rogee/superpowers
1
0
Fork 0
mirror of https://github.com/obra/superpowers.git synced 2026-08-07 14:48:47 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix(e2e): scope tmux cleanup to owned sessions

Browse Source 
Replace deterministic shared tmux names and preemptive deletion with a readable run-unique name, an ownership bit set only after successful creation, and cleanup guarded by that ownership. A collision now fails closed: the runner chooses another unique name or reports failure and never reclassifies the existing session as stale.\n\nwriting-skills pressure evidence used three fresh subjects per arm with real sentinel sessions. RED was 0/3: every subject followed the old recipe by killing the pre-existing session, recreating the same name, and cleaning it again. GREEN was 3/3: every subject preserved the original session identity, created a distinct owned session, captured READY, and removed only the owned session. The controller independently verified original IDs $9, $10, and $11 before removing the disposable sentinels.
This commit is contained in:
Drew Ritter
2026-08-06 14:57:57 -07:00
parent 7dff0b2d48
commit b994fedcca
1 changed files with 37 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
skills/agentic-end-to-end-testing/driving-cli-tui.md
View File

@@ -1,16 +1,29 @@
# Driving a CLI / TUI (tmux) # Driving a CLI / TUI (tmux)
Each scenario gets its own named tmux session (cleanup needs a deterministic Each scenario gets its own run-unique tmux session. Record ownership only after
name). Fix the size for deterministic capture; prefer the app's plain-text/inline creation succeeds, then clean up only that owned session. Fix the size for
mode if it has one. deterministic capture; prefer the app's plain-text/inline mode if it has one.
## The four-command recipe ## Ownership-safe recipe
```bash ```bash
tmux new-session -d -s <name> -x 200 -y 50 "<cmd> 2>/tmp/<name>-stderr.log" scenario="widget-form"
tmux send-keys -t <name> -l "literal text" # -l = no key-name parsing (paths, slashes) session="${scenario}-$(date +%s)-$$"
tmux send-keys -t <name> Enter stderr_log="/tmp/${session}-stderr.log"
tmux capture-pane -t <name> -p # -p = plain text; add -e only for styling session_owned=0
cleanup() {
if [ "$session_owned" -eq 1 ]; then
tmux kill-session -t "$session" 2>/dev/null || true
fi
}
trap cleanup EXIT
tmux new-session -d -s "$session" -x 200 -y 50 "<cmd> 2>\"$stderr_log\""
session_owned=1
tmux send-keys -t "$session" -l "literal text" # -l = no key-name parsing (paths, slashes)
tmux send-keys -t "$session" Enter
tmux capture-pane -t "$session" -p # -p = plain text; add -e only for styling
``` ```
- `-x 200 -y 50` fixes the pane size so `capture-pane` output is deterministic - `-x 200 -y 50` fixes the pane size so `capture-pane` output is deterministic
@@ -19,13 +32,10 @@ tmux capture-pane -t <name> -p # -p = plain text; add -e only for
`/foo/bar` gets parsed as arrow-key escapes instead of typed characters. `/foo/bar` gets parsed as arrow-key escapes instead of typed characters.
- Redirect stderr to a file — panics, log lines, and debug probes land there, - Redirect stderr to a file — panics, log lines, and debug probes land there,
not in the pane, so they won't show up in a `capture-pane` snapshot at all. not in the pane, so they won't show up in a `capture-pane` snapshot at all.
- The timestamp/PID suffix keeps the name readable and run-unique. If creation
Kill any leftover session with the same name before starting a new one, so still reports a collision, choose another unique name or report the failure.
reruns don't attach to a stale process: Never kill the colliding session: `session_owned` remains `0`, so cleanup
cannot touch it.
```bash
tmux kill-session -t <name> 2>/dev/null # idempotent: fine if nothing to kill
```
## Form fill: send-keys patterns ## Form fill: send-keys patterns
@@ -33,19 +43,19 @@ tmux kill-session -t <name> 2>/dev/null # idempotent: fine if nothing to kill
`-l` for literal text. A typical field-by-field fill mixes both: `-l` for literal text. A typical field-by-field fill mixes both:
```bash ```bash
tmux send-keys -t <name> "n" # tap a key to open the form tmux send-keys -t "$session" "n" # tap a key to open the form
sleep 1 sleep 1
tmux send-keys -t <name> BTab # shift-tab back one field tmux send-keys -t "$session" BTab # shift-tab back one field
sleep 0.3 sleep 0.3
tmux send-keys -t <name> C-u # clear the current line tmux send-keys -t "$session" C-u # clear the current line
sleep 0.3 sleep 0.3
tmux send-keys -t <name> -l "some/literal/path" # literal — no key parsing tmux send-keys -t "$session" -l "some/literal/path" # literal — no key parsing
sleep 0.3 sleep 0.3
tmux send-keys -t <name> Tab # forward to next field tmux send-keys -t "$session" Tab # forward to next field
sleep 0.3 sleep 0.3
tmux send-keys -t <name> -l "text the user would type" tmux send-keys -t "$session" -l "text the user would type"
sleep 0.3 sleep 0.3
tmux send-keys -t <name> Enter # submit tmux send-keys -t "$session" Enter # submit
``` ```
`sleep 0.3` between keys is usually enough; bump to 0.5–1.0s for field `sleep 0.3` between keys is usually enough; bump to 0.5–1.0s for field
@@ -60,7 +70,7 @@ glyph:
```bash ```bash
for i in $(seq 1 30); do for i in $(seq 1 30); do
pane=$(tmux capture-pane -t <name> -p) pane=$(tmux capture-pane -t "$session" -p)
echo "$pane" | grep -q "state: processing" && break echo "$pane" | grep -q "state: processing" && break
sleep 1 sleep 1
done done
@@ -78,11 +88,11 @@ immediate capture you can't tell "rendered then reconciled" from "never
rendered": rendered":
```bash ```bash
tmux send-keys -t <name> -l "trigger the optimistic action" tmux send-keys -t "$session" -l "trigger the optimistic action"
tmux send-keys -t <name> Enter tmux send-keys -t "$session" Enter
echo "=== synchronous ===" ; tmux capture-pane -t <name> -p | grep -E "pending-glyph" echo "=== synchronous ===" ; tmux capture-pane -t "$session" -p | grep -E "pending-glyph"
sleep 6 sleep 6
echo "=== reconciled ===" ; tmux capture-pane -t <name> -p | grep -E "pending-glyph" || echo "[no pending — reconciled]" echo "=== reconciled ===" ; tmux capture-pane -t "$session" -p | grep -E "pending-glyph" || echo "[no pending — reconciled]"
``` ```
## Plain-text mode over the alt-screen buffer ## Plain-text mode over the alt-screen buffer