From b1e4718205257aaac97c8a1cb397703e0784f0ca Mon Sep 17 00:00:00 2001 From: Jesse Vincent Date: Sat, 4 Jul 2026 15:51:31 -0700 Subject: [PATCH] feat(skills): add e2e browser and CLI/TUI driving recipes --- .../driving-cli-tui.md | 93 ++++++++++++++++++ .../driving-web-browser.md | 95 +++++++++++++++++++ 2 files changed, 188 insertions(+) create mode 100644 skills/agentic-end-to-end-testing/driving-cli-tui.md create mode 100644 skills/agentic-end-to-end-testing/driving-web-browser.md diff --git a/skills/agentic-end-to-end-testing/driving-cli-tui.md b/skills/agentic-end-to-end-testing/driving-cli-tui.md new file mode 100644 index 00000000..6ec424e2 --- /dev/null +++ b/skills/agentic-end-to-end-testing/driving-cli-tui.md @@ -0,0 +1,93 @@ +# Driving a CLI / TUI (tmux) + +Each scenario gets its own named tmux session (cleanup needs a deterministic +name). Fix the size for deterministic capture; prefer the app's plain-text/inline +mode if it has one. + +## The four-command recipe + +```bash +tmux new-session -d -s -x 200 -y 50 " 2>/tmp/-stderr.log" +tmux send-keys -t -l "literal text" # -l = no key-name parsing (paths, slashes) +tmux send-keys -t Enter +tmux capture-pane -t -p # -p = plain text; add -e only for styling +``` + +- `-x 200 -y 50` fixes the pane size so `capture-pane` output is deterministic + run to run — a resized pane reflows text differently. +- Always `-l` for user-typed strings; without it a literal path like + `/foo/bar` gets parsed as arrow-key escapes instead of typed characters. +- Redirect stderr to a file — panics, log lines, and debug probes land there, + not in the pane, so they won't show up in a `capture-pane` snapshot at all. + +Kill any leftover session with the same name before starting a new one, so +reruns don't attach to a stale process: + +```bash +tmux kill-session -t 2>/dev/null # idempotent: fine if nothing to kill +``` + +## Form fill: send-keys patterns + +`send-keys` parses keystrokes by name (`Enter`, `BTab`, `C-u`) unless you pass +`-l` for literal text. A typical field-by-field fill mixes both: + +```bash +tmux send-keys -t BTab # shift-tab to a prior field +tmux send-keys -t C-u # clear the current line +tmux send-keys -t -l "some/literal/path" # literal — no key parsing +tmux send-keys -t Tab # forward to the next field +tmux send-keys -t Enter +``` + +`sleep 0.3` between keys is usually enough; bump to 0.5–1.0s for field +transitions where the UI re-renders. + +## Polling capture-pane for state + +Poll `capture-pane -p` for a state string and grep the **glyph or word**, not +the color — `-p` drops ANSI styling by default (add `-e` only if you need +styling), and colors are also just harder to grep reliably than a fixed +glyph: + +```bash +for i in $(seq 1 30); do + pane=$(tmux capture-pane -t -p) + echo "$pane" | grep -q "state: processing" && break + sleep 1 +done +``` + +TUIs commonly use a distinct glyph per state, e.g. a Braille spinner (`⠋`) +while pending and an X mark (`✗`) on failure, with the glyph simply removed +once reconciled. Grep for the glyph itself, not for a color code. + +## Two captures for optimistic UI + +Mirror the web sync/async pattern: capture the pane immediately after the +triggering keypress, then again after a reconcile window. Without the +immediate capture you can't tell "rendered then reconciled" from "never +rendered": + +```bash +tmux send-keys -t -l "trigger the optimistic action" +tmux send-keys -t Enter +echo "=== synchronous ===" ; tmux capture-pane -t -p | grep -E "pending-glyph" +sleep 6 +echo "=== reconciled ===" ; tmux capture-pane -t -p | grep -E "pending-glyph" || echo "[no pending — reconciled]" +``` + +## Plain-text mode over the alt-screen buffer + +If the TUI has a flag that disables its alternate-screen buffer (a debug or +plain-output mode), use it when launching under tmux. `capture-pane` then sees +plain scrollback text instead of raw escape sequences from a full-screen +redraw, which is much easier to grep. + +## Non-interactive CLIs don't need tmux + +If the surface under test is a one-shot command rather than an interactive +session, skip tmux entirely — run the command and capture its stdout/stderr +directly. The tmux machinery exists for interaction, not for driving a binary +in general. Still run it against a real, freshly built instance, not a stale +one left over from an earlier session. diff --git a/skills/agentic-end-to-end-testing/driving-web-browser.md b/skills/agentic-end-to-end-testing/driving-web-browser.md new file mode 100644 index 00000000..bf9c43c6 --- /dev/null +++ b/skills/agentic-end-to-end-testing/driving-web-browser.md @@ -0,0 +1,95 @@ +# Driving a Web UI (Browser) + +Use a Chrome/CDP browser tool. After authenticated navigation, drive the page +through `eval` against the app's own JS entry points rather than synthesizing +clicks where possible — it's more robust to layout change than clicking +coordinates or brittle selectors. + +## Authenticated navigation + +If the app's login flow is a token-bearing redirect (e.g. a URL like +`/auth?token=&next=`), navigate straight to that URL and then wait +for an element you expect to exist once the session is live: + +```text +navigate http:///auth?token=&next= +await_element [data-some-marker] +``` + +Use the literal token value, not the path to the file that contains it. Passing +the path instead of the token itself typically renders as an "invalid token" +page rather than an obvious stack trace — if you see that error, check which +one you passed. + +## Optimistic-vs-settled assertions + +For any "did the optimistic UI update happen before the request resolved?" +scenario, fire the action but *don't await it*, take a synchronous DOM +snapshot (the pending placeholder is there *now*), then await and snapshot +again: + +```javascript +(async () => { + const before = { + pendingCount: document.querySelectorAll(".optimistic-pending").length, + }; + // Fire — capture the promise but don't await yet. + const promise = window.App.doAction(id, payload).catch(e => e); + // Synchronous: the pending placeholder is in the DOM RIGHT NOW. + const sync = { + pendingCount: document.querySelectorAll(".optimistic-pending").length, + pendingText: document.querySelector(".optimistic-pending")?.textContent, + }; + await promise; + await new Promise(r => setTimeout(r, 200)); // let the DOM settle + const after = { + pendingCount: document.querySelectorAll(".optimistic-pending").length, + failedCount: document.querySelectorAll(".optimistic-failed").length, + reason: document.querySelector(".optimistic-failed-reason")?.textContent, + }; + return JSON.stringify({ before, sync, after }, null, 2); +})() +``` + +Without the no-await capture you can't tell "rendered then reconciled" from +"never rendered" — both look identical in the post-await snapshot alone. + +## Return a plain string from eval + +Join your findings into a string (e.g. `JSON.stringify(..., null, 2)` or +`\n`-joined lines) before returning from `eval`. Some bridges stringify a +returned object as `[object Object]`, silently discarding everything you +wanted to inspect. + +## Probing internal state when the DOM is ambiguous + +Inspect the app's singleton via `window.?.state` (or whatever it exposes) +when the DOM alone can't tell you what happened: + +```javascript +JSON.stringify({ + state: window.App?.state, // idle | processing | … + hydrated: window.App?.hydrated, + pendingType: typeof window.App?.pending, + windowKeys: Object.keys(window).filter(k => k.toLowerCase().includes("app")), +}) +``` + +The `windowKeys` scan is useful when you don't already know the singleton's +name — grep the result for something plausible. If a hydration/connection +flag is `false` when you expect `true`, or a registry that should be an object +comes back `"undefined"`, that's usually the real bug, not a DOM timing issue. + +## Prefer labels over selectors + +When a step needs a concrete locator, prefer a label the user actually sees +(button text, aria-label, visible heading) over a brittle structural selector +like `#nav > li:nth-child(3)`. A layout shuffle breaks the selector; it rarely +changes the label. + +## When console capture is unreliable + +If the browser tool's console-log capture is flaky or stubbed, route debug +output through `eval` instead: push entries to a `window.__DEBUG_LOG` array +from the page, then read it back with a follow-up `eval` call. This sidesteps +the capture path entirely and gives you an ordinary string to inspect.