mirror of
https://github.com/obra/superpowers.git
synced 2026-07-09 11:39:03 +08:00
feat(skills): add e2e browser and CLI/TUI driving recipes
This commit is contained in:
93
skills/agentic-end-to-end-testing/driving-cli-tui.md
Normal file
93
skills/agentic-end-to-end-testing/driving-cli-tui.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# Driving a CLI / TUI (tmux)
|
||||
|
||||
Each scenario gets its own named tmux session (cleanup needs a deterministic
|
||||
name). Fix the size for deterministic capture; prefer the app's plain-text/inline
|
||||
mode if it has one.
|
||||
|
||||
## The four-command recipe
|
||||
|
||||
```bash
|
||||
tmux new-session -d -s <name> -x 200 -y 50 "<cmd> 2>/tmp/<name>-stderr.log"
|
||||
tmux send-keys -t <name> -l "literal text" # -l = no key-name parsing (paths, slashes)
|
||||
tmux send-keys -t <name> Enter
|
||||
tmux capture-pane -t <name> -p # -p = plain text; add -e only for styling
|
||||
```
|
||||
|
||||
- `-x 200 -y 50` fixes the pane size so `capture-pane` output is deterministic
|
||||
run to run — a resized pane reflows text differently.
|
||||
- Always `-l` for user-typed strings; without it a literal path like
|
||||
`/foo/bar` gets parsed as arrow-key escapes instead of typed characters.
|
||||
- Redirect stderr to a file — panics, log lines, and debug probes land there,
|
||||
not in the pane, so they won't show up in a `capture-pane` snapshot at all.
|
||||
|
||||
Kill any leftover session with the same name before starting a new one, so
|
||||
reruns don't attach to a stale process:
|
||||
|
||||
```bash
|
||||
tmux kill-session -t <name> 2>/dev/null # idempotent: fine if nothing to kill
|
||||
```
|
||||
|
||||
## Form fill: send-keys patterns
|
||||
|
||||
`send-keys` parses keystrokes by name (`Enter`, `BTab`, `C-u`) unless you pass
|
||||
`-l` for literal text. A typical field-by-field fill mixes both:
|
||||
|
||||
```bash
|
||||
tmux send-keys -t <name> BTab # shift-tab to a prior field
|
||||
tmux send-keys -t <name> C-u # clear the current line
|
||||
tmux send-keys -t <name> -l "some/literal/path" # literal — no key parsing
|
||||
tmux send-keys -t <name> Tab # forward to the next field
|
||||
tmux send-keys -t <name> Enter
|
||||
```
|
||||
|
||||
`sleep 0.3` between keys is usually enough; bump to 0.5–1.0s for field
|
||||
transitions where the UI re-renders.
|
||||
|
||||
## Polling capture-pane for state
|
||||
|
||||
Poll `capture-pane -p` for a state string and grep the **glyph or word**, not
|
||||
the color — `-p` drops ANSI styling by default (add `-e` only if you need
|
||||
styling), and colors are also just harder to grep reliably than a fixed
|
||||
glyph:
|
||||
|
||||
```bash
|
||||
for i in $(seq 1 30); do
|
||||
pane=$(tmux capture-pane -t <name> -p)
|
||||
echo "$pane" | grep -q "state: processing" && break
|
||||
sleep 1
|
||||
done
|
||||
```
|
||||
|
||||
TUIs commonly use a distinct glyph per state, e.g. a Braille spinner (`⠋`)
|
||||
while pending and an X mark (`✗`) on failure, with the glyph simply removed
|
||||
once reconciled. Grep for the glyph itself, not for a color code.
|
||||
|
||||
## Two captures for optimistic UI
|
||||
|
||||
Mirror the web sync/async pattern: capture the pane immediately after the
|
||||
triggering keypress, then again after a reconcile window. Without the
|
||||
immediate capture you can't tell "rendered then reconciled" from "never
|
||||
rendered":
|
||||
|
||||
```bash
|
||||
tmux send-keys -t <name> -l "trigger the optimistic action"
|
||||
tmux send-keys -t <name> Enter
|
||||
echo "=== synchronous ===" ; tmux capture-pane -t <name> -p | grep -E "pending-glyph"
|
||||
sleep 6
|
||||
echo "=== reconciled ===" ; tmux capture-pane -t <name> -p | grep -E "pending-glyph" || echo "[no pending — reconciled]"
|
||||
```
|
||||
|
||||
## Plain-text mode over the alt-screen buffer
|
||||
|
||||
If the TUI has a flag that disables its alternate-screen buffer (a debug or
|
||||
plain-output mode), use it when launching under tmux. `capture-pane` then sees
|
||||
plain scrollback text instead of raw escape sequences from a full-screen
|
||||
redraw, which is much easier to grep.
|
||||
|
||||
## Non-interactive CLIs don't need tmux
|
||||
|
||||
If the surface under test is a one-shot command rather than an interactive
|
||||
session, skip tmux entirely — run the command and capture its stdout/stderr
|
||||
directly. The tmux machinery exists for interaction, not for driving a binary
|
||||
in general. Still run it against a real, freshly built instance, not a stale
|
||||
one left over from an earlier session.
|
||||
95
skills/agentic-end-to-end-testing/driving-web-browser.md
Normal file
95
skills/agentic-end-to-end-testing/driving-web-browser.md
Normal file
@@ -0,0 +1,95 @@
|
||||
# Driving a Web UI (Browser)
|
||||
|
||||
Use a Chrome/CDP browser tool. After authenticated navigation, drive the page
|
||||
through `eval` against the app's own JS entry points rather than synthesizing
|
||||
clicks where possible — it's more robust to layout change than clicking
|
||||
coordinates or brittle selectors.
|
||||
|
||||
## Authenticated navigation
|
||||
|
||||
If the app's login flow is a token-bearing redirect (e.g. a URL like
|
||||
`/auth?token=<TOKEN>&next=<path>`), navigate straight to that URL and then wait
|
||||
for an element you expect to exist once the session is live:
|
||||
|
||||
```text
|
||||
navigate http://<host>/auth?token=<TOKEN>&next=<path>
|
||||
await_element [data-some-marker]
|
||||
```
|
||||
|
||||
Use the literal token value, not the path to the file that contains it. Passing
|
||||
the path instead of the token itself typically renders as an "invalid token"
|
||||
page rather than an obvious stack trace — if you see that error, check which
|
||||
one you passed.
|
||||
|
||||
## Optimistic-vs-settled assertions
|
||||
|
||||
For any "did the optimistic UI update happen before the request resolved?"
|
||||
scenario, fire the action but *don't await it*, take a synchronous DOM
|
||||
snapshot (the pending placeholder is there *now*), then await and snapshot
|
||||
again:
|
||||
|
||||
```javascript
|
||||
(async () => {
|
||||
const before = {
|
||||
pendingCount: document.querySelectorAll(".optimistic-pending").length,
|
||||
};
|
||||
// Fire — capture the promise but don't await yet.
|
||||
const promise = window.App.doAction(id, payload).catch(e => e);
|
||||
// Synchronous: the pending placeholder is in the DOM RIGHT NOW.
|
||||
const sync = {
|
||||
pendingCount: document.querySelectorAll(".optimistic-pending").length,
|
||||
pendingText: document.querySelector(".optimistic-pending")?.textContent,
|
||||
};
|
||||
await promise;
|
||||
await new Promise(r => setTimeout(r, 200)); // let the DOM settle
|
||||
const after = {
|
||||
pendingCount: document.querySelectorAll(".optimistic-pending").length,
|
||||
failedCount: document.querySelectorAll(".optimistic-failed").length,
|
||||
reason: document.querySelector(".optimistic-failed-reason")?.textContent,
|
||||
};
|
||||
return JSON.stringify({ before, sync, after }, null, 2);
|
||||
})()
|
||||
```
|
||||
|
||||
Without the no-await capture you can't tell "rendered then reconciled" from
|
||||
"never rendered" — both look identical in the post-await snapshot alone.
|
||||
|
||||
## Return a plain string from eval
|
||||
|
||||
Join your findings into a string (e.g. `JSON.stringify(..., null, 2)` or
|
||||
`\n`-joined lines) before returning from `eval`. Some bridges stringify a
|
||||
returned object as `[object Object]`, silently discarding everything you
|
||||
wanted to inspect.
|
||||
|
||||
## Probing internal state when the DOM is ambiguous
|
||||
|
||||
Inspect the app's singleton via `window.<App>?.state` (or whatever it exposes)
|
||||
when the DOM alone can't tell you what happened:
|
||||
|
||||
```javascript
|
||||
JSON.stringify({
|
||||
state: window.App?.state, // idle | processing | …
|
||||
hydrated: window.App?.hydrated,
|
||||
pendingType: typeof window.App?.pending,
|
||||
windowKeys: Object.keys(window).filter(k => k.toLowerCase().includes("app")),
|
||||
})
|
||||
```
|
||||
|
||||
The `windowKeys` scan is useful when you don't already know the singleton's
|
||||
name — grep the result for something plausible. If a hydration/connection
|
||||
flag is `false` when you expect `true`, or a registry that should be an object
|
||||
comes back `"undefined"`, that's usually the real bug, not a DOM timing issue.
|
||||
|
||||
## Prefer labels over selectors
|
||||
|
||||
When a step needs a concrete locator, prefer a label the user actually sees
|
||||
(button text, aria-label, visible heading) over a brittle structural selector
|
||||
like `#nav > li:nth-child(3)`. A layout shuffle breaks the selector; it rarely
|
||||
changes the label.
|
||||
|
||||
## When console capture is unreliable
|
||||
|
||||
If the browser tool's console-log capture is flaky or stubbed, route debug
|
||||
output through `eval` instead: push entries to a `window.__DEBUG_LOG` array
|
||||
from the page, then read it back with a follow-up `eval` call. This sidesteps
|
||||
the capture path entirely and gives you an ordinary string to inspect.
|
||||
Reference in New Issue
Block a user