rogee/superpowers
1
0
Fork 0
mirror of https://github.com/obra/superpowers.git synced 2026-08-08 23:32:58 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix(e2e): preserve evidence pipeline failures

Browse Source
This commit is contained in:
Drew Ritter
2026-08-06 16:51:54 -07:00
parent a0885c03bc
commit 9f4f46639c
1 changed files with 18 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/agentic-end-to-end-testing/recording-a-proof-movie.md
+13 -6
View File
@@ -31,14 +31,18 @@ markers. Use `bash`, not `zsh` — zsh's read-only `$status` injects a spurious
error *after* a passing run and pollutes the movie. error *after* a passing run and pollutes the movie.
```bash ```bash
run_log=<evidence-dir>/run.log
bash -o pipefail -c ' bash -o pipefail -c '
{
printf "MANUAL_E2E_KIND=<name>\n"; printf "MANUAL_E2E_KIND=<name>\n";
printf "STARTED_AT="; date -u +%Y-%m-%dT%H:%M:%SZ; printf "STARTED_AT="; date -u +%Y-%m-%dT%H:%M:%SZ;
<the real e2e command>; # e.g. xcodebuild test-without-building ... -resultBundlePath ... <the real e2e command>; # e.g. xcodebuild test-without-building ... -resultBundlePath ...
rc=$?; rc=$?;
printf "FINISHED_AT="; date -u +%Y-%m-%dT%H:%M:%SZ; printf "FINISHED_AT="; date -u +%Y-%m-%dT%H:%M:%SZ;
printf "EXIT_STATUS=%s\n" "$rc"; exit "$rc" printf "EXIT_STATUS=%s\n" "$rc";
' 2>&1 | tee <evidence-dir>/run.log exit "$rc"
} 2>&1 | tee "$1"
' bash "$run_log"
``` ```
## Snapshot external state before and after ## Snapshot external state before and after
@@ -47,10 +51,13 @@ If the run touches a remote host or a shared tmux, snapshot it identically
pre- and post-run and diff. Equal snapshots prove the run left no residue. pre- and post-run and diff. Equal snapshots prove the run left no residue.
```bash ```bash
ssh <host> 'date -Is; tmux list-sessions -F "#{session_name}|#{session_windows}|attached=#{session_attached}"; \ host=<host>
ps -eo pid=,args= | awk "/<helper>/ {print}"; find /tmp -maxdepth 1 -name "<sock-glob>" | wc -l' \ snapshot_path=<evidence-dir>/pre-snapshot.txt
| tee <evidence-dir>/pre-snapshot.txt bash -o pipefail -c 'ssh "$1" "$2" | tee "$3"' bash \
# ... run gate ... then repeat with SNAPSHOT_KIND=post => post-snapshot.txt ; assert they match "$host" \
'date -Is; tmux list-sessions -F "#{session_name}|#{session_windows}|attached=#{session_attached}"; ps -eo pid=,args= | awk "/<helper>/ {print}"; find /tmp -maxdepth 1 -name "<sock-glob>" | wc -l' \
"$snapshot_path"
# ... run gate ... then repeat for post-snapshot.txt. Begin comparison only after both capture commands succeed.
``` ```
## Render the reel from the log ## Render the reel from the log