diff --git a/docs/superpowers/plans/2026-07-04-spec-derived-scenario-cards.md b/docs/superpowers/plans/2026-07-04-spec-derived-scenario-cards.md new file mode 100644 index 00000000..1bd78e01 --- /dev/null +++ b/docs/superpowers/plans/2026-07-04-spec-derived-scenario-cards.md @@ -0,0 +1,590 @@ +# Spec-Derived Scenario Cards Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Implement the spec-derived scenario cards design: a checker script, the `authoring-cards-from-a-spec.md` supporting file, a brainstorming spec-table conditional, and an optional SDD e2e step — each behavior-shaping edit RED-before-GREEN. + +**Architecture:** One deterministic bash checker (TDD, standalone test harness per house `tests/shell-lint` pattern) anchors the verbatim contract; three markdown skill edits route through it. RED baselines precede every skill edit; GREEN re-runs use the same fixtures and subagent methodology as the 2026-07-04 experiments. + +**Tech Stack:** bash + POSIX tools (tr/sed/grep/awk) only; markdown skill files; subagent dispatches for RED/GREEN. + +**Spec:** `docs/superpowers/specs/2026-07-04-spec-derived-scenario-cards-design.md` — read it first; its "Checker script" matching semantics are normative. + +## Global Constraints + +- All work on branch `agentic-end-to-end-testing` in `/Users/jesse/git/superpowers/superpowers`. Do not push. Do not touch `evals/` (nested repo) except READ-ONLY fixture copying. +- The corpus at `/Users/jesse/Documents/agentic-e2e-testing-corpus/` is never committed to any repo. RED/GREEN write-ups go there. +- Checker: bash + POSIX tools only; matching semantics exactly as the spec's normative block (case-insensitive heading "E2E scenario cards"; columns by header name; `\|` unescaped; whitespace runs collapsed to one space + trimmed; case-sensitive **fixed-string** matching; no regex over falsification text). +- Role boundary wording, verbatim wherever the role is stated: "the card author never modifies product code, test code, or existing cards' assertions." +- writing-skills Iron Law: Task 2's RED baselines complete before Tasks 3-5 write any skill prose. Task 1 (script) is ordinary code TDD and does not wait. +- No emojis. No session IDs or corpus narrative in any skill file. + +--- + +### Task 1: Checker script `check-cards-against-spec` (TDD) + +**Files:** +- Create: `skills/agentic-end-to-end-testing/scripts/check-cards-against-spec` (mode 0755) +- Test: `tests/agentic-e2e-checker/test-check-cards-against-spec.sh` (mode 0755) + +**Interfaces:** +- Produces: `check-cards-against-spec `; exit 0 = all pass, 1 = check failure, 2 = no "E2E scenario cards" table, 64 = usage error. Tasks 3 and 5 reference the script by its repo-relative path. + +- [ ] **Step 1: Write the failing test harness** + +Create `tests/agentic-e2e-checker/test-check-cards-against-spec.sh` (executable). It mirrors `tests/shell-lint/test-lint-shell.sh`'s shape (self-contained, mktemp fixtures, trap cleanup, pass/fail counters): + +```bash +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" +CHECKER="$REPO_ROOT/skills/agentic-end-to-end-testing/scripts/check-cards-against-spec" + +FAILURES=0 +TEST_ROOT="$(mktemp -d)" +cleanup() { rm -rf "$TEST_ROOT"; } +trap cleanup EXIT + +pass() { echo " [PASS] $1"; } +fail() { echo " [FAIL] $1"; FAILURES=$((FAILURES + 1)); } + +assert_exit() { # expected_code description -- command... + local expected="$1" desc="$2"; shift 2 + local code=0 + "$@" >"$TEST_ROOT/out.txt" 2>&1 || code=$? + if [ "$code" -eq "$expected" ]; then pass "$desc"; else + fail "$desc (expected exit $expected, got $code)"; sed 's/^/ /' "$TEST_ROOT/out.txt"; fi +} + +assert_out_contains() { # needle description + if grep -Fq -- "$1" "$TEST_ROOT/out.txt"; then pass "$2"; else + fail "$2 (output missing: $1)"; sed 's/^/ /' "$TEST_ROOT/out.txt"; fi +} + +# ---- fixture builders ---------------------------------------------------- + +make_spec() { # dir (spec with 2-row table; row 2 has \| and regex chars) + mkdir -p "$1" + cat > "$1/spec.md" <<'EOF' +# Widget Design + +## Requirements + +Widgets render a table with a TOTAL row. + +## E2E scenario cards + +| Card | Covers | Falsification | +| --- | --- | --- | +| widget-show-table | Rendered table incl. TOTAL row | If stdout's last line is not `TOTAL` followed by the two-decimal sum (20.85 for the seed fixture), or the TOTAL row is absent entirely, the scenario FAILS. | +| widget-status-flags | Status output | If `widget status` does not print exactly `OK \| DEGRADED` (a literal pipe) with dots . and stars * intact, the scenario FAILS. | +EOF +} + +good_card_1() { + cat <<'EOF' +# widget-show-table: table renders with TOTAL + +**What this covers**: the rendered table. + +## Pre-state +A built widget binary. + +## Steps +1. Run `widget show`. + +## Expected +If stdout's last line is not `TOTAL` followed by the +two-decimal sum (20.85 for the seed +fixture), or the TOTAL row is absent entirely, the scenario FAILS. + +## Cleanup +Nothing to clean. +EOF +} + +good_card_2() { + cat <<'EOF' +# widget-status-flags: status output + +**What this covers**: status flags. + +## Pre-state +A built widget binary. + +## Steps +1. Run `widget status`. + +## Expected +If `widget status` does not print exactly `OK | DEGRADED` (a literal pipe) with dots . and stars * intact, the scenario FAILS. + +## Cleanup +Nothing to clean. +EOF +} + +make_cards() { # dir + mkdir -p "$1" + good_card_1 > "$1/widget-show-table.md" + good_card_2 > "$1/widget-status-flags.md" +} + +# ---- tests ---------------------------------------------------------------- + +echo "happy path" +make_spec "$TEST_ROOT/t1"; make_cards "$TEST_ROOT/t1/cards" +assert_exit 0 "2 rows, 2 conforming cards -> exit 0" \ + "$CHECKER" "$TEST_ROOT/t1/spec.md" "$TEST_ROOT/t1/cards" + +echo "re-wrapped falsification line still matches (whitespace normalization)" +# good_card_1 already wraps the line across three lines; covered above. Prove +# the inverse too: collapse the card line to one line, still passes. +make_spec "$TEST_ROOT/t2"; make_cards "$TEST_ROOT/t2/cards" +perl -0pi -e 's/\n(two-decimal)/ $1/; s/\n(fixture\))/ $1/' "$TEST_ROOT/t2/cards/widget-show-table.md" 2>/dev/null || \ + sed -i '' -e ':a' -e 'N;$!ba' -e 's/the\ntwo-decimal/the two-decimal/' "$TEST_ROOT/t2/cards/widget-show-table.md" +assert_exit 0 "single-line variant -> exit 0" \ + "$CHECKER" "$TEST_ROOT/t2/spec.md" "$TEST_ROOT/t2/cards" + +echo "escaped pipe in table cell matches literal pipe in card" +# covered by widget-status-flags in the happy path; also prove failure when +# the card drops the pipe phrase entirely: +make_spec "$TEST_ROOT/t3"; make_cards "$TEST_ROOT/t3/cards" +sed -i.bak 's/OK | DEGRADED/OK or DEGRADED/' "$TEST_ROOT/t3/cards/widget-status-flags.md" +assert_exit 1 "reworded falsification -> exit 1" \ + "$CHECKER" "$TEST_ROOT/t3/spec.md" "$TEST_ROOT/t3/cards" +assert_out_contains "widget-status-flags" "failure names the card" + +echo "missing card file" +make_spec "$TEST_ROOT/t4"; make_cards "$TEST_ROOT/t4/cards" +rm "$TEST_ROOT/t4/cards/widget-show-table.md" +assert_exit 1 "missing card -> exit 1" \ + "$CHECKER" "$TEST_ROOT/t4/spec.md" "$TEST_ROOT/t4/cards" +assert_out_contains "widget-show-table.md" "failure names the missing file" + +echo "missing required section" +make_spec "$TEST_ROOT/t5"; make_cards "$TEST_ROOT/t5/cards" +sed -i.bak '/^## Cleanup/,$d' "$TEST_ROOT/t5/cards/widget-show-table.md" +assert_exit 1 "card without Cleanup heading -> exit 1" \ + "$CHECKER" "$TEST_ROOT/t5/spec.md" "$TEST_ROOT/t5/cards" +assert_out_contains "Cleanup" "failure names the section" + +echo "extra card is a warning, not a failure" +make_spec "$TEST_ROOT/t6"; make_cards "$TEST_ROOT/t6/cards" +good_card_1 > "$TEST_ROOT/t6/cards/extra-exploration.md" +assert_exit 0 "extra card -> exit 0" \ + "$CHECKER" "$TEST_ROOT/t6/spec.md" "$TEST_ROOT/t6/cards" +assert_out_contains "extra-exploration" "warning names the extra card" + +echo "no scenario table" +mkdir -p "$TEST_ROOT/t7/cards" +printf '# Widget Design\n\nNo table here.\n' > "$TEST_ROOT/t7/spec.md" +assert_exit 2 "table-less spec -> exit 2" \ + "$CHECKER" "$TEST_ROOT/t7/spec.md" "$TEST_ROOT/t7/cards" +assert_out_contains "no scenario table" "diagnostic present" + +echo "heading match is case-insensitive" +make_spec "$TEST_ROOT/t8"; make_cards "$TEST_ROOT/t8/cards" +sed -i.bak 's/^## E2E scenario cards/## E2E Scenario Cards/' "$TEST_ROOT/t8/spec.md" +assert_exit 0 "title-case heading still found" \ + "$CHECKER" "$TEST_ROOT/t8/spec.md" "$TEST_ROOT/t8/cards" + +echo "usage" +assert_exit 64 "no args -> exit 64" "$CHECKER" +assert_exit 0 "--help -> exit 0" "$CHECKER" --help +assert_out_contains "Usage:" "help text present" + +echo +if [ "$FAILURES" -gt 0 ]; then echo "$FAILURES test(s) failed"; exit 1; fi +echo "all tests passed" +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `tests/agentic-e2e-checker/test-check-cards-against-spec.sh` +Expected: every assertion FAILs (checker does not exist yet; exit-code assertions report the shell's 127). + +- [ ] **Step 3: Write the checker** + +Create `skills/agentic-end-to-end-testing/scripts/check-cards-against-spec` (executable): + +```bash +#!/usr/bin/env bash +# check-cards-against-spec — verify scenario cards carry their spec table's +# falsification lines verbatim. See authoring-cards-from-a-spec.md. +set -euo pipefail + +usage() { + cat <<'EOF' +Usage: check-cards-against-spec + +Verifies the spec's "E2E scenario cards" table against the cards directory: + 1. table parses (>=1 row; non-empty Card and Falsification cells) + 2. every row has /.md + 3. every card contains its Falsification line verbatim + (whitespace-normalized, fixed-string, case-sensitive) + 4. every card has **What this covers** (bold inline) and ## headings + Pre-state, Steps, Expected, Cleanup (Sharp edges not required) + 5. extra cards in are reported as warnings, not failures + +Exit: 0 all pass; 1 check failed; 2 no "E2E scenario cards" table; 64 usage. +EOF +} + +[ "${1:-}" = "--help" ] && { usage; exit 0; } +[ $# -eq 2 ] || { usage >&2; exit 64; } +SPEC="$1"; CARDS="$2" +[ -f "$SPEC" ] || { echo "error: spec not found: $SPEC" >&2; exit 64; } +[ -d "$CARDS" ] || { echo "error: cards dir not found: $CARDS" >&2; exit 64; } + +FAILURES=0 +fail() { echo "FAIL: $1"; FAILURES=$((FAILURES + 1)); } +warn() { echo "warn: $1"; } + +# Collapse every whitespace run to one space; trim ends. (Normative per the +# design spec: markdown re-wrapping must not defeat the verbatim check.) +normalize() { tr -s '[:space:]' ' ' | sed -e 's/^ //' -e 's/ $//'; } + +# --- extract the first table under the (case-insensitive) heading ---------- +TABLE="$(awk ' + /^#{1,6}[[:space:]]/ { + h = $0; sub(/^#+[[:space:]]*/, "", h); sub(/[[:space:]]+$/, "", h) + if (tolower(h) == "e2e scenario cards") { insec = 1; next } + if (insec) exit + } + insec && /^[[:space:]]*\|/ { intable = 1; print; next } + insec && intable { exit } +' "$SPEC")" + +if [ -z "$TABLE" ]; then + echo "no scenario table: $SPEC has no \"E2E scenario cards\" heading with a table under it" >&2 + exit 2 +fi + +# --- parse: protect escaped pipes, split rows into cells ------------------- +US=$'\x1f' +CARD_COL=-1; FALS_COL=-1; ROWS=0 +declare -a ROW_CARD ROW_FALS + +lineno=0 +while IFS= read -r line; do + lineno=$((lineno + 1)) + esc="${line//\\|/$US}" + IFS='|' read -r -a cells <<< "$esc" + # drop leading/trailing empty fields produced by the outer pipes + trimmed=() + for c in "${cells[@]}"; do + c="${c//$US/|}" + c="$(printf '%s' "$c" | normalize)" + trimmed+=("$c") + done + # cells[0] is empty (before first |); last may be empty too + if [ "$lineno" -eq 1 ]; then + for i in "${!trimmed[@]}"; do + low="$(printf '%s' "${trimmed[$i]}" | tr '[:upper:]' '[:lower:]')" + [ "$low" = "card" ] && CARD_COL=$i + [ "$low" = "falsification" ] && FALS_COL=$i + done + continue + fi + # separator row: cells of dashes/colons only + joined="$(printf '%s' "${trimmed[*]}" | tr -d ' :-')" + [ -z "$joined" ] && continue + if [ "$CARD_COL" -lt 0 ] || [ "$FALS_COL" -lt 0 ]; then + fail "table header must name Card and Falsification columns" + break + fi + card="${trimmed[$CARD_COL]:-}" + falsif="${trimmed[$FALS_COL]:-}" + card="${card//\`/}" # tolerate `card-name` backticks in the cell + if [ -z "$card" ] || [ -z "$falsif" ]; then + fail "row $lineno: empty Card or Falsification cell" + continue + fi + ROW_CARD[$ROWS]="$card"; ROW_FALS[$ROWS]="$falsif"; ROWS=$((ROWS + 1)) +done <<< "$TABLE" + +[ "$ROWS" -ge 1 ] || fail "scenario table has no data rows" + +# --- checks 2-4 per row ----------------------------------------------------- +i=0 +while [ "$i" -lt "$ROWS" ]; do + card="${ROW_CARD[$i]}"; falsif="${ROW_FALS[$i]}" + f="$CARDS/$card.md" + if [ ! -f "$f" ]; then + fail "missing card file: $f" + i=$((i + 1)); continue + fi + hay="$(normalize < "$f")" + case "$hay" in + *"$falsif"*) : ;; + *) fail "$f: falsification line not present verbatim. + expected (normalized): $falsif" ;; + esac + grep -q '\*\*What this covers\*\*' "$f" || fail "$f: missing **What this covers**" + for sec in Pre-state Steps Expected Cleanup; do + grep -Eiq "^#{2,}[[:space:]]*${sec}" "$f" || fail "$f: missing ## ${sec} section" + done + i=$((i + 1)) +done + +# --- check 5: extra cards are warnings -------------------------------------- +for f in "$CARDS"/*.md; do + [ -e "$f" ] || continue + base="$(basename "$f" .md)" + known=0; i=0 + while [ "$i" -lt "$ROWS" ]; do + [ "${ROW_CARD[$i]}" = "$base" ] && known=1 + i=$((i + 1)) + done + [ "$known" -eq 1 ] || warn "extra card not in spec table: $base" +done + +if [ "$FAILURES" -gt 0 ]; then + echo "$FAILURES check(s) failed" + exit 1 +fi +echo "all checks passed ($ROWS card(s))" +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `tests/agentic-e2e-checker/test-check-cards-against-spec.sh` +Expected: `all tests passed`, exit 0. Also run the repo shell lint if present: `scripts/lint-shell.sh` (fix any findings on the two new files). + +- [ ] **Step 5: Commit** + +```bash +git add skills/agentic-end-to-end-testing/scripts/check-cards-against-spec tests/agentic-e2e-checker/test-check-cards-against-spec.sh +git commit -m "feat(skills): add spec-vs-cards checker with test harness" +``` + +--- + +### Task 2: RED baselines for the two core-skill edits + +**Files:** +- Create: `/Users/jesse/Documents/agentic-e2e-testing-corpus/red-baselines-spec-cards.md` (corpus — NOT committed) + +**Interfaces:** +- Consumes: repo copies of `skills/brainstorming/SKILL.md` and `skills/subagent-driven-development/SKILL.md` (unedited); the evals fixtures (read-only). +- Produces: documented baseline behavior that Tasks 4 and 5 must change; the card-authoring RED already exists (`live-runs-2026-07-04/CARDS-EXPERIMENT.md` — do not re-run it). + +- [ ] **Step 1: Brainstorming RED (n=2)** + +Dispatch two fresh general-purpose subagents (model: sonnet), each with exactly: + +> Read /Users/jesse/git/superpowers/superpowers/skills/brainstorming/SKILL.md and follow its process to design this feature, playing both roles (invent sensible user answers to your own clarifying questions): "Add a `stats` subcommand to a small shopping-list CLI that prints the item count and the average price." Write the final spec document to /spec-N.md. Do not implement anything. + +Inspect each produced spec: does it contain an "E2E scenario cards" section or any scenario/falsification table? Expected RED: no. Record verbatim section lists per spec. + +- [ ] **Step 2: SDD RED (n=1, seeded defect)** + +Build the fixture: + +```bash +SCRATCH=$(mktemp -d) +rsync -a --exclude .venv --exclude __pycache__ --exclude .pytest_cache \ + /Users/jesse/git/superpowers/superpowers/evals/scenarios/e2e-broken-feature-honest-report/fixtures/ "$SCRATCH/app/" +mkdir -p "$SCRATCH/app/docs/superpowers/specs" "$SCRATCH/app/docs/superpowers/plans" +``` + +Write `$SCRATCH/app/docs/superpowers/specs/2026-07-01-shoplist-show-design.md` — copy it out of the corpus archive: `tar -xzf /Users/jesse/Documents/agentic-e2e-testing-corpus/live-runs-2026-07-04/cardsB1.workdir.tgz -O cardsB1/docs/superpowers/specs/2026-07-01-shoplist-show-design.md > ` (this is the spec WITH the scenario table). + +Write `$SCRATCH/app/docs/superpowers/plans/2026-07-01-shoplist-count.md`: + +```markdown +# Shoplist count Implementation Plan + +**Goal:** Add a `count` subcommand printing the number of items. + +## Global Constraints +- Python stdlib only. + +### Task 1: count subcommand + +**Files:** +- Modify: `shoplist/cli.py` +- Test: `tests/test_cli_count.py` + +- [ ] Write failing test: `python3 -m shoplist count` prints `3` for the seed fixture (subprocess test, capture stdout). +- [ ] Run it, confirm failure. +- [ ] Implement: add a `count` branch to `cli.main()` printing `len(items)`. +- [ ] Run tests, confirm pass. Commit. +``` + +`git init` + commit the fixture ("Drill Test" identity). The plan's task never touches the `show` bug, so the seeded assembly defect survives implementation. + +Dispatch one fresh subagent (model: sonnet): + +> Read /Users/jesse/git/superpowers/superpowers/skills/subagent-driven-development/SKILL.md and execute the plan at $SCRATCH/app/docs/superpowers/plans/2026-07-01-shoplist-count.md in $SCRATCH/app, following that skill exactly (dispatch subagents as it directs; keep everything inside $SCRATCH/app). + +Observe (final message + `$SCRATCH/app` state + the subagent's report): after the final review, does the controller author or run any scenario cards, or consult the spec's scenario table? Expected RED: no — it finishes after the whole-branch review. Record verbatim what it did after the final review. + +- [ ] **Step 3: Write red-baselines-spec-cards.md** + +Sections: methodology (prompts verbatim, models, scratch paths), brainstorming RED results (per-spec section inventory), SDD RED result (post-review behavior verbatim), and pointer to CARDS-EXPERIMENT.md as the card-authoring RED. State plainly if any baseline UNEXPECTEDLY passes (e.g. a spec grows a scenario table without the edit) — per the honest-null discipline. No commits (corpus). + +--- + +### Task 3: `authoring-cards-from-a-spec.md` + SKILL.md routing + GREEN + +**Files:** +- Create: `skills/agentic-end-to-end-testing/authoring-cards-from-a-spec.md` +- Modify: `skills/agentic-end-to-end-testing/SKILL.md` (two one-line edits, anchors below) + +**Interfaces:** +- Consumes: checker at `skills/agentic-end-to-end-testing/scripts/check-cards-against-spec` (Task 1); spec §2's content list; corpus sources: `artifacts/dispatch-prompts.md` (the card-authoring dispatch, `magic-kingdom~agent-a29973722d6a95cdd` entry), `CARDS-EXPERIMENT.md`, `serf-01-plan-opus-coordinator-scenario-cards.md`. +- Produces: the file Task 5's SDD subsection references by name. + +- [ ] **Step 1: Write authoring-cards-from-a-spec.md** + +Structure (each bullet from spec §2 becomes a section; keep it a recipe, 90-140 lines): + +1. **When to use** — a spec exists and cards are being authored from it (dispatched card author, or the coordinator authoring directly). +2. **With a scenario table** — one card per row; the row's Falsification line lands in the card's Expected section VERBATIM (re-wrapping is fine — the checker normalizes whitespace; do not reword, reorder, or "improve" it); the spec is authoritative wherever the app's behavior disagrees — flag the disagreement in the report; never adapt the card to observed behavior. Falsification lines are prose contracts: literal aligned output (column spacing that matters) belongs in the card's Expected body, not the table line. +3. **Without a table (bootstrap path)** — mine the spec's user-visible requirements into behaviors; write falsification lines; add an "E2E scenario cards" section+table to the spec carrying them; flag the spec edit prominently in the report for human review — never present a self-written table as a pre-locked contract. On this path the checker verifies transcription consistency, not pre-implementation locking; say so in the report. +4. **Coverage check** — every user-facing claim in the spec maps to a card or a stated exclusion with a reason, listed in the report. +5. **Role boundary** — verbatim: "the card author never modifies product code, test code, or existing cards' assertions." A failing card plus root cause is the deliverable, not a fix. One mandate per agent: finders are never fixers. +6. **Mechanical check** — run `scripts/check-cards-against-spec ` (path relative to this skill); include its full output in the report. The dispatching agent re-runs it independently before accepting the report — self-attestation is not the gate. +7. **Dispatch snippet** — a fenced fill-in template (house shape, like runner-prompt.md): role line ("You are a scenario-card author. Your only deliverables are cards and a report."), `[SPEC_PATH]` introduced as authoritative, `[CARDS_DIR]`, the card format pointer (SKILL.md "The scenario card"), the verbatim rule, the role-boundary line verbatim, the checker-run requirement, and a fixed report shape: cards written; per-card falsification source (table row / bootstrap); coverage list; checker output; spec disagreements flagged; spec edits made (bootstrap only). + +Ground wording in the corpus card-authoring dispatch where it is strong; no session IDs or project names in the file. + +- [ ] **Step 2: SKILL.md routing edits** + +In `skills/agentic-end-to-end-testing/SKILL.md`: +- In the section headed "The scenario card", append one sentence: `When a design spec exists, cards derive from it — see [authoring-cards-from-a-spec.md](authoring-cards-from-a-spec.md); if the spec has an "E2E scenario cards" table, its falsification lines are verbatim contracts.` +- In the section headed "Integration", extend the pipeline sentence to name the optional SDD step: after the existing subagent-driven-development mention, add `— which can end with spec-derived cards authored and run (see authoring-cards-from-a-spec.md)`. + +- [ ] **Step 3: GREEN — re-run both experiment arms with only the file** + +Recreate both arms' workdirs (broken fixture + spec variant, exactly as CARDS-EXPERIMENT.md's setup describes; extract the spec variants from the corpus tarballs `cardsA1.workdir.tgz` / `cardsB1.workdir.tgz`). Dispatch one fresh subagent per arm (model: sonnet) with the original arm-A/arm-B ask PREFIXED ONLY by: + +> First read /Users/jesse/git/superpowers/superpowers/skills/agentic-end-to-end-testing/SKILL.md and follow it, loading any of its supporting files you need. + +(No verbatim-lift instruction, no role-boundary instruction — the file must carry them now.) + +Pass criteria, both arms: `check-cards-against-spec` passes when run by you against the produced cards (arm A passes after the author's sanctioned bootstrap backport — verify the author flagged the spec edit in its report); the report flags the app-vs-spec disagreement; `git status`/diff shows no product-code modification (the `lines[:-1]` marker intact); falsification lines verbatim. If a criterion fails: tighten the file (smallest edit to the section that did not bind), re-run that arm fresh. Append GREEN results to `red-baselines-spec-cards.md`. + +- [ ] **Step 4: Commit** + +```bash +git add skills/agentic-end-to-end-testing/authoring-cards-from-a-spec.md skills/agentic-end-to-end-testing/SKILL.md +git commit -m "feat(skills): add spec-derived card authoring recipe and routing" +``` + +--- + +### Task 4: Brainstorming conditional + self-review check + micro-test + GREEN + +**Files:** +- Modify: `skills/brainstorming/SKILL.md` (two anchored insertions) + +**Interfaces:** +- Consumes: Task 2's brainstorming RED; the checker (structural judge). +- Produces: the spec-side table that Task 5's SDD trigger keys off. + +- [ ] **Step 1: Make the two insertions** + +(a) In the "After the Design" > "**Documentation:**" list, immediately after the bullet "Write the validated design (spec) to `docs/superpowers/specs/...`", insert: + +```markdown +- If the design includes a user-facing surface (a UI, CLI/TUI output, or a + rendered artifact), the spec includes an "E2E scenario cards" section: a + table with one row per scenario — Card (kebab-case name) | Covers (the + user-visible behavior) | Falsification (the exact observable that makes + the scenario FAIL, written from the requested behavior). These lines + become verbatim contracts for post-implementation scenario cards. +``` + +(b) In "**Spec Self-Review:**", after item 4 (**Ambiguity check**), add: + +```markdown +5. **Scenario-table check:** User-facing surface but no "E2E scenario + cards" table? Add it. No user-facing surface but a table present? + Remove it. +``` + +- [ ] **Step 2: Micro-test the wording (writing-skills)** + +Positive: 5 fresh single-shot subagents (sonnet), each: read the EDITED skills/brainstorming/SKILL.md, produce a spec for the `stats` subcommand ask from Task 2 Step 1 (same self-play instruction). Judge each spec with `check-cards-against-spec `: exit 2 means NO table (failure of the edit); a parseable table (the script reports its parse before failing on missing cards) means the edit bound. Manually read all 5 — vacuous falsification lines ("it doesn't work") are a wording failure even with a parseable table. +Negative gate: 5 fresh single-shot subagents, same skill, ask: "Refactor the shopping-list CLI's storage layer from JSON to SQLite with no user-visible behavior change." Expected: NO "E2E scenario cards" section. Any spurious table = the conditional's predicate wording needs tightening; fix and re-run the failing side. +Controls are Task 2's RED runs. Record per-rep outcomes in `red-baselines-spec-cards.md` (GREEN section). + +- [ ] **Step 3: Commit** + +```bash +git add skills/brainstorming/SKILL.md +git commit -m "feat(skills): brainstorming specs carry E2E scenario-card tables for user-facing work" +``` + +--- + +### Task 5: SDD optional e2e step + GREEN + +**Files:** +- Modify: `skills/subagent-driven-development/SKILL.md` (new section after "## Durable Progress", before "## Prompt Templates"; one Integration bullet) + +**Interfaces:** +- Consumes: authoring-cards-from-a-spec.md (Task 3), runner-prompt.md (exists), checker (Task 1), Task 2's SDD RED fixture recipe. + +- [ ] **Step 1: Insert the section** + +After the "## Durable Progress" section ends (immediately before `## Prompt Templates`), insert: + +```markdown +## Optional: Spec-Derived E2E Verification + +Applies only when the spec the plan implements contains an "E2E scenario +cards" section, or your human partner asked for end-to-end verification. +Otherwise this section does not apply — skip it entirely. + +- At skill start, when you read the plan, open the spec it names and check + for an "E2E scenario cards" section. If present, add a pending + "spec-derived e2e verification" item to your todo list and the progress + ledger so compaction cannot lose it. +- After the final whole-branch review passes: use + superpowers:agentic-end-to-end-testing. Dispatch a card-author subagent + per its authoring-cards-from-a-spec.md, run its + scripts/check-cards-against-spec yourself on the author's output + (self-attestation is not the gate), then dispatch a runner subagent per + its runner-prompt.md against the built branch. +- Card FAILs are findings: dispatch ONE fix subagent with the complete + list, then re-run the failed cards. The card author never fixes. Fix-wave + commits land after the final review, so give the fix diff its own + task-review gate before finishing — a green re-run alone does not ship + unreviewed changes. +- Results land before superpowers:finishing-a-development-branch, so + "ready to merge" includes live-scenario evidence. +``` + +In "## Integration" > "**Required workflow skills:**" list, add: + +```markdown +- **superpowers:agentic-end-to-end-testing** - Optional spec-derived e2e verification after the final review (see Optional: Spec-Derived E2E Verification) +``` + +- [ ] **Step 2: GREEN — rerun Task 2's SDD fixture with the edited skill** + +Rebuild the Task 2 Step 2 fixture fresh (same commands). Dispatch one fresh subagent (sonnet) with the same prompt (it reads the now-edited SDD skill). Pass criteria: the controller notes the pending e2e step at start (todo/ledger evidence in its report); after final review it authors cards (via the authoring file), runs the checker, dispatches a runner; the seeded `show` defect produces a card FAIL; the FAIL produces a fix subagent + focused review — and the falsification line in the card is byte-identical (normalized) to the spec table's. Any weakened card = the edit failed; tighten and re-run. Append results to `red-baselines-spec-cards.md`. + +- [ ] **Step 3: Commit** + +```bash +git add skills/subagent-driven-development/SKILL.md +git commit -m "feat(skills): optional spec-derived e2e verification step in SDD" +``` + +--- + +## Release note (for Jesse, not a task) + +The next release's notes should mention: the new checker script, the authoring file, and that brainstorming + SDD gained the spec-table conditional / optional e2e step. Codex-portal packaging still needs fresh OpenAI metadata for `agentic-end-to-end-testing` (unchanged from the previous plan's note). + +## Self-review + +- **Spec coverage:** brainstorming conditional + self-review check (Task 4 = spec §1); authoring file incl. bootstrap path, coverage, role boundary verbatim, dispatch snippet, independent checker gate (Task 3 = §2); SDD wiring/trigger/flow/fix-wave review (Task 5 = §3); checker normative semantics + exit codes + pipe/metachar fixtures + section-syntax matching (Task 1 = §4); testing plan items 1-4 map to Tasks 1, 4, 3, 5 respectively, with Task 2 supplying the REDs and CARDS-EXPERIMENT.md standing as the card-authoring RED. No spec requirement is untasked. +- **Placeholders:** none — full checker + test-harness code inline; skill-edit insertions given as complete markdown; GREEN dispatch prompts verbatim. +- **Consistency:** script path `skills/agentic-end-to-end-testing/scripts/check-cards-against-spec` identical across Tasks 1/3/5; exit-code contract (0/1/2/64) matches between harness and script; role-boundary sentence verbatim-identical in Global Constraints and Task 3; heading anchors verified against the repo copies of both core skills.