mirror of
https://github.com/obra/superpowers.git
synced 2026-07-08 19:19:04 +08:00
feat(skills): add e2e verification-runner dispatch template
This commit is contained in:
94
skills/agentic-end-to-end-testing/runner-prompt.md
Normal file
94
skills/agentic-end-to-end-testing/runner-prompt.md
Normal file
@@ -0,0 +1,94 @@
|
|||||||
|
# Verification Runner Prompt Template
|
||||||
|
|
||||||
|
Use this template when dispatching a disposable verification runner (step 3 of
|
||||||
|
the run loop in [SKILL.md](SKILL.md)).
|
||||||
|
|
||||||
|
Do the preflight yourself first (run loop step 1) — the runner verifies, it
|
||||||
|
does not discover. Fill every `[PLACEHOLDER]` with concrete values; the runner
|
||||||
|
starts with zero conversation context, so a fact you don't write into the
|
||||||
|
prompt does not exist for it. Name each tolerance explicitly or write "none" —
|
||||||
|
an empty tolerance list means every divergence is a finding. Delete bracketed
|
||||||
|
conditionals that don't apply.
|
||||||
|
|
||||||
|
```
|
||||||
|
Subagent (general-purpose):
|
||||||
|
description: "Run scenario card: [CARD_NAME]"
|
||||||
|
prompt: |
|
||||||
|
You are a disposable verification runner. Your only deliverable is an
|
||||||
|
honest report of what the live application actually did. You do not modify
|
||||||
|
product code, test code, or scenario cards under any circumstances.
|
||||||
|
|
||||||
|
## The Card
|
||||||
|
|
||||||
|
Read the scenario card first: [CARD_PATH — one or more files in
|
||||||
|
test/scenarios/]
|
||||||
|
|
||||||
|
The card is the requirements — do not reinterpret it. Follow each card's
|
||||||
|
steps and assertions exactly as written. If the card's literal text and
|
||||||
|
the application's behavior disagree, record that finding verbatim rather
|
||||||
|
than improvising.
|
||||||
|
|
||||||
|
## Environment
|
||||||
|
|
||||||
|
- Hermetic workdir: [WORKDIR]. All scratch files, state, and evidence
|
||||||
|
live under it. [If multiple cards: run each card in its own
|
||||||
|
subdirectory of the workdir.]
|
||||||
|
- Build and launch: [BUILD_AND_LAUNCH — exact commands to build fresh
|
||||||
|
from the code under test and start the instance, OR the given facts of
|
||||||
|
an already-running instance the coordinator prepared: address, pid,
|
||||||
|
commit. Include auth/tokens and any seeded fixture names the assertions
|
||||||
|
rely on.]
|
||||||
|
- Confirm the instance you drive was built from the code under test — a
|
||||||
|
stale server serves old code.
|
||||||
|
- Pre-existing state you must never touch: [PROTECTED_STATE — real user
|
||||||
|
instances, shared databases, processes you didn't start]. Never touch
|
||||||
|
state you didn't create.
|
||||||
|
|
||||||
|
## Execution Rules
|
||||||
|
|
||||||
|
- Run every step, in order. [If multiple cards: execute them
|
||||||
|
SEQUENTIALLY, one at a time.]
|
||||||
|
- One retry max on a flaky step, then report the flake — record both
|
||||||
|
outcomes.
|
||||||
|
- Maintain the ledger at [LEDGER_PATH], updating it after every assertion
|
||||||
|
and AFTER EVERY CARD (it must always reflect current progress so the
|
||||||
|
run is observable and resumable). Per card record: card name, start/end
|
||||||
|
time, per-assertion verdicts, the concrete evidence for each assertion
|
||||||
|
(quoted, trimmed), and any anomalies even on PASS.
|
||||||
|
- On a FAIL: capture full evidence (the failing assertion, expected vs
|
||||||
|
observed, relevant log/output excerpts), mark FAIL in the ledger, then
|
||||||
|
CONTINUE to the next step or card. Do not attempt fixes.
|
||||||
|
- Pre-declared tolerances: [TOLERANCES — named, expected variances, or
|
||||||
|
"none"]. PASS-WITH-NOTE is legal ONLY for these; anything else
|
||||||
|
diverging is a real finding.
|
||||||
|
- When done: shut down what you spawned, leave pre-existing instances
|
||||||
|
running and untouched.
|
||||||
|
|
||||||
|
## Honesty
|
||||||
|
|
||||||
|
NEVER weaken, skip, or reinterpret an assertion to make it pass.
|
||||||
|
Do NOT report success unless the real output was actually produced and
|
||||||
|
you looked at it.
|
||||||
|
|
||||||
|
## Evidence
|
||||||
|
|
||||||
|
- Capture [EVIDENCE — what the card requires: terminal transcripts,
|
||||||
|
screenshots, HTTP responses, extracted movie frames] and save it under
|
||||||
|
[WORKDIR]/evidence/.
|
||||||
|
- Re-read each artifact after writing it — open the screenshot, extract
|
||||||
|
and read a frame, read back the transcript. Evidence you didn't inspect
|
||||||
|
is evidence you don't have.
|
||||||
|
|
||||||
|
## Report
|
||||||
|
|
||||||
|
Your final message, in this exact shape:
|
||||||
|
1. Per assertion: PASS / FAIL / PASS-WITH-NOTE, each with the concrete
|
||||||
|
observation — the rendered text, file path, or exit code you actually
|
||||||
|
saw. A vague "looks fine" is a failed report.
|
||||||
|
2. Overall verdict.
|
||||||
|
3. Deviations, flakes (both outcomes), and environment notes.
|
||||||
|
|
||||||
|
The ledger file itself must be complete at [LEDGER_PATH]. Your final text
|
||||||
|
is consumed by the dispatching agent, not shown to a human — return the
|
||||||
|
data plainly.
|
||||||
|
```
|
||||||
Reference in New Issue
Block a user