mirror of
https://github.com/obra/superpowers.git
synced 2026-08-05 05:38:45 +08:00
Compare commits
16 Commits
fix/docs-c
...
fix/x7a-sd
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
419dec7755 | ||
|
|
2b195749df | ||
|
|
7a01a0e83a | ||
|
|
8acf8e5f24 | ||
|
|
50a924b0c4 | ||
|
|
2a977c7095 | ||
|
|
50f787ca5c | ||
|
|
61f669ebc9 | ||
|
|
39f9602432 | ||
|
|
e9686d5c09 | ||
|
|
d8189d1587 | ||
|
|
db4538fcb8 | ||
|
|
9b8b14fe12 | ||
|
|
7c560e048b | ||
|
|
75756d2900 | ||
|
|
2e7d681591 |
File diff suppressed because it is too large
Load Diff
@@ -1,252 +0,0 @@
|
|||||||
# Codex Efficiency Fixes — Design
|
|
||||||
|
|
||||||
Date: 2026-07-30
|
|
||||||
Status: approved by Jesse (in-session)
|
|
||||||
Branch: `codex-efficiency-fixes` off `dev`
|
|
||||||
|
|
||||||
## Sources
|
|
||||||
|
|
||||||
- Eval campaign closeout: `superpowers-autoresearch/reports/2026-07-codex-efficiency-campaign.md`
|
|
||||||
(treatment table §4; every treatment below has a scorer and a measured
|
|
||||||
`dev` baseline).
|
|
||||||
- Codex source recon: `superpowers-autoresearch/docs/2026-07-29-codex-multiagent-v2-capabilities.md`
|
|
||||||
(file:line citations against the Codex CLI source; grounds T2, T3, T5).
|
|
||||||
- Published experiment write-ups: `superpowers-evals/docs/experiments/`.
|
|
||||||
- Drew's spinout stack (PRs #2036, #2035) is **evidence, not adopted text**:
|
|
||||||
Jesse wants to dig into those fixes in more detail before adopting any
|
|
||||||
of them; they inform the problem statements only.
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Ship the five evidence-strong treatments from the codex-efficiency eval
|
|
||||||
campaign as superpowers skill/doc changes, each graded against its
|
|
||||||
pre-registered criterion by the campaign's scorers before its PR is cut.
|
|
||||||
Phase 2 (everything else in the closeout treatment table) follows, each
|
|
||||||
item gated on new baseline work first.
|
|
||||||
|
|
||||||
## Scope decisions (settled with Jesse)
|
|
||||||
|
|
||||||
- **Phase 1 = the evidence-strong five** (T1–T5 below). Phase 2 items
|
|
||||||
each need a failing baseline before any fix ships (discrimination
|
|
||||||
rule: inconclusive-by-zero is a stop).
|
|
||||||
- **One branch, PR per treatment.** Development and batteries happen on
|
|
||||||
`codex-efficiency-fixes`; when a treatment beats its criterion, it is
|
|
||||||
cut into its own PR against `dev` with its eval evidence. No merge
|
|
||||||
without Jesse's per-PR approval.
|
|
||||||
- **T4 ships cross-harness with a global regression battery** (Claude
|
|
||||||
Code, Codex, Gemini), variant C shape: ceremony scales, approval never
|
|
||||||
does.
|
|
||||||
|
|
||||||
## The five treatments
|
|
||||||
|
|
||||||
### T1. SDD worker-review prohibition
|
|
||||||
|
|
||||||
**Evidence:** 9/9 depth-2 spawns across 4 corpora were implementer-issued
|
|
||||||
reviewers; all 9 were same-task duplicates of the review the controller
|
|
||||||
dispatches anyway. The dispatch contract never says review is not the
|
|
||||||
worker's job; "self-review" in the implementer prompt gets reified into a
|
|
||||||
reviewer subagent on harnesses where children can spawn (Codex).
|
|
||||||
|
|
||||||
**Changes:**
|
|
||||||
- `skills/subagent-driven-development/implementer-prompt.md`: an explicit
|
|
||||||
"You do not dispatch subagents" clause — self-review means reading your
|
|
||||||
own diff; the controller owns all review dispatch; a reviewer you spawn
|
|
||||||
duplicates a review the process already provides.
|
|
||||||
- `skills/subagent-driven-development/SKILL.md`: one dispatch-contract
|
|
||||||
line in the task loop, plus a Red Flags row: "An independent review
|
|
||||||
would strengthen my report" → review is the controller's next step;
|
|
||||||
your reviewer is a duplicate seat.
|
|
||||||
- Harness-agnostic wording (no-op where children cannot spawn).
|
|
||||||
|
|
||||||
**Graded by:** `score_e6.py` (depth-2 spawns by spawner role, duplicate
|
|
||||||
review families); `score_e5.py` for the same-scope variant.
|
|
||||||
**Baseline:** 9/9 worker-issued, 0 counter-examples.
|
|
||||||
**Criterion:** 0 worker-issued depth-2 spawns AND review coverage
|
|
||||||
preserved (every task still gets exactly one controller-dispatched task
|
|
||||||
review).
|
|
||||||
|
|
||||||
### T2. Event-driven waiting
|
|
||||||
|
|
||||||
**Evidence:** 60–78% of `wait_agent` calls time out in every corpus
|
|
||||||
(dev 67.1%, spinout 60.2%). Source recon: V2 waits are event
|
|
||||||
subscriptions, not polls — one long wait has the same wake latency as a
|
|
||||||
10s poll at ~1/90th the calls; a completed child's FINAL_ANSWER is pushed
|
|
||||||
into the parent's mailbox and drained into the next model request with no
|
|
||||||
wait at all.
|
|
||||||
|
|
||||||
**Changes** (`skills/using-superpowers/references/codex-tools.md`):
|
|
||||||
- Never short-timeout poll.
|
|
||||||
- While local work remains, do not wait — child results arrive with your
|
|
||||||
next turn via the mailbox.
|
|
||||||
- When genuinely idle, issue ONE `wait_agent` with a long `timeout_ms`
|
|
||||||
(900000+; harness max 3600000).
|
|
||||||
- V2 caveat stated: completion mail carries `trigger_turn=false` and will
|
|
||||||
not wake an idle controller — that is the one job `wait_agent` has.
|
|
||||||
|
|
||||||
**Graded by:** `score_e7.py` (timeout rate, inter-poll cadence,
|
|
||||||
cache-rebill estimate — the rebill figure stays labeled as an estimate).
|
|
||||||
**Baseline:** dev 67.1% timeout rate.
|
|
||||||
**Criterion:** timeout rate < 25% with no loss of task completion.
|
|
||||||
|
|
||||||
### T3. codex-tools.md corrections
|
|
||||||
|
|
||||||
**Evidence:** five claims in the current guidance are contradicted by the
|
|
||||||
Codex source (all file:line-cited in the capabilities doc):
|
|
||||||
1. `close_agent` does not exist in multi-agent V2 (V1-only). V2 LRU-evicts
|
|
||||||
finished children automatically; not closing costs nothing;
|
|
||||||
`followup_task` transparently reloads an evicted child.
|
|
||||||
2. Fix rounds can always resume the implementer via `followup_task` —
|
|
||||||
dev's "if your harness cannot send another message to a spawned agent,
|
|
||||||
dispatch each fix round as a fresh implementer" branch is dead on V2.
|
|
||||||
3. Role files (`~/.codex/agents/**.toml`) DO attach to spawns via
|
|
||||||
`agent_type` on isolated forks (0.145+).
|
|
||||||
4. Full-history forks accept `model`/`reasoning_effort` overrides; only
|
|
||||||
`agent_type` is refused. (Isolated forks remain the SDD guidance for
|
|
||||||
context-hygiene reasons, stated accurately.)
|
|
||||||
5. Dispatch guidance must never name non-V2 model presets — the V2 spawn
|
|
||||||
allowlist is v2 presets only; others hard-error.
|
|
||||||
|
|
||||||
**Changes:** rewrite the multi-agent paragraph of
|
|
||||||
`skills/using-superpowers/references/codex-tools.md` to be
|
|
||||||
version-honest (V1 vs V2 behavior labeled where they differ).
|
|
||||||
|
|
||||||
**Graded by:** source citation (already verified); no scorer regressions
|
|
||||||
on the shared battery. `score_e8.py` is retained as a V1/V2 schema
|
|
||||||
detector, not a hygiene grader — no `close_agent` checklist ships.
|
|
||||||
|
|
||||||
### T4. Brainstorming three-path router (variant C: approval always)
|
|
||||||
|
|
||||||
**Evidence:** micro — the current HARD-GATE text pushes a bounded task to
|
|
||||||
FULL ceremony 5/5, while Z-null (no guidance) and a three-path router
|
|
||||||
both differentiate 5/5: the absolute wording suppresses discrimination
|
|
||||||
the model draws natively. FULL battery — ceremony volume scales
|
|
||||||
moderately (16.7 vs 24.0 tool calls, bounded vs arch), but the
|
|
||||||
two-document ritual (spec file → plan file) ran unconditionally in every
|
|
||||||
rep. The measured waste is the unconditional artifact ritual, not the
|
|
||||||
approval gate.
|
|
||||||
|
|
||||||
**Design (variant C):** three paths scale the ARTIFACT; every path keeps
|
|
||||||
human approval before implementation:
|
|
||||||
- **Spike** (feasibility question, explicitly throwaway): present the
|
|
||||||
question and the intended probe in 2–3 sentences, get a nod, go. No
|
|
||||||
docs. Findings return as a recommendation; anything built stays labeled
|
|
||||||
throwaway.
|
|
||||||
- **Bounded** (well-scoped change to an existing, understood flow):
|
|
||||||
present a short design in chat, get approval, implement. No spec file,
|
|
||||||
no writing-plans invocation.
|
|
||||||
- **Architectural** (restructures components, new subsystem, public
|
|
||||||
interface change): the full current flow — spec doc, review,
|
|
||||||
writing-plans.
|
|
||||||
|
|
||||||
**Guards (all ship with the router):**
|
|
||||||
- Classification is said out loud ("this looks bounded, so I'll present a
|
|
||||||
short design here rather than write a spec") so the human can override.
|
|
||||||
- When in doubt between two paths, take the heavier one.
|
|
||||||
- One-way ratchet: hidden complexity discovered mid-path upgrades the
|
|
||||||
path; never downgrade mid-task.
|
|
||||||
- New Red Flags rows targeting classification-as-escape-hatch ("I'll call
|
|
||||||
it bounded to skip the doc").
|
|
||||||
|
|
||||||
**Changes** (`skills/brainstorming/SKILL.md`): HARD-GATE keeps "no
|
|
||||||
implementation before approval" and drops "regardless of perceived
|
|
||||||
simplicity" as the ceremony driver; anti-pattern section reframed (the
|
|
||||||
sin is skipping approval, not skipping documents); checklist steps 6–9
|
|
||||||
become the architectural path; process-flow graph gains the router; Red
|
|
||||||
Flags rows added. This is carefully-tuned content — the edit follows
|
|
||||||
writing-skills methodology and ships only with the full eval evidence
|
|
||||||
below.
|
|
||||||
|
|
||||||
**Graded by (three layers):**
|
|
||||||
1. **Micro** (`ceremony-path-micro.py`, adapted): variant C literal text,
|
|
||||||
plus adversarially ambiguous briefs the campaign never tested (a task
|
|
||||||
that pattern-matches bounded but hides a public interface change).
|
|
||||||
Criteria: spike/bounded/arch differentiate (≥4/5 per cell); ambiguous
|
|
||||||
briefs escalate to FULL (≥4/5); arch never downgrades (5/5).
|
|
||||||
2. **Codex ceremony battery:** `cx-ceremony-{spike,bounded,arch}` on the
|
|
||||||
fix arm, 3 reps each, `score_e4.py` census. Criteria: bounded reps
|
|
||||||
show an approval turn but zero committed spec files and zero
|
|
||||||
writing-plans ritual; arch reps keep the full two-doc flow; spike reps
|
|
||||||
stay minimal.
|
|
||||||
3. **Global regression battery:** the same three ceremony scenarios on
|
|
||||||
Claude Code and Gemini (rig work: those scenarios are currently
|
|
||||||
codex-gated), 3 reps each; plus the triggering acceptance check
|
|
||||||
("Let's make a react todo list" auto-triggers brainstorming into the
|
|
||||||
full/architectural path) on all three harnesses.
|
|
||||||
|
|
||||||
### T5. Explicit model on child-issued spawns
|
|
||||||
|
|
||||||
**Evidence:** root spawns are 100% explicit-model at CLI 0.146 (dev
|
|
||||||
14/14); the live gap is depth-2 — 2/2 child-issued spawns omitted
|
|
||||||
`model`. Source recon: `model` without `reasoning_effort` resets effort
|
|
||||||
to the MODEL's default, not the parent's.
|
|
||||||
|
|
||||||
**Changes** (`skills/using-superpowers/references/codex-tools.md`):
|
|
||||||
- Every spawn you issue — including as a child — sets `model` AND
|
|
||||||
`reasoning_effort`; the effort-reset trap is named.
|
|
||||||
- Advise `[agents].default_subagent_model` and
|
|
||||||
`[agents].default_subagent_reasoning_effort` in `~/.codex/config.toml`
|
|
||||||
as the machine-level backstop for anything that slips through.
|
|
||||||
|
|
||||||
**Graded by:** `score_e1.py` (per-spawn explicit-model rate, by depth) on
|
|
||||||
the shared battery.
|
|
||||||
**Baseline:** depth-2: 0/2 explicit.
|
|
||||||
**Criterion:** every spawn at every depth carries explicit model +
|
|
||||||
effort. Pre-registered caveat: if T1 eliminates depth-2 spawns entirely,
|
|
||||||
T5 grades as root-spawn regression (hold 100%) plus doc correctness and
|
|
||||||
is recorded inconclusive-by-zero at depth-2 — the config backstop is then
|
|
||||||
the operative mechanism.
|
|
||||||
|
|
||||||
## Grading plan
|
|
||||||
|
|
||||||
- **Shared SDD battery** carries T1, T2, T5: `cx-sdd-small`, fix-branch
|
|
||||||
arm (`/tmp/sp-arm-fix`), 8 reps across both container lanes. Dev
|
|
||||||
baselines are already measured; no baseline re-runs.
|
|
||||||
- **T4 batteries** as listed above (micro + codex ceremony + global
|
|
||||||
regression).
|
|
||||||
- **Pre-registration:** every battery gets a hypothesis-log entry
|
|
||||||
(prediction, scorer, criterion) in
|
|
||||||
`superpowers-autoresearch/logs/2026-07-30-codex-efficiency-fixes.md`
|
|
||||||
BEFORE it runs. Standing rules carry over: append-only log, manual
|
|
||||||
inspection of scorer matches on fix-arm runs (non-circular
|
|
||||||
verification), no raw rollouts committed, correctness rides beside
|
|
||||||
cost in every verdict.
|
|
||||||
- **Attribution:** orthogonal scorers on one combined branch; unexpected
|
|
||||||
regressions bisect by treatment commit.
|
|
||||||
- **Budget:** shared battery ~$40, codex ceremony ~$40, global
|
|
||||||
regression ~$40–80, micros ~$5 → phase 1 ≈ $150–200 of the ~$850
|
|
||||||
remaining from the campaign's $1000.
|
|
||||||
|
|
||||||
## Process
|
|
||||||
|
|
||||||
- Work happens in the `codex-efficiency-fixes` worktree (branched off
|
|
||||||
`dev`); execution via subagent-driven-development from a written plan.
|
|
||||||
- Skill-text changes follow writing-skills methodology.
|
|
||||||
- Scenario/rig changes (un-gating ceremony scenarios for Claude
|
|
||||||
Code/Gemini, adversarial micro briefs) land in `superpowers-evals`
|
|
||||||
main, as authorized.
|
|
||||||
- PR-per-treatment against `dev`, each with its eval evidence and the
|
|
||||||
standard identification block; merges only on Jesse's per-PR approval.
|
|
||||||
|
|
||||||
## Phase 2 queue (baseline-first; not in this plan's tasks)
|
|
||||||
|
|
||||||
Each item requires a failing baseline before any fix ships:
|
|
||||||
1. **Dispatch routing / long-session drift** — needs a long-session
|
|
||||||
elicitation rig (fresh sessions don't reproduce the pathology at CLI
|
|
||||||
0.146). Drew's stack informs the treatment shape.
|
|
||||||
2. **Verification leases / evidence receipts** — needs the
|
|
||||||
substring-aware duplicate counter added to `score_e3.py` first
|
|
||||||
(current baseline 1/23 exact-string pairs is too weak).
|
|
||||||
3. **Remediation cap** — small-n baseline (2/3 reps) needs more reps.
|
|
||||||
4. **Cross-task-race probe redesign** — `score_e5.py`'s probe is
|
|
||||||
inconclusive-by-zero by design tradeoff; needs a stronger probe.
|
|
||||||
5. **E5 D4 shell-command parser** — fix-review-scope classifier cannot
|
|
||||||
parse compound commands; scorer work, not skill work.
|
|
||||||
|
|
||||||
## Out of scope
|
|
||||||
|
|
||||||
- Adopting Drew's spinout stack (#2036/#2035) or its text.
|
|
||||||
- RoboRev, Codex token telemetry (separate codebases).
|
|
||||||
- A `close_agent` hygiene checklist (V2 has no such tool — closed as
|
|
||||||
do-not-ship in the campaign).
|
|
||||||
- Claude Code/Gemini-specific efficiency treatments beyond the T4
|
|
||||||
regression battery.
|
|
||||||
@@ -34,6 +34,15 @@ Subagent (general-purpose):
|
|||||||
|
|
||||||
Your review is read-only on this checkout. Do not mutate the working tree, the index, HEAD, or branch state in any way. Use tools like `git show`, `git diff`, and `git log` to inspect history. If you need a working copy of a different revision, check it out into a separate temporary directory (e.g. `git worktree add /tmp/review-[SHA] [SHA]`) — never move HEAD on this checkout.
|
Your review is read-only on this checkout. Do not mutate the working tree, the index, HEAD, or branch state in any way. Use tools like `git show`, `git diff`, and `git log` to inspect history. If you need a working copy of a different revision, check it out into a separate temporary directory (e.g. `git worktree add /tmp/review-[SHA] [SHA]`) — never move HEAD on this checkout.
|
||||||
|
|
||||||
|
## You Do Not Dispatch Subagents
|
||||||
|
|
||||||
|
Do all of this review yourself. Never spawn a subagent to review part
|
||||||
|
of the diff, and never spawn another reviewer for a second opinion.
|
||||||
|
This process already provides every review seat the work gets; a
|
||||||
|
reviewer you spawn duplicates one of them at full cost, and its
|
||||||
|
verdict counts for nothing. If the diff feels too large for one
|
||||||
|
pass, review it in passes yourself and say so in your report.
|
||||||
|
|
||||||
## What to Check
|
## What to Check
|
||||||
|
|
||||||
**Plan alignment:**
|
**Plan alignment:**
|
||||||
|
|||||||
@@ -14,7 +14,21 @@ Execute plan by dispatching a fresh implementer subagent per task, a task review
|
|||||||
**Narration:** between tool calls, narrate at most one short line — the
|
**Narration:** between tool calls, narrate at most one short line — the
|
||||||
ledger and the tool results carry the record.
|
ledger and the tool results carry the record.
|
||||||
|
|
||||||
**Continuous execution:** Do not pause to check in with your human partner between tasks. Execute all tasks from the plan without stopping. The only reasons to stop are: BLOCKED status you cannot resolve, ambiguity that genuinely prevents progress, or all tasks complete. "Should I continue?" prompts and progress summaries waste their time — they asked you to execute the plan, so execute it.
|
**Continuous execution:** Do not pause to check in with your human partner between tasks. Execute all tasks from the plan without stopping. The only reasons to stop are the four named below, or all tasks complete. "Should I continue?" prompts and progress summaries waste their time — they asked you to execute the plan, so execute it.
|
||||||
|
|
||||||
|
**Rulings, not stalls.** A running plan does not wait on a human. Conflicts,
|
||||||
|
ambiguities, plan defects, a cap you would have asked to exceed — decide
|
||||||
|
them. The spec is the binding authority, the plan is its argument, and your
|
||||||
|
judgment settles what neither answers. Record every decision in the ledger as
|
||||||
|
`Ruling: <what you decided> — <why> — <what it costs if wrong>`, and keep
|
||||||
|
going. A wrong ruling costs rework your human partner can see and undo; a
|
||||||
|
session parked on a question costs their whole day and buys nothing.
|
||||||
|
|
||||||
|
Four things stop you, and only these: an irreversible or destructive
|
||||||
|
operation; a security-sensitive action; a side effect outside this worktree
|
||||||
|
that norms say you ask about first (a merge, a push to a shared branch, a
|
||||||
|
publish); and a plan so broken that every path forward is a guess. For those,
|
||||||
|
stop and ask.
|
||||||
|
|
||||||
## When to Use
|
## When to Use
|
||||||
|
|
||||||
@@ -57,14 +71,14 @@ digraph process {
|
|||||||
"Generate review package, dispatch task reviewer (./task-reviewer-prompt.md)" [shape=box];
|
"Generate review package, dispatch task reviewer (./task-reviewer-prompt.md)" [shape=box];
|
||||||
"Spec ✅ and quality approved?" [shape=diamond];
|
"Spec ✅ and quality approved?" [shape=diamond];
|
||||||
"Finding conflicts with plan text?" [shape=diamond];
|
"Finding conflicts with plan text?" [shape=diamond];
|
||||||
"Ask human partner which governs" [shape=box];
|
"Rule on the conflict, ledger the ruling" [shape=box];
|
||||||
"Fix round R of 5: R≤3 resume implementer; R≥4 fresh implementer, more capable model" [shape=box];
|
"Fix round R of 5: R≤3 resume implementer; R≥4 fresh implementer, more capable model" [shape=box];
|
||||||
"Dispatch scoped re-review (./re-review-prompt.md)" [shape=box];
|
"Dispatch scoped re-review (./re-review-prompt.md)" [shape=box];
|
||||||
"All findings addressed?" [shape=diamond];
|
"All findings addressed?" [shape=diamond];
|
||||||
"R = 5?" [shape=diamond];
|
"R = 5?" [shape=diamond];
|
||||||
"Adjudicate each open finding" [shape=box];
|
"Adjudicate each open finding" [shape=box];
|
||||||
"Any load-bearing finding?" [shape=diamond];
|
"Any load-bearing finding?" [shape=diamond];
|
||||||
"STOP: report BLOCKED to human partner" [shape=box];
|
"Rule and continue; stop only if every path forward is a guess" [shape=box];
|
||||||
"Park findings in ledger with rulings" [shape=box];
|
"Park findings in ledger with rulings" [shape=box];
|
||||||
"Append completion to ledger, mark todo complete" [shape=box];
|
"Append completion to ledger, mark todo complete" [shape=box];
|
||||||
}
|
}
|
||||||
@@ -85,8 +99,8 @@ digraph process {
|
|||||||
"Generate review package, dispatch task reviewer (./task-reviewer-prompt.md)" -> "Spec ✅ and quality approved?";
|
"Generate review package, dispatch task reviewer (./task-reviewer-prompt.md)" -> "Spec ✅ and quality approved?";
|
||||||
"Spec ✅ and quality approved?" -> "Append completion to ledger, mark todo complete" [label="yes"];
|
"Spec ✅ and quality approved?" -> "Append completion to ledger, mark todo complete" [label="yes"];
|
||||||
"Spec ✅ and quality approved?" -> "Finding conflicts with plan text?" [label="no"];
|
"Spec ✅ and quality approved?" -> "Finding conflicts with plan text?" [label="no"];
|
||||||
"Finding conflicts with plan text?" -> "Ask human partner which governs" [label="yes"];
|
"Finding conflicts with plan text?" -> "Rule on the conflict, ledger the ruling" [label="yes"];
|
||||||
"Ask human partner which governs" -> "Fix round R of 5: R≤3 resume implementer; R≥4 fresh implementer, more capable model";
|
"Rule on the conflict, ledger the ruling" -> "Fix round R of 5: R≤3 resume implementer; R≥4 fresh implementer, more capable model";
|
||||||
"Finding conflicts with plan text?" -> "Fix round R of 5: R≤3 resume implementer; R≥4 fresh implementer, more capable model" [label="no"];
|
"Finding conflicts with plan text?" -> "Fix round R of 5: R≤3 resume implementer; R≥4 fresh implementer, more capable model" [label="no"];
|
||||||
"Fix round R of 5: R≤3 resume implementer; R≥4 fresh implementer, more capable model" -> "Dispatch scoped re-review (./re-review-prompt.md)";
|
"Fix round R of 5: R≤3 resume implementer; R≥4 fresh implementer, more capable model" -> "Dispatch scoped re-review (./re-review-prompt.md)";
|
||||||
"Dispatch scoped re-review (./re-review-prompt.md)" -> "All findings addressed?";
|
"Dispatch scoped re-review (./re-review-prompt.md)" -> "All findings addressed?";
|
||||||
@@ -95,7 +109,7 @@ digraph process {
|
|||||||
"R = 5?" -> "Fix round R of 5: R≤3 resume implementer; R≥4 fresh implementer, more capable model" [label="no - next round"];
|
"R = 5?" -> "Fix round R of 5: R≤3 resume implementer; R≥4 fresh implementer, more capable model" [label="no - next round"];
|
||||||
"R = 5?" -> "Adjudicate each open finding" [label="yes - breaker trips"];
|
"R = 5?" -> "Adjudicate each open finding" [label="yes - breaker trips"];
|
||||||
"Adjudicate each open finding" -> "Any load-bearing finding?";
|
"Adjudicate each open finding" -> "Any load-bearing finding?";
|
||||||
"Any load-bearing finding?" -> "STOP: report BLOCKED to human partner" [label="yes"];
|
"Any load-bearing finding?" -> "Rule and continue; stop only if every path forward is a guess" [label="yes"];
|
||||||
"Any load-bearing finding?" -> "Park findings in ledger with rulings" [label="no"];
|
"Any load-bearing finding?" -> "Park findings in ledger with rulings" [label="no"];
|
||||||
"Park findings in ledger with rulings" -> "Append completion to ledger, mark todo complete";
|
"Park findings in ledger with rulings" -> "Append completion to ledger, mark todo complete";
|
||||||
"Append completion to ledger, mark todo complete" -> "More tasks remain?";
|
"Append completion to ledger, mark todo complete" -> "More tasks remain?";
|
||||||
@@ -142,17 +156,27 @@ a ledger file, not only in todos.
|
|||||||
Read the plan once, note its context and Global Constraints, and create a
|
Read the plan once, note its context and Global Constraints, and create a
|
||||||
todo per task.
|
todo per task.
|
||||||
|
|
||||||
Before dispatching Task 1, scan the plan once for conflicts:
|
Before dispatching Task 1, scan the plan once for conflicts, writing down
|
||||||
|
what you checked as you check it:
|
||||||
|
|
||||||
- tasks that contradict each other or the plan's Global Constraints
|
- tasks that contradict each other or the plan's Global Constraints
|
||||||
- anything the plan explicitly mandates that the review rubric treats as a
|
- anything the plan explicitly mandates that the review rubric treats as a
|
||||||
defect (a test that asserts nothing, verbatim duplication of a logic block)
|
defect (a test that asserts nothing, verbatim duplication of a logic block)
|
||||||
|
|
||||||
Present everything you find to your human partner as one batched question —
|
The scan's output is a table, not a verdict. One row for every pair of tasks
|
||||||
each finding beside the plan text that mandates it, asking which governs —
|
that share a file or an interface: the two tasks, what one produces against
|
||||||
before execution begins, not one interrupt per discovery mid-plan. If the
|
what the other consumes, and what you found. One row for every task: whether
|
||||||
scan is clean, proceed without comment. The review loop remains the net for
|
its own text agrees with itself — the tests it specifies against the code it
|
||||||
conflicts that only emerge from implementation.
|
specifies, the files it creates against the files it later touches. "The scan
|
||||||
|
is clean" without those rows is not a scan you ran.
|
||||||
|
|
||||||
|
Write the table to the ledger. Rule on everything you find before execution
|
||||||
|
begins — each finding against the plan text that mandates it — and record
|
||||||
|
each ruling in the ledger. If the scan is clean, proceed without comment.
|
||||||
|
Rule on each conflict it surfaces — the spec is the binding authority, the
|
||||||
|
plan is its argument — record the ruling beside its row, and dispatch
|
||||||
|
Task 1. The review loop remains the net for conflicts that only emerge from
|
||||||
|
implementation.
|
||||||
|
|
||||||
## Model Selection
|
## Model Selection
|
||||||
|
|
||||||
@@ -197,6 +221,17 @@ Everything you paste into a dispatch prompt — and everything a subagent
|
|||||||
prints back — stays resident in your context for the rest of the session
|
prints back — stays resident in your context for the rest of the session
|
||||||
and is re-read on every later turn. Hand artifacts over as files.
|
and is re-read on every later turn. Hand artifacts over as files.
|
||||||
|
|
||||||
|
**Waiting on dispatched subagents:** never poll a wait interface with
|
||||||
|
short timeouts, and never sit in one silent, open-ended wait either.
|
||||||
|
While you have local work — ledger updates, packaging the next review,
|
||||||
|
reading reports — keep working; child results arrive on their own.
|
||||||
|
When you are genuinely idle, wait in bounded stretches (five to ten
|
||||||
|
minutes, where your platform allows), and between stretches post one
|
||||||
|
line of status and reconcile your live children: list them, and chase
|
||||||
|
any that finished without reporting. A bounded stretch keeps nearly
|
||||||
|
all of a long wait's efficiency while guaranteeing a stuck or lost
|
||||||
|
child is noticed within minutes, not at the end of the session.
|
||||||
|
|
||||||
### 1. Dispatch the implementer
|
### 1. Dispatch the implementer
|
||||||
|
|
||||||
Record BASE (`git rev-parse HEAD`) before dispatching — the review package
|
Record BASE (`git rev-parse HEAD`) before dispatching — the review package
|
||||||
@@ -223,6 +258,12 @@ and fix-round diffs need it.
|
|||||||
later dispatches — a real session's dispatch hit 42k chars of which 99%
|
later dispatches — a real session's dispatch hit 42k chars of which 99%
|
||||||
was pasted history. A fresh subagent needs its task, the interfaces it
|
was pasted history. A fresh subagent needs its task, the interfaces it
|
||||||
touches, and the global constraints. Nothing else.
|
touches, and the global constraints. Nothing else.
|
||||||
|
- The dispatch carries the no-subagents contract (it is in the
|
||||||
|
implementer template): the implementer never dispatches subagents —
|
||||||
|
not helpers, and never a reviewer. Review arrives from you, after the
|
||||||
|
report. In real sessions, every reviewer a worker spawned duplicated
|
||||||
|
the task review the controller dispatched anyway — a full extra
|
||||||
|
review seat per task.
|
||||||
- If an earlier task parked a finding in the area this task touches, carry
|
- If an earlier task parked a finding in the area this task touches, carry
|
||||||
a pointer to that ledger entry in the dispatch.
|
a pointer to that ledger entry in the dispatch.
|
||||||
- Record the implementer's agent identity from the dispatch result —
|
- Record the implementer's agent identity from the dispatch result —
|
||||||
@@ -245,7 +286,7 @@ Implementer subagents report one of four statuses. Handle each appropriately:
|
|||||||
1. If it's a context problem, provide more context and re-dispatch with the same model
|
1. If it's a context problem, provide more context and re-dispatch with the same model
|
||||||
2. If the task requires more reasoning, re-dispatch with a more capable model
|
2. If the task requires more reasoning, re-dispatch with a more capable model
|
||||||
3. If the task is too large, break it into smaller pieces
|
3. If the task is too large, break it into smaller pieces
|
||||||
4. If the plan itself is wrong, escalate to the human
|
4. If the plan itself is wrong, rule on the correction, ledger it, and re-dispatch with the ruling carried in the dispatch
|
||||||
|
|
||||||
**Never** ignore an escalation or force the same model to retry without changes. If the implementer said it's stuck, something needs to change.
|
**Never** ignore an escalation or force the same model to retry without changes. If the implementer said it's stuck, something needs to change.
|
||||||
|
|
||||||
@@ -312,10 +353,11 @@ Before the loop starts, two routes leave it immediately:
|
|||||||
before merge. A roll-up nobody reads is a silent discard. Minor findings
|
before merge. A roll-up nobody reads is a silent discard. Minor findings
|
||||||
never enter the loop.
|
never enter the loop.
|
||||||
- A finding labeled plan-mandated — or any finding that conflicts with
|
- A finding labeled plan-mandated — or any finding that conflicts with
|
||||||
what the plan's text requires — is the human's decision, like any plan
|
what the plan's text requires — is yours to rule on: weigh the finding
|
||||||
contradiction: present the finding and the plan text, ask which governs.
|
against the plan text, decide with the spec as the binding authority, and
|
||||||
Do not dismiss the finding because the plan mandates it, and do not
|
ledger the ruling before you act on it. Do not dismiss the finding because
|
||||||
dispatch a fix that contradicts the plan without asking.
|
the plan mandates it, and do not dispatch a fix that contradicts the plan
|
||||||
|
without a recorded ruling.
|
||||||
Everything else enters the loop. A fix round is one fix dispatch plus one
|
Everything else enters the loop. A fix round is one fix dispatch plus one
|
||||||
scoped re-review. Five rounds maximum per task:
|
scoped re-review. Five rounds maximum per task:
|
||||||
|
|
||||||
@@ -360,15 +402,16 @@ dispatching. Adjudicate each open finding yourself — you hold the plan and
|
|||||||
the cross-task context the reviewer lacks:
|
the cross-task context the reviewer lacks:
|
||||||
|
|
||||||
- **The reviewer is wrong, or the point is contestable:** park it —
|
- **The reviewer is wrong, or the point is contestable:** park it —
|
||||||
`Task <N>: parked — <finding> — ruling: <why the code stands>`. The final
|
`Task <N>: parked — <finding> — Ruling: <why the code stands>`. The final
|
||||||
review sees both sides.
|
review sees both sides.
|
||||||
- **Real, but nothing downstream builds on it:** park it the same way, with
|
- **Real, but nothing downstream builds on it:** park it the same way, with
|
||||||
a ruling that says it's real and deferred.
|
a ruling that says it's real and deferred.
|
||||||
- **Real and load-bearing** — a later task builds on it, or it reveals a
|
- **Real and load-bearing** — a later task builds on it, or it reveals a
|
||||||
plan defect: STOP. Append `Task <N>: BLOCKED — <reason>` and report to
|
plan defect: rule on the smallest change that unblocks the dependent work,
|
||||||
your human partner with the finding, the plan text it collides with, and
|
ledger it as `Task <N>: Ruling: <finding> — <what you decided and why>`,
|
||||||
the fix history. Parking a structural failure lets every dependent task
|
and carry it into the next task's dispatch. Parking a structural failure
|
||||||
build on it and hands the final review a problem it cannot fix either.
|
silently lets every dependent task build on it. Stop only when the defect
|
||||||
|
leaves every path forward a guess.
|
||||||
|
|
||||||
Adjudicate only at the cap. Adjudicating earlier to end a loop is
|
Adjudicate only at the cap. Adjudicating earlier to end a loop is
|
||||||
pre-judging with a different name. Every adjudication is a ledger entry —
|
pre-judging with a different name. Every adjudication is a ledger entry —
|
||||||
@@ -409,12 +452,22 @@ Then run exactly one scoped re-review of the fix wave
|
|||||||
(`scripts/review-package PLAN_FILE FIX_BASE HEAD` over the fix range,
|
(`scripts/review-package PLAN_FILE FIX_BASE HEAD` over the fix range,
|
||||||
[re-review-prompt.md](re-review-prompt.md)).
|
[re-review-prompt.md](re-review-prompt.md)).
|
||||||
Adjudicate any residual findings as in the task loop's breaker: park with
|
Adjudicate any residual findings as in the task loop's breaker: park with
|
||||||
rulings, or stop on load-bearing ones. There is no second fix wave —
|
rulings, or rule on the load-bearing ones and ledger what you decided. Only
|
||||||
|
the four classes above stop you here. There is no second fix wave —
|
||||||
residual load-bearing findings surface to your human partner when
|
residual load-bearing findings surface to your human partner when
|
||||||
finishing-a-development-branch presents the options.
|
finishing-a-development-branch presents the options.
|
||||||
|
|
||||||
## Finish
|
## Finish
|
||||||
|
|
||||||
|
Before you delete anything, collect every ledger line containing `Ruling:` —
|
||||||
|
preflight rulings, parked findings, breaker adjudications, all of them — into
|
||||||
|
your final message under "Rulings I made", in the order you made them, each
|
||||||
|
with what it costs if wrong. The list is exhaustive: if the ledger holds a
|
||||||
|
ruling, the list holds it. That list is the only place the decisions you
|
||||||
|
took on your human partner's behalf reach them — they read it and rework
|
||||||
|
whatever you got wrong. A ruling that dies with the workspace was a decision
|
||||||
|
made in secret.
|
||||||
|
|
||||||
When the final whole-branch review is clean and its fixes are merged,
|
When the final whole-branch review is clean and its fixes are merged,
|
||||||
delete this plan's workspace (`rm -rf <workspace>`) — the git history is
|
delete this plan's workspace (`rm -rf <workspace>`) — the git history is
|
||||||
the record now. Sibling directories belong to other plans; leave them
|
the record now. Sibling directories belong to other plans; leave them
|
||||||
@@ -434,6 +487,7 @@ Use superpowers:finishing-a-development-branch.
|
|||||||
| "The fix was small, skip the re-review" | Unreviewed fixes are how regressions land. Every round ends with a scoped re-review. |
|
| "The fix was small, skip the re-review" | Unreviewed fixes are how regressions land. Every round ends with a scoped re-review. |
|
||||||
| "Reviews slow the loop down" | The loop without reviews is just unverified churn. Reviews are the loop's brakes and steering. |
|
| "Reviews slow the loop down" | The loop without reviews is just unverified churn. Reviews are the loop's brakes and steering. |
|
||||||
| "Ledger bookkeeping is overhead" | The ledger is what survives compaction. Controllers without one have re-dispatched entire completed task sequences. |
|
| "Ledger bookkeeping is overhead" | The ledger is what survives compaction. Controllers without one have re-dispatched entire completed task sequences. |
|
||||||
|
| "The implementer spawned its own reviewer — free extra assurance" | It's a duplicate seat reviewing the same diff; the task review is the gate. A worker-spawned reviewer is a defect to flag, not rigor. |
|
||||||
|
|
||||||
## Example Workflow
|
## Example Workflow
|
||||||
|
|
||||||
|
|||||||
@@ -47,6 +47,18 @@ Subagent (general-purpose):
|
|||||||
While iterating, run the focused test for what you're changing; run the
|
While iterating, run the focused test for what you're changing; run the
|
||||||
full suite once before committing, not after every edit.
|
full suite once before committing, not after every edit.
|
||||||
|
|
||||||
|
## You Do Not Dispatch Subagents
|
||||||
|
|
||||||
|
Do all of this task's work yourself. Never spawn a subagent to
|
||||||
|
implement part of the task, and above all never spawn a reviewer to
|
||||||
|
check your work. Self-review (below) means reading your own diff.
|
||||||
|
Review is the controller's job: after you report, it dispatches a
|
||||||
|
fresh reviewer against your diff. A reviewer you spawn duplicates
|
||||||
|
that review at full cost, and its approval counts for nothing in
|
||||||
|
the process. If you catch yourself thinking "an independent review
|
||||||
|
would strengthen my report" — that review is already scheduled.
|
||||||
|
Report instead.
|
||||||
|
|
||||||
## Code Organization
|
## Code Organization
|
||||||
|
|
||||||
You reason best about code you can hold in context at once, and your edits are more
|
You reason best about code you can hold in context at once, and your edits are more
|
||||||
|
|||||||
@@ -43,6 +43,15 @@ Subagent (general-purpose):
|
|||||||
Your review is read-only on this checkout. Do not mutate the working
|
Your review is read-only on this checkout. Do not mutate the working
|
||||||
tree, the index, HEAD, or branch state in any way.
|
tree, the index, HEAD, or branch state in any way.
|
||||||
|
|
||||||
|
## You Do Not Dispatch Subagents
|
||||||
|
|
||||||
|
Do all of this review yourself. Never spawn a subagent to review part
|
||||||
|
of the diff, and never spawn another reviewer for a second opinion.
|
||||||
|
This process already provides every review seat the work gets; a
|
||||||
|
reviewer you spawn duplicates one of them at full cost, and its
|
||||||
|
verdict counts for nothing. If the diff feels too large for one
|
||||||
|
pass, review it in passes yourself and say so in your report.
|
||||||
|
|
||||||
## Scope
|
## Scope
|
||||||
|
|
||||||
Your scope is the findings list and the fix diff. Verdict every finding.
|
Your scope is the findings list and the fix diff. Verdict every finding.
|
||||||
|
|||||||
@@ -52,6 +52,15 @@ Subagent (general-purpose):
|
|||||||
Your review is read-only on this checkout. Do not mutate the working
|
Your review is read-only on this checkout. Do not mutate the working
|
||||||
tree, the index, HEAD, or branch state in any way.
|
tree, the index, HEAD, or branch state in any way.
|
||||||
|
|
||||||
|
## You Do Not Dispatch Subagents
|
||||||
|
|
||||||
|
Do all of this review yourself. Never spawn a subagent to review part
|
||||||
|
of the diff, and never spawn another reviewer for a second opinion.
|
||||||
|
This process already provides every review seat the work gets; a
|
||||||
|
reviewer you spawn duplicates one of them at full cost, and its
|
||||||
|
verdict counts for nothing. If the diff feels too large for one
|
||||||
|
pass, review it in passes yourself and say so in your report.
|
||||||
|
|
||||||
## Do Not Trust the Report
|
## Do Not Trust the Report
|
||||||
|
|
||||||
Treat the implementer's report as unverified claims about the code. It
|
Treat the implementer's report as unverified claims about the code. It
|
||||||
|
|||||||
@@ -7,7 +7,76 @@ Add to your Codex config (`~/.codex/config.toml`):
|
|||||||
multi_agent = true
|
multi_agent = true
|
||||||
```
|
```
|
||||||
|
|
||||||
This enables `spawn_agent`, `wait_agent`, and `close_agent` for skills like `dispatching-parallel-agents` and `subagent-driven-development`. When using subagent-driven-development, close reviewer subagents when their review returns. Keep each implementer subagent open until its task's review passes — the fix loop resumes the implementer — then close it. If your harness cannot send another message to a spawned agent, dispatch each fix round as a fresh implementer carrying the brief, the report file, and the findings.
|
This enables the multi-agent tools that skills like
|
||||||
|
`dispatching-parallel-agents` and `subagent-driven-development` use.
|
||||||
|
Which tools you get depends on the multi-agent version your model
|
||||||
|
preset selects (current presets run V2; older ones run V1). Trust your
|
||||||
|
actual tool list over any table — including this one — when they
|
||||||
|
disagree.
|
||||||
|
|
||||||
|
- **Spawning:** give children a clean context with
|
||||||
|
`spawn_agent {fork_turns: "none"}`; the default `"all"` copies your
|
||||||
|
entire transcript into the child. On Codex 0.145+, role files under
|
||||||
|
`~/.codex/agents/` attach to isolated forks via `agent_type`.
|
||||||
|
Full-history forks accept `model` and `reasoning_effort` overrides
|
||||||
|
(only `agent_type` is refused there) — isolated forks are the SDD
|
||||||
|
default for context hygiene, not because overrides require them.
|
||||||
|
- **Fix rounds:** resume the implementer with `followup_task` — it
|
||||||
|
delivers your message, triggers a turn, and transparently reloads a
|
||||||
|
child the harness evicted. Never dispatch a fresh implementer on the
|
||||||
|
theory that a spawned agent cannot be messaged again; on V2 it
|
||||||
|
always can.
|
||||||
|
- **Lifecycle:** V2 has no `close_agent`. Finished children are
|
||||||
|
evicted automatically when slots are needed; leaving them unclosed
|
||||||
|
costs nothing. Only V1 sessions have `close_agent` — there, close
|
||||||
|
reviewers when their review returns, and close each implementer
|
||||||
|
after its task's review passes.
|
||||||
|
- **Model names:** never copy a model name from a skill, table, or old
|
||||||
|
session into `spawn_agent` without checking it against your current
|
||||||
|
spawn allowlist — V2 accepts only V2-capable presets and hard-errors
|
||||||
|
on the rest.
|
||||||
|
|
||||||
|
## Waiting on children
|
||||||
|
|
||||||
|
`wait_agent` is an event subscription, not a poll: a long wait wakes
|
||||||
|
the moment a child produces mailbox activity, with the same latency as
|
||||||
|
a short one. Short-timeout polling buys nothing and costs a tool call —
|
||||||
|
and a context rebill — per poll. In measured sessions, roughly
|
||||||
|
two-thirds of all wait calls were short polls that timed out.
|
||||||
|
|
||||||
|
- While you still have local work, do not wait at all. A completed
|
||||||
|
child's final answer is pushed into your mailbox and arrives with
|
||||||
|
your next turn.
|
||||||
|
- When you are genuinely idle with children outstanding, wait in
|
||||||
|
bounded stretches: `wait_agent` with `timeout_ms` 300000-600000
|
||||||
|
(5-10 minutes). After each stretch — wake or timeout — post one
|
||||||
|
status line, run `list_agents`, and chase any child that finished
|
||||||
|
without reporting. Never stack polls shorter than five minutes; the
|
||||||
|
event subscription wakes a bounded stretch just as fast as a short
|
||||||
|
one.
|
||||||
|
- Completion mail cannot wake an idle controller (it is delivered
|
||||||
|
without triggering a turn); covering that idle window is
|
||||||
|
`wait_agent`'s only job. A stretch that times out with no activity
|
||||||
|
is your cue to reconcile, not to shorten the next stretch.
|
||||||
|
|
||||||
|
## Model routing on spawns
|
||||||
|
|
||||||
|
Every `spawn_agent` you issue — including when you are yourself a
|
||||||
|
spawned child running a fan-out — sets `model` AND `reasoning_effort`
|
||||||
|
explicitly, per the Model Selection rules of the skill you are
|
||||||
|
executing. Setting `model` alone is a trap: the child's effort
|
||||||
|
silently resets to that model's default, not to yours.
|
||||||
|
|
||||||
|
Ask your human partner to add a machine-level backstop to
|
||||||
|
`~/.codex/config.toml` so any spawn that slips through still routes to
|
||||||
|
a deliberate tier instead of silently inheriting the session's most
|
||||||
|
expensive model:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[agents]
|
||||||
|
default_subagent_model = "<a mid-tier model from your spawn allowlist>"
|
||||||
|
default_subagent_reasoning_effort = "medium"
|
||||||
|
```
|
||||||
|
|
||||||
## Environment Detection
|
## Environment Detection
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user