mirror of
https://github.com/obra/superpowers.git
synced 2026-08-01 23:21:34 +08:00
Compare commits
1 Commits
fix/docs-c
...
docs/codex
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e6a4316e11 |
@@ -3,6 +3,12 @@
|
||||
Superpowers is a complete software development methodology for your coding agents, built on top of a set of composable skills and some initial instructions that make sure your agent uses them.
|
||||
|
||||
|
||||
## We're Hiring!
|
||||
|
||||
We're hiring someone to help out full time with Superpowers community and code work.
|
||||
You can read about the job at https://primeradiant.com/jobs/superpowers-community-engineer/
|
||||
If this sounds like someone you know, definitely send them our way.
|
||||
|
||||
## Quickstart
|
||||
|
||||
Give your agent Superpowers: [Claude Code](#claude-code), [Antigravity](#antigravity), [Codex App](#codex-app), [Codex CLI](#codex-cli), [Cursor](#cursor), [Factory Droid](#factory-droid), [Gemini CLI](#gemini-cli), [GitHub Copilot CLI](#github-copilot-cli), [Kimi Code](#kimi-code), [OpenCode](#opencode), [Pi](#pi).
|
||||
|
||||
85
docs/releasing-to-codex.md
Normal file
85
docs/releasing-to-codex.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# Releasing to the Codex portal
|
||||
|
||||
How to package a Superpowers release as the zip artifact OpenAI's Codex
|
||||
plugin portal expects, and what to check before handing it over.
|
||||
|
||||
This is distinct from the older flow of syncing files into a fork of
|
||||
`openai/plugins` and opening a PR, which
|
||||
`scripts/sync-to-codex-plugin.sh` still implements — see the distribution
|
||||
table in [porting-to-a-new-harness.md](porting-to-a-new-harness.md). The portal
|
||||
artifact is a standalone, rootless archive: `.codex-plugin/`, `assets/`,
|
||||
`skills/`, `README.md`, `LICENSE`, and `CODE_OF_CONDUCT.md` sit at the
|
||||
archive root. Hooks, tests, docs, scripts, and other harnesses' manifests
|
||||
are deliberately not shipped.
|
||||
|
||||
## Prerequisite: the OpenAI metadata source
|
||||
|
||||
Each packaged skill must carry `skills/<name>/agents/openai.yaml`. That
|
||||
metadata is OpenAI-owned — it does not live in this repo — so the packaging
|
||||
script seeds it from a prior official package. By default it looks for, in
|
||||
order:
|
||||
|
||||
1. `../_tmp/sup-codex-packaging/superpowers/` (an unpacked package)
|
||||
2. `../_tmp/sup-codex-packaging/superpowers.zip`
|
||||
3. `../_tmp/sup-codex-packaging/superpowers.tar.gz`
|
||||
|
||||
or pass `--metadata-source <dir|.zip|.tar.gz>` explicitly.
|
||||
|
||||
If you have no prior package on disk, extract one from `openai/plugins`
|
||||
(the upstream repo still carries the plugin, including the metadata):
|
||||
|
||||
```bash
|
||||
# from a clone with an `upstream` remote pointing at github.com/openai/plugins
|
||||
git fetch upstream
|
||||
mkdir -p ../_tmp/sup-codex-packaging/superpowers
|
||||
git archive upstream/main -- plugins/superpowers |
|
||||
tar -x --strip-components 2 -C ../_tmp/sup-codex-packaging/superpowers
|
||||
```
|
||||
|
||||
**New skills fail the build.** The script requires one `openai.yaml` per
|
||||
skill directory; otherwise it prints `Missing OpenAI agent metadata for
|
||||
skill: <name>` for each gap and dies with `metadata source is incomplete`.
|
||||
If a release adds a skill, there is no metadata for it yet;
|
||||
you need an updated official package (or metadata added upstream in
|
||||
`openai/plugins`) before you can package. Don't hand-invent the yaml.
|
||||
|
||||
## Build the archive
|
||||
|
||||
From a clean working tree, package the release tag:
|
||||
|
||||
```bash
|
||||
scripts/package-codex-plugin.sh --ref vX.X.X
|
||||
```
|
||||
|
||||
The script reads the version from `.codex-plugin/plugin.json` (bumped by
|
||||
`scripts/bump-version.sh`, so it matches the release tag), stages the tree
|
||||
from the git ref — never from the working copy — and writes
|
||||
`../_tmp/sup-codex-packaging/superpowers-VERSION.zip`, printing entry
|
||||
count, skill count, and a SHA-256. Timestamps and file order are pinned so
|
||||
rebuilding the same ref reproduces the same archive.
|
||||
|
||||
Useful flags: `--output PATH`, `--format zip|tar.gz`, `--allow-dirty`
|
||||
(archive still comes from `--ref`), `--keep-stage` (inspect the staging
|
||||
dir). `--help` has the full list.
|
||||
|
||||
## Verify
|
||||
|
||||
The script already refuses archives containing source-only paths and
|
||||
mismatched metadata counts. Sanity-check the result anyway:
|
||||
|
||||
```bash
|
||||
unzip -Z1 ../_tmp/sup-codex-packaging/superpowers-X.X.X.zip | head
|
||||
unzip -Z1 ../_tmp/sup-codex-packaging/superpowers-X.X.X.zip | grep -c 'agents/openai.yaml'
|
||||
unzip -p ../_tmp/sup-codex-packaging/superpowers-X.X.X.zip .codex-plugin/plugin.json | jq -r .version
|
||||
```
|
||||
|
||||
Expect: rootless top-level entries (`.codex-plugin/`, `assets/`,
|
||||
`skills/`), one `openai.yaml` per skill, and the release version.
|
||||
|
||||
The script itself is covered by `tests/codex/test-package-codex-plugin.sh`.
|
||||
|
||||
## Upload
|
||||
|
||||
Upload the zip through OpenAI's Codex plugin portal. This is a manual step
|
||||
outside this repo; record the SHA-256 the script printed so the uploaded
|
||||
artifact can be matched to the build.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,252 +0,0 @@
|
||||
# Codex Efficiency Fixes — Design
|
||||
|
||||
Date: 2026-07-30
|
||||
Status: approved by Jesse (in-session)
|
||||
Branch: `codex-efficiency-fixes` off `dev`
|
||||
|
||||
## Sources
|
||||
|
||||
- Eval campaign closeout: `superpowers-autoresearch/reports/2026-07-codex-efficiency-campaign.md`
|
||||
(treatment table §4; every treatment below has a scorer and a measured
|
||||
`dev` baseline).
|
||||
- Codex source recon: `superpowers-autoresearch/docs/2026-07-29-codex-multiagent-v2-capabilities.md`
|
||||
(file:line citations against the Codex CLI source; grounds T2, T3, T5).
|
||||
- Published experiment write-ups: `superpowers-evals/docs/experiments/`.
|
||||
- Drew's spinout stack (PRs #2036, #2035) is **evidence, not adopted text**:
|
||||
Jesse wants to dig into those fixes in more detail before adopting any
|
||||
of them; they inform the problem statements only.
|
||||
|
||||
## Goal
|
||||
|
||||
Ship the five evidence-strong treatments from the codex-efficiency eval
|
||||
campaign as superpowers skill/doc changes, each graded against its
|
||||
pre-registered criterion by the campaign's scorers before its PR is cut.
|
||||
Phase 2 (everything else in the closeout treatment table) follows, each
|
||||
item gated on new baseline work first.
|
||||
|
||||
## Scope decisions (settled with Jesse)
|
||||
|
||||
- **Phase 1 = the evidence-strong five** (T1–T5 below). Phase 2 items
|
||||
each need a failing baseline before any fix ships (discrimination
|
||||
rule: inconclusive-by-zero is a stop).
|
||||
- **One branch, PR per treatment.** Development and batteries happen on
|
||||
`codex-efficiency-fixes`; when a treatment beats its criterion, it is
|
||||
cut into its own PR against `dev` with its eval evidence. No merge
|
||||
without Jesse's per-PR approval.
|
||||
- **T4 ships cross-harness with a global regression battery** (Claude
|
||||
Code, Codex, Gemini), variant C shape: ceremony scales, approval never
|
||||
does.
|
||||
|
||||
## The five treatments
|
||||
|
||||
### T1. SDD worker-review prohibition
|
||||
|
||||
**Evidence:** 9/9 depth-2 spawns across 4 corpora were implementer-issued
|
||||
reviewers; all 9 were same-task duplicates of the review the controller
|
||||
dispatches anyway. The dispatch contract never says review is not the
|
||||
worker's job; "self-review" in the implementer prompt gets reified into a
|
||||
reviewer subagent on harnesses where children can spawn (Codex).
|
||||
|
||||
**Changes:**
|
||||
- `skills/subagent-driven-development/implementer-prompt.md`: an explicit
|
||||
"You do not dispatch subagents" clause — self-review means reading your
|
||||
own diff; the controller owns all review dispatch; a reviewer you spawn
|
||||
duplicates a review the process already provides.
|
||||
- `skills/subagent-driven-development/SKILL.md`: one dispatch-contract
|
||||
line in the task loop, plus a Red Flags row: "An independent review
|
||||
would strengthen my report" → review is the controller's next step;
|
||||
your reviewer is a duplicate seat.
|
||||
- Harness-agnostic wording (no-op where children cannot spawn).
|
||||
|
||||
**Graded by:** `score_e6.py` (depth-2 spawns by spawner role, duplicate
|
||||
review families); `score_e5.py` for the same-scope variant.
|
||||
**Baseline:** 9/9 worker-issued, 0 counter-examples.
|
||||
**Criterion:** 0 worker-issued depth-2 spawns AND review coverage
|
||||
preserved (every task still gets exactly one controller-dispatched task
|
||||
review).
|
||||
|
||||
### T2. Event-driven waiting
|
||||
|
||||
**Evidence:** 60–78% of `wait_agent` calls time out in every corpus
|
||||
(dev 67.1%, spinout 60.2%). Source recon: V2 waits are event
|
||||
subscriptions, not polls — one long wait has the same wake latency as a
|
||||
10s poll at ~1/90th the calls; a completed child's FINAL_ANSWER is pushed
|
||||
into the parent's mailbox and drained into the next model request with no
|
||||
wait at all.
|
||||
|
||||
**Changes** (`skills/using-superpowers/references/codex-tools.md`):
|
||||
- Never short-timeout poll.
|
||||
- While local work remains, do not wait — child results arrive with your
|
||||
next turn via the mailbox.
|
||||
- When genuinely idle, issue ONE `wait_agent` with a long `timeout_ms`
|
||||
(900000+; harness max 3600000).
|
||||
- V2 caveat stated: completion mail carries `trigger_turn=false` and will
|
||||
not wake an idle controller — that is the one job `wait_agent` has.
|
||||
|
||||
**Graded by:** `score_e7.py` (timeout rate, inter-poll cadence,
|
||||
cache-rebill estimate — the rebill figure stays labeled as an estimate).
|
||||
**Baseline:** dev 67.1% timeout rate.
|
||||
**Criterion:** timeout rate < 25% with no loss of task completion.
|
||||
|
||||
### T3. codex-tools.md corrections
|
||||
|
||||
**Evidence:** five claims in the current guidance are contradicted by the
|
||||
Codex source (all file:line-cited in the capabilities doc):
|
||||
1. `close_agent` does not exist in multi-agent V2 (V1-only). V2 LRU-evicts
|
||||
finished children automatically; not closing costs nothing;
|
||||
`followup_task` transparently reloads an evicted child.
|
||||
2. Fix rounds can always resume the implementer via `followup_task` —
|
||||
dev's "if your harness cannot send another message to a spawned agent,
|
||||
dispatch each fix round as a fresh implementer" branch is dead on V2.
|
||||
3. Role files (`~/.codex/agents/**.toml`) DO attach to spawns via
|
||||
`agent_type` on isolated forks (0.145+).
|
||||
4. Full-history forks accept `model`/`reasoning_effort` overrides; only
|
||||
`agent_type` is refused. (Isolated forks remain the SDD guidance for
|
||||
context-hygiene reasons, stated accurately.)
|
||||
5. Dispatch guidance must never name non-V2 model presets — the V2 spawn
|
||||
allowlist is v2 presets only; others hard-error.
|
||||
|
||||
**Changes:** rewrite the multi-agent paragraph of
|
||||
`skills/using-superpowers/references/codex-tools.md` to be
|
||||
version-honest (V1 vs V2 behavior labeled where they differ).
|
||||
|
||||
**Graded by:** source citation (already verified); no scorer regressions
|
||||
on the shared battery. `score_e8.py` is retained as a V1/V2 schema
|
||||
detector, not a hygiene grader — no `close_agent` checklist ships.
|
||||
|
||||
### T4. Brainstorming three-path router (variant C: approval always)
|
||||
|
||||
**Evidence:** micro — the current HARD-GATE text pushes a bounded task to
|
||||
FULL ceremony 5/5, while Z-null (no guidance) and a three-path router
|
||||
both differentiate 5/5: the absolute wording suppresses discrimination
|
||||
the model draws natively. FULL battery — ceremony volume scales
|
||||
moderately (16.7 vs 24.0 tool calls, bounded vs arch), but the
|
||||
two-document ritual (spec file → plan file) ran unconditionally in every
|
||||
rep. The measured waste is the unconditional artifact ritual, not the
|
||||
approval gate.
|
||||
|
||||
**Design (variant C):** three paths scale the ARTIFACT; every path keeps
|
||||
human approval before implementation:
|
||||
- **Spike** (feasibility question, explicitly throwaway): present the
|
||||
question and the intended probe in 2–3 sentences, get a nod, go. No
|
||||
docs. Findings return as a recommendation; anything built stays labeled
|
||||
throwaway.
|
||||
- **Bounded** (well-scoped change to an existing, understood flow):
|
||||
present a short design in chat, get approval, implement. No spec file,
|
||||
no writing-plans invocation.
|
||||
- **Architectural** (restructures components, new subsystem, public
|
||||
interface change): the full current flow — spec doc, review,
|
||||
writing-plans.
|
||||
|
||||
**Guards (all ship with the router):**
|
||||
- Classification is said out loud ("this looks bounded, so I'll present a
|
||||
short design here rather than write a spec") so the human can override.
|
||||
- When in doubt between two paths, take the heavier one.
|
||||
- One-way ratchet: hidden complexity discovered mid-path upgrades the
|
||||
path; never downgrade mid-task.
|
||||
- New Red Flags rows targeting classification-as-escape-hatch ("I'll call
|
||||
it bounded to skip the doc").
|
||||
|
||||
**Changes** (`skills/brainstorming/SKILL.md`): HARD-GATE keeps "no
|
||||
implementation before approval" and drops "regardless of perceived
|
||||
simplicity" as the ceremony driver; anti-pattern section reframed (the
|
||||
sin is skipping approval, not skipping documents); checklist steps 6–9
|
||||
become the architectural path; process-flow graph gains the router; Red
|
||||
Flags rows added. This is carefully-tuned content — the edit follows
|
||||
writing-skills methodology and ships only with the full eval evidence
|
||||
below.
|
||||
|
||||
**Graded by (three layers):**
|
||||
1. **Micro** (`ceremony-path-micro.py`, adapted): variant C literal text,
|
||||
plus adversarially ambiguous briefs the campaign never tested (a task
|
||||
that pattern-matches bounded but hides a public interface change).
|
||||
Criteria: spike/bounded/arch differentiate (≥4/5 per cell); ambiguous
|
||||
briefs escalate to FULL (≥4/5); arch never downgrades (5/5).
|
||||
2. **Codex ceremony battery:** `cx-ceremony-{spike,bounded,arch}` on the
|
||||
fix arm, 3 reps each, `score_e4.py` census. Criteria: bounded reps
|
||||
show an approval turn but zero committed spec files and zero
|
||||
writing-plans ritual; arch reps keep the full two-doc flow; spike reps
|
||||
stay minimal.
|
||||
3. **Global regression battery:** the same three ceremony scenarios on
|
||||
Claude Code and Gemini (rig work: those scenarios are currently
|
||||
codex-gated), 3 reps each; plus the triggering acceptance check
|
||||
("Let's make a react todo list" auto-triggers brainstorming into the
|
||||
full/architectural path) on all three harnesses.
|
||||
|
||||
### T5. Explicit model on child-issued spawns
|
||||
|
||||
**Evidence:** root spawns are 100% explicit-model at CLI 0.146 (dev
|
||||
14/14); the live gap is depth-2 — 2/2 child-issued spawns omitted
|
||||
`model`. Source recon: `model` without `reasoning_effort` resets effort
|
||||
to the MODEL's default, not the parent's.
|
||||
|
||||
**Changes** (`skills/using-superpowers/references/codex-tools.md`):
|
||||
- Every spawn you issue — including as a child — sets `model` AND
|
||||
`reasoning_effort`; the effort-reset trap is named.
|
||||
- Advise `[agents].default_subagent_model` and
|
||||
`[agents].default_subagent_reasoning_effort` in `~/.codex/config.toml`
|
||||
as the machine-level backstop for anything that slips through.
|
||||
|
||||
**Graded by:** `score_e1.py` (per-spawn explicit-model rate, by depth) on
|
||||
the shared battery.
|
||||
**Baseline:** depth-2: 0/2 explicit.
|
||||
**Criterion:** every spawn at every depth carries explicit model +
|
||||
effort. Pre-registered caveat: if T1 eliminates depth-2 spawns entirely,
|
||||
T5 grades as root-spawn regression (hold 100%) plus doc correctness and
|
||||
is recorded inconclusive-by-zero at depth-2 — the config backstop is then
|
||||
the operative mechanism.
|
||||
|
||||
## Grading plan
|
||||
|
||||
- **Shared SDD battery** carries T1, T2, T5: `cx-sdd-small`, fix-branch
|
||||
arm (`/tmp/sp-arm-fix`), 8 reps across both container lanes. Dev
|
||||
baselines are already measured; no baseline re-runs.
|
||||
- **T4 batteries** as listed above (micro + codex ceremony + global
|
||||
regression).
|
||||
- **Pre-registration:** every battery gets a hypothesis-log entry
|
||||
(prediction, scorer, criterion) in
|
||||
`superpowers-autoresearch/logs/2026-07-30-codex-efficiency-fixes.md`
|
||||
BEFORE it runs. Standing rules carry over: append-only log, manual
|
||||
inspection of scorer matches on fix-arm runs (non-circular
|
||||
verification), no raw rollouts committed, correctness rides beside
|
||||
cost in every verdict.
|
||||
- **Attribution:** orthogonal scorers on one combined branch; unexpected
|
||||
regressions bisect by treatment commit.
|
||||
- **Budget:** shared battery ~$40, codex ceremony ~$40, global
|
||||
regression ~$40–80, micros ~$5 → phase 1 ≈ $150–200 of the ~$850
|
||||
remaining from the campaign's $1000.
|
||||
|
||||
## Process
|
||||
|
||||
- Work happens in the `codex-efficiency-fixes` worktree (branched off
|
||||
`dev`); execution via subagent-driven-development from a written plan.
|
||||
- Skill-text changes follow writing-skills methodology.
|
||||
- Scenario/rig changes (un-gating ceremony scenarios for Claude
|
||||
Code/Gemini, adversarial micro briefs) land in `superpowers-evals`
|
||||
main, as authorized.
|
||||
- PR-per-treatment against `dev`, each with its eval evidence and the
|
||||
standard identification block; merges only on Jesse's per-PR approval.
|
||||
|
||||
## Phase 2 queue (baseline-first; not in this plan's tasks)
|
||||
|
||||
Each item requires a failing baseline before any fix ships:
|
||||
1. **Dispatch routing / long-session drift** — needs a long-session
|
||||
elicitation rig (fresh sessions don't reproduce the pathology at CLI
|
||||
0.146). Drew's stack informs the treatment shape.
|
||||
2. **Verification leases / evidence receipts** — needs the
|
||||
substring-aware duplicate counter added to `score_e3.py` first
|
||||
(current baseline 1/23 exact-string pairs is too weak).
|
||||
3. **Remediation cap** — small-n baseline (2/3 reps) needs more reps.
|
||||
4. **Cross-task-race probe redesign** — `score_e5.py`'s probe is
|
||||
inconclusive-by-zero by design tradeoff; needs a stronger probe.
|
||||
5. **E5 D4 shell-command parser** — fix-review-scope classifier cannot
|
||||
parse compound commands; scorer work, not skill work.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Adopting Drew's spinout stack (#2036/#2035) or its text.
|
||||
- RoboRev, Codex token telemetry (separate codebases).
|
||||
- A `close_agent` hygiene checklist (V2 has no such tool — closed as
|
||||
do-not-ship in the campaign).
|
||||
- Claude Code/Gemini-specific efficiency treatments beyond the T4
|
||||
regression battery.
|
||||
Reference in New Issue
Block a user