mirror of
https://github.com/obra/superpowers.git
synced 2026-07-06 09:49:06 +08:00
Compare commits
25 Commits
codex/pack
...
tdd-writin
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0e69a4d32c | ||
|
|
8afa64b49d | ||
|
|
deb9d855cb | ||
|
|
78fb4643da | ||
|
|
6f3eca4f2e | ||
|
|
0cfc0a16b4 | ||
|
|
c809093a2a | ||
|
|
97506cefd7 | ||
|
|
4ecbbcd0b4 | ||
|
|
53106e6536 | ||
|
|
89338e5113 | ||
|
|
c842f8871a | ||
|
|
6752471ad9 | ||
|
|
371a26cf99 | ||
|
|
3bb0a3faa3 | ||
|
|
2d05b63edc | ||
|
|
f268f7c953 | ||
|
|
e1753f6e77 | ||
|
|
777cc2fae4 | ||
|
|
e7ddc25e51 | ||
|
|
711d895ce7 | ||
|
|
640ce6c0e9 | ||
|
|
879ae59c33 | ||
|
|
d376057029 | ||
|
|
add6a283b1 |
@@ -9,7 +9,7 @@
|
||||
{
|
||||
"name": "superpowers",
|
||||
"description": "Core skills library for Claude Code: TDD, debugging, collaboration patterns, and proven techniques",
|
||||
"version": "6.1.0",
|
||||
"version": "6.1.1",
|
||||
"source": "./",
|
||||
"author": {
|
||||
"name": "Jesse Vincent",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "superpowers",
|
||||
"description": "Core skills library for Claude Code: TDD, debugging, collaboration patterns, and proven techniques",
|
||||
"version": "6.1.0",
|
||||
"version": "6.1.1",
|
||||
"author": {
|
||||
"name": "Jesse Vincent",
|
||||
"email": "jesse@fsck.com"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "superpowers",
|
||||
"version": "6.1.0",
|
||||
"version": "6.1.1",
|
||||
"description": "An agentic skills framework & software development methodology that works: planning, TDD, debugging, and collaboration workflows.",
|
||||
"author": {
|
||||
"name": "Jesse Vincent",
|
||||
@@ -21,12 +21,13 @@
|
||||
"workflow"
|
||||
],
|
||||
"skills": "./skills/",
|
||||
"hooks": {},
|
||||
"interface": {
|
||||
"displayName": "Superpowers",
|
||||
"shortDescription": "Planning, TDD, debugging, and delivery workflows for coding agents",
|
||||
"longDescription": "Use Superpowers to guide agent work through brainstorming, implementation planning, test-driven development, systematic debugging, parallel execution, code review, and finish-the-branch workflows.",
|
||||
"developerName": "Jesse Vincent",
|
||||
"category": "Coding",
|
||||
"category": "Developer Tools",
|
||||
"capabilities": [
|
||||
"Interactive",
|
||||
"Read",
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"name": "superpowers",
|
||||
"displayName": "Superpowers",
|
||||
"description": "Core skills library: TDD, debugging, collaboration patterns, and proven techniques",
|
||||
"version": "6.1.0",
|
||||
"version": "6.1.1",
|
||||
"author": {
|
||||
"name": "Jesse Vincent",
|
||||
"email": "jesse@fsck.com"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "superpowers",
|
||||
"version": "6.1.0",
|
||||
"version": "6.1.1",
|
||||
"description": "An agentic skills framework and software development methodology.",
|
||||
"author": {
|
||||
"name": "Jesse Vincent",
|
||||
|
||||
@@ -1,5 +1,16 @@
|
||||
# Superpowers Release Notes
|
||||
|
||||
## v6.1.1 (2026-07-02)
|
||||
|
||||
### Codex
|
||||
|
||||
- **Codex no longer re-registers the Claude SessionStart hook.** v6.1.0 removed the Codex hook config and its manifest `hooks` pointer, meaning to stop Codex from installing a SessionStart hook — but with no `hooks` field, Codex fell back to auto-discovering `hooks/hooks.json`, the Claude Code SessionStart hook that the marketplace ships from the repo root, and re-registered it along with its install-time trust prompt. The Codex manifest now declares an explicit empty hooks object (`hooks: {}`), which Codex reads as "no hooks" instead of reaching the auto-discovery fallback. An absent field, `[]`, and an empty inline list all collapse back to the fallback, so the value has to be exactly `{}`.
|
||||
- **Removed orphaned Codex session-start dead code.** `hooks/session-start-codex` had no caller once the Codex hook config was deleted, so it and its redundant test cases are gone. The worked shell-hook example in `docs/porting-to-a-new-harness.md` moves from Codex — now native skill discovery with no session-start hook — to Cursor, a live shell-hook harness, and the stale `hooks-codex.json` pointer in `docs/windows/polyglot-hooks.md` is corrected. The Codex plugin category is also fixed to "Developer Tools".
|
||||
|
||||
### Packaging
|
||||
|
||||
- **New `package-codex-plugin.sh` for building the Codex portal package.** A maintainer script produces a deterministic Codex "portal" archive — `.zip` by default, `tar.gz` on request — that normalizes entry timestamps, preserves executable modes, verifies every packaged skill ships its OpenAI metadata, includes the app and composer icons, and refuses to run against a dirty worktree. The packaged manifest keeps the source `hooks: {}` object so a portal-installed plugin avoids the same SessionStart auto-discovery, and the script can rebuild a byte-identical archive from a saved metadata source. Covered by a new test suite.
|
||||
|
||||
## v6.1.0 (2026-06-30)
|
||||
|
||||
### Lower Per-Session Token Cost
|
||||
|
||||
@@ -90,7 +90,7 @@ every session, with no per-session opt-in by your human partner.** This is the
|
||||
one non-negotiable capability. It can take any form:
|
||||
|
||||
- a **hook/event system** that runs a shell command at session start and reads
|
||||
its stdout (Claude Code, Codex, Cursor, Copilot CLI), or
|
||||
its stdout (Claude Code, Cursor, Copilot CLI), or
|
||||
- an **in-process plugin/extension** with a session-start or message lifecycle
|
||||
callback that can mutate the message array (OpenCode, pi), or
|
||||
- an **instructions-file** convention where the harness loads a context file that
|
||||
@@ -227,18 +227,20 @@ you may **not** do is bridge a gap by editing the user's global config.
|
||||
The harness has a hook system that runs a shell command at session start and
|
||||
reads JSON from its stdout. The configured command runs `run-hook.cmd`, a
|
||||
polyglot wrapper that just locates bash and dispatches the named script; the
|
||||
script (`hooks/session-start`, or a harness-specific variant like
|
||||
`hooks/session-start-codex`) is what reads `using-superpowers/SKILL.md` and
|
||||
prints a JSON object whose **field name and nesting differ per harness**.
|
||||
script (`hooks/session-start`, or a harness-specific variant) is what reads
|
||||
`using-superpowers/SKILL.md` and prints a JSON object whose **field name and
|
||||
nesting differ per harness**.
|
||||
|
||||
- Reference: `hooks/session-start` (and `hooks/session-start-codex`),
|
||||
`hooks/run-hook.cmd`, and the per-harness hook config `hooks/hooks.json`
|
||||
(Claude Code), `hooks/hooks-codex.json` (Codex), `hooks/hooks-cursor.json`
|
||||
- Reference: `hooks/session-start`, `hooks/run-hook.cmd`, and the per-harness
|
||||
hook config `hooks/hooks.json` (Claude Code) and `hooks/hooks-cursor.json`
|
||||
(Cursor).
|
||||
- Manifests: `.codex-plugin/plugin.json`, `.cursor-plugin/plugin.json` point the
|
||||
harness at `./skills/` and the right `hooks-*.json`. (Claude Code's
|
||||
- Manifests: `.cursor-plugin/plugin.json` is the Shape A manifest example that
|
||||
points the harness at `./skills/` and the right `hooks-*.json`. Claude Code's
|
||||
`.claude-plugin/plugin.json` sets neither field — it auto-discovers `skills/`
|
||||
and `hooks/hooks.json` by convention.)
|
||||
and `hooks/hooks.json` by convention. Do **not** copy Codex's
|
||||
`.codex-plugin/plugin.json` for Shape A: it declares an empty `hooks` object
|
||||
specifically to suppress Codex's `hooks/hooks.json` auto-discovery, because
|
||||
Codex surfaces skills natively and runs no session-start hook.
|
||||
|
||||
> **A hook *system* is not a session-start *event*.** A harness can have a
|
||||
> `hooks.json` mechanism — and even contain the literal string `SessionStart` in
|
||||
@@ -287,7 +289,7 @@ part of the installed extension** — never substitute "edit the user's global
|
||||
|
||||
| If the harness… | Use shape | Copy from |
|
||||
|---|---|---|
|
||||
| runs a shell command at session start and reads its stdout | A (shell-hook) | Codex (`hooks/session-start-codex` + `hooks/hooks-codex.json` + `.codex-plugin/`) |
|
||||
| runs a shell command at session start and reads its stdout | A (shell-hook) | Cursor (`hooks/session-start` + `hooks/hooks-cursor.json` + `.cursor-plugin/`) |
|
||||
| is a JS/TS plugin host with session/message lifecycle callbacks | B (in-process) | OpenCode (`.opencode/`) — or pi (`.pi/`) if it has no native skill tool |
|
||||
| ships an extension-declared context file it always loads | C (instructions-file) | Gemini (`gemini-extension.json` + `GEMINI.md` + `references/gemini-tools.md`) |
|
||||
| has a plugin install command and a manifest `contextFileName` (or equivalent) the installer keeps | C via the plugin installer | Antigravity (`.antigravity-plugin/` — `agy plugin install` ships a generated context file; verify the installer preserves it — Part 6) |
|
||||
@@ -309,7 +311,7 @@ patterns below are summaries; the code is the spec.
|
||||
Create whatever the harness uses to recognize the plugin. Match the existing
|
||||
ones in spirit:
|
||||
|
||||
- **Shape A:** a `*-plugin/plugin.json` (see `.codex-plugin/plugin.json`) with
|
||||
- **Shape A:** a `*-plugin/plugin.json` (see `.cursor-plugin/plugin.json`) with
|
||||
`name`, `version`, `description`, author/license/keywords, `"skills":
|
||||
"./skills/"`, and `"hooks": "./hooks/hooks-<harness>.json"`. Plus the
|
||||
`hooks-<harness>.json` itself, registering a session-start hook whose command
|
||||
@@ -375,25 +377,24 @@ both double-injects). Find the
|
||||
exact field, nesting, and event-matcher values your harness expects. Then
|
||||
decide: add a fourth branch to `hooks/session-start`, or — if the harness needs
|
||||
a different bootstrap message or env contract — add a dedicated
|
||||
`hooks/session-start-<harness>` script, the way Codex did. If you add a branch
|
||||
`hooks/session-start-<harness>` script. If you add a branch
|
||||
and your harness *also* sets an env var an earlier branch keys on (some harnesses
|
||||
set `CLAUDE_PLUGIN_ROOT` too), order your branch before the one that would
|
||||
otherwise shadow it. Match the harness's
|
||||
own event-matcher strings (Claude Code uses `startup|clear|compact`, Codex
|
||||
`startup|resume|clear`, Cursor `sessionStart`); wrong matchers mean the hook
|
||||
silently never fires.
|
||||
own event-matcher strings (Claude Code uses `startup|clear|compact`, Cursor
|
||||
`sessionStart`); wrong matchers mean the hook silently never fires.
|
||||
|
||||
The **hook-config schema itself varies per harness** — don't assume the
|
||||
Claude/Codex shape is universal. Compare `hooks/hooks.json`,
|
||||
`hooks/hooks-codex.json`, and `hooks/hooks-cursor.json`: Cursor's uses
|
||||
Claude Code shape is universal. Compare `hooks/hooks.json` and
|
||||
`hooks/hooks-cursor.json`: Cursor's uses
|
||||
`"version": 1`, a lowercase `sessionStart` key, a relative
|
||||
`./hooks/run-hook.cmd` command, and omits the `matcher`/`type`/`async` fields the
|
||||
others use. Match your `hooks-<harness>.json` to whichever existing file is
|
||||
`./hooks/run-hook.cmd` command, and omits the `matcher`/`type`/`async` fields
|
||||
Claude Code uses. Match your `hooks-<harness>.json` to whichever existing file is
|
||||
closest, not to a single canonical template.
|
||||
|
||||
The hook **command string references a harness-provided plugin-root variable**,
|
||||
and its name differs per harness: `hooks.json` uses `${CLAUDE_PLUGIN_ROOT}`,
|
||||
`hooks-codex.json` uses `${PLUGIN_ROOT}`, Cursor uses a relative path. Use
|
||||
`hooks-cursor.json` uses a relative path. Use
|
||||
whatever your harness exports. (The `session-start` script re-derives the root
|
||||
itself via `dirname`, so the script body doesn't depend on this — but the
|
||||
command in the manifest does.)
|
||||
@@ -784,7 +785,7 @@ Use this as the live index; when in doubt, read the files, not this table.
|
||||
| Harness | Entry point | Bootstrap mechanism | Tool mapping | Tests | Distribution |
|
||||
|---|---|---|---|---|---|
|
||||
| Claude Code | `.claude-plugin/plugin.json` + `hooks/hooks.json` | shell hook → `hooks/session-start` (`hookSpecificOutput.additionalContext`) | native `Skill` tool; `references/claude-code-tools.md` | `tests/hooks/` | marketplace |
|
||||
| Codex | `.codex-plugin/plugin.json` + `hooks/hooks-codex.json` | shell hook → `hooks/session-start-codex` | `references/codex-tools.md` | `tests/codex-plugin-sync/`, `tests/hooks/` | fork sync (`scripts/sync-to-codex-plugin.sh`) |
|
||||
| Codex | `.codex-plugin/plugin.json` (declares empty `hooks`) | native skill discovery (no session-start hook) | `references/codex-tools.md` | `tests/codex/`, `tests/codex-plugin-sync/` | fork sync (`scripts/sync-to-codex-plugin.sh`) |
|
||||
| Cursor | `.cursor-plugin/plugin.json` + `hooks/hooks-cursor.json` | shell hook → `hooks/session-start` (`additional_context`) | `references/claude-code-tools.md` | `tests/hooks/` | hand-authored |
|
||||
| Copilot CLI | (shares Claude Code hook path; `COPILOT_CLI` env) | shell hook → `hooks/session-start` (`additionalContext`) | `references/copilot-tools.md` | `tests/hooks/` | — |
|
||||
| Gemini CLI | `gemini-extension.json` + `GEMINI.md` | instructions file `@`-includes bootstrap + mapping | `references/gemini-tools.md` | — | `gemini extensions install` |
|
||||
@@ -799,10 +800,10 @@ Use this as the live index; when in doubt, read the files, not this table.
|
||||
- **Wrong JSON field → silent failure or double injection.** Shape A only.
|
||||
Confirm the exact field/nesting; Claude Code reads two fields without dedup.
|
||||
- **Hook-config schema varies per harness.** Shape A. Cursor's `hooks-cursor.json`
|
||||
looks nothing like the Claude/Codex one (`version`, lowercase `sessionStart`,
|
||||
looks nothing like the Claude Code one (`version`, lowercase `sessionStart`,
|
||||
relative command, no `matcher`/`type`/`async`). Match the closest existing file.
|
||||
- **Plugin-root env var differs per harness.** Shape A. The hook command uses
|
||||
`${CLAUDE_PLUGIN_ROOT}` (Claude), `${PLUGIN_ROOT}` (Codex), or a relative path
|
||||
`${CLAUDE_PLUGIN_ROOT}` (Claude) or a relative path
|
||||
(Cursor). Use what your harness exports; the script re-derives the root itself.
|
||||
- **System-message injection.** Shape B injects a *user* message on purpose
|
||||
(#750, #894). Don't "fix" it to a system message.
|
||||
|
||||
@@ -140,7 +140,7 @@ Check that the script filename is **extensionless** in `hooks.json`. A command l
|
||||
|
||||
### Hook doesn't fire at all
|
||||
|
||||
Verify the `matcher` in `hooks.json` matches the event type your harness emits. Claude Code uses `startup|clear|compact`; Codex uses `startup|resume|clear`. Check `hooks-codex.json` for the Codex variant.
|
||||
Verify the `matcher` in `hooks.json` matches the event type your harness emits. Claude Code uses `startup|clear|compact`; Cursor uses `sessionStart`. Check `hooks-cursor.json` for the Cursor variant.
|
||||
|
||||
## Related Issues
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "superpowers",
|
||||
"description": "Core skills library: TDD, debugging, collaboration patterns, and proven techniques",
|
||||
"version": "6.1.0",
|
||||
"version": "6.1.1",
|
||||
"contextFileName": "GEMINI.md"
|
||||
}
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Codex SessionStart hook for superpowers plugin
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
PLUGIN_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
using_superpowers_content=$(cat "${PLUGIN_ROOT}/skills/using-superpowers/SKILL.md" 2>&1 || echo "Error reading using-superpowers skill")
|
||||
|
||||
escape_for_json() {
|
||||
local s="$1"
|
||||
s="${s//\\/\\\\}"
|
||||
s="${s//\"/\\\"}"
|
||||
s="${s//$'\n'/\\n}"
|
||||
s="${s//$'\r'/\\r}"
|
||||
s="${s//$'\t'/\\t}"
|
||||
printf '%s' "$s"
|
||||
}
|
||||
|
||||
using_superpowers_escaped=$(escape_for_json "$using_superpowers_content")
|
||||
session_context="<EXTREMELY_IMPORTANT>\nYou have superpowers.\n\n**Below is the full content of your 'superpowers:using-superpowers' skill - your introduction to using skills. For all other skills, follow the Codex skill-loading instructions in that skill:**\n\n${using_superpowers_escaped}\n</EXTREMELY_IMPORTANT>"
|
||||
|
||||
printf '{\n "hookSpecificOutput": {\n "hookEventName": "SessionStart",\n "additionalContext": "%s"\n }\n}\n' "$session_context" | cat
|
||||
|
||||
exit 0
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "superpowers",
|
||||
"version": "6.1.0",
|
||||
"version": "6.1.1",
|
||||
"description": "Superpowers skills and runtime bootstrap for coding agents",
|
||||
"type": "module",
|
||||
"main": ".opencode/plugins/superpowers.js",
|
||||
|
||||
@@ -242,10 +242,6 @@ git -C "$REPO_ROOT" archive --format=tar "$REF" -- \
|
||||
VERSION="$(jq -r '.version // empty' "$STAGE/.codex-plugin/plugin.json")"
|
||||
[[ -n "$VERSION" ]] || die "could not read version from .codex-plugin/plugin.json"
|
||||
|
||||
if jq -e 'has("hooks")' "$STAGE/.codex-plugin/plugin.json" >/dev/null; then
|
||||
die "Codex manifest must not declare hooks for the portal package"
|
||||
fi
|
||||
|
||||
if [[ -z "$OUTPUT" ]]; then
|
||||
case "$FORMAT" in
|
||||
zip)
|
||||
|
||||
@@ -203,6 +203,12 @@ Next failing test for next feature.
|
||||
| **Clear** | Name describes behavior | `test('test1')` |
|
||||
| **Shows intent** | Demonstrates desired API | Obscures what code should do |
|
||||
|
||||
When writing or changing any test, read [writing-good-tests.md](writing-good-tests.md) for the rules that keep tests honest:
|
||||
- Name the production change that would make the test fail — before writing it
|
||||
- Assert on real behavior, never on mock behavior
|
||||
- Keep test-only code in test utilities, out of production classes
|
||||
- Understand a dependency's side effects before mocking it
|
||||
|
||||
## Why Order Matters
|
||||
|
||||
**"I'll write tests after to verify it works"**
|
||||
@@ -354,13 +360,6 @@ Bug found? Write failing test reproducing it. Follow TDD cycle. Test proves fix
|
||||
|
||||
Never fix bugs without a test.
|
||||
|
||||
## Testing Anti-Patterns
|
||||
|
||||
When adding mocks or test utilities, read [testing-anti-patterns.md](testing-anti-patterns.md) to avoid common pitfalls:
|
||||
- Testing mock behavior instead of real behavior
|
||||
- Adding test-only methods to production classes
|
||||
- Mocking without understanding dependencies
|
||||
|
||||
## Final Rule
|
||||
|
||||
```
|
||||
|
||||
@@ -1,299 +0,0 @@
|
||||
# Testing Anti-Patterns
|
||||
|
||||
**Load this reference when:** writing or changing tests, adding mocks, or tempted to add test-only methods to production code.
|
||||
|
||||
## Overview
|
||||
|
||||
Tests must verify real behavior, not mock behavior. Mocks are a means to isolate, not the thing being tested.
|
||||
|
||||
**Core principle:** Test what the code does, not what the mocks do.
|
||||
|
||||
**Following strict TDD prevents these anti-patterns.**
|
||||
|
||||
## The Iron Laws
|
||||
|
||||
```
|
||||
1. NEVER test mock behavior
|
||||
2. NEVER add test-only methods to production classes
|
||||
3. NEVER mock without understanding dependencies
|
||||
```
|
||||
|
||||
## Anti-Pattern 1: Testing Mock Behavior
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: Testing that the mock exists
|
||||
test('renders sidebar', () => {
|
||||
render(<Page />);
|
||||
expect(screen.getByTestId('sidebar-mock')).toBeInTheDocument();
|
||||
});
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- You're verifying the mock works, not that the component works
|
||||
- Test passes when mock is present, fails when it's not
|
||||
- Tells you nothing about real behavior
|
||||
|
||||
**your human partner's correction:** "Are we testing the behavior of a mock?"
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Test real component or don't mock it
|
||||
test('renders sidebar', () => {
|
||||
render(<Page />); // Don't mock sidebar
|
||||
expect(screen.getByRole('navigation')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
// OR if sidebar must be mocked for isolation:
|
||||
// Don't assert on the mock - test Page's behavior with sidebar present
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE asserting on any mock element:
|
||||
Ask: "Am I testing real component behavior or just mock existence?"
|
||||
|
||||
IF testing mock existence:
|
||||
STOP - Delete the assertion or unmock the component
|
||||
|
||||
Test real behavior instead
|
||||
```
|
||||
|
||||
## Anti-Pattern 2: Test-Only Methods in Production
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: destroy() only used in tests
|
||||
class Session {
|
||||
async destroy() { // Looks like production API!
|
||||
await this._workspaceManager?.destroyWorkspace(this.id);
|
||||
// ... cleanup
|
||||
}
|
||||
}
|
||||
|
||||
// In tests
|
||||
afterEach(() => session.destroy());
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- Production class polluted with test-only code
|
||||
- Dangerous if accidentally called in production
|
||||
- Violates YAGNI and separation of concerns
|
||||
- Confuses object lifecycle with entity lifecycle
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Test utilities handle test cleanup
|
||||
// Session has no destroy() - it's stateless in production
|
||||
|
||||
// In test-utils/
|
||||
export async function cleanupSession(session: Session) {
|
||||
const workspace = session.getWorkspaceInfo();
|
||||
if (workspace) {
|
||||
await workspaceManager.destroyWorkspace(workspace.id);
|
||||
}
|
||||
}
|
||||
|
||||
// In tests
|
||||
afterEach(() => cleanupSession(session));
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE adding any method to production class:
|
||||
Ask: "Is this only used by tests?"
|
||||
|
||||
IF yes:
|
||||
STOP - Don't add it
|
||||
Put it in test utilities instead
|
||||
|
||||
Ask: "Does this class own this resource's lifecycle?"
|
||||
|
||||
IF no:
|
||||
STOP - Wrong class for this method
|
||||
```
|
||||
|
||||
## Anti-Pattern 3: Mocking Without Understanding
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: Mock breaks test logic
|
||||
test('detects duplicate server', () => {
|
||||
// Mock prevents config write that test depends on!
|
||||
vi.mock('ToolCatalog', () => ({
|
||||
discoverAndCacheTools: vi.fn().mockResolvedValue(undefined)
|
||||
}));
|
||||
|
||||
await addServer(config);
|
||||
await addServer(config); // Should throw - but won't!
|
||||
});
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- Mocked method had side effect test depended on (writing config)
|
||||
- Over-mocking to "be safe" breaks actual behavior
|
||||
- Test passes for wrong reason or fails mysteriously
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Mock at correct level
|
||||
test('detects duplicate server', () => {
|
||||
// Mock the slow part, preserve behavior test needs
|
||||
vi.mock('MCPServerManager'); // Just mock slow server startup
|
||||
|
||||
await addServer(config); // Config written
|
||||
await addServer(config); // Duplicate detected ✓
|
||||
});
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE mocking any method:
|
||||
STOP - Don't mock yet
|
||||
|
||||
1. Ask: "What side effects does the real method have?"
|
||||
2. Ask: "Does this test depend on any of those side effects?"
|
||||
3. Ask: "Do I fully understand what this test needs?"
|
||||
|
||||
IF depends on side effects:
|
||||
Mock at lower level (the actual slow/external operation)
|
||||
OR use test doubles that preserve necessary behavior
|
||||
NOT the high-level method the test depends on
|
||||
|
||||
IF unsure what test depends on:
|
||||
Run test with real implementation FIRST
|
||||
Observe what actually needs to happen
|
||||
THEN add minimal mocking at the right level
|
||||
|
||||
Red flags:
|
||||
- "I'll mock this to be safe"
|
||||
- "This might be slow, better mock it"
|
||||
- Mocking without understanding the dependency chain
|
||||
```
|
||||
|
||||
## Anti-Pattern 4: Incomplete Mocks
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: Partial mock - only fields you think you need
|
||||
const mockResponse = {
|
||||
status: 'success',
|
||||
data: { userId: '123', name: 'Alice' }
|
||||
// Missing: metadata that downstream code uses
|
||||
};
|
||||
|
||||
// Later: breaks when code accesses response.metadata.requestId
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- **Partial mocks hide structural assumptions** - You only mocked fields you know about
|
||||
- **Downstream code may depend on fields you didn't include** - Silent failures
|
||||
- **Tests pass but integration fails** - Mock incomplete, real API complete
|
||||
- **False confidence** - Test proves nothing about real behavior
|
||||
|
||||
**The Iron Rule:** Mock the COMPLETE data structure as it exists in reality, not just fields your immediate test uses.
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Mirror real API completeness
|
||||
const mockResponse = {
|
||||
status: 'success',
|
||||
data: { userId: '123', name: 'Alice' },
|
||||
metadata: { requestId: 'req-789', timestamp: 1234567890 }
|
||||
// All fields real API returns
|
||||
};
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE creating mock responses:
|
||||
Check: "What fields does the real API response contain?"
|
||||
|
||||
Actions:
|
||||
1. Examine actual API response from docs/examples
|
||||
2. Include ALL fields system might consume downstream
|
||||
3. Verify mock matches real response schema completely
|
||||
|
||||
Critical:
|
||||
If you're creating a mock, you must understand the ENTIRE structure
|
||||
Partial mocks fail silently when code depends on omitted fields
|
||||
|
||||
If uncertain: Include all documented fields
|
||||
```
|
||||
|
||||
## Anti-Pattern 5: Integration Tests as Afterthought
|
||||
|
||||
**The violation:**
|
||||
```
|
||||
✅ Implementation complete
|
||||
❌ No tests written
|
||||
"Ready for testing"
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- Testing is part of implementation, not optional follow-up
|
||||
- TDD would have caught this
|
||||
- Can't claim complete without tests
|
||||
|
||||
**The fix:**
|
||||
```
|
||||
TDD cycle:
|
||||
1. Write failing test
|
||||
2. Implement to pass
|
||||
3. Refactor
|
||||
4. THEN claim complete
|
||||
```
|
||||
|
||||
## When Mocks Become Too Complex
|
||||
|
||||
**Warning signs:**
|
||||
- Mock setup longer than test logic
|
||||
- Mocking everything to make test pass
|
||||
- Mocks missing methods real components have
|
||||
- Test breaks when mock changes
|
||||
|
||||
**your human partner's question:** "Do we need to be using a mock here?"
|
||||
|
||||
**Consider:** Integration tests with real components often simpler than complex mocks
|
||||
|
||||
## TDD Prevents These Anti-Patterns
|
||||
|
||||
**Why TDD helps:**
|
||||
1. **Write test first** → Forces you to think about what you're actually testing
|
||||
2. **Watch it fail** → Confirms test tests real behavior, not mocks
|
||||
3. **Minimal implementation** → No test-only methods creep in
|
||||
4. **Real dependencies** → You see what the test actually needs before mocking
|
||||
|
||||
**If you're testing mock behavior, you violated TDD** - you added mocks without watching test fail against real code first.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Anti-Pattern | Fix |
|
||||
|--------------|-----|
|
||||
| Assert on mock elements | Test real component or unmock it |
|
||||
| Test-only methods in production | Move to test utilities |
|
||||
| Mock without understanding | Understand dependencies first, mock minimally |
|
||||
| Incomplete mocks | Mirror real API completely |
|
||||
| Tests as afterthought | TDD - tests first |
|
||||
| Over-complex mocks | Consider integration tests |
|
||||
|
||||
## Red Flags
|
||||
|
||||
- Assertion checks for `*-mock` test IDs
|
||||
- Methods only called in test files
|
||||
- Mock setup is >50% of test
|
||||
- Test fails when you remove mock
|
||||
- Can't explain why mock is needed
|
||||
- Mocking "just to be safe"
|
||||
|
||||
## The Bottom Line
|
||||
|
||||
**Mocks are tools to isolate, not things to test.**
|
||||
|
||||
If TDD reveals you're testing mock behavior, you've gone wrong.
|
||||
|
||||
Fix: Test real behavior or question why you're mocking at all.
|
||||
198
skills/test-driven-development/writing-good-tests.md
Normal file
198
skills/test-driven-development/writing-good-tests.md
Normal file
@@ -0,0 +1,198 @@
|
||||
# Writing Good Tests
|
||||
|
||||
**Load this reference when:** writing or changing tests, adding mocks, or
|
||||
adding cleanup/helper methods for tests.
|
||||
|
||||
## Overview
|
||||
|
||||
A test exists to catch a specific break. Two principles govern everything
|
||||
here:
|
||||
|
||||
```
|
||||
1. Every test names the break it catches
|
||||
2. Every test exercises the real thing
|
||||
```
|
||||
|
||||
Strict TDD produces both naturally: a test written first and watched
|
||||
failing against real code has already proven it can fail, and only earns
|
||||
a mock when the real dependency proves slow or external.
|
||||
|
||||
## Principle 1: Name the Break
|
||||
|
||||
Before writing the test body, answer: **what production change should
|
||||
make this test fail — and is that change a bug or a decision?** A test
|
||||
earns its place by catching a wrong branch, missing side effect, wrong
|
||||
argument, boundary case, or broken contract.
|
||||
|
||||
**Derive expectations independently.** Use literals and hand-checked
|
||||
fixtures; table-driven tests with literal `want` values are the preferred
|
||||
shape. An expectation computed by the code under test — or its helpers —
|
||||
passes no matter what that code does:
|
||||
|
||||
```typescript
|
||||
// ❌ Mirror assertion: the same builder computes both sides — always true
|
||||
const expected = buildSearchQuery({ tag: 'urgent' });
|
||||
expect(buildSearchQuery({ tag: 'urgent' })).toBe(expected);
|
||||
|
||||
// ✅ Hand-derived literal
|
||||
expect(buildSearchQuery({ tag: 'urgent' })).toBe('tag:"urgent"');
|
||||
```
|
||||
|
||||
**No change detectors.** If only intentional decisions can fail a test —
|
||||
a constant's value, exact message wording, private structure — it fires
|
||||
on redesign and sleeps through bugs. Test the behavior that depends on
|
||||
the decision: not `expect(MAX_RETRIES).toBe(5)` but "a failing call is
|
||||
retried 5 times and the 6th attempt never happens."
|
||||
|
||||
**Behavior, not text.** Asserting that a script, skill, or config
|
||||
contains an exact line proves only that the source is the source. Run
|
||||
scripts against controlled inputs and assert outputs, side effects, or
|
||||
exit codes. Documents that instruct agents are tested by the consuming
|
||||
agent's behavior (superpowers:writing-skills); prose for humans earns no
|
||||
test at all.
|
||||
|
||||
**Your code, not the framework.** Test the contract your code makes at
|
||||
its boundaries — the route you register, the query you emit, the payload
|
||||
you produce. Upstream mechanics are their maintainers' tests to write
|
||||
(the classic: asserting your router invokes a registered handler — that
|
||||
is the framework's test, not yours). When upstream behavior genuinely
|
||||
surprised you, write one narrow characterization test naming the
|
||||
assumption. The same boundary applies inside your code: constructors,
|
||||
getters, constants, and trivial forwarding earn tests only when they
|
||||
validate, normalize, default, derive, enforce, or cause side effects —
|
||||
otherwise assert the first consumer-visible result that depends on them.
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE writing the test body:
|
||||
Name the production change that would make this test fail.
|
||||
|
||||
Cannot name one → redesign around an observable behavior
|
||||
"The source text changed" → run the artifact and assert its effects
|
||||
Only intentional decisions → change detector; test the behavior
|
||||
that depends on the decision
|
||||
|
||||
Confirm the expected value is derived without the code under test.
|
||||
IF it reuses the code's logic or helpers:
|
||||
Replace it with a literal or hand-checked fixture
|
||||
```
|
||||
|
||||
## Principle 2: Exercise the Real Thing
|
||||
|
||||
**The mock earns no assertions.** A mock assertion passes when the mock
|
||||
is present and fails when it is absent — it says nothing about the
|
||||
component. Assert the real component's behavior; if the mock is what you
|
||||
are checking, unmock it or delete the assertion.
|
||||
|
||||
```typescript
|
||||
// ✅ Real behavior
|
||||
expect(screen.getByRole('navigation')).toBeInTheDocument();
|
||||
|
||||
// ❌ Mock existence
|
||||
expect(screen.getByTestId('sidebar-mock')).toBeInTheDocument();
|
||||
```
|
||||
|
||||
**your human partner's correction:** "Are we testing the behavior of a
|
||||
mock?"
|
||||
|
||||
**Mock at the right level.** Learn every side effect of the real method
|
||||
before replacing it; mock the slow or external operation and keep what
|
||||
the test depends on real. When unsure, run the test against the real
|
||||
implementation first and observe what actually needs to happen.
|
||||
|
||||
```typescript
|
||||
// ❌ The mock swallows the config write that duplicate detection reads
|
||||
vi.mock('ToolCatalog', () => ({
|
||||
discoverAndCacheTools: vi.fn().mockResolvedValue(undefined)
|
||||
}));
|
||||
|
||||
// ✅ Mock only the slow server startup; the config write stays real
|
||||
vi.mock('MCPServerManager');
|
||||
```
|
||||
|
||||
**Make doubles specific.** When arguments, call counts, or ordering are
|
||||
part of the contract, assert them — a fake that accepts anything verifies
|
||||
nothing. Give each branch (success, error, malformed) its own fixture or
|
||||
spy, so the wrong branch cannot satisfy the expectation.
|
||||
|
||||
**Mirror real data completely.** Mock the complete structure as it exists
|
||||
in reality — all documented fields — not just the ones your test reads.
|
||||
Partial mocks fail silently when downstream code reads an omitted field:
|
||||
the test passes while integration breaks.
|
||||
|
||||
**Production classes carry production methods only.** Cleanup that only
|
||||
tests need lives in test utilities, never as a `destroy()` on the
|
||||
production class. Ask: is this method called only from tests? Does this
|
||||
class own this resource's lifecycle? Wrong answers → test utility.
|
||||
|
||||
**Prefer real components over complex mocks.** When mock setup outgrows
|
||||
the test logic, mocks miss methods the real components have, or tests
|
||||
break when the mock changes, switch to an integration test with real
|
||||
components. **your human partner's question:** "Do we need to be using a
|
||||
mock here?"
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE adding a mock or test helper:
|
||||
List the real method's side effects; keep the ones the test
|
||||
depends on real — mock the slow/external level below them.
|
||||
|
||||
Mock responses mirror the complete real structure.
|
||||
|
||||
A method only tests call lives in test utilities, not production.
|
||||
|
||||
About to assert on the mock itself?
|
||||
Unmock it or delete the assertion.
|
||||
```
|
||||
|
||||
## Tests Ship With the Implementation
|
||||
|
||||
The TDD cycle — failing test, minimal implementation, refactor — is what
|
||||
"complete" means. Ship the tests the behavior needs and only those:
|
||||
trivial code and human prose earn none, and a test written to satisfy
|
||||
process costs maintenance forever.
|
||||
|
||||
## The Mutation Check
|
||||
|
||||
Before finishing, mentally mutate the production code; at least one test
|
||||
should fail for each realistic mutation:
|
||||
|
||||
- Wrong constant or argument
|
||||
- Wrong branch handler
|
||||
- Missing state change or side effect
|
||||
- Empty or default return
|
||||
- Missing validation for zero, empty, nil, unauthorized, or malformed input
|
||||
|
||||
A mutation nothing catches marks the behavior as unprotected — or the
|
||||
test as tautological.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| When you... | Do |
|
||||
|-------------|-----|
|
||||
| Write any test | Name the break it catches — a bug, not a decision |
|
||||
| Build an expected value | Derive it by hand; never with the code under test |
|
||||
| Test a script or document | Run it / pressure-test its consumer; never grep its text |
|
||||
| Reach for a dependency test | Test your boundary contract, not their documented mechanics |
|
||||
| Want to assert on a mocked element | Test the real component, or unmock it |
|
||||
| Are about to mock a method | Learn its side effects; mock the slow/external level |
|
||||
| Build a mock response | Mirror the real structure completely |
|
||||
| Need cleanup only tests use | Put it in test utilities |
|
||||
| Watch mock setup balloon | Switch to an integration test with real components |
|
||||
| Finish a test file | Run the mutation check |
|
||||
|
||||
## Warning Signs
|
||||
|
||||
- Setup and assertion share the same object, guaranteeing equality
|
||||
- The test can fail only through a panic, crash, or missing selector
|
||||
- The test fails on every intentional change, never on accidental breakage
|
||||
- Expected values are hidden behind loops, builders, or helpers
|
||||
- The test greps source text, or asserts a removed symbol stays removed
|
||||
- The test would still matter if only the framework remained
|
||||
- The test exists for coverage, checking no side effect or outcome
|
||||
- An assertion checks a `*-mock` test ID, or fails if you remove the mock
|
||||
- A method is called only from test files
|
||||
- Mock setup is more than half the test, or you can't explain why the mock is needed
|
||||
- Mocking "just to be safe"
|
||||
@@ -51,10 +51,25 @@ if not plugin_manifest.exists():
|
||||
|
||||
manifest = json.loads(plugin_manifest.read_text(encoding="utf-8"))
|
||||
assert_equal(manifest.get("name"), plugin.get("name"), "plugin manifest name")
|
||||
|
||||
# Codex auto-discovers a plugin's hooks/hooks.json whenever the Codex manifest
|
||||
# has no `hooks` field: load_plugin_hooks falls back to a hardcoded
|
||||
# DEFAULT_HOOKS_CONFIG_FILE = "hooks/hooks.json" and registers it. That file is
|
||||
# the Claude Code SessionStart hook, it is tracked in this repo, and this
|
||||
# marketplace installs the whole repo root (source url "./"), so on Codex the
|
||||
# fallback re-registers the SessionStart hook and its install-time trust prompt.
|
||||
# Declaring an empty inline hooks object ({}) parses as an empty inline hook set
|
||||
# and suppresses the auto-discovery. An absent field, an empty array ([]), and
|
||||
# an empty inline list all collapse back to the fallback, so the value must be
|
||||
# exactly an empty object.
|
||||
hooks_config = repo_root / "hooks" / "hooks.json"
|
||||
if not hooks_config.exists():
|
||||
raise AssertionError("hooks/hooks.json must exist (Claude Code SessionStart hook)")
|
||||
|
||||
assert_equal(
|
||||
manifest.get("hooks"),
|
||||
None,
|
||||
"Codex manifest ships no hooks",
|
||||
{},
|
||||
"Codex manifest must declare empty hooks {} to suppress hooks/hooks.json auto-discovery",
|
||||
)
|
||||
|
||||
print("Codex marketplace manifest looks good")
|
||||
|
||||
@@ -140,6 +140,9 @@ extracted="$TEST_ROOT/extracted"
|
||||
tar_extracted="$TEST_ROOT/tar-extracted"
|
||||
write_metadata_fixture "$metadata_source"
|
||||
|
||||
source_hooks="$(python3 -c 'import json; print(json.load(open("'"$REPO_ROOT"'/.codex-plugin/plugin.json")).get("hooks"))')"
|
||||
assert_equals "$source_hooks" "{}" "source Codex manifest suppresses local hook auto-discovery"
|
||||
|
||||
if output="$("$SCRIPT_UNDER_TEST" --allow-dirty --metadata-source "$metadata_source" --output "$archive" 2>&1)"; then
|
||||
pass "package script exits successfully"
|
||||
else
|
||||
@@ -170,7 +173,7 @@ assert_contains "$archive_paths" "assets/superpowers-small.svg" "archive include
|
||||
|
||||
manifest_summary="$(read_archive_file "$archive" .codex-plugin/plugin.json | python3 -c 'import json,sys; data=json.load(sys.stdin); print("\t".join([data["name"], data["version"], data["skills"], str(data.get("hooks"))]))')"
|
||||
expected_version="$(python3 -c 'import json; print(json.load(open("'"$REPO_ROOT"'/.codex-plugin/plugin.json"))["version"])')"
|
||||
assert_equals "$manifest_summary" "superpowers $expected_version ./skills/ None" "archive manifest is current and hook-free"
|
||||
assert_equals "$manifest_summary" "superpowers $expected_version ./skills/ $source_hooks" "archive manifest preserves source hooks"
|
||||
|
||||
skill_count="$(find "$extracted/skills" -mindepth 1 -maxdepth 1 -type d | wc -l | tr -d ' ')"
|
||||
metadata_count="$(find "$extracted/skills" -path '*/agents/openai.yaml' -type f | wc -l | tr -d ' ')"
|
||||
|
||||
@@ -4,7 +4,6 @@ set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
HOOK_UNDER_TEST="$REPO_ROOT/hooks/session-start"
|
||||
CODEX_HOOK_UNDER_TEST="$REPO_ROOT/hooks/session-start-codex"
|
||||
WRAPPER_UNDER_TEST="$REPO_ROOT/hooks/run-hook.cmd"
|
||||
|
||||
FAILURES=0
|
||||
@@ -154,35 +153,15 @@ assert_command_output \
|
||||
CLAUDE_PLUGIN_ROOT="$REPO_ROOT" \
|
||||
bash "$HOOK_UNDER_TEST"
|
||||
|
||||
codex_home="$(make_home codex-plugin-hooks)"
|
||||
codex_data="$TEST_ROOT/codex-plugin-hooks/data"
|
||||
mkdir -p "$codex_data"
|
||||
wrapper_home="$(make_home run-hook-wrapper)"
|
||||
assert_command_output \
|
||||
"Codex plugin hooks use dedicated script and emit nested SessionStart additionalContext" \
|
||||
"run-hook.cmd wrapper dispatches to the named session-start script" \
|
||||
"nested" \
|
||||
"" \
|
||||
"" \
|
||||
"$codex_home" \
|
||||
PLUGIN_DATA="$codex_data" \
|
||||
CLAUDE_PLUGIN_DATA="$codex_data" \
|
||||
PLUGIN_ROOT="$REPO_ROOT" \
|
||||
"$wrapper_home" \
|
||||
CLAUDE_PLUGIN_ROOT="$REPO_ROOT" \
|
||||
bash "$CODEX_HOOK_UNDER_TEST"
|
||||
|
||||
codex_wrapper_home="$(make_home codex-wrapper)"
|
||||
codex_wrapper_data="$TEST_ROOT/codex-wrapper/data"
|
||||
mkdir -p "$codex_wrapper_data"
|
||||
assert_command_output \
|
||||
"Codex wrapper path dispatches to dedicated script" \
|
||||
"nested" \
|
||||
"" \
|
||||
"" \
|
||||
"$codex_wrapper_home" \
|
||||
PLUGIN_DATA="$codex_wrapper_data" \
|
||||
CLAUDE_PLUGIN_DATA="$codex_wrapper_data" \
|
||||
PLUGIN_ROOT="$REPO_ROOT" \
|
||||
CLAUDE_PLUGIN_ROOT="$REPO_ROOT" \
|
||||
bash "$WRAPPER_UNDER_TEST" session-start-codex
|
||||
bash "$WRAPPER_UNDER_TEST" session-start
|
||||
|
||||
cursor_home="$(make_home cursor)"
|
||||
assert_command_output \
|
||||
@@ -217,21 +196,6 @@ assert_command_output \
|
||||
CLAUDE_PLUGIN_ROOT="$REPO_ROOT" \
|
||||
bash "$HOOK_UNDER_TEST"
|
||||
|
||||
codex_legacy_home="$(make_home codex-legacy-warning-removed)"
|
||||
codex_legacy_data="$TEST_ROOT/codex-legacy-warning-removed/data"
|
||||
mkdir -p "$codex_legacy_home/.config/superpowers/skills" "$codex_legacy_data"
|
||||
assert_command_output \
|
||||
"Codex SessionStart omits obsolete legacy custom-skill warning" \
|
||||
"nested" \
|
||||
"" \
|
||||
"Superpowers now uses"$'\037'"~/.config/superpowers/skills"$'\037'"~/.claude/skills"$'\037'"legacy" \
|
||||
"$codex_legacy_home" \
|
||||
PLUGIN_DATA="$codex_legacy_data" \
|
||||
CLAUDE_PLUGIN_DATA="$codex_legacy_data" \
|
||||
PLUGIN_ROOT="$REPO_ROOT" \
|
||||
CLAUDE_PLUGIN_ROOT="$REPO_ROOT" \
|
||||
bash "$CODEX_HOOK_UNDER_TEST"
|
||||
|
||||
if [[ "$FAILURES" -gt 0 ]]; then
|
||||
echo "STATUS: FAILED ($FAILURES failure(s))"
|
||||
exit 1
|
||||
|
||||
Reference in New Issue
Block a user