rogee/superpowers
1
0
Fork 0
mirror of https://github.com/obra/superpowers.git synced 2026-07-10 03:59:04 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs(skills): name concrete macOS tools in the computer-use ladder

Browse Source
This commit is contained in:
Jesse Vincent
2026-07-04 16:03:54 -07:00
parent 8b7625df19
commit bc044af154
1 changed files with 13 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
skills/agentic-end-to-end-testing/driving-computer-use.md
View File

@@ -48,19 +48,19 @@ Accessibility automation on a real desktop is not always available cleanly.
Climb a ladder of approaches, and when a rung is blocked, record *why* before Climb a ladder of approaches, and when a rung is blocked, record *why* before
trying the next one: trying the next one:
1. **Scripting API** (e.g. an OS-level accessibility scripting bridge) — the 1. **Accessibility scripting** (on macOS, `osascript`/AppleScript) — the cheap
cheap default. Blocked signature: a permission error before any command default. Blocked signature: a permission error before any command runs
runs (no Accessibility grant). (no Accessibility grant, e.g. `osascript` error `-1719`).
2. **UI-test harness** (a first-class automated-testing framework for the 2. **UI-test harness** (on macOS, an XCUITest automation session) — the
platform) — the "proper" way to drive the real app end to end. Blocked "proper" way to drive the real app end to end. Blocked signature: the
signature: the harness process itself never establishes its automation harness process itself never establishes its automation session (an
session (e.g. an unsigned test runner killed before it attaches) — that's unsigned test runner killed before it attaches) — that's the harness
the harness failing to bootstrap, not a bug in the app under test. failing to bootstrap, not a bug in the app under test.
3. **Raw input injection** (a coordinate-based click/keystroke tool plus a 3. **Raw input injection** (on macOS, a coordinate-clicking tool such as
screenshot after each action) — the fallback of last resort when both of `cliclick` plus `screencapture` after each action) — the fallback of last
the above are blocked. Coarser than element-indexed driving, so screenshot resort when both of the above are blocked. Coarser than element-indexed
after every action and confirm the click landed on the intended window driving, so screenshot after every action and confirm the click landed on
before trusting the result. the intended window before trusting the result.
Every rung you tried belongs in the report, including the ones that failed — Every rung you tried belongs in the report, including the ones that failed —
not only the one that worked. Diagnose each blocked rung enough to state the not only the one that worked. Diagnose each blocked rung enough to state the