rogee/superpowers
1
0
Fork 0
mirror of https://github.com/obra/superpowers.git synced 2026-07-09 19:49:05 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs(skills): name concrete macOS tools in the computer-use ladder

Browse Source
This commit is contained in:
Jesse Vincent
2026-07-04 16:03:54 -07:00
parent 8b7625df19
commit bc044af154
1 changed files with 13 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
skills/agentic-end-to-end-testing/driving-computer-use.md
View File

@@ -48,19 +48,19 @@ Accessibility automation on a real desktop is not always available cleanly.
Climb a ladder of approaches, and when a rung is blocked, record *why* before
trying the next one:
1. **Scripting API** (e.g. an OS-level accessibility scripting bridge) — the
cheap default. Blocked signature: a permission error before any command
runs (no Accessibility grant).
2. **UI-test harness** (a first-class automated-testing framework for the
platform) — the "proper" way to drive the real app end to end. Blocked
signature: the harness process itself never establishes its automation
session (e.g. an unsigned test runner killed before it attaches) — that's
the harness failing to bootstrap, not a bug in the app under test.
3. **Raw input injection** (a coordinate-based click/keystroke tool plus a
screenshot after each action) — the fallback of last resort when both of
the above are blocked. Coarser than element-indexed driving, so screenshot
after every action and confirm the click landed on the intended window
before trusting the result.
1. **Accessibility scripting** (on macOS, `osascript`/AppleScript) — the cheap
default. Blocked signature: a permission error before any command runs
(no Accessibility grant, e.g. `osascript` error `-1719`).
2. **UI-test harness** (on macOS, an XCUITest automation session) — the
"proper" way to drive the real app end to end. Blocked signature: the
harness process itself never establishes its automation session (an
unsigned test runner killed before it attaches) — that's the harness
failing to bootstrap, not a bug in the app under test.
3. **Raw input injection** (on macOS, a coordinate-clicking tool such as
`cliclick` plus `screencapture` after each action) — the fallback of last
resort when both of the above are blocked. Coarser than element-indexed
driving, so screenshot after every action and confirm the click landed on
the intended window before trusting the result.
Every rung you tried belongs in the report, including the ones that failed —
not only the one that worked. Diagnose each blocked rung enough to state the