lume-ctrl/docs/m0-spike.md

M0 CDP network spike

Run against the Clark binary declared by clark-browser 0.2.1:

cd ../clark-browser
python3 -m clarkbrowser fetch
cd ../lume-ctrl
M0_CDP_PORT=40746 M0_HTTP_PORT=40745 \
  CLARK_BINARY_PATH="$HOME/.clarkbrowser/chromium-148.0.7778.96/chrome" npm run spike
npm test

The two explicit loopback ports make the recorded run directly comparable; omit them to select free ports automatically, or change them if already occupied. One command performs three independent startup/full-traffic samples by default. Override M0_STARTUP_SAMPLES only when gathering a larger baseline; values below two are rejected.

The spike starts only loopback listeners and uses generated, one-day self-signed TLS material in a temporary directory. Chromium accepts that local certificate with --ignore-certificate-errors; no proxy, CA installation, traffic interception, VNC, or external fixture is involved. artifacts/m0-events.sample.jsonl is the sanitized event output and artifacts/m0-report.json is the run result/resource snapshot.

Connection and schema

Clark is launched directly with --remote-debugging-address=127.0.0.1. The runner waits on /json/version, discovers the startup page through /json/list, connects to its webSocketDebuggerUrl, and navigates to the already-listening loopback fixture before triggering test traffic. This avoids clarkserve's current 0.0.0.0 default; a protected gateway and short-lived credentials remain required outside a local Spike.

Every JSONL line has schema_version, session_id, target_id, and kind. HTTP records use request_id; WebSocket records use the CDP requestId as connection_id. Kind-specific fields are nested under request, response, headers, body, or error. Query strings are removed, sensitive headers are replaced by [REDACTED], and WebSocket payloads are never persisted. In schema 1.0, websocket_frame.payload_bytes is the payload's original byte count (base64-decoded for opcode 2) and drives the frame-size limit. Binary frames additionally expose encoded_payload_bytes for the UTF-8 byte length of CDP's base64 payloadData text.

Response bodies are safe by default and never contain content. Their stable metadata is {state,source,encoding,storage,reason,bytes}: source is network, memory_cache, disk_cache, prefetch_cache, or service_worker; encoding is utf8, base64, or unknown; and storage remains omitted. Oversize and CDP-unavailable bodies keep size_limit and unavailable states. The local fixture fetches a cacheable response twice and an application/octet-stream response to assert cached and binary semantics.

Session filtering happens before pagination and export serialization. The validation closes two distinct CDP targets bound to two generated session IDs, then asserts that queries, exports, and disconnect records contain only the requested session/target.

Coverage and limits

Case	Clark 0.2.1 / Chromium 148.0.7778.96	Evidence or limit
HTTP request → response → finished	Pass	Same request_id asserted for /ok, /large, and /secure.
Failed HTTP request	Pass	Unused loopback port produces loadingFailed; error text retained.
HTTPS without MITM	Pass	Direct browser connection to local TLS fixture; no network intermediary.
WebSocket lifecycle and frames	Pass	Created, handshake, sent, received, and closed share one connection_id; payload omitted.
WebSocket pressure limits	Pass	Defaults: 1,024 bytes/frame, 16 events/second, and 16 emitted events/connection. Drops are aggregated into one payload-free record with frame_size_limit, rate_limit, or event_limit counts.
getResponseBody success	Pass	Called after loadingFinished; fetched bytes are recorded but content is omitted by default.
Cached and binary body	Pass	Repeated force-cache response records its cache source; octet-stream records encoding=base64; neither persists content.
Sensitive response body	Pass	Random secret fixture is fetched; serialized JSONL is asserted not to contain its value.
Body size limit	Pass	2 KiB fixture is reported as size_limit and not persisted with the 1 KiB default.
Body unavailable	Pass	Calling on the failed request returns the original CDP error as structured cdp_error.
Target close	Pass	Target connection closes; browser process remains alive.
CDP disconnect/reconnect	Pass	Browser-side target close triggers the real socket close callback; the browser stays alive and a new target accepts a verified CDP command.
Cookie / Authorization / Set-Cookie	Pass	Network.*ExtraInfo values are redacted before JSONL serialization.
Multi-target/session isolation	Pass	Two target/session pairs keep query, export, and real socket-close records isolated.
Startup Page.navigate	Sampled	Three successful startup/full-traffic samples are required; exact Render process gone. retries are recorded and must stay at or below a 0.34 failure rate. Other discovery, WebSocket, timeout, and post-startup errors fail immediately.

Each full sample takes five ps samples 200 ms apart from the browser process tree and reports min/max/mean/p95. resource_samples retains the summary for every startup/full-traffic sample; resources remains the latest summary for existing consumers. The local guardrails are RSS max 1,500,000 KiB and at most 20 processes; exceeding either fails the run. These are single-instance development thresholds, not production capacity claims. Tune with M0_RESOURCE_SAMPLES, M0_RESOURCE_SAMPLE_INTERVAL_MS, M0_MAX_RSS_KIB, and M0_MAX_PROCESS_COUNT under the intended container limits.

The runner restarts Clark at most three times per startup sample only from the local navigation/readiness block, and only after CDP reports the exact detached reason Render process gone.. Every failed attempt is retained with sample, attempt, error, and detached reason. A timeout without that reason and every discovery, connection, or post-startup error fail immediately with the original message. The default renderer failure-rate guardrail is 0.34 and can be tightened with M0_MAX_RENDERER_FAILURE_RATE after more samples.