H-391: bound network collection follow-ups (#3)

This commit was merged in pull request #3.
This commit is contained in:
2026-08-21 13:44:16 +08:00
parent b4fb8ddbde
commit d4b9c66629
5 changed files with 471 additions and 103 deletions
+12 -8
View File
@@ -11,7 +11,7 @@ M0_CDP_PORT=40746 M0_HTTP_PORT=40745 \
npm test
```
The two explicit loopback ports make the recorded run directly comparable; omit them to select free ports automatically, or change them if already occupied.
The two explicit loopback ports make the recorded run directly comparable; omit them to select free ports automatically, or change them if already occupied. One command performs three independent startup/full-traffic samples by default. Override `M0_STARTUP_SAMPLES` only when gathering a larger baseline; values below two are rejected.
The spike starts only loopback listeners and uses generated, one-day self-signed TLS material in a temporary directory. Chromium accepts that local certificate with `--ignore-certificate-errors`; no proxy, CA installation, traffic interception, VNC, or external fixture is involved. `artifacts/m0-events.sample.jsonl` is the sanitized event output and `artifacts/m0-report.json` is the run result/resource snapshot.
@@ -19,7 +19,11 @@ The spike starts only loopback listeners and uses generated, one-day self-signed
Clark is launched directly with `--remote-debugging-address=127.0.0.1`. The runner waits on `/json/version`, discovers the startup page through `/json/list`, connects to its `webSocketDebuggerUrl`, and navigates to the already-listening loopback fixture before triggering test traffic. This avoids `clarkserve`'s current `0.0.0.0` default; a protected gateway and short-lived credentials remain required outside a local Spike.
Every JSONL line has `schema_version`, `session_id`, `target_id`, and `kind`. HTTP records use `request_id`; WebSocket records use the CDP `requestId` as `connection_id`. Kind-specific fields are nested under `request`, `response`, `headers`, `body`, or `error`. Query strings are removed, sensitive headers are replaced by `[REDACTED]`, and WebSocket payloads are never persisted. Response bodies are safe by default: a fetched body is recorded as `{state:"fetched",storage:"omitted",reason:"default_body_policy",bytes:N}` with no content; oversize and unavailable bodies retain distinct structured states.
Every JSONL line has `schema_version`, `session_id`, `target_id`, and `kind`. HTTP records use `request_id`; WebSocket records use the CDP `requestId` as `connection_id`. Kind-specific fields are nested under `request`, `response`, `headers`, `body`, or `error`. Query strings are removed, sensitive headers are replaced by `[REDACTED]`, and WebSocket payloads are never persisted.
Response bodies are safe by default and never contain content. Their stable metadata is `{state,source,encoding,storage,reason,bytes}`: `source` is `network`, `memory_cache`, `disk_cache`, `prefetch_cache`, or `service_worker`; `encoding` is `utf8`, `base64`, or `unknown`; and `storage` remains `omitted`. Oversize and CDP-unavailable bodies keep `size_limit` and `unavailable` states. The local fixture fetches a cacheable response twice and an `application/octet-stream` response to assert cached and binary semantics.
Session filtering happens before pagination and export serialization. The validation closes two distinct CDP targets bound to two generated session IDs, then asserts that queries, exports, and disconnect records contain only the requested session/target.
### Coverage and limits
@@ -29,18 +33,18 @@ Every JSONL line has `schema_version`, `session_id`, `target_id`, and `kind`. HT
| Failed HTTP request | Pass | Unused loopback port produces `loadingFailed`; error text retained. |
| HTTPS without MITM | Pass | Direct browser connection to local TLS fixture; no network intermediary. |
| WebSocket lifecycle and frames | Pass | Created, handshake, sent, received, and closed share one `connection_id`; payload omitted. |
| WebSocket pressure limits | Pass | Defaults: 1,024 bytes/frame, 16 events/second, and 16 emitted events/connection. Drops are aggregated into one payload-free record with `frame_size_limit`, `rate_limit`, or `event_limit` counts. |
| `getResponseBody` success | Pass | Called after `loadingFinished`; fetched bytes are recorded but content is omitted by default. |
| Cached and binary body | Pass | Repeated `force-cache` response records its cache source; octet-stream records `encoding=base64`; neither persists content. |
| Sensitive response body | Pass | Random secret fixture is fetched; serialized JSONL is asserted not to contain its value. |
| Body size limit | Pass | 2 KiB fixture is reported as `size_limit` and not persisted with the 1 KiB default. |
| Body unavailable | Pass | Calling on the failed request returns the original CDP error as structured `cdp_error`. |
| Target close | Pass | Target connection closes; browser process remains alive. |
| CDP disconnect/reconnect | Pass | Browser-side target close triggers the real socket `close` callback; the browser stays alive and a new target accepts a verified CDP command. |
| Cookie / Authorization / Set-Cookie | Pass | `Network.*ExtraInfo` values are redacted before JSONL serialization. |
| Cache-specific body availability | Not covered | Cache behavior varies by response/cache mode; retain optional-body semantics in M1. |
| Sustained high-rate WebSocket | Not covered | M0 stores metadata only; add frame/event caps before M1 load testing. |
| Multi-target/session isolation | Not covered | Explicitly M1 scope. |
| Startup `Page.navigate` | Intermittent | This build can report `Inspector.detached: Render process gone`; the exact startup failure is bounded and recorded. |
| Multi-target/session isolation | Pass | Two target/session pairs keep query, export, and real socket-close records isolated. |
| Startup `Page.navigate` | Sampled | Three successful startup/full-traffic samples are required; exact `Render process gone.` retries are recorded and must stay at or below a 0.34 failure rate. Other discovery, WebSocket, timeout, and post-startup errors fail immediately. |
The resource numbers are a one-time `ps` snapshot of the browser process tree, not a benchmark. Repeat under the intended container limits before setting capacity targets.
Each full sample takes five `ps` samples 200 ms apart from the browser process tree and reports min/max/mean/p95. The local guardrails are RSS max 1,500,000 KiB and at most 20 processes; exceeding either fails the run. These are single-instance development thresholds, not production capacity claims. Tune with `M0_RESOURCE_SAMPLES`, `M0_RESOURCE_SAMPLE_INTERVAL_MS`, `M0_MAX_RSS_KIB`, and `M0_MAX_PROCESS_COUNT` under the intended container limits.
The runner restarts Clark at most three times only from the local startup navigation/readiness block, and only after CDP reports the exact detached reason `Render process gone.`. Every failed attempt is retained in `attempt_failures`; a timeout without that reason and every post-startup error fail immediately with the original message.
The runner restarts Clark at most three times per startup sample only from the local navigation/readiness block, and only after CDP reports the exact detached reason `Render process gone.`. Every failed attempt is retained with sample, attempt, error, and detached reason. A timeout without that reason and every discovery, connection, or post-startup error fail immediately with the original message. The default renderer failure-rate guardrail is 0.34 and can be tightened with `M0_MAX_RENDERER_FAILURE_RATE` after more samples.