From 9c197219919f7fb9708f90ccfafdae4f05a8852a Mon Sep 17 00:00:00 2001 From: Rogee Date: Thu, 20 Aug 2026 22:28:39 +0800 Subject: [PATCH 1/2] docs(H-389): consolidate repository guidance Co-authored-by: multica-agent --- AGENTS.md | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 19ec799..63c4739 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,12 +1,55 @@ # Repository Guidelines -## Scope +## Scope and current phase - Build a CDP-only network observability and debugging control plane for clark-browser/Chromium. - Collect and correlate CDP `Network`/`Fetch` HTTP(S), failure, cache, and WebSocket events. - Keep the MVP read-only. Any future request or response mutation must be explicit opt-in and default off. +- Work in M0 architecture/Spike: validate one browser instance, a local test page, CDP event coverage, stable schema, redaction/truncation, and disconnect behavior before M1. +- Optimize first for one or a few isolated browser instances; do not assume cluster-scale concurrency in M0/M1. - Prefer the smallest correct change and reuse existing patterns before adding dependencies or abstractions. +## Users and supported scenarios + +- Help automation developers diagnose request failures, ordering, retries, and page behavior. +- Let data-pipeline developers consume structured browser network events downstream. +- Let operations and test users inspect session-scoped HTTP and WebSocket failure chains. + +## Architecture and data flow + +- CDP access validates short-lived credentials, protects the WebSocket connection, discovers targets, binds sessions, maintains heartbeats, and reports disconnects. +- The collector enables CDP `Network`/`Fetch` as needed, correlates `requestId`, records HTTP and WebSocket lifecycles, and reads response bodies only on demand. +- The policy layer normalizes events, filters by URL/resource/status, and applies redaction, sampling, size, event, and retention limits. +- Start storage and query with JSONL containing session, target, request, timing, URL/method, safe header summaries, body availability/reference, and error fields. Do not persist full bodies by default. +- Keep query and redacted export session-isolated and paginated. Re-evaluate Postgres, object storage, and asynchronous streams only after MVP evidence. +- A later console may show sessions, timelines, filters, and errors, but never control the browser GUI. + +Data flow: `clark-browser (CDP enabled)` → protected CDP WebSocket → target/session binding → Network/Fetch events and on-demand body reads → normalize/filter/redact/limit → JSONL/event stream → session-scoped API/export. + +CDP observes browser-internal decrypted events. It is not a client/server proxy, does not generate certificates, and must not change the network path. + +## Priority + +- P0: single-instance CDP access; HTTP request/response/failure and WebSocket events; `requestId` correlation; default redaction; body/event limits; JSONL export; session isolation; disconnect observability. +- P1: multiple targets/sessions, paginated query, metrics/alerts, permission audit, asynchronous event streams, and tiered storage. +- P2: Fetch mutation, versioned rules, replay, browser-cluster access, and a minimal console. + +## MVP acceptance + +- A page load reconstructs request → response → completion/failure by `session_id`, with fixture-defined key fields 100% correct. +- WebSocket open/close and sent/received frames correlate to their connection; HTTPS and HTTP/2 remain observable without MITM. +- Sensitive values are redacted by default. Oversized bodies are not persisted and retain a truncation or unavailable reason. +- Query and export are paginated and session-isolated; unauthorized sessions cannot read or export another session. +- CDP disconnects, target closure, dropped events, and individual body-read failures are reported or recoverable without terminating the browser workload. +- Local end-to-end coverage includes connection, collection, correlation, redaction, storage, query, export, and disconnect, with no VNC, MITM, proxy certificate, or CA dependency. + +## Iteration plan + +- M0: confirm clark-browser CDP startup/connection, event coverage, stable schema, and one local page-load loop. +- M1: deliver single-instance collection, redaction/limits, JSONL query/export, disconnect recovery, and local fixtures. +- M2: add multiple targets/sessions, tiered storage, an asynchronous queue, metrics/alerts, and permission audit when evidence warrants them. +- M3: add explicitly authorized Fetch mutation, replay, versioned rules, and browser-cluster access only after earlier phases validate the need. + ## Non-goals - Do not add VNC/noVNC, desktop streaming, or GUI browser control. @@ -22,6 +65,14 @@ - Report disconnects, target closure, dropped events, and body-read failures without stopping the browser workload. - Validate all external input and preserve compatibility of persisted schemas and exported data. +## Constraints and validation + +- CDP does not cover non-browser processes, WebRTC media contents, or final rendered page state. +- Large bodies, high-frequency WebSocket frames, and multi-instance concurrency create memory, throughput, and privacy costs; prefer bounded, sampled, or discardable handling in the MVP. +- Advance beyond M0 only when local fixtures and a local test page demonstrate event/correlation completeness, zero sensitive-value leakage, body-limit behavior, disconnect recovery, and acceptable single-instance resource cost. +- Trigger ordinary HTTP, a failed request, HTTPS, HTTP/2 where available, and a WebSocket during validation; never require external production systems. +- Validate the product by reducing repeated debugging and collection glue code before building clusters, paid proxy integrations, or an operations console. + ## Development and tests - Keep commits focused; do not add speculative scaffolding, dependencies, CI, or an unconfirmed application stack. -- 2.54.0 From ab04c21254b1efa39528ec6f85197d2040447708 Mon Sep 17 00:00:00 2001 From: Rogee Date: Thu, 20 Aug 2026 22:37:28 +0800 Subject: [PATCH 2/2] docs(H-389): tighten CDP security guidance Co-authored-by: multica-agent --- AGENTS.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 63c4739..eb69ce1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -59,7 +59,8 @@ CDP observes browser-internal decrypted events. It is not a client/server proxy, ## Security and reliability - Never commit or log CDP tokens, cookies, `Authorization`, `Set-Cookie`, credentials, full sensitive bodies, or production data. -- Use short-lived CDP credentials over a protected internal or TLS connection, with session isolation and auditable access. +- Never put CDP tokens, `Cookie`, `Authorization`, credentials, or unredacted sensitive values in any URL. +- Treat CDP access as browser-high-privilege: use short-lived credentials, protected internal or TLS transport, least privilege, session isolation, and auditable access. - Redact sensitive headers and bodies before persistence or export; enforce body, event, frame, and retention limits. - Treat response bodies as optional: cache, target closure, protocol behavior, or lifecycle timing may make them unavailable. - Report disconnects, target closure, dropped events, and body-read failures without stopping the browser workload. -- 2.54.0