rogee/gochat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: land active parity tracking board

Browse Source
This commit is contained in:
rogee
2026-06-05 18:27:22 +08:00
parent cda7d0266e
commit 0744843743
1 changed files with 53 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CHATWOOT_PARITY_DEVELOPMENT_PLAN.md
+53 -35
View File
@@ -16,10 +16,10 @@ Build GoChat as a Go backend that can directly reuse the frontend from `referenc
## Current Baseline
- Plan freeze checkpoint: 2026-06-05 docs-only tracker landing after `13cb750 feat(captain): align task payload persistence`.
- Latest implementation checkpoint: this checkpoint, prepared as `feat(conversations): queue bulk actions`.
- Latest documentation checkpoint before this freeze: `3263ed9 docs: land copilot task execution plan`; this document now carries the active follow-up plan directly.
- Worktree status at this implementation checkpoint: B11.1a aligns Captain assistant CRUD/tools/inbox bindings; B11.1b aligns Captain scenarios and custom tools; B11.1c aligns Captain documents, assistant responses, bulk actions, and custom-tool test payloads; B11.2 aligns Copilot thread/message create/list/get/delete payloads, account/user scoping, and no-LLM fallback persistence; B11.3a aligns Captain preferences show/update payloads and account-level model/feature storage; B11.3b aligns Captain playground request/response payloads, account scoping, v2 history handling, and no-LLM fallback; B11.3c adds the fakeable Captain document sync backend gate with disabled, failed, and fake-success states; B11.3d aligns Captain task request/response payloads, no-provider disabled states, follow-up context, suggestion persistence, and Copilot message tool-call key validation; B11.3e aligns Captain stream DTOs/disabled SSE fallbacks and Copilot push-event payload shapes; B12.1 adds the reusable GoChat server/seed entrypoint plus a Meilisearch-first reused Chatwoot frontend smoke harness and report; B12.2a adds API smoke assertions for auth/profile, inbox, conversation/messages, contact/company, widget config/message, and public CSAT; B12.2b adds a zero-dependency Chrome DevTools browser smoke that loads the reused Chatwoot login and dashboard entrypoints through Vite and checks browser auth/dashboard API requests; B12.3a adds enterprise API smoke assertions for SLA reports/download, CSAT reports/download, automation/macros, audit/custom roles, capacity, Captain, and Copilot; B12.3b adds reused-frontend enterprise browser route navigation for SLA, CSAT, automation, macros, audit logs, custom roles, capacity, Captain, and Copilot request coverage; P5.1 adds the PostgreSQL-backed durable `background_jobs` model/migration plus WorkerPool enqueue, schedule, retry/backoff, dead-letter, idempotency, stale-lock recovery, and focused tests; P5.2 wires `channel.Dispatcher` and `dispatch.EventDispatcher` async paths into durable event jobs with worker replay tests; P5.3 queues Meilisearch write-side index/delete jobs for conversations, messages, contacts, companies, and articles while keeping search reads Meilisearch-first; P5.4 queues automation webhook and email transcript side effects as durable jobs while preserving fakeable delivery boundaries; P5.5 queues Chatwoot-style macro execute fan-out through durable `automation:macro_execution` jobs; P5.6 queues resolve-triggered CSAT survey sends and WhatsApp/Twilio CSAT template creation through durable jobs; P5.7 queues Chatwoot enterprise SLA account scans and applied-SLA evaluation jobs through the durable worker; P5.8 queues Chatwoot-style contact export artifact generation through durable `contact:export` jobs; P5.11 queues Captain document sync requests through durable `captain:document_sync` jobs; P5.12 queues scheduled item fan-out, one-off campaigns, snoozed conversation reopening, account auto-resolution, widget/public message status updates, and account conversation bulk actions through durable jobs. Next active implementation slice is remaining Captain/Copilot jobs, provider webhook/outbound job fan-out, inbound webhook job deferral, and analytics aggregation.
- Current tracking checkpoint: 2026-06-05 after `cda7d02 feat(conversations): queue bulk actions`.
- Latest implementation checkpoint: `cda7d02 feat(conversations): queue bulk actions`.
- Latest documentation-only checkpoint: this checkpoint, prepared as `docs: land active parity tracking board`; this document is now the active follow-up plan and supersedes `.hermes/plans/*`.
- Worktree status at this implementation checkpoint: B11.1a aligns Captain assistant CRUD/tools/inbox bindings; B11.1b aligns Captain scenarios and custom tools; B11.1c aligns Captain documents, assistant responses, bulk actions, and custom-tool test payloads; B11.2 aligns Copilot thread/message create/list/get/delete payloads, account/user scoping, and no-LLM fallback persistence; B11.3a aligns Captain preferences show/update payloads and account-level model/feature storage; B11.3b aligns Captain playground request/response payloads, account scoping, v2 history handling, and no-LLM fallback; B11.3c adds the fakeable Captain document sync backend gate with disabled, failed, and fake-success states; B11.3d aligns Captain task request/response payloads, no-provider disabled states, follow-up context, suggestion persistence, and Copilot message tool-call key validation; B11.3e aligns Captain stream DTOs/disabled SSE fallbacks and Copilot push-event payload shapes; B12.1 adds the reusable GoChat server/seed entrypoint plus a Meilisearch-first reused Chatwoot frontend smoke harness and report; B12.2a adds API smoke assertions for auth/profile, inbox, conversation/messages, contact/company, widget config/message, and public CSAT; B12.2b adds a zero-dependency Chrome DevTools browser smoke that loads the reused Chatwoot login and dashboard entrypoints through Vite and checks browser auth/dashboard API requests; B12.3a adds enterprise API smoke assertions for SLA reports/download, CSAT reports/download, automation/macros, audit/custom roles, capacity, Captain, and Copilot; B12.3b adds reused-frontend enterprise browser route navigation for SLA, CSAT, automation, macros, audit logs, custom roles, capacity, Captain, and Copilot request coverage; P5.1 adds the PostgreSQL-backed durable `background_jobs` model/migration plus WorkerPool enqueue, schedule, retry/backoff, dead-letter, idempotency, stale-lock recovery, and focused tests; P5.2 wires `channel.Dispatcher` and `dispatch.EventDispatcher` async paths into durable event jobs with worker replay tests; P5.3 queues Meilisearch write-side index/delete jobs for conversations, messages, contacts, companies, and articles while keeping search reads Meilisearch-first; P5.4 queues automation webhook and email transcript side effects as durable jobs while preserving fakeable delivery boundaries; P5.5 queues Chatwoot-style macro execute fan-out through durable `automation:macro_execution` jobs; P5.6 queues resolve-triggered CSAT survey sends and WhatsApp/Twilio CSAT template creation through durable jobs; P5.7 queues Chatwoot enterprise SLA account scans and applied-SLA evaluation jobs through the durable worker; P5.8 queues Chatwoot-style contact export artifact generation through durable `contact:export` jobs; P5.11 queues Captain document sync requests through durable `captain:document_sync` jobs; P5.12 queues scheduled item fan-out, one-off campaigns, snoozed conversation reopening, account auto-resolution, widget/public message status updates, and account conversation bulk actions through durable jobs. Next active implementation slice is P5.10 outbound message delivery, followed by P5.9 inbound webhook deferral, remaining P5.11 Captain/Copilot jobs, and P5.13 analytics aggregation.
- `go test ./...` passes.
- Route dump succeeds with `TOTAL: 830` after adding the Chatwoot-compatible applied-SLA index route.
- Route parity artifacts now exist under `docs/parity/` and are generated by `cmd/route_parity`.
@@ -45,16 +45,18 @@ Next ordered checkpoints:
| Order | Slice | Required outcome | Primary verification |
| --- | --- | --- | --- |
| 1 | B12 | Reused Chatwoot frontend smoke runs repeatably against GoChat. | Checked smoke command plus gap report under `docs/parity/`. |
| 2 | Phase 5 job parity | Any async behavior deferred from B7-B11 has a durable worker decision, fakeable delivery boundary, and replay/idempotency tests. | Focused worker/job tests plus a Phase 5 gap ledger update. |
| 3 | Phase 2/3 audit pass | Route/controller/serializer drift found by B12 or new reference inspection is captured as named slices, not free-form TODOs. | Regenerated route parity artifacts and fixture-backed serializer tests. |
| 4 | Phase 6 placeholder burn-down | Remaining account/contact/conversation/message/inbox placeholder handlers are either real Chatwoot-compatible flows or explicitly tracked as unsupported reference gaps. | `rg` placeholder audit, route smoke, and endpoint-family tests. |
| 1 | P5.10 outbound jobs | Queue Chatwoot `SendReplyJob`-style outbound provider delivery and provider delivery-status updates. | Focused message delivery worker tests, provider fake tests, and `go test ./...`. |
| 2 | P5.9 inbound jobs | Move provider webhook persistence/dispatch that Chatwoot defers into durable jobs while keeping HTTP acknowledgement behavior compatible. | Provider webhook handler tests plus worker replay/idempotency tests. |
| 3 | P5.11 remaining Captain/Copilot jobs | Queue Captain crawl/response/embedding and Copilot response work behind fakeable provider gates. | Captain/Copilot worker tests and disabled/failure-state fixtures. |
| 4 | P5.13 analytics aggregation | Replace frontend-visible placeholder report values with scheduled or cached real aggregations. | Report service/handler fixtures and freshness/idempotency tests. |
| 5 | Phase 2/3 audit pass | Route/controller/serializer drift found by B12 or new reference inspection is captured as named slices, not free-form TODOs. | Regenerated route parity artifacts and fixture-backed serializer tests. |
| 6 | Phase 6 placeholder burn-down | Remaining account/contact/conversation/message/inbox placeholder handlers are either real Chatwoot-compatible flows or explicitly tracked as unsupported reference gaps. | `rg` placeholder audit, route smoke, and endpoint-family tests. |
## Handoff Contract
This checkpoint is intended to make the development plan complete enough to track without reading Hermes notes first.
- The next active implementation slice is B12. B11.3e has deterministic Captain stream disabled/error SSE behavior and Copilot push payload tests; provider-specific realtime delivery depth remains a Phase 5 integration concern if it needs durable dispatch infrastructure.
- The next active implementation slice is Phase 5 remaining job parity, starting with P5.10 outbound message delivery. B12 has repeatable API/browser/enterprise smoke harnesses in Review; optional live failures should be converted into named slices instead of blocking Phase 5 job work.
- The Hermes search plan is fully represented by Phase 1/B6. Future search changes must be Meilisearch-first and must not reintroduce production DB fallback.
- The Hermes automation/macro/CSAT plan is fully represented by B8/B9 and Phase 5. Durable delayed execution and channel-specific template delivery remain visible Phase 5 work, not hidden Done work.
- Enterprise scope is fixed: SLA, Audit, CustomRole, AgentCapacity, Captain/Copilot, CSAT, InboxLimit, automation, macros, assignment policies, and related limits/workflows are in scope; SSO/SAML/LDAP/OIDC are out of scope.
@@ -64,8 +66,8 @@ Open work after the current checkpoint:
| Area | Next concrete action | Tracking location | Done boundary |
| --- | --- | --- | --- |
| Phase 5 jobs | Finish outbound delivery, inbound webhook deferral, remaining Captain/Copilot jobs, and analytics aggregation on top of the committed durable worker. | `Phase 5: Background Jobs And Integrations` | Worker tests prove enqueue, retry, idempotency, and fakeable external effects. |
| B12 smoke | Run optional live API/browser/enterprise smoke in a full PostgreSQL/Redis/Meilisearch/Vite environment and convert failures into named slices. | `B12 reused frontend verification breakdown` | `docs/parity/frontend_smoke_report.md` records checked pass/fail results and maps failures to slices. |
| Phase 5 jobs | Decide durable worker mechanism and wire deferred SLA/automation/macro/export/template delivery jobs behind idempotent boundaries. | `Phase 5: Background Jobs And Integrations` | Worker tests prove enqueue, retry, idempotency, and fakeable external effects. |
| Phase 2/3 drift | Expand tracked route/serializer fixtures when B12 exposes frontend-critical gaps. | `Phase 2`, `Phase 3`, `docs/parity/` | Route parity remains 0 missing for tracked frontend routes; serializers have reference fixtures. |
| Phase 6 placeholders | Re-run placeholder audit and burn down any frontend-reachable stub. | `Phase 6: Core Product Placeholder Burn-down` | Stub list has no reused-frontend critical path without a named owner. |
@@ -77,11 +79,25 @@ Open work after the current checkpoint:
| Phase 1 | Meilisearch search engine | Review | B6 payload parity, optional live gate, and DB-fallback hardening are implemented; an actual live Meilisearch run is optional and environment-dependent |
| Phase 2 | Route and controller parity audit | Doing | Ruby/Bundler unavailable, so Chatwoot route extraction currently uses static `routes.rb` fallback |
| Phase 3 | Data and serializer parity | Doing | JSON fixture coverage is partial and still endpoint-family based |
| Phase 4 | Enterprise feature completion | Doing | B7, B8, B9, B10, and B11 are in Review; B12 reused frontend smoke is the next broad verification gate |
| Phase 4 | Enterprise feature completion | Doing | B7, B8, B9, B10, and B11 are in Review; B12 reused frontend smoke harnesses exist and optional live runs can expose follow-up slices |
| Phase 5 | Background jobs and integrations | Doing | P5.1/P5.2/P5.3/P5.4/P5.5/P5.6/P5.7/P5.8/P5.11/P5.12 durable worker, event dispatch, search indexing, automation delivery, macro, CSAT survey/template, SLA scan, contact export, Captain document sync, conversation maintenance, message status update, and account bulk-action cores are in Review; remaining Captain/Copilot/provider/outbound job integration remains open |
| Phase 6 | Core placeholder burn-down | Doing | account/contact/conversation/message/inbox placeholder groups remain broad |
| Phase 7 | Verification harness | Review | B12.1 boot/readiness, B12.2a API assertions, B12.2b browser smoke harness, B12.3a enterprise API assertions, and B12.3b enterprise browser route navigation exist; optional live Meilisearch/full-browser runs remain environment-dependent |
## Active Remaining Work Board
This table is the shortest authoritative handoff view. If an older lower section still says a completed checkpoint is `Todo`, prefer this board and update the older row in the same commit that touches that area.
| Priority | Workstream | Current state | Next checkpoint | Commit close rule |
| --- | --- | --- | --- | --- |
| 1 | P5.10 outbound message delivery | Not started after durable worker foundations. Existing outgoing message creation still needs Chatwoot `SendReplyJob` separation and duplicate-send review. | Queue outbound provider delivery and delivery-status updates through durable jobs. | Commit code, tests, P5.10 table row, commit ledger row, and progress log together. |
| 2 | P5.9 inbound webhook deferral | Provider ingress routes verify, persist, and dispatch synchronously. | Preserve provider HTTP ack behavior while moving Chatwoot-deferred persistence/dispatch into retryable jobs. | Provider HTTP tests plus worker replay/idempotency tests must pass. |
| 3 | P5.11 Captain/Copilot jobs | Captain document sync is queued; crawl/response/embedding/Copilot response jobs remain open. | Add fakeable durable jobs for remaining Captain/Copilot async work and disabled-provider states. | Focused Captain/Copilot worker tests plus full `go test ./...`. |
| 4 | P5.13 reports/analytics | Frontend smoke harness exists; `analytics_service` still has placeholder aggregation paths. | Replace frontend-visible report placeholders with real scheduled/cached aggregations. | Report fixtures verify values, cache/freshness behavior, and no hidden placeholder JSON. |
| 5 | Phase 2/3 drift | Tracked route parity is 0 missing for the current critical set; serializer fixtures remain partial. | Expand route/serializer fixtures when smoke or reference inspection exposes drift. | Regenerate parity artifacts and add endpoint-family fixture tests. |
| 6 | Phase 6 placeholder audit | Widget/public/webhook critical placeholders are burned down; account/contact/conversation/message/inbox audit remains broad. | Run a fresh placeholder audit and assign every frontend-reachable stub to a tracked owner. | `rg` audit result is recorded and no reused-frontend blocker is ownerless. |
| 7 | B12 optional live smoke | API/browser/enterprise smoke commands are checked in; live runs need PostgreSQL, Redis, Meilisearch, Vite, and Chrome. | Run full live smoke when environment is available and map failures to the board. | `docs/parity/frontend_smoke_report.md` records pass/fail and linked owners. |
## Tracking Artifacts
| Artifact | Purpose | Update rule |
@@ -101,6 +117,7 @@ This ledger records the committed parity checkpoints that future slices should b
| Commit | Scope | Verification summary | Follow-up state |
| --- | --- | --- | --- |
| `docs: land active parity tracking board` | Documentation-only checkpoint that reconciles the current state after `cda7d02`. Adds the front-loaded Active Remaining Work Board, makes Phase 5 the active slice, records P5.10/P5.9/P5.11/P5.13 as the next implementation order, and updates stale B12/M6/M7 tracker rows to Review/Doing. | `git diff --check`; documentation-only change. | Start P5.10 outbound message delivery from the new active board. |
| `feat(conversations): queue bulk actions` | Advances P5.12 with a durable Chatwoot `BulkActionsJob` equivalent for account conversation bulk actions. `POST /bulk_actions` now accepts frontend Chatwoot payloads without `action_name`, including `fields`, nested `labels.add/remove`, `snoozed_until`, and display-ID `ids`; when a WorkerPool is configured it enqueues `conversation:bulk_action` on the `medium` queue and returns empty `200 OK`. Worker replay scopes by account and display IDs, merges labels, applies status/team/assignee/snooze updates, and leaves same-display conversations in other accounts untouched. | `go test ./internal/service -run 'ConversationMaintenanceJobs.*BulkAction\|ConversationMaintenanceJobs' -count=1`; `go test ./internal/handler/api/v1 -run 'BulkActionHandler' -count=1`; `go test ./internal/service ./internal/handler/api/v1 ./internal/worker ./internal/app -count=1`; `go test ./...`; `git diff --check`; full verification recorded in the P5.12 section. | Move P5.12 account bulk actions to Review; continue remaining Captain/Copilot jobs, provider webhook/outbound delivery, inbound webhook job deferral, and analytics aggregation. |
| `feat(conversations): queue message status updates` | Advances P5.12 with a durable Chatwoot `Conversations::UpdateMessageStatusJob` equivalent. Widget and public inbox `update_last_seen` paths now enqueue `conversation:update_message_status` on the `deferred` queue when a WorkerPool is configured. Worker replay validates `read`/`delivered`, skips missing conversations, and updates only non-incoming messages created at or before the viewing timestamp while preserving already-read and post-cutoff messages. | `go test ./internal/service -run 'ConversationMaintenanceJobs\|WidgetService_UpdateLastSeenQueuesMessageStatusJob' -count=1`; `go test ./internal/service ./internal/worker ./internal/app -count=1`; `go test ./...`; `git diff --check`; full verification recorded in the P5.12 section. | Move P5.12 update-message-status depth to Review; continue durable bulk-action fan-out, remaining Captain/Copilot jobs, provider webhook/outbound delivery, and analytics aggregation. |
| `feat(captain): queue document syncs` | Advances P5.11 with a durable Chatwoot `Captain::Documents::PerformSyncJob` equivalent. `POST /captain/documents/:id/sync` still returns `202` after marking the account-scoped document as syncing, but now enqueues `captain:document_sync` on the `low` queue when a WorkerPool is configured. Worker replay calls the existing fakeable `CaptainDocumentSyncBackend`, persists synced/failed/disabled states through `SyncDocumentByAccount`, and lets missing documents or malformed payloads retry through `background_jobs`. | `go test ./internal/service -run 'CaptainDocumentService' -count=1`; `go test ./internal/handler/api/v1 -run 'CaptainDocument\|CaptainResource' -count=1`; `go test ./internal/service ./internal/handler/api/v1 ./internal/worker ./internal/app -count=1`; `go test ./...`; `git diff --check`; full verification recorded in the P5.11 section. | Move P5.11 document sync core to Review; continue Captain crawl/response-building/embedding/Copilot response jobs, provider webhook/outbound delivery, update-message-status/bulk-action jobs, and analytics aggregation. |
@@ -467,10 +484,10 @@ Upcoming enterprise task boards:
| B10 | B10.4 | Review remaining InboxLimit/account-limit create-path enforcement outside capacity policies. | Chatwoot enterprise account/inbox limit policies. | Inbox/channel create tests for over-limit behavior and frontend-readable errors. | Done by `feat(inboxes): enforce chatwoot inbox limits` |
| B11 | B11.1 | Align Captain assistant CRUD, inbox bindings, responses, documents, scenarios, and custom tools payloads. | Captain controllers/services/frontend clients under `reference/chatwoot`. | Handler/service fixtures for every Captain dashboard client path. | Review; assistant CRUD/tools/inbox binding, scenarios, custom tools, documents, assistant responses, bulk actions, and custom-tool test payloads are landed |
| B11 | B11.2 | Align Copilot threads, messages, tasks, preferences, playground/tool-call behavior, and disabled-state feature gates. | Copilot controllers/services/frontend clients under `reference/chatwoot`. | Copilot handler/service tests for persistence, disabled LLM state, and frontend payloads. | Review; thread/message payloads, account/user scoping, assistant scope, and no-LLM fallback are landed; tasks/preferences/tool-call/playground depth remains in B11.3 follow-up |
| B11 | B11.3 | Add document sync/embedding/LLM job boundaries where external dependencies are required and finish remaining Copilot task/preference/tool-call/streaming depth. | Captain/Copilot jobs, document services, Copilot controllers/services/frontend clients. | Worker tests or explicit feature-gated fallback tests plus Copilot task/preference/tool-call fixtures. | Doing; B11.3a Captain preferences show/update payloads are landed, while document sync/indexing and remaining Copilot task/tool-call depth remain active |
| B11 | B11.3 | Add document sync/embedding/LLM job boundaries where external dependencies are required and finish remaining Copilot task/preference/tool-call/streaming depth. | Captain/Copilot jobs, document services, Copilot controllers/services/frontend clients. | Worker tests or explicit feature-gated fallback tests plus Copilot task/preference/tool-call fixtures. | Review; B11.3a preferences, B11.3b playground, B11.3c document sync gate, B11.3d task/tool-call persistence, and B11.3e streaming fallback landed. Remaining durable crawl/response/embedding/Copilot jobs are P5.11 |
| B12 | B12.1 | Add a repeatable command to run the reused Chatwoot frontend against GoChat. | `reference/chatwoot` frontend boot/auth/API clients. | Smoke command documented and runnable locally. | Done by `test(parity): add frontend smoke harness` |
| B12 | B12.2 | Cover login, inbox list/settings, conversation list/detail/message send, contact/company views, widget init/message, public CSAT, SLA/CSAT reports, and enterprise admin screens. | Dashboard route usage and frontend stores/API modules. | Smoke report checked into `docs/parity/` with pass/fail gaps. | Review; B12.2a API assertion command and B12.2b browser smoke harness landed; live run is environment-dependent |
| B12 | B12.3 | Add enterprise smoke coverage for SLA reports/download, CSAT reports/download, automation/macros, audit/custom roles, capacity, Captain, and Copilot. | Enterprise dashboard clients and `reference/chatwoot` enterprise screens. | Enterprise smoke command records pass/fail and names failed route/serializer owners. | Doing; B12.3a API assertion command landed, browser screen navigation remains open |
| B12 | B12.3 | Add enterprise smoke coverage for SLA reports/download, CSAT reports/download, automation/macros, audit/custom roles, capacity, Captain, and Copilot. | Enterprise dashboard clients and `reference/chatwoot` enterprise screens. | Enterprise smoke command records pass/fail and names failed route/serializer owners. | Review; B12.3a enterprise API assertions and B12.3b enterprise browser navigation landed. Optional live execution is environment-dependent |
B8 CSAT execution breakdown:
@@ -905,9 +922,9 @@ B11.3c-B11.3e follow-up contracts:
| Slice | Landing rule | Status gate |
| --- | --- | --- |
| B11.3c document sync/indexing | Document sync must expose frontend-visible `syncing/synced/failed` states and call a fakeable Meilisearch/embedding boundary. Missing provider config must be a raw disabled/failure state, not a placeholder success. | B11 remains `Doing` until disabled, failed, and fake-success sync tests are present. |
| B11.3d Copilot tasks/tool-calls | Rewrite, summarize, reply suggestion, label suggestion, and follow-up task routes must accept Chatwoot request payloads, persist enough task/tool-call output for reloads, and serialize raw frontend payloads. | Done by `feat(captain): align task payload persistence`; B11 remains `Doing` for B11.3e streaming/realtime fallback. |
| B11.3e streaming/realtime | Streaming can be deferred only with a deterministic non-streaming response or explicit disabled state. Realtime push payloads should use the same thread/message serializer already landed in B11.2. | B11 moves to `Review` only after every streaming/realtime endpoint has a tested frontend-compatible fallback. |
| B11.3c document sync/indexing | Document sync must expose frontend-visible `syncing/synced/failed` states and call a fakeable Meilisearch/embedding boundary. Missing provider config must be a raw disabled/failure state, not a placeholder success. | Done by `feat(captain): gate document sync backend`; remaining crawl/embedding jobs are tracked in P5.11. |
| B11.3d Copilot tasks/tool-calls | Rewrite, summarize, reply suggestion, label suggestion, and follow-up task routes must accept Chatwoot request payloads, persist enough task/tool-call output for reloads, and serialize raw frontend payloads. | Done by `feat(captain): align task payload persistence`. |
| B11.3e streaming/realtime | Streaming can be deferred only with a deterministic non-streaming response or explicit disabled state. Realtime push payloads should use the same thread/message serializer already landed in B11.2. | Done by `feat(captain): align streaming fallbacks`; B11 is in Review. |
B11.3c current checkpoint:
@@ -1036,7 +1053,7 @@ B12 smoke harness contract:
| Seed data | Create or document deterministic seed data for admin login, account, inbox, contact, conversation, CSAT, SLA, macro, automation, audit, and custom-role screens. | Seed command or fixture reference in `docs/parity/`. |
| Core smoke paths | Cover login/current-user, inbox list/settings, conversation list/detail/message send, contact/company views, widget config/message, and public CSAT. | `docs/parity/frontend_smoke_report.md` with pass/fail status and failed API calls. |
| Enterprise smoke paths | Cover SLA reports, CSAT reports/download, automation rules, macros, audit logs, custom roles, capacity settings, Captain, and Copilot as their slices land. | Same smoke report links each failure to the owning B-slice. |
| Exit rule | A failing smoke does not block code commits if the failure is named, scoped, and tracked; hidden failures block moving B12 out of `Doing`. | B12 moves to `Review` only with a repeatable command and checked report. |
| Exit rule | A failing optional live smoke does not block code commits if the failure is named, scoped, and tracked; hidden failures block moving B12 to `Done`. | B12 stays in `Review` until live pass/fail results are recorded or every environment-dependent gap has an owner. |
B12.1 current checkpoint:
@@ -1044,7 +1061,7 @@ B12.1 current checkpoint:
- `scripts/parity_frontend_smoke.sh --check` verifies local command prerequisites, validates `cmd/gochat`, and writes `docs/parity/frontend_smoke_report.md` without requiring live PostgreSQL/Redis/Meilisearch/frontend boot.
- `scripts/parity_frontend_smoke.sh --print` prints the exact GoChat and reused Chatwoot frontend commands.
- `scripts/parity_frontend_smoke.sh --boot-only` starts GoChat plus `reference/chatwoot` Vite and verifies backend `/health` plus frontend HTTP readiness. The default backend command is Meilisearch-first via `GOCHAT_SEARCH_ENGINE=meilisearch`; DB fallback is only opt-in through `GOCHAT_SMOKE_SEARCH_ENGINE=db` for local debugging.
- The current report is a readiness report, not a real browser pass. B12.2 must add login/current-user/inbox/conversation/CRM/widget/CSAT browser or API assertions and update the smoke matrix with pass/fail results.
- B12.1 itself is a readiness checkpoint. B12.2 and B12.3 below add the repeatable API/browser assertions and enterprise route coverage; optional live execution still depends on the full local service stack.
B12.1 verification:
@@ -1062,7 +1079,7 @@ B12.2a current checkpoint:
- `scripts/parity_frontend_smoke.sh --api-smoke` now runs against a live GoChat backend. It seeds deterministic data, logs in with `/auth/sign_in`, captures DeviseTokenAuth-compatible headers, and exercises the frontend-critical API paths without frontend adapters.
- Covered API assertions: `/auth/validate_token`, `/api/v1/profile`, inbox list, conversation list/messages/message create, contact show, company show, widget config, widget message create, and public CSAT show.
- `cmd/gochat seed` now emits `company_id`, `conversation_display_id`, and `csat_message_id`; it also creates a deterministic company relation and an `input_csat` message so B12 API smoke can verify CRM and public CSAT surfaces.
- This is still not a browser-navigation pass. B12.2b must drive the reused Vite frontend and capture failed API calls or UI blockers in the report; B12.3 must add enterprise screens.
- This is the core API assertion checkpoint. B12.2b below adds reused Vite browser navigation, and B12.3 adds enterprise API/browser coverage.
B12.2a verification:
@@ -1197,7 +1214,7 @@ This is the ordered queue for the next implementation slices. Do not skip the ro
| Q5 | Review Meilisearch document shape and endpoint payloads against Chatwoot frontend consumers. | Phase 1 and Phase 3 | Search remains Meilisearch-first and payload mismatches are fixed or tracked. | Review |
| Q6 | Implement provider-specific webhook ingress for Chatwoot public webhook paths. | Phase 6 | Generic webhook placeholder no longer masks provider gaps; Telegram, LINE, SMS/Twilio, WhatsApp, Instagram/Twitter/TikTok routes resolve and verify like Chatwoot where supported. | Done |
| Q7 | Burn down enterprise gaps in this order: SLA, assignment policy and capacity, CSAT, automation/macros, Audit, CustomRole, InboxLimit, Captain/Copilot. | Phase 4 and Phase 5 | Each feature passes route, persistence, auth, side-effect, response, and test checks. | Doing |
| Q8 | Add frontend smoke harness using the reused Chatwoot frontend once core API flows boot end-to-end. | Phase 7 | Login, inbox list, conversation list/detail, message send, contact view, and widget init run without frontend adapters. | Todo |
| Q8 | Add frontend smoke harness using the reused Chatwoot frontend once core API flows boot end-to-end. | Phase 7 | Login, inbox list, conversation list/detail, message send, contact view, widget init/message, public CSAT, and key enterprise routes have repeatable API/browser smoke commands. | Review |
## Current Decision Ledger
@@ -1223,8 +1240,8 @@ These milestones are the tracking spine for the remaining Chatwoot frontend reus
| M3 | Serializer parity for frontend API families. | Fixture tests cover auth/profile, accounts/users, inboxes, conversations/messages, contacts/companies, notifications, reports, widget/public, and search. | Doing |
| M4 | Core handler placeholder burn-down. | Account/contact/conversation/message/inbox/webhook handlers are repository-backed and no frontend-critical route returns placeholder JSON. | Doing |
| M5 | Paid feature parity excluding SSO family. | SLA, Audit, CustomRole, AgentCapacity, Captain/Copilot, CSAT, InboxLimit, automation/macros, and assignment policies pass route, persistence, auth, side-effect, serializer, and tests. | Doing |
| M6 | Durable jobs and external integrations. | Search indexing, CSAT send, automation actions, notifications, webhooks, and external deliveries are queued, retryable, logged, and idempotent. | Todo |
| M7 | Reused Chatwoot frontend smoke validation. | Chatwoot frontend boots against GoChat for login, inbox list, conversation detail, message send, contact view, widget init/message, public CSAT, and key enterprise screens without adapters. | Todo |
| M6 | Durable jobs and external integrations. | Search indexing, CSAT send, automation actions, notifications, webhooks, and external deliveries are queued, retryable, logged, and idempotent. | Doing |
| M7 | Reused Chatwoot frontend smoke validation. | Chatwoot frontend boots against GoChat for login, inbox list, conversation detail, message send, contact view, widget init/message, public CSAT, and key enterprise screens without adapters. | Review |
## Slice Backlog
@@ -1243,7 +1260,7 @@ Work proceeds top-down unless a failing test or frontend blocker forces a narrow
| B9 | Automation/macros durable side effects. | Chatwoot automation/macro services and jobs. | Action execution, logs, webhook/email transcript retry tests. | Review |
| B10 | Audit, CustomRole, InboxLimit. | Chatwoot enterprise admin behavior and policies. | Authorization, audit emission, limits enforcement, admin payload fixtures. | Review |
| B11 | Captain/Copilot deep behavior. | Chatwoot Captain/Copilot controllers, services, frontend clients. | Assistant/tool/document/scenario/copilot thread/task tests and feature gates. | Review |
| B12 | Frontend smoke harness. | `reference/chatwoot` frontend. | Repeatable smoke command and checked gap report. | Doing; B12.1 harness landed, B12.2 browser/API assertions pending |
| B12 | Frontend smoke harness. | `reference/chatwoot` frontend. | Repeatable smoke command and checked gap report. | Review; B12.1 boot harness, B12.2 API/browser assertions, and B12.3 enterprise API/browser assertions landed. Optional live runs remain environment-dependent |
Remaining slice landing plan:
@@ -1255,7 +1272,7 @@ Remaining slice landing plan:
| B9 | Done: B9.1 automation rule CRUD/listener/log/external-action parity and B9.2 macro frontend CRUD/execute side effects. | Delayed actions, durable queued worker scheduling, and deeper macro attachment/file parity remain named B9.3/B9.4 follow-ups. | Review after `feat(macros): align chatwoot macro payloads`; move to Done only after durable worker/attachment gaps are implemented or formally split out. |
| B10 | Done: audit list payload, audit writer boundary for representative mutating core resources, CustomRole permission-key parity, AccountUser permission resolution, admin gates, delete nullification, and account-level InboxLimit enforcement in inbox/channel creation paths. | Frontend smoke coverage for enterprise settings remains B12, not a hidden B10 blocker. | Review after B10.4; move to Done only after reused frontend smoke confirms audit/custom-role/limit settings flows or any smoke gaps are split into owned follow-ups. |
| B11 | Done through B11.3e: Captain Assistant CRUD, inbox binding, scenarios, documents, responses, custom tools, Copilot threads/messages, Captain preferences, playground, document sync gates, task/tool-call payload persistence, stream disabled-state SSE, and Copilot push payload fixtures. | Reused frontend smoke remains B12; provider-specific durable realtime/LLM delivery can move to Phase 5 if smoke exposes it. | Review after B11.3e; move to Done only after B12 proves reused frontend Captain/Copilot screens or names any remaining provider/deployment follow-ups. |
| B12 | B12.1 boot/readiness harness landed: GoChat `serve`/`seed`, Meilisearch-first smoke script, and checked report. | Add B12.2 browser/API assertions for auth/profile/inbox/conversation/contact/widget/CSAT, then B12.3 enterprise screens. | Done only after checked smoke paths have pass/fail results and every failure is mapped to an owning slice. |
| B12 | B12.1 boot/readiness, B12.2 core API/browser assertions, and B12.3 enterprise API/browser route assertions landed. | Optional live smoke runs in a full PostgreSQL/Redis/Meilisearch/Vite/Chrome environment must map every failure to an owning slice. | Move to Done only after live-smoke pass/fail results are recorded and every failure is owned. |
Per-slice documentation rule:
@@ -1575,7 +1592,7 @@ Excluded tracking table:
## Phase 5: Background Jobs And Integrations
Status: doing; this is the next active implementation slice after B12 smoke harness review.
Status: doing; this is the current active implementation slice.
Default implementation decision for tracking: use a PostgreSQL-backed durable job table first, with Redis used only for optional locks/rate limiting where existing code already depends on it. This keeps job state inspectable in the Go backend, avoids adding a new hard runtime beyond the already-required database, and still allows Chatwoot-style retry, scheduling, idempotency, and observability. If a specific reference behavior requires Redis timing semantics, record that under the owning P5 task before implementation.
@@ -1590,20 +1607,20 @@ Reference job families to keep in scope:
Known hotspots:
- `internal/worker/worker.go` now has the P5.1 durable core, but no feature-specific producers/handlers are wired yet.
- `internal/automation/action_service.go` has pending webhook/email transcript work.
- `internal/automation/csat_survey_listener.go` now handles CSAT enablement, survey rules, generic send idempotency, and message-updated response building; durable queueing and external WhatsApp/Twilio template delivery remain follow-ups.
- `internal/auth/webhook_registry.go` has pending signature verification for Facebook/WhatsApp.
- `internal/service/analytics_service.go` has placeholder analytics paths.
- `internal/worker/worker.go` has the committed P5.1 durable core and is already wired to dispatcher, search indexing, automation delivery, macro fan-out, CSAT survey/template sends, SLA scans, contact exports, Captain document sync, and conversation maintenance jobs.
- `internal/service/message_service.go`, `internal/channel/registry.go`, and provider outbound listeners remain the next P5.10 focus for Chatwoot `SendReplyJob` parity and duplicate-send prevention.
- `internal/handler/webhook/*` and provider services have synchronous verified ingress/persistence; P5.9 must move Chatwoot-deferred webhook work behind durable jobs without changing provider acknowledgement semantics.
- `internal/service/captain_*` and `internal/service/copilot_*` have document sync job coverage only; P5.11 still needs crawl/response/embedding/Copilot response jobs.
- `internal/service/analytics_service.go` has placeholder analytics/report paths and is the owner for P5.13 aggregation work.
Checklist:
- [x] Add `background_jobs` persistence with job type, payload, queue, status, attempt counters, scheduled/locked timestamps, idempotency key, last error, and completion timestamps.
- [x] Replace `internal/worker/worker.go` placeholder with enqueue, schedule, perform, retry/backoff, dead-letter, and graceful shutdown behavior.
- [ ] Map Chatwoot jobs/listeners to Go worker responsibilities and record out-of-scope migration-only jobs explicitly.
- [ ] Implement durable job dispatch for automation, CSAT, notifications, webhooks, SLA processing, Captain/Copilot, contact export, macros, and search indexing.
- [ ] Add retry and failure logging for external calls.
- [ ] Add tests for enqueueing, uniqueness/idempotency, delayed execution, retries, worker restart pickup, and fakeable external side effects.
- [x] Map the committed Chatwoot job/listener families to Go worker responsibilities in the tracking table.
- [ ] Finish durable job dispatch for outbound provider delivery/status, inbound provider webhook deferral, remaining Captain/Copilot jobs, delayed automation actions, and report aggregation.
- [ ] Add retry and failure logging for the remaining external provider calls.
- [ ] Add tests for the remaining enqueueing, uniqueness/idempotency, delayed execution, retries, worker restart pickup, and fakeable external side effects.
Acceptance:
@@ -1634,7 +1651,7 @@ P5.1 current checkpoint:
- Replaced the `WorkerPool` stub with enqueue/schedule APIs, handler registration, `ProcessOne`, start/stop worker loops, PostgreSQL `FOR UPDATE SKIP LOCKED` claiming, retry/backoff, dead-letter state, and stale-lock recovery for worker restart pickup.
- Kept older `NewWorkerPool()` construction as a no-op-compatible path while adding `NewWorkerPoolWithOptions(db, ...)` for durable wiring.
- Added focused tests proving enqueue/idempotency, due job completion, retry to dead-letter, queue/schedule filtering, and stale running job requeue.
- Remaining Phase 5 work must now wire real producers/handlers: dispatcher async events, Meilisearch indexing, automation webhooks/transcripts, delayed actions/macros, CSAT templates, SLA scans, contact exports, provider webhooks, outbound delivery, Captain/Copilot, and conversation maintenance.
- Remaining Phase 5 work: delayed automation scheduled-item execution, inbound provider webhook deferral, outbound provider delivery/status jobs, Captain crawl/response/embedding/Copilot response jobs, and analytics/report aggregation.
P5.1 verification:
@@ -1895,6 +1912,7 @@ Verification milestone gates:
## Progress Log
- 2026-06-05: Active parity tracker checkpoint prepared as `docs: land active parity tracking board`; the plan now front-loads the current authoritative remaining-work board, points the next implementation slice at P5.10 outbound message delivery, records P5.9 inbound job deferral, P5.11 remaining Captain/Copilot jobs, P5.13 analytics aggregation, Phase 2/3 drift, Phase 6 placeholder audit, and B12 optional live smoke as tracked owners, and reconciles stale B12/M6/M7 status rows. Documentation-only checkpoint; `git diff --check` passed.
- 2026-06-04: Baseline stabilized and committed as `42cdab8 chore: stabilize chatwoot parity baseline`; `go test ./...` passed and route dump reported `TOTAL: 704`.
- 2026-06-04: Phase 1 search foundation added: Meilisearch config/env defaults, `SearchEngine` contract, Meilisearch HTTP wrapper with bootstrap/settings, DB fallback adapter, document builders, reindex command, and no-live-Meilisearch tests. Verified `go test ./...` in unsandboxed mode because miniredis/httptest need local sockets; route dump still reports `TOTAL: 704`.
- 2026-06-04: Phase 1 indexing hooks wired for conversations, messages, contacts, companies, and articles. Create/update/delete paths now call the service-layer `SearchIndexer` boundary, bootstrap injects the Meilisearch-backed search service, and unit tests cover each entity hook path.