diff --git a/docs/CHATWOOT_PARITY_DEVELOPMENT_PLAN.md b/docs/CHATWOOT_PARITY_DEVELOPMENT_PLAN.md index c27ef8ae..21f968d9 100644 --- a/docs/CHATWOOT_PARITY_DEVELOPMENT_PLAN.md +++ b/docs/CHATWOOT_PARITY_DEVELOPMENT_PLAN.md @@ -17,8 +17,8 @@ Build GoChat as a Go backend that can directly reuse the frontend from `referenc ## Current Baseline - Latest implementation checkpoint: `feat(captain): gate document sync backend`. -- Latest documentation checkpoint: this checkpoint, recorded with the B11.3c Captain document sync backend gate implementation. -- Worktree status at this implementation checkpoint: B11.1a aligns Captain assistant CRUD/tools/inbox bindings; B11.1b aligns Captain scenarios and custom tools; B11.1c aligns Captain documents, assistant responses, bulk actions, and custom-tool test payloads; B11.2 aligns Copilot thread/message create/list/get/delete payloads, account/user scoping, and no-LLM fallback persistence; B11.3a aligns Captain preferences show/update payloads and account-level model/feature storage; B11.3b aligns Captain playground request/response payloads, account scoping, v2 history handling, and no-LLM fallback; B11.3c adds the fakeable Captain document sync backend gate with disabled, failed, and fake-success states. Next active slice is B11.3d Copilot tasks/tool-call depth, followed by B11.3e streaming/realtime compatibility and B12 reused frontend smoke. +- Latest documentation checkpoint: this checkpoint, recorded with the B11.3d/B11.3e/B12 executable landing plan. +- Worktree status at this documentation checkpoint: B11.1a aligns Captain assistant CRUD/tools/inbox bindings; B11.1b aligns Captain scenarios and custom tools; B11.1c aligns Captain documents, assistant responses, bulk actions, and custom-tool test payloads; B11.2 aligns Copilot thread/message create/list/get/delete payloads, account/user scoping, and no-LLM fallback persistence; B11.3a aligns Captain preferences show/update payloads and account-level model/feature storage; B11.3b aligns Captain playground request/response payloads, account scoping, v2 history handling, and no-LLM fallback; B11.3c adds the fakeable Captain document sync backend gate with disabled, failed, and fake-success states. Next active implementation slice is B11.3d Copilot tasks/tool-call depth, followed by B11.3e streaming/realtime compatibility and B12 reused frontend smoke. - `go test ./...` passes. - Route dump succeeds with `TOTAL: 830` after adding the Chatwoot-compatible applied-SLA index route. - Route parity artifacts now exist under `docs/parity/` and are generated by `cmd/route_parity`. @@ -155,6 +155,7 @@ This ledger records the committed parity checkpoints that future slices should b | `docs: land captain copilot completion plan` | Documentation-only handoff checkpoint that breaks the remaining B11.3 work into explicit Captain playground, document sync/indexing, Copilot task/tool-call, streaming/realtime, and B12 smoke slices. It also records the reference files, required payload contracts, fallback behavior, verification commands, and status gates so no Hermes-era plan remains implicit. | `git diff --check`; no Go code changed. | Start B11.3b Captain playground implementation from the recorded contract. | | `feat(captain): align playground fallback` | Completes B11.3b Captain assistant playground parity: `POST /captain/assistants/:assistant_id/playground` now accepts the Chatwoot frontend top-level `message_content` and `message_history` payload, enforces account-scoped assistant lookup, returns raw legacy `{ content }` or v2 `{ response }` payloads, appends the current user message once for `captain_integration_v2`, preserves history ordering through the LLM boundary, and returns a deterministic raw disabled response when no LLM provider is configured. | `go test ./internal/handler/api/v1 -run CaptainAssistant -count=1`; `go test ./internal/service -run Captain -count=1`; `go test ./internal/handler/api/v1 -run 'Captain\|Copilot' -count=1`; `go test ./internal/service -run 'Captain\|Copilot' -count=1`; `go test ./internal/handler/api/v1 -count=1`; `go test ./internal/service -count=1`; `go test ./...`; `git diff --check`. Focused handler fixtures cover raw no-envelope fallback, no-history default, cross-account rejection, v2 append/no-duplicate history behavior, and fake LLM request capture. | Continue B11.3c with Captain document sync/indexing and Meilisearch/embedding gates. | | `feat(captain): gate document sync backend` | Completes B11.3c's first document sync gate: Captain document sync now has a fakeable `CaptainDocumentSyncBackend` boundary and account-scoped `SyncDocumentByAccount` worker entry that marks disabled config as `sync_disabled`, records backend errors as `sync_error`, preserves backend error codes such as `access_denied`, stores successful content/title/fingerprint updates as `synced`, and avoids external HTTP/LLM calls in default tests. | `go test ./internal/service -run CaptainDocument -count=1`; `go test ./internal/service -run 'Captain\|Copilot' -count=1`; `go test ./internal/handler/api/v1 -run 'Captain\|Copilot' -count=1`; `go test ./internal/service -count=1`; `go test ./internal/handler/api/v1 -count=1`; `go test ./...`; `git diff --check`. Focused service fixtures cover disabled, fake success, backend error, backend error-code, content fingerprint normalization, and account scoping. | Continue B11.3d with Copilot task/tool-call persistence and suggestion payload depth. | +| `docs: land copilot task execution plan` | Documentation-only checkpoint that fully lands the B11.3d/B11.3e/B12 execution plan: Captain task routes now have exact Chatwoot request/response contracts, current Go gaps, task persistence rules, follow-up context behavior, no-provider disabled-state rules, focused verification commands, streaming/realtime fallback gates, and reused frontend smoke deliverables. | `git diff --check`; no Go code changed. | Start B11.3d implementation from the recorded task/tool-call contract before moving to streaming or frontend smoke. | ## Next Slice Contract @@ -875,6 +876,54 @@ B11.3c current checkpoint: - Account-scoped document lookup prevents another account from invoking sync work against a document primary key. - Default tests do not open external sockets or call an LLM provider; fake backend tests cover success/failure states. +B11.3d Copilot task/tool-call landing contract: + +| Area | Current Chatwoot contract | Current Go gap | Required Go work | Required tests | +| --- | --- | --- | --- | --- | +| Task routes and request bodies | `reference/chatwoot/app/controllers/api/v1/accounts/captain/tasks_controller.rb` exposes `rewrite`, `summarize`, `reply_suggestion`, `label_suggestion`, and `follow_up` under account-scoped Captain task routes. Dashboard `api/captain/tasks.js` posts `content`, `operation`, `conversation_display_id`, `follow_up_context`, and `message`. | `CaptainTaskHandler` returns the local `response.OK` envelope, `CaptainTaskExtendedHandler` still expects query `conversation_ids`, and request structs use local `tone`, `language`, `assistant_id`, or DB `conversation_id` fields. | Bind the exact frontend JSON bodies while keeping harmless legacy fields only behind service adapters. Resolve conversations by account plus display ID with `FindByAccountAndDisplayIDOrID`. Return raw Chatwoot JSON, never `{ success, data }`. | Handler tests post exact frontend payloads for all five actions and assert raw shapes, account scoping, display-ID lookup, and no local envelope. | +| Task response shape | `render_result` returns `{ message: nil }` for nil results, `422 { error }` for errors, or `{ message, follow_up_context }` on success. `useCaptain.js` consumes `data.message` and optional `data.follow_up_context`. | Current Go returns task-specific `rewritten_content`, `summary`, `suggestions`, `source`, or local error envelopes. Missing LLM provider can panic through `s.llmProvider.ChatCompletion`. | Standardize service output to a Chatwoot task result with `message`, optional `follow_up_context`, and raw `422 { error }`. Nil or disabled states must be deterministic and frontend-readable. | No-provider tests for each route assert raw disabled/error payloads and no panic/500. Success tests assert `data.message` and optional `follow_up_context`. | +| Rewrite operations | `Captain::RewriteService` allows `fix_spelling_grammar`, `improve`, `casual`, `professional`, `friendly`, `confident`, and `straightforward`; `improve` may use conversation context by display ID. | Current Go uses `Tone`/`Language` and does not validate Chatwoot `operation`. | Add `operation` validation, map operations to prompts, and include conversation context for `improve` through account-scoped display-ID lookup. | Rewrite tests cover every accepted operation group, invalid operation `422`, context lookup, and raw success/error shape. | +| Summary/reply/label suggestions | `SummaryService`, `ReplySuggestionService`, and `LabelSuggestionService` all take `conversation_display_id`; label suggestion may return nil/empty as `{ message: nil }`. | Current Go uses DB IDs or query `conversation_ids`; label/follow-up handler contract is from an older local design. | Implement single-conversation display-ID tasks for summarize, reply suggestion, and label suggestion. Keep any bulk query endpoints only as legacy-compatible extras. | Tests cover display-ID lookup, wrong-account rejection, label nil result shape, and frontend request bodies. | +| Follow-up context | `Captain::BaseTaskService` emits `follow_up_context` with `event_name`, `original_context`, `last_response`, `conversation_history`, and `channel_type`. `Captain::FollowUpService` accepts only known event names and appends user/assistant turns into `conversation_history`. | Current Go has no Chatwoot follow-up refinement path and the handler expects query params. | Add a follow-up DTO for `follow_up_context`, `message`, and `conversation_display_id`; validate known event names; append history; preserve `channel_type`; return updated `{ message, follow_up_context }`. | Service tests cover missing/invalid context `422`, allowed event names, history append order, and updated `last_response`. | +| Reloadable task/tool-call state | Chatwoot Copilot messages allow JSON keys `content`, `reasoning`, `function_name`, and `reply_suggestion`; thread/message serializers already landed in B11.2. | Task endpoints currently produce transient responses and do not persist enough frontend state for reload/tool-call review. | Persist task outputs/tool-call-like state either as scoped `copilot_messages` attached to a thread/conversation context or as a documented task record that serializes with the same allowed keys. Do not store unknown JSON keys. | Repository/service tests prove persisted records survive list/reload and reject or strip disallowed message keys. Serializer tests cover `content`, `reasoning`, `function_name`, and `reply_suggestion`. | + +B11.3d reference files already inspected: + +- `reference/chatwoot/app/controllers/api/v1/accounts/captain/tasks_controller.rb` for route actions, request params, and `render_result` response rules. +- `reference/chatwoot/app/javascript/dashboard/api/captain/tasks.js` and `reference/chatwoot/app/javascript/dashboard/composables/useCaptain.js` for frontend request/response usage. +- `reference/chatwoot/lib/captain/base_task_service.rb`, `rewrite_service.rb`, `summary_service.rb`, `reply_suggestion_service.rb`, `label_suggestion_service.rb`, and `follow_up_service.rb` for operation names, disabled/error behavior, follow-up context, and event names. +- `reference/chatwoot/enterprise/app/models/copilot_message.rb` for reloadable Copilot message JSON keys. + +B11.3d exit commands: + +```bash +env GOCACHE=/tmp/gochat-gocache GOMODCACHE=/tmp/gochat-gomodcache go test ./internal/handler/api/v1 -run 'CaptainTask|Captain|Copilot' -count=1 +env GOCACHE=/tmp/gochat-gocache GOMODCACHE=/tmp/gochat-gomodcache go test ./internal/service -run 'CaptainTask|Captain|Copilot' -count=1 +env GOCACHE=/tmp/gochat-gocache GOMODCACHE=/tmp/gochat-gomodcache go test ./internal/repository -run 'Copilot|Captain' -count=1 +env GOCACHE=/tmp/gochat-gocache GOMODCACHE=/tmp/gochat-gomodcache go test ./internal/handler/api/v1 -count=1 +env GOCACHE=/tmp/gochat-gocache GOMODCACHE=/tmp/gochat-gomodcache go test ./internal/service -count=1 +env GOCACHE=/tmp/gochat-gocache GOMODCACHE=/tmp/gochat-gomodcache go test ./... +git diff --check +``` + +B11.3e streaming/realtime landing contract: + +| Area | Landing requirement | Done signal | +| --- | --- | --- | +| Captain task streams | Any `*/stream` task route must accept the same request DTOs as its non-streaming route. If the configured LLM provider cannot stream, return a deterministic frontend-compatible disabled event sequence or documented non-streaming fallback. | SSE tests cover no-provider, provider error, and success chunk shapes without placeholder success. | +| Copilot realtime | Copilot message broadcasts should reuse the B11.2 raw thread/message serializer and allowed message keys. | Event/push payload tests compare created message payloads with list/show serializers. | +| Feature gates | Missing provider credentials, disabled Captain task features, or unimplemented streaming should be visible as raw disabled/error states. | No route returns a local envelope or opaque `500` for a known disabled state. | +| Route stability | Existing Chatwoot-compatible paths stay mounted; legacy local paths may remain only if they do not change frontend route matching. | Route dump remains stable unless a tracked route is intentionally added and parity artifacts are regenerated. | + +B12 smoke harness landing contract: + +| Area | Landing requirement | Done signal | +| --- | --- | --- | +| Boot path | Add or document one command that starts GoChat and the reused `reference/chatwoot` frontend pointed at GoChat without adapters. | The command is checked into docs or script form and can be repeated by a new agent. | +| Seed path | Provide deterministic login/account/inbox/contact/conversation data plus enterprise fixtures for SLA, CSAT, automation, macro, audit, custom role, capacity, Captain, and Copilot. | Seed command or fixture instructions are recorded under `docs/parity/`. | +| Core smoke | Cover auth/profile, inbox list/settings, conversation list/detail/message send, contact/company views, widget config/message, public inbox flow, and public CSAT. | `docs/parity/frontend_smoke_report.md` records pass/fail with failed API calls mapped to owning slices. | +| Enterprise smoke | Cover SLA reports, CSAT reports/download, automation rules, macros, audit logs, custom roles, inbox limits, agent capacity, Captain, and Copilot. | Every enterprise failure becomes a named B-slice or Phase 5 follow-up, not hidden browser-only debt. | + B11 landing rules: | Area | Landing requirement | Done signal | @@ -1552,3 +1601,4 @@ Verification milestone gates: - 2026-06-05: B11.3a Captain preferences checkpoint prepared as `feat(captain): align preferences payloads`; Captain preferences now use Chatwoot's raw `{ providers, models, features }` show/update payload, persist `captain_models`/`captain_features` on accounts, merge partial preference updates, reject invalid model selections with `422`, gate updates to administrators, and expose exact no-trailing-slash GET/PUT preference routes. Focused CaptainPreference handler/service tests, Captain/Copilot handler/service tests, handler/service/model/router package tests, migration tests, route dump/parity regeneration, escalated full `go test ./...`, and `git diff --check` passed. Next slice is B11.3b Captain playground parity. - 2026-06-05: B11.3b Captain playground checkpoint prepared as `feat(captain): align playground fallback`; playground now accepts Chatwoot top-level `message_content/message_history`, scopes assistant lookup to the account, returns raw legacy `{ content }` or v2 `{ response }`, appends current v2 user messages without duplication, preserves history into the fakeable LLM boundary, and returns deterministic no-LLM fallback JSON instead of 500. Focused CaptainAssistant/Captain tests, handler/service package tests, full `go test ./...`, and `git diff --check` passed. Next slice is B11.3c document sync/indexing and Meilisearch/embedding gates. - 2026-06-05: B11.3c Captain document sync gate checkpoint prepared as `feat(captain): gate document sync backend`; document sync now has a fakeable backend boundary and account-scoped worker entry, disabled config is recorded as `sync_disabled`, backend failures become `sync_error` or preserve explicit backend error codes, fake success updates content/title/fingerprint/timestamps as `synced`, and default tests avoid external network/LLM calls. Focused Captain/Copilot tests, handler/service package tests, full `go test ./...`, and `git diff --check` passed. Next slice is B11.3d Copilot task/tool-call persistence. +- 2026-06-05: B11.3d/B11.3e/B12 planning checkpoint prepared as `docs: land copilot task execution plan`; the active tracker now records exact Chatwoot Captain task request/response contracts, current Go handler/service gaps, follow-up context and allowed operation rules, reloadable Copilot message/tool-call persistence requirements, no-provider disabled-state expectations, streaming/realtime fallback gates, B12 smoke harness deliverables, and focused exit commands. Documentation-only checkpoint; `git diff --check` passed. Next slice is B11.3d implementation.