gochat/docs/CHATWOOT_PARITY_DEVELOPMENT_PLAN.md

Chatwoot Parity Development Plan

Updated: 2026-06-04

Goal

Build GoChat as a Go backend that can directly reuse the frontend from reference/chatwoot. The backend API, data contracts, side effects, permissions, and runtime behavior must match the local reference/chatwoot repository first. Existing docs are secondary when they conflict with the reference implementation.

Confirmed Decisions

• Frontend: reuse Chatwoot frontend directly. Backend compatibility is mandatory.
• Baseline: reference/chatwoot is the source of truth for routes, controllers, models, serializers, jobs, and service behavior.
• Immediate order: keep go test ./... green first, then deepen Chatwoot behavior parity.
• Search: final implementation must use Meilisearch. DB/LIKE search is not acceptable as the final engine.
• Enterprise scope: exclude SSO/SAML/LDAP/OIDC. Include the remaining paid features already present in planning and code: SLA, Audit, CustomRole, AgentCapacity, Captain/Copilot, CSAT, InboxLimit, automation, macros, assignment policies, and related limits/workflows.

Current Baseline

• go test ./... passes.
• Route dump succeeds with TOTAL: 704.
• Handler test stability fixes are committed into the baseline before feature parity work continues.
• .codegraph/ is generated indexing output and is not part of tracked product code.

Phase 0: Test And Route Baseline

Status: done.

Checklist:

• Fix internal/handler/api/v1 route param mismatches around :account_id, :id, and resource IDs.
• Fix malformed test routes that conflict with Gin wildcard rules.
• Add defensive handling for zero-value services in handler edge tests.
• Align handler error responses where tests encode the expected Chatwoot-compatible status category.
• Keep platform routes bootable and dumpable.
• Verify go test ./....
• Verify go run ./cmd/dump_routes.

Verification commands:

env GOCACHE=/tmp/gochat-gocache GOMODCACHE=/tmp/gochat-gomodcache go test ./...
env GOCACHE=/tmp/gochat-gocache GOMODCACHE=/tmp/gochat-gomodcache go run ./cmd/dump_routes

Phase 1: Meilisearch Search Engine

Status: next.

Source material:

• .hermes/plans/2025-05-24-global-search-meilisearch.md
• reference/chatwoot search controllers, models, and indexing behavior
• Current local packages under internal/search, internal/handler/api/v1/search_handler.go, and entity repositories

Checklist:

• Define a stable SearchEngine interface for Meilisearch-backed search and indexing.
• Add search config for engine, host, API key, and index prefix.
• Implement Meilisearch client wrapper with index bootstrapping and settings.
• Define per-entity documents for conversations, messages, contacts, companies, articles, and help-center content as required by Chatwoot frontend behavior.
• Wire create/update/delete hooks from services into indexing.
• Add batch reindex command for existing data.
• Keep DB search only as explicit development fallback, not as final production mode.
• Add tests with a mocked search engine and integration hooks that can run without a live Meilisearch instance.
• Document required Meilisearch environment variables and local startup flow.

Acceptance:

• Global search endpoint and entity search endpoints return Chatwoot-compatible payloads.
• Search results are account-scoped.
• Search indexing survives entity updates and deletes.
• go test ./... stays green.

Tracking table:

ID	Task	Target files	Reference source	Status
P1.1	Add SearchConfig with engine, host, API key, index prefix, env binding, and defaults.	internal/config/config.go, configs/config.yaml, config tests	.hermes/plans/2025-05-24-global-search-meilisearch.md	Done
P1.2	Add stable engine contract for search, indexing, deletes, batch indexing, and close.	internal/search/engine.go	Chatwoot global/entity search behavior	Done
P1.3	Implement Meilisearch engine wrapper, index naming, bootstrap, sortable/filterable/searchable settings.	internal/search/engine_meili.go	Chatwoot search models/services	Review
P1.4	Keep existing DB search as explicit dev fallback only. Production config must prefer Meilisearch.	internal/search/engine_db.go, internal/search/search_service.go	User decision on Meilisearch	Done
P1.5	Define documents and serializers for conversations, messages, contacts, companies, articles, and help-center content.	internal/search/engine.go	reference/chatwoot models/serializers	Review
P1.6	Wire create/update/delete hooks from entity services into async or synchronous indexing boundary.	internal/service/*, internal/search/search_service.go, internal/app/bootstrap.go	Chatwoot callbacks/jobs	Done
P1.7	Add batch reindex command and account/entity filters.	cmd/reindex_search	Chatwoot reindex/search tasks	Done
P1.8	Add mocked engine tests and service integration tests without requiring live Meilisearch.	internal/search/engine_test.go, internal/config/config_test.go	Existing test style	Done
P1.9	Document Meilisearch env vars and local startup flow.	this doc, ops docs if needed	Hermes plan	Done

Meilisearch local flow:

docker run --rm -p 7700:7700 -e MEILI_MASTER_KEY=gochat_dev getmeili/meilisearch:latest
GOCHAT_SEARCH_ENGINE=meilisearch GOCHAT_SEARCH_HOST=http://localhost:7700 GOCHAT_SEARCH_API_KEY=gochat_dev go run ./cmd/reindex_search -types all

Search environment variables:

Variable	Default	Notes
GOCHAT_SEARCH_ENGINE	meilisearch	Use db only for explicit local fallback.
GOCHAT_SEARCH_HOST	http://localhost:7700	Meilisearch endpoint.
GOCHAT_SEARCH_API_KEY	empty	Set to Meilisearch master/search key when enabled.
GOCHAT_SEARCH_INDEX_PREFIX	gochat_	Prefixes indexes such as gochat_conversations.
GOCHAT_SEARCH_TIMEOUT_SECONDS	5	HTTP timeout for search/index requests.

Phase 2: Route And Controller Parity Audit

Status: planned.

Checklist:

• Dump Chatwoot routes from reference/chatwoot.
• Dump GoChat routes with cmd/dump_routes.
• Build a tracked route parity table covering method, path, controller, auth scope, request params, and response serializer.
• Prioritize frontend-critical routes used by the Chatwoot web app.
• Patch route names and wildcard params where Gin constraints require different internal names, while preserving external URLs.
• Add regression tests for route groups that previously conflicted.

Acceptance:

• Route gap report is generated and checked in.
• All currently implemented routes boot without panic.
• Missing frontend-critical routes have tickets or implementation tasks.

Tracking table:

ID	Task	Artifact	Status
P2.1	Generate Chatwoot route dump from reference/chatwoot.	docs/parity/chatwoot_routes.*	Todo
P2.2	Generate GoChat route dump with cmd/dump_routes.	docs/parity/gochat_routes.*	Todo
P2.3	Produce route parity table: method, path, controller/handler, auth, request params, serializer, status.	docs/parity/route_parity.md	Todo
P2.4	Mark frontend-critical gaps from Chatwoot web app route usage.	docs/parity/route_parity.md	Todo
P2.5	Convert existing placeholders/stubs into tracked feature tasks instead of hidden debt.	this doc and gap report	Todo
P2.6	Add route boot regression tests for Gin wildcard/param conflicts.	router and handler tests	Todo

Phase 3: Data And Serializer Parity

Status: planned.

Checklist:

• Compare key Chatwoot serializers/entities with Go response payloads.
• Align account, user, inbox, conversation, message, contact, company, team, label, canned response, campaign, help center, and notification payload shapes.
• Verify timestamps, IDs, enum strings, nested objects, pagination metadata, and error envelopes.
• Add fixture-driven tests for payload compatibility.

Acceptance:

• Chatwoot frontend can consume the payloads without adapter code.
• Serializer deviations are documented only where GoChat intentionally differs.

Frontend-critical API groups to audit first:

ID	Area	Scope	Status
P3.1	Auth/session/profile	Login, logout, current user, profile, availability, notification settings.	Todo
P3.2	Accounts/users/teams	Account settings, users, agents, teams, invitations, roles, permissions.	Todo
P3.3	Inboxes/channels	Inbox CRUD, assignable agents, avatars, channel config, business hours, widget config.	Todo
P3.4	Conversations/messages	List filters, status changes, assignment, labels, private notes, attachments, drafts, typing/read events.	Todo
P3.5	Contacts/companies	CRUD, merge, labels, notes, custom attributes, import/export, conversations relation.	Todo
P3.6	Labels/custom attributes/custom filters	Create/update/list behavior and exact response shapes.	Todo
P3.7	Notifications/reports/help center/campaigns	Frontend-visible payloads and pagination/error envelopes.	Todo

Serializer comparison rules:

• Compare against reference/chatwoot serializers/entities before changing Go responses.
• Prefer fixture-driven tests for exact JSON shape, enum strings, pagination metadata, and error envelopes.
• Preserve Chatwoot field names even if Go internal naming differs.

Phase 4: Enterprise Feature Completion

Status: planned.

Excluded:

• SSO
• SAML
• LDAP
• OIDC

Included checklist:

• SLA policies and SLA event tracking.
• Audit logs and admin-readable audit endpoints.
• Custom roles and permission checks.
• Agent capacity and assignment limits.
• Assignment policies and auto-assignment compatibility.
• Captain/Copilot assistant, custom tools, scenarios, documents, responses, and inbox bindings.
• CSAT survey response flow, metrics, filters, and review notes.
• Inbox limits and account/inbox usage enforcement.
• Automation rules, macros, execution logs, and action side effects.

Acceptance:

• Each included feature has routes, persistence, authorization, tests, and Chatwoot-compatible response behavior.
• Unsupported SSO family features are explicitly disabled or omitted without breaking frontend navigation for enabled features.

Enterprise tracking table:

ID	Feature	Existing Go surface	Required next work	Status
P4.1	SLA policies/events	internal/model/sla_policy.go, internal/model/sla_event.go, internal/service/sla_policy_service.go, internal/service/sla_event_service.go, internal/handler/api/v1/sla_policy_handler.go	Compare with Chatwoot SLA behavior; wire lifecycle events and breach tracking; add frontend payload tests.	Todo
P4.2	Audit logs	internal/model/audit.go, internal/service/audit_service.go, internal/repository/audit_repo.go, internal/handler/api/v1/audit_handler.go	Ensure every mutating enterprise/core action emits audit events and filters match Chatwoot.	Todo
P4.3	Custom roles/permissions	internal/model/custom_role.go, internal/service/custom_role_service.go, internal/middleware/role_check.go, internal/handler/api/v1/custom_role_handler.go	Align permission keys, inherited roles, authorization failures, and admin UX payloads.	Todo
P4.4	Agent capacity	internal/model/agent_capacity_policy.go, internal/service/agent_capacity_policy_service.go, internal/handler/api/v1/agent_capacity_handler.go, internal/autoassignment/*	Enforce capacity in assignment path and auto-assignment; add limit tests.	Todo
P4.5	Inbox limits	internal/model/inbox_limit.go, internal/service/inbox_limit_service.go, internal/repository/inbox_limit_repo.go, internal/handler/api/v1/inbox_limit_handler.go	Enforce account/inbox usage limits and frontend-compatible responses.	Todo
P4.6	Captain/Copilot	internal/model/captain_models.go, internal/model/copilot_models.go, internal/service/captain_*, internal/service/copilot_*, internal/handler/api/v1/captain_*, internal/handler/api/v1/copilot_handler.go	Complete assistant, tools, scenarios, documents, responses, inbox bindings, suggestions, and streaming compatibility.	Todo
P4.7	CSAT	internal/csat/*, internal/automation/csat_survey_*, internal/handler/api/v1/csat_*, internal/service/csat_metrics_service.go	Wire resolve-triggered survey send, public update flow, metrics, review notes, filters.	Todo
P4.8	Automation and macros	internal/automation/*, internal/handler/api/v1/automation_rule_handler.go, internal/handler/api/v1/macro_handler.go	Finish action side effects, execution logs, webhook/email transcript delivery, and rule trigger coverage.	Todo
P4.9	Assignment policies	internal/autoassignment/*, internal/automation/agent_bot_rule_listener.go	Match Chatwoot assignment policy behavior and availability/capacity rules.	Todo

Excluded tracking table:

Feature	Decision	Required handling
SSO	Excluded	Do not prioritize implementation. Existing surfaces should not block core frontend use.
SAML	Excluded	Do not expand beyond already present code unless needed to disable safely.
LDAP	Excluded	Omit from parity scope.
OIDC	Excluded	Omit from parity scope.

Phase 5: Background Jobs And Integrations

Status: planned.

Known hotspots:

• internal/worker/worker.go is still mostly placeholder.
• internal/automation/action_service.go has pending webhook/email transcript work.
• internal/automation/csat_survey_listener.go has pending CSAT enable/send behavior.
• internal/auth/webhook_registry.go has pending signature verification for Facebook/WhatsApp.
• internal/service/analytics_service.go has placeholder analytics paths.

Checklist:

• Map Chatwoot jobs/listeners to Go worker responsibilities.
• Implement durable job dispatch for automation, CSAT, notifications, webhooks, and search indexing.
• Add retry and failure logging for external calls.
• Add tests for job enqueueing and idempotency.

Acceptance:

• User-visible side effects do not depend on synchronous handler-only execution.
• Failed background work is observable and retryable.

Tracking table:

ID	Task	Current hotspot	Status
P5.1	Choose and wire durable job runner compatible with current Go stack.	internal/worker/worker.go, internal/channel/dispatcher.go	Todo
P5.2	Move search indexing into retryable jobs where Chatwoot uses callbacks/jobs.	search services and worker	Todo
P5.3	Implement automation webhook delivery with retry, timeout, and logs.	internal/automation/action_service.go	Todo
P5.4	Implement email transcript delivery once email infrastructure is available.	internal/automation/action_service.go	Todo
P5.5	Complete CSAT survey send listener and idempotency.	internal/automation/csat_survey_listener.go, internal/csat/listener.go	Todo
P5.6	Complete webhook signature verification for Facebook/WhatsApp/Twilio/provider paths.	internal/auth/webhook_registry.go, channel providers	Todo
P5.7	Replace placeholder analytics with real report builders/queries.	internal/service/analytics_service.go, reporting services	Todo

Phase 6: Core Product Placeholder Burn-down

Status: planned.

Purpose: route parity is not enough; existing handlers that return placeholder JSON must be converted into real Chatwoot-compatible behavior before the frontend can be reused directly.

Tracking table:

ID	Area	Known references	Required next work	Status
P6.1	Account APIs	docs/ROUTE_GAP_ANALYSIS.md, account handlers	Replace placeholder responses with repository-backed behavior and serializer tests.	Todo
P6.2	Contact APIs	docs/ROUTE_GAP_ANALYSIS.md, contact handlers/services	Implement show/create/update/list/search relations, labels, notes, merge.	Todo
P6.3	Conversation APIs	docs/ROUTE_GAP_ANALYSIS.md, conversation handlers/services	Implement frontend-critical filters, assignment, status, snooze, merge, bulk actions.	Todo
P6.4	Message APIs	docs/ROUTE_GAP_ANALYSIS.md, message handlers/services	Implement create/list/delete, private notes, attachments, source attribution, events.	Todo
P6.5	Inbox APIs	docs/ROUTE_GAP_ANALYSIS.md, inbox handlers/services	Implement CRUD, assignable agents, avatar, campaigns, channel settings, reset secret.	Todo
P6.6	Widget APIs	docs/ROUTE_GAP_ANALYSIS.md, widget/channel provider code	Implement init, contact, conversations, messages, cable token with Chatwoot widget contracts.	Todo
P6.7	Webhook ingress	internal/router/router.go, internal/handler/webhook/*, channel providers	Replace generic placeholder with provider-specific verified ingestion and dispatch.	Todo

Phase 7: Verification Harness

Status: planned.

Checklist:

• Add repeatable command to compare Chatwoot and GoChat route dumps.
• Add fixture-based JSON parity tests for major frontend endpoints.
• Add Meilisearch mock tests and optional integration test mode gated by env vars.
• Add frontend smoke path using reused Chatwoot frontend once backend boot flow is ready.
• Keep a generated gap report in docs/parity/ with date, reference commit, Go commit, and pass/fail summary.

Required verification before each feature commit:

env GOCACHE=/tmp/gochat-gocache GOMODCACHE=/tmp/gochat-gomodcache go test ./...
env GOCACHE=/tmp/gochat-gocache GOMODCACHE=/tmp/gochat-gomodcache go run ./cmd/dump_routes

Task Status Legend

• Todo: not started or only skeleton exists.
• Doing: implementation is underway in the current working tree.
• Review: code is implemented and needs parity/test verification.
• Done: committed, tested, and linked to the matching reference/chatwoot behavior.
• Blocked: needs a decision, missing dependency, or reference behavior cannot yet be reproduced.

Ongoing Tracking Rules

• Every parity task should name the matching file or behavior in reference/chatwoot.
• Every route change should keep cmd/dump_routes passing.
• Every phase must keep go test ./... green before moving on.
• Test-only route fixes should be limited to malformed tests. Production route changes must preserve external Chatwoot-compatible URLs.
• New search work must target Meilisearch first.
• Update this document in the same commit as each completed parity slice: change task status, add verification command output summary, and link the touched reference/chatwoot behavior.
• Do not count a feature as done because a model/handler exists; it is done only when route behavior, persistence, authorization, side effects, response shape, and tests are covered.
• Keep .codegraph/ and other generated local analysis artifacts out of product commits unless explicitly requested.

Progress Log

• 2026-06-04: Baseline stabilized and committed as 42cdab8 chore: stabilize chatwoot parity baseline; go test ./... passed and route dump reported TOTAL: 704.
• 2026-06-04: Phase 1 search foundation added: Meilisearch config/env defaults, SearchEngine contract, Meilisearch HTTP wrapper with bootstrap/settings, DB fallback adapter, document builders, reindex command, and no-live-Meilisearch tests. Verified go test ./... in unsandboxed mode because miniredis/httptest need local sockets; route dump still reports TOTAL: 704.
• 2026-06-04: Phase 1 indexing hooks wired for conversations, messages, contacts, companies, and articles. Create/update/delete paths now call the service-layer SearchIndexer boundary, bootstrap injects the Meilisearch-backed search service, and unit tests cover each entity hook path.