rogee/gochat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: record search live gate checkpoint

Browse Source
This commit is contained in:
rogee
2026-06-05 06:48:57 +08:00
parent f08c74381e
commit 5033cf17b0
1 changed files with 21 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CHATWOOT_PARITY_DEVELOPMENT_PLAN.md
+21 -20
View File
@@ -16,9 +16,9 @@ Build GoChat as a Go backend that can directly reuse the frontend from `referenc
## Current Baseline
- Latest implementation checkpoint: `a16c23c feat(search): align chatwoot search payloads`.
- Latest documentation checkpoint before this update: `178bc36 docs: record channel route cleanup checkpoint`.
- Worktree status at this planning checkpoint: clean after the matching implementation checkpoint; next active slice is B6 live Meilisearch gate and DB-fallback hardening.
- Latest implementation checkpoint: `f08c743 test(search): add meilisearch live gate`.
- Latest documentation checkpoint before this update: `f36512e docs: record search payload checkpoint`.
- Worktree status at this planning checkpoint: clean after the matching implementation checkpoint; next active slice is either running the optional live Meilisearch gate or starting B7 SLA/assignment capacity.
- `go test ./...` passes.
- Route dump succeeds with `TOTAL: 829` after adding Chatwoot-compatible nested AgentCapacityPolicy users and inbox-limit routes.
- Route parity artifacts now exist under `docs/parity/` and are generated by `cmd/route_parity`.
@@ -32,7 +32,7 @@ Build GoChat as a Go backend that can directly reuse the frontend from `referenc
| Phase | Name | Status | Blocking gaps |
| --- | --- | --- | --- |
| Phase 0 | Test and route baseline | Done | none |
| Phase 1 | Meilisearch search engine | Doing | B6 payload parity has a mocked Meilisearch checkpoint; live integration and DB-fallback hardening still need verification |
| Phase 1 | Meilisearch search engine | Review | B6 payload parity, optional live gate, and DB-fallback hardening are implemented; an actual live Meilisearch run is optional and environment-dependent |
| Phase 2 | Route and controller parity audit | Doing | Ruby/Bundler unavailable, so Chatwoot route extraction currently uses static `routes.rb` fallback |
| Phase 3 | Data and serializer parity | Doing | JSON fixture coverage is partial and still endpoint-family based |
| Phase 4 | Enterprise feature completion | Doing | excluded SSO family must stay out of scope; all other enterprise features remain included |
@@ -94,21 +94,20 @@ This ledger records the committed parity checkpoints that future slices should b
| `b197e54 feat(capacity): align chatwoot inbox capacity limits` | Completed B5.5c AgentCapacityPolicy/InboxCapacityLimit API/data parity: policy CRUD now returns raw Chatwoot payloads with Unix timestamps, `assigned_agent_count`, and `inbox_capacity_limits`; `assignment_logic` is optional/defaulted; nested policy users and inbox limits are registered; `InboxCapacityLimit` validates account scope, duplicate inbox assignment, and non-negative limits; account users can be assigned/unassigned to capacity policies. | `go test ./internal/service -run AgentCapacity -count=1`; `go test ./internal/handler/api/v1 -run AgentCapacity -count=1`; `go test ./internal/router -count=1`; `go test ./internal/handler/api/v1 -count=1`; sandboxed `go test ./...` failed on local socket restrictions; escalated `go test ./...` passed; route dump regenerated with `TOTAL: 829`; route parity is `267 exact, 7 parameter-compatible, 0 missing`; `git diff --check`. | Continue final B5.5 review for remaining channel-specific route response cleanup, then move to B6 Meilisearch live-shape review. |
| `f04a03b feat(channels): align channel route inbox payloads` | Completed the final B5.5 channel route-response cleanup: Email, Twilio SMS, and LINE dedicated channel create/get/update/list routes now return Chatwoot frontend-compatible raw inbox payloads or `{ payload: [...] }` lists instead of `{ channel }`, `{ channels }`, `{ channel, inbox }`, or success-message envelopes. Delete routes now return empty `200 OK`, and channel creation binds the dedicated channel ID/config back onto the inbox before serialization. | `go test ./internal/handler/api/v1 -run 'Test(Email\|Twilio\|LINE)Channel' -count=1`; `go test ./internal/service -run Inbox -count=1`; `go test ./internal/handler/api/v1 -count=1`; `go test ./...`; `git diff --check`. No route changes; route dump remains `TOTAL: 829`. | B5 inbox/channel API parity moves to Done for current frontend-critical scope; next active slice is B6 Meilisearch live-shape review. |
| `a16c23c feat(search): align chatwoot search payloads` | Advanced B6 search live-shape parity: global and entity search endpoints now return Chatwoot frontend `payload` envelopes, search filters accept Chatwoot `since`, `until`, and `from=contact:id/agent:id` params, message sender IDs are indexed/filterable in Meilisearch and DB fallback, SearchController default page size is aligned to 15, and Meilisearch hit maps are serialized into frontend message/contact/conversation/article shapes. | `go test ./internal/handler/api/v1 -run SearchHandler -count=1`; `go test ./internal/search -run 'SearchFilter\|Meili\|Engine' -count=1`; `go test ./internal/repository -run Search -count=1`; `go test ./internal/handler/api/v1 -count=1`; `go test ./internal/search -count=1`; `go test ./internal/repository -count=1`; sandboxed `go test ./...` failed on local socket restrictions; escalated `go test ./...` passed; `git diff --check`. No route changes; route dump remains `TOTAL: 829`. | B6.1 and B6.3 are Done; B6.2 is in Review with mocked Meilisearch coverage. Continue B6.4 live integration gate and B6.5 DB-fallback hardening before moving B6 to Review/Done. |
| `f08c743 test(search): add meilisearch live gate` | Completed B6 gate/hardening work: added an env-gated live Meilisearch test that bootstraps isolated indexes, indexes message/contact documents, verifies account-scoped searches and sender filters, rejects `search.engine=db` in release mode, and prevents `cmd/reindex_search` from silently running against the DB fallback. | `go test ./internal/config -count=1`; `go test ./cmd/reindex_search -count=1`; `go test ./internal/search -count=1`; `go test ./...`; `git diff --check`. Live gate is skipped unless `GOCHAT_LIVE_MEILI_HOST` is set. No route changes; route dump remains `TOTAL: 829`. | B6 moves to Review. Optional next verification is running `GOCHAT_LIVE_MEILI_HOST=http://localhost:7700 GOCHAT_LIVE_MEILI_API_KEY=... go test ./internal/search -run TestLiveMeiliSearchEngineIndexesAndSearchesChatwootShapes -count=1`; otherwise continue B7 SLA/assignment capacity. |
## Next Slice Contract
Completed implementation slice: B5.1-B5.5 now cover inbox serializer shape, Chatwoot frontend create/update binding, working-hours persistence, out-of-office behavior, inbox member assignment payload/mutation semantics, channel-specific config depth, AgentCapacityPolicy/InboxCapacityLimit API data contracts, and dedicated Email/Twilio/LINE channel route response shapes. B6 now has its first search payload checkpoint in `a16c23c`: Chatwoot search response envelopes, frontend query params, Meilisearch sender filters, and mocked hit serialization are covered. B3 and B4 remain in review for deeper side effects and browser validation.
Completed implementation slice: B5.1-B5.5 now cover inbox serializer shape, Chatwoot frontend create/update binding, working-hours persistence, out-of-office behavior, inbox member assignment payload/mutation semantics, channel-specific config depth, AgentCapacityPolicy/InboxCapacityLimit API data contracts, and dedicated Email/Twilio/LINE channel route response shapes. B6 is now in Review after `a16c23c` and `f08c743`: Chatwoot search response envelopes, frontend query params, Meilisearch sender filters, mocked hit serialization, env-gated live Meilisearch validation, release-mode DB fallback rejection, and reindex Meilisearch-only guard are covered. B3 and B4 remain in review for deeper side effects and browser validation.
Next implementation slice: finish B6.4 live Meilisearch gate and B6.5 DB-fallback hardening, then move to B7 SLA/assignment capacity unless a reused-frontend search smoke exposes a blocker.
Next implementation slice: start B7 SLA/assignment capacity unless an operator wants to run the optional live Meilisearch gate first.
| Step | Required result | Reference source | Verification |
| --- | --- | --- | --- |
| N1 | Keep B6 search payload checkpoint as the current baseline. | `SearchController`, `search.js`, `conversationSearch.js`, search Jbuilder views. | Done by `a16c23c`; response envelopes, Chatwoot params, sender filters, and mocked Meilisearch hit shape are covered. |
| N2 | Add the live Meilisearch integration gate. | `cmd/reindex_search`, `internal/search/engine_meili.go`, local Meilisearch env. | Env-gated test or documented command starts from indexed data, runs global/messages/contacts searches, and records result shape. |
| N3 | Harden and document DB fallback as development-only. | Search config defaults, `NewSearchEngine`, bootstrap wiring. | Config tests keep `meilisearch` as default; docs call DB fallback non-production and explain how to fail/alert when Meilisearch is unavailable. |
| N4 | Decide B6 close status after live gate. | B6 task board and Phase 1 checklist. | Mark B6 `Review` or `Done` only after live gate and fallback hardening are recorded; otherwise keep B6 `Doing`. |
| N5 | Start B7 SLA and assignment capacity. | Chatwoot enterprise SLA policies, assignment policies, capacity enforcement. | First B7 checkpoint must include routes/data/service tests before deeper timer/report behavior. |
| N6 | Update this tracker after every implementation checkpoint. | This document. | `git diff --check`; `go test ./...` for Go changes. |
| N1 | Keep B6 search payload/gate checkpoint as the current baseline. | `SearchController`, `search.js`, `conversationSearch.js`, search Jbuilder views, `cmd/reindex_search`. | Done by `a16c23c` and `f08c743`; response envelopes, Chatwoot params, sender filters, mocked hit shape, live gate, and fallback guards are covered. |
| N2 | Optionally run the live Meilisearch gate when a local Meilisearch instance is available. | `internal/search/engine_meili_live_test.go`. | `GOCHAT_LIVE_MEILI_HOST=http://localhost:7700 GOCHAT_LIVE_MEILI_API_KEY=... go test ./internal/search -run TestLiveMeiliSearchEngineIndexesAndSearchesChatwootShapes -count=1`. |
| N3 | Start B7 SLA and assignment capacity. | Chatwoot enterprise SLA policies, assignment policies, capacity enforcement. | First B7 checkpoint must include routes/data/service tests before deeper timer/report behavior. |
| N4 | Update this tracker after every implementation checkpoint. | This document. | `git diff --check`; `go test ./...` for Go changes. |
Current B2 profile checkpoint:
@@ -264,8 +263,8 @@ Active B6 task board:
| B6.1 | Inventory Chatwoot search frontend request/response consumers and current Go search routes. | `reference/chatwoot/app/javascript/dashboard/api/search.js`, `conversationSearch.js`, `SearchController`, `internal/handler/api/v1/search_handler.go`. | Done | Matrix below records global, contacts, conversations, messages, and articles params/payload fields. |
| B6.2 | Compare Meilisearch document fields and filters against frontend payload needs. | `internal/search/engine.go`, `engine_meili.go`, document builders, Chatwoot search views/entities. | Review | `a16c23c` adds mocked Meilisearch sender filter and hit-shape tests; live Meilisearch gate remains B6.4. |
| B6.3 | Tighten endpoint serializers for `/search`, `/search/contacts`, `/search/conversations`, `/search/messages`, and `/search/articles`. | `reference/chatwoot/app/controllers/api/v1/accounts/search_controller.rb`, dashboard search API specs. | Done | `SearchHandler` tests assert Chatwoot `payload` envelopes and Meilisearch hit serialization. |
| B6.4 | Add optional live Meilisearch integration gate. | Local Meilisearch flow in Phase 1 and `cmd/reindex_search`. | Todo | Env-gated test or documented command runs reindex plus representative searches against a live Meilisearch instance. |
| B6.5 | Document DB fallback as development-only and verify production config remains Meilisearch-first. | User decision ledger and search config. | Review | Existing config defaults to `meilisearch`; next step is a focused config/doc warning around DB fallback usage. |
| B6.4 | Add optional live Meilisearch integration gate. | Local Meilisearch flow in Phase 1 and `cmd/reindex_search`. | Done | `f08c743` adds `TestLiveMeiliSearchEngineIndexesAndSearchesChatwootShapes`, skipped unless `GOCHAT_LIVE_MEILI_HOST` is set. |
| B6.5 | Document DB fallback as development-only and verify production config remains Meilisearch-first. | User decision ledger and search config. | Done | `f08c743` rejects DB fallback in release mode and prevents `reindex_search` from using DB fallback. |
B6 request and payload matrix after `a16c23c`:
@@ -283,7 +282,8 @@ B6 current checkpoint:
- `search.ParseSearchFilter` now understands Chatwoot dashboard `since`/`until` Unix seconds and `from=contact:id`/`from=agent:id` sender filters while preserving legacy `date_from`/`date_to` and `sender_id` inputs.
- Meilisearch documents now store `sender_id`, index settings mark it filterable, and DB fallback applies the same sender ID filter for local tests/dev mode.
- Search handler pagination defaults to Chatwoot's 15 items when `per_page` is not supplied.
- Remaining B6 risk: live Meilisearch reindex/search has not been run in this checkpoint, and DB fallback needs an explicit production-safety warning/test before B6 can leave Doing.
- The optional live gate is implemented but not run by default. It requires `GOCHAT_LIVE_MEILI_HOST` and validates isolated live indexes, account-scoped contact/message search, and message sender filters.
- DB fallback remains available for explicit local development, but release config validation and `reindex_search` now refuse it so production parity stays Meilisearch-first.
Active B4 task board:
@@ -313,7 +313,7 @@ This is the ordered queue for the next implementation slices. Do not skip the ro
| Q2 | Patch route aliases discovered by Q1, especially Chatwoot widget/public paths such as `/api/v1/widget/...` versus existing `/widget/...`. | Phase 2 | Missing tracked routes return to zero or are explicitly documented with implementation tasks. | Done |
| Q3 | Add route boot regression coverage for wildcard conflict groups before expanding more Rails-style resources. | Phase 2 | Router tests cover nested dynamic resources that previously risked Gin conflicts. | Done |
| Q4 | Start serializer parity fixtures for auth/session, conversations/messages, contacts/companies, inboxes, notifications, and search. | Phase 3 | Each area has at least one reference fixture and Go response test. | Doing |
| Q5 | Review Meilisearch document shape and endpoint payloads against Chatwoot frontend consumers. | Phase 1 and Phase 3 | Search remains Meilisearch-first and payload mismatches are fixed or tracked. | Doing |
| Q5 | Review Meilisearch document shape and endpoint payloads against Chatwoot frontend consumers. | Phase 1 and Phase 3 | Search remains Meilisearch-first and payload mismatches are fixed or tracked. | Review |
| Q6 | Implement provider-specific webhook ingress for Chatwoot public webhook paths. | Phase 6 | Generic webhook placeholder no longer masks provider gaps; Telegram, LINE, SMS/Twilio, WhatsApp, Instagram/Twitter/TikTok routes resolve and verify like Chatwoot where supported. | Done |
| Q7 | Burn down enterprise gaps in this order: SLA, assignment policy and capacity, CSAT, automation/macros, Audit, CustomRole, InboxLimit, Captain/Copilot. | Phase 4 and Phase 5 | Each feature passes route, persistence, auth, side-effect, response, and test checks. | Todo |
| Q8 | Add frontend smoke harness using the reused Chatwoot frontend once core API flows boot end-to-end. | Phase 7 | Login, inbox list, conversation list/detail, message send, contact view, and widget init run without frontend adapters. | Todo |
@@ -337,7 +337,7 @@ These milestones are the tracking spine for the remaining Chatwoot frontend reus
| Milestone | Scope | Exit gate | Status |
| --- | --- | --- | --- |
| M0 | Test, route, and documentation baseline. | Clean worktree, `go test ./...`, route dump/parity artifacts current. | Done |
| M1 | Meilisearch-first search foundation. | Config, engine, indexing hooks, reindex command, mocked tests, and live-shape review tracked. | Doing |
| M1 | Meilisearch-first search foundation. | Config, engine, indexing hooks, reindex command, mocked tests, and live-shape review tracked. | Review |
| M2 | Route parity expansion for frontend-critical routes. | Tracked route set has zero missing routes and every new route group has router boot coverage. | Doing |
| M3 | Serializer parity for frontend API families. | Fixture tests cover auth/profile, accounts/users, inboxes, conversations/messages, contacts/companies, notifications, reports, widget/public, and search. | Doing |
| M4 | Core handler placeholder burn-down. | Account/contact/conversation/message/inbox/webhook handlers are repository-backed and no frontend-critical route returns placeholder JSON. | Doing |
@@ -356,7 +356,7 @@ Work proceeds top-down unless a failing test or frontend blocker forces a narrow
| B3 | Conversation/message serializer and behavior fixtures. | Chatwoot conversation/message controllers, entities, jobs. | Fixture tests for list/show/create/update/private notes/attachments/status/assignment. | Review |
| B4 | Contact/company behavior fixtures. | Chatwoot contact/company controllers, merge/import/export/notes/labels. | Fixture tests for CRUD/search/merge/relation/import-export shells and frontend CRM API smoke. | Review |
| B5 | Inbox/channel behavior fixtures. | Chatwoot inbox/channel controllers and channel models. | Fixture tests for inbox CRUD, settings, business hours, members, avatar, channel config. | Done |
| B6 | Meilisearch live-shape review. | Chatwoot frontend search usage and search controllers. | Meilisearch-backed response fixtures plus optional live integration gate. | Doing |
| B6 | Meilisearch live-shape review. | Chatwoot frontend search usage and search controllers. | Meilisearch-backed response fixtures plus optional live integration gate. | Review |
| B7 | SLA and assignment capacity. | Chatwoot enterprise SLA and assignment policy behavior. | Unit/integration tests for SLA state, breach, assignment capacity, policy selection. | Todo |
| B8 | CSAT account-side completion. | Chatwoot CSAT survey responses, reports, downloads, listeners. | Metrics/list/review/download/send idempotency tests. | Todo |
| B9 | Automation/macros durable side effects. | Chatwoot automation/macro services and jobs. | Action execution, logs, webhook/email transcript retry tests. | Todo |
@@ -443,11 +443,11 @@ Tracking table:
| P1.1 | Add `SearchConfig` with engine, host, API key, index prefix, env binding, and defaults. | `internal/config/config.go`, `configs/config.yaml`, config tests | `.hermes/plans/2025-05-24-global-search-meilisearch.md` | Done |
| P1.2 | Add stable engine contract for search, indexing, deletes, batch indexing, and close. | `internal/search/engine.go` | Chatwoot global/entity search behavior | Done |
| P1.3 | Implement Meilisearch engine wrapper, index naming, bootstrap, sortable/filterable/searchable settings. | `internal/search/engine_meili.go` | Chatwoot search models/services | Review |
| P1.4 | Keep existing DB search as explicit dev fallback only. Production config must prefer Meilisearch. | `internal/search/engine_db.go`, `internal/search/search_service.go` | User decision on Meilisearch | Done |
| P1.4 | Keep existing DB search as explicit dev fallback only. Production config must prefer Meilisearch. | `internal/search/engine_db.go`, `internal/search/search_service.go`, `internal/config/validator.go`, `cmd/reindex_search` | User decision on Meilisearch | Done |
| P1.5 | Define documents and serializers for conversations, messages, contacts, companies, articles, and help-center content. | `internal/search/engine.go` | `reference/chatwoot` models/serializers | Review |
| P1.6 | Wire create/update/delete hooks from entity services into async or synchronous indexing boundary. | `internal/service/*`, `internal/search/search_service.go`, `internal/app/bootstrap.go` | Chatwoot callbacks/jobs | Done |
| P1.7 | Add batch reindex command and account/entity filters. | `cmd/reindex_search` | Chatwoot reindex/search tasks | Done |
| P1.8 | Add mocked engine tests and service integration tests without requiring live Meilisearch. | `internal/search/engine_test.go`, `internal/config/config_test.go` | Existing test style | Done |
| P1.8 | Add mocked engine tests and service integration tests without requiring live Meilisearch. | `internal/search/engine_test.go`, `internal/search/engine_meili_live_test.go`, `internal/config/config_test.go` | Existing test style | Done |
| P1.9 | Document Meilisearch env vars and local startup flow. | this doc, ops docs if needed | Hermes plan | Done |
Meilisearch local flow:
@@ -876,3 +876,4 @@ Verification milestone gates:
- 2026-06-05: B5.5c AgentCapacityPolicy/InboxCapacityLimit checkpoint committed as `b197e54 feat(capacity): align chatwoot inbox capacity limits`; policy CRUD now returns raw Chatwoot serializers, nested policy users and inbox limits are implemented, `account_users.agent_capacity_policy_id` is persisted, duplicate/wrong-account/non-negative limit checks are covered, route dump is `TOTAL: 829`, tracked route parity is `267 exact, 7 parameter-compatible, 0 missing`, focused service/handler/router tests passed, sandboxed full tests failed only on socket restrictions, escalated full `go test ./...` passed, and `git diff --check` passed.
- 2026-06-05: B5.5 final channel route-response cleanup committed as `f04a03b feat(channels): align channel route inbox payloads`; Email, Twilio SMS, and LINE dedicated channel routes now return raw Chatwoot-compatible inbox payloads for create/get/update, `{ payload: [...] }` for list, and empty `200 OK` for delete. `InboxService.BindChannel` keeps dedicated channel IDs/configs reflected in inbox serialization. Focused Email/Twilio/LINE channel tests, service inbox tests, handler package tests, full `go test ./...`, and `git diff --check` passed. Route dump unchanged at `TOTAL: 829`; next active slice is B6 Meilisearch live-shape review.
- 2026-06-05: B6 search payload checkpoint committed as `a16c23c feat(search): align chatwoot search payloads`; global/entity search endpoints now return Chatwoot `payload` envelopes, default to 15 results, accept `since`/`until`/`from` frontend params, serialize model and Meilisearch-hit data into frontend-ready conversation/contact/message/article shapes, and filter message sender IDs through both Meilisearch and DB fallback. Focused handler/search/repository tests, handler/search/repository package tests, escalated full `go test ./...`, and `git diff --check` passed. Route dump unchanged at `TOTAL: 829`; B6.4 live Meilisearch gate and B6.5 fallback hardening remain active.
- 2026-06-05: B6 live-gate/fallback checkpoint committed as `f08c743 test(search): add meilisearch live gate`; added an env-gated live Meilisearch test for isolated bootstrap/index/search of contacts and messages, kept it skipped unless `GOCHAT_LIVE_MEILI_HOST` is set, rejected `search.engine=db` in release validation, and made `cmd/reindex_search` fail fast if configured with DB fallback. Focused config/reindex/search tests, full `go test ./...`, and `git diff --check` passed. B6 moves to Review; next implementation slice is B7 SLA/assignment capacity unless the optional live gate is explicitly run first.