feat(search): index conversation bulk actions

This commit is contained in:
2026-06-07 14:29:39 +08:00
parent af624f7728
commit 4ba1e6e847
4 changed files with 83 additions and 7 deletions
+10 -6
View File
@@ -49,11 +49,11 @@ Hermes task landing checklist:
## Current Baseline
- Current tracking checkpoint: 2026-06-07 P5.3c contact label search indexing, prepared as `feat(search): index contact labels`.
- Latest implementation checkpoint: this checkpoint, prepared as `feat(search): index contact labels`.
- Latest documentation/tooling checkpoint: this tracker update records Meilisearch contact label document fields, contact-label mutation indexing hooks, and contact bulk-action search index fan-out.
- Current tracking checkpoint: 2026-06-07 P5.3d conversation bulk-action search indexing, prepared as `feat(search): index conversation bulk actions`.
- Latest implementation checkpoint: this checkpoint, prepared as `feat(search): index conversation bulk actions`.
- Latest documentation/tooling checkpoint: this tracker update records Meilisearch conversation reindex fan-out after account bulk-action status, assignment, snooze, and label mutations.
- Plan landing status: complete for the current known Hermes plans and user-confirmed scope. Future work should update this file directly instead of opening a parallel tracker.
- Worktree status at this implementation checkpoint: contact search documents now carry label arrays for Meilisearch filter parity, durable contact indexing reloads current label assignments before replay, `UpdateLabels` triggers contact reindexing, and contact bulk-action label/delete jobs enqueue search index/delete follow-up jobs. This retains P3.103 contact bulk-action parity, P3.102 enterprise account route parity, P6.1 webhook placeholder burn-down, and prior checkpoints. Live API/browser/enterprise smoke still needs the full PostgreSQL/Redis/Meilisearch/GoChat/Vite/Chrome stack.
- Worktree status at this implementation checkpoint: account conversation bulk actions now enqueue durable search reindex follow-ups for updated conversations, so Meilisearch documents stay fresh after bulk status/team/assignee/snooze/label mutations. This retains P5.3c contact label search indexing, P3.103 contact bulk-action parity, P3.102 enterprise account route parity, P6.1 webhook placeholder burn-down, and prior checkpoints. Live API/browser/enterprise smoke still needs the full PostgreSQL/Redis/Meilisearch/GoChat/Vite/Chrome stack.
- Next executable implementation checkpoint: continue Phase 2/3 drift audit for the next reused-frontend mismatch, or run B12 live smoke when the full PostgreSQL/Redis/Meilisearch/GoChat/Vite/Chrome stack is available. Re-run Phase 6 placeholder audit after future route/smoke changes.
- `go test ./...` passes when run outside the restricted socket sandbox for the latest implementation baseline; the latest docs/tooling checkpoint verified `scripts/parity_frontend_smoke.sh --check` with workspace-local temp/cache dirs after `/tmp` was full.
- Route dump succeeds with `972` registered routes after enterprise account route tracking.
@@ -156,6 +156,7 @@ This table is the shortest authoritative handoff view. If an older lower section
| Priority | Workstream | Current state | Next checkpoint | Commit close rule |
| --- | --- | --- | --- | --- |
| 0 | P5.3d conversation bulk-action search indexing | Implemented for Meilisearch conversation bulk-action parity: `conversation:bulk_action` now reindexes updated account conversations after status, assignee, team, snooze, or label mutations, and the app registers a search-aware bulk-action runner after the durable search indexer is wired. | Keep in Review; reopen from live Meilisearch gate, conversation bulk-action search smoke, or a fresh mutation path that changes searchable conversation fields without indexing. | Focused conversation bulk-action search test passed; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. No route artifacts change. |
| 0 | P5.3c contact label search indexing | Implemented for Meilisearch contact-label parity: contact search documents now include `labels`, durable `search:index` contact replay loads current `contact_labels`/`tags`, direct contact label replacement reindexes the contact, and contact bulk-action label/delete workers enqueue search index/delete follow-ups after account-scoped side effects. | Keep in Review; reopen from live Meilisearch gate, contact label search smoke, or a fresh mutation path that changes contact labels without indexing. | Focused service/search tests passed; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. No route artifacts change. |
| 0 | P3.103 contact bulk actions | Implemented for reused CRM contact index actions: `POST /api/v1/accounts/:account_id/bulk_actions` now normalizes Chatwoot `type`, accepts Contact payloads without requiring `action_name`, enqueues durable `contact:bulk_action` jobs on the `medium` queue, returns empty `200 OK`, applies label add/remove account-scoped through `contact_labels`/`tags`, soft-deletes only current-account selected contacts, and treats unknown contact operations as no-op success like `Contacts::BulkActionService`. | Keep in Review; reopen from B12 contacts smoke or fresh reference evidence for exact Pundit authorization, label serializer, deleted-association cleanup, or notification side effects beyond the inspected controller/service/frontend contract. | Focused BulkActionHandler tests passed; focused conversation maintenance contact bulk-action worker tests passed; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. No route artifacts change. |
| 0 | P3.102 enterprise account limits and billing routes | Implemented for reused enterprise account frontend calls: `GET/POST /enterprise/api/v1/accounts/:account_id/{limits,checkout,subscription,toggle_deletion,topup_checkout}` are now registered and tracked from `routes.rb:523-527`; `limits` returns Chatwoot-shaped usage data for agents, Captain documents/responses, and default-plan conversation/non-web-inbox counts; `toggle_deletion` mutates `marked_for_deletion_at` and `marked_for_deletion_reason`; `subscription` persists the `is_creating_customer` guard when no Stripe customer exists; checkout/top-up return explicit billing-provider errors instead of missing routes. | Keep in Review; reopen from B12 enterprise account smoke or fresh reference evidence for actual Stripe session creation, cloud-env gating, plan-config defaults, or account deletion notification/cancellation jobs beyond the local persisted boundary. | Focused EnterpriseAccountHandler tests passed; route dump/parity artifacts regenerated to `TOTAL: 972` and `435 exact, 0 method-compatible, 9 parameter-compatible, 0 missing out of 444`; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. |
@@ -393,6 +394,7 @@ This ledger records the committed parity checkpoints that future slices should b
| Commit | Scope | Verification summary | Follow-up state |
| --- | --- | --- | --- |
| `feat(search): index conversation bulk actions` | Advances P5.3/B6 for mandatory Meilisearch conversation bulk-action parity. Account conversation bulk-action worker replay now reindexes every updated conversation after bulk status/team/assignee/snooze/label changes, while the app registers the search-aware runner after durable search indexing is wired. | Focused conversation bulk-action search test passed; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. No route artifacts change. | Move P5.3d to Review; continue Phase 2/3 drift audit, Phase 6 placeholder audit, or B12 live smoke. |
| `feat(search): index contact labels` | Advances P5.3/B6 for mandatory Meilisearch contact label parity. Contact documents now carry label arrays, durable contact index replay preloads current label assignments, `ContactService.UpdateLabels` reindexes contacts after replacement, and contact bulk-action add/remove/delete jobs enqueue search index/delete follow-up jobs through the durable indexer. | Focused service/search tests passed; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. No route artifacts change. | Move P5.3c to Review; continue Phase 2/3 drift audit, Phase 6 placeholder audit, or B12 live smoke. |
| `feat(crm): queue contact bulk actions` | Advances P3.103/P5.12 with Chatwoot contact bulk-action parity. GoChat now accepts reused frontend Contact bulk payloads for label add/remove and delete, normalizes `type` like the Rails controller, enqueues `contact:bulk_action` on the `medium` queue when a WorkerPool is configured, returns empty `200 OK`, and replays account-scoped label/delete side effects through durable workers. | Focused BulkActionHandler tests and conversation maintenance worker contact bulk-action tests passed; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. No route artifacts change. | Move P3.103 to Review; continue Phase 2/3 drift audit, Phase 6 placeholder audit, or B12 live smoke. |
| `fix(routes): align dashboard app ids` | Advances P3.100 with exact Chatwoot dashboard app member route parameter parity. GoChat now registers `GET/PATCH/PUT/DELETE /api/v1/accounts/:account_id/dashboard_apps/:id`, keeps handler compatibility with legacy local `:dashboard_app_id`, and regenerates route parity artifacts. | Focused DashboardAppHandler and router tests passed; route dump/parity regenerated to `TOTAL: 967` and `425 exact, 0 method-compatible, 14 parameter-compatible, 0 missing out of 439`; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. | Move P3.100 to Review; continue reducing remaining parameter-compatible rows or run B12 live smoke. |
@@ -2117,7 +2119,7 @@ Tracking table:
| --- | --- | --- | --- | --- | --- |
| P5.1 | Implement durable worker core and job model. | `reference/chatwoot/app/jobs/application_job.rb`, `mutex_application_job.rb` | `internal/worker/worker.go`, `internal/model/background_job.go`, `migrations/000026_add_background_jobs.*.sql` | Job table, worker persistence API, enqueue API, worker loop, retry/backoff, scheduled jobs, mutex/idempotency keys, dead-letter state, and restart pickup tests exist. | Review by `feat(worker): add durable background jobs` |
| P5.2 | Route async dispatcher events through durable jobs. | `event_dispatcher_job.rb`, Chatwoot async dispatcher listeners | `internal/dispatch/dispatcher.go`, `internal/channel/dispatcher.go` | Heavy listeners can enqueue durable jobs without changing sync listener behavior; tests cover sync vs async routing and replay. | Review by `feat(dispatch): queue async events durably` |
| P5.3 | Move Meilisearch indexing and reindex fan-out into retryable jobs. | Meilisearch plan plus Chatwoot callbacks/jobs that index searchable records | search services, contact/company/conversation indexing hooks | Create/update/delete/label indexing survives handler success, retries on Meilisearch failure, and optional live Meilisearch gate remains green. | Review by `feat(search): queue index updates durably` and `feat(search): index contact labels` |
| P5.3 | Move Meilisearch indexing and reindex fan-out into retryable jobs. | Meilisearch plan plus Chatwoot callbacks/jobs that index searchable records | search services, contact/company/conversation indexing hooks | Create/update/delete/label/bulk-action indexing survives handler success, retries on Meilisearch failure, and optional live Meilisearch gate remains green. | Review by `feat(search): queue index updates durably`, `feat(search): index contact labels`, and `feat(search): index conversation bulk actions` |
| P5.4 | Queue automation webhook and transcript delivery. | `webhook_job.rb`, automation action execution services | `internal/automation/action_delivery.go`, `internal/automation/action_service.go` | Existing timeout/retry fakeable delivery is invoked by durable jobs; logs preserve attempt metadata and idempotency. | Review by `feat(automation): queue external action deliveries` |
| P5.5 | Queue delayed automation actions and macro execution. | `trigger_scheduled_items_job.rb`, `macros_execution_job.rb` | automation rule listener, macro service | Delayed actions execute after schedule time if reference params exist; macro execute supports multi-conversation job fan-out, and repeated workers do not duplicate side effects. | Review by `feat(automation): queue macro and csat jobs` and `feat(automation): close delayed action parity`; current reference exposes no delayed automation action params |
| P5.6 | Queue CSAT survey sends and channel-specific templates. | CSAT listener/services, WhatsApp/Twilio template services/jobs | `internal/automation/csat_survey_listener.go`, `internal/service/csat_template_service.go`, channel send services | Resolve-triggered CSAT send is durable; WhatsApp/Twilio template delivery and failure states are fakeable and observable. | Review by `feat(automation): queue macro and csat jobs` and `feat(csat): queue channel templates` |
@@ -2168,10 +2170,11 @@ P5.3 current checkpoint:
- Worker replay reloads each account-scoped record before indexing so Meilisearch receives current database state instead of stale request-time payloads.
- Contact replay now also reloads current `contact_labels`/`tags` so Meilisearch contact documents carry the same label filter field used by reused CRM contact/search flows.
- Contact label replacement and contact bulk-action label/delete jobs now reindex or delete contact documents after the label side effect, keeping Meilisearch label filters fresh.
- Conversation bulk-action replay now reindexes every updated conversation after bulk status, assignment, team, snooze, or label mutations so conversation search documents do not stale after reused inbox bulk actions.
- Missing records during an index replay are converted into delegate delete calls, which keeps delayed create/update jobs from resurrecting documents after a database delete.
- Search read paths remain Meilisearch-first through `searchService`; contact and company services still use the live `SearchService` reader instead of the durable wrapper.
- No-worker construction still falls back to synchronous indexing for focused tests and development paths that do not start the durable worker.
- Remaining P5.3 work is limited to explicit reindex fan-out/CLI scheduling and optional live Meilisearch environment gates; normal service-layer writes and contact-label writes are now durable.
- Remaining P5.3 work is limited to explicit reindex fan-out/CLI scheduling and optional live Meilisearch environment gates; normal service-layer writes, contact-label writes, and account bulk-action conversation writes are now durable.
P5.3 verification:
@@ -2796,3 +2799,4 @@ Verification milestone gates:
- 2026-06-07: P3.102 enterprise account limits checkpoint prepared as `feat(enterprise): align account limits API`; audited Chatwoot enterprise `AccountsController#limits/#toggle_deletion/#subscription/#checkout/#topup_checkout`, `BillingHelper`, `Enterprise::Account::PlanUsageAndLimits`, reused dashboard `api/enterprise/account.js`, and `routes.rb:523-527`. GoChat now registers the enterprise account route family under `/enterprise/api/v1/accounts/:account_id`, returns Chatwoot-shaped account limit payloads, persists scheduled deletion custom attributes, records subscription customer-creation guards, and exposes explicit local billing-provider errors for checkout/top-up paths. Focused EnterpriseAccountHandler tests passed; route dump/parity regenerated to `TOTAL: 972` and `435 exact, 0 method-compatible, 9 parameter-compatible, 0 missing out of 444`; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed.
- 2026-06-07: P3.103 contact bulk-action checkpoint prepared as `feat(crm): queue contact bulk actions`; audited Chatwoot `BulkActionsController`, `Contacts::BulkActionJob`, `Contacts::BulkActionService`, reused dashboard `api/bulkActions.js`, and `ContactsIndex.vue` label/delete callers. GoChat account bulk actions now accept Contact payloads for label add/remove and delete, normalize lower-case type values, enqueue durable `contact:bulk_action` jobs on the `medium` queue, return empty `200 OK`, replay account-scoped contact label add/remove through tags/contact_labels, soft-delete only selected current-account contacts, and preserve no-op success for unknown contact bulk payloads. Focused BulkActionHandler and conversation maintenance worker contact bulk-action tests passed; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. No route artifacts change.
- 2026-06-07: P5.3c contact label search-index checkpoint prepared as `feat(search): index contact labels`; audited Meilisearch contact document/filter behavior, contact label endpoints, and the new contact bulk-action mutation path. GoChat contact search documents now include `labels`, durable contact index replay preloads current contact labels from `contact_labels`/`tags`, direct contact label replacement reindexes the contact, and contact bulk-action label/delete jobs enqueue search index/delete follow-ups through the durable search indexer. Focused service/search tests passed; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. No route artifacts change.
- 2026-06-07: P5.3d conversation bulk-action search-index checkpoint prepared as `feat(search): index conversation bulk actions`; audited the Chatwoot account `BulkActionsJob` mutation path and the P5.3 Meilisearch durability contract. GoChat conversation bulk-action worker replay now reindexes every updated account-scoped conversation after bulk status, assignee, team, snooze, or label updates, and bootstrap registers the search-aware bulk-action runner after the durable search indexer is wired. Focused conversation bulk-action search test passed; full `go test ./...` passed outside the restricted socket sandbox; `git diff --check` passed. No route artifacts change.
+1 -1
View File
@@ -679,7 +679,7 @@ func Bootstrap(env string) (*App, error) {
}
searchService := search.NewSearchServiceWithEngine(searchEngine, searchRepo)
searchIndexer := service.NewDurableSearchIndexer(db, workerPool, searchService)
service.RegisterContactBulkActionSearchIndexer(workerPool, db, searchIndexer)
service.RegisterBulkActionSearchIndexer(workerPool, db, searchIndexer)
conversationService.SetSearchIndexer(searchIndexer)
messageService.SetSearchIndexer(searchIndexer)
contactService.SetSearchIndexer(searchIndexer)
@@ -94,10 +94,15 @@ func RegisterConversationMaintenanceJobs(wp *worker.WorkerPool, db *gorm.DB) {
}
func RegisterContactBulkActionSearchIndexer(wp *worker.WorkerPool, db *gorm.DB, indexer SearchIndexer) {
RegisterBulkActionSearchIndexer(wp, db, indexer)
}
func RegisterBulkActionSearchIndexer(wp *worker.WorkerPool, db *gorm.DB, indexer SearchIndexer) {
if wp == nil || db == nil {
return
}
runner := &conversationMaintenanceRunner{wp: wp, db: db, now: time.Now, searchIndexer: indexer}
wp.Register(TaskTypeConversationBulkAction, runner.performConversationBulkAction)
wp.Register(TaskTypeContactBulkAction, runner.performContactBulkAction)
}
@@ -361,6 +366,7 @@ func (r *conversationMaintenanceRunner) performConversationBulkAction(ctx contex
Find(&conversations).Error; err != nil {
return fmt.Errorf("load bulk action conversations: %w", err)
}
updatedConversationIDs := make([]uint, 0, len(conversations))
for i := range conversations {
conversation := conversations[i]
updates := map[string]any{}
@@ -395,6 +401,29 @@ func (r *conversationMaintenanceRunner) performConversationBulkAction(ctx contex
if err := r.db.WithContext(ctx).Model(&model.Conversation{}).Where("id = ?", conversation.ID).Updates(updates).Error; err != nil {
return fmt.Errorf("bulk update conversation %d: %w", conversation.ID, err)
}
updatedConversationIDs = append(updatedConversationIDs, conversation.ID)
}
return r.indexConversationSearchDocuments(ctx, payload.AccountID, updatedConversationIDs)
}
func (r *conversationMaintenanceRunner) indexConversationSearchDocuments(ctx context.Context, accountID uint, conversationIDs []uint) error {
if r.searchIndexer == nil || len(conversationIDs) == 0 {
return nil
}
var conversations []model.Conversation
if err := r.db.WithContext(ctx).
Preload("Contact").
Preload("Inbox").
Preload("Assignee").
Preload("Messages", func(db *gorm.DB) *gorm.DB { return db.Order("created_at ASC, id ASC") }).
Where("account_id = ? AND id IN ?", accountID, conversationIDs).
Find(&conversations).Error; err != nil {
return err
}
for i := range conversations {
if err := r.searchIndexer.IndexConversation(ctx, &conversations[i]); err != nil {
return err
}
}
return nil
}
@@ -318,6 +318,49 @@ func TestConversationMaintenanceJobsConversationBulkAction(t *testing.T) {
}
}
func TestConversationMaintenanceJobsConversationBulkActionQueuesSearchIndex(t *testing.T) {
now := time.Date(2026, 6, 6, 0, 20, 0, 0, time.UTC)
db := setupServiceTestDB(t)
wp := worker.NewWorkerPoolWithOptions(db, worker.WithNow(func() time.Time { return now }))
registerConversationMaintenanceJobsWithNow(wp, db, func() time.Time { return now })
delegate := &recordingDurableSearchIndexer{}
searchIndexer := NewDurableSearchIndexer(db, wp, delegate)
RegisterBulkActionSearchIndexer(wp, db, searchIndexer)
account := createTestAccount(t, db)
inbox := createTestInbox(t, db, account.ID, "web_widget")
contact := createTestContact(t, db, account.ID)
conversation := createTestConversation(t, db, account.ID, inbox.ID, contact.ID)
displayID := uint(501)
if err := db.Model(conversation).Updates(map[string]any{"display_id": displayID, "labels": "old"}).Error; err != nil {
t.Fatalf("set display id: %v", err)
}
status := string(model.ConversationStatusResolved)
_, err := EnqueueConversationBulkAction(context.Background(), wp, account.ID, 42, ConversationBulkActionParams{
Type: "Conversation",
IDs: []uint{displayID},
Fields: ConversationBulkActionFields{Status: &status},
Labels: ConversationBulkActionLabels{Add: []string{"urgent"}, Remove: []string{"old"}},
})
if err != nil {
t.Fatalf("enqueue conversation bulk action: %v", err)
}
processRequiredJob(t, wp, "conversation bulk action")
processRequiredJob(t, wp, "conversation search index")
if len(delegate.indexedConversations) != 1 {
t.Fatalf("expected one indexed conversation, got %#v", delegate.indexedConversations)
}
indexed := delegate.indexedConversations[0]
if indexed.ID != conversation.ID || indexed.Status != status || indexed.Labels != "urgent" {
t.Fatalf("expected indexed updated conversation, got id=%d status=%s labels=%q", indexed.ID, indexed.Status, indexed.Labels)
}
if indexed.Contact == nil || indexed.Inbox == nil {
t.Fatalf("expected indexed conversation relations, got contact=%#v inbox=%#v", indexed.Contact, indexed.Inbox)
}
}
func TestConversationMaintenanceJobsContactBulkAction(t *testing.T) {
now := time.Date(2026, 6, 6, 0, 0, 0, 0, time.UTC)
db := setupServiceTestDB(t)