rogee/gochat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(reports): align summary aggregates

Browse Source
This commit is contained in:
rogee
2026-06-07 00:44:48 +08:00
parent 5333a356e9
commit ea56eb16c2
6 changed files with 261 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CHATWOOT_PARITY_DEVELOPMENT_PLAN.md
+7 -4
View File
@@ -49,11 +49,11 @@ Hermes task landing checklist:
## Current Baseline
- Current tracking checkpoint: 2026-06-07 documentation consolidation checkpoint, prepared as `docs: consolidate parity execution tracker`.
- Latest implementation checkpoint: committed baseline `aa159f6 feat(reports): align bot count strategies`.
- Latest documentation/tooling checkpoint: this tracker update records the committed P3.45 report bot raw-count strategy parity plus the P3.44/P3.43/P3.42/P3.41/P3.40 reports baseline, lands the known Hermes plan families into this document, and keeps this file as the active follow-up plan that supersedes `.hermes/plans/*`.
- Current tracking checkpoint: 2026-06-07 report summary aggregate parity checkpoint, prepared as `feat(reports): align summary aggregates`.
- Latest implementation checkpoint: this checkpoint, prepared as `feat(reports): align summary aggregates`.
- Latest documentation/tooling checkpoint: this tracker update records P3.46 report summary aggregate parity plus the committed P3.45/P3.44/P3.43/P3.42/P3.41/P3.40 reports baseline, lands the known Hermes plan families into this document, and keeps this file as the active follow-up plan that supersedes `.hermes/plans/*`.
- Plan landing status: complete for the current known Hermes plans and user-confirmed scope. Future work should update this file directly instead of opening a parallel tracker.
- Worktree status at this implementation checkpoint: report bot metric count strategies from `reference/chatwoot/app/services/reports/raw_data_source.rb`, `ReportMetricRegistry`, and `spec/builders/v2/reports/timeseries/report_builder_spec.rb` are aligned for v2 report timeseries. `bot_handoffs_count` now counts distinct non-null `conversation_id` values per bucket, and `bot_resolutions_count` now excludes conversations that also have a bot handoff in the same scoped range before bucketing. P3.44 live additive rollup parity, P3.43 rollup/backfill timezone parity, P3.42 timeseries timezone parity, P3.41 CSV/download parity, and P3.40 raw JSON/date parity for report, live-report, and summary-report endpoints remain implemented. Profile MFA, Facebook callbacks, account WhatsApp calls, onboarding, notification scoping, API v2 year-in-review, and WhatsApp calling toggle parity remain implemented from previous checkpoints. Live API/browser/enterprise smoke still needs the full PostgreSQL/Redis/Meilisearch/GoChat/Vite/Chrome stack.
- Worktree status at this implementation checkpoint: API v2 report summary responses now match inspected Chatwoot `Api::V2::Accounts::ReportsController#summary/#bot_summary`, `V2::Reports::Conversations::MetricBuilder`, `Reports::RawDataSource#aggregate`, and `ReportMetricRegistry` behavior. `/reports/summary` returns raw Chatwoot metric keys plus `previous`, scoped by current/previous ranges, report type/id, and business-hours average selection. `/reports/bot_summary` uses aggregate raw count strategies: `bot_handoffs_count` counts distinct non-null conversations, while `bot_resolutions_count` counts event rows after excluding conversations with bot handoffs in the same scoped range. P3.45 report bot metric timeseries count strategies, P3.44 live additive rollup parity, P3.43 rollup/backfill timezone parity, P3.42 timeseries timezone parity, P3.41 CSV/download parity, and P3.40 raw JSON/date parity for report, live-report, and summary-report endpoints remain implemented. Profile MFA, Facebook callbacks, account WhatsApp calls, onboarding, notification scoping, API v2 year-in-review, and WhatsApp calling toggle parity remain implemented from previous checkpoints. Live API/browser/enterprise smoke still needs the full PostgreSQL/Redis/Meilisearch/GoChat/Vite/Chrome stack.
- Next executable implementation checkpoint: continue Phase 2/3 drift audit for deeper report data-source/metric-builder drift or the next reused-frontend mismatch, Phase 6 placeholder audit, or B12 live smoke from fresh reference/smoke evidence.
- `go test ./...` passes when run outside the restricted socket sandbox; focused Facebook callback handler/router/route-parity tests pass in the sandbox.
- Route dump succeeds with `967` registered routes after profile MFA route tracking.
@@ -156,6 +156,7 @@ This table is the shortest authoritative handoff view. If an older lower section
| Priority | Workstream | Current state | Next checkpoint | Commit close rule |
| --- | --- | --- | --- | --- |
| 0 | P3.46 report summary aggregate parity | Implemented for reused v2 reports summary clients: GoChat now follows inspected Chatwoot `Api::V2::Accounts::ReportsController#build_summary`, `V2::Reports::Conversations::MetricBuilder#summary/#bot_summary`, `Reports::RawDataSource#aggregate`, and `Reports::ReportMetricRegistry`. `/reports/summary` returns raw Chatwoot metric keys with a `previous` object instead of the local `{ metrics: [...] }` rollup envelope, uses current/previous Unix-second ranges, honors report `type/id` filters, and switches average values to `value_in_business_hours` when requested. `/reports/bot_summary` now applies aggregate raw count strategies: handoffs are distinct non-null conversations, and bot resolutions count event rows after excluding handoff conversations in the same scoped range. | Keep in Review; reopen from B12 reports summary smoke, fresh reference evidence for rollup-backed `Reports::DataSource.for`, invalid timezone/range controller behavior, or summary dimension scope drift for team/label/report-type filters. | Focused Analytics service bot-summary tests, Analytics handler summary tests, escalated full `go test ./...`, and `git diff --check` must pass. No route artifacts change. |
| 0 | P3.45 report bot raw-count strategy parity | Implemented for reused v2 reports chart clients: GoChat now follows inspected Chatwoot `Reports::RawDataSource#reporting_event_count_scope`, `Reports::ReportMetricRegistry`, and v2 timeseries specs for bot metrics. `bot_handoffs_count` uses the `distinct_conversation` strategy and ignores nil conversation IDs, so duplicate handoff rows for the same conversation count once per bucket. `bot_resolutions_count` uses the `exclude_bot_handoffs` strategy and removes conversations that also have a bot handoff in the same scoped range before timeseries bucketing. Dimension scopes reuse the existing account/inbox/agent/team/label filters. | Keep in Review; reopen from B12 reports smoke or fresh reference evidence for aggregate-value endpoints beyond the current timeseries route, data-source scope behavior that differs for team/label dimensions, or future `ReportMetricRegistry` count strategies. | Focused Analytics service bot-timeseries tests, full `go test ./...`, and `git diff --check` must pass. No route artifacts change. |
| 0 | P3.44 reporting live additive rollup parity | Implemented for live reporting-event freshness: GoChat now follows inspected Chatwoot `ReportingEventListener#safe_rollup`, `ReportingEvents::RollupService#upsert_rollups`, and `ReportingEvents::EventMetricRegistry` behavior when `AnalyticsService.RecordEvent` persists a raw event. Valid account `reporting_timezone` gates the live write, event `created_at` determines the local rollup date, account/agent/inbox dimensions are emitted only when present, known event metrics expand to Chatwoot rollup metrics, unknown raw events emit no rollup rows, and conflicts add incoming `count`, `sum_value`, and `sum_value_business_hours` to the existing unique row. | Keep in Review; reopen from B12 reports smoke or fresh reference evidence for dispatcher/listener event creation paths that bypass `AnalyticsService.RecordEvent`, Sidekiq retry semantics beyond swallowed live-rollup errors, or report metric registry additions beyond the inspected reference event registry. | Focused Reporting live rollup/repository tests, full `go test ./...`, and `git diff --check` must pass. No route artifacts change. |
| 0 | P3.43 reporting rollup/backfill timezone parity | Implemented for reused report data freshness paths: GoChat now follows inspected Chatwoot `ReportingEvents::RollupService`, `ReportingEvents::BackfillService`, reporting-event service specs, and `Account#reporting_timezone` validation behavior by persisting/serializing `reporting_timezone`, treating blank or invalid values as a rollup-write skip, converting local report dates into UTC windows through the account reporting timezone, querying reporting events by `created_at >= start AND created_at < end`, skipping nil agent/inbox dimensions instead of falling back to the account id, zeroing count-metric sum fields, and counting distinct `conversation_bot_handoff` conversations per account/agent/inbox dimension. | Keep in Review; reopen from B12 reports smoke or fresh reference evidence for ActiveSupport timezone-name aliases beyond IANA names, data-source rollup read feature flags, or report metric registry drift beyond the inspected rollup/backfill contract. | Focused Reporting rollup/backfill/service tests, focused Analytics/Account handler tests, full `go test ./...`, and `git diff --check` must pass. No route artifacts change. |
@@ -284,6 +285,7 @@ This ledger records the committed parity checkpoints that future slices should b
| Commit | Scope | Verification summary | Follow-up state |
| --- | --- | --- | --- |
| `feat(reports): align summary aggregates` | Advances P3.46 report summary aggregate parity by matching Chatwoot `Api::V2::Accounts::ReportsController#build_summary`, `V2::Reports::Conversations::MetricBuilder`, `Reports::RawDataSource#aggregate`, and `ReportMetricRegistry`. GoChat `/reports/summary` now returns raw current metric keys plus `previous`, computes current/previous range aggregates from persisted conversations/messages/reporting events, honors report `type/id` dimensions and business-hours averages, and makes `bot_summary` aggregate counts follow the same `distinct_conversation` and `exclude_bot_handoffs` raw count strategies as the current timeseries path. | `go test ./internal/service -run 'Analytics.*Summary\|BotSummary\|BotTimeseries\|AnalyticsReports' -count=1`; `go test ./internal/handler/api/v1 -run AnalyticsHandler -count=1`; escalated full `go test ./...`; `git diff --check`. Route artifacts are unchanged. | P3.46 moves to Review for current summary aggregate evidence; continue Phase 2/3 drift audit for report builder depth, Phase 6 placeholder audit, B12 live smoke, or fresh reference/smoke drift. |
| `docs: consolidate parity execution tracker` | Documentation-only checkpoint requested before continuing implementation. It records the already-committed `aa159f6 feat(reports): align bot count strategies` implementation baseline, adds a one-page tracking dashboard, and makes the next-turn execution rule explicit: Phase 2/3 drift audit first unless the full B12 live stack is available. | `git diff --check`. No Go code, migrations, or route artifacts changed. | Continue Phase 2/3 drift audit, Phase 6 placeholder audit, or B12 live smoke from this tracker; `.hermes/plans/*` remain source notes only. |
| `feat(reports): align bot count strategies` | Advances P3.45 report bot raw-count strategy parity by matching Chatwoot `Reports::RawDataSource#reporting_event_count_scope`, `Reports::ReportMetricRegistry`, and v2 report timeseries specs. GoChat now applies `distinct_conversation` for `bot_handoffs_count`, ignores nil handoff conversation IDs, and applies `exclude_bot_handoffs` for `bot_resolutions_count` before bucketing so conversations that both resolve and hand off are not double counted. | `go test ./internal/service -run 'Analytics.*Timeseries\|BotTimeseries\|Reporting.*Rollup\|Backfill' -count=1`; full `go test ./...`; `git diff --check`. Route artifacts are unchanged. | P3.45 moves to Review for current bot metric raw-count evidence; continue Phase 2/3 drift audit for aggregate-value endpoints, report data-source/metric-builder depth, Phase 6 placeholder audit, B12 live smoke, or fresh reference/smoke drift. |
| `feat(reports): add live rollup upserts` | Advances P3.44 reporting live additive rollup parity by matching Chatwoot `ReportingEventListener#safe_rollup`, `ReportingEvents::RollupService`, and `ReportingEvents::EventMetricRegistry`. GoChat now applies rollups immediately after `AnalyticsService.RecordEvent` persists the raw reporting event, computes the local rollup date from the account `reporting_timezone`, emits only known event-registry metrics, and additively upserts unique rollup rows so repeated live events increment existing `count`, `sum_value`, and `sum_value_business_hours`. | `go test ./internal/service ./internal/repository -run 'Reporting.*Rollup\|Backfill\|Analytics\|ReportingEventsRollup' -count=1`; full `go test ./...`; `git diff --check`. Route artifacts are unchanged. | P3.44 moves to Review for current live-rollup evidence; continue Phase 2/3 drift audit for event-listener paths bypassing `AnalyticsService.RecordEvent`, report data-source/metric-builder depth, Phase 6 placeholder audit, B12 live smoke, or fresh reference/smoke drift. |
@@ -2545,3 +2547,4 @@ Verification milestone gates:
- 2026-06-06: P3.44 reporting live additive rollup checkpoint prepared as `feat(reports): add live rollup upserts`; audited Chatwoot reporting event listener safe-rollup path, rollup service additive `upsert_all`, and event metric registry. GoChat now applies live rollups from `AnalyticsService.RecordEvent` after raw event persistence, uses account reporting timezone for the event-created local date, ignores unknown raw metrics, and additively upserts existing rollup rows instead of waiting for lazy daily recompute. Focused reporting live rollup/repository tests, escalated full `go test ./...`, and `git diff --check` passed. No route artifacts change.
- 2026-06-07: P3.45 report bot raw-count strategy checkpoint prepared as `feat(reports): align bot count strategies`; audited Chatwoot `Reports::RawDataSource`, `Reports::ReportMetricRegistry`, and v2 timeseries bot-resolution specs. GoChat `bot_handoffs_count` timeseries now counts distinct non-null conversations per bucket, while `bot_resolutions_count` excludes conversations that also handed off in the scoped range before bucketing. Focused Analytics timeseries tests, escalated full `go test ./...`, and `git diff --check` passed. No route artifacts change.
- 2026-06-07: Documentation checkpoint prepared as `docs: consolidate parity execution tracker`; confirms `aa159f6 feat(reports): align bot count strategies` is already committed before new implementation work, adds the one-page tracking dashboard, keeps `.hermes/plans/*` as source notes only, and makes Phase 2/3 drift audit the next default action unless the full B12 live stack is available. Verification for this docs-only checkpoint: `git diff --check`.
- 2026-06-07: P3.46 report summary aggregate checkpoint prepared as `feat(reports): align summary aggregates`; audited Chatwoot v2 reports controller summary builder, conversation metric builder, raw data-source aggregate, metric registry, and reused dashboard `reports.js` summary/bot-summary callers. GoChat `/reports/summary` now returns raw Chatwoot metric keys plus `previous`, derives current/previous aggregates from persisted rows, honors report `type/id` filters and business-hours averages, and fixes `/reports/bot_summary` aggregate counting so handoffs are distinct non-null conversations while bot resolutions count event rows after excluding handoff conversations. Focused Analytics service and handler tests plus escalated full `go test ./...` passed; `git diff --check` passed. No route artifacts change.