feat(reports): align rollup timezones
This commit is contained in:
@@ -49,12 +49,12 @@ Hermes task landing checklist:
|
||||
|
||||
## Current Baseline
|
||||
|
||||
- Current tracking checkpoint: 2026-06-06 API v2 reports timeseries timezone parity checkpoint, prepared as `feat(reports): align v2 timeseries timezones`.
|
||||
- Latest implementation checkpoint: this checkpoint, prepared as `feat(reports): align v2 timeseries timezones`.
|
||||
- Latest documentation/tooling checkpoint: this tracker update records P3.42 API v2 report timeseries timezone parity plus the committed P3.41/P3.40 baseline and landed parity tracker history; this document is the active follow-up plan and supersedes `.hermes/plans/*`.
|
||||
- Current tracking checkpoint: 2026-06-06 reporting rollup/backfill timezone parity checkpoint, prepared as `feat(reports): align rollup timezones`.
|
||||
- Latest implementation checkpoint: this checkpoint, prepared as `feat(reports): align rollup timezones`.
|
||||
- Latest documentation/tooling checkpoint: this tracker update records P3.43 reporting rollup/backfill timezone parity plus the committed P3.42/P3.41/P3.40 reports baseline and landed parity tracker history; this document is the active follow-up plan and supersedes `.hermes/plans/*`.
|
||||
- Plan landing status: complete for the current known Hermes plans and user-confirmed scope. Future work should update this file directly instead of opening a parallel tracker.
|
||||
- Worktree status at this implementation checkpoint: API v2 report timeseries from `reference/chatwoot/config/routes.rb:488`, `Api::V2::Accounts::ReportsController#index`, `V2::Reports::Timeseries::BaseTimeseriesBuilder`, `Reports::RawDataSource`, `TimezoneHelper`, `DateRangeHelper`, and reused dashboard `reports.js` are aligned for frontend `timezone_offset` grouping. `GET /api/v2/accounts/:account_id/reports` now parses `timezone_offset`, groups conversations/messages/reporting-events by local hour/day/week/month/year bucket boundaries, returns bucket timestamps at the local bucket start like Chatwoot `group_by_period(time_zone: ...)`, and includes zero-valued buckets across the requested range. P3.41 CSV/download parity and P3.40 raw JSON/date parity for report, live-report, and summary-report endpoints remain implemented. Profile MFA, Facebook callbacks, account WhatsApp calls, onboarding, notification scoping, API v2 year-in-review, and WhatsApp calling toggle parity remain implemented from previous checkpoints. Live API/browser/enterprise smoke still needs the full PostgreSQL/Redis/Meilisearch/GoChat/Vite/Chrome stack.
|
||||
- Next executable implementation checkpoint: continue Phase 2/3 drift audit for deeper report metric-builder drift or the next reused-frontend mismatch, Phase 6 placeholder audit, or B12 live smoke from fresh reference/smoke evidence.
|
||||
- Worktree status at this implementation checkpoint: reporting rollup/backfill behavior from `reference/chatwoot/app/services/reporting_events/rollup_service.rb`, `backfill_service.rb`, `spec/services/reporting_events/*`, and `Account#reporting_timezone` is aligned for local reporting dates. GoChat now stores/serializes `accounts.reporting_timezone`, skips rollup writes when it is blank or invalid, converts requested report dates to UTC boundaries through the account reporting timezone, queries `reporting_events.created_at` in `[start, end)`, skips nil agent/inbox dimensions, writes count-metric sums as zero, and deduplicates `conversation_bot_handoff` by `conversation_id` per account/agent/inbox dimension. P3.42 timeseries timezone parity, P3.41 CSV/download parity, and P3.40 raw JSON/date parity for report, live-report, and summary-report endpoints remain implemented. Profile MFA, Facebook callbacks, account WhatsApp calls, onboarding, notification scoping, API v2 year-in-review, and WhatsApp calling toggle parity remain implemented from previous checkpoints. Live API/browser/enterprise smoke still needs the full PostgreSQL/Redis/Meilisearch/GoChat/Vite/Chrome stack.
|
||||
- Next executable implementation checkpoint: continue Phase 2/3 drift audit for deeper report rollup/data-source drift or the next reused-frontend mismatch, Phase 6 placeholder audit, or B12 live smoke from fresh reference/smoke evidence.
|
||||
- `go test ./...` passes when run outside the restricted socket sandbox; focused Facebook callback handler/router/route-parity tests pass in the sandbox.
|
||||
- Route dump succeeds with `967` registered routes after profile MFA route tracking.
|
||||
- Route parity artifacts now exist under `docs/parity/` and are generated by `cmd/route_parity`.
|
||||
@@ -140,6 +140,7 @@ This table is the shortest authoritative handoff view. If an older lower section
|
||||
|
||||
| Priority | Workstream | Current state | Next checkpoint | Commit close rule |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 0 | P3.43 reporting rollup/backfill timezone parity | Implemented for reused report data freshness paths: GoChat now follows inspected Chatwoot `ReportingEvents::RollupService`, `ReportingEvents::BackfillService`, reporting-event service specs, and `Account#reporting_timezone` validation behavior by persisting/serializing `reporting_timezone`, treating blank or invalid values as a rollup-write skip, converting local report dates into UTC windows through the account reporting timezone, querying reporting events by `created_at >= start AND created_at < end`, skipping nil agent/inbox dimensions instead of falling back to the account id, zeroing count-metric sum fields, and counting distinct `conversation_bot_handoff` conversations per account/agent/inbox dimension. | Keep in Review; reopen from B12 reports smoke or fresh reference evidence for per-event additive `upsert_all` live rollup wiring, ActiveSupport timezone-name aliases beyond IANA names, data-source rollup read feature flags, or report metric registry drift beyond the inspected rollup/backfill contract. | Focused Reporting rollup/backfill/service tests, focused Analytics/Account handler tests, full `go test ./...`, and `git diff --check` must pass. No route artifacts change. |
|
||||
| 0 | P3.42 API v2 reports timeseries timezone parity | Implemented for reused dashboard chart clients: `GET /api/v2/accounts/:account_id/reports` now follows inspected Chatwoot `ReportsController#index`, `V2::Reports::Timeseries::BaseTimeseriesBuilder`, `Reports::RawDataSource`, and `TimezoneHelper` behavior by honoring frontend `timezone_offset` for bucket boundaries and returned timestamps. Conversation, message, and reporting-event timeseries now seed zero buckets across the requested range, group by local hour/day/week/month/year starts, preserve data totals while redistributing values across timezone-local buckets, and keep summary-report totals timezone-boundary driven by the frontend `since/until` values. | Keep in Review; reopen from B12 reports smoke or fresh reference evidence for ActiveSupport named timezone/DST drift, exact Groupdate range edge cases, rollup-data-source behavior, or metric-specific builder differences beyond the inspected raw timeseries contract. | Focused Analytics handler/service timeseries tests, full `go test ./...`, and `git diff --check` must pass. No route artifacts change. |
|
||||
| 0 | P3.41 API v2 reports CSV/download parity | Implemented for reused dashboard report download clients: `GET /api/v2/accounts/:account_id/reports/agents`, `/inboxes`, `/labels`, `/teams`, `/conversations_summary`, and `/conversation_traffic` now follow the inspected Chatwoot v2 reports controller, CSV templates, `ReportsHelper`, `HeatmapHelper`, and `reports.js` download actions by returning `text/csv` attachments instead of JSON. Standard download reports emit Chatwoot filenames, reporting-period row, localized English header text from the reference locale, readable duration formatting, and account-scoped agent/inbox/team/label/conversation-summary rows. `conversation_traffic` accepts frontend `days_before` plus `timezone_offset` without requiring `since/until`, emits `conversation_traffic_reports.csv`, a timezone row, and hourly heatmap rows; explicit `since/until` remains compatible for local callers. | Keep in Review; reopen from B12 reports smoke or fresh reference evidence for exact metric-builder aggregation drift, ActiveSupport timezone display-name drift, business-hours summary drift, channel display-name drift, or CSV row ordering/body differences beyond the inspected download contract. | Focused Analytics handler CSV tests, full `go test ./...`, and `git diff --check` must pass. No route artifacts change. |
|
||||
| 0 | P3.40 API v2 reports envelope/date parity | Implemented for reused dashboard reports clients: `GET /api/v2/accounts/:account_id/reports*`, `/summary_reports/*`, and `/live_reports/*` now follow the inspected Chatwoot v2 controllers by accepting Unix-second `since/until` params from `reports.js`/`summaryReports.js`, preserving existing RFC3339 compatibility for local callers, and returning raw JSON arrays/objects for reports, live reports, and summary reports instead of the local `{ success, data }` envelope. | Keep in Review; reopen from B12 reports smoke or fresh reference evidence for CSV body column drift, deeper metric-builder shape drift, authorization payload drift, or report CSV download content differences beyond this raw JSON/date-parameter slice. | Focused Analytics/SummaryReport/LiveReport handler tests, full `go test ./...`, and `git diff --check` must pass. No route artifacts change. |
|
||||
@@ -265,6 +266,7 @@ This ledger records the committed parity checkpoints that future slices should b
|
||||
|
||||
| Commit | Scope | Verification summary | Follow-up state |
|
||||
| --- | --- | --- | --- |
|
||||
| `feat(reports): align rollup timezones` | Advances P3.43 reporting rollup/backfill timezone parity by matching Chatwoot `ReportingEvents::RollupService`, `ReportingEvents::BackfillService`, reporting event rollup/backfill specs, `ReportingEvents::EventMetricRegistry`, and `Account#reporting_timezone`. GoChat now adds account `reporting_timezone` storage/serialization, skips blank/invalid rollup writes, computes UTC windows from local reporting dates, uses `reporting_events.created_at` half-open ranges, skips nil agent/inbox dimensions, zeroes count-metric sum columns, and deduplicates `conversation_bot_handoff` by distinct conversation per dimension. | `go test ./internal/service ./internal/handler/api/v1 -run 'Reporting.*Rollup\|Backfill\|Analytics\|Account' -count=1`; full `go test ./...`; `git diff --check`. Route artifacts are unchanged. | P3.43 moves to Review for current rollup/backfill timezone evidence; continue Phase 2/3 drift audit for per-event additive rollup wiring, ActiveSupport timezone alias depth, rollup data-source feature flags, Phase 6 placeholder audit, B12 live smoke, or fresh reference/smoke drift. |
|
||||
| `feat(reports): align v2 timeseries timezones` | Advances P3.42 API v2 reports timeseries timezone parity by matching Chatwoot `Api::V2::Accounts::ReportsController#index`, `V2::Reports::Timeseries::BaseTimeseriesBuilder`, `Reports::RawDataSource`, `TimezoneHelper`, `DateRangeHelper`, controller timezone specs, and reused dashboard `reports.js`. GoChat now parses frontend `timezone_offset` on `/reports`, seeds zero buckets across the range, groups conversation/message/reporting-event timeseries by local hour/day/week/month/year starts, and returns timestamps for the local bucket boundary so timezone offsets redistribute values while preserving totals. | `go test ./internal/handler/api/v1 ./internal/service -run 'Analytics\|Timeseries' -count=1`; full `go test ./...`; `git diff --check`. Route artifacts are unchanged. | P3.42 moves to Review for current timeseries timezone evidence; continue Phase 2/3 drift audit for Groupdate/DST/rollup builder depth, Phase 6 placeholder audit, B12 live smoke, or fresh reference/smoke drift. |
|
||||
| `feat(reports): align v2 csv downloads` | Advances P3.41 API v2 reports CSV/download parity by matching Chatwoot `Api::V2::Accounts::ReportsController`, CSV templates `agents/inboxes/labels/teams/conversations_summary/conversation_traffic`, `ReportsHelper`, `HeatmapHelper`, locale CSV labels, and reused dashboard `reports.js` download actions. GoChat now returns `text/csv` attachments for report downloads, emits Chatwoot filenames and headers, keeps Unix-second `since/until` report period support, formats durations like Chatwoot's time presenter, and lets `conversation_traffic` run from frontend `days_before`/`timezone_offset` without requiring `since/until`. | `go test ./internal/handler/api/v1 -run Analytics -count=1`; full `go test ./...`; `git diff --check`. Route artifacts are unchanged. | P3.41 moves to Review for current CSV/download evidence; continue Phase 2/3 drift audit for metric-builder/timezone/body-depth drift, Phase 6 placeholder audit, B12 live smoke, or fresh reference/smoke drift. |
|
||||
| `feat(reports): align v2 report envelopes` | Advances P3.40 API v2 reports envelope/date-parameter parity by matching Chatwoot `Api::V2::Accounts::ReportsController`, `SummaryReportsController`, `LiveReportsController`, routes `481-509`, and reused dashboard `reports.js`, `summaryReports.js`, and `liveReports.js`. GoChat now accepts Unix-second `since/until` report params from the frontend while preserving RFC3339 compatibility, and returns raw Chatwoot-style JSON arrays/objects for reports, summary reports, and live reports instead of the local `{ success, data }` envelope. | `go test ./internal/handler/api/v1 -run 'Analytics\|SummaryReport\|LiveReport' -count=1`; full `go test ./...`; `git diff --check`. Route artifacts are unchanged. | P3.40 moves to Review for the current v2 report JSON/date evidence; continue Phase 2/3 drift audit for CSV/metric-builder depth, Phase 6 placeholder audit, B12 live smoke, or fresh reference/smoke drift. |
|
||||
@@ -2518,3 +2520,4 @@ Verification milestone gates:
|
||||
- 2026-06-06: P3.40 API v2 reports checkpoint prepared as `feat(reports): align v2 report envelopes`; audited Chatwoot v2 reports, summary reports, and live reports controllers plus reused dashboard `reports.js`, `summaryReports.js`, and `liveReports.js`. GoChat report handlers now accept frontend Unix-second `since/until` params while preserving RFC3339 compatibility, and v2 reports/live_reports/summary_reports return raw Chatwoot JSON arrays/objects instead of the local `{ success, data }` envelope. Focused Analytics/SummaryReport/LiveReport handler tests, escalated full `go test ./...`, and `git diff --check` passed. No route artifacts change.
|
||||
- 2026-06-06: P3.41 API v2 reports CSV/download checkpoint prepared as `feat(reports): align v2 csv downloads`; audited Chatwoot v2 reports CSV actions, templates, ReportsHelper/HeatmapHelper, locale headers, and reused dashboard report download clients. GoChat now returns `text/csv` attachments for agents/inboxes/labels/teams/conversations_summary/conversation_traffic, emits Chatwoot filenames/reporting-period/header rows, renders readable durations, and accepts `conversation_traffic` frontend `days_before`/`timezone_offset` without requiring `since/until`. Focused Analytics CSV tests, escalated full `go test ./...`, and `git diff --check` passed. No route artifacts change.
|
||||
- 2026-06-06: P3.42 API v2 reports timeseries timezone checkpoint prepared as `feat(reports): align v2 timeseries timezones`; audited Chatwoot report index controller, Timeseries builder, RawDataSource, TimezoneHelper, DateRangeHelper, controller specs, and reused dashboard reports client. GoChat `/reports` timeseries now honors frontend `timezone_offset` for local hour/day/week/month/year bucket starts, returns local bucket-boundary timestamps, and seeds zero-value buckets across the requested range while preserving totals. Focused Analytics/Timeseries tests, escalated full `go test ./...`, and `git diff --check` passed. No route artifacts change.
|
||||
- 2026-06-06: P3.43 reporting rollup/backfill timezone checkpoint prepared as `feat(reports): align rollup timezones`; audited Chatwoot `ReportingEvents::RollupService`, `ReportingEvents::BackfillService`, rollup/backfill specs, event metric registry, and `Account#reporting_timezone`. GoChat now stores and serializes `reporting_timezone`, skips blank/invalid rollups, computes local-date UTC windows through account reporting timezone, filters reporting events by `created_at` half-open ranges, skips nil agent/inbox dimensions, preserves zero sum fields for count metrics, and deduplicates bot handoffs by distinct conversation per account/agent/inbox. Focused reporting rollup/backfill/service and analytics/account handler tests, escalated full `go test ./...`, and `git diff --check` passed. No route artifacts change.
|
||||
|
||||
@@ -456,6 +456,7 @@ func serializeAccountSettings(account *model.Account) map[string]any {
|
||||
"auto_resolve_ignore_waiting": false,
|
||||
"audio_transcriptions": false,
|
||||
"auto_resolve_label": "",
|
||||
"reporting_timezone": account.ReportingTimezone,
|
||||
}
|
||||
return settings
|
||||
}
|
||||
|
||||
@@ -51,7 +51,7 @@ func (s *AnalyticsHandlerTestSuite) SetupSuite() {
|
||||
))
|
||||
|
||||
// Create test account
|
||||
acct := model.Account{Name: "TestAccount"}
|
||||
acct := model.Account{Name: "TestAccount", ReportingTimezone: "UTC"}
|
||||
s.Require().NoError(db.Create(&acct).Error)
|
||||
s.accountID = acct.ID
|
||||
|
||||
|
||||
@@ -13,6 +13,7 @@ type Account struct {
|
||||
Domain string `gorm:"size:255" json:"domain,omitempty"`
|
||||
Locale string `gorm:"size:10;default:en" json:"locale"`
|
||||
Timezone string `gorm:"size:50;default:UTC" json:"timezone"`
|
||||
ReportingTimezone string `gorm:"size:100" json:"reporting_timezone,omitempty"`
|
||||
Active bool `gorm:"default:true" json:"active"`
|
||||
Status string `gorm:"size:50;default:active" json:"status"`
|
||||
OnboardingStep string `gorm:"size:100" json:"onboarding_step,omitempty"`
|
||||
|
||||
@@ -118,12 +118,12 @@ func (r *ReportingEventRepo) AggregateByMetric(ctx context.Context, accountID ui
|
||||
return result.SumValue, result.Count, err
|
||||
}
|
||||
|
||||
// FindByAccountIDAndTimeRange retrieves all events for an account in a time range,
|
||||
// using event_start_time / event_end_time for precise boundary matching.
|
||||
// FindByAccountIDAndTimeRange retrieves all events created in [since, until).
|
||||
// Chatwoot rollup/backfill uses reporting_events.created_at for local-date boundaries.
|
||||
func (r *ReportingEventRepo) FindByAccountIDAndTimeRange(ctx context.Context, accountID uint, since, until time.Time) ([]model.ReportingEvent, error) {
|
||||
var events []model.ReportingEvent
|
||||
err := r.db.WithContext(ctx).
|
||||
Where("account_id = ? AND event_start_time >= ? AND event_end_time <= ?", accountID, since, until).
|
||||
Where("account_id = ? AND created_at >= ? AND created_at < ?", accountID, since, until).
|
||||
Find(&events).Error
|
||||
return events, err
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ package service
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/gochat/gochat/internal/model"
|
||||
@@ -224,6 +225,7 @@ func (s *AccountService) RemoveUser(ctx context.Context, accountID, userID uint)
|
||||
type UpdateAccountSettingsRequest struct {
|
||||
AutoResolveDuration int `json:"auto_resolve_duration" validate:"gte=0"`
|
||||
Locale string `json:"locale" validate:"omitempty,len=2"`
|
||||
ReportingTimezone string `json:"reporting_timezone" validate:"omitempty"`
|
||||
}
|
||||
|
||||
// UpdateSettings updates account-level settings.
|
||||
@@ -241,6 +243,12 @@ func (s *AccountService) UpdateSettings(ctx context.Context, id uint, req Update
|
||||
if req.Locale != "" {
|
||||
account.Locale = req.Locale
|
||||
}
|
||||
if req.ReportingTimezone != "" {
|
||||
if _, err := time.LoadLocation(req.ReportingTimezone); err != nil {
|
||||
return nil, fmt.Errorf("invalid reporting_timezone")
|
||||
}
|
||||
account.ReportingTimezone = req.ReportingTimezone
|
||||
}
|
||||
|
||||
if err := s.repo.Update(ctx, account); err != nil {
|
||||
return nil, err
|
||||
|
||||
@@ -39,7 +39,7 @@ func setupAnalyticsP513Test(t *testing.T) (*gorm.DB, *AnalyticsService, *model.A
|
||||
sqlDB, _ := db.DB()
|
||||
sqlDB.Close()
|
||||
})
|
||||
account := &model.Account{Name: "Analytics"}
|
||||
account := &model.Account{Name: "Analytics", ReportingTimezone: "UTC"}
|
||||
require.NoError(t, db.Create(account).Error)
|
||||
user := &model.User{AccountID: account.ID, Name: "Agent", Email: "agent@example.com", Password: "secret", Active: true}
|
||||
require.NoError(t, db.Create(user).Error)
|
||||
|
||||
@@ -2,12 +2,12 @@ package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/gochat/gochat/internal/model"
|
||||
"github.com/gochat/gochat/internal/repository"
|
||||
applogger "github.com/gochat/gochat/pkg/logger"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// ReportingBackfillService backfills missing rollup data for a given account and date.
|
||||
@@ -48,37 +48,42 @@ var DistinctCountEvents = []string{
|
||||
}
|
||||
|
||||
// BackfillDate performs backfill for a single account on a single date.
|
||||
// 1. Delete existing rollups for that date
|
||||
// 2. Aggregate raw events into rollup rows
|
||||
// 3. Bulk insert rollup rows
|
||||
// 1. Build replacement rollups for the account reporting timezone date
|
||||
// 2. Delete existing rollups for that date and insert replacements transactionally
|
||||
func (s *ReportingBackfillService) BackfillDate(ctx context.Context, accountID uint, date time.Time) error {
|
||||
// Step 1: Delete existing rollups for this account+date
|
||||
if err := s.rollupRepo.DeleteByAccountAndDate(ctx, accountID, date); err != nil {
|
||||
applogger.L().Errorf("BackfillDate delete existing rollups: %v", err)
|
||||
db := reportingServiceDB(s.eventRepo, s.rollupRepo)
|
||||
loc, enabled, err := accountReportingLocation(ctx, db, accountID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !enabled {
|
||||
return nil
|
||||
}
|
||||
startUTC, endUTC, rollupDate := utcBoundariesForReportingDate(date, loc)
|
||||
|
||||
// Step 2: Determine UTC boundaries for the date
|
||||
// TODO: Use account.reporting_timezone for proper TZ conversion (currently UTC)
|
||||
startUTC := time.Date(date.Year(), date.Month(), date.Day(), 0, 0, 0, 0, time.UTC)
|
||||
endUTC := startUTC.Add(24 * time.Hour)
|
||||
|
||||
// Step 3: Build rollup rows by aggregating raw events
|
||||
rollupRows, err := s.buildRollupRows(ctx, accountID, date, startUTC, endUTC)
|
||||
rollupRows, err := s.buildRollupRows(ctx, accountID, rollupDate, startUTC, endUTC)
|
||||
if err != nil {
|
||||
applogger.L().Errorf("BackfillDate build rollup rows: %v", err)
|
||||
return err
|
||||
}
|
||||
|
||||
// Step 4: Bulk insert if any rows were produced
|
||||
if len(rollupRows) > 0 {
|
||||
if err := s.rollupRepo.BulkCreate(ctx, rollupRows); err != nil {
|
||||
if db == nil {
|
||||
return nil
|
||||
}
|
||||
return db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
if err := tx.Unscoped().Where("account_id = ? AND date = ?", accountID, rollupDate).Delete(&model.ReportingEventsRollup{}).Error; err != nil {
|
||||
applogger.L().Errorf("BackfillDate delete existing rollups: %v", err)
|
||||
return err
|
||||
}
|
||||
if len(rollupRows) == 0 {
|
||||
return nil
|
||||
}
|
||||
if err := tx.CreateInBatches(rollupRows, 100).Error; err != nil {
|
||||
applogger.L().Errorf("BackfillDate bulk insert: %v", err)
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// BackfillRange performs backfill for a date range (inclusive).
|
||||
@@ -103,83 +108,9 @@ type RollupAggregate struct {
|
||||
}
|
||||
|
||||
func (s *ReportingBackfillService) buildRollupRows(ctx context.Context, accountID uint, date time.Time, startUTC, endUTC time.Time) ([]model.ReportingEventsRollup, error) {
|
||||
var rollupRows []model.ReportingEventsRollup
|
||||
|
||||
// For each dimension, aggregate events
|
||||
for _, dim := range BackfillDimensions {
|
||||
aggregates, err := s.aggregateForDimension(ctx, accountID, dim, startUTC, endUTC)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for _, agg := range aggregates {
|
||||
// Map raw event metrics to rollup metrics via MetricRegistry
|
||||
rollupMetrics := ExpandEventToRollupMetrics(agg.Metric, agg.Count, agg.SumValue, agg.SumBizHours)
|
||||
for rm, data := range rollupMetrics {
|
||||
rollupRows = append(rollupRows, model.ReportingEventsRollup{
|
||||
AccountID: accountID,
|
||||
Date: date,
|
||||
DimensionType: agg.DimensionType,
|
||||
DimensionID: agg.DimensionID,
|
||||
Metric: rm,
|
||||
Count: data.Count,
|
||||
SumValue: data.SumValue,
|
||||
SumValueBusinessHours: data.SumBizHours,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return rollupRows, nil
|
||||
}
|
||||
|
||||
func (s *ReportingBackfillService) aggregateForDimension(ctx context.Context, accountID uint, dim DimensionSpec, startUTC, endUTC time.Time) ([]RollupAggregate, error) {
|
||||
// Query raw events grouped by the dimension's group column + metric name
|
||||
events, err := s.eventRepo.FindByAccountIDAndTimeRange(ctx, accountID, startUTC, endUTC)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Group events by (dimension_type, dimension_id, metric_name) and aggregate
|
||||
groupMap := make(map[string]*RollupAggregate)
|
||||
|
||||
for _, event := range events {
|
||||
dimensionID := accountID // account dimension uses account_id
|
||||
if dim.GroupColumn == "user_id" && event.UserID != nil {
|
||||
dimensionID = *event.UserID
|
||||
} else if dim.GroupColumn == "inbox_id" && event.InboxID != nil {
|
||||
dimensionID = *event.InboxID
|
||||
}
|
||||
|
||||
key := dimKey(dim.Type, dimensionID, event.Name)
|
||||
agg, ok := groupMap[key]
|
||||
if !ok {
|
||||
agg = &RollupAggregate{
|
||||
DimensionType: dim.Type,
|
||||
DimensionID: dimensionID,
|
||||
Metric: model.RollupMetric(event.Name),
|
||||
Count: 0,
|
||||
SumValue: 0,
|
||||
SumBizHours: 0,
|
||||
}
|
||||
groupMap[key] = agg
|
||||
}
|
||||
|
||||
// For distinct-count events, we track unique conversation IDs separately
|
||||
// The backfill uses COUNT(DISTINCT conversation_id) at DB level, but here
|
||||
// we approximate by counting each event once per conversation
|
||||
agg.Count++
|
||||
agg.SumValue += event.Value
|
||||
agg.SumBizHours += event.ValueInBusinessHours
|
||||
}
|
||||
|
||||
var result []RollupAggregate
|
||||
for _, agg := range groupMap {
|
||||
result = append(result, *agg)
|
||||
}
|
||||
return result, nil
|
||||
return rollupRowsFromAggregates(accountID, date, aggregateReportingEvents(accountID, events)), nil
|
||||
}
|
||||
|
||||
func dimKey(dimType model.DimensionType, dimID uint, metric string) string {
|
||||
return fmt.Sprintf("%s_%d_%s", dimType, dimID, metric)
|
||||
}
|
||||
@@ -6,8 +6,8 @@ import (
|
||||
|
||||
// RollupMetricData holds the computed count and value for a single rollup metric.
|
||||
type RollupMetricData struct {
|
||||
Count int64
|
||||
SumValue float64
|
||||
Count int64
|
||||
SumValue float64
|
||||
SumBizHours float64
|
||||
}
|
||||
|
||||
@@ -15,11 +15,12 @@ type RollupMetricData struct {
|
||||
// it contributes to. This mirrors Chatwoot's EVENT_METRICS registry.
|
||||
//
|
||||
// Chatwoot EVENT_METRICS mapping:
|
||||
// conversation_resolved → resolutions_count (count) + resolution_time (duration)
|
||||
// first_response → first_response (duration)
|
||||
// reply_time → reply_time (duration)
|
||||
// conversation_bot_resolved → bot_resolutions_count (count)
|
||||
// conversation_bot_handoff → bot_handoffs_count (count)
|
||||
//
|
||||
// conversation_resolved → resolutions_count (count) + resolution_time (duration)
|
||||
// first_response → first_response (duration)
|
||||
// reply_time → reply_time (duration)
|
||||
// conversation_bot_resolved → bot_resolutions_count (count)
|
||||
// conversation_bot_handoff → bot_handoffs_count (count)
|
||||
func ExpandEventToRollupMetrics(rawMetric model.RollupMetric, count int64, sumValue, sumBizHours float64) map[model.RollupMetric]RollupMetricData {
|
||||
result := map[model.RollupMetric]RollupMetricData{}
|
||||
|
||||
@@ -27,50 +28,50 @@ func ExpandEventToRollupMetrics(rawMetric model.RollupMetric, count int64, sumVa
|
||||
case model.RollupMetric("conversation_resolved"):
|
||||
// Count metric: resolutions_count
|
||||
result[model.MetricResolutionsCount] = RollupMetricData{
|
||||
Count: count,
|
||||
SumValue: float64(count),
|
||||
SumBizHours: float64(count),
|
||||
Count: count,
|
||||
SumValue: 0,
|
||||
SumBizHours: 0,
|
||||
}
|
||||
// Duration metric: resolution_time
|
||||
result[model.MetricResolutionTime] = RollupMetricData{
|
||||
Count: count,
|
||||
SumValue: sumValue,
|
||||
Count: count,
|
||||
SumValue: sumValue,
|
||||
SumBizHours: sumBizHours,
|
||||
}
|
||||
|
||||
case model.RollupMetric("first_response"):
|
||||
result[model.MetricFirstResponse] = RollupMetricData{
|
||||
Count: count,
|
||||
SumValue: sumValue,
|
||||
Count: count,
|
||||
SumValue: sumValue,
|
||||
SumBizHours: sumBizHours,
|
||||
}
|
||||
|
||||
case model.RollupMetric("reply_time"):
|
||||
result[model.MetricReplyTime] = RollupMetricData{
|
||||
Count: count,
|
||||
SumValue: sumValue,
|
||||
Count: count,
|
||||
SumValue: sumValue,
|
||||
SumBizHours: sumBizHours,
|
||||
}
|
||||
|
||||
case model.RollupMetric("conversation_bot_resolved"):
|
||||
result[model.MetricBotResolutionsCount] = RollupMetricData{
|
||||
Count: count,
|
||||
SumValue: float64(count),
|
||||
SumBizHours: float64(count),
|
||||
Count: count,
|
||||
SumValue: 0,
|
||||
SumBizHours: 0,
|
||||
}
|
||||
|
||||
case model.RollupMetric("conversation_bot_handoff"):
|
||||
result[model.MetricBotHandoffsCount] = RollupMetricData{
|
||||
Count: count,
|
||||
SumValue: float64(count),
|
||||
SumBizHours: float64(count),
|
||||
Count: count,
|
||||
SumValue: 0,
|
||||
SumBizHours: 0,
|
||||
}
|
||||
|
||||
default:
|
||||
// Unknown metric — still record it as a direct rollup
|
||||
result[rawMetric] = RollupMetricData{
|
||||
Count: count,
|
||||
SumValue: sumValue,
|
||||
Count: count,
|
||||
SumValue: sumValue,
|
||||
SumBizHours: sumBizHours,
|
||||
}
|
||||
}
|
||||
@@ -81,23 +82,23 @@ func ExpandEventToRollupMetrics(rawMetric model.RollupMetric, count int64, sumVa
|
||||
// ReportMetricDefinition describes how a report metric is sourced and aggregated.
|
||||
// Mirrors Chatwoot REPORT_METRICS registry.
|
||||
type ReportMetricDefinition struct {
|
||||
RawEventName string // The raw ReportingEvent name that feeds this metric
|
||||
RawEventName string // The raw ReportingEvent name that feeds this metric
|
||||
RollupMetric model.RollupMetric // The rollup metric column to read from
|
||||
AggregateType string // "count", "average", "sum"
|
||||
AggregateType string // "count", "average", "sum"
|
||||
}
|
||||
|
||||
// ReportMetricsRegistry maps summary response keys to metric definitions.
|
||||
// Mirrors Chatwoot MetricRegistry::REPORT_METRICS.
|
||||
var ReportMetricsRegistry = map[string]ReportMetricDefinition{
|
||||
"conversations_count": {AggregateType: "count"},
|
||||
"incoming_messages_count": {AggregateType: "count"},
|
||||
"outgoing_messages_count": {AggregateType: "count"},
|
||||
"avg_first_response_time": {RawEventName: "first_response", RollupMetric: model.MetricFirstResponse, AggregateType: "average"},
|
||||
"avg_resolution_time": {RawEventName: "conversation_resolved", RollupMetric: model.MetricResolutionTime, AggregateType: "average"},
|
||||
"reply_time": {RawEventName: "reply_time", RollupMetric: model.MetricReplyTime, AggregateType: "average"},
|
||||
"resolutions_count": {RawEventName: "conversation_resolved", RollupMetric: model.MetricResolutionsCount, AggregateType: "count"},
|
||||
"bot_resolutions_count": {RawEventName: "conversation_bot_resolved", RollupMetric: model.MetricBotResolutionsCount, AggregateType: "count"},
|
||||
"bot_handoffs_count": {RawEventName: "conversation_bot_handoff", RollupMetric: model.MetricBotHandoffsCount, AggregateType: "count"},
|
||||
"conversations_count": {AggregateType: "count"},
|
||||
"incoming_messages_count": {AggregateType: "count"},
|
||||
"outgoing_messages_count": {AggregateType: "count"},
|
||||
"avg_first_response_time": {RawEventName: "first_response", RollupMetric: model.MetricFirstResponse, AggregateType: "average"},
|
||||
"avg_resolution_time": {RawEventName: "conversation_resolved", RollupMetric: model.MetricResolutionTime, AggregateType: "average"},
|
||||
"reply_time": {RawEventName: "reply_time", RollupMetric: model.MetricReplyTime, AggregateType: "average"},
|
||||
"resolutions_count": {RawEventName: "conversation_resolved", RollupMetric: model.MetricResolutionsCount, AggregateType: "count"},
|
||||
"bot_resolutions_count": {RawEventName: "conversation_bot_resolved", RollupMetric: model.MetricBotResolutionsCount, AggregateType: "count"},
|
||||
"bot_handoffs_count": {RawEventName: "conversation_bot_handoff", RollupMetric: model.MetricBotHandoffsCount, AggregateType: "count"},
|
||||
"avg_first_response_time_business_hours": {RawEventName: "first_response", RollupMetric: model.MetricFirstResponse, AggregateType: "average_biz"},
|
||||
"avg_resolution_time_business_hours": {RawEventName: "conversation_resolved", RollupMetric: model.MetricResolutionTime, AggregateType: "average_biz"},
|
||||
"reply_time_business_hours": {RawEventName: "reply_time", RollupMetric: model.MetricReplyTime, AggregateType: "average_biz"},
|
||||
@@ -118,4 +119,4 @@ func GetReportMetricDefinition(key string) *ReportMetricDefinition {
|
||||
return &def
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gochat/gochat/internal/model"
|
||||
"github.com/gochat/gochat/internal/repository"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func reportingServiceDB(eventRepo *repository.ReportingEventRepo, rollupRepo *repository.ReportingEventsRollupRepo) *gorm.DB {
|
||||
if rollupRepo != nil && rollupRepo.DB() != nil {
|
||||
return rollupRepo.DB()
|
||||
}
|
||||
if eventRepo != nil {
|
||||
return eventRepo.DB()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func accountReportingLocation(ctx context.Context, db *gorm.DB, accountID uint) (*time.Location, bool, error) {
|
||||
if db == nil {
|
||||
return nil, false, nil
|
||||
}
|
||||
var account model.Account
|
||||
if err := db.WithContext(ctx).First(&account, accountID).Error; err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
tz := strings.TrimSpace(account.ReportingTimezone)
|
||||
if tz == "" {
|
||||
return nil, false, nil
|
||||
}
|
||||
loc, err := time.LoadLocation(tz)
|
||||
if err != nil {
|
||||
return nil, false, nil
|
||||
}
|
||||
return loc, true, nil
|
||||
}
|
||||
|
||||
func utcBoundariesForReportingDate(date time.Time, loc *time.Location) (time.Time, time.Time, time.Time) {
|
||||
localDate := time.Date(date.Year(), date.Month(), date.Day(), 0, 0, 0, 0, loc)
|
||||
startUTC := localDate.UTC()
|
||||
endUTC := localDate.AddDate(0, 0, 1).UTC()
|
||||
rollupDate := time.Date(localDate.Year(), localDate.Month(), localDate.Day(), 0, 0, 0, 0, time.UTC)
|
||||
return startUTC, endUTC, rollupDate
|
||||
}
|
||||
|
||||
func dimensionIDForEvent(accountID uint, event model.ReportingEvent, dim DimensionSpec) (uint, bool) {
|
||||
switch dim.GroupColumn {
|
||||
case "":
|
||||
return accountID, true
|
||||
case "user_id":
|
||||
if event.UserID == nil {
|
||||
return 0, false
|
||||
}
|
||||
return *event.UserID, true
|
||||
case "inbox_id":
|
||||
if event.InboxID == nil {
|
||||
return 0, false
|
||||
}
|
||||
return *event.InboxID, true
|
||||
default:
|
||||
return 0, false
|
||||
}
|
||||
}
|
||||
|
||||
func isDistinctCountEvent(name string) bool {
|
||||
for _, eventName := range DistinctCountEvents {
|
||||
if name == eventName {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func aggregateReportingEvents(accountID uint, events []model.ReportingEvent) []RollupAggregate {
|
||||
type groupKey struct {
|
||||
dimType model.DimensionType
|
||||
dimID uint
|
||||
metric model.RollupMetric
|
||||
}
|
||||
|
||||
groups := map[groupKey]*RollupAggregate{}
|
||||
seenDistinct := map[groupKey]map[uint]struct{}{}
|
||||
|
||||
for _, event := range events {
|
||||
for _, dim := range BackfillDimensions {
|
||||
dimID, ok := dimensionIDForEvent(accountID, event, dim)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
key := groupKey{dimType: dim.Type, dimID: dimID, metric: model.RollupMetric(event.Name)}
|
||||
agg, ok := groups[key]
|
||||
if !ok {
|
||||
agg = &RollupAggregate{DimensionType: dim.Type, DimensionID: dimID, Metric: model.RollupMetric(event.Name)}
|
||||
groups[key] = agg
|
||||
}
|
||||
|
||||
if isDistinctCountEvent(event.Name) {
|
||||
if event.ConversationID == nil {
|
||||
continue
|
||||
}
|
||||
if seenDistinct[key] == nil {
|
||||
seenDistinct[key] = map[uint]struct{}{}
|
||||
}
|
||||
if _, exists := seenDistinct[key][*event.ConversationID]; exists {
|
||||
continue
|
||||
}
|
||||
seenDistinct[key][*event.ConversationID] = struct{}{}
|
||||
agg.Count++
|
||||
continue
|
||||
}
|
||||
|
||||
agg.Count++
|
||||
agg.SumValue += event.Value
|
||||
agg.SumBizHours += event.ValueInBusinessHours
|
||||
}
|
||||
}
|
||||
|
||||
aggregates := make([]RollupAggregate, 0, len(groups))
|
||||
for _, agg := range groups {
|
||||
if agg.Count == 0 {
|
||||
continue
|
||||
}
|
||||
aggregates = append(aggregates, *agg)
|
||||
}
|
||||
return aggregates
|
||||
}
|
||||
|
||||
func rollupRowsFromAggregates(accountID uint, date time.Time, aggregates []RollupAggregate) []model.ReportingEventsRollup {
|
||||
var rollupRows []model.ReportingEventsRollup
|
||||
for _, agg := range aggregates {
|
||||
rollupMetrics := ExpandEventToRollupMetrics(agg.Metric, agg.Count, agg.SumValue, agg.SumBizHours)
|
||||
for rm, data := range rollupMetrics {
|
||||
rollupRows = append(rollupRows, model.ReportingEventsRollup{
|
||||
AccountID: accountID,
|
||||
Date: date,
|
||||
DimensionType: agg.DimensionType,
|
||||
DimensionID: agg.DimensionID,
|
||||
Metric: rm,
|
||||
Count: data.Count,
|
||||
SumValue: data.SumValue,
|
||||
SumValueBusinessHours: data.SumBizHours,
|
||||
})
|
||||
}
|
||||
}
|
||||
return rollupRows
|
||||
}
|
||||
@@ -13,9 +13,9 @@ import (
|
||||
// from raw ReportingEvents. Reference: Chatwoot app/services/reporting_events/rollup_service.rb (81行)
|
||||
//
|
||||
// Rollup computation:
|
||||
// 1. Group raw events by (account_id, date, dimension_type, dimension_id, metric)
|
||||
// 2. Compute COUNT, SUM(value), SUM(value_in_business_hours) per group
|
||||
// 3. Upsert into reporting_events_rollups table
|
||||
// 1. Group raw events by (account_id, date, dimension_type, dimension_id, metric)
|
||||
// 2. Compute COUNT, SUM(value), SUM(value_in_business_hours) per group
|
||||
// 3. Upsert into reporting_events_rollups table
|
||||
type ReportingRollupService struct {
|
||||
eventRepo *repository.ReportingEventRepo
|
||||
rollupRepo *repository.ReportingEventsRollupRepo
|
||||
@@ -28,8 +28,14 @@ func NewReportingRollupService(eventRepo *repository.ReportingEventRepo, rollupR
|
||||
|
||||
// ComputeDailyRollup computes rollup for a specific account and date.
|
||||
func (s *ReportingRollupService) ComputeDailyRollup(ctx context.Context, accountID uint, date time.Time) error {
|
||||
startUTC := time.Date(date.Year(), date.Month(), date.Day(), 0, 0, 0, 0, time.UTC)
|
||||
endUTC := startUTC.Add(24 * time.Hour)
|
||||
loc, enabled, err := accountReportingLocation(ctx, reportingServiceDB(s.eventRepo, s.rollupRepo), accountID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !enabled {
|
||||
return nil
|
||||
}
|
||||
startUTC, endUTC, rollupDate := utcBoundariesForReportingDate(date, loc)
|
||||
|
||||
// Get raw events for this time range
|
||||
events, err := s.eventRepo.FindByAccountIDAndTimeRange(ctx, accountID, startUTC, endUTC)
|
||||
@@ -38,15 +44,11 @@ func (s *ReportingRollupService) ComputeDailyRollup(ctx context.Context, account
|
||||
return err
|
||||
}
|
||||
|
||||
if len(events) == 0 {
|
||||
return nil // No events to rollup
|
||||
}
|
||||
|
||||
// Build rollup rows using the backfill logic
|
||||
rollupRows := s.buildRollupFromEvents(accountID, date, events)
|
||||
rollupRows := s.buildRollupFromEvents(accountID, rollupDate, events)
|
||||
|
||||
// Delete existing rollups for this date and replace
|
||||
if err := s.rollupRepo.DeleteByAccountAndDate(ctx, accountID, date); err != nil {
|
||||
if err := s.rollupRepo.DeleteByAccountAndDate(ctx, accountID, rollupDate); err != nil {
|
||||
applogger.L().Errorf("ComputeDailyRollup delete old: %v", err)
|
||||
return err
|
||||
}
|
||||
@@ -110,58 +112,5 @@ func (s *ReportingRollupService) GetSummaryMetrics(ctx context.Context, accountI
|
||||
}
|
||||
|
||||
func (s *ReportingRollupService) buildRollupFromEvents(accountID uint, date time.Time, events []model.ReportingEvent) []model.ReportingEventsRollup {
|
||||
// Group events by (dimension_type, dimension_id, raw_metric_name)
|
||||
type groupKey struct {
|
||||
dimType model.DimensionType
|
||||
dimID uint
|
||||
metric model.RollupMetric
|
||||
}
|
||||
|
||||
groups := make(map[groupKey]*RollupAggregate)
|
||||
dims := BackfillDimensions
|
||||
|
||||
for _, event := range events {
|
||||
for _, dim := range dims {
|
||||
dimID := accountID
|
||||
if dim.GroupColumn == "user_id" && event.UserID != nil {
|
||||
dimID = *event.UserID
|
||||
} else if dim.GroupColumn == "inbox_id" && event.InboxID != nil {
|
||||
dimID = *event.InboxID
|
||||
}
|
||||
|
||||
key := groupKey{dimType: dim.Type, dimID: dimID, metric: model.RollupMetric(event.Name)}
|
||||
agg, ok := groups[key]
|
||||
if !ok {
|
||||
agg = &RollupAggregate{
|
||||
DimensionType: dim.Type,
|
||||
DimensionID: dimID,
|
||||
Metric: model.RollupMetric(event.Name),
|
||||
}
|
||||
groups[key] = agg
|
||||
}
|
||||
agg.Count++
|
||||
agg.SumValue += event.Value
|
||||
agg.SumBizHours += event.ValueInBusinessHours
|
||||
}
|
||||
}
|
||||
|
||||
// Expand each group into rollup metrics
|
||||
var rollupRows []model.ReportingEventsRollup
|
||||
for _, agg := range groups {
|
||||
rollupMetrics := ExpandEventToRollupMetrics(agg.Metric, agg.Count, agg.SumValue, agg.SumBizHours)
|
||||
for rm, data := range rollupMetrics {
|
||||
rollupRows = append(rollupRows, model.ReportingEventsRollup{
|
||||
AccountID: accountID,
|
||||
Date: date,
|
||||
DimensionType: agg.DimensionType,
|
||||
DimensionID: agg.DimensionID,
|
||||
Metric: rm,
|
||||
Count: data.Count,
|
||||
SumValue: data.SumValue,
|
||||
SumValueBusinessHours: data.SumBizHours,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
return rollupRows
|
||||
}
|
||||
return rollupRowsFromAggregates(accountID, date, aggregateReportingEvents(accountID, events))
|
||||
}
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gochat/gochat/internal/model"
|
||||
"github.com/gochat/gochat/internal/repository"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func setupReportingRollupTZTest(t *testing.T) (*gorm.DB, *ReportingRollupService, *ReportingBackfillService) {
|
||||
t.Helper()
|
||||
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=private"), &gorm.Config{})
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, db.AutoMigrate(
|
||||
&model.Account{},
|
||||
&model.User{},
|
||||
&model.Inbox{},
|
||||
&model.Contact{},
|
||||
&model.Conversation{},
|
||||
&model.ReportingEvent{},
|
||||
&model.ReportingEventsRollup{},
|
||||
))
|
||||
t.Cleanup(func() {
|
||||
sqlDB, _ := db.DB()
|
||||
sqlDB.Close()
|
||||
})
|
||||
eventRepo := repository.NewReportingEventRepo(db)
|
||||
rollupRepo := repository.NewReportingEventsRollupRepo(db)
|
||||
return db, NewReportingRollupService(eventRepo, rollupRepo), NewReportingBackfillService(eventRepo, rollupRepo)
|
||||
}
|
||||
|
||||
func createReportingAccount(t *testing.T, db *gorm.DB, reportingTimezone string) *model.Account {
|
||||
t.Helper()
|
||||
account := &model.Account{Name: "Reports", ReportingTimezone: reportingTimezone}
|
||||
require.NoError(t, db.Create(account).Error)
|
||||
return account
|
||||
}
|
||||
|
||||
func createReportingActorRows(t *testing.T, db *gorm.DB, accountID uint) (*model.User, *model.Inbox, *model.Conversation) {
|
||||
t.Helper()
|
||||
user := &model.User{AccountID: accountID, Name: "Agent", Email: "agent@example.com", Password: "secret", Active: true}
|
||||
require.NoError(t, db.Create(user).Error)
|
||||
inbox := &model.Inbox{AccountID: accountID, Name: "Web", ChannelType: "web_widget", ChannelID: 1, Enabled: true}
|
||||
require.NoError(t, db.Create(inbox).Error)
|
||||
contact := &model.Contact{AccountID: accountID, Name: "Customer"}
|
||||
require.NoError(t, db.Create(contact).Error)
|
||||
conversation := &model.Conversation{AccountID: accountID, InboxID: inbox.ID, ContactID: contact.ID, AssigneeID: &user.ID, ChannelType: inbox.ChannelType, Channel: inbox.ChannelType}
|
||||
require.NoError(t, db.Create(conversation).Error)
|
||||
return user, inbox, conversation
|
||||
}
|
||||
|
||||
func createReportingEvent(t *testing.T, db *gorm.DB, accountID uint, name string, createdAt time.Time, userID, inboxID, conversationID *uint) {
|
||||
t.Helper()
|
||||
require.NoError(t, db.Create(&model.ReportingEvent{
|
||||
Base: model.Base{CreatedAt: createdAt},
|
||||
AccountID: accountID,
|
||||
Name: name,
|
||||
Value: 100,
|
||||
ValueInBusinessHours: 50,
|
||||
UserID: userID,
|
||||
InboxID: inboxID,
|
||||
ConversationID: conversationID,
|
||||
EventStartTime: createdAt.AddDate(0, 0, -3),
|
||||
EventEndTime: createdAt.AddDate(0, 0, -3).Add(time.Minute),
|
||||
}).Error)
|
||||
}
|
||||
|
||||
func TestReportingRollupSkipsBlankAndInvalidReportingTimezone(t *testing.T) {
|
||||
db, rollups, _ := setupReportingRollupTZTest(t)
|
||||
blank := createReportingAccount(t, db, "")
|
||||
invalid := createReportingAccount(t, db, "Invalid/Timezone")
|
||||
|
||||
createReportingEvent(t, db, blank.ID, model.MetricNameFirstResponse, time.Date(2026, 2, 11, 15, 0, 0, 0, time.UTC), nil, nil, nil)
|
||||
createReportingEvent(t, db, invalid.ID, model.MetricNameFirstResponse, time.Date(2026, 2, 11, 15, 0, 0, 0, time.UTC), nil, nil, nil)
|
||||
|
||||
require.NoError(t, rollups.ComputeDailyRollup(context.Background(), blank.ID, time.Date(2026, 2, 11, 0, 0, 0, 0, time.UTC)))
|
||||
require.NoError(t, rollups.ComputeDailyRollup(context.Background(), invalid.ID, time.Date(2026, 2, 11, 0, 0, 0, 0, time.UTC)))
|
||||
|
||||
var count int64
|
||||
require.NoError(t, db.Model(&model.ReportingEventsRollup{}).Count(&count).Error)
|
||||
assert.Equal(t, int64(0), count)
|
||||
}
|
||||
|
||||
func TestReportingRollupUsesReportingTimezoneCreatedAtBoundaries(t *testing.T) {
|
||||
db, rollups, _ := setupReportingRollupTZTest(t)
|
||||
account := createReportingAccount(t, db, "America/New_York")
|
||||
user, inbox, conversation := createReportingActorRows(t, db, account.ID)
|
||||
createdAt := time.Date(2026, 2, 12, 4, 0, 0, 0, time.UTC)
|
||||
createReportingEvent(t, db, account.ID, "conversation_resolved", createdAt, &user.ID, &inbox.ID, &conversation.ID)
|
||||
|
||||
require.NoError(t, rollups.ComputeDailyRollup(context.Background(), account.ID, time.Date(2026, 2, 11, 0, 0, 0, 0, time.UTC)))
|
||||
|
||||
rollupDate := time.Date(2026, 2, 11, 0, 0, 0, 0, time.UTC)
|
||||
var resolutionTime model.ReportingEventsRollup
|
||||
require.NoError(t, db.Where("account_id = ? AND date = ? AND dimension_type = ? AND dimension_id = ? AND metric = ?", account.ID, rollupDate, model.DimensionAccount, account.ID, model.MetricResolutionTime).First(&resolutionTime).Error)
|
||||
assert.Equal(t, int64(1), resolutionTime.Count)
|
||||
assert.Equal(t, 100.0, resolutionTime.SumValue)
|
||||
|
||||
var resolutionsCount model.ReportingEventsRollup
|
||||
require.NoError(t, db.Where("account_id = ? AND date = ? AND dimension_type = ? AND dimension_id = ? AND metric = ?", account.ID, rollupDate, model.DimensionAccount, account.ID, model.MetricResolutionsCount).First(&resolutionsCount).Error)
|
||||
assert.Equal(t, int64(1), resolutionsCount.Count)
|
||||
assert.Equal(t, 0.0, resolutionsCount.SumValue)
|
||||
}
|
||||
|
||||
func TestReportingRollupSkipsNilAgentAndInboxDimensions(t *testing.T) {
|
||||
db, rollups, _ := setupReportingRollupTZTest(t)
|
||||
account := createReportingAccount(t, db, "America/New_York")
|
||||
createReportingEvent(t, db, account.ID, model.MetricNameFirstResponse, time.Date(2026, 2, 11, 15, 0, 0, 0, time.UTC), nil, nil, nil)
|
||||
|
||||
require.NoError(t, rollups.ComputeDailyRollup(context.Background(), account.ID, time.Date(2026, 2, 11, 0, 0, 0, 0, time.UTC)))
|
||||
|
||||
var accountRows int64
|
||||
require.NoError(t, db.Model(&model.ReportingEventsRollup{}).Where("account_id = ? AND dimension_type = ?", account.ID, model.DimensionAccount).Count(&accountRows).Error)
|
||||
assert.Equal(t, int64(1), accountRows)
|
||||
|
||||
var nilDimensionFallbacks int64
|
||||
require.NoError(t, db.Model(&model.ReportingEventsRollup{}).Where("account_id = ? AND dimension_type IN ?", account.ID, []model.DimensionType{model.DimensionAgent, model.DimensionInbox}).Count(&nilDimensionFallbacks).Error)
|
||||
assert.Equal(t, int64(0), nilDimensionFallbacks)
|
||||
}
|
||||
|
||||
func TestReportingBackfillUsesReportingTimezoneAndDistinctBotHandoffs(t *testing.T) {
|
||||
db, _, backfill := setupReportingRollupTZTest(t)
|
||||
account := createReportingAccount(t, db, "America/New_York")
|
||||
user, inbox, conversationA := createReportingActorRows(t, db, account.ID)
|
||||
conversationB := &model.Conversation{AccountID: account.ID, InboxID: inbox.ID, ContactID: conversationA.ContactID, AssigneeID: &user.ID, ChannelType: inbox.ChannelType, Channel: inbox.ChannelType}
|
||||
require.NoError(t, db.Create(conversationB).Error)
|
||||
|
||||
createReportingEvent(t, db, account.ID, "conversation_bot_handoff", time.Date(2026, 2, 11, 4, 59, 0, 0, time.UTC), &user.ID, &inbox.ID, &conversationA.ID)
|
||||
createReportingEvent(t, db, account.ID, "conversation_bot_handoff", time.Date(2026, 2, 11, 5, 0, 0, 0, time.UTC), &user.ID, &inbox.ID, &conversationA.ID)
|
||||
createReportingEvent(t, db, account.ID, "conversation_bot_handoff", time.Date(2026, 2, 11, 6, 0, 0, 0, time.UTC), &user.ID, &inbox.ID, &conversationA.ID)
|
||||
createReportingEvent(t, db, account.ID, "conversation_bot_handoff", time.Date(2026, 2, 12, 4, 59, 0, 0, time.UTC), &user.ID, &inbox.ID, &conversationB.ID)
|
||||
createReportingEvent(t, db, account.ID, "conversation_bot_handoff", time.Date(2026, 2, 12, 5, 0, 0, 0, time.UTC), &user.ID, &inbox.ID, &conversationB.ID)
|
||||
|
||||
require.NoError(t, backfill.BackfillDate(context.Background(), account.ID, time.Date(2026, 2, 11, 0, 0, 0, 0, time.UTC)))
|
||||
|
||||
rollupDate := time.Date(2026, 2, 11, 0, 0, 0, 0, time.UTC)
|
||||
for _, dimension := range []struct {
|
||||
typ model.DimensionType
|
||||
id uint
|
||||
}{
|
||||
{model.DimensionAccount, account.ID},
|
||||
{model.DimensionAgent, user.ID},
|
||||
{model.DimensionInbox, inbox.ID},
|
||||
} {
|
||||
var rollup model.ReportingEventsRollup
|
||||
require.NoError(t, db.Where("account_id = ? AND date = ? AND dimension_type = ? AND dimension_id = ? AND metric = ?", account.ID, rollupDate, dimension.typ, dimension.id, model.MetricBotHandoffsCount).First(&rollup).Error)
|
||||
assert.Equal(t, int64(2), rollup.Count)
|
||||
assert.Equal(t, 0.0, rollup.SumValue)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
ALTER TABLE accounts DROP COLUMN IF EXISTS reporting_timezone;
|
||||
@@ -0,0 +1 @@
|
||||
ALTER TABLE accounts ADD COLUMN IF NOT EXISTS reporting_timezone VARCHAR(100);
|
||||
Reference in New Issue
Block a user