Commit Graph
5 Commits
Author SHA1 Message Date
rogee 6d8eda28b5 Phase 3.4: Context window optimization — token estimation + sliding window
- token_estimator.go (new): TokenEstimator with ~4 chars/token heuristic,
  EstimateText/EstimateMessages methods, TruncateMessages sliding window
  that drops oldest messages to fit token budget, BuildContextWindow
  entry point that converts conversation messages to LLM format with
  token-budgeted truncation (default 4096 tokens)
- copilot_context_service.go: fetch up to 100 messages (was 20), then
  apply BuildContextWindow truncation to fit within 4096 token budget;
  log how many messages were dropped

Verified: go build + go vet + go test all pass
Semantic search route reaches handler (times out on LLM call without API key,
confirming route + service wiring is correct)
2026-07-08 15:38:08 +08:00
rogee b2f36a20a0 Phase 3.2: Function Calling — tool_call loop for AI services
- llm/provider.go: extend ChatMessage with ToolCalls, ToolCallID, Name
  fields; add ToolCall + ToolCallFunction structs for parsing LLM
  function call responses
- tool_execution_service.go (new): ToolExecutionService that converts
  CaptainCustomTool → LLM ToolDefinition, executes HTTP tool calls
  (GET/POST/PUT with bearer/basic/api-key auth), and runs the full
  tool_call loop (LLM → tool_call → execute → result → LLM → final
  answer) with maxIterations safeguard
- captain_conversation_service.go: add toolExecSvc field +
  SetToolExecutionService method; use RunToolCallLoop in
  generateConversationResponse when tools are available, with graceful
  fallback to plain LLM call on error
- bootstrap.go: instantiate ToolExecutionService and inject into
  CaptainConversationService

Verified: go build + go vet + go test all pass
2026-07-08 15:38:08 +08:00
rogee c72e359e48 Phase 3.1: Help Center semantic search with pgvector
- ArticleEmbeddingRepo (new): Upsert, GetByArticleID, DeleteByArticleID,
  SearchByEmbedding using pgvector cosine distance (vector_embedding column)
- ArticleEmbedding model: add VectorEmbedding pgvector.Vector field
  alongside existing JSONB Embedding (backward compatible)
- ArticleService: add SemanticSearch() — generates query embedding via LLM,
  searches articles by cosine similarity; add GenerateEmbedding() — creates
  and stores article embedding from title+description+content
- ArticleHandler: add SemanticSearch endpoint
  GET /portals/:portal_id/articles/semantic_search?query=...
- bootstrap.go: inject articleEmbeddingRepo + llmProvider into ArticleService
- router.go: register /articles/semantic_search route
- migration 000049: add vector(1536) column to article_embeddings table,
  create ivfflat index, migrate existing JSONB data to vector format

Verified: go build + go vet + go test all pass
2026-07-08 15:38:08 +08:00
rogee b769b9a3e4 Phase 2: AutoReplyRule integration + AgentBot Captain type
AutoReplyRule complete integration:
- bootstrap.go: instantiate AutoReplyRuleService + AutoReplyListener,
  register listener on channel dispatcher for message.created events
- router.go: register /captain/auto_reply_rules CRUD + /evaluate routes
- auto_reply_rule_handler.go: fix c.Param(id) → c.Param(account_id),
  override evalCtx.AccountID from path param
- auto_reply_rule_service.go: add JSON tags to AutoReplyEvaluationContext
  for correct request body binding
- auto_reply_listener.go (new): EventListener that triggers on incoming
  messages, evaluates active rules, composes reply (static/LLM/mixed),
  respects DelaySeconds and OneTimeOnly flags, sends via MessageService
- migration 000048: create captain_auto_reply_rules table

AgentBot + Captain integration:
- agent_bot_listener.go: add captainConvSvc field + SetCaptainConversationService
  method. In HandleEvent loop, check bot.BotType == captain and route
  to CaptainConversationService.BuildConversationResponseByAccount
  instead of webhook push. Extract assistant_id from bot.Config JSONB,
  extract conversation_id from event data.
- bootstrap.go: inject captainConversationService into agentBotListener

CaptainConversationService improvement:
- generateConversationResponse: use assistant config for system prompt,
  model name, and temperature instead of hardcoded values

Verified:
- go build ./... passes
- go vet passes on all internal packages
- go test passes (service + repository + llm, SQLite mode)
- Auto-reply CRUD: create/get/update/delete all work
- Auto-reply evaluate: correctly matches hello → should_reply=true,
  correctly rejects non-matching message
- Existing routes unaffected (assistants, RAG, conversation respond)
- Migration 000048 creates captain_auto_reply_rules table successfully
2026-07-08 15:38:08 +08:00
rogee aeddedf2a3 Reorganize repo: backend/, deploy/, docs/ layout + AGENTS.md
Restructure the monorepo into clear top-level directories:
- backend/: Go module root (cmd, internal, pkg, configs, migrations,
  docs/swagger, scripts, tests, go.mod, Makefile, .air.toml)
- deploy/: Docker (Dockerfile, docker-compose*), quickstart, fluentd
- docs/: project documentation + reports/ (moved from repo root)
- AGENTS.md: new AI coding-agent guide at repo root

Update all references to the new layout:
- Dockerfile: COPY backend/go.mod, COPY backend/ (context = repo root)
- docker-compose files: context ../.., dockerfile deploy/docker/Dockerfile,
  env_file ../../.env, volume mounts ../../backend:/app
- deploy/quickstart/compose.yaml: dockerfile deploy/docker/Dockerfile
- CI: working-directory: backend for go commands, file deploy/docker/Dockerfile,
  coverage path backend/coverage.out, health_check backend/scripts/
- backend/Makefile: docker target uses -f ../deploy/docker/Dockerfile ../
- README: architecture tree, quickstart, config paths updated

Move root stray scripts (rename_models.*, run_m11_tests.sh, verify_build.sh,
gorm_bool_main.go) to backend/scripts/legacy/. All moves via git mv to
preserve history. Build, vet, SQLite tests, and docker compose config verified.
2026-07-07 14:44:12 +08:00