rogee
0dabb8cfa5
清理: - 删除 34 份过时文档（gap reports/QA临时报告/验收报告/阶段性文档） - 删除 docs/.hermes/skills 第三方 skills 副本（16 文件） - 删除 skills-lock.json 目录归集: - 根目录仅保留 README.md 索引 - product/ — 产品与架构设计（PRD + ARCHITECTURE + P2设计文档 + AI/企业路线图） - tracking/ — Chatwoot parity 开发跟踪 - requirements/ — M01-M12 模块需求 - plans/ — 历史实现计划 - parity/ — 路由 parity 与前端契约 - qa/ — QA 报告与测试计划 - ops/ — 运维部署 命名规范: - 全小写 kebab-case，禁止全大写文件名 - product/tracking/ops 用 NN- 序号前缀 - requirements 用 MNN- 两位零填充模块号 - plans/qa 用 YYYY-MM-DD- 日期前缀 - requirements M1-M9 零填充为 M01-M09（修复字典序） 同步更新: - backend/cmd/route_parity/main.go 路径默认值 - backend/scripts/parity_frontend_smoke.sh 报告路径 - 所有 docs 内部交叉引用 - .gitignore 排除编译产物 (backend/gochat, backend/route_parity) - 新增迁移 000052/000053 - 前端 WS 相关修改
452 lines
15 KiB
Markdown
452 lines
15 KiB
Markdown
## Code Context
|
|
|
|
**Query:** knowledge base, portal, article, category, folder, related category, article embedding, help center, kbase search
|
|
|
|
### Entry Points
|
|
|
|
- **Onboarding::HelpCenterArticleBuilder** (class) - enterprise/app/services/onboarding/help_center_article_builder.rb:1
|
|
- **Onboarding::HelpCenterArticleWriterJob** (class) - enterprise/app/jobs/onboarding/help_center_article_writer_job.rb:1
|
|
- **Onboarding::HelpCenterArticleGenerationJob** (class) - enterprise/app/jobs/onboarding/help_center_article_generation_job.rb:1
|
|
|
|
### Related Symbols
|
|
|
|
- app/jobs/application_job.rb: ApplicationJob:1
|
|
- enterprise/app/services/onboarding/help_center_article_builder.rb: initialize:4, perform:15, scrape:36, normalize:44, rewrite:57
|
|
- enterprise/app/jobs/onboarding/help_center_article_writer_job.rb: perform:12, on_writer_failure:27, failure_context:33, finalize:38
|
|
|
|
### Code
|
|
|
|
#### Onboarding::HelpCenterArticleBuilder (enterprise/app/services/onboarding/help_center_article_builder.rb:1)
|
|
|
|
```ruby
|
|
class Onboarding::HelpCenterArticleBuilder
|
|
BuildFailed = Onboarding::HelpCenterErrors::ArticleBuildFailed
|
|
|
|
def initialize(account:, portal:, user:, article:)
|
|
@account = account
|
|
@portal = portal
|
|
@user = user
|
|
|
|
spec = article.with_indifferent_access
|
|
@urls = Array(spec[:urls]).map(&:to_s).reject(&:blank?)
|
|
@title = spec[:title]
|
|
@category_id = spec[:category_id]
|
|
end
|
|
|
|
def perform
|
|
raise BuildFailed, 'no source urls supplied' if @urls.empty?
|
|
|
|
source_pages = scrape(@urls)
|
|
raise BuildFailed, "scrape produced no usable pages for #{@urls.join(', ')}" if source_pages.empty?
|
|
|
|
payload = rewrite(source_pages)
|
|
|
|
@portal.articles.create!(
|
|
title: payload[:title],
|
|
description: payload[:description].presence,
|
|
content: payload[:content],
|
|
author_id: @user.id,
|
|
category_id: @category_id,
|
|
status: :draft,
|
|
meta: { source_urls: source_pages.pluck(:url) }
|
|
)
|
|
end
|
|
|
|
private
|
|
|
|
def scrape(urls)
|
|
job = Firecrawl::Configuration.client.batch_scrape(
|
|
urls,
|
|
Firecrawl::Models::BatchScrapeOptions.new(options: Firecrawl::Configuration.default_scrape_options)
|
|
)
|
|
Array(job.data).filter_map { |doc| normalize(doc) }
|
|
end
|
|
|
|
def normalize(doc)
|
|
metadata = doc&.metadata || {}
|
|
status = metadata['statusCode']
|
|
return nil if status.present? && !(200..299).cover?(status)
|
|
return nil if doc.markdown.to_s.blank?
|
|
|
|
{
|
|
url: metadata['sourceURL'] || metadata['url'],
|
|
markdown: doc.m
|
|
// ... truncated ...
|
|
```
|
|
|
|
#### Onboarding::HelpCenterArticleWriterJob (enterprise/app/jobs/onboarding/help_center_article_writer_job.rb:1)
|
|
|
|
```ruby
|
|
class Onboarding::HelpCenterArticleWriterJob < ApplicationJob
|
|
queue_as :low
|
|
|
|
retry_on Firecrawl::FirecrawlError, wait: :polynomially_longer, attempts: 3 do |job, error|
|
|
job.send(:on_writer_failure, error)
|
|
end
|
|
|
|
discard_on Onboarding::HelpCenterErrors::ArticleBuildFailed do |job, error|
|
|
job.send(:on_writer_failure, error)
|
|
end
|
|
|
|
def perform(account_id, portal_id, user_id, generation_id, article_payload)
|
|
user = User.find(user_id)
|
|
payload = article_payload.with_indifferent_access
|
|
article = Onboarding::HelpCenterArticleBuilder.new(
|
|
account: Account.find(account_id),
|
|
portal: Portal.find(portal_id),
|
|
user: user,
|
|
article: payload[:article]
|
|
).perform
|
|
|
|
finalize(user: user, generation_id: generation_id, article: article)
|
|
end
|
|
|
|
private
|
|
|
|
def on_writer_failure(error)
|
|
user, generation_id = failure_context
|
|
Rails.logger.warn "[HelpCenterWriterJob] gen=#{generation_id} failed: #{error.class} #{error.message}"
|
|
finalize(user: user, generation_id: generation_id, article: nil)
|
|
end
|
|
|
|
def failure_context
|
|
_account_id, _portal_id, user_id, generation_id = arguments
|
|
[User.find_by(id: user_id), generation_id]
|
|
end
|
|
|
|
def finalize(user:, generation_id:, article:)
|
|
result = Onboarding::HelpCenterGenerationState.record_article_finished(generation_id)
|
|
|
|
if article
|
|
Onboarding::HelpCenterBroadcaster.article_generated(
|
|
user: user, generation_id: generation_id, article: article, articles_finished: resul
|
|
// ... truncated ...
|
|
```
|
|
|
|
#### Onboarding::HelpCenterArticleGenerationJob (enterprise/app/jobs/onboarding/help_center_article_generation_job.rb:1)
|
|
|
|
```ruby
|
|
class Onboarding::HelpCenterArticleGenerationJob < ApplicationJob
|
|
queue_as :low
|
|
|
|
retry_on Firecrawl::FirecrawlError, wait: :polynomially_longer, attempts: 3 do |job, error|
|
|
_account_id, _portal_id, user_id, generation_id = job.arguments
|
|
reason = "firecrawl exhausted: #{error.message}"
|
|
Rails.logger.warn "[HelpCenterGenerationJob] gen=#{generation_id} #{reason}"
|
|
job.send(:skip_and_broadcast, user: User.find_by(id: user_id), generation_id: generation_id, reason: reason)
|
|
end
|
|
|
|
def perform(account_id, portal_id, user_id, generation_id)
|
|
return if Onboarding::HelpCenterGenerationState.current(generation_id).present?
|
|
|
|
process(
|
|
account: Account.find(account_id),
|
|
portal: Portal.find(portal_id),
|
|
user: User.find(user_id),
|
|
generation_id: generation_id
|
|
)
|
|
rescue Onboarding::HelpCenterErrors::CurationSkipped => e
|
|
Rails.logger.info "[HelpCenterGenerationJob] gen=#{generation_id} skipped: #{e.message}"
|
|
skip_and_broadcast(user: User.find_by(id: user_id), generation_id: generation_id, reason: e.message)
|
|
end
|
|
|
|
private
|
|
|
|
def process(account:, portal:, user:, generation_id:)
|
|
plan = Onboarding::HelpCenterCurator.new(account: account).perform
|
|
articles = create_categories_and_build_article_payloads(portal, plan)
|
|
|
|
Onboarding::HelpCenterGenerationState.start(generation_id, total: articles.size)
|
|
enqueue_writer_jobs(
|
|
account_id: account.id,
|
|
portal_id: portal.id,
|
|
user_id: user.id,
|
|
generation_id:
|
|
// ... truncated ...
|
|
```
|
|
|
|
#### initialize (enterprise/app/services/onboarding/help_center_article_builder.rb:4)
|
|
|
|
```ruby
|
|
def initialize(account:, portal:, user:, article:)
|
|
@account = account
|
|
@portal = portal
|
|
@user = user
|
|
|
|
spec = article.with_indifferent_access
|
|
@urls = Array(spec[:urls]).map(&:to_s).reject(&:blank?)
|
|
@title = spec[:title]
|
|
@category_id = spec[:category_id]
|
|
end
|
|
```
|
|
|
|
#### perform (enterprise/app/services/onboarding/help_center_article_builder.rb:15)
|
|
|
|
```ruby
|
|
def perform
|
|
raise BuildFailed, 'no source urls supplied' if @urls.empty?
|
|
|
|
source_pages = scrape(@urls)
|
|
raise BuildFailed, "scrape produced no usable pages for #{@urls.join(', ')}" if source_pages.empty?
|
|
|
|
payload = rewrite(source_pages)
|
|
|
|
@portal.articles.create!(
|
|
title: payload[:title],
|
|
description: payload[:description].presence,
|
|
content: payload[:content],
|
|
author_id: @user.id,
|
|
category_id: @category_id,
|
|
status: :draft,
|
|
meta: { source_urls: source_pages.pluck(:url) }
|
|
)
|
|
end
|
|
```
|
|
|
|
#### scrape (enterprise/app/services/onboarding/help_center_article_builder.rb:36)
|
|
|
|
```ruby
|
|
def scrape(urls)
|
|
job = Firecrawl::Configuration.client.batch_scrape(
|
|
urls,
|
|
Firecrawl::Models::BatchScrapeOptions.new(options: Firecrawl::Configuration.default_scrape_options)
|
|
)
|
|
Array(job.data).filter_map { |doc| normalize(doc) }
|
|
end
|
|
```
|
|
|
|
#### normalize (enterprise/app/services/onboarding/help_center_article_builder.rb:44)
|
|
|
|
```ruby
|
|
def normalize(doc)
|
|
metadata = doc&.metadata || {}
|
|
status = metadata['statusCode']
|
|
return nil if status.present? && !(200..299).cover?(status)
|
|
return nil if doc.markdown.to_s.blank?
|
|
|
|
{
|
|
url: metadata['sourceURL'] || metadata['url'],
|
|
markdown: doc.markdown.to_s,
|
|
page_title: metadata['title'].to_s.strip
|
|
}
|
|
end
|
|
```
|
|
|
|
#### rewrite (enterprise/app/services/onboarding/help_center_article_builder.rb:57)
|
|
|
|
```ruby
|
|
def rewrite(source_pages)
|
|
response = Captain::Llm::ArticleWriterService.new(
|
|
account: @account,
|
|
source_pages: source_pages,
|
|
hint_title: @title.presence || source_pages.first[:page_title]
|
|
).perform
|
|
raise BuildFailed, "writer LLM error: #{response[:error]}" if response[:error]
|
|
|
|
payload = response[:message] || {}
|
|
raise BuildFailed, 'writer returned blank content' if payload[:content].blank?
|
|
raise BuildFailed, 'writer returned blank title' if payload[:title].blank?
|
|
|
|
payload
|
|
end
|
|
```
|
|
|
|
#### perform (enterprise/app/jobs/onboarding/help_center_article_writer_job.rb:12)
|
|
|
|
```ruby
|
|
def perform(account_id, portal_id, user_id, generation_id, article_payload)
|
|
user = User.find(user_id)
|
|
payload = article_payload.with_indifferent_access
|
|
article = Onboarding::HelpCenterArticleBuilder.new(
|
|
account: Account.find(account_id),
|
|
portal: Portal.find(portal_id),
|
|
user: user,
|
|
article: payload[:article]
|
|
).perform
|
|
|
|
finalize(user: user, generation_id: generation_id, article: article)
|
|
end
|
|
```
|
|
|
|
#### on_writer_failure (enterprise/app/jobs/onboarding/help_center_article_writer_job.rb:27)
|
|
|
|
```ruby
|
|
def on_writer_failure(error)
|
|
user, generation_id = failure_context
|
|
Rails.logger.warn "[HelpCenterWriterJob] gen=#{generation_id} failed: #{error.class} #{error.message}"
|
|
finalize(user: user, generation_id: generation_id, article: nil)
|
|
end
|
|
```
|
|
|
|
#### failure_context (enterprise/app/jobs/onboarding/help_center_article_writer_job.rb:33)
|
|
|
|
```ruby
|
|
def failure_context
|
|
_account_id, _portal_id, user_id, generation_id = arguments
|
|
[User.find_by(id: user_id), generation_id]
|
|
end
|
|
```
|
|
|
|
#### finalize (enterprise/app/jobs/onboarding/help_center_article_writer_job.rb:38)
|
|
|
|
```ruby
|
|
def finalize(user:, generation_id:, article:)
|
|
result = Onboarding::HelpCenterGenerationState.record_article_finished(generation_id)
|
|
|
|
if article
|
|
Onboarding::HelpCenterBroadcaster.article_generated(
|
|
user: user, generation_id: generation_id, article: article, articles_finished: result[:finished]
|
|
)
|
|
end
|
|
return unless result[:completed]
|
|
|
|
Onboarding::HelpCenterBroadcaster.completed(user: user, generation_id: generation_id, status: 'completed')
|
|
rescue Onboarding::HelpCenterGenerationState::Missing => e
|
|
Rails.logger.warn "[HelpCenterWriterJob] gen=#{generation_id} #{e.message}"
|
|
end
|
|
```
|
|
|
|
#### enqueue_writer_jobs (enterprise/app/jobs/onboarding/help_center_article_generation_job.rb:89)
|
|
|
|
```ruby
|
|
def enqueue_writer_jobs(account_id:, portal_id:, user_id:, generation_id:, articles:)
|
|
articles.each do |article|
|
|
Onboarding::HelpCenterArticleWriterJob.perform_later(
|
|
account_id, portal_id, user_id, generation_id, { article: article }
|
|
)
|
|
end
|
|
end
|
|
```
|
|
|
|
#### perform (enterprise/app/jobs/onboarding/help_center_article_generation_job.rb:11)
|
|
|
|
```ruby
|
|
def perform(account_id, portal_id, user_id, generation_id)
|
|
return if Onboarding::HelpCenterGenerationState.current(generation_id).present?
|
|
|
|
process(
|
|
account: Account.find(account_id),
|
|
portal: Portal.find(portal_id),
|
|
user: User.find(user_id),
|
|
generation_id: generation_id
|
|
)
|
|
rescue Onboarding::HelpCenterErrors::CurationSkipped => e
|
|
Rails.logger.info "[HelpCenterGenerationJob] gen=#{generation_id} skipped: #{e.message}"
|
|
skip_and_broadcast(user: User.find_by(id: user_id), generation_id: generation_id, reason: e.message)
|
|
end
|
|
```
|
|
|
|
#### process (enterprise/app/jobs/onboarding/help_center_article_generation_job.rb:27)
|
|
|
|
```ruby
|
|
def process(account:, portal:, user:, generation_id:)
|
|
plan = Onboarding::HelpCenterCurator.new(account: account).perform
|
|
articles = create_categories_and_build_article_payloads(portal, plan)
|
|
|
|
Onboarding::HelpCenterGenerationState.start(generation_id, total: articles.size)
|
|
enqueue_writer_jobs(
|
|
account_id: account.id,
|
|
portal_id: portal.id,
|
|
user_id: user.id,
|
|
generation_id: generation_id,
|
|
articles: articles
|
|
)
|
|
end
|
|
```
|
|
|
|
#### create_categories_and_build_article_payloads (enterprise/app/jobs/onboarding/help_center_article_generation_job.rb:41)
|
|
|
|
```ruby
|
|
def create_categories_and_build_article_payloads(portal, plan)
|
|
ActiveRecord::Base.transaction do
|
|
categories_by_name = create_categories(portal, plan['categories'])
|
|
articles = build_article_payloads(
|
|
plan['articles'],
|
|
categories_by_name,
|
|
plan['allowed_urls']
|
|
)
|
|
|
|
if articles.empty?
|
|
raise Onboarding::HelpCenterErrors::CurationSkipped,
|
|
'no articles after category or URL filtering'
|
|
end
|
|
|
|
articles
|
|
end
|
|
end
|
|
```
|
|
|
|
#### create_categories (enterprise/app/jobs/onboarding/help_center_article_generation_job.rb:59)
|
|
|
|
```ruby
|
|
def create_categories(portal, categories)
|
|
locale = portal.default_locale
|
|
Array(categories).each_with_index.with_object({}) do |(cat, idx), acc|
|
|
name = cat['name'].to_s.strip
|
|
next if name.blank?
|
|
|
|
record = portal.categories.create!(
|
|
name: name,
|
|
description: cat['description'].to_s.strip.presence,
|
|
slug: "#{name.parameterize}-#{SecureRandom.hex(3)}",
|
|
locale: locale,
|
|
position: (idx + 1) * 10
|
|
)
|
|
acc[name] = record
|
|
end
|
|
end
|
|
```
|
|
|
|
#### build_article_payloads (enterprise/app/jobs/onboarding/help_center_article_generation_job.rb:76)
|
|
|
|
```ruby
|
|
def build_article_payloads(articles, categories_by_name, allowed_urls)
|
|
allowed_urls = Array(allowed_urls).to_set
|
|
Array(articles).filter_map do |article|
|
|
category_id = categories_by_name[article['category_name'].to_s]&.id
|
|
next if category_id.nil?
|
|
|
|
urls = Array(article['urls']).select { |url| allowed_urls.include?(url) }
|
|
next if urls.empty?
|
|
|
|
article.merge('category_id' => category_id, 'urls' => urls)
|
|
end
|
|
end
|
|
```
|
|
|
|
#### skip_and_broadcast (enterprise/app/jobs/onboarding/help_center_article_generation_job.rb:97)
|
|
|
|
```ruby
|
|
def skip_and_broadcast(user:, generation_id:, reason:)
|
|
Onboarding::HelpCenterGenerationState.skip(generation_id, reason: reason)
|
|
Onboarding::HelpCenterBroadcaster.completed(
|
|
user: user, generation_id: generation_id, status: 'skipped', skip_reason: reason
|
|
)
|
|
end
|
|
```
|
|
|
|
#### enqueue_article_generation (enterprise/app/services/onboarding/help_center_creation_service.rb:75)
|
|
|
|
```ruby
|
|
def enqueue_article_generation(portal)
|
|
return if homepage_link.blank?
|
|
|
|
generation_id = SecureRandom.uuid
|
|
Onboarding::HelpCenterArticleGenerationJob.perform_later(@account.id, portal.id, @user.id, generation_id)
|
|
rescue StandardError => e
|
|
Rails.logger.error "[HelpCenterCreation] Failed to enqueue article generation for account #{@account.id}: #{e.class} - #{e.message}"
|
|
end
|
|
```
|
|
|
|
#### ApplicationJob (app/jobs/application_job.rb:1)
|
|
|
|
```ruby
|
|
class ApplicationJob < ActiveJob::Base
|
|
# https://api.rubyonrails.org/v5.2.1/classes/ActiveJob/Exceptions/ClassMethods.html
|
|
discard_on ActiveJob::DeserializationError do |job, error|
|
|
Rails.logger.info("Skipping #{job.class} with #{
|
|
job.instance_variable_get(:@serialized_arguments)
|
|
} because of ActiveJob::DeserializationError (#{error.message})")
|
|
end
|
|
end
|
|
``` |