4 Commits

Author SHA1 Message Date
9a57949147 Fix upstream timeout for large downloads
Some checks failed
docker-release / build-and-push (push) Failing after 25m53s
2025-12-12 17:36:37 +08:00
3685b2129a fix: update content type handling for release.gpg 2025-11-18 16:13:05 +08:00
fc2c46a9df Refactor module binding to rely on Type 2025-11-18 16:11:13 +08:00
347eb3adc5 feat: remove module/rollout config key 2025-11-18 15:37:21 +08:00
41 changed files with 221 additions and 635 deletions

View File

@@ -38,7 +38,7 @@ Password = "s3cr3t"
1. 复制 `configs/config.example.toml` 为工作目录下的 `config.toml` 并调整 `[[Hub]]` 配置: 1. 复制 `configs/config.example.toml` 为工作目录下的 `config.toml` 并调整 `[[Hub]]` 配置:
- 在全局段添加/修改 `ListenPort`,并从每个 Hub 中移除 `Port` - 在全局段添加/修改 `ListenPort`,并从每个 Hub 中移除 `Port`
- 为 Hub 填写 `Type`并按需添加 `Module`(缺省为 `legacy`,自定义模块需在 `internal/hubmodule/<module-key>/` 注册) - 为 Hub 填写 `Type`any-hub 会根据类型挑选对应模块 Hook如需扩展模块需在 `internal/hubmodule/<type>/` 注册并将新的类型纳入配置校验
- 根据 quickstart 示例设置 `Domain``Upstream``StoragePath` 等字段,并按需添加 `Username`/`Password` - 根据 quickstart 示例设置 `Domain``Upstream``StoragePath` 等字段,并按需添加 `Username`/`Password`
2. 参考 [`specs/003-hub-auth-fields/quickstart.md`](specs/003-hub-auth-fields/quickstart.md) 完成配置校验、凭证验证与日志检查。 2. 参考 [`specs/003-hub-auth-fields/quickstart.md`](specs/003-hub-auth-fields/quickstart.md) 完成配置校验、凭证验证与日志检查。
3. 常用命令: 3. 常用命令:
@@ -48,20 +48,20 @@ Password = "s3cr3t"
## 模块化代理与示例 ## 模块化代理与示例
- `configs/config.example.toml` 展示了多个 Hub 的组合Docker Hub省略 `Module`,自动使用 `Type` 同名 Hook、Composer Hub显式指定 `Module = "composer"`)以及 legacy 兜底 Hub可直接复制修改 - `configs/config.example.toml` 展示了多个 Hub 的组合Docker/NPM/Composer 等类型均自动绑定同名模块,仅需设置 `Type` 与上游信息即可启动
- 运行 `./scripts/demo-proxy.sh docker`(或 `npm`)即可加载示例配置并启动代理,日志中会附带 `module_key` 字段,便于确认命中的是 `legacy` 还是自定义模块。 - 运行 `./scripts/demo-proxy.sh docker`(或 `npm`)即可加载示例配置并启动代理,日志中会附带 `module_key` 字段,便于确认命中的是 `docker``npm`模块。
- Hook 开发流程: - Hook 开发流程:
1. 复制 `internal/hubmodule/template/``internal/hubmodule/<module-key>/`,补全 `module.go``module_test.go` 1. 复制 `internal/hubmodule/template/``internal/hubmodule/<module-key>/`,补全 `module.go``module_test.go`
2. 在模块 `init()` 中调用 `hubmodule.MustRegister` 注册 metadata并使用 `hooks.MustRegister` 注册 HookNormalizePath/ResolveUpstream/RewriteResponse 等)。 2. 在模块 `init()` 中调用 `hubmodule.MustRegister` 注册 metadata并使用 `hooks.MustRegister` 注册 HookNormalizePath/ResolveUpstream/RewriteResponse 等)。
3. 为模块补充单元测试、`tests/integration/` 覆盖 miss→hit 流程,运行 `make modules-test`/`go test ./...` 3. 为模块补充单元测试、`tests/integration/` 覆盖 miss→hit 流程,运行 `make modules-test`/`go test ./...`
4. 更新配置:`[[Hub]].Module` 留空,将根据 `Type` 自动选择 Hook也可显式设置 `Module = "<module-key>"` 并通过 `Rollout` 控制 legacy/dual/modular 4. 更新配置:为新的模块挑选一个唯一的 `Type`需要同步到配置校验列表Hub 只需填写该 `Type` 即可路由至新模块
5. 启动服务前,可通过 `curl -s /-/modules | jq '.hook_registry'` 确认 hook 注册情况;缺失时启动会直接失败,避免运行期回退到 legacy。 5. 启动服务前,可通过 `curl -s /-/modules | jq '.hook_registry'` 确认 hook 注册情况;缺失时启动会直接失败,避免运行期回退到 legacy。
### 模块选择与 legacy ### 模块选择
- `[[Hub]].Module` 为空时会自动回退到与 `Type` 同名的模块(若已注册),否则使用 `legacy` 兜底。
- diagnostics `/-/modules` 将展示 `hook_status`,当模块仍使用 legacy 时会标记 `legacy-only`,便于排查 - Hub 的 `Type` 直接映射到同名模块;新增模块时需同步扩展 `internal/config/validation.go` 中的支持列表
- legacy 模块仅提供最小兜底能力,迁移完成后应显式将 `Module` 设置为对应仓库Hook。 - diagnostics `/-/modules` 仍会展示每个模块hook 注册状态与所有 Hub 绑定关系,便于排查配置错误
- 示例操作手册、常见问题参见 [`specs/003-hub-auth-fields/quickstart.md`](specs/003-hub-auth-fields/quickstart.md) 以及本特性的 [`quickstart.md`](specs/004-modular-proxy-cache/quickstart.md) - legacy 模块仅作为历史兼容存在,不再通过配置字段触发
## CLI 标志 ## CLI 标志

View File

@@ -19,7 +19,6 @@ Name = "docker"
Upstream = "https://registry-1.docker.io" Upstream = "https://registry-1.docker.io"
Proxy = "" Proxy = ""
Type = "docker" Type = "docker"
Module = "docker"
Username = "" Username = ""
Password = "" Password = ""
@@ -29,7 +28,6 @@ Name = "ghcr"
Upstream = "https://ghcr.io" Upstream = "https://ghcr.io"
Proxy = "" Proxy = ""
Type = "docker" Type = "docker"
Module = "docker"
Username = "" Username = ""
Password = "" Password = ""
@@ -39,7 +37,6 @@ Name = "quay"
Upstream = "https://quay.io" Upstream = "https://quay.io"
Proxy = "" Proxy = ""
Type = "docker" Type = "docker"
Module = "docker"
Username = "" Username = ""
Password = "" Password = ""
@@ -50,7 +47,6 @@ Name = "go"
Upstream = "https://proxy.golang.org" Upstream = "https://proxy.golang.org"
Proxy = "" Proxy = ""
Type = "go" Type = "go"
Module = "go"
Username = "" Username = ""
Password = "" Password = ""
@@ -61,7 +57,6 @@ Name = "npm"
Upstream = "https://registry.npmjs.org" Upstream = "https://registry.npmjs.org"
Proxy = "" Proxy = ""
Type = "npm" Type = "npm"
Module = "npm"
Username = "" Username = ""
Password = "" Password = ""
@@ -74,7 +69,6 @@ Proxy = ""
Username = "" Username = ""
Password = "" Password = ""
Type = "pypi" Type = "pypi"
Module = "pypi"
# Composer Repository # Composer Repository
[[Hub]] [[Hub]]
@@ -85,7 +79,6 @@ Proxy = ""
Username = "" Username = ""
Password = "" Password = ""
Type = "composer" Type = "composer"
Module = "composer"
# Debian/Ubuntu APT 示例 # Debian/Ubuntu APT 示例
[[Hub]] [[Hub]]
@@ -93,8 +86,6 @@ Name = "apt-cache"
Domain = "apt.hub.local" Domain = "apt.hub.local"
Upstream = "https://mirrors.edge.kernel.org/ubuntu" Upstream = "https://mirrors.edge.kernel.org/ubuntu"
Type = "debian" Type = "debian"
Module = "debian"
Rollout = "modular"
# Alpine APK 示例 # Alpine APK 示例
[[Hub]] [[Hub]]
@@ -102,5 +93,3 @@ Name = "apk-cache"
Domain = "apk.hub.local" Domain = "apk.hub.local"
Upstream = "https://dl-cdn.alpinelinux.org/alpine" Upstream = "https://dl-cdn.alpinelinux.org/alpine"
Type = "apk" Type = "apk"
Module = "apk"
Rollout = "modular"

View File

@@ -17,9 +17,7 @@ Name = "docker-cache"
Domain = "docker.hub.local" Domain = "docker.hub.local"
Upstream = "https://registry-1.docker.io" Upstream = "https://registry-1.docker.io"
Proxy = "" Proxy = ""
Type = "docker" # 省略 Module 时自动选择与 Type 同名的 Hook此处为 docker Type = "docker"
# Module = "docker" # 如需明确指定,可取消注释
Rollout = "modular"
Username = "" Username = ""
Password = "" Password = ""
CacheTTL = 43200 CacheTTL = 43200
@@ -30,8 +28,6 @@ Name = "composer-cache"
Domain = "composer.hub.local" Domain = "composer.hub.local"
Upstream = "https://repo.packagist.org" Upstream = "https://repo.packagist.org"
Type = "composer" Type = "composer"
Module = "composer" # 显式绑定 composer Hook启动时会验证 hook 是否已注册
Rollout = "dual" # 可选legacy-only/dual/modular
CacheTTL = 21600 CacheTTL = 21600
[[Hub]] [[Hub]]
@@ -39,8 +35,6 @@ Name = "legacy-fallback"
Domain = "legacy.hub.local" Domain = "legacy.hub.local"
Upstream = "https://registry.npmjs.org" Upstream = "https://registry.npmjs.org"
Type = "npm" Type = "npm"
Module = "legacy" # 仍未迁移的 Hub 可显式指定 legacy诊断会标记为 legacy-only
Rollout = "legacy-only"
# Debian/Ubuntu APT 示例 # Debian/Ubuntu APT 示例
[[Hub]] [[Hub]]
@@ -48,8 +42,6 @@ Name = "apt-cache"
Domain = "apt.hub.local" Domain = "apt.hub.local"
Upstream = "https://mirrors.edge.kernel.org/ubuntu" Upstream = "https://mirrors.edge.kernel.org/ubuntu"
Type = "debian" Type = "debian"
Module = "debian"
Rollout = "modular"
# Alpine APK 示例 # Alpine APK 示例
[[Hub]] [[Hub]]
@@ -57,5 +49,3 @@ Name = "apk-cache"
Domain = "apk.hub.local" Domain = "apk.hub.local"
Upstream = "https://dl-cdn.alpinelinux.org/alpine" Upstream = "https://dl-cdn.alpinelinux.org/alpine"
Type = "apk" Type = "apk"
Module = "apk"
Rollout = "modular"

View File

@@ -2,10 +2,9 @@
## Common Fields ## Common Fields
- `hub`/`domain`/`hub_type`:当前 Hub 标识与协议类型,例如 `debian`/`apk` - `hub`/`domain`/`hub_type`:当前 Hub 标识与协议类型,例如 `debian`/`apk`
- `module_key`:命中的模块键( `legacy` 时表示新模块生效)。 - `module_key`:命中的模块键( `Type` 同名)。
- `cache_hit``true` 表示直接复用缓存;`false` 表示从上游获取或已刷新。 - `cache_hit``true` 表示直接复用缓存;`false` 表示从上游获取或已刷新。
- `upstream`/`upstream_status`:实际访问的上游地址与状态码。 - `upstream`/`upstream_status`:实际访问的上游地址与状态码。
- `rollout_flag``legacy-only`/`dual`/`modular`,便于排查路由与灰度。
- `action``proxy`,表明代理链路日志。 - `action``proxy`,表明代理链路日志。
## APT (debian 模块) ## APT (debian 模块)
@@ -19,4 +18,4 @@
## Quick Checks ## Quick Checks
- 观察 `cache_hit``upstream_status``cache_hit=true``upstream_status=200/304` 表示缓存复用成功;`cache_hit=false` 表示回源或刷新。 - 观察 `cache_hit``upstream_status``cache_hit=true``upstream_status=200/304` 表示缓存复用成功;`cache_hit=false` 表示回源或刷新。
-期望模块日志字段但出现 `module_key":"legacy"`,检查 `Module``Rollout` 配置是否指向新模块。 -`module_key` 与配置的 `Type` 不符,检查该类型的 hook 是否已注册,或是否误用了旧版二进制。

View File

@@ -1,58 +1,33 @@
# Modular Hub Migration Playbook # Module Binding Notes
This playbook describes how to cut a hub over from the shared legacy adapter to a dedicated module using the new rollout flags, diagnostics endpoint, and structured logs delivered in feature `004-modular-proxy-cache`. Legacy rollout flags (`Module`/`Rollout`) have been removed. Hubs now bind to modules solely through their `Type` values, which map 1:1 to registered modules (docker, npm, go, pypi, composer, debian, apk, ...).
## Prerequisites ## Migrating to a New Module
- Target module must be registered via `hubmodule.MustRegister` and expose a proxy handler through `proxy.RegisterModuleHandler`. 1. **Register the module**
- `config.toml` must already map the hub to its target module through `[[Hub]].Module`. Implement the new module under `internal/hubmodule/<type>/`, call `hubmodule.MustRegister` in `init()`, and register hooks via `hooks.MustRegister`.
- Operators must have access to the running binary port (default `:5000`) to query `/-/modules`.
## Rollout Workflow 2. **Expose a handler**
New modules continue to reuse the shared proxy handler registered via `proxy.RegisterModule`. No per-module handler wiring is required unless the module supplies a bespoke handler.
1. **Snapshot current state** 3. **Update the config schema**
Run `curl -s http://localhost:5000/-/modules | jq '.hubs[] | select(.hub_name=="<hub>")'` to capture the current `module_key` and `rollout_flag`. Legacy hubs report `module_key=legacy` and `rollout_flag=legacy-only`. Add the new type to `internal/config/validation.go`s `supportedHubTypes`, then redeploy. Every hub that should use the new module only needs `Type = "<type>"` plus the usual `Domain`/`Upstream` fields.
2. **Prepare config for dual traffic**
Edit the hub block to target the new module while keeping rollback safety:
```toml
[[Hub]]
Name = "npm-prod"
Domain = "npm.example.com"
Upstream = "https://registry.npmjs.org"
Module = "npm"
Rollout = "dual"
```
Dual mode keeps routing on the new module but keeps observability tagged as a partial rollout.
3. **Deploy and monitor**
Restart the service and tail logs filtered by `module_key`:
```sh
jq 'select(.module_key=="npm" and .rollout_flag=="dual")' /var/log/any-hub.json
```
Every request now carries `module_key`/`rollout_flag`, allowing dashboards or `grep`-based analyses without extra parsing.
4. **Verify diagnostics** 4. **Verify diagnostics**
Query `/-/modules/npm` to inspect the registered metadata and confirm cache strategy, or `/-/modules` to ensure the hub binding reflects `rollout_flag=dual`. `curl http://127.0.0.1:<port>/-/modules` to ensure the new type appears under `modules[]` and that the desired hubs show `module_key="<type>"`.
5. **Promote to modular** 5. **Monitor logs**
Once metrics are healthy, change `Rollout = "modular"` in config and redeploy. Continue monitoring logs to make sure both `module_key` and `rollout_flag` show the fully promoted state. Structured logs still carry `module_key`, making it easy to confirm that traffic is flowing through the expected module. Example:
6. **Rollback procedure** ```json
To rollback, set `Rollout = "legacy-only"` (without touching `Module`). The runtime forces traffic through the legacy module while keeping the desired module declaration for later reattempts. Confirm via diagnostics (`module_key` reverts to `legacy`) before announcing rollback complete. {"action":"proxy","hub":"npm","module_key":"npm","cache_hit":false,"upstream_status":200}
```
## Observability Checklist 6. **Rollback**
Since modules are now type-driven, rollback is as simple as reverting the `Type` value (or config deployment) back to the previous modules type.
- **Logs**: Every proxy log line now contains `hub`, `module_key`, `rollout_flag`, upstream status, and `request_id`. Capture at least five minutes of traffic per flag change.
- **Diagnostics**: Store JSON snapshots from `/-/modules` before and after each rollout stage for incident timelines.
- **Config History**: Keep the `config.toml` diff (especially `Rollout` changes) attached to change records for auditability.
## Troubleshooting ## Troubleshooting
- **Error: `module_not_found` during diagnostics** → module key not registered; ensure the module packages `init()` calls `hubmodule.MustRegister`. - **`module_not_found` in diagnostics** → ensure the module registered via `hubmodule.MustRegister` before the hub references its type.
- **Requests still tagged with `legacy-only` after promotion** → double-check the running process uses the updated config path (`ANY_HUB_CONFIG` vs `--config`) and restart the service. - **Hooks missing** → `/-/modules` exposes `hook_registry`; confirm the new type reports `registered`.
- **Diagnostics 404** → confirm you are hitting the correct port and that the CLI user/network path allows HTTP access; the endpoint ignores Host headers, so `curl http://127.0.0.1:<port>/-/modules` should succeed locally. - **Unexpected module key in logs** → confirm the running binary includes your module (imported in `internal/config/modules.go`) and that the config `/--config` path matches the deployed file.

View File

@@ -5,7 +5,6 @@ import (
"path/filepath" "path/filepath"
"reflect" "reflect"
"strconv" "strconv"
"strings"
"time" "time"
"github.com/mitchellh/mapstructure" "github.com/mitchellh/mapstructure"
@@ -89,25 +88,12 @@ func applyHubDefaults(h *HubConfig) {
if h.CacheTTL.DurationValue() < 0 { if h.CacheTTL.DurationValue() < 0 {
h.CacheTTL = Duration(0) h.CacheTTL = Duration(0)
} }
if trimmed := strings.TrimSpace(h.Module); trimmed == "" {
typeKey := strings.ToLower(strings.TrimSpace(h.Type))
if meta, ok := hubmodule.Resolve(typeKey); ok {
h.Module = meta.Key
} else {
h.Module = hubmodule.DefaultModuleKey()
}
} else {
h.Module = strings.ToLower(trimmed)
}
if rollout := strings.TrimSpace(h.Rollout); rollout != "" {
h.Rollout = strings.ToLower(rollout)
}
if h.ValidationMode == "" { if h.ValidationMode == "" {
h.ValidationMode = string(hubmodule.ValidationModeETag) h.ValidationMode = string(hubmodule.ValidationModeETag)
} }
} }
// NormalizeHubConfig 公开给无需依赖 loader 的调用方(例如测试)以填充模块/rollout 默认值。 // NormalizeHubConfig 公开给无需依赖 loader 的调用方(例如测试)以应用 TTL/校验默认值。
func NormalizeHubConfig(h HubConfig) HubConfig { func NormalizeHubConfig(h HubConfig) HubConfig {
applyHubDefaults(&h) applyHubDefaults(&h)
return h return h

View File

@@ -4,7 +4,6 @@ import (
"time" "time"
"github.com/any-hub/any-hub/internal/hubmodule" "github.com/any-hub/any-hub/internal/hubmodule"
"github.com/any-hub/any-hub/internal/hubmodule/legacy"
) )
// HubRuntime 将 Hub 配置与模块元数据合并,方便运行时快速取用策略。 // HubRuntime 将 Hub 配置与模块元数据合并,方便运行时快速取用策略。
@@ -12,16 +11,14 @@ type HubRuntime struct {
Config HubConfig Config HubConfig
Module hubmodule.ModuleMetadata Module hubmodule.ModuleMetadata
CacheStrategy hubmodule.CacheStrategyProfile CacheStrategy hubmodule.CacheStrategyProfile
Rollout legacy.RolloutFlag
} }
// BuildHubRuntime 根据 Hub 配置和模块元数据创建运行时描述,应用最终 TTL 覆盖。 // BuildHubRuntime 根据 Hub 配置和模块元数据创建运行时描述,应用最终 TTL 覆盖。
func BuildHubRuntime(cfg HubConfig, meta hubmodule.ModuleMetadata, ttl time.Duration, flag legacy.RolloutFlag) HubRuntime { func BuildHubRuntime(cfg HubConfig, meta hubmodule.ModuleMetadata, ttl time.Duration) HubRuntime {
strategy := hubmodule.ResolveStrategy(meta, cfg.StrategyOverrides(ttl)) strategy := hubmodule.ResolveStrategy(meta, cfg.StrategyOverrides(ttl))
return HubRuntime{ return HubRuntime{
Config: cfg, Config: cfg,
Module: meta, Module: meta,
CacheStrategy: strategy, CacheStrategy: strategy,
Rollout: flag,
} }
} }

View File

@@ -1,69 +0,0 @@
package config
import (
"fmt"
"strings"
"github.com/any-hub/any-hub/internal/hubmodule"
"github.com/any-hub/any-hub/internal/hubmodule/legacy"
)
// Rollout 字段说明legacy → modular 平滑迁移控制):
// - legacy-only强制使用 legacy 模块EffectiveModuleKey → legacy用于未迁移或需要快速回滚时。
// - dual新模块为默认保留 legacy 以便诊断/灰度;仅当 Module 非空时生效,否则回退 legacy-only。
// - modular仅使用新模块Module 为空或 legacy 模块时自动回退 legacy-only。
// 默认行为:未填写 Rollout 时,空 Module/legacy 模块默认 legacy-only其它模块默认 modular。
// 影响范围动态选择执行的模块键EffectiveModuleKey、路由日志中的 rollout_flag方便区分迁移阶段。
// parseRolloutFlag 将配置中的 rollout 字段标准化,并结合模块类型输出最终状态。
func parseRolloutFlag(raw string, moduleKey string) (legacy.RolloutFlag, error) {
normalized := strings.ToLower(strings.TrimSpace(raw))
if normalized == "" {
return defaultRolloutFlag(moduleKey), nil
}
switch normalized {
case string(legacy.RolloutLegacyOnly):
return legacy.RolloutLegacyOnly, nil
case string(legacy.RolloutDual):
if moduleKey == hubmodule.DefaultModuleKey() {
return legacy.RolloutLegacyOnly, nil
}
return legacy.RolloutDual, nil
case string(legacy.RolloutModular):
if moduleKey == hubmodule.DefaultModuleKey() {
return legacy.RolloutLegacyOnly, nil
}
return legacy.RolloutModular, nil
default:
return "", fmt.Errorf("不支持的 rollout 值: %s", raw)
}
}
func defaultRolloutFlag(moduleKey string) legacy.RolloutFlag {
if strings.TrimSpace(moduleKey) == "" || moduleKey == hubmodule.DefaultModuleKey() {
return legacy.RolloutLegacyOnly
}
return legacy.RolloutModular
}
// EffectiveModuleKey 根据 rollout 状态计算真实运行的模块。
func EffectiveModuleKey(moduleKey string, flag legacy.RolloutFlag) string {
if flag == legacy.RolloutLegacyOnly {
return hubmodule.DefaultModuleKey()
}
normalized := strings.ToLower(strings.TrimSpace(moduleKey))
if normalized == "" {
return hubmodule.DefaultModuleKey()
}
return normalized
}
// RolloutFlagValue 返回当前 Hub 的 rollout flag假定 Validate 已经通过)。
func (h HubConfig) RolloutFlagValue() legacy.RolloutFlag {
flag := legacy.RolloutFlag(strings.ToLower(strings.TrimSpace(h.Rollout)))
if flag == "" {
return defaultRolloutFlag(h.Module)
}
return flag
}

View File

@@ -69,8 +69,6 @@ type HubConfig struct {
Upstream string `mapstructure:"Upstream"` Upstream string `mapstructure:"Upstream"`
Proxy string `mapstructure:"Proxy"` Proxy string `mapstructure:"Proxy"`
Type string `mapstructure:"Type"` Type string `mapstructure:"Type"`
Module string `mapstructure:"Module"`
Rollout string `mapstructure:"Rollout"`
Username string `mapstructure:"Username"` Username string `mapstructure:"Username"`
Password string `mapstructure:"Password"` Password string `mapstructure:"Password"`
CacheTTL Duration `mapstructure:"CacheTTL"` CacheTTL Duration `mapstructure:"CacheTTL"`

View File

@@ -79,23 +79,9 @@ func (c *Config) Validate() error {
} }
hub.Type = normalizedType hub.Type = normalizedType
moduleKey := strings.ToLower(strings.TrimSpace(hub.Module)) if _, ok := hubmodule.Resolve(normalizedType); !ok {
if moduleKey == "" { return newFieldError(hubField(hub.Name, "Type"), fmt.Sprintf("未注册模块: %s", normalizedType))
if _, ok := hubmodule.Resolve(normalizedType); ok && normalizedType != "" {
moduleKey = normalizedType
} else {
moduleKey = hubmodule.DefaultModuleKey()
} }
}
if _, ok := hubmodule.Resolve(moduleKey); !ok {
return newFieldError(hubField(hub.Name, "Module"), fmt.Sprintf("未注册模块: %s", moduleKey))
}
hub.Module = moduleKey
flag, err := parseRolloutFlag(hub.Rollout, hub.Module)
if err != nil {
return newFieldError(hubField(hub.Name, "Rollout"), err.Error())
}
hub.Rollout = string(flag)
if hub.ValidationMode != "" { if hub.ValidationMode != "" {
mode := strings.ToLower(strings.TrimSpace(hub.ValidationMode)) mode := strings.ToLower(strings.TrimSpace(hub.ValidationMode))
switch mode { switch mode {

View File

@@ -24,7 +24,7 @@ internal/hubmodule/
2. 填写模块特有逻辑与缓存策略,并确保包含中文注释解释设计。 2. 填写模块特有逻辑与缓存策略,并确保包含中文注释解释设计。
3. 在模块目录添加 `module_test.go`,使用 `httptest.Server``t.TempDir()` 复现真实流量。 3. 在模块目录添加 `module_test.go`,使用 `httptest.Server``t.TempDir()` 复现真实流量。
4. 运行 `make modules-test` 验证模块单元测试。 4. 运行 `make modules-test` 验证模块单元测试。
5. `[[Hub]].Module` 留空时会优先选择与 `Type` 同名的模块,实际迁移时仍建议显式填写,便于 diagnostics 标记 rollout 5. `[[Hub]].Type` 现已直接映射到同名模块;新增模块时记得将类型加入配置校验与示例配置
## 术语 ## 术语
- **Module Key**:模块唯一标识(如 `legacy``npm-tarball`)。 - **Module Key**:模块唯一标识(如 `legacy``npm-tarball`)。

View File

@@ -25,76 +25,49 @@ func normalizePath(_ *hooks.RequestContext, p string, rawQuery []byte) (string,
func cachePolicy(_ *hooks.RequestContext, locatorPath string, current hooks.CachePolicy) hooks.CachePolicy { func cachePolicy(_ *hooks.RequestContext, locatorPath string, current hooks.CachePolicy) hooks.CachePolicy {
clean := canonicalPath(locatorPath) clean := canonicalPath(locatorPath)
switch { if strings.Contains(clean, "/by-hash/") || strings.Contains(clean, "/pool/") {
case isAptIndexPath(clean):
// 索引类Release/Packages需要 If-None-Match/If-Modified-Since 再验证。
current.AllowCache = true
current.AllowStore = true
current.RequireRevalidate = true
case isAptImmutablePath(clean):
// pool/*.deb 与 by-hash 路径视为不可变,直接缓存后续不再 HEAD。 // pool/*.deb 与 by-hash 路径视为不可变,直接缓存后续不再 HEAD。
current.AllowCache = true current.AllowCache = true
current.AllowStore = true current.AllowStore = true
current.RequireRevalidate = false current.RequireRevalidate = false
default: return current
}
if strings.Contains(clean, "/dists/") {
// 索引类Release/Packages/Contents需要 If-None-Match/If-Modified-Since 再验证。
if strings.HasSuffix(clean, "/release") ||
strings.HasSuffix(clean, "/inrelease") ||
strings.HasSuffix(clean, "/release.gpg") {
current.AllowCache = true
current.AllowStore = true
current.RequireRevalidate = true
return current
}
}
current.AllowCache = false current.AllowCache = false
current.AllowStore = false current.AllowStore = false
current.RequireRevalidate = false current.RequireRevalidate = false
}
return current return current
} }
func contentType(_ *hooks.RequestContext, locatorPath string) string { func contentType(_ *hooks.RequestContext, locatorPath string) string {
clean := canonicalPath(locatorPath)
switch { switch {
case strings.HasSuffix(locatorPath, ".gz"): case strings.HasSuffix(clean, ".gz"):
return "application/gzip" return "application/gzip"
case strings.HasSuffix(locatorPath, ".xz"): case strings.HasSuffix(clean, ".xz"):
return "application/x-xz" return "application/x-xz"
case strings.HasSuffix(locatorPath, "Release.gpg"): case strings.HasSuffix(clean, "release.gpg"):
return "application/pgp-signature" return "application/pgp-signature"
case isAptIndexPath(locatorPath): case strings.Contains(clean, "/dists/") &&
(strings.HasSuffix(clean, "/release") || strings.HasSuffix(clean, "/inrelease") || strings.HasSuffix(clean, "/release.gpg")):
return "text/plain" return "text/plain"
default: default:
return "" return ""
} }
} }
func isAptIndexPath(p string) bool {
clean := canonicalPath(p)
if isByHashPath(clean) {
return false
}
if strings.Contains(clean, "/dists/") {
if strings.HasSuffix(clean, "/release") ||
strings.HasSuffix(clean, "/inrelease") ||
strings.HasSuffix(clean, "/release.gpg") {
return true
}
}
return false
}
func isAptImmutablePath(p string) bool {
clean := canonicalPath(p)
if isByHashPath(clean) {
return true
}
if strings.Contains(clean, "/pool/") {
return true
}
return false
}
func isByHashPath(p string) bool {
clean := canonicalPath(p)
if strings.Contains(clean, "/dists/") {
return false
}
return strings.Contains(clean, "/by-hash/")
}
func canonicalPath(p string) string { func canonicalPath(p string) string {
if p == "" { if p == "" {
return "/" return "/"

View File

@@ -1,65 +0,0 @@
package legacy
import (
"sort"
"strings"
"sync"
)
// RolloutFlag 描述 legacy 模块迁移阶段。
type RolloutFlag string
const (
RolloutLegacyOnly RolloutFlag = "legacy-only"
RolloutDual RolloutFlag = "dual"
RolloutModular RolloutFlag = "modular"
)
// AdapterState 记录特定 Hub 在 legacy 适配器中的运行状态。
type AdapterState struct {
HubName string
ModuleKey string
Rollout RolloutFlag
}
var (
stateMu sync.RWMutex
state = make(map[string]AdapterState)
)
// RecordAdapterState 更新指定 Hub 的 rollout 状态,供诊断端和日志使用。
func RecordAdapterState(hubName, moduleKey string, flag RolloutFlag) {
if hubName == "" {
return
}
key := strings.ToLower(hubName)
stateMu.Lock()
state[key] = AdapterState{
HubName: hubName,
ModuleKey: moduleKey,
Rollout: flag,
}
stateMu.Unlock()
}
// SnapshotAdapterStates 返回所有 Hub 的 rollout 状态,按名称排序。
func SnapshotAdapterStates() []AdapterState {
stateMu.RLock()
defer stateMu.RUnlock()
if len(state) == 0 {
return nil
}
keys := make([]string, 0, len(state))
for k := range state {
keys = append(keys, k)
}
sort.Strings(keys)
result := make([]AdapterState, 0, len(keys))
for _, key := range keys {
result = append(result, state[key])
}
return result
}

View File

@@ -11,15 +11,13 @@ func BaseFields(action, configPath string) logrus.Fields {
} }
// RequestFields 提供 hub/domain/命中状态字段,供代理请求日志复用。 // RequestFields 提供 hub/domain/命中状态字段,供代理请求日志复用。
func RequestFields(hub, domain, hubType, authMode, moduleKey, rolloutFlag string, cacheHit bool, legacyOnly bool) logrus.Fields { func RequestFields(hub, domain, hubType, authMode, moduleKey string, cacheHit bool) logrus.Fields {
return logrus.Fields{ return logrus.Fields{
"hub": hub, "hub": hub,
"domain": domain, "domain": domain,
"hub_type": hubType, "hub_type": hubType,
"auth_mode": authMode, "auth_mode": authMode,
"cache_hit": cacheHit, "cache_hit": cacheHit,
"legacy_only": legacyOnly,
"module_key": moduleKey, "module_key": moduleKey,
"rollout_flag": rolloutFlag,
} }
} }

View File

@@ -8,7 +8,6 @@ import (
"github.com/gofiber/fiber/v3" "github.com/gofiber/fiber/v3"
"github.com/sirupsen/logrus" "github.com/sirupsen/logrus"
"github.com/any-hub/any-hub/internal/hubmodule"
"github.com/any-hub/any-hub/internal/logging" "github.com/any-hub/any-hub/internal/logging"
"github.com/any-hub/any-hub/internal/server" "github.com/any-hub/any-hub/internal/server"
) )
@@ -36,7 +35,7 @@ func RegisterModuleHandler(key string, handler server.ProxyHandler) {
MustRegisterModule(ModuleRegistration{Key: key, Handler: handler}) MustRegisterModule(ModuleRegistration{Key: key, Handler: handler})
} }
// Handle 实现 server.ProxyHandler根据 route.ModuleKey 选择 handler。 // Handle 实现 server.ProxyHandler根据 route.Module.Key 选择 handler。
func (f *Forwarder) Handle(c fiber.Ctx, route *server.HubRoute) error { func (f *Forwarder) Handle(c fiber.Ctx, route *server.HubRoute) error {
requestID := server.RequestID(c) requestID := server.RequestID(c)
handler := f.lookup(route) handler := f.lookup(route)
@@ -91,7 +90,7 @@ func (f *Forwarder) logModuleError(route *server.HubRoute, code string, err erro
func (f *Forwarder) lookup(route *server.HubRoute) server.ProxyHandler { func (f *Forwarder) lookup(route *server.HubRoute) server.ProxyHandler {
if route != nil { if route != nil {
if handler := lookupModuleHandler(route.ModuleKey); handler != nil { if handler := lookupModuleHandler(route.Module.Key); handler != nil {
return handler return handler
} }
} }
@@ -132,10 +131,8 @@ func (f *Forwarder) routeFields(route *server.HubRoute, requestID string) logrus
route.Config.Domain, route.Config.Domain,
route.Config.Type, route.Config.Type,
route.Config.AuthMode(), route.Config.AuthMode(),
route.ModuleKey, route.Module.Key,
string(route.RolloutFlag),
false, false,
route.ModuleKey == hubmodule.DefaultModuleKey(),
) )
if requestID != "" { if requestID != "" {
fields["request_id"] = requestID fields["request_id"] = requestID

View File

@@ -10,7 +10,7 @@ import (
"github.com/valyala/fasthttp" "github.com/valyala/fasthttp"
"github.com/any-hub/any-hub/internal/config" "github.com/any-hub/any-hub/internal/config"
"github.com/any-hub/any-hub/internal/hubmodule/legacy" "github.com/any-hub/any-hub/internal/hubmodule"
"github.com/any-hub/any-hub/internal/server" "github.com/any-hub/any-hub/internal/server"
) )
@@ -103,7 +103,8 @@ func testRouteWithModule(moduleKey string) *server.HubRoute {
Domain: "test.local", Domain: "test.local",
Type: "custom", Type: "custom",
}, },
ModuleKey: moduleKey, Module: hubmodule.ModuleMetadata{
RolloutFlag: legacy.RolloutModular, Key: moduleKey,
},
} }
} }

View File

@@ -65,8 +65,7 @@ func buildHookContext(route *server.HubRoute, c fiber.Ctx) *hooks.RequestContext
HubName: route.Config.Name, HubName: route.Config.Name,
Domain: route.Config.Domain, Domain: route.Config.Domain,
HubType: route.Config.Type, HubType: route.Config.Type,
ModuleKey: route.ModuleKey, ModuleKey: route.Module.Key,
RolloutFlag: string(route.RolloutFlag),
UpstreamHost: baseHost, UpstreamHost: baseHost,
Method: c.Method(), Method: c.Method(),
} }
@@ -84,7 +83,7 @@ func hasHook(def hooks.Hooks) bool {
func (h *Handler) Handle(c fiber.Ctx, route *server.HubRoute) error { func (h *Handler) Handle(c fiber.Ctx, route *server.HubRoute) error {
started := time.Now() started := time.Now()
requestID := server.RequestID(c) requestID := server.RequestID(c)
hooksDef, ok := hooks.Fetch(route.ModuleKey) hooksDef, ok := hooks.Fetch(route.Module.Key)
hookCtx := buildHookContext(route, c) hookCtx := buildHookContext(route, c)
rawQuery := append([]byte(nil), c.Request().URI().QueryString()...) rawQuery := append([]byte(nil), c.Request().URI().QueryString()...)
cleanPath := normalizeRequestPath(route, string(c.Request().URI().Path())) cleanPath := normalizeRequestPath(route, string(c.Request().URI().Path()))
@@ -121,7 +120,7 @@ func (h *Handler) Handle(c fiber.Ctx, route *server.HubRoute) error {
// miss, continue // miss, continue
default: default:
h.logger.WithError(err). h.logger.WithError(err).
WithFields(logrus.Fields{"hub": route.Config.Name, "module_key": route.ModuleKey}). WithFields(logrus.Fields{"hub": route.Config.Name, "module_key": route.Module.Key}).
Warn("cache_get_failed") Warn("cache_get_failed")
} }
} }
@@ -135,7 +134,7 @@ func (h *Handler) Handle(c fiber.Ctx, route *server.HubRoute) error {
fresh, err := h.isCacheFresh(c, route, locator, cached.Entry, &hookState) fresh, err := h.isCacheFresh(c, route, locator, cached.Entry, &hookState)
if err != nil { if err != nil {
h.logger.WithError(err). h.logger.WithError(err).
WithFields(logrus.Fields{"hub": route.Config.Name, "module_key": route.ModuleKey}). WithFields(logrus.Fields{"hub": route.Config.Name, "module_key": route.Module.Key}).
Warn("cache_revalidate_failed") Warn("cache_revalidate_failed")
serve = false serve = false
} else if !fresh { } else if !fresh {
@@ -518,10 +517,8 @@ func (h *Handler) logResult(
route.Config.Domain, route.Config.Domain,
route.Config.Type, route.Config.Type,
route.Config.AuthMode(), route.Config.AuthMode(),
route.ModuleKey, route.Module.Key,
string(route.RolloutFlag),
cacheHit, cacheHit,
route.ModuleKey == hubmodule.DefaultModuleKey(),
) )
fields["action"] = "proxy" fields["action"] = "proxy"
fields["upstream"] = upstream fields["upstream"] = upstream
@@ -971,10 +968,8 @@ func (h *Handler) logAuthRetry(route *server.HubRoute, upstream string, requestI
route.Config.Domain, route.Config.Domain,
route.Config.Type, route.Config.Type,
route.Config.AuthMode(), route.Config.AuthMode(),
route.ModuleKey, route.Module.Key,
string(route.RolloutFlag),
false, false,
route.ModuleKey == hubmodule.DefaultModuleKey(),
) )
fields["action"] = "proxy_retry" fields["action"] = "proxy_retry"
fields["upstream"] = upstream fields["upstream"] = upstream
@@ -992,10 +987,8 @@ func (h *Handler) logAuthFailure(route *server.HubRoute, upstream string, reques
route.Config.Domain, route.Config.Domain,
route.Config.Type, route.Config.Type,
route.Config.AuthMode(), route.Config.AuthMode(),
route.ModuleKey, route.Module.Key,
string(route.RolloutFlag),
false, false,
route.ModuleKey == hubmodule.DefaultModuleKey(),
) )
fields["action"] = "proxy" fields["action"] = "proxy"
fields["upstream"] = upstream fields["upstream"] = upstream

View File

@@ -13,7 +13,6 @@ type RequestContext struct {
Domain string Domain string
HubType string HubType string
ModuleKey string ModuleKey string
RolloutFlag string
UpstreamHost string UpstreamHost string
Method string Method string
} }

View File

@@ -31,9 +31,12 @@ func NewUpstreamClient(cfg *config.Config) *http.Client {
timeout = cfg.Global.UpstreamTimeout.DurationValue() timeout = cfg.Global.UpstreamTimeout.DurationValue()
} }
transport := defaultTransport.Clone()
// Use UpstreamTimeout as ResponseHeaderTimeout to avoid killing long streaming downloads.
transport.ResponseHeaderTimeout = timeout
return &http.Client{ return &http.Client{
Timeout: timeout, Transport: transport,
Transport: defaultTransport.Clone(),
} }
} }

View File

@@ -16,8 +16,12 @@ func TestNewUpstreamClientUsesConfigTimeout(t *testing.T) {
} }
client := NewUpstreamClient(cfg) client := NewUpstreamClient(cfg)
if client.Timeout != 45*time.Second { transport, ok := client.Transport.(*http.Transport)
t.Fatalf("expected timeout 45s, got %s", client.Timeout) if !ok {
t.Fatalf("expected *http.Transport, got %T", client.Transport)
}
if transport.ResponseHeaderTimeout != 45*time.Second {
t.Fatalf("expected response header timeout 45s, got %s", transport.ResponseHeaderTimeout)
} }
} }

View File

@@ -11,7 +11,6 @@ import (
"github.com/any-hub/any-hub/internal/config" "github.com/any-hub/any-hub/internal/config"
"github.com/any-hub/any-hub/internal/hubmodule" "github.com/any-hub/any-hub/internal/hubmodule"
"github.com/any-hub/any-hub/internal/hubmodule/legacy"
) )
// HubRoute 将 Hub 配置与派生属性(如缓存 TTL、解析后的 Upstream/Proxy URL // HubRoute 将 Hub 配置与派生属性(如缓存 TTL、解析后的 Upstream/Proxy URL
@@ -26,13 +25,10 @@ type HubRoute struct {
// UpstreamURL/ProxyURL 在构造 Registry 时提前解析完成,便于后续请求快速复用。 // UpstreamURL/ProxyURL 在构造 Registry 时提前解析完成,便于后续请求快速复用。
UpstreamURL *url.URL UpstreamURL *url.URL
ProxyURL *url.URL ProxyURL *url.URL
// ModuleKey/Module 记录当前 hub 选用的模块及其元数据,便于日志与观测。 // Module 记录当前 hub 选用的模块元数据,便于日志与观测。
ModuleKey string
Module hubmodule.ModuleMetadata Module hubmodule.ModuleMetadata
// CacheStrategy 代表模块默认策略与 hub 覆盖后的最终结果。 // CacheStrategy 代表模块默认策略与 hub 覆盖后的最终结果。
CacheStrategy hubmodule.CacheStrategyProfile CacheStrategy hubmodule.CacheStrategyProfile
// RolloutFlag 反映当前 hub 的 legacy → modular 迁移状态,供日志/诊断使用。
RolloutFlag legacy.RolloutFlag
} }
// HubRegistry 提供 Host/Host:port 到 HubRoute 的查询能力,所有 Hub 共享同一个监听端口。 // HubRegistry 提供 Host/Host:port 到 HubRoute 的查询能力,所有 Hub 共享同一个监听端口。
@@ -106,9 +102,11 @@ func (r *HubRegistry) List() []HubRoute {
} }
func buildHubRoute(cfg *config.Config, hub config.HubConfig) (*HubRoute, error) { func buildHubRoute(cfg *config.Config, hub config.HubConfig) (*HubRoute, error) {
flag := hub.RolloutFlagValue() moduleKey := strings.ToLower(strings.TrimSpace(hub.Type))
effectiveKey := config.EffectiveModuleKey(hub.Module, flag) if moduleKey == "" {
meta, err := moduleMetadataForKey(effectiveKey) return nil, fmt.Errorf("hub %s: 缺少 Type", hub.Name)
}
meta, err := moduleMetadataForKey(moduleKey)
if err != nil { if err != nil {
return nil, fmt.Errorf("hub %s: %w", hub.Name, err) return nil, fmt.Errorf("hub %s: %w", hub.Name, err)
} }
@@ -127,8 +125,7 @@ func buildHubRoute(cfg *config.Config, hub config.HubConfig) (*HubRoute, error)
} }
effectiveTTL := cfg.EffectiveCacheTTL(hub) effectiveTTL := cfg.EffectiveCacheTTL(hub)
runtime := config.BuildHubRuntime(hub, meta, effectiveTTL, flag) runtime := config.BuildHubRuntime(hub, meta, effectiveTTL)
legacy.RecordAdapterState(hub.Name, runtime.Module.Key, flag)
return &HubRoute{ return &HubRoute{
Config: hub, Config: hub,
@@ -136,10 +133,8 @@ func buildHubRoute(cfg *config.Config, hub config.HubConfig) (*HubRoute, error)
CacheTTL: effectiveTTL, CacheTTL: effectiveTTL,
UpstreamURL: upstreamURL, UpstreamURL: upstreamURL,
ProxyURL: proxyURL, ProxyURL: proxyURL,
ModuleKey: runtime.Module.Key,
Module: runtime.Module, Module: runtime.Module,
CacheStrategy: runtime.CacheStrategy, CacheStrategy: runtime.CacheStrategy,
RolloutFlag: runtime.Rollout,
}, nil }, nil
} }

View File

@@ -5,7 +5,6 @@ import (
"time" "time"
"github.com/any-hub/any-hub/internal/config" "github.com/any-hub/any-hub/internal/config"
"github.com/any-hub/any-hub/internal/hubmodule/legacy"
) )
func TestHubRegistryLookupByHost(t *testing.T) { func TestHubRegistryLookupByHost(t *testing.T) {
@@ -55,8 +54,8 @@ func TestHubRegistryLookupByHost(t *testing.T) {
if route.CacheStrategy.ValidationMode == "" { if route.CacheStrategy.ValidationMode == "" {
t.Fatalf("cache strategy validation mode should not be empty") t.Fatalf("cache strategy validation mode should not be empty")
} }
if route.RolloutFlag != legacy.RolloutModular { if route.Module.Key != "docker" {
t.Fatalf("default rollout flag should be modular") t.Fatalf("expected docker module, got %s", route.Module.Key)
} }
if route.UpstreamURL.String() != "https://registry-1.docker.io" { if route.UpstreamURL.String() != "https://registry-1.docker.io" {

View File

@@ -65,8 +65,6 @@ type hubBindingPayload struct {
ModuleKey string `json:"module_key"` ModuleKey string `json:"module_key"`
Domain string `json:"domain"` Domain string `json:"domain"`
Port int `json:"port"` Port int `json:"port"`
Rollout string `json:"rollout_flag"`
Legacy bool `json:"legacy_only"`
} }
func encodeModules(mods []hubmodule.ModuleMetadata, status map[string]string) []modulePayload { func encodeModules(mods []hubmodule.ModuleMetadata, status map[string]string) []modulePayload {
@@ -115,11 +113,9 @@ func encodeHubBindings(routes []server.HubRoute) []hubBindingPayload {
for _, route := range routes { for _, route := range routes {
result = append(result, hubBindingPayload{ result = append(result, hubBindingPayload{
HubName: route.Config.Name, HubName: route.Config.Name,
ModuleKey: route.ModuleKey, ModuleKey: route.Module.Key,
Domain: route.Config.Domain, Domain: route.Config.Domain,
Port: route.ListenPort, Port: route.ListenPort,
Rollout: string(route.RolloutFlag),
Legacy: route.ModuleKey == hubmodule.DefaultModuleKey(),
}) })
} }
return result return result

View File

@@ -15,7 +15,7 @@
- `MaxMemoryCacheSize` (bytes, optional, default 268435456) - `MaxMemoryCacheSize` (bytes, optional, default 268435456)
- `MaxRetries` (int >=0, default 3) - `MaxRetries` (int >=0, default 3)
- `InitialBackoff` (duration, default 1s) - `InitialBackoff` (duration, default 1s)
- `UpstreamTimeout` (duration, default 30s) - `UpstreamTimeout` (duration, default 30s, used as upstream response header timeout; body can stream longer)
- **Validation Rules**: 路径必须存在或可创建;数值必须 >0LogLevel 必须匹配允许枚举。 - **Validation Rules**: 路径必须存在或可创建;数值必须 >0LogLevel 必须匹配允许枚举。
- **Relationships**: 被 `Config` 聚合并为 `HubConfig` 提供默认值。 - **Relationships**: 被 `Config` 聚合并为 `HubConfig` 提供默认值。

View File

@@ -94,6 +94,3 @@ components:
type: string type: string
port: port:
type: integer type: integer
rollout_flag:
type: string
enum: [legacy-only, dual, modular]

View File

@@ -13,13 +13,12 @@ The modular architecture introduces explicit metadata describing which proxy+cac
- `Domain` *(string, required)* hostname clients access; must be unique per process. - `Domain` *(string, required)* hostname clients access; must be unique per process.
- `Port` *(int, required)* listen port; validated to 165535. - `Port` *(int, required)* listen port; validated to 165535.
- `Upstream` *(string, required)* base URL for upstream registry; must be HTTPS or explicitly whitelisted HTTP. - `Upstream` *(string, required)* base URL for upstream registry; must be HTTPS or explicitly whitelisted HTTP.
- `Module` *(string, optional, default `"legacy"`)* key resolved through module registry. Validation ensures module exists at load time. - `Type` *(string, required)* module selector; validated against the registered module set.
- `CacheTTL`, `Proxy`, and other overrides *(optional)* reuse existing schema; modules may read these via dependency injection. - `CacheTTL`, `Proxy`, and other overrides *(optional)* reuse existing schema; modules may read these via dependency injection.
- **Relationships**: - **Relationships**:
- `HubConfigEntry.Module``ModuleMetadata.Key` (many-to-one). - `HubConfigEntry.Type``ModuleMetadata.Key` (many-to-one).
- **Validation Rules**: - **Validation Rules**:
- Missing `Module` implicitly maps to `legacy` to preserve backward compatibility. - Invalid `Type` values reject config load; operators must add the type to the supported list alongside module registration.
- Changing `Module` requires a migration plan; config loader logs module name for observability.
### 2. ModuleMetadata ### 2. ModuleMetadata
- **Fields**: - **Fields**:
@@ -57,20 +56,12 @@ The modular architecture introduces explicit metadata describing which proxy+cac
- TTL must be positive. - TTL must be positive.
- Modules flagged as `SupportsStreamingWrite=false` must document fallback behavior before registration. - Modules flagged as `SupportsStreamingWrite=false` must document fallback behavior before registration.
### 5. LegacyAdapterState
- **Purpose**: Tracks which hubs still run through the old shared implementation to support progressive migration.
- **Fields**:
- `HubName` *(string)* references `HubConfigEntry.Name`.
- ` rolloutFlag` *(enum: `legacy-only`, `dual`, `modular`)* indicates traffic split for that hub.
- `FallbackDeadline` *(timestamp, optional)* when legacy path will be removed.
- **Storage**: In-memory map derived from config + environment flags; optionally surfaced via diagnostics endpoint.
## State Transitions ## State Transitions
1. **Module Adoption** 1. **Module Adoption**
- Start: `HubConfigEntry.Module = "legacy"`. - Start: `HubConfigEntry.Type = "legacy"` (or other baseline).
- Transition: operator edits config to new module key, runs validation. - Transition: operator edits config to new module type, runs validation.
- Result: registry resolves new module, `LegacyAdapterState` updated to `dual` until rollout flag toggled fully. - Result: registry resolves new module and routes all traffic to it immediately.
2. **Cache Strategy Update** 2. **Cache Strategy Update**
- Start: Module uses default TTL. - Start: Module uses default TTL.

View File

@@ -21,7 +21,7 @@ Modularize the proxy and cache layers so every hub type (npm, Docker, PyPI, futu
**Constraints**: 禁止 Web UI 或账号体系;所有行为受单一 TOML 配置控制;每个 Hub 需独立 Domain/Port 绑定;仅匿名访问 **Constraints**: 禁止 Web UI 或账号体系;所有行为受单一 TOML 配置控制;每个 Hub 需独立 Domain/Port 绑定;仅匿名访问
**Scale/Scope**: 支撑 Docker/NPM/Go/PyPI 等多仓代理,面向弱网及离线缓存复用场景 **Scale/Scope**: 支撑 Docker/NPM/Go/PyPI 等多仓代理,面向弱网及离线缓存复用场景
**Module Registry Location**: `internal/hubmodule/registry.go` 暴露注册/解析 API模块子目录位于 `internal/hubmodule/<name>/` **Module Registry Location**: `internal/hubmodule/registry.go` 暴露注册/解析 API模块子目录位于 `internal/hubmodule/<name>/`
**Config Binding for Modules**: `[[Hub]].Module` 字段控制模块名,默认 `legacy`,配置加载阶段校验必须命中已注册模块 **Config Binding for Modules**: `[[Hub]].Type` 字段控制模块名(同名映射),配置加载阶段校验类型对应的模块已注册
## Constitution Check ## Constitution Check
@@ -29,7 +29,7 @@ Modularize the proxy and cache layers so every hub type (npm, Docker, PyPI, futu
- Feature 仍然是“轻量多仓 CLI 代理”,未引入 Web UI、账号体系或与代理无关的能力。 - Feature 仍然是“轻量多仓 CLI 代理”,未引入 Web UI、账号体系或与代理无关的能力。
- 仅使用 Go + 宪法指定依赖;任何新第三方库都已在本计划中说明理由与审核结论。 - 仅使用 Go + 宪法指定依赖;任何新第三方库都已在本计划中说明理由与审核结论。
- 行为完全由 `config.toml` 控制,新增 `[[Hub]].Module` 配置项已规划默认值、校验与迁移策略 - 行为完全由 `config.toml` 控制,`[[Hub]].Type` 直接驱动模块绑定,校验列表随模块扩展更新
- 方案维持缓存优先 + 流式回源路径,并给出命中/回源/失败的日志与观测手段。 - 方案维持缓存优先 + 流式回源路径,并给出命中/回源/失败的日志与观测手段。
- 计划内列出了配置解析、缓存读写、Host Header 路由等强制测试与中文注释交付范围。 - 计划内列出了配置解析、缓存读写、Host Header 路由等强制测试与中文注释交付范围。
@@ -103,14 +103,14 @@ tests/ # `go test` 下的单元/集成测试,用临时目
### Post-Design Constitution Check ### Post-Design Constitution Check
- New diagnostics endpoint remains internal and optional; no UI/login introduced. ✅ Principle I - New diagnostics endpoint remains internal and optional; no UI/login introduced. ✅ Principle I
- Code still single Go binary with existing dependency set. ✅ Principle II - Code still single Go binary with existing dependency set. ✅ Principle II
- `Module` field documented with defaults, validation, and migration path; no extra config sources. ✅ Principle III - `Type` 字段即模块绑定点,文档与校验同步更新;无额外配置源。 ✅ Principle III
- Cache strategy enforces“原始路径 == 磁盘路径”的布局与流式回源,相关观测需求写入 contracts。✅ Principle IV - Cache strategy enforces“原始路径 == 磁盘路径”的布局与流式回源,相关观测需求写入 contracts。✅ Principle IV
- Logs/quickstart/test guidance ensure observability and Chinese documentation continue. ✅ Principle V - Logs/quickstart/test guidance ensure observability and Chinese documentation continue. ✅ Principle V
## Phase 2 Implementation Outlook (pre-tasks) ## Phase 2 Implementation Outlook (pre-tasks)
1. **Module Registry & Interfaces**: Create `internal/hubmodule` package, define shared interfaces, implement registry with tests, and expose diagnostics data source reused by HTTP endpoints. 1. **Module Registry & Interfaces**: Create `internal/hubmodule` package, define shared interfaces, implement registry with tests, and expose diagnostics data source reused by HTTP endpoints.
2. **Config Loader & Validation**: Extend `internal/config/types.go` and `validation.go` to include `Module` with default `legacy`, plus wiring to registry resolution during startup. 2. **Config Loader & Validation**: Extend `internal/config/types.go` and `validation.go` to bind modules via `Type`, plus wiring to registry resolution during startup.
3. **Legacy Adapter & Migration Switches**: Provide adapter module that wraps current shared proxy/cache, plus feature flags or config toggles to control rollout states per hub. 3. **Legacy Adapter & Migration Switches**: Provide adapter module that wraps current shared proxy/cache, plus feature flags or config toggles to control rollout states per hub.
4. **Module Implementations**: Carve existing npm/docker/pypi logic into dedicated modules within `internal/hubmodule/`, ensuring cache writer复用原始请求路径与必要的 telemetry 标签。 4. **Module Implementations**: Carve existing npm/docker/pypi logic into dedicated modules within `internal/hubmodule/`, ensuring cache writer复用原始请求路径与必要的 telemetry 标签。
5. **Observability/Diagnostics**: Implement `//modules` endpoint (Fiber route) and log tags showing `module_key` on cache/proxy events. 5. **Observability/Diagnostics**: Implement `//modules` endpoint (Fiber route) and log tags showing `module_key` on cache/proxy events.

View File

@@ -12,20 +12,20 @@
4. Add tests under the module directory and run `make modules-test` (delegates to `go test ./internal/hubmodule/...`). 4. Add tests under the module directory and run `make modules-test` (delegates to `go test ./internal/hubmodule/...`).
## 3. Bind Module via Config ## 3. Bind Module via Config
1. Edit `config.toml` and set `Module = "<module-key>"` inside the target `[[Hub]]` block (omit to use `legacy`). 1. Add your module type to `internal/config/validation.go` and the sample config if it represents a new protocol.
2. While validating a new module, set `Rollout = "dual"` so you can flip back to legacy without editing other fields. 2. Edit `config.toml` and set `Type = "<module-type>"` inside the target `[[Hub]]` block.
3. (Optional) Override cache behavior per hub using existing fields (`CacheTTL`, etc.). 3. (Optional) Override cache behavior per hub using existing fields (`CacheTTL`, etc.).
4. Run `ANY_HUB_CONFIG=./config.toml go test ./...` (or `make modules-test`) to ensure loader validation passes and the module registry sees your key. 4. Run `ANY_HUB_CONFIG=./config.toml go test ./...` (or `make modules-test`) to ensure loader validation passes and the module registry sees your key.
## 4. Run and Verify ## 4. Run and Verify
1. Start the binary: `go run ./cmd/any-hub --config ./config.toml`. 1. Start the binary: `go run ./cmd/any-hub --config ./config.toml`.
2. Use `curl -H "Host: <hub-domain>" http://127.0.0.1:<port>/<path>` to produce traffic, then hit `curl http://127.0.0.1:<port>/-/modules` and confirm the hub binding points to your module with the expected `rollout_flag`. 2. Use `curl -H "Host: <hub-domain>" http://127.0.0.1:<port>/<path>` to produce traffic, then hit `curl http://127.0.0.1:<port>/-/modules` and confirm the hub binding points to your module key.
3. Inspect `./storage/<hub>/` to confirm the cached files mirror the upstream path (no suffix). When a path also has child entries (e.g., `/pkg` metadata plus `/pkg/-/...` tarballs), the metadata payload is stored in a `__content` file under that directory so both artifacts can coexist. PyPI Simple responses rewrite distribution links to `/files/<scheme>/<host>/<path>` so that wheels/tarballs are fetched through the proxy and cached alongside the HTML/JSON index. Verify TTL overrides are propagated. 3. Inspect `./storage/<hub>/` to confirm the cached files mirror the upstream path (no suffix). When a path also has child entries (e.g., `/pkg` metadata plus `/pkg/-/...` tarballs), the metadata payload is stored in a `__content` file under that directory so both artifacts can coexist. PyPI Simple responses rewrite distribution links to `/files/<scheme>/<host>/<path>` so that wheels/tarballs are fetched through the proxy and cached alongside the HTML/JSON index. Verify TTL overrides are propagated.
4. Monitor `logs/any-hub.log` (or the sample `logs/module_migration_sample.log`) to verify each entry exposes `module_key` + `rollout_flag`. Example: 4. Monitor `logs/any-hub.log` (or the sample `logs/module_migration_sample.log`) to verify each entry exposes `module_key`. Example:
```json ```json
{"action":"proxy","hub":"testhub","module_key":"testhub","rollout_flag":"dual","cache_hit":false,"upstream_status":200} {"action":"proxy","hub":"testhub","module_key":"testhub","cache_hit":false,"upstream_status":200}
``` ```
5. Exercise rollback by switching `Rollout = "legacy-only"` (or `Module = "legacy"` if needed) and re-running the traffic to ensure diagnostics/logs show the transition. 5. Exercise rollback by reverting the config change (or type rename) and re-running the traffic to ensure diagnostics/logs show the transition.
## 5. Ship ## 5. Ship
1. Commit module code + config docs. 1. Commit module code + config docs.

View File

@@ -15,7 +15,7 @@
## Phase 2: Foundational (Blocking Prerequisites) ## Phase 2: Foundational (Blocking Prerequisites)
- [X] T003 Create shared module interfaces + registry in `internal/hubmodule/interfaces.go` and `internal/hubmodule/registry.go` - [X] T003 Create shared module interfaces + registry in `internal/hubmodule/interfaces.go` and `internal/hubmodule/registry.go`
- [X] T004 Extend config schema with `[[Hub]].Module` defaults/validation plus sample configs in `internal/config/{types.go,validation.go,loader.go}` and `configs/*.toml` - [X] T004 Extend config schema with module defaults/validation2025-03 起由 `Type` 直接驱动,`[[Hub]].Module` 已淘汰)
- [X] T005 [P] Wire server bootstrap to resolve modules once and inject into proxy/cache layers (`internal/server/bootstrap.go`, `internal/proxy/handler.go`) - [X] T005 [P] Wire server bootstrap to resolve modules once and inject into proxy/cache layers (`internal/server/bootstrap.go`, `internal/proxy/handler.go`)
**Checkpoint**: Registry + config plumbing complete; user story work may begin. **Checkpoint**: Registry + config plumbing complete; user story work may begin.
@@ -61,6 +61,8 @@
## Phase 5: User Story 3 - Operate Mixed Generations During Migration (Priority: P3) ## Phase 5: User Story 3 - Operate Mixed Generations During Migration (Priority: P3)
> 2025-03: Rollout flags were removed; this section remains for historical tracking only.
**Goal**: Support dual-path deployments with diagnostics/logging to track legacy vs. modular hubs. **Goal**: Support dual-path deployments with diagnostics/logging to track legacy vs. modular hubs.
**Independent Test**: Run mixed legacy/modular hubs, flip rollout flags, and confirm logs + diagnostics show module ownership and allow rollback. **Independent Test**: Run mixed legacy/modular hubs, flip rollout flags, and confirm logs + diagnostics show module ownership and allow rollback.
@@ -73,7 +75,7 @@
- [X] T019 [US3] Implement `LegacyAdapterState` tracker + rollout flag parsing (`internal/hubmodule/legacy/state.go`, `internal/config/runtime_flags.go`) - [X] T019 [US3] Implement `LegacyAdapterState` tracker + rollout flag parsing (`internal/hubmodule/legacy/state.go`, `internal/config/runtime_flags.go`)
- [X] T020 [US3] Implement Fiber handler + routing for `//modules` diagnostics (`internal/server/routes/modules.go`, `internal/server/router.go`) - [X] T020 [US3] Implement Fiber handler + routing for `//modules` diagnostics (`internal/server/routes/modules.go`, `internal/server/router.go`)
- [X] T021 [US3] Add structured log fields (`module_key`, `rollout_flag`) across logging middleware (`internal/server/middleware/logging.go`, `internal/proxy/logging.go`) - [X] T021 [US3] Add structured log fields (`module_key`) across logging middleware (`internal/server/middleware/logging.go`, `internal/proxy/logging.go`)
- [X] T022 [US3] Document operational playbook for phased migration (`docs/operations/migration.md`) - [X] T022 [US3] Document operational playbook for phased migration (`docs/operations/migration.md`)
--- ---

View File

@@ -9,5 +9,5 @@ If future external endpoints are added, document them here with request/response
## Error Behaviors ## Error Behaviors
- **module_handler_missing**: Forwarder无法找到给定 module_key 的 handler 时返回 `500 {"error":"module_handler_missing"}`,并记录 `hub/domain/module_key/rollout_flag` 等日志字段,便于排查配置缺失或注册遗漏。 - **module_handler_missing**: Forwarder无法找到给定 module_key 的 handler 时返回 `500 {"error":"module_handler_missing"}`,并记录 `hub/domain/module_key` 等日志字段,便于排查配置缺失或注册遗漏。
- **module_handler_panic**: Module handler panic 被捕获后返回 `500 {"error":"module_handler_panic"}`,同时输出结构化日志 `error=module_handler_panic`,防止进程崩溃并提供观测。 - **module_handler_panic**: Module handler panic 被捕获后返回 `500 {"error":"module_handler_panic"}`,同时输出结构化日志 `error=module_handler_panic`,防止进程崩溃并提供观测。

View File

@@ -14,7 +14,7 @@
- **Proxy Dispatcher** - **Proxy Dispatcher**
- Attributes: handler map (module_key → handler), default handler fallback. - Attributes: handler map (module_key → handler), default handler fallback.
- Behavior: Lookup by route.ModuleKey and invoke handler; wraps errors/logging. - Behavior: Lookup by the hub's module key (derived from Type / route.Module.Key) and invoke handler; wraps errors/logging.
- Constraints: If handler missing, return 5xx with observable logging. - Constraints: If handler missing, return 5xx with observable logging.
- **Cache Policy** - **Cache Policy**

View File

@@ -27,7 +27,7 @@
## Relationships ## Relationships
- Hub module 注册时同时在 HookRegistry 与 Forwarder handler map 建立关联。 - Hub module 注册时同时在 HookRegistry 与 Forwarder handler map 建立关联。
- ProxyDispatcher 在请求进入后根据 route.ModuleKey 查询 Hook + handler。 - ProxyDispatcher 在请求进入后根据 route.Module.Key(来自 Hub Type查询 Hook + handler。
- Diagnostics 依赖 HookRegistry 与 HubRegistry 联合输出状态。 - Diagnostics 依赖 HookRegistry 与 HubRegistry 联合输出状态。
## Lifecycle ## Lifecycle

View File

@@ -40,5 +40,5 @@ curl -s http://localhost:8080/-/modules | jq '.modules[].hook_status'
``` ```
- 确认新模块标记为 `registered`,未注册模块显示 `missing`legacy handler 仍可作为兜底。 - 确认新模块标记为 `registered`,未注册模块显示 `missing`legacy handler 仍可作为兜底。
- 如果需要查看全局状态,可检查 `hook_registry` 字段,它返回每个 module_key 的注册情况。 - 如果需要查看全局状态,可检查 `hook_registry` 字段,它返回每个 module_key 的注册情况。
- `hubs[].legacy_only``true` 时表示该 Hub 仍绑定 legacy 模块;迁移完成后应显式设置 `[[Hub]].Module` - `hubs[].module_key` 应与配置中的 `Type` 对齐legacy 模块仅作为兜底存在,推荐尽快替换为协议专用模块
- 启动阶段会验证每个模块是否注册 Hook缺失则直接退出避免运行期静默回退。 - 启动阶段会验证每个模块是否注册 Hook缺失则直接退出避免运行期静默回退。

View File

@@ -41,7 +41,6 @@ func TestAPKProxyCachesIndexAndPackages(t *testing.T) {
Name: "apk", Name: "apk",
Domain: "apk.hub.local", Domain: "apk.hub.local",
Type: "apk", Type: "apk",
Module: "apk",
Upstream: stub.URL, Upstream: stub.URL,
}, },
}, },

View File

@@ -41,7 +41,6 @@ func TestAptPackagesCachedWithoutRevalidate(t *testing.T) {
Name: "apt", Name: "apt",
Domain: "apt.hub.local", Domain: "apt.hub.local",
Type: "debian", Type: "debian",
Module: "debian",
Upstream: stub.URL, Upstream: stub.URL,
}, },
}, },

View File

@@ -36,7 +36,6 @@ func TestAptUpdateCachesIndexes(t *testing.T) {
Name: "apt", Name: "apt",
Domain: "apt.hub.local", Domain: "apt.hub.local",
Type: "debian", Type: "debian",
Module: "debian",
Upstream: stub.URL, Upstream: stub.URL,
}, },
}, },

View File

@@ -44,7 +44,6 @@ func TestCacheFlowWithConditionalRequest(t *testing.T) {
Name: "docker", Name: "docker",
Domain: "docker.hub.local", Domain: "docker.hub.local",
Type: "docker", Type: "docker",
Module: "docker",
Upstream: upstream.URL, Upstream: upstream.URL,
}, },
}, },
@@ -146,7 +145,6 @@ func TestDockerManifestHeadDoesNotOverwriteCache(t *testing.T) {
Name: "docker", Name: "docker",
Domain: "docker.hub.local", Domain: "docker.hub.local",
Type: "docker", Type: "docker",
Module: "docker",
Upstream: upstream.URL, Upstream: upstream.URL,
}, },
}, },

View File

@@ -35,7 +35,6 @@ func TestCacheStrategyOverrides(t *testing.T) {
Name: "npm-ttl", Name: "npm-ttl",
Domain: "ttl.npm.local", Domain: "ttl.npm.local",
Type: "npm", Type: "npm",
Module: "npm",
Upstream: stub.URL, Upstream: stub.URL,
CacheTTL: config.Duration(ttl), CacheTTL: config.Duration(ttl),
}, },
@@ -111,7 +110,6 @@ func TestCacheStrategyOverrides(t *testing.T) {
Name: "npm-novalidation", Name: "npm-novalidation",
Domain: "novalidation.npm.local", Domain: "novalidation.npm.local",
Type: "npm", Type: "npm",
Module: "npm",
Upstream: stub.URL, Upstream: stub.URL,
CacheTTL: config.Duration(ttl), CacheTTL: config.Duration(ttl),
ValidationMode: string(hubmodule.ValidationModeNever), ValidationMode: string(hubmodule.ValidationModeNever),

View File

@@ -1,118 +0,0 @@
package integration
import (
"io"
"net/http/httptest"
"testing"
"time"
"github.com/gofiber/fiber/v3"
"github.com/sirupsen/logrus"
"github.com/any-hub/any-hub/internal/config"
"github.com/any-hub/any-hub/internal/hubmodule"
"github.com/any-hub/any-hub/internal/hubmodule/legacy"
"github.com/any-hub/any-hub/internal/server"
)
func TestLegacyAdapterRolloutToggle(t *testing.T) {
const moduleKey = "rollout-toggle-test"
_ = hubmodule.Register(hubmodule.ModuleMetadata{Key: moduleKey})
logger := logrus.New()
logger.SetOutput(io.Discard)
baseHub := config.HubConfig{
Name: "dual-mode",
Domain: "dual.local",
Type: "docker",
Upstream: "https://registry.npmjs.org",
Module: moduleKey,
}
testCases := []struct {
name string
rolloutFlag string
expectKey string
expectFlag legacy.RolloutFlag
}{
{
name: "force legacy",
rolloutFlag: "legacy-only",
expectKey: hubmodule.DefaultModuleKey(),
expectFlag: legacy.RolloutLegacyOnly,
},
{
name: "dual mode",
rolloutFlag: "dual",
expectKey: moduleKey,
expectFlag: legacy.RolloutDual,
},
{
name: "full modular",
rolloutFlag: "modular",
expectKey: moduleKey,
expectFlag: legacy.RolloutModular,
},
{
name: "rollback to legacy",
rolloutFlag: "legacy-only",
expectKey: hubmodule.DefaultModuleKey(),
expectFlag: legacy.RolloutLegacyOnly,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
cfg := &config.Config{
Global: config.GlobalConfig{
ListenPort: 6100,
CacheTTL: config.Duration(time.Minute),
},
Hubs: []config.HubConfig{
func() config.HubConfig {
h := baseHub
h.Rollout = tc.rolloutFlag
return h
}(),
},
}
registry, err := server.NewHubRegistry(cfg)
if err != nil {
t.Fatalf("failed to build registry: %v", err)
}
recorder := &routeRecorder{}
app := mustNewApp(t, cfg.Global.ListenPort, logger, registry, recorder)
req := httptest.NewRequest("GET", "http://dual.local/v2/", nil)
req.Host = "dual.local"
resp, err := app.Test(req)
if err != nil {
t.Fatalf("request failed: %v", err)
}
if resp.StatusCode != fiber.StatusNoContent {
t.Fatalf("unexpected status: %d", resp.StatusCode)
}
if recorder.moduleKey != tc.expectKey {
t.Fatalf("expected module %s, got %s", tc.expectKey, recorder.moduleKey)
}
if recorder.rolloutFlag != tc.expectFlag {
t.Fatalf("expected rollout flag %s, got %s", tc.expectFlag, recorder.rolloutFlag)
}
})
}
}
type routeRecorder struct {
moduleKey string
rolloutFlag legacy.RolloutFlag
}
func (r *routeRecorder) Handle(c fiber.Ctx, route *server.HubRoute) error {
r.moduleKey = route.ModuleKey
r.rolloutFlag = route.RolloutFlag
return c.SendStatus(fiber.StatusNoContent)
}

View File

@@ -13,7 +13,6 @@ import (
"github.com/any-hub/any-hub/internal/config" "github.com/any-hub/any-hub/internal/config"
"github.com/any-hub/any-hub/internal/hubmodule" "github.com/any-hub/any-hub/internal/hubmodule"
"github.com/any-hub/any-hub/internal/hubmodule/legacy"
"github.com/any-hub/any-hub/internal/proxy/hooks" "github.com/any-hub/any-hub/internal/proxy/hooks"
"github.com/any-hub/any-hub/internal/server" "github.com/any-hub/any-hub/internal/server"
"github.com/any-hub/any-hub/internal/server/routes" "github.com/any-hub/any-hub/internal/server/routes"
@@ -37,21 +36,17 @@ func TestModuleDiagnosticsEndpoints(t *testing.T) {
CacheTTL: config.Duration(30 * time.Minute), CacheTTL: config.Duration(30 * time.Minute),
}, },
Hubs: []config.HubConfig{ Hubs: []config.HubConfig{
{
Name: "legacy-hub",
Domain: "legacy.local",
Type: "docker",
Upstream: "https://registry-1.docker.io",
Module: hubmodule.DefaultModuleKey(),
Rollout: string(legacy.RolloutLegacyOnly),
},
{ {
Name: "modern-hub", Name: "modern-hub",
Domain: "modern.local", Domain: "modern.local",
Type: "npm", Type: "npm",
Upstream: "https://registry.npmjs.org", Upstream: "https://registry.npmjs.org",
Module: moduleKey, },
Rollout: "dual", {
Name: "docker-hub",
Domain: "docker.local",
Type: "docker",
Upstream: "https://registry-1.docker.io",
}, },
}, },
} }
@@ -79,10 +74,8 @@ func TestModuleDiagnosticsEndpoints(t *testing.T) {
Hubs []struct { Hubs []struct {
HubName string `json:"hub_name"` HubName string `json:"hub_name"`
ModuleKey string `json:"module_key"` ModuleKey string `json:"module_key"`
Rollout string `json:"rollout_flag"`
Domain string `json:"domain"` Domain string `json:"domain"`
Port int `json:"port"` Port int `json:"port"`
LegacyOnly bool `json:"legacy_only"`
} `json:"hubs"` } `json:"hubs"`
} }
body, _ := io.ReadAll(resp.Body) body, _ := io.ReadAll(resp.Body)
@@ -111,22 +104,13 @@ func TestModuleDiagnosticsEndpoints(t *testing.T) {
} }
for _, hub := range payload.Hubs { for _, hub := range payload.Hubs {
switch hub.HubName { switch hub.HubName {
case "legacy-hub":
if hub.ModuleKey != hubmodule.DefaultModuleKey() {
t.Fatalf("legacy hub should expose legacy module, got %s", hub.ModuleKey)
}
if !hub.LegacyOnly {
t.Fatalf("legacy hub should be marked legacy_only")
}
case "modern-hub": case "modern-hub":
if hub.ModuleKey != moduleKey { if hub.ModuleKey != "npm" {
t.Fatalf("modern hub should expose %s, got %s", moduleKey, hub.ModuleKey) t.Fatalf("modern hub should expose npm, got %s", hub.ModuleKey)
} }
if hub.Rollout != "dual" { case "docker-hub":
t.Fatalf("modern hub rollout flag should be dual, got %s", hub.Rollout) if hub.ModuleKey != "docker" {
} t.Fatalf("docker hub should expose docker, got %s", hub.ModuleKey)
if hub.LegacyOnly {
t.Fatalf("modern hub should not be marked legacy_only")
} }
default: default:
t.Fatalf("unexpected hub %s", hub.HubName) t.Fatalf("unexpected hub %s", hub.HubName)

View File

@@ -9,13 +9,10 @@ import (
"github.com/sirupsen/logrus" "github.com/sirupsen/logrus"
"github.com/any-hub/any-hub/internal/config" "github.com/any-hub/any-hub/internal/config"
"github.com/any-hub/any-hub/internal/hubmodule"
"github.com/any-hub/any-hub/internal/server" "github.com/any-hub/any-hub/internal/server"
) )
func TestModuleRoutingIsolation(t *testing.T) { func TestModuleRoutingIsolation(t *testing.T) {
_ = hubmodule.Register(hubmodule.ModuleMetadata{Key: "module-routing-test"})
cfg := &config.Config{ cfg := &config.Config{
Global: config.GlobalConfig{ Global: config.GlobalConfig{
ListenPort: 6000, ListenPort: 6000,
@@ -23,17 +20,15 @@ func TestModuleRoutingIsolation(t *testing.T) {
}, },
Hubs: []config.HubConfig{ Hubs: []config.HubConfig{
{ {
Name: "legacy", Name: "docker-hub",
Domain: "legacy.hub.local", Domain: "legacy.hub.local",
Type: "docker", Type: "docker",
Module: "legacy",
Upstream: "https://registry-1.docker.io", Upstream: "https://registry-1.docker.io",
}, },
{ {
Name: "test", Name: "npm-hub",
Domain: "test.hub.local", Domain: "test.hub.local",
Type: "npm", Type: "npm",
Module: "module-routing-test",
Upstream: "https://registry.example.com", Upstream: "https://registry.example.com",
}, },
}, },
@@ -60,8 +55,8 @@ func TestModuleRoutingIsolation(t *testing.T) {
if resp.StatusCode != fiber.StatusNoContent { if resp.StatusCode != fiber.StatusNoContent {
t.Fatalf("legacy hub should return 204, got %d", resp.StatusCode) t.Fatalf("legacy hub should return 204, got %d", resp.StatusCode)
} }
if recorder.moduleKey != "legacy" { if recorder.moduleKey != "docker" {
t.Fatalf("expected legacy module, got %s", recorder.moduleKey) t.Fatalf("expected docker module, got %s", recorder.moduleKey)
} }
testReq := httptest.NewRequest("GET", "http://test.hub.local/v2/", nil) testReq := httptest.NewRequest("GET", "http://test.hub.local/v2/", nil)
@@ -74,8 +69,8 @@ func TestModuleRoutingIsolation(t *testing.T) {
if resp2.StatusCode != fiber.StatusNoContent { if resp2.StatusCode != fiber.StatusNoContent {
t.Fatalf("test hub should return 204, got %d", resp2.StatusCode) t.Fatalf("test hub should return 204, got %d", resp2.StatusCode)
} }
if recorder.moduleKey != "module-routing-test" { if recorder.moduleKey != "npm" {
t.Fatalf("expected module-routing-test module, got %s", recorder.moduleKey) t.Fatalf("expected npm module, got %s", recorder.moduleKey)
} }
} }
@@ -96,12 +91,10 @@ func mustNewApp(t *testing.T, port int, logger *logrus.Logger, registry *server.
type moduleRecorder struct { type moduleRecorder struct {
routeName string routeName string
moduleKey string moduleKey string
rollout string
} }
func (p *moduleRecorder) Handle(c fiber.Ctx, route *server.HubRoute) error { func (p *moduleRecorder) Handle(c fiber.Ctx, route *server.HubRoute) error {
p.routeName = route.Config.Name p.routeName = route.Config.Name
p.moduleKey = route.ModuleKey p.moduleKey = route.Module.Key
p.rollout = string(route.RolloutFlag)
return c.SendStatus(fiber.StatusNoContent) return c.SendStatus(fiber.StatusNoContent)
} }