feat: remove module/rollout config key
This commit is contained in:
@@ -94,6 +94,3 @@ components:
|
||||
type: string
|
||||
port:
|
||||
type: integer
|
||||
rollout_flag:
|
||||
type: string
|
||||
enum: [legacy-only, dual, modular]
|
||||
|
||||
@@ -13,13 +13,12 @@ The modular architecture introduces explicit metadata describing which proxy+cac
|
||||
- `Domain` *(string, required)* – hostname clients access; must be unique per process.
|
||||
- `Port` *(int, required)* – listen port; validated to 1–65535.
|
||||
- `Upstream` *(string, required)* – base URL for upstream registry; must be HTTPS or explicitly whitelisted HTTP.
|
||||
- `Module` *(string, optional, default `"legacy"`)* – key resolved through module registry. Validation ensures module exists at load time.
|
||||
- `Type` *(string, required)* – module selector; validated against the registered module set.
|
||||
- `CacheTTL`, `Proxy`, and other overrides *(optional)* – reuse existing schema; modules may read these via dependency injection.
|
||||
- **Relationships**:
|
||||
- `HubConfigEntry.Module` → `ModuleMetadata.Key` (many-to-one).
|
||||
- `HubConfigEntry.Type` → `ModuleMetadata.Key` (many-to-one).
|
||||
- **Validation Rules**:
|
||||
- Missing `Module` implicitly maps to `legacy` to preserve backward compatibility.
|
||||
- Changing `Module` requires a migration plan; config loader logs module name for observability.
|
||||
- Invalid `Type` values reject config load; operators must add the type to the supported list alongside module registration.
|
||||
|
||||
### 2. ModuleMetadata
|
||||
- **Fields**:
|
||||
@@ -57,20 +56,12 @@ The modular architecture introduces explicit metadata describing which proxy+cac
|
||||
- TTL must be positive.
|
||||
- Modules flagged as `SupportsStreamingWrite=false` must document fallback behavior before registration.
|
||||
|
||||
### 5. LegacyAdapterState
|
||||
- **Purpose**: Tracks which hubs still run through the old shared implementation to support progressive migration.
|
||||
- **Fields**:
|
||||
- `HubName` *(string)* – references `HubConfigEntry.Name`.
|
||||
- ` rolloutFlag` *(enum: `legacy-only`, `dual`, `modular`)* – indicates traffic split for that hub.
|
||||
- `FallbackDeadline` *(timestamp, optional)* – when legacy path will be removed.
|
||||
- **Storage**: In-memory map derived from config + environment flags; optionally surfaced via diagnostics endpoint.
|
||||
|
||||
## State Transitions
|
||||
|
||||
1. **Module Adoption**
|
||||
- Start: `HubConfigEntry.Module = "legacy"`.
|
||||
- Transition: operator edits config to new module key, runs validation.
|
||||
- Result: registry resolves new module, `LegacyAdapterState` updated to `dual` until rollout flag toggled fully.
|
||||
- Start: `HubConfigEntry.Type = "legacy"` (or other baseline).
|
||||
- Transition: operator edits config to new module type, runs validation.
|
||||
- Result: registry resolves new module and routes all traffic to it immediately.
|
||||
|
||||
2. **Cache Strategy Update**
|
||||
- Start: Module uses default TTL.
|
||||
|
||||
@@ -21,7 +21,7 @@ Modularize the proxy and cache layers so every hub type (npm, Docker, PyPI, futu
|
||||
**Constraints**: 禁止 Web UI 或账号体系；所有行为受单一 TOML 配置控制；每个 Hub 需独立 Domain/Port 绑定；仅匿名访问
|
||||
**Scale/Scope**: 支撑 Docker/NPM/Go/PyPI 等多仓代理，面向弱网及离线缓存复用场景
|
||||
**Module Registry Location**: `internal/hubmodule/registry.go` 暴露注册/解析 API，模块子目录位于 `internal/hubmodule/<name>/`
|
||||
**Config Binding for Modules**: `[[Hub]].Module` 字段控制模块名，默认 `legacy`，配置加载阶段校验必须命中已注册模块
|
||||
**Config Binding for Modules**: `[[Hub]].Type` 字段控制模块名（同名映射），配置加载阶段需校验类型对应的模块已注册
|
||||
|
||||
## Constitution Check
|
||||
|
||||
@@ -29,7 +29,7 @@ Modularize the proxy and cache layers so every hub type (npm, Docker, PyPI, futu
|
||||
|
||||
- Feature 仍然是“轻量多仓 CLI 代理”，未引入 Web UI、账号体系或与代理无关的能力。
|
||||
- 仅使用 Go + 宪法指定依赖；任何新第三方库都已在本计划中说明理由与审核结论。
|
||||
- 行为完全由 `config.toml` 控制，新增 `[[Hub]].Module` 配置项已规划默认值、校验与迁移策略。
|
||||
- 行为完全由 `config.toml` 控制，`[[Hub]].Type` 直接驱动模块绑定，校验列表随模块扩展更新。
|
||||
- 方案维持缓存优先 + 流式回源路径，并给出命中/回源/失败的日志与观测手段。
|
||||
- 计划内列出了配置解析、缓存读写、Host Header 路由等强制测试与中文注释交付范围。
|
||||
|
||||
@@ -103,14 +103,14 @@ tests/ # `go test` 下的单元/集成测试，用临时目
|
||||
### Post-Design Constitution Check
|
||||
- New diagnostics endpoint remains internal and optional; no UI/login introduced. ✅ Principle I
|
||||
- Code still single Go binary with existing dependency set. ✅ Principle II
|
||||
- `Module` field documented with defaults, validation, and migration path; no extra config sources. ✅ Principle III
|
||||
- `Type` 字段即模块绑定点，文档与校验同步更新；无额外配置源。 ✅ Principle III
|
||||
- Cache strategy enforces“原始路径 == 磁盘路径”的布局与流式回源，相关观测需求写入 contracts。✅ Principle IV
|
||||
- Logs/quickstart/test guidance ensure observability and Chinese documentation continue. ✅ Principle V
|
||||
|
||||
## Phase 2 – Implementation Outlook (pre-tasks)
|
||||
|
||||
1. **Module Registry & Interfaces**: Create `internal/hubmodule` package, define shared interfaces, implement registry with tests, and expose diagnostics data source reused by HTTP endpoints.
|
||||
2. **Config Loader & Validation**: Extend `internal/config/types.go` and `validation.go` to include `Module` with default `legacy`, plus wiring to registry resolution during startup.
|
||||
2. **Config Loader & Validation**: Extend `internal/config/types.go` and `validation.go` to bind modules via `Type`, plus wiring to registry resolution during startup.
|
||||
3. **Legacy Adapter & Migration Switches**: Provide adapter module that wraps current shared proxy/cache, plus feature flags or config toggles to control rollout states per hub.
|
||||
4. **Module Implementations**: Carve existing npm/docker/pypi logic into dedicated modules within `internal/hubmodule/`, ensuring cache writer复用原始请求路径与必要的 telemetry 标签。
|
||||
5. **Observability/Diagnostics**: Implement `/−/modules` endpoint (Fiber route) and log tags showing `module_key` on cache/proxy events.
|
||||
|
||||
@@ -12,20 +12,20 @@
|
||||
4. Add tests under the module directory and run `make modules-test` (delegates to `go test ./internal/hubmodule/...`).
|
||||
|
||||
## 3. Bind Module via Config
|
||||
1. Edit `config.toml` and set `Module = "<module-key>"` inside the target `[[Hub]]` block (omit to use `legacy`).
|
||||
2. While validating a new module, set `Rollout = "dual"` so you can flip back to legacy without editing other fields.
|
||||
1. Add your module type to `internal/config/validation.go` and the sample config if it represents a new protocol.
|
||||
2. Edit `config.toml` and set `Type = "<module-type>"` inside the target `[[Hub]]` block.
|
||||
3. (Optional) Override cache behavior per hub using existing fields (`CacheTTL`, etc.).
|
||||
4. Run `ANY_HUB_CONFIG=./config.toml go test ./...` (or `make modules-test`) to ensure loader validation passes and the module registry sees your key.
|
||||
|
||||
## 4. Run and Verify
|
||||
1. Start the binary: `go run ./cmd/any-hub --config ./config.toml`.
|
||||
2. Use `curl -H "Host: <hub-domain>" http://127.0.0.1:<port>/<path>` to produce traffic, then hit `curl http://127.0.0.1:<port>/-/modules` and confirm the hub binding points to your module with the expected `rollout_flag`.
|
||||
2. Use `curl -H "Host: <hub-domain>" http://127.0.0.1:<port>/<path>` to produce traffic, then hit `curl http://127.0.0.1:<port>/-/modules` and confirm the hub binding points to your module key.
|
||||
3. Inspect `./storage/<hub>/` to confirm the cached files mirror the upstream path (no suffix). When a path also has child entries (e.g., `/pkg` metadata plus `/pkg/-/...` tarballs), the metadata payload is stored in a `__content` file under that directory so both artifacts can coexist. PyPI Simple responses rewrite distribution links to `/files/<scheme>/<host>/<path>` so that wheels/tarballs are fetched through the proxy and cached alongside the HTML/JSON index. Verify TTL overrides are propagated.
|
||||
4. Monitor `logs/any-hub.log` (or the sample `logs/module_migration_sample.log`) to verify each entry exposes `module_key` + `rollout_flag`. Example:
|
||||
4. Monitor `logs/any-hub.log` (or the sample `logs/module_migration_sample.log`) to verify each entry exposes `module_key`. Example:
|
||||
```json
|
||||
{"action":"proxy","hub":"testhub","module_key":"testhub","rollout_flag":"dual","cache_hit":false,"upstream_status":200}
|
||||
{"action":"proxy","hub":"testhub","module_key":"testhub","cache_hit":false,"upstream_status":200}
|
||||
```
|
||||
5. Exercise rollback by switching `Rollout = "legacy-only"` (or `Module = "legacy"` if needed) and re-running the traffic to ensure diagnostics/logs show the transition.
|
||||
5. Exercise rollback by reverting the config change (or type rename) and re-running the traffic to ensure diagnostics/logs show the transition.
|
||||
|
||||
## 5. Ship
|
||||
1. Commit module code + config docs.
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
## Phase 2: Foundational (Blocking Prerequisites)
|
||||
|
||||
- [X] T003 Create shared module interfaces + registry in `internal/hubmodule/interfaces.go` and `internal/hubmodule/registry.go`
|
||||
- [X] T004 Extend config schema with `[[Hub]].Module` defaults/validation plus sample configs in `internal/config/{types.go,validation.go,loader.go}` and `configs/*.toml`
|
||||
- [X] T004 Extend config schema with module defaults/validation（2025-03 起由 `Type` 直接驱动，`[[Hub]].Module` 已淘汰）
|
||||
- [X] T005 [P] Wire server bootstrap to resolve modules once and inject into proxy/cache layers (`internal/server/bootstrap.go`, `internal/proxy/handler.go`)
|
||||
|
||||
**Checkpoint**: Registry + config plumbing complete; user story work may begin.
|
||||
@@ -61,6 +61,8 @@
|
||||
|
||||
## Phase 5: User Story 3 - Operate Mixed Generations During Migration (Priority: P3)
|
||||
|
||||
> 2025-03: Rollout flags were removed; this section remains for historical tracking only.
|
||||
|
||||
**Goal**: Support dual-path deployments with diagnostics/logging to track legacy vs. modular hubs.
|
||||
**Independent Test**: Run mixed legacy/modular hubs, flip rollout flags, and confirm logs + diagnostics show module ownership and allow rollback.
|
||||
|
||||
@@ -73,7 +75,7 @@
|
||||
|
||||
- [X] T019 [US3] Implement `LegacyAdapterState` tracker + rollout flag parsing (`internal/hubmodule/legacy/state.go`, `internal/config/runtime_flags.go`)
|
||||
- [X] T020 [US3] Implement Fiber handler + routing for `/−/modules` diagnostics (`internal/server/routes/modules.go`, `internal/server/router.go`)
|
||||
- [X] T021 [US3] Add structured log fields (`module_key`, `rollout_flag`) across logging middleware (`internal/server/middleware/logging.go`, `internal/proxy/logging.go`)
|
||||
- [X] T021 [US3] Add structured log fields (`module_key`) across logging middleware (`internal/server/middleware/logging.go`, `internal/proxy/logging.go`)
|
||||
- [X] T022 [US3] Document operational playbook for phased migration (`docs/operations/migration.md`)
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user