add oc
This commit is contained in:
@@ -10,6 +10,8 @@ Auto-generated from all feature plans. Last updated: 2025-11-13
|
||||
- 本地文件系统缓存目录 `StoragePath/<Hub>/<path>`,模块需直接复用原始路径布局 (004-modular-proxy-cache)
|
||||
- 本地文件系统缓存目录 `StoragePath/<Hub>/<path>`(按模块定义的布局) (005-proxy-module-delegation)
|
||||
- 本地文件系统缓存目录 `StoragePath/<Hub>/<path>`(由模块 Hook 定义布局) (006-module-hook-refactor)
|
||||
- Go 1.25+ (静态链接,单二进制) + Fiber v3(HTTP 服务)、Viper(配置)、Logrus + Lumberjack、标准库 `net/http`/`io` (007-apt-apk-cache)
|
||||
- 本地 `StoragePath/<Hub>/<path>` + `.meta`;索引需带 Last-Modified/ETag 元信息;包体按原路径落盘 (007-apt-apk-cache)
|
||||
|
||||
- Go 1.25+ (静态链接,单二进制交付) + Fiber v3(HTTP 服务)、Viper(配置)、Logrus + Lumberjack(结构化日志 [EXTRACTED FROM ALL PLAN.MD FILES] 滚动)、标准库 `net/http`/`io` (001-config-bootstrap)
|
||||
|
||||
@@ -29,9 +31,9 @@ tests/
|
||||
Go 1.25+ (静态链接,单二进制交付): Follow standard conventions
|
||||
|
||||
## Recent Changes
|
||||
- 007-apt-apk-cache: Added Go 1.25+ (静态链接,单二进制) + Fiber v3(HTTP 服务)、Viper(配置)、Logrus + Lumberjack、标准库 `net/http`/`io`
|
||||
- 006-module-hook-refactor: Added Go 1.25+ (静态链接,单二进制交付) + Fiber v3(HTTP 服务)、Viper(配置)、Logrus + Lumberjack(结构化日志 & 滚动)、标准库 `net/http`/`io`
|
||||
- 005-proxy-module-delegation: Added Go 1.25+ (静态链接,单二进制交付) + Fiber v3(HTTP 服务)、Viper(配置)、Logrus + Lumberjack(结构化日志 & 滚动)、标准库 `net/http`/`io`
|
||||
- 004-modular-proxy-cache: Added Go 1.25+ (静态链接,单二进制交付) + Fiber v3(HTTP 服务)、Viper(配置)、Logrus + Lumberjack(结构化日志 & 滚动)、标准库 `net/http`/`io`
|
||||
|
||||
|
||||
<!-- MANUAL ADDITIONS START -->
|
||||
|
||||
34
specs/007-apt-apk-cache/checklists/requirements.md
Normal file
34
specs/007-apt-apk-cache/checklists/requirements.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Specification Quality Checklist: APT/APK 包缓存模块
|
||||
|
||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||
**Created**: 2025-11-17
|
||||
**Feature**: /home/rogee/Projects/any-hub/specs/007-apt-apk-cache/spec.md
|
||||
|
||||
## Content Quality
|
||||
|
||||
- [x] No implementation details (languages, frameworks, APIs)
|
||||
- [x] Focused on user value and business needs
|
||||
- [x] Written for non-technical stakeholders
|
||||
- [x] All mandatory sections completed
|
||||
|
||||
## Requirement Completeness
|
||||
|
||||
- [x] No [NEEDS CLARIFICATION] markers remain
|
||||
- [x] Requirements are testable and unambiguous
|
||||
- [x] Success criteria are measurable
|
||||
- [x] Success criteria are technology-agnostic (no implementation details)
|
||||
- [x] All acceptance scenarios are defined
|
||||
- [x] Edge cases are identified
|
||||
- [x] Scope is clearly bounded
|
||||
- [x] Dependencies and assumptions identified
|
||||
|
||||
## Feature Readiness
|
||||
|
||||
- [x] All functional requirements have clear acceptance criteria
|
||||
- [x] User scenarios cover primary flows
|
||||
- [x] Feature meets measurable outcomes defined in Success Criteria
|
||||
- [x] No implementation details leak into specification
|
||||
|
||||
## Notes
|
||||
|
||||
- Items marked complete; ready to proceed to planning.
|
||||
26
specs/007-apt-apk-cache/contracts/proxy-paths.md
Normal file
26
specs/007-apt-apk-cache/contracts/proxy-paths.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Proxy Path Contracts: APT/APK
|
||||
|
||||
## APT
|
||||
|
||||
| Downstream Request | Upstream Target | Caching | Notes |
|
||||
|--------------------|-----------------|---------|-------|
|
||||
| `/dists/<suite>/<component>/binary-<arch>/Packages[.gz/.xz]` | same path on Upstream | RequireRevalidate=true | Send If-None-Match/If-Modified-Since if cached |
|
||||
| `/dists/<suite>/Release` | same path | RequireRevalidate=true | Preserve content; may have accompanying `.gpg` |
|
||||
| `/dists/<suite>/InRelease` | same path | RequireRevalidate=true | Signed inline; no body changes |
|
||||
| `/dists/<suite>/Release.gpg` | same path | RequireRevalidate=true | Signature only; no rewrite |
|
||||
| `/pool/<vendor>/<name>.deb` | same path | AllowCache/AllowStore=true, RequireRevalidate=false | Treat as immutable |
|
||||
| `/dists/<suite>/by-hash/<algo>/<hash>` | same path | AllowCache/AllowStore=true, RequireRevalidate=false | Path encodes hash, no extra validation |
|
||||
|
||||
## Alpine APK
|
||||
|
||||
| Downstream Request | Upstream Target | Caching | Notes |
|
||||
|--------------------|-----------------|---------|-------|
|
||||
| `/v3.<branch>/main/<arch>/APKINDEX.tar.gz` (and variants) | same path | RequireRevalidate=true | Preserve signature/headers |
|
||||
| `/v3.<branch>/<repo>/<arch>/APKINDEX.tar.gz` | same path | RequireRevalidate=true | Same handling as above |
|
||||
| `/v3.<branch>/<repo>/<arch>/packages/<file>.apk` | same path | AllowCache/AllowStore=true, RequireRevalidate=false | Immutable package |
|
||||
| APKINDEX signature files | same path | RequireRevalidate=true | No rewrite |
|
||||
|
||||
## Behaviors
|
||||
- No body/URL rewrite; proxy only normalizes cache policy per path shape.
|
||||
- Preserve status codes, headers, and signature files exactly as upstream.
|
||||
- Conditional requests (If-None-Match/If-Modified-Since) applied to all index/signature paths when cached.
|
||||
36
specs/007-apt-apk-cache/data-model.md
Normal file
36
specs/007-apt-apk-cache/data-model.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# Data Model: APT/APK 包缓存模块
|
||||
|
||||
## Entities
|
||||
|
||||
### HubConfig (APT/APK)
|
||||
- Fields: `Name`, `Domain`, `Port`, `Upstream`, `Type` (`debian`/`apk`), optional `CacheTTL` override.
|
||||
- Rules: `Type` 必须新增枚举;`Upstream` 必须为 HTTP/HTTPS;每个 Hub 独立缓存前缀。
|
||||
|
||||
### IndexFile
|
||||
- Represents: APT Release/InRelease/Packages*,Alpine APKINDEX。
|
||||
- Attributes: `Path`, `ETag` (if present), `Last-Modified`, `Hash` (if provided in index), `Size`, `ContentType`.
|
||||
- Rules: RequireRevalidate=true;缓存命中需携带条件请求;内容不得修改。
|
||||
|
||||
### PackageFile
|
||||
- Represents: APT `pool/*/*.deb` 包体,Alpine `packages/<arch>/*.apk`。
|
||||
- Attributes: `Path`, `Size`, `Hash` (from upstream metadata if available), `StoredAt`.
|
||||
- Rules: 视作不可变;AllowCache/AllowStore=true;不做本地内容改写。
|
||||
|
||||
### SignatureFile
|
||||
- Represents: APT `Release.gpg`/`InRelease` 自带签名,Alpine APKINDEX 签名文件。
|
||||
- Attributes: `Path`, `Size`, `StoredAt`.
|
||||
- Rules: 原样透传;与对应 IndexFile 绑定,同步缓存与再验证。
|
||||
|
||||
### CacheEntry
|
||||
- Represents: 本地缓存记录(索引或包体)。
|
||||
- Attributes: `LocatorPath`, `ModTime`, `ETag`, `Size`, `AllowStore`, `RequireRevalidate`.
|
||||
- Rules: 读取时决定是否回源;写入需同步 `.meta`;路径格式 `StoragePath/<Hub>/<Path>`.
|
||||
|
||||
## Relationships
|
||||
- HubConfig → IndexFile/PackageFile/SignatureFile 通过 Domain/Upstream 绑定。
|
||||
- IndexFile ↔ SignatureFile:同一索引文件的签名文件需同周期缓存与再验证。
|
||||
- CacheEntry 聚合 IndexFile/PackageFile/SignatureFile 的落盘信息,用于策略判定。
|
||||
|
||||
## State/Transitions
|
||||
- CacheEntry states: `miss` → `fetched` → `validated` (304) / `refreshed` (200) → `stale` (TTL 或上游 200 新内容) → `removed` (404 或清理)。
|
||||
- Transitions triggered by upstream响应:304 保持内容,200 刷新,404 删除相关缓存。
|
||||
94
specs/007-apt-apk-cache/plan.md
Normal file
94
specs/007-apt-apk-cache/plan.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# Implementation Plan: APT/APK 包缓存模块
|
||||
|
||||
**Branch**: `007-apt-apk-cache` | **Date**: 2025-11-17 | **Spec**: `/home/rogee/Projects/any-hub/specs/007-apt-apk-cache/spec.md`
|
||||
**Input**: Feature specification from `/specs/007-apt-apk-cache/spec.md`
|
||||
|
||||
## Summary
|
||||
|
||||
为 any-hub 增加 APT(Debian/Ubuntu)与 Alpine APK 的缓存代理模块:索引(Release/InRelease/Packages*/APKINDEX)必须再验证,包体(pool/*.deb、packages/*)视作不可变直接缓存,支持 Acquire-By-Hash/签名透传,不影响现有各模块。
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Go 1.25+ (静态链接,单二进制)
|
||||
**Primary Dependencies**: Fiber v3(HTTP 服务)、Viper(配置)、Logrus + Lumberjack、标准库 `net/http`/`io`
|
||||
**Storage**: 本地 `StoragePath/<Hub>/<path>` + `.meta`;索引需带 Last-Modified/ETag 元信息;包体按原路径落盘
|
||||
**Protocols**: APT (`/dists/<suite>/<component>/binary-<arch>/Packages*`, `Release`, `InRelease`, `pool/*`), Acquire-By-Hash;Alpine (`APKINDEX.tar.gz`, `packages/<arch>/…`)
|
||||
**Caching Rules**: 索引 RequireRevalidate=true;包体 AllowStore/AllowCache=true、RequireRevalidate=false;签名/校验文件原样透传
|
||||
**Testing**: `go test ./...`,构造 httptest 上游返回 Release/Packages/APKINDEX 与包体,验证首次回源+再验证+命中;使用临时目录校验缓存落盘
|
||||
**Target Platform**: Linux/Unix CLI,systemd/supervisor 托管,匿名客户端
|
||||
**Constraints**: 单一 config.toml 控制;禁止新增第三方依赖;保持现有模块行为不变
|
||||
**Risk/Unknowns**: Acquire-By-Hash 路径映射是否需额外校验逻辑(选择原样透传,路径即校验)
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||
|
||||
- Feature 仍然是“轻量多仓 CLI 代理”,未引入 Web UI、账号体系或与代理无关的能力。
|
||||
- 仅使用 Go + 宪法指定依赖;任何新第三方库都已在本计划中说明理由与审核结论。
|
||||
- 行为完全由 `config.toml` 控制,新增配置项已规划默认值、校验与迁移策略。
|
||||
- 方案维持缓存优先 + 流式回源路径,并给出命中/回源/失败的日志与观测手段。
|
||||
- 计划内列出了配置解析、缓存读写、Host Header 路由等强制测试与中文注释交付范围。
|
||||
|
||||
**Status**: PASS(未引入新依赖/界面,新增配置仅增加模块枚举与示例)
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/007-apt-apk-cache/
|
||||
├── plan.md # 本文件
|
||||
├── research.md # Phase 0 输出
|
||||
├── data-model.md # Phase 1 输出
|
||||
├── quickstart.md # Phase 1 输出
|
||||
├── contracts/ # Phase 1 输出
|
||||
└── tasks.md # Phase 2 (/speckit.tasks 生成)
|
||||
```
|
||||
|
||||
### Source Code (repository root)
|
||||
```text
|
||||
cmd/any-hub/main.go # CLI 入口、参数解析
|
||||
internal/config/ # TOML 加载、默认值、校验
|
||||
internal/server/ # Fiber 服务、路由、中间件
|
||||
internal/cache/ # 磁盘/内存缓存与 .meta 管理
|
||||
internal/proxy/ # 上游访问、缓存策略、流式复制
|
||||
internal/hubmodule/* # 各模块 Hooks/元数据(新增 debian/apk 模块)
|
||||
configs/ # 示例 config.toml(如需)
|
||||
tests/ # go test 下的单元/集成测试
|
||||
```
|
||||
|
||||
**Structure Decision**: 新增模块位于 `internal/hubmodule/debian` 或类似命名;复用现有 hook 注册与缓存策略接口;配置扩展在 `internal/config` 追加枚举与默认值。
|
||||
|
||||
## Complexity Tracking
|
||||
|
||||
> **Fill ONLY if Constitution Check has violations that must be justified**
|
||||
|
||||
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
||||
|-----------|------------|-------------------------------------|
|
||||
| None | — | — |
|
||||
|
||||
## Phase 0: Outline & Research
|
||||
|
||||
### Unknowns / Research Tasks
|
||||
- Acquire-By-Hash 处理策略:是否需要额外校验逻辑 → 选择透传路径并依赖上游校验。
|
||||
- Alpine APKINDEX 签名校验需求 → 方案为透传签名文件,保持客户端校验。
|
||||
|
||||
### Research Output
|
||||
- `/home/rogee/Projects/any-hub/specs/007-apt-apk-cache/research.md`
|
||||
|
||||
## Phase 1: Design & Contracts
|
||||
|
||||
### Artifacts
|
||||
- `/home/rogee/Projects/any-hub/specs/007-apt-apk-cache/data-model.md`
|
||||
- `/home/rogee/Projects/any-hub/specs/007-apt-apk-cache/contracts/proxy-paths.md`
|
||||
- `/home/rogee/Projects/any-hub/specs/007-apt-apk-cache/quickstart.md`
|
||||
- Agent context updated via `.specify/scripts/bash/update-agent-context.sh codex`
|
||||
|
||||
### Design Focus
|
||||
- 定义 APT/APK 路径匹配与缓存策略(索引再验证、包体直接缓存)。
|
||||
- Acquire-By-Hash/签名文件透传,避免破坏校验。
|
||||
- 示例配置与测试入口(httptest 上游 + 临时缓存目录)。
|
||||
|
||||
## Phase 2: Task Breakdown (deferred to /speckit.tasks)
|
||||
|
||||
- 基于用户故事和设计生成 tasks.md(后续命令)
|
||||
42
specs/007-apt-apk-cache/quickstart.md
Normal file
42
specs/007-apt-apk-cache/quickstart.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Quickstart: APT/APK 缓存代理
|
||||
|
||||
## 1) 启用新 Hub
|
||||
|
||||
在 `config.toml` 增加示例:
|
||||
|
||||
```toml
|
||||
[[Hub]]
|
||||
Domain = "apt.hub.local"
|
||||
Name = "apt"
|
||||
Port = 5001
|
||||
Upstream = "https://mirrors.edge.kernel.org/ubuntu"
|
||||
Type = "debian"
|
||||
Module = "debian" # 待实现模块键
|
||||
|
||||
[[Hub]]
|
||||
Domain = "apk.hub.local"
|
||||
Name = "apk"
|
||||
Port = 5002
|
||||
Upstream = "https://dl-cdn.alpinelinux.org/alpine"
|
||||
Type = "apk"
|
||||
Module = "apk" # 待实现模块键
|
||||
```
|
||||
|
||||
## 2) 指向代理
|
||||
|
||||
- APT:将 `/etc/apt/sources.list` 中的 `http://apt.hub.local:5001` 替换官方源域名(需匹配 suite/component 路径)。
|
||||
- APK:在 `/etc/apk/repositories` 中写入 `http://apk.hub.local:5002/v3.19/main` 等路径。
|
||||
|
||||
## 3) 验证
|
||||
|
||||
```bash
|
||||
# APT
|
||||
apt-get update
|
||||
apt-get install -y curl
|
||||
|
||||
# Alpine
|
||||
apk update
|
||||
apk add curl
|
||||
```
|
||||
|
||||
观察 `logs/` 输出:首次请求应为回源,二次请求命中缓存(索引可能返回 304)。如上游不可达且缓存已有包体,应继续命中缓存;无缓存则透传错误。
|
||||
18
specs/007-apt-apk-cache/research.md
Normal file
18
specs/007-apt-apk-cache/research.md
Normal file
@@ -0,0 +1,18 @@
|
||||
# Research Notes: APT/APK 包缓存模块
|
||||
|
||||
## Decision Records
|
||||
|
||||
### 1) Acquire-By-Hash 处理
|
||||
- **Decision**: 路径原样透传,缓存按完整路径存储;不做额外本地哈希校验,交由上游与客户端校验。
|
||||
- **Rationale**: APT 自带哈希校验,路径即校验信息;本地重复计算增加 CPU/IO 成本且风险与上游标准重复。
|
||||
- **Alternatives**: 额外本地哈希校验并拒绝不匹配(增加复杂度、可能与上游行为不一致);跳过缓存(失去加速价值)。
|
||||
|
||||
### 2) APT 索引再验证策略
|
||||
- **Decision**: Release/InRelease/Packages* 请求统一带 If-None-Match/If-Modified-Since,缓存 RequireRevalidate=true;命中 304 继续用缓存,200 刷新。
|
||||
- **Rationale**: 与现有代理模式一致,确保“latest” 索引及时更新,避免 stale。
|
||||
- **Alternatives**: 固定 TTL 不再验证(风险:索引过期);强制每次全量 GET(浪费带宽)。
|
||||
|
||||
### 3) APKINDEX 签名处理
|
||||
- **Decision**: APKINDEX 及其签名文件原样透传并缓存,索引 RequireRevalidate=true,包体直接缓存。
|
||||
- **Rationale**: Alpine 客户端依靠签名文件校验,代理不应修改或剥离;再验证确保索引更新。
|
||||
- **Alternatives**: 不缓存 APKINDEX(失去加速效果);仅缓存包体(无法验证包版本更新)。
|
||||
88
specs/007-apt-apk-cache/spec.md
Normal file
88
specs/007-apt-apk-cache/spec.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# Feature Specification: APT/APK 包缓存模块
|
||||
|
||||
**Feature Branch**: `007-apt-apk-cache`
|
||||
**Created**: 2025-11-17
|
||||
**Status**: Draft
|
||||
**Input**: User description: "Title: 设计并实现 APT/APK 包缓存模块以加速 Docker 构建 Goal: 为 any-hub 新增一个 debian/apk 模块,代理本地 Docker 构建中的 apt-get/apk add 请求,支持缓存索引与包文件并透明回源。 Constraints: - 适配 Debian/Ubuntu APT 路径(Release/InRelease、Packages(.gz|.xz)、pool/*.deb)及 Alpine APKINDEX/packages。 - 索引/Release 需 RequireRevalidate=true;二进制包 AllowStore=true;签名文件透传。 - 兼容 Acquire-By-Hash(APT)与 APKINDEX 签名校验,不破坏客户端校验。 - 复用现有缓存架构(hooks.go + module.go + CacheStrategyProfile),不得影响现有 docker/npm/pypi/go/composer 模块。 Inputs: - 代码库路径:/home/rogee/Projects/any-hub - 参考模块:internal/hubmodule/docker/hooks.go (cachePolicy)、internal/hubmodule/golang/hooks.go、internal/proxy/handler.go (ETag/Last-Modified 再验证) Deliverables: - 新增 internal/hubmodule/debian(或 alpine)下的 hooks.go、module.go 实现 - 配套测试(模拟 apt-get update/install 或 apk update/add 的最小集) - 使用说明(简要配置示例,如何在 config.toml 添加 Hub) AcceptanceCriteria: - “apt-get update” 或 “apk update” 指向代理时,首次 miss 回源,二次命中缓存;索引文件可被最新上游版本触发再验证;二进制包稳定命中缓存。 新分支开头数字是 007"
|
||||
|
||||
## User Scenarios & Testing *(mandatory)*
|
||||
|
||||
### User Story 1 - APT 更新通过代理 (Priority: P1)
|
||||
|
||||
本地构建镜像的开发者将 apt 源指向代理,运行 `apt-get update` 获取最新索引并希望后续重复构建无需再次访问官方仓。
|
||||
|
||||
**Why this priority**: APT 更新是后续安装的前提,需保证首次拉取成功且缓存生效以降低构建时间与外网依赖。
|
||||
|
||||
**Independent Test**: 将 apt 源指向代理后执行两次 `apt-get update`,验证首轮回源、次轮命中缓存且返回内容与官方一致。
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** 缓存为空,**When** 运行 `apt-get update` 指向代理,**Then** 返回官方索引内容且缓存写入成功(Release/InRelease/Packages)。
|
||||
2. **Given** 同一索引已缓存,**When** 再次运行 `apt-get update`,**Then** 返回 200/304 且无额外上游 GET(仅必要的条件请求),输出内容与首轮一致。
|
||||
|
||||
---
|
||||
|
||||
### User Story 2 - APT 安装包命中缓存 (Priority: P2)
|
||||
|
||||
在构建步骤中安装常用 `.deb` 包,希望重复构建时直接从本地缓存提供 pool 下的包体。
|
||||
|
||||
**Why this priority**: 包文件体积大、下载耗时,命中缓存可显著缩短镜像构建时长。
|
||||
|
||||
**Independent Test**: 在完成 User Story 1 后,执行 `apt-get install <常见包>` 两次,确认首轮回源、次轮包体命中缓存,且校验/签名不受影响。
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** 缓存已有对应索引且无包体,**When** 首次安装某包,**Then** 包体成功下载并写入缓存,安装过程不因代理失败。
|
||||
2. **Given** 同一包体已缓存,**When** 再次安装,**Then** 不触发上游包体下载,安装成功且校验通过。
|
||||
|
||||
---
|
||||
|
||||
### User Story 3 - Alpine APK 加速 (Priority: P3)
|
||||
|
||||
将 Alpine 利用代理执行 `apk update && apk add`,希望索引与包体同样缓存并可重复命中。
|
||||
|
||||
**Why this priority**: Alpine 镜像在容器场景常用,通过同一代理获得一致的构建加速。
|
||||
|
||||
**Independent Test**: 将 `/etc/apk/repositories` 指向代理,执行两轮 `apk update && apk add <常用包>`,验证首轮回源、次轮命中缓存且 APKINDEX/包体校验正常。
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** 代理配置完毕,**When** 首次执行 `apk update`,**Then** 获取 APKINDEX 并写入缓存,随后 `apk add` 成功下载包体。
|
||||
2. **Given** 同一索引与包体已缓存,**When** 再次执行 `apk update && apk add`,**Then** 索引再验证后命中,包体直接命中缓存且安装成功。
|
||||
|
||||
---
|
||||
|
||||
### Edge Cases
|
||||
|
||||
- 上游索引更新:Release/InRelease 或 APKINDEX 发生变更时,代理应通过条件请求检测到并刷新缓存。
|
||||
- 校验失败:Acquire-By-Hash 校验或 APKINDEX 签名不匹配时,应返回上游原始错误而非吞掉错误。
|
||||
- 离线/超时:上游暂不可达时,若有缓存且策略允许,应优先返回缓存;无缓存则向客户端返回明确错误。
|
||||
- 部分源未配置:多源场景下,某源缺失或路径拼写错误时,代理应返回与上游一致的 404/403。
|
||||
- 大文件/磁盘不足:缓存写入包体时磁盘不足,应优雅失败并告警,不影响后续直接透传回源。
|
||||
|
||||
## Requirements *(mandatory)*
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
- **FR-001**: 代理必须支持 APT 获取与缓存 Release/InRelease、Packages(.gz/.xz) 及 pool 下的包体;索引请求需携带条件请求并可根据上游变更刷新缓存。
|
||||
- **FR-002**: 代理必须支持 Acquire-By-Hash 路径,若哈希不匹配需返回上游一致的错误,不得用缓存污染返回。
|
||||
- **FR-003**: 代理必须支持 Alpine APKINDEX 与 packages 路径的获取与缓存,索引需再验证,包体可直接命中缓存。
|
||||
- **FR-004**: 首次命中失败的索引/包体要回源写入缓存;后续同一路径在缓存有效期内应直接命中,除非索引再验证判定有新版本。
|
||||
- **FR-005**: 代理行为须保持透明:返回状态码、头信息、签名/校验文件与上游一致,日志需记录命中/回源/失败原因,便于构建诊断。
|
||||
- **FR-006**: 新增 Hub 配置字段(如仓类型、上游地址)需有默认值与示例,且不得影响现有 docker/npm/pypi/go/composer 的配置与行为。
|
||||
|
||||
### Key Entities *(include if feature involves data)*
|
||||
|
||||
- **索引文件**:APT 的 Release/InRelease/Packages*,Alpine 的 APKINDEX;含校验或签名信息,需可再验证。
|
||||
- **包体文件**:`pool/*/*.deb`、Alpine `packages/*`;内容不可变,按路径/哈希缓存。
|
||||
- **签名/校验文件**:APT 的 Release.gpg、Acquire-By-Hash 路径;用于客户端校验,需原样透传。
|
||||
- **缓存条目**:存储索引或包体的本地副本,包含写入时间与可选校验标识,用于命中与再验证判定。
|
||||
|
||||
## Success Criteria *(mandatory)*
|
||||
|
||||
### Measurable Outcomes
|
||||
|
||||
- **SC-001**: 在同一构建机器上,第二次执行 `apt-get update` 或 `apk update` 对同一源时,上游请求数较首次减少 ≥90%(仅保留必要条件请求),且耗时降低至少 50%。
|
||||
- **SC-002**: 已缓存的包体再次安装时,不触发上游下载,安装成功率达到 100%(以连续 20 次同包安装为样本)。
|
||||
- **SC-003**: 当上游索引有新版本发布时,下一次索引请求能够在一次交互内获取最新数据(无旧数据残留),并保持客户端校验/签名通过率 100%。
|
||||
- **SC-004**: 引入新模块后,现有 docker/npm/pypi/go/composer 代理的功能及配置使用无变化,回归用例全部通过。
|
||||
119
specs/007-apt-apk-cache/tasks.md
Normal file
119
specs/007-apt-apk-cache/tasks.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# Tasks: APT/APK 包缓存模块
|
||||
|
||||
**Input**: Design documents from `/specs/007-apt-apk-cache/`
|
||||
**Prerequisites**: plan.md (required), spec.md (user stories), research.md, data-model.md, contracts/
|
||||
|
||||
## Format: `[ID] [P?] [Story] Description`
|
||||
|
||||
- **[P]**: Can run in parallel (different files, no dependencies)
|
||||
- **[Story]**: User story label (US1, US2, US3)
|
||||
- Include exact file paths in descriptions
|
||||
|
||||
## Phase 1: Setup (Shared Infrastructure)
|
||||
|
||||
**Purpose**: Prepare module folders and examples for new hubs.
|
||||
|
||||
- [ ] T001 Create module directories `internal/hubmodule/debian/` and `internal/hubmodule/apk/` with placeholder go files (module.go/hooks.go scaffolds).
|
||||
- [ ] T002 Add sample hub entries for APT/APK in `configs/config.example.toml`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Foundational (Blocking Prerequisites)
|
||||
|
||||
**Purpose**: Core wiring for new hub types before story work.
|
||||
|
||||
- [ ] T003 Update hub type validation to accept `debian` 和 `apk` in `internal/config/validation.go`.
|
||||
- [ ] T004 Register new modules in `internal/config/modules.go` and `internal/hubmodule/registry.go` (init side-effect includes debian/apk).
|
||||
- [ ] T005 [P] Define debian module metadata (cache strategy, TTL, validation mode) in `internal/hubmodule/debian/module.go`.
|
||||
- [ ] T006 [P] Define apk module metadata in `internal/hubmodule/apk/module.go`.
|
||||
- [ ] T007 Ensure path locator rewrite strategy (raw_path) reused for new modules in `internal/hubmodule/strategy.go` or module options if needed.
|
||||
- [ ] T008 Add constitution-mandated Chinese comments for new module metadata files.
|
||||
|
||||
**Checkpoint**: Foundation ready—new hub types recognized, modules load without runtime errors.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: User Story 1 - APT 更新通过代理 (Priority: P1) 🎯 MVP
|
||||
|
||||
**Goal**: 代理 APT 索引(Release/InRelease/Packages*),首次回源,后续带条件请求再验证并命中缓存。
|
||||
|
||||
**Independent Test**: `apt-get update` 两次指向代理,首轮回源,次轮 304/命中缓存且内容与官方一致。
|
||||
|
||||
### Tests for User Story 1
|
||||
|
||||
- [ ] T009 [P] [US1] Add unit tests for path classification与缓存策略(索引 RequireRevalidate)在 `internal/hubmodule/debian/hooks_test.go`.
|
||||
- [ ] T010 [US1] Add integration test `tests/integration/apt_update_proxy_test.go` covering first/second `apt-get update` (Release/InRelease/Packages) with httptest upstream and temp storage.
|
||||
|
||||
### Implementation for User Story 1
|
||||
|
||||
- [ ] T011 [P] [US1] Implement APT hooks (NormalizePath/CachePolicy/ContentType/ResolveUpstream if needed) for index paths in `internal/hubmodule/debian/hooks.go`.
|
||||
- [ ] T012 [P] [US1] Support conditional requests (ETag/Last-Modified passthrough) for index responses in `internal/hubmodule/debian/hooks.go`.
|
||||
- [ ] T013 [US1] Wire debian module registration to use hooks in `internal/hubmodule/debian/module.go` and ensure hook registration in `hooks.go`.
|
||||
- [ ] T014 [US1] Ensure logging fields include cache hit/upstream for APT requests (reuse proxy logging) and document in comments `internal/hubmodule/debian/hooks.go`.
|
||||
- [ ] T015 [US1] Update quickstart instructions with APT usage validation steps in `specs/007-apt-apk-cache/quickstart.md`.
|
||||
|
||||
**Checkpoint**: APT 索引更新可独立验证并缓存。
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: User Story 2 - APT 安装包命中缓存 (Priority: P2)
|
||||
|
||||
**Goal**: pool 下 `.deb` 包首次回源、后续直接命中缓存;保持 Acquire-By-Hash 路径透传且不污染哈希校验。
|
||||
|
||||
**Independent Test**: `apt-get install <包>` 两次,首轮下载并缓存,次轮无上游下载,安装成功且校验通过。
|
||||
|
||||
### Tests for User Story 2
|
||||
|
||||
- [ ] T016 [P] [US2] Extend debian hook unit tests to cover `/pool/...` 与 `/by-hash/...` 缓存策略 in `internal/hubmodule/debian/hooks_test.go`.
|
||||
- [ ] T017 [US2] Integration test for package download caching and Acquire-By-Hash passthrough in `tests/integration/apt_package_proxy_test.go`.
|
||||
|
||||
### Implementation for User Story 2
|
||||
|
||||
- [ ] T018 [P] [US2] Implement package/dist path handling (AllowCache/AllowStore, RequireRevalidate=false) in `internal/hubmodule/debian/hooks.go`.
|
||||
- [ ] T019 [P] [US2] Handle `/dists/<suite>/by-hash/<algo>/<hash>` as immutable cached resources in `internal/hubmodule/debian/hooks.go`.
|
||||
- [ ] T020 [US2] Validate cache writer/reader streaming for large deb files in `internal/proxy/handler.go` (ensure no full-buffer reads) with comments/tests if changes required.
|
||||
- [ ] T021 [US2] Update config docs/examples if additional APT-specific knobs are added in `configs/config.example.toml` or `README.md`.
|
||||
|
||||
**Checkpoint**: APT 包体可命中缓存且哈希/签名校验保持一致。
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: User Story 3 - Alpine APK 加速 (Priority: P3)
|
||||
|
||||
**Goal**: 缓存 APKINDEX 并再验证;包体(packages/*.apk)首次回源后直接命中缓存。
|
||||
|
||||
**Independent Test**: `apk update && apk add <包>` 两次,索引次轮 304/命中,包体次轮直接命中,安装成功。
|
||||
|
||||
### Tests for User Story 3
|
||||
|
||||
- [ ] T022 [P] [US3] Add apk hook unit tests for index/package path policy in `internal/hubmodule/apk/hooks_test.go`.
|
||||
- [ ] T023 [US3] Integration test for apk update/install caching in `tests/integration/apk_proxy_test.go`.
|
||||
|
||||
### Implementation for User Story 3
|
||||
|
||||
- [ ] T024 [P] [US3] Implement APK hooks (CachePolicy/ContentType/NormalizePath) for APKINDEX and packages in `internal/hubmodule/apk/hooks.go`.
|
||||
- [ ] T025 [P] [US3] Ensure APKINDEX/signature files RequireRevalidate and package files immutable cache in `internal/hubmodule/apk/hooks.go`.
|
||||
- [ ] T026 [US3] Register apk hooks in module init and update logging/observability comments in `internal/hubmodule/apk/module.go`.
|
||||
- [ ] T027 [US3] Add Alpine repository usage notes to `specs/007-apt-apk-cache/quickstart.md`.
|
||||
|
||||
**Checkpoint**: Alpine 索引与包体缓存可独立验证。
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Polish & Cross-Cutting Concerns
|
||||
|
||||
- [ ] T028 [P] Add Chinese comments for key caching logic and path handling in new hook files (`internal/hubmodule/debian/hooks.go`, `internal/hubmodule/apk/hooks.go`).
|
||||
- [ ] T029 [P] Document log fields and cache semantics in `docs/` or `README.md` (structure log examples for APT/APK).
|
||||
- [ ] T030 Validate gofmt/go test ./... and update `specs/007-apt-apk-cache/quickstart.md` with final verification steps.
|
||||
- [ ] T031 [P] Confirm no regressions to existing modules via smoke test list in `tests/integration/` (reuse existing suites, adjust configs if needed).
|
||||
|
||||
---
|
||||
|
||||
## Dependencies & Execution Order
|
||||
|
||||
- Phase 1 → Phase 2 → User stories (Phase 3/4/5) → Phase 6.
|
||||
- User Story 1 (P1) must complete before US2 (shares debian hooks); US3 can start after Phase 2 independently.
|
||||
- Parallel opportunities:
|
||||
- T005/T006 module metadata in parallel; hook/unit work can run in parallel within each story where marked [P].
|
||||
- US3 tasks can run in parallel with US1 late-stage tasks once foundational ready (different modules/files).
|
||||
- Suggested MVP: Complete Phases 1-3 (US1) to deliver APT 更新加速;US2/US3 incrementally after validation.
|
||||
Reference in New Issue
Block a user