rogee/any-hub
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: support apk

Browse Source
This commit is contained in:
Rogee
2025-11-18 12:16:28 +08:00
parent 68b6bb78e6
commit ba5544c28d
28 changed files with 1412 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
specs/007-apt-apk-cache/quickstart.md
View File

@@ -8,7 +8,6 @@
[[Hub]]
Domain = "apt.hub.local"
Name = "apt"
Port = 5001
Upstream = "https://mirrors.edge.kernel.org/ubuntu"
Type = "debian"
Module = "debian" # 待实现模块键
@@ -16,7 +15,6 @@ Module = "debian" # 待实现模块键
[[Hub]]
Domain = "apk.hub.local"
Name = "apk"
Port = 5002
Upstream = "https://dl-cdn.alpinelinux.org/alpine"
Type = "apk"
Module = "apk" # 待实现模块键
@@ -24,8 +22,8 @@ Module = "apk" # 待实现模块键
## 2) 指向代理
- APT：将 `/etc/apt/sources.list` 中的 `http://apt.hub.local:5001` 替换官方源域名（需匹配 suite/component 路径）。
- APK：在 `/etc/apk/repositories` 中写入 `http://apk.hub.local:5002/v3.19/main` 等路径。
- APT：将 `/etc/apt/sources.list` 中的官方域名替换为 `http://apt.hub.local:5000`（或全局 ListenPort），保持原 suite/component 路径。
- APK：在 `/etc/apk/repositories` 中写入 `http://apk.hub.local:5000/v3.19/main` 等路径（与全局 ListenPort 一致）。
## 3) 验证
@@ -33,10 +31,12 @@ Module = "apk" # 待实现模块键
# APT
apt-get update
apt-get install -y curl
apt-get install -y curl # 第二次安装应直接命中缓存（pool/* 与 by-hash/* 不再 HEAD 再验证）
# Alpine
apk update
apk add curl
apk add curl # 第二次安装应直接命中缓存（APKINDEX 再验证，packages/*.apk 直接命中）
```
观察 `logs/` 输出：首次请求应为回源，二次请求命中缓存（索引可能返回 304）。如上游不可达且缓存已有包体，应继续命中缓存；无缓存则透传错误。

62
specs/007-apt-apk-cache/tasks.md
View File

@@ -13,8 +13,8 @@
**Purpose**: Prepare module folders and examples for new hubs.
- [ ] T001 Create module directories `internal/hubmodule/debian/` and `internal/hubmodule/apk/` with placeholder go files (module.go/hooks.go scaffolds).
- [ ] T002 Add sample hub entries for APT/APK in `configs/config.example.toml`.
- [X] T001 Create module directories `internal/hubmodule/debian/` and `internal/hubmodule/apk/` with placeholder go files (module.go/hooks.go scaffolds).
- [X] T002 Add sample hub entries for APT/APK in `configs/config.example.toml`.
---
@@ -22,12 +22,12 @@
**Purpose**: Core wiring for new hub types before story work.
- [ ] T003 Update hub type validation to accept `debian` 和 `apk` in `internal/config/validation.go`.
- [ ] T004 Register new modules in `internal/config/modules.go` and `internal/hubmodule/registry.go` (init side-effect includes debian/apk).
- [ ] T005 [P] Define debian module metadata (cache strategy, TTL, validation mode) in `internal/hubmodule/debian/module.go`.
- [ ] T006 [P] Define apk module metadata in `internal/hubmodule/apk/module.go`.
- [ ] T007 Ensure path locator rewrite strategy (raw_path) reused for new modules in `internal/hubmodule/strategy.go` or module options if needed.
- [ ] T008 Add constitution-mandated Chinese comments for new module metadata files.
- [X] T003 Update hub type validation to accept `debian` 和 `apk` in `internal/config/validation.go`.
- [X] T004 Register new modules in `internal/config/modules.go` and `internal/hubmodule/registry.go` (init side-effect includes debian/apk).
- [X] T005 [P] Define debian module metadata (cache strategy, TTL, validation mode) in `internal/hubmodule/debian/module.go`.
- [X] T006 [P] Define apk module metadata in `internal/hubmodule/apk/module.go`.
- [X] T007 Ensure path locator rewrite strategy (raw_path) reused for new modules in `internal/hubmodule/strategy.go` or module options if needed.
- [X] T008 Add constitution-mandated Chinese comments for new module metadata files.
**Checkpoint**: Foundation ready—new hub types recognized, modules load without runtime errors.
@@ -41,16 +41,16 @@
### Tests for User Story 1
- [ ] T009 [P] [US1] Add unit tests for path classification与缓存策略（索引 RequireRevalidate）在 `internal/hubmodule/debian/hooks_test.go`.
- [ ] T010 [US1] Add integration test `tests/integration/apt_update_proxy_test.go` covering first/second `apt-get update` (Release/InRelease/Packages) with httptest upstream and temp storage.
- [X] T009 [P] [US1] Add unit tests for path classification与缓存策略（索引 RequireRevalidate）在 `internal/hubmodule/debian/hooks_test.go`.
- [X] T010 [US1] Add integration test `tests/integration/apt_update_proxy_test.go` covering first/second `apt-get update` (Release/InRelease/Packages) with httptest upstream and temp storage.
### Implementation for User Story 1
- [ ] T011 [P] [US1] Implement APT hooks (NormalizePath/CachePolicy/ContentType/ResolveUpstream if needed) for index paths in `internal/hubmodule/debian/hooks.go`.
- [ ] T012 [P] [US1] Support conditional requests (ETag/Last-Modified passthrough) for index responses in `internal/hubmodule/debian/hooks.go`.
- [ ] T013 [US1] Wire debian module registration to use hooks in `internal/hubmodule/debian/module.go` and ensure hook registration in `hooks.go`.
- [ ] T014 [US1] Ensure logging fields include cache hit/upstream for APT requests (reuse proxy logging) and document in comments `internal/hubmodule/debian/hooks.go`.
- [ ] T015 [US1] Update quickstart instructions with APT usage validation steps in `specs/007-apt-apk-cache/quickstart.md`.
- [X] T011 [P] [US1] Implement APT hooks (NormalizePath/CachePolicy/ContentType/ResolveUpstream if needed) for index paths in `internal/hubmodule/debian/hooks.go`.
- [X] T012 [P] [US1] Support conditional requests (ETag/Last-Modified passthrough) for index responses in `internal/hubmodule/debian/hooks.go`.
- [X] T013 [US1] Wire debian module registration to use hooks in `internal/hubmodule/debian/module.go` and ensure hook registration in `hooks.go`.
- [X] T014 [US1] Ensure logging fields include cache hit/upstream for APT requests (reuse proxy logging) and document in comments `internal/hubmodule/debian/hooks.go`.
- [X] T015 [US1] Update quickstart instructions with APT usage validation steps in `specs/007-apt-apk-cache/quickstart.md`.
**Checkpoint**: APT 索引更新可独立验证并缓存。
@@ -64,15 +64,15 @@
### Tests for User Story 2
- [ ] T016 [P] [US2] Extend debian hook unit tests to cover `/pool/...` 与 `/by-hash/...` 缓存策略 in `internal/hubmodule/debian/hooks_test.go`.
- [ ] T017 [US2] Integration test for package download caching and Acquire-By-Hash passthrough in `tests/integration/apt_package_proxy_test.go`.
- [X] T016 [P] [US2] Extend debian hook unit tests to cover `/pool/...` 与 `/by-hash/...` 缓存策略 in `internal/hubmodule/debian/hooks_test.go`.
- [X] T017 [US2] Integration test for package download caching and Acquire-By-Hash passthrough in `tests/integration/apt_package_proxy_test.go`.
### Implementation for User Story 2
- [ ] T018 [P] [US2] Implement package/dist path handling (AllowCache/AllowStore, RequireRevalidate=false) in `internal/hubmodule/debian/hooks.go`.
- [ ] T019 [P] [US2] Handle `/dists/<suite>/by-hash/<algo>/<hash>` as immutable cached resources in `internal/hubmodule/debian/hooks.go`.
- [ ] T020 [US2] Validate cache writer/reader streaming for large deb files in `internal/proxy/handler.go` (ensure no full-buffer reads) with comments/tests if changes required.
- [ ] T021 [US2] Update config docs/examples if additional APT-specific knobs are added in `configs/config.example.toml` or `README.md`.
- [X] T018 [P] [US2] Implement package/dist path handling (AllowCache/AllowStore, RequireRevalidate=false) in `internal/hubmodule/debian/hooks.go`.
- [X] T019 [P] [US2] Handle `/dists/<suite>/by-hash/<algo>/<hash>` as immutable cached resources in `internal/hubmodule/debian/hooks.go`.
- [X] T020 [US2] Validate cache writer/reader streaming for large deb files in `internal/proxy/handler.go` (ensure no full-buffer reads) with comments/tests if changes required.
- [X] T021 [US2] Update config docs/examples if additional APT-specific knobs are added in `configs/config.example.toml` or `README.md`.
**Checkpoint**: APT 包体可命中缓存且哈希/签名校验保持一致。
@@ -86,15 +86,15 @@
### Tests for User Story 3
- [ ] T022 [P] [US3] Add apk hook unit tests for index/package path policy in `internal/hubmodule/apk/hooks_test.go`.
- [ ] T023 [US3] Integration test for apk update/install caching in `tests/integration/apk_proxy_test.go`.
- [X] T022 [P] [US3] Add apk hook unit tests for index/package path policy in `internal/hubmodule/apk/hooks_test.go`.
- [X] T023 [US3] Integration test for apk update/install caching in `tests/integration/apk_proxy_test.go`.
### Implementation for User Story 3
- [ ] T024 [P] [US3] Implement APK hooks (CachePolicy/ContentType/NormalizePath) for APKINDEX and packages in `internal/hubmodule/apk/hooks.go`.
- [ ] T025 [P] [US3] Ensure APKINDEX/signature files RequireRevalidate and package files immutable cache in `internal/hubmodule/apk/hooks.go`.
- [ ] T026 [US3] Register apk hooks in module init and update logging/observability comments in `internal/hubmodule/apk/module.go`.
- [ ] T027 [US3] Add Alpine repository usage notes to `specs/007-apt-apk-cache/quickstart.md`.
- [X] T024 [P] [US3] Implement APK hooks (CachePolicy/ContentType/NormalizePath) for APKINDEX and packages in `internal/hubmodule/apk/hooks.go`.
- [X] T025 [P] [US3] Ensure APKINDEX/signature files RequireRevalidate and package files immutable cache in `internal/hubmodule/apk/hooks.go`.
- [X] T026 [US3] Register apk hooks in module init and update logging/observability comments in `internal/hubmodule/apk/module.go`.
- [X] T027 [US3] Add Alpine repository usage notes to `specs/007-apt-apk-cache/quickstart.md`.
**Checkpoint**: Alpine 索引与包体缓存可独立验证。
@@ -102,10 +102,10 @@
## Phase 6: Polish & Cross-Cutting Concerns
- [ ] T028 [P] Add Chinese comments for key caching logic and path handling in new hook files (`internal/hubmodule/debian/hooks.go`, `internal/hubmodule/apk/hooks.go`).
- [ ] T029 [P] Document log fields and cache semantics in `docs/` or `README.md` (structure log examples for APT/APK).
- [ ] T030 Validate gofmt/go test ./... and update `specs/007-apt-apk-cache/quickstart.md` with final verification steps.
- [ ] T031 [P] Confirm no regressions to existing modules via smoke test list in `tests/integration/` (reuse existing suites, adjust configs if needed).
- [X] T028 [P] Add Chinese comments for key caching logic and path handling in new hook files (`internal/hubmodule/debian/hooks.go`, `internal/hubmodule/apk/hooks.go`).
- [X] T029 [P] Document log fields and cache semantics in `docs/` or `README.md` (structure log examples for APT/APK).
- [X] T030 Validate gofmt/go test ./... and update `specs/007-apt-apk-cache/quickstart.md` with final verification steps.
- [X] T031 [P] Confirm no regressions to existing modules via smoke test list in `tests/integration/` (reuse existing suites, adjust configs if needed).
---