chore: stabilize lint and verify builds
This commit is contained in:
107
docs/plans/2026-02-04-p3-17.md
Normal file
107
docs/plans/2026-02-04-p3-17.md
Normal file
@@ -0,0 +1,107 @@
|
||||
# Implementation Plan: p3-17-media-processing-s3
|
||||
|
||||
**Branch**: `[p3-17-media-processing-s3]` | **Date**: 2026-02-04 | **Spec**: `docs/todo_list.md#17`
|
||||
**Input**: P3-17 “媒体处理管线适配对象存储（S3/MinIO）” + user request to extract a cover from `fixtures/demo.mp4` via ffmpeg.
|
||||
|
||||
## Summary
|
||||
|
||||
Adapt the media processing worker to support S3/MinIO storage by downloading source media to a temp directory, running ffmpeg to generate a cover image, uploading derived assets via the storage provider, and preserving the existing local filesystem path. Include fixture-based verification using `fixtures/demo.mp4`.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Go 1.x (project standard)
|
||||
**Primary Dependencies**: River (jobs), MinIO SDK (S3 compatible), Fiber
|
||||
**Storage**: PostgreSQL + local filesystem / S3-compatible storage
|
||||
**Testing**: `go test` (service/job tests), ffmpeg CLI for fixture validation
|
||||
**Target Platform**: Linux server
|
||||
**Project Type**: Web application (backend + frontend, backend-only changes)
|
||||
**Performance Goals**: N/A (processing path only)
|
||||
**Constraints**: Preserve local storage behavior; do not edit generated files; follow `backend/llm.txt`; ensure temp files are cleaned
|
||||
**Scale/Scope**: Media processing worker + storage provider only
|
||||
|
||||
## Constitution Check
|
||||
|
||||
- Follow `backend/llm.txt` conventions and Chinese comments for business logic.
|
||||
- Do not edit generated files (`*.gen.go`, `backend/docs/docs.go`).
|
||||
- Keep controller thin; changes limited to jobs/services/providers.
|
||||
- Preserve local provider behavior; add S3 path without regression.
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
docs/
|
||||
└── plan.md # This plan
|
||||
```
|
||||
|
||||
### Source Code (repository root)
|
||||
|
||||
```text
|
||||
backend/
|
||||
├── app/
|
||||
│ ├── jobs/
|
||||
│ │ ├── media_process_job.go
|
||||
│ │ └── args/media_asset_process.go
|
||||
│ └── services/
|
||||
│ └── common.go
|
||||
└── providers/
|
||||
└── storage/provider.go
|
||||
|
||||
fixtures/
|
||||
└── demo.mp4
|
||||
```
|
||||
|
||||
**Structure Decision**: Web application backend; scope limited to media processing worker and storage provider integration.
|
||||
|
||||
## Plan Phases
|
||||
|
||||
1. **Design & plumbing**: Define temp file conventions and storage download API for S3/local.
|
||||
2. **Implementation**: Add S3 processing flow to worker and cover asset registration; keep local path intact.
|
||||
3. **Verification**: Add tests (or integration checks) and run fixture-based ffmpeg validation.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. **Define storage download interface**
|
||||
- Add a storage provider helper to download an object to a local temp file (local: copy from `LocalPath/objectKey` without rename; S3: `FGetObject`).
|
||||
- Ensure helper never mutates/deletes the source object, creates parent dirs for destination, and overwrites the destination if it already exists.
|
||||
- Ensure API is used by jobs without leaking provider-specific logic.
|
||||
|
||||
2. **Adapt `MediaProcessWorker` for S3/MinIO**
|
||||
- For non-local providers, download the source object into a temp directory.
|
||||
- Run ffmpeg to extract a cover image from the temp file.
|
||||
- Upload the cover via `storage.PutObject` and register the derived media asset.
|
||||
- Ensure temp directories/files are cleaned on success or failure.
|
||||
|
||||
3. **Update cover asset registration**
|
||||
- For non-local providers, avoid filesystem rename; upload via storage provider and keep object key conventions.
|
||||
- Use `storage.PutObject(ctx, objectKey, coverTempPath, "image/jpeg")`, then cleanup temp files/dirs.
|
||||
- Keep local path move behavior unchanged.
|
||||
|
||||
4. **Add tests / verification hooks**
|
||||
- Add a job/service test for S3 path (gated by MinIO env if needed).
|
||||
- Keep/extend local path test to ensure no regression.
|
||||
- Validate `fixtures/demo.mp4` can produce a cover with ffmpeg.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Task 1 must complete before Task 2 (worker needs download API).
|
||||
- Task 2 must complete before Task 3 (cover registration requires new flow).
|
||||
- Task 4 depends on Task 2 & 3 (tests rely on updated pipeline).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- **Local path unchanged**: `MediaProcessWorker` still generates a cover image for local provider when ffmpeg is available.
|
||||
- **S3/MinIO path works**: For `Storage.Type = s3`, the worker downloads source media, generates cover via ffmpeg, uploads cover to bucket, and creates a derived `media_assets` record with correct object key.
|
||||
- **Fixture validation**: `ffmpeg -y -i fixtures/demo.mp4 -ss 00:00:00.000 -vframes 1 /tmp/demo_cover.jpg` succeeds locally and produces a non-empty file.
|
||||
- **S3/MinIO test config**: tests load Storage config with `Type=s3`, `Endpoint`, `AccessKey`, `SecretKey`, `Bucket`, `PathStyle` (true for MinIO).
|
||||
- **Automated checks** (to be added in this phase):
|
||||
- `go test ./backend/app/jobs -run TestMediaProcessWorkerS3 -count=1` passes (with S3/MinIO config loaded).
|
||||
- `go test ./backend/app/jobs -run TestMediaProcessWorkerLocal -count=1` passes.
|
||||
|
||||
## Risks
|
||||
|
||||
- **ffmpeg not installed**: Worker should detect and log; processing should fail gracefully.
|
||||
- **Temp storage pressure**: Large media downloads may exceed disk; ensure cleanup on all paths.
|
||||
- **S3 connectivity/transient errors**: Add retry or error propagation to avoid silent failures.
|
||||
- **Access policies**: Misconfigured bucket policies may prevent download/upload; surface clear errors.
|
||||
80
docs/plans/2026-02-05.md
Normal file
80
docs/plans/2026-02-05.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# Implementation Plan: comment-hook-fix
|
||||
|
||||
**Branch**: `[fix/comment-hook]` | **Date**: 2026-02-05 | **Spec**: `docs/todo_list.md#18`
|
||||
**Input**: Resolve the comment-hook failure introduced by the recharge code/superadmin DTO changes.
|
||||
|
||||
## Summary
|
||||
|
||||
Identify the comment-hook rule being violated in the DTO updates, adjust or remove offending comments in the affected backend DTOs, and verify the hook/lints pass without modifying generated files.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Go 1.x
|
||||
**Primary Dependencies**: Fiber, GORM-Gen
|
||||
**Storage**: PostgreSQL
|
||||
**Testing**: `go test ./...` (if required), LSP diagnostics for changed files
|
||||
**Target Platform**: Linux server
|
||||
**Project Type**: Web application (backend + frontend)
|
||||
**Performance Goals**: N/A
|
||||
**Constraints**: Follow `backend/llm.txt`; no generated file edits; respect comment-hook rules
|
||||
**Scale/Scope**: Backend DTO comments only
|
||||
|
||||
## Constitution Check
|
||||
|
||||
- Follow `backend/llm.txt` (thin controllers, services handle DB, Chinese comments for business logic).
|
||||
- Do not edit generated files (`*.gen.go`, `backend/docs/docs.go`).
|
||||
- Verify hook compliance before finishing.
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
docs/
|
||||
└── plan.md
|
||||
```
|
||||
|
||||
### Source Code (repository root)
|
||||
|
||||
```text
|
||||
backend/
|
||||
└── app/http/super/v1/dto/super.go
|
||||
```
|
||||
|
||||
**Structure Decision**: Web application; scope is limited to backend DTO comment fixes.
|
||||
|
||||
## Plan Phases
|
||||
|
||||
1. **Diagnose hook failure**: Inspect the comment-hook rule and locate offending comments in DTO files.
|
||||
2. **Apply fixes**: Adjust/remove comments to satisfy the hook while preserving API docs via struct tags if needed.
|
||||
3. **Verify**: Run gofmt on touched files, LSP diagnostics, and re-run hook/tests as required.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. Review the hook failure output and hook rules to identify the forbidden comment pattern.
|
||||
2. Update DTO comment blocks to comply with the rule (no generated-file edits).
|
||||
3. Run gofmt and LSP diagnostics; re-run the hook/tests if needed.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Task 1 must complete before Task 2 (need exact rule to apply correct fix).
|
||||
- Task 2 must complete before Task 3 (verification after edits).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Comment-hook passes without errors.
|
||||
- Updated DTO file(s) are gofmt’d and LSP diagnostics are clean.
|
||||
- No generated files are edited manually.
|
||||
|
||||
## Risks
|
||||
|
||||
- Removing comments could reduce Swagger clarity; mitigate by keeping struct tags or non-offending comment styles if required.
|
||||
- Hook may flag multiple files; ensure all offenders are addressed before verification.
|
||||
|
||||
## Complexity Tracking
|
||||
|
||||
> **Fill ONLY if Constitution Check has violations that must be justified**
|
||||
|
||||
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
||||
|-----------|------------|-------------------------------------|
|
||||
| N/A | N/A | N/A |
|
||||
Reference in New Issue
Block a user