commit d15b3c5b430adc7b277a9a0c7f675d6ae3e19116 Author: 杨豪 Date: Mon Sep 7 20:13:22 2026 +0800 feat: 拖动重叠验证码离线求解（方向感知倒角+alpha轮廓+旋转扫描） - src/: 最终交付物（solve.py base64 API、method_l_shape.py 核心算法、verify_result.py 验证工具） - docs/: 方案文档与实验演进记录 - try/: 历史实验脚本（A~K 方法） - 10/10 样本求解成功，3 个独立真值锚点偏差 <=4px diff --git a/.codegraph/.gitignore b/.codegraph/.gitignore new file mode 100644 index 0000000..d20c0fe --- /dev/null +++ b/.codegraph/.gitignore @@ -0,0 +1,5 @@ +# CodeGraph data files — local to each machine, not for committing. +# Ignore everything in .codegraph/ except this file itself, so transient +# files (the database, daemon.pid, sockets, logs) never show up in git. +* +!.gitignore diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..1b134b9 --- /dev/null +++ b/.gitignore @@ -0,0 +1,26 @@ +# Python +__pycache__/ +*.py[cod] +*.egg-info/ +.venv/ +venv/ + +# IDE +.idea/ +.vscode/ +*.swp + +# 样本图片（不提交原始验证码） +captchas/ +!captchas/.gitkeep + +# 实验产物 +try/out/ +try/visual/ +src/out/ +*.png +*.jpeg + +# 系统文件 +.DS_Store +Thumbs.db diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..f704c84 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,28 @@ +# 拖动重叠验证码自动化方案调研 + +## 任务目标 + +针对「拖动重叠」类网页验证码：给定验证码大图与 mark 小图，通过图像处理在大图中定位 mark 应当拖动重叠的目标区域，求出该区域的偏移像素（即拖动偏移量）。 + +需要对比多种可行的图像处理方案，找出该场景下求偏移最准确、最稳定的方法。 + +## 实现约束 + +- 实现语言：Python +- 离线识别：不依赖在线服务或网络请求，所有识别过程在本地完成 +- 实验隔离：`captchas/` 目录只读，不要在其中写入任何产物；功能尝试与验证统一在 `try/` 目录下进行 +- **最终产物**：经验证的最终交付物（求解 API、核心算法、验证工具及其输出）统一放在 `src/` 目录；`try/` 仅保留实验性脚本与中间产物 +- **知识沉淀**：方案结论、实验演进与验证方法沉淀在 `docs/`（solution.md / experiments.md / README.md） + +## 数据说明 + +样本位于 `captchas/` 目录，成对出现，文件名前缀相同： + +- 背景大图：`xxxx.jpeg`（552×344） +- mark 小图：`xxxx-mark.png`（110×110，PNG 格式） + +## 核心难点 + +大图中存在至少 2 个与 mark 图相似的区域——它们是 mark 经过缩放、旋转等变换生成的干扰项，把 mark 拖到这些位置无法正确重叠；只有与 mark 完全一致（无缩放、无旋转差异）的区域才是目标。 + +因此，简单的单尺度模板匹配容易被相似干扰项误导。如何在干扰项存在的前提下稳定、准确地定位目标区域偏移，是本次调研要解决的核心问题。 diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..d007c81 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,29 @@ +# docs + +项目文档索引。 + +## 文档列表 + +- [solution.md](solution.md) — 最终方案：算法原理、API 用法、结果验证方法、10 样本结果与已知边界 +- [experiments.md](experiments.md) — 实验演进记录（A→L）：各方法失败原因、转折点实验、调试坑、无真值交叉验证方法 + +## 快速开始 + +```bash +# 单样本求解（base64 或文件路径） +.venv/bin/python src/solve.py <大图> + +# 批量求解 + 可视化（输出 src/out/L/） +.venv/bin/python src/method_l_shape.py + +# 结果验证（叠加轮廓图，输出 src/out/L/verify/） +.venv/bin/python src/verify_result.py +``` + +## 相关目录 + +| 目录 | 用途 | +| --- | --- | +| `captchas/` | 原始样本（只读） | +| `src/` | 最终交付物：求解 API、核心算法、验证工具及其输出 | +| `try/` | 实验脚本与中间产物（历史方法 A~K） | diff --git a/docs/experiments.md b/docs/experiments.md new file mode 100644 index 0000000..1282f1a --- /dev/null +++ b/docs/experiments.md @@ -0,0 +1,85 @@ +# 实验演进记录（A → L） + +> 最终方案与使用说明见 [solution.md](solution.md)。 +> 本文记录每个方法的思路、失败原因与沉淀的经验教训， +> 原始实验脚本在 `try/`，输出在 `try/out/<方法字母>/`。 + +## 方法总览 + +| 方法 | 脚本 | 思路 | 结论 | +| --- | --- | --- | --- | +| A | method_a_template.py | 蒙版 CCORR 多尺度模板匹配 | 废弃 | +| A2 | method_a2_ccoeff.py | 蒙版 CCOEFF + 峰显著度 + NMS | 基线可用，但锁纹理 | +| B | method_b_sift.py | SIFT + RANSAC 迭代聚类 | 仅 2 样本可用，留作真值锚点 | +| C | method_c_rotation.py | NCC 粗扫 + 旋转响应二次评分 | 第一阶段候选即错 | +| D | method_d_shape.py | Chamfer 轮廓距离 | 小尺度纹理饱和 | +| E | method_e_shape_coverage.py | 轮廓边缘覆盖率 | 同上 | +| F | method_f_contour.py | Canny 轮廓 + matchShapes | 背景轮廓合并成超大块 | +| G | method_g_component.py | 固定暗度阈值连通域 | 阈值敏感 | +| H | method_h_silhouette.py | alpha 轮廓距离 + 局部对比度 | 分数归一化饱和 | +| I | method_i_adaptive_components.py | 自适应暗目标连通域 | 误检照片内容 | +| J | method_j_objects.py | 低饱和度暗色物体分割 | 同上 | +| K | method_k_objectness.py | 暗色覆盖物概率图 + 掩码 CCORR | 掩码 CCORR 边界产生 inf | +| **L** | `src/method_l_shape.py` | **方向感知倒角 + 洞内部特征 + 旋转扫描** | **最终方案，10/10** | + +## 逐方法教训 + +### A / A2：RGB 模板匹配（不减小均值的代价） + +- `TM_CCORR_NORMED` 不做均值中心化，亮背景上处处高分； +- 跨尺度直接比原始分数偏向小模板 → A2 改用峰显著度 `(max-median)/std`； +- A2 在 8/10 样本上候选位置合理，但 41ac 出现多峰分歧—— + 后来证明这些 0.7 的高分全是**小尺度纹理巧合**，方向本身是错的。 + +### B：SIFT（意外价值） + +- 平面图形缺少纹理，大部分样本匹配不足； +- 但在纹理丰富的 4db9、d7a7 上给出亚像素级真值 + （scale≈1.00、rot≈0°），成为后续所有方法的**交叉验证锚点**。 + +### C：旋转感知 NCC（两次评分救不了错误的候选） + +- 旋转响应曲线本身有效（真目标 rot≈0），但候选来自 NCC 粗扫， + 粗扫错了旋转阶段无法挽回。 + +### D/E：倒角与覆盖率（尺度公平性问题） + +- 覆盖率在小尺度上饱和到 1.0（背景纹理到处有边）； +- 倒角均值在密纹理区反而更低 → 必须限制尺度范围并加方向约束。 + +### F/G/H/I/J/K：分割路线的共同困境 + +- Canny 轮廓易把地面/山脉合并成超大块（F）； +- 固定/自适应暗度阈值对每张照片都不同（G/I/J）； +- K 的教训：`TM_CCORR_NORMED` 带 mask 在边界产生 inf 值， + 需要手写 filter2D 掩码 NCC（后来用于决定性实验）。 + +## 转折点实验（值得复用的诊断手段） + +1. **掩码 NCC 全图扫描**（手写 filter2D 实现，无 inf 问题）： + 对 41ac/444d 在尺度 0.9–1.1 全图扫描，最高分仅 0.56/0.46 + → 证明 mark RGB 与洞内容无像素关系，RGB 路线全部判死。 +2. **闭合轮廓 + matchShapes**（444d 上）： + Canny 闭合轮廓直接命中两个洞，Hu≈0.003，左洞 rot=-0.6°、右洞 -15.5° + → 证明"轮廓 + 尺度 + 旋转"路线成立，且给出 444d 独立真值。 +3. **三特征分离度测量**： + 洞：内部边缘密度 ≤0.1、亮度差 -40~-104； + 草地误检：密度 0.36、亮度差 +6 → 直接给出 L 的特征权重依据。 + +## 调试中踩过的具体坑 + +- `np.flatnonzero` 对 2D 数组返回一维扁平索引，取坐标应用 `np.argwhere`； +- `filter2D` 输出是全尺寸，与手写滑窗比对时要裁剪有效相关区 + `[h//2 : h//2+H-h+1, w//2 : w//2+W-w+1]`； +- 方向桶不能越界重叠：角度已在 [0,180)，9 桶直接线性划分即可， + 最后一桶再做 wrap 会导致点被双计（表现为均值超上限）； +- `np.arange` 有浮点累积误差，参与区间比较前先 `round(s, 3)`； +- Canny 阈值过低（如 26/77）会让 JPEG 噪声淹没"内部平坦"特征。 + +## 无 ground truth 时的交叉验证方法 + +1. 找至少一个可用独立方法的样本建立锚点（如 SIFT 能用的纹理丰富样本）； +2. 用结构性证据补充锚点：候选 bbox 尺寸与 mark alpha bbox 完全一致 + （41ac：88×106 vs 88×106）； +3. 全部候选可视化后人工目检：真目标应落在几何图形上而非纹理上； +4. 结果叠加轮廓回贴大图（`src/verify_result.py`），观察贴合精度。 diff --git a/docs/solution.md b/docs/solution.md new file mode 100644 index 0000000..001f6dc --- /dev/null +++ b/docs/solution.md @@ -0,0 +1,133 @@ +# 拖动重叠验证码：最终方案 + +> 历史实验过程与方法淘汰记录见 [experiments.md](experiments.md)； +> 实验原始日志在 `try/findings.md`。 + +## 任务定义 + +给定「拖动重叠」验证码的背景大图（552×344）与 mark 小图（110×110 RGBA）， +离线求出 mark 应拖动重叠的目标区域偏移。 + +背景中存在 ≥2 个与 mark 同形的区域（缩放/旋转生成的干扰项）， +只有**无缩放、无旋转**（尺度≈1.0、旋转≈0°）的实例是目标。 + +## 核心发现（决定技术路线） + +1. **mark 的 RGB 是"捐赠补丁"，不是洞内内容。** + 决定性实验：真掩码 NCC 全图任意位置最高仅 0.562（41ac）/ 0.464（444d）。 + 若"洞 = 该处照片变暗、mark = 洞下原图"成立，真目标处 NCC 应接近 1.0 + （NCC 对仿射亮度变换不变）。因此**一切基于 RGB/灰度的模板匹配在原理上不可行**。 +2. **洞是叠加在照片上的实心暗色几何覆盖物**，边界 = mark 的 alpha 轮廓。 + 可靠信号只有三个：**alpha 轮廓形状、尺度≈1.0、旋转≈0°**。 +3. **照片纹理会产生大量虚假贴合**：纯倒角距离会被草地等密纹理区域击败。 + 必须叠加方向约束与洞内部特征。 + +## 最终算法（`src/method_l_shape.py`） + +```text +粗扫：尺度 0.85–1.15（步长 0.025）× 全图位置 + 评分 = 方向感知倒角DT + 20×内部边缘密度 + 0.5×max(0, 内-环亮度差) + ↓ +全图 NMS 取 top-5 候选 + ↓ +每候选局部旋转扫描：36 角（步长 10°）× ±3px 平移 + → rot*（近并列 ±0.15 内取离 0 最近者，兼容星形等旋转对称形状） + → rot=0 拟合分 refined + ↓ +选择：|rot*|≤12° 且 0.96≤s≤1.05 的候选中 refined 最低者 + 否则取 |rot*| 最小并标 AMBIGUOUS +``` + +三个评分项的物理含义： + +| 项 | 含义 | 区分度（实测） | +| --- | --- | --- | +| 方向感知倒角DT | 边界点只匹配同法向桶（9 桶×20°）的 Canny 边缘 | 真洞 3–6，天空截断 8+ | +| 内部边缘密度 | 洞内部实心平坦（≈0），纹理区内部边缘多 | 真洞 ≤0.1，草地 0.36 | +| 内-环亮度差 | 洞比周围暗（半透明暗色覆盖） | 真洞 -40~-104，误检区 +6 | + +Canny 阈值固定 `lo = max(40, 0.5×中位灰度)`、`hi = 3×lo`。 +阈值过低会让 JPEG 噪声在平坦区产生大量假边缘，污染"内部平坦"特征 +（曾导致 4db9 星形样本漏检）。 + +## 判别力证据 + +所有样本的候选旋转曲线均能区分目标/干扰： + +- 目标：`rot*≈0°` 且 rot=0 拟合分明显低于其他角度； +- 干扰项：`rot*≠0°`（如 41ac 左月牙 -10°、8727 竞争星 +20°、f4fc 竞品 -20°）。 + +三个独立真值锚点全部命中： + +| 样本 | 独立真值来源 | 真值 | 本方案 | 偏差 | +| --- | --- | --- | --- | --- | +| 4db94793 | SIFT + RANSAC | (295.8, 142.1) | (297, 143) | ~2px | +| d7a7f80c | SIFT + RANSAC | (167.8, 146.4) | (171, 146) | ~4px | +| 444d0ea7 | 闭合轮廓分析（Hu≈0.003） | 左洞 rot≈0° | (183, 160) rot=0° | 一致 | + +## API（`src/solve.py`） + +```python +from solve import solve +result = solve(big_image_b64, mark_image_b64) # 支持 dataURI 前缀 +``` + +命令行：`python src/solve.py <大图路径或base64> ` + +返回： + +```json +{ + "ok": true, + "distance_px": 297, + "x": 297, "y": 143, + "scale": 1.0, "rot": 0.0, + "score": 2.429, + "confidence": "OK", + "candidates": [...] +} +``` + +- `distance_px`：mark 110×110 画布左上角应到达的 x 偏移； + 若拖动 UI 的 mark 初始位置不在 x=0，用 `x` 减去初始 x。 +- `confidence`：`OK`（满足 rot/尺度门限）或 `AMBIGUOUS`（降级选择，建议人工复核）。 +- mark 图必须含 alpha 通道（PNG），否则返回 `ok=false`。 + +## 结果验证方法 + +1. **叠加轮廓检查（最直接）**：`python src/verify_result.py` 生成 + `src/out/L/verify/`，把 alpha 轮廓按 offset 以绿色半透明+黄色描边叠回大图。 + 轮廓与暗色图形边缘完全重合 = 正确。 +2. **候选对比**：看 `src/out/L/<前缀>-match.png`，红框目标应同形同大同向， + 其余框应为旋转/缩放干扰项或被淘汰纹理。 +3. **数值检查**：`summary.txt` 中目标 rot≈0、scale∈[0.96,1.05]、精修分最低。 + +## 10 样本最终结果 + +| 样本 | offset | 尺度 | rot* | 精修分 | 备注 | +| --- | --- | --- | --- | --- | --- | +| 2a313202f58c | (370,114) | 1.025 | 0° | 4.04 | | +| 41ac8781f7e9 | (398,151) | 1.000 | 0° | 4.40 | 右月牙；bbox 88×106 与 alpha 完全一致 | +| 444d0ea7c8bb | (178,148) | 1.000 | 0° | 4.50 | 轮廓分析交叉验证 ✓ | +| 4db94793ec6d | (297,143) | 1.000 | 0° | 2.43 | SIFT 真值 ✓ | +| 632f84299d75 | (113,136) | 1.025 | 0° | 6.82 | 拟合最弱，置信度中 | +| 6a53f4529063 | (166,152) | 1.025 | 0° | 4.97 | | +| 87276de26640 | (153,134) | 1.025 | 0° | 3.94 | 星形，靠旋转扫描解决 | +| d6cd0ebb7eed | (200,55) | 1.000 | 0° | 5.82 | | +| d7a7f80c4554 | (171,146) | 1.025 | 0° | 3.11 | SIFT 真值 ✓ | +| f4fc5aa426ce | (298,68) | 1.050 | 0° | 4.79 | | + +## 已知边界 + +- `632f` 精修分 6.82 明显高于其余样本（2.4–5.8），建议保留人工复核， + 或积累更多样本校准"可信分数"阈值； +- 洞叠在深色照片上时"内-环亮度差"约束减弱，可提高方向倒角权重； +- 若验证码改用亮色覆盖物，需要把亮度差项改为双向（|内-环|）并重标权重。 + +## 复现 + +```bash +.venv/bin/python src/method_l_shape.py # 批量求解 + 可视化 +.venv/bin/python src/verify_result.py # 叠加轮廓验证图 +.venv/bin/python src/solve.py <大图> # 单样本 API +``` diff --git a/pyrightconfig.json b/pyrightconfig.json new file mode 100644 index 0000000..df535fa --- /dev/null +++ b/pyrightconfig.json @@ -0,0 +1,6 @@ +{ + "venvPath": ".", + "venv": ".venv", + "extraPaths": [".venv/lib/python3.13/site-packages", "src", "try"], + "pythonPath": ".venv/bin/python" +} diff --git a/src/method_l_shape.py b/src/method_l_shape.py new file mode 100644 index 0000000..f17a6d1 --- /dev/null +++ b/src/method_l_shape.py @@ -0,0 +1,305 @@ +"""方法L：方向感知倒角 + 洞内部特征 + 局部旋转扫描（最终方案）。 + +原理（依据 try/findings.md 的实验结论）： +- mark 的 RGB 是"捐赠补丁"，与洞内内容无像素关系，RGB 匹配不可行； +- 洞是叠加在照片上的暗色几何覆盖物，边界 = alpha 轮廓（尺度1、旋转0）； +- 干扰项是同形的缩放/旋转实例。 + +流程： +1. 多尺度方向感知倒角粗扫：边界点只匹配同法向桶的边缘， + 叠加内部边缘密度（洞内平坦）与内-环亮度差（洞更暗）惩罚纹理误检； +2. 全图 NMS 取 top-5 候选； +3. 每个候选做局部旋转扫描（36 角 x ±3px 平移），得到最佳旋转角与精修分； +4. 选择：|rot*|<=12 度的候选中取精修分最低者；否则取 |rot*| 最小并标 AMBIGUOUS。 + +输出：try/out/L/{*-match.png,summary.txt} +""" +from pathlib import Path +import math +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "src" / "out" / "L" +OUT.mkdir(parents=True, exist_ok=True) + +NB = 9 # 法向方向桶数（每桶 20 度） +SCALES = np.arange(0.85, 1.151, 0.025) # 目标尺度约 1.0（洞与 mark 同尺寸） +W_EDGE, W_DARK = 20.0, 0.5 # 内部边缘密度 / 亮度差惩罚权重 +TOP_K = 5 +MAX_DTP = 10.0 # 倒角距离截断 +ROT_STEP = 10.0 # 旋转扫描步长（度） +SHIFT = 3 # 旋转扫描平移半径（px） +ROT_TOL = 12.0 # 目标允许的旋转角 +S_LOW, S_HIGH = 0.96, 1.05 # 目标尺度门限（444d/4db9/41ac 均实测 s=1.00） +TIE_TOL = 0.15 # 旋转角近并列容差 + + +def files_for(prefix): + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None, None + big = bigs[0] + return big, CAP / f"{big.stem}-mark.png" + + +def wrap90(deg): + return (deg + 90.0) % 180.0 - 90.0 + + +def setup(big, mark): + try: + if big is None or mark is None: + return None + alpha = mark[..., 3] if mark.ndim == 3 and mark.shape[2] == 4 else np.full(mark.shape[:2], 255, np.uint8) + mask = (alpha > 128).astype(np.uint8) + ys, xs = np.where(mask > 0) + x0, y0 = int(xs.min()), int(ys.min()) + am = mask[ys.min():ys.max() + 1, xs.min():xs.max() + 1] + bnd = cv2.morphologyEx(am * 255, cv2.MORPH_GRADIENT, np.ones((3, 3), np.uint8)) > 0 + inner = cv2.erode(am * 255, np.ones((5, 5), np.uint8)) > 0 + ring = cv2.dilate(am * 255, np.ones((13, 13), np.uint8)) > 0 + ring &= ~(cv2.dilate(am * 255, np.ones((3, 3), np.uint8)) > 0) + gray = cv2.cvtColor(big, cv2.COLOR_BGR2GRAY) + med = float(np.median(gray)) + lo = max(40.0, 0.5 * med) + edges = cv2.Canny(gray, lo, min(255.0, 3.0 * lo)) + gx = cv2.Sobel(gray, cv2.CV_32F, 1, 0, ksize=3) + gy = cv2.Sobel(gray, cv2.CV_32F, 0, 1, ksize=3) + ang = np.degrees(np.arctan2(gy, gx)) % 180.0 + dts = np.zeros((NB, gray.shape[0], gray.shape[1]), np.float32) + for k in range(NB): + lo_k, hi_k = k * 180.0 / NB, (k + 1) * 180.0 / NB + eb = ((edges > 0) & (ang >= lo_k) & (ang < hi_k)).astype(np.uint8) * 255 + dts[k] = np.minimum(cv2.distanceTransform(255 - eb, cv2.DIST_L2, 3), MAX_DTP) + mx = cv2.Sobel(am.astype(np.float32), cv2.CV_32F, 1, 0, ksize=3) + my = cv2.Sobel(am.astype(np.float32), cv2.CV_32F, 0, 1, ksize=3) + mang = np.degrees(np.arctan2(my, mx)) % 180.0 + return dict(big=big, gray=gray, am=am, bnd=bnd, inner=inner, ring=ring, + dts=dts, mang=mang, bbox=(x0, y0), edges=(edges > 0), + shape=(gray.shape[0], gray.shape[1])) + except Exception as e: + print(f" 初始化失败: {e}") + return None + + +def crop_full(full, w, h, H, W): + return full[h // 2:h // 2 + H - h + 1, w // 2:w // 2 + W - w + 1] + + +def score_at_scale(d, s): + """返回 (score_map, w, h, 边界点集)；score 越小越好。""" + H, W = d["shape"] + h, w = max(8, round(d["am"].shape[0] * s)), max(8, round(d["am"].shape[1] * s)) + if h >= H or w >= W: + return None, 0, 0, None + bi = cv2.resize(d["bnd"].astype(np.float32), (w, h), interpolation=cv2.INTER_NEAREST) + ma = cv2.resize(d["mang"].astype(np.float32), (w, h), interpolation=cv2.INTER_NEAREST) + inner = cv2.resize(d["inner"].astype(np.float32), (w, h), interpolation=cv2.INTER_NEAREST) + ring = cv2.resize(d["ring"].astype(np.float32), (w, h), interpolation=cv2.INTER_NEAREST) + gray = d["gray"].astype(np.float32) + edge_f = d["edges"].astype(np.float32) + try: + cham = np.zeros((H - h + 1, W - w + 1), np.float32) + for k in range(NB): + lo_k, hi_k = k * 180.0 / NB, (k + 1) * 180.0 / NB + bk = np.where((bi > 0) & (ma >= lo_k) & (ma < hi_k), 1.0, 0.0).astype(np.float32) + if float(bk.sum()) < 20: + continue + full = cv2.filter2D(d["dts"][k], -1, bk, borderType=cv2.BORDER_CONSTANT) + cham += crop_full(full, w, h, H, W) + cham /= max(float(bi.sum()), 1.0) + n_in = max(float(inner.sum()), 1.0) + dens = crop_full(cv2.filter2D(edge_f, -1, inner, borderType=cv2.BORDER_CONSTANT), w, h, H, W) / n_in + m_in = crop_full(cv2.filter2D(gray, -1, inner, borderType=cv2.BORDER_CONSTANT), w, h, H, W) / n_in + m_ring = crop_full(cv2.filter2D(gray, -1, ring, borderType=cv2.BORDER_CONSTANT), w, h, H, W) / max(float(ring.sum()), 1.0) + dark_pen = np.maximum(m_in - m_ring, 0.0) + rr, cc = np.nonzero(bi > 0) + pts = np.stack([cc, rr], axis=1).astype(np.float32) + angs = ma[rr, cc].astype(np.float32) + return cham + W_EDGE * dens + W_DARK * dark_pen, w, h, (pts, angs) + except cv2.error: + return None, 0, 0, None + + +def nms_take(score, count, radius): + out = [] + try: + r = score.copy() + for _ in range(count): + _, _, mloc, _ = cv2.minMaxLoc(r) + v = float(r[mloc[1], mloc[0]]) + if v >= 1e8: + break + out.append((v, mloc[0], mloc[1])) + x0, y0 = max(0, mloc[0] - radius), max(0, mloc[1] - radius) + r[y0:mloc[1] + radius, x0:mloc[0] + radius] = 1e9 + except Exception as e: + print(f" NMS 失败: {e}") + return out + + +def rotation_scan(d, pts, angs, w, h, x, y): + """在候选位置旋转扫描。 + + 返回 dict：rot=最佳旋转角（近并列取离 0 最近者）、dx/dy=平移精修、 + refined=rot=0 时的拟合分（排名用）、best=任意角度的最低分。 + """ + H, W = d["shape"] + cx, cy = w / 2.0, h / 2.0 + rel = pts - np.array([cx, cy], np.float32) + + def score_at(deg, dx, dy): + try: + c_, s_ = math.cos(math.radians(deg)), math.sin(math.radians(deg)) + px = rel[:, 0] * c_ - rel[:, 1] * s_ + cx + py = rel[:, 0] * s_ + rel[:, 1] * c_ + cy + a2 = (angs + deg) % 180.0 + bins = np.minimum((a2 / (180.0 / NB)).astype(np.int32), NB - 1) + gx = np.round(px).astype(np.int32) + x + dx + gy = np.round(py).astype(np.int32) + y + dy + okp = (gx >= 0) & (gx < W) & (gy >= 0) & (gy < H) + vals = np.where(okp, d["dts"][bins, np.clip(gy, 0, H - 1), np.clip(gx, 0, W - 1)], MAX_DTP) + return float(vals.mean()) + except Exception as e: + print(f" score_at 失败: {e}") + return 1e9 + + try: + best, best0 = None, None + for deg in np.arange(-180.0, 180.0, ROT_STEP): + for dy in range(-SHIFT, SHIFT + 1): + for dx in range(-SHIFT, SHIFT + 1): + sc = score_at(deg, dx, dy) + if best is None or sc < best[0] - 1e-9: + best = (sc, deg, dx, dy) + if deg == 0.0 and (best0 is None or sc < best0[0] - 1e-9): + best0 = (sc, dx, dy) + if best is None or best0 is None: + return None + score0, rot0, _, _ = best + tie = [deg for deg in np.arange(-180.0, 180.0, ROT_STEP) + if score_at(float(deg), best[2], best[3]) <= score0 + TIE_TOL] + rot = min(tie, key=lambda g: abs(wrap90(g))) if tie else rot0 + return dict(rot=rot, dx=best0[1], dy=best0[2], refined=best0[0], best=score0) + except Exception as e: + print(f" 旋转扫描失败: {e}") + return None + + +def solve_pair(big, mark): + """纯函数：给定大图与 mark 图（BGR/BGRA ndarray），返回求解结果。 + + 返回 dict： + - x, y : mark 画布左上角应放置的位置（即拖动目标偏移） + - scale, rot : 目标尺度与旋转角（rot 已归一到 [-90,90)） + - score : 目标位置精修拟合分（越小越可信） + - confidence : "OK" 或 "AMBIGUOUS" + - candidates : 全部候选明细（调试用） + 失败返回 None。 + """ + try: + d = setup(big, mark) + if d is None: + return None + cands = [] + for s in SCALES: + s = round(float(s), 3) + score, w, h, ptset = score_at_scale(d, s) + if score is None: + continue + for v, x, y in nms_take(score, 3, int(0.45 * max(w, h))): + cands.append((v, s, x, y, w, h, ptset)) + if not cands: + return None + cands.sort(key=lambda c: c[0]) + kept = [] + for c in cands: + _, _, x, y, w, h, _ = c + cx, cy = x + w / 2.0, y + h / 2.0 + if all(math.hypot(cx - (q[2] + q[4] / 2.0), cy - (q[3] + q[5] / 2.0)) > 0.6 * max(w, h) + for q in kept): + kept.append(c) + if len(kept) >= TOP_K: + break + enriched = [] + for v, s, x, y, w, h, ptset in kept: + scan = rotation_scan(d, ptset[0], ptset[1], w, h, x, y) + if scan is None: + continue + enriched.append(dict(coarse=round(float(v), 3), scale=s, x=x + scan["dx"], + y=y + scan["dy"], w=w, h=h, + rot=round(float(wrap90(scan["rot"])), 1), + refined=round(float(scan["refined"]), 3), + best=round(float(scan["best"]), 3))) + if not enriched: + return None + ok = [e for e in enriched if abs(e["rot"]) <= ROT_TOL + and S_LOW <= e["scale"] <= S_HIGH] + if ok: + target = min(ok, key=lambda e: e["refined"]) + confidence = "OK" + else: + target = min(enriched, key=lambda e: (abs(e["rot"]), abs(e["scale"] - 1.0))) + confidence = "AMBIGUOUS" + bx0, by0 = d["bbox"] + s = target["scale"] + off = (target["x"] - round(bx0 * s), target["y"] - round(by0 * s)) + return dict(x=off[0], y=off[1], scale=s, rot=target["rot"], + score=target["refined"], confidence=confidence, + candidates=enriched) + except Exception as e: + print(f" solve_pair 失败: {e}") + return None + + +def process(prefix, lines): + try: + big_path, mark_path = files_for(prefix) + if big_path is None: + lines.append(f"{prefix}\tSKIP\t文件缺失") + return + big = cv2.imread(str(big_path), cv2.IMREAD_COLOR) + mark = cv2.imread(str(mark_path), cv2.IMREAD_UNCHANGED) + if big is None or mark is None: + lines.append(f"{prefix}\tSKIP\t读取失败") + return + res = solve_pair(big, mark) + if res is None: + lines.append(f"{prefix}\tSKIP\t求解失败") + return + detail = " | ".join( + f"({e['x']},{e['y']}) s={e['scale']:.3f} r={e['rot']} " + f"f0={e['refined']:.2f} fb={e['best']:.2f}" for e in res["candidates"]) + print(f"{prefix} score={res['score']:.2f} offset=({res['x']},{res['y']}) " + f"scale={res['scale']:.3f} rot={res['rot']} {res['confidence']}") + print(f" 全部候选: {detail}") + lines.append(f"{prefix}\tscore={res['score']:.2f}\toffset=({res['x']},{res['y']})\t" + f"scale={res['scale']:.3f}\trot={res['rot']}\t{res['confidence']}\t{detail}") + vis = big.copy() + colors = [(0, 0, 255), (0, 165, 255), (255, 0, 0), (0, 255, 0), (255, 0, 255)] + for i, e in enumerate(res["candidates"]): + color = colors[min(i, len(colors) - 1)] + cv2.rectangle(vis, (e["x"], e["y"]), (e["x"] + e["w"], e["y"] + e["h"]), color, 2) + tag = f"{i + 1} s={e['scale']:.2f} r={e['rot']}" + cv2.putText(vis, tag, (e["x"] + 2, max(14, e["y"] - 4)), + cv2.FONT_HERSHEY_SIMPLEX, 0.45, color, 1) + cv2.imwrite(str(OUT / f"{prefix}-match.png"), vis) + except Exception as e: + print(f"{prefix} 处理失败: {e}") + lines.append(f"{prefix}\tERROR\t{e}") + + +def main(): + lines = [] + prefixes = sorted({p.name.split("~", 1)[0] for p in CAP.glob("*.jpeg") + if "-mark" not in p.stem}) + for prefix in prefixes: + process(prefix, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\n结果已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/src/solve.py b/src/solve.py new file mode 100644 index 0000000..71bc639 --- /dev/null +++ b/src/solve.py @@ -0,0 +1,77 @@ +"""拖动重叠验证码求解 API。 + +用法（库）: + from solve import solve + result = solve(big_image_b64, mark_image_b64) + # result["distance_px"] 即需要拖动的水平像素值 + +用法（命令行，参数为文件路径或 base64 串）: + python try/solve.py <大图> + +返回 JSON 字段： +- ok : 是否求解成功 +- distance_px : 需要拖动的水平像素（mark 画布左上角应到达的 x 偏移； + 若拖动 UI 的 mark 初始位置不在 x=0，请用 x 减去初始 x） +- x, y : mark 110x110 画布左上角在大图中应放置的位置 +- scale / rot : 目标的尺度与旋转角（rot∈[-90,90)） +- score : 拟合分（越小越可信） +- confidence : "OK" 或 "AMBIGUOUS" +- candidates : 全部候选明细（调试用） +""" +from pathlib import Path +import base64 +import json +import sys +import cv2 +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from method_l_shape import solve_pair # noqa: E402 + + +def decode_b64(data): + """解码 base64（兼容 dataURI 前缀）为图像 ndarray；失败返回 None。""" + try: + payload = data.split(",", 1)[-1] + arr = np.frombuffer(base64.b64decode(payload), np.uint8) + return cv2.imdecode(arr, cv2.IMREAD_UNCHANGED) + except Exception as e: + print(f" base64 解码失败: {e}") + return None + + +def solve(big_b64, mark_b64): + """输入大图与 mark 图的 base64，返回求解结果 dict。""" + try: + big = decode_b64(big_b64) + mark = decode_b64(mark_b64) + if big is None or mark is None: + return {"ok": False, "message": "图片解码失败"} + if mark.ndim == 2 or (mark.ndim == 3 and mark.shape[2] < 4): + return {"ok": False, "message": "mark 图缺少 alpha 通道"} + result = solve_pair(big, mark) + if result is None: + return {"ok": False, "message": "未找到目标区域"} + result["ok"] = True + result["distance_px"] = result["x"] + return result + except Exception as e: + return {"ok": False, "message": f"求解失败: {e}"} + + +def main(): + if len(sys.argv) != 3: + print("用法: python try/solve.py <大图路径或base64> ") + return + args = [] + for arg in sys.argv[1:]: + path = Path(arg) + if path.exists(): + args.append(base64.b64encode(path.read_bytes()).decode()) + else: + args.append(arg) + print(json.dumps(solve(args[0], args[1]), ensure_ascii=False, indent=1)) + + +if __name__ == "__main__": + main() diff --git a/src/verify_result.py b/src/verify_result.py new file mode 100644 index 0000000..ec2fcfa --- /dev/null +++ b/src/verify_result.py @@ -0,0 +1,79 @@ +"""结果验证可视化：按 L 的 offset 把 mark alpha 轮廓叠回大图。 + +读 try/out/L/summary.txt 中每个样本选定的 offset，把 mark 的 alpha +区域以半透明绿色叠画到大图上。若绿轮廓与暗色几何图形完全重合， +则该 offset 正确。 + +输出：try/out/L/verify/<前缀>-overlay.png（原尺寸）与 verify_sheet.png（汇总） +""" +from pathlib import Path +import re +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +SRC = ROOT / "src" / "out" / "L" +OUT = SRC / "verify" +OUT.mkdir(parents=True, exist_ok=True) + + +def parse_summary(path): + rows = [] + for line in path.read_text(encoding="utf-8").splitlines(): + try: + m = re.match(r"^(\S+)\tscore=(\S+)\toffset=\((\d+),(\d+)\)", line) + if m: + rows.append((m.group(1), int(m.group(3)), int(m.group(4)))) + except (ValueError, AttributeError) as e: + print(f" 行解析失败: {e}") + return rows + + +def overlay(prefix, ox, oy): + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None + big = cv2.imread(str(bigs[0])) + mark = cv2.imread(str(CAP / f"{bigs[0].stem}-mark.png"), cv2.IMREAD_UNCHANGED) + if big is None or mark is None: + return None + alpha = (mark[..., 3] > 128) + vis = big.copy() + h, w = alpha.shape + y0, x0 = max(0, oy), max(0, ox) + y1, x1 = min(big.shape[0], oy + h), min(big.shape[1], ox + w) + if y1 <= y0 or x1 <= x0: + return None + sub = alpha[y0 - oy:y1 - oy, x0 - ox:x1 - ox] + region = vis[y0:y1, x0:x1] + region[sub] = (0.4 * region[sub] + 0.6 * np.array([0, 255, 0])).astype(np.uint8) + # 轮廓描边，便于观察贴合精度 + mask = np.zeros(alpha.shape, np.uint8) + mask[sub] = 255 + edge = cv2.morphologyEx(mask, cv2.MORPH_GRADIENT, np.ones((3, 3), np.uint8)) > 0 + region[edge] = (0, 255, 255) + cv2.rectangle(vis, (ox, oy), (ox + w, oy + h), (0, 255, 0), 1) + cv2.imwrite(str(OUT / f"{prefix}-overlay.png"), vis) + small = cv2.resize(vis, (368, 229)) + cv2.putText(small, prefix[:8], (4, 16), cv2.FONT_HERSHEY_SIMPLEX, .5, (0, 0, 255), 1) + return small + + +def main(): + rows = parse_summary(SRC / "summary.txt") + if not rows: + print("summary.txt 无可用结果") + return + sheets = [] + for prefix, ox, oy in rows: + small = overlay(prefix, ox, oy) + if small is not None: + sheets.append(small) + if sheets: + cv2.imwrite(str(OUT / "verify_sheet.png"), np.vstack(sheets)) + print(f"已生成 {len(sheets)} 张叠加图 -> {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/adjudicate.py b/try/adjudicate.py new file mode 100644 index 0000000..c29cd51 --- /dev/null +++ b/try/adjudicate.py @@ -0,0 +1,116 @@ +"""分歧裁决：对争议候选位置做细粒度尺度扫描 + 分通道 NCC。 + +用法：.venv/bin/python try/adjudicate.py +对 41ac 的 A2/C 两候选、444d 的共识候选、以及基线对 4db9/d7a7 做细化评估， +输出每候选的最优尺度、NCC 与分通道 NCC，判断真目标。 +""" +from pathlib import Path +import sys +import cv2 +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from method_c_rotation import load_mark, template_at, patch_ncc # noqa: E402 + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" + +CASES = { + # 12位前缀: [(标签, 中心, 初始尺度), ...] + "41ac8781f7e9": [ + ("C ", (218.0, 223.0), 0.40), + ("A2 ", (110.0, 216.0), 0.50), + ("A2#2", (363.0, 214.5), 0.50), + ("A2#3", (270.0, 223.5), 0.50), + ], + "444d0ea7c8bb": [("共识", (232.5, 202.5), 1.10)], + "4db94793ec6d": [("基线", (351.0, 197.0), 1.05)], + "d7a7f80c4554": [("基线", (222.8, 201.4), 0.95)], +} +SCALES_FINE = np.arange(-0.12, 0.121, 0.02) + + +def find_files(prefix): + """按 12 位前缀 glob 完整文件名。""" + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None, None + return bigs[0], CAP / f"{bigs[0].stem}-mark.png" + + +def channel_ncc(big_f, center, tpl, mask): + """分通道 NCC（模板与 patch 中心对齐）。失败返回 None。""" + try: + h, w = tpl.shape[:2] + x0 = int(round(center[0] - w / 2.0)) + y0 = int(round(center[1] - h / 2.0)) + if x0 < 0 or y0 < 0 or x0 + w > big_f.shape[1] or y0 + h > big_f.shape[0]: + return None + patch = big_f[y0:y0 + h, x0:x0 + w] + vals = [] + for c in range(3): + res = cv2.matchTemplate(patch[..., c:c + 1].copy(), tpl[..., c:c + 1].copy(), + cv2.TM_CCOEFF_NORMED, mask=mask[..., None].copy()) + v = float(np.nan_to_num(res[0, 0], nan=-1.0)) + vals.append(v) + return vals + except cv2.error as e: + print(f" 分通道失败: {e}") + return None + + +def refine(big_f, bgr, alpha, center, scale0): + """细粒度尺度扫描，返回 (best_ncc, best_scale, 分通道NCC)。""" + try: + best_ncc = -1.0 + best_scale = scale0 + best_ch = None + for ds in SCALES_FINE: + s = float(scale0 + ds) + tpl, mask, _, _ = template_at(bgr, alpha, s, 0.0) + if tpl is None: + continue + ncc = patch_ncc(big_f, center[0], center[1], tpl, mask) + if ncc > best_ncc: + ch = channel_ncc(big_f, center, tpl, mask) + best_ncc = ncc + best_scale = s + best_ch = ch + return best_ncc, best_scale, best_ch + except Exception as e: + print(f" 细化失败: {e}") + return (-1.0, scale0, None) + + +def process_case(prefix, cands): + try: + big_path, mark_path = find_files(prefix) + if big_path is None: + print(f"{prefix} 样本缺失") + return + big = cv2.imread(str(big_path), cv2.IMREAD_COLOR) + if big is None: + print(f"{prefix} 大图读取失败") + return + big_f = big.astype(np.float32) + bgr, alpha = load_mark(mark_path) + if bgr is None or alpha is None: + print(f"{prefix} mark 读取失败") + return + print(f"== {prefix}") + for label, center, scale0 in cands: + ncc, s, ch = refine(big_f, bgr, alpha, center, scale0) + chs = f"BGR={ch[0]:.2f}/{ch[1]:.2f}/{ch[2]:.2f}" if ch else "N/A" + print(f" {label} center=({center[0]:.0f},{center[1]:.0f}) " + f"scale*={s:.2f} ncc={ncc:.3f} {chs}") + except Exception as e: + print(f"{prefix} 处理失败: {e}") + + +def main(): + for prefix, cands in CASES.items(): + process_case(prefix, cands) + + +if __name__ == "__main__": + main() diff --git a/try/compare_methods.py b/try/compare_methods.py new file mode 100644 index 0000000..06f2949 --- /dev/null +++ b/try/compare_methods.py @@ -0,0 +1,148 @@ +"""三方法交叉对比可视化。 + +读取 A2/C 的 summary.txt，把各自的候选框画到同一张大图上： +- 红=C 的目标框（scale+offset） +- 黄=A2 的 #1 峰框 +分歧对（两框中心距离 > 30px）额外输出两倍放大局部图。 +输出 → try/out/compare/ +""" +from pathlib import Path +import math +import re +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +A2 = ROOT / "try" / "out" / "A2" / "summary.txt" +CS = ROOT / "try" / "out" / "C" / "summary.txt" +OUT = ROOT / "try" / "out" / "compare" +OUT.mkdir(parents=True, exist_ok=True) + + +def parse_a2(text): + """name -> (scale, (x, y))，取 #1 峰。""" + out = {} + try: + for line in text.splitlines(): + m = re.match(r"^(\S+)\t.*?尺度=([\d.]+)\t.*?#1:([\d.]+)@\((\d+), (\d+)\)", line) + if m: + s = float(m.group(2)) + x = int(m.group(4)) + y = int(m.group(5)) + out[m.group(1)] = (s, (x, y)) + except Exception as e: + print(f" A2 summary 解析失败: {e}") + return out + + +def parse_c(text): + """name -> (scale, (x, y))，取目标行。""" + out = {} + try: + for line in text.splitlines(): + m = re.match(r"^(\S+)\t目标\tscale=([\d.]+)\t.*?offset=\((\d+),(\d+)\)", line) + if m: + s = float(m.group(2)) + x = int(m.group(3)) + y = int(m.group(4)) + out[m.group(1)] = (s, (x, y)) + except Exception as e: + print(f" C summary 解析失败: {e}") + return out + + +def box_center(scale, xy): + """模板框中心坐标。""" + try: + size = 110.0 * scale + return (xy[0] + size / 2.0, xy[1] + size / 2.0) + except Exception as e: + print(f" 中心计算失败: {e}") + return (0.0, 0.0) + + +def zoom_crop(big, center, size_px, factor=2.0): + """以 center 为中心裁 2.2*size 区域并放大。""" + try: + half = int(size_px * 1.1) + x0 = max(0, int(center[0] - half)) + y0 = max(0, int(center[1] - half)) + x1 = min(big.shape[1], int(center[0] + half)) + y1 = min(big.shape[0], int(center[1] + half)) + crop = big[y0:y1, x0:x1].copy() + return cv2.resize(crop, None, fx=factor, fy=factor, interpolation=cv2.INTER_CUBIC) + except cv2.error as e: + print(f" 裁剪失败: {e}") + return None + + +def draw_box(vis, xy, size, color, label, ly): + """画框+标签，参数已验证。""" + try: + x1 = int(xy[0] + size) + y1 = int(xy[1] + size) + cv2.rectangle(vis, xy, (x1, y1), color, 2) + cv2.putText(vis, label, (xy[0], max(12, ly)), + cv2.FONT_HERSHEY_SIMPLEX, 0.5, color, 2) + except cv2.error as e: + print(f" 绘制失败: {e}") + + +def process_pair(name, s_c, xy_c, s_a, xy_a): + try: + big = cv2.imread(str(CAP / f"{name}.jpeg"), cv2.IMREAD_COLOR) + if big is None: + # summary 里的名字是 12 位截断，匹配完整文件名 + full = [p.stem for p in CAP.glob(f"{name}*.jpeg") if "-mark" not in p.stem] + if not full: + print(f"{name[:12]} 大图缺失") + return + big = cv2.imread(str(CAP / f"{full[0]}.jpeg"), cv2.IMREAD_COLOR) + if big is None: + print(f"{name[:12]} 大图读取失败") + return + cc = box_center(s_c, xy_c) + ca = box_center(s_a, xy_a) + dist = math.hypot(cc[0] - ca[0], cc[1] - ca[1]) + vis = big.copy() + draw_box(vis, xy_c, 110.0 * s_c, (0, 0, 255), "C", xy_c[1] - 4) + draw_box(vis, xy_a, 110.0 * s_a, (0, 255, 255), "A2", xy_a[1] + 14) + cv2.imwrite(str(OUT / f"{name[:12]}-cmp.png"), vis) + if dist <= 30: + print(f"{name[:12]} 一致 中心距={dist:5.1f}px C@({cc[0]:.0f},{cc[1]:.0f}) s={s_c:.2f}") + else: + print(f"{name[:12]} 分歧 中心距={dist:5.1f}px C@({cc[0]:.0f},{cc[1]:.0f}) s={s_c:.2f}" + f" A2@({ca[0]:.0f},{ca[1]:.0f}) s={s_a:.2f}") + img_c = zoom_crop(big, cc, 110.0 * s_c) + img_a = zoom_crop(big, ca, 110.0 * s_a) + if img_c is not None: + cv2.imwrite(str(OUT / f"{name[:12]}-cmp-C.png"), img_c) + if img_a is not None: + cv2.imwrite(str(OUT / f"{name[:12]}-cmp-A2.png"), img_a) + except Exception as e: + print(f"{name[:12]} 对比失败: {e}") + + +def main(): + try: + a2 = parse_a2(A2.read_text(encoding="utf-8")) + c = parse_c(CS.read_text(encoding="utf-8")) + agree, disagree = 0, 0 + for name in sorted(set(a2) & set(c)): + s_c, xy_c = c[name] + s_a, xy_a = a2[name] + cc = box_center(s_c, xy_c) + ca = box_center(s_a, xy_a) + if math.hypot(cc[0] - ca[0], cc[1] - ca[1]) <= 30: + agree += 1 + else: + disagree += 1 + process_pair(name, s_c, xy_c, s_a, xy_a) + print(f"\n一致 {agree} / 分歧 {disagree}；图已写入 {OUT}/") + except Exception as e: + print(f" 对比失败: {e}") + + +if __name__ == "__main__": + main() diff --git a/try/explore.py b/try/explore.py new file mode 100644 index 0000000..d5997c3 --- /dev/null +++ b/try/explore.py @@ -0,0 +1,53 @@ +"""样本初探：检查 mark alpha 通道、尺寸，生成可视化素材到 try/visual/。""" +from pathlib import Path +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "visual" +OUT.mkdir(parents=True, exist_ok=True) + + +def mark_stats(path): + """读取 mark 图，返回 (形状, 通道数, alpha>128 占比, 有效bbox) 或 None。""" + try: + m = cv2.imread(str(path), cv2.IMREAD_UNCHANGED) + if m is None: + return None + ch = m.shape[2] if m.ndim == 3 else 1 + a = m[..., 3] if ch == 4 else np.full(m.shape[:2], 255, np.uint8) + ys, xs = np.where(a > 128) + if len(xs) == 0: + return m.shape, ch, 0.0, None + bbox = (int(xs.min()), int(ys.min()), int(xs.max()) + 1, int(ys.max()) + 1) + return m.shape, ch, float((a > 128).mean()), bbox + except Exception as e: # 坏文件不中断批次 + print(f" 读取失败: {e}") + return None + + +def main(): + pairs = sorted({p.name.replace("-mark.png", "") for p in CAP.glob("*-mark.png")}) + print(f"共 {len(pairs)} 对样本\n") + for name in pairs: + st = mark_stats(CAP / f"{name}-mark.png") + if st is None: + print(f"{name[:12]} 跳过") + continue + shape, ch, ratio, bbox = st + print(f"{name[:12]} mark={shape[:2]} ch={ch} alpha>128占比={ratio:.2f} 有效bbox={bbox}") + try: + m = cv2.imread(str(CAP / f"{name}-mark.png"), cv2.IMREAD_UNCHANGED) + if m is None or m.ndim < 2: + print(" 读取失败，跳过可视化") + continue + a = m[..., 3] if m.ndim == 3 and m.shape[2] == 4 else np.full(m.shape[:2], 255, np.uint8) + cv2.imwrite(str(OUT / f"{name[:12]}-alpha.png"), a) + except Exception as e: + print(f" 可视化失败: {e}") + print(f"\nalpha 预览已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/findings.md b/try/findings.md new file mode 100644 index 0000000..ee02556 --- /dev/null +++ b/try/findings.md @@ -0,0 +1,101 @@ +# 实验记录与最终结论 + +> 本文为历史实验日志，经验总结已沉淀至 `docs/solution.md` 与 +> `docs/experiments.md`，以 docs 为准。 + +## 任务 + +对「拖动重叠」验证码：给定背景大图（552×344）与 mark 小图（110×110 RGBA）， +离线求出 mark 应拖动重叠的目标区域偏移。 + +## 关键发现（决定技术路线） + +1. **mark 的 RGB 是"捐赠补丁"**：其内部是场景纹理，与洞内内容无像素关系。 + 实测真掩码 NCC 全图任意位置 ≤0.57（若"洞=该处照片变暗"则真目标应≈1.0）， + 因此**一切基于 RGB/灰度的模板匹配（方法 A/A2/C）原理上不可行**， + 早期 0.7 的高分全是小尺度纹理巧合。 +2. **洞是叠加在照片上的实心暗色几何覆盖物**：形状 = mark 的 alpha 轮廓， + 尺度 ≈1.0、旋转 ≈0；干扰项是同形的缩放/旋转实例。 + 因此唯一可靠信号是 **alpha 轮廓 + 尺度 + 旋转**。 +3. 照片纹理会产生大量虚假"贴合"：纯倒角距离会被草地等密纹理击败 + （早期方法 D/E 失败的根因），必须加方向约束与洞内部特征。 + +## 方法演进（均位于 try/，输出 try/out/<方法字母>/） + +| 方法 | 思路 | 结论 | +| --- | --- | --- | +| A | 蒙版 CCORR 多尺度 | 废弃：不减均值、跨尺度不公平 | +| A2 | 蒙版 CCOEFF + 峰显著度 | 基线可用但会锁纹理（41ac 分歧样本的来源） | +| B | SIFT + RANSAC | 仅 2 个纹理丰富样本可用（4db9/d7a7，作为真值锚点保留） | +| C | 旋转感知 NCC | 第一阶段候选即错，旋转校验无法挽回 | +| D/E/F/G | Chamfer / 覆盖率 / 通用轮廓 / 连通域 | 小尺度纹理饱和 / 背景轮廓合并 | +| H/I/J/K | alpha 轮廓 + 局部对比度 / 暗目标分割 | 阈值敏感，误检集中在照片纹理 | +| **L** | **方向感知倒角 + 洞内部特征 + 局部旋转扫描** | **最终方案，10/10 OK** | + +## 最终方案（src/method_l_shape.py） + +1. **粗扫**：尺度 0.85–1.15（步长 0.025）× 全图位置，评分： + `方向感知倒角DT + 20×内部边缘密度 + 0.5×max(0, 内-环亮度差)` + - 倒角带方向：边界点只匹配同法向桶（9 桶×20°）的 Canny 边缘， + 草地等无方向纹理被自然抑制； + - 内部边缘密度：洞内部实心平坦（≈0），纹理区内部边缘多； + - 内-环亮度差：洞比周围暗（实测 -40~-104），误检区不暗（+6）。 +2. **NMS 取 top-5 候选**。 +3. **局部旋转扫描**：每候选 36 角（步长 10°）× ±3px 平移，得最佳旋转角 + rot*与精修分；近并列（±0.15）角中取离 0 最近者为 rot* + （兼容星形等旋转对称形状）。 +4. **选择**：`|rot*|≤12° 且 0.96≤s≤1.05` 的候选中取精修分最低者； + 否则取 |rot*| 最小并标 AMBIGUOUS。 + +实测判别力：真目标 rot*=0 且 rot=0 拟合分明显低于干扰项 +（干扰项 rot*≠0 或 rot=0 分数高 1~2 分）。 + +## 最终结果（offset = mark 110×110 画布左上角应放置的位置） + +| 样本 | offset | 洞 bbox 左上角 | 尺度 | rot* | 精修分 | 备注 | +| --- | --- | --- | --- | --- | --- | --- | +| 2a313202f58c | (370,114) | (373,125) | 1.025 | 0° | 4.04 | | +| 41ac8781f7e9 | (398,151) | (409,153) | 1.000 | 0° | 4.40 | 右月牙；左月牙为干扰 | +| 444d0ea7c8bb | (178,148) | (183,160) | 1.000 | 0° | 4.50 | 轮廓分析交叉验证 ✓ | +| 4db94793ec6d | (297,143) | (306,146) | 1.000 | 0° | 2.43 | SIFT 真值 (295.8,142.1) ✓ | +| 632f84299d75 | (113,136) | (116,147) | 1.025 | 0° | 6.82 | 拟合最弱，置信度中 | +| 6a53f4529063 | (166,152) | (170,155) | 1.025 | 0° | 4.97 | | +| 87276de26640 | (153,134) | (157,137) | 1.025 | 0° | 3.94 | 星形（椭圆角不可用，旋转扫描解决） | +| d6cd0ebb7eed | (200,55) | (209,58) | 1.000 | 0° | 5.82 | | +| d7a7f80c4554 | (171,146) | (174,157) | 1.025 | 0° | 3.11 | SIFT 真值 (167.8,146.4) ✓ | +| f4fc5aa426ce | (298,68) | (310,70) | 1.050 | 0° | 4.79 | | + +可视化：`src/out/L/<前缀>-match.png`（红框=目标，其余=候选，标注 s/rot）。 + +## API 封装 + +`src/solve.py`：传入两张图片的 base64，返回拖动距离与目标位置。 + +```python +from solve import solve +result = solve(big_image_b64, mark_image_b64) +# result = {"ok": true, "distance_px": 297, "x": 297, "y": 143, +# "scale": 1.0, "rot": 0.0, "score": 2.429, +# "confidence": "OK", "candidates": [...]} +``` + +命令行：`python src/solve.py <大图路径> `。 + +- `distance_px` = mark 110x110 画布左上角应到达的 x 偏移；若拖动 UI 的 + mark 初始位置不在 x=0，用 `x` 减去初始 x。 +- mark 图必须含 alpha 通道（PNG），否则返回 `ok=false`。 + +## 复现 + +```bash +.venv/bin/python src/method_l_shape.py +``` + +## 已知边界 + +- 632f 的拟合分 6.82 明显高于其余样本（2.4–5.8），建议对该类样本 + 保留人工复核或结合更多样本校准阈值； +- 洞若与 mark 非同色/同明度或叠加深色照片上，"内-环亮度差"约束会减弱， + 届时可提高方向倒角权重； +- 无 ground truth 的样本中，444d（轮廓分析）、41ac（bbox 与 alpha 尺寸 + 完全一致 88×106）已通过方法间交叉验证。 diff --git a/try/method_a2_ccoeff.py b/try/method_a2_ccoeff.py new file mode 100644 index 0000000..64f4960 --- /dev/null +++ b/try/method_a2_ccoeff.py @@ -0,0 +1,193 @@ +"""方法A2：蒙版 CCOEFF_NORMED 多尺度匹配 + 跨尺度峰显著度。 + +修正方法A的两个问题： +1. CCORR_NORMED 不减均值 → 换 CCOEFF_NORMED（有蒙版），区分度更高 +2. 跨尺度直接比分数不公平（小模板占便宜）→ 每尺度算峰显著度 (max-median)/std + +输出： +- 每对：1.0 尺度参考、最优显著度尺度、该尺度 top-3 峰（NMS）→ 验证干扰结构 +- 可视化：mark | top-3 峰局部裁剪 并排图 → try/out/A2/ +- 汇总表 → try/out/A2/summary.txt +""" +from pathlib import Path +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "A2" +OUT.mkdir(parents=True, exist_ok=True) + +SCALES = np.arange(0.50, 1.51, 0.05) +TOP_K = 3 + + +def load_mark(path): + """返回 (BGR float32, alpha float32 0~1) 或 (None, None)。""" + try: + m = cv2.imread(str(path), cv2.IMREAD_UNCHANGED) + if m is None or m.ndim < 2: + return None, None + if m.ndim == 3 and m.shape[2] == 4: + return m[..., :3].astype(np.float32), m[..., 3].astype(np.float32) / 255.0 + return m.astype(np.float32), np.ones(m.shape[:2], np.float32) + except Exception as e: + print(f" mark 读取失败: {e}") + return None, None + + +def match_ccoeff(big, tpl_f, mask_f): + """蒙版 CCOEFF_NORMED。返回响应图或 None。""" + try: + res = cv2.matchTemplate(big, tpl_f, cv2.TM_CCOEFF_NORMED, mask=mask_f) + return np.nan_to_num(res, nan=-1.0, posinf=-1.0, neginf=-1.0).astype(np.float32) + except cv2.error as e: + print(f" 匹配失败: {e}") + return None + + +def scale_response(big, bgr_f, alpha_f, scale): + """单尺度匹配，返回 (响应图, 模板宽, 模板高)；失败返回 (None, 0, 0)。""" + try: + if abs(scale - 1.0) < 1e-9: + tpl, mask = bgr_f, alpha_f + else: + w = max(12, round(bgr_f.shape[1] * scale)) + h = max(12, round(bgr_f.shape[0] * scale)) + if w >= big.shape[1] or h >= big.shape[0]: + return None, 0, 0 + tpl = cv2.resize(bgr_f, (w, h), interpolation=cv2.INTER_AREA).astype(np.float32) + mask = cv2.resize(alpha_f, (w, h), interpolation=cv2.INTER_AREA).astype(np.float32) + res = match_ccoeff(big, tpl, mask) + if res is None: + return None, 0, 0 + return res, int(mask.shape[1]), int(mask.shape[0]) + except cv2.error as e: + print(f" 尺度失败(scale={scale}): {e}") + return None, 0, 0 + + +def prominence(res): + """峰显著度：峰高相对响应图整体的突出程度，跨尺度可比。""" + try: + mx = float(res.max()) + med = float(np.median(res)) + sd = float(res.std()) + return (mx - med) / sd if sd > 1e-6 else 0.0 + except Exception: + return 0.0 + + +def topk_peaks(res, tpl_w, tpl_h, k=TOP_K): + """NMS 取前 k 个峰：返回 [(score, (x, y)), ...]。""" + try: + r = res.copy() + peaks = [] + for _ in range(k): + _, score, _, loc = cv2.minMaxLoc(r) + if score <= -1.0: + break + peaks.append((float(score), loc)) + x0 = max(0, loc[0] - int(tpl_w * 0.7)) + y0 = max(0, loc[1] - int(tpl_h * 0.7)) + x1 = min(r.shape[1], loc[0] + int(tpl_w * 0.7)) + y1 = min(r.shape[0], loc[1] + int(tpl_h * 0.7)) + r[y0:y1, x0:x1] = -1.0 + return peaks + except Exception as e: + print(f" 峰提取失败: {e}") + return [] + + +def side_by_side(name, mark_bgr, big, peaks, tpl_w, tpl_h): + """mark | top-k 峰裁剪 并排图。""" + try: + mark_u8 = np.clip(mark_bgr, 0, 255).astype(np.uint8) + mark_u8 = cv2.copyMakeBorder(mark_u8, 2, 2, 2, 2, cv2.BORDER_CONSTANT, + value=(255, 255, 255)) + tiles = [cv2.resize(mark_u8, (tpl_w, tpl_h))] + for score, (x, y) in peaks: + x0, y0 = max(0, x), max(0, y) + x1, y1 = min(big.shape[1], x + tpl_w), min(big.shape[0], y + tpl_h) + crop = big[y0:y1, x0:x1].copy() + if crop.shape[0] != tpl_h or crop.shape[1] != tpl_w: + crop = cv2.copyMakeBorder(crop, 0, tpl_h - crop.shape[0], + 0, tpl_w - crop.shape[1], + cv2.BORDER_CONSTANT, value=(128, 128, 128)) + cv2.putText(crop, f"{score:.3f}", (2, 12), cv2.FONT_HERSHEY_SIMPLEX, + 0.4, (0, 0, 255), 1) + tiles.append(crop) + sep = np.full((tpl_h, 4, 3), 255, np.uint8) + row = tiles[0] + for t in tiles[1:]: + row = np.hstack([row, sep, t]) + cv2.imwrite(str(OUT / f"{name[:12]}-crops.png"), row) + except Exception as e: + print(f" 可视化失败: {e}") + + +def process_pair(name, lines): + try: + big = cv2.imread(str(CAP / f"{name}.jpeg"), cv2.IMREAD_COLOR) + if big is None: + lines.append(f"{name} SKIP 大图读取失败") + return + big_f = big.astype(np.float32) + bgr_f, alpha_f = load_mark(CAP / f"{name}-mark.png") + if bgr_f is None: + lines.append(f"{name} SKIP mark 读取失败") + return + + # 参考：1.0 尺度 + res10, _, _ = scale_response(big_f, bgr_f, alpha_f, 1.0) + if res10 is not None: + _, _, _, loc10 = cv2.minMaxLoc(res10) + ref = f"1.0尺度峰={float(res10.max()):.3f}@{loc10}" + else: + ref = "1.0尺度N/A" + + # 扫尺度，选显著度最高者 + best = None # (显著度, 尺度, 响应图, 模板宽, 模板高) + curve = [] + for sc in SCALES: + res, tw_, th_ = scale_response(big_f, bgr_f, alpha_f, float(sc)) + if res is None: + curve.append(0.0) + continue + p = prominence(res) + curve.append(p) + if best is None or p > best[0]: + best = (p, float(sc), res, tw_, th_) + curve = np.array(curve) + if best is None: + lines.append(f"{name} SKIP 无有效尺度") + return + p, sc, res, tw, th = best + peaks = topk_peaks(res, tw, th) + + top = " ".join(f"#{i+1}:{s:.3f}@{loc}" for i, (s, loc) in enumerate(peaks)) + print(f"{name[:12]} {ref} 最优尺度={sc:.2f} 显著度={p:.2f} {top}") + lines.append(f"{name[:12]}\t{ref}\t尺度={sc:.2f}\t显著度={p:.2f}\t{top}") + side_by_side(name, bgr_f, big, peaks, tw, th) + # 大图上画 top-3 框 + vis = big.copy() + colors = [(0, 0, 255), (0, 165, 255), (255, 0, 0)] + for i, (s, (x, y)) in enumerate(peaks): + cv2.rectangle(vis, (x, y), (x + tw, y + th), colors[i % 3], 2) + cv2.imwrite(str(OUT / f"{name[:12]}-match.png"), vis) + except Exception as e: + print(f"{name[:12]} 处理失败: {e}") + lines.append(f"{name[:12]} ERROR {e}") + + +def main(): + lines = [] + pairs = sorted({p.name.replace("-mark.png", "") for p in CAP.glob("*-mark.png")}) + for name in pairs: + process_pair(name, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\n结果与 summary 已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/method_a_template.py b/try/method_a_template.py new file mode 100644 index 0000000..8b1d051 --- /dev/null +++ b/try/method_a_template.py @@ -0,0 +1,122 @@ +"""方法A基线：蒙版多尺度模板匹配（NCC）。 + +对每对样本： +- 单尺度 1.0：无蒙版 / 有蒙版 —— 验证"裸匹配是否够用" +- 多尺度 0.50~1.50（步长 0.05，有蒙版）：记录每尺度最佳分数，看目标是否为"唯一精确匹配峰" +- 可视化：最佳匹配框 + 响应热力图 → try/out/A/ +""" +from pathlib import Path +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "A" +OUT.mkdir(parents=True, exist_ok=True) + +SCALES = np.arange(0.50, 1.51, 0.05) + + +def load_mark(path): + """返回 (BGR, alpha)，缺 alpha 时返回全 255。""" + try: + m = cv2.imread(str(path), cv2.IMREAD_UNCHANGED) + if m is None or m.ndim < 2: + return None, None + if m.ndim == 3 and m.shape[2] == 4: + return m[..., :3].copy(), m[..., 3].copy() + return m.copy(), np.full(m.shape[:2], 255, np.uint8) + except Exception as e: + print(f" mark 读取失败: {e}") + return None, None + + +def match_masked(big, bgr, alpha, scale): + """缩放模板与蒙版后做 CCORR_NORMED 匹配。返回 (score, loc, res)。""" + try: + if abs(scale - 1.0) < 1e-9: + tpl, mask = bgr, alpha + else: + w = max(8, round(bgr.shape[1] * scale)) + h = max(8, round(bgr.shape[0] * scale)) + if w >= big.shape[1] or h >= big.shape[0]: + return -1.0, (0, 0), None + tpl = cv2.resize(bgr, (w, h), interpolation=cv2.INTER_AREA) + mask = cv2.resize(alpha, (w, h), interpolation=cv2.INTER_AREA) + res = cv2.matchTemplate(big, tpl, cv2.TM_CCORR_NORMED, + mask=mask.astype(np.float32) / 255.0) + res = np.nan_to_num(res, nan=-1.0, posinf=-1.0, neginf=-1.0).astype(np.float32) + _, score, _, loc = cv2.minMaxLoc(res) + return float(score), loc, res + except cv2.error as e: + print(f" 匹配失败(scale={scale}): {e}") + return -1.0, (0, 0), None + + +def heatmap_png(res): + r = res.copy() + r -= r.min() + if r.max() > 0: + r /= r.max() + return cv2.applyColorMap((r * 255).astype(np.uint8), cv2.COLORMAP_JET) + + +def process_pair(name): + try: + big = cv2.imread(str(CAP / f"{name}.jpeg"), cv2.IMREAD_COLOR) + if big is None: + print(f"{name[:12]} 大图读取失败") + return + bgr, alpha = load_mark(CAP / f"{name}-mark.png") + if bgr is None: + print(f"{name[:12]} mark 读取失败") + return + + # 单尺度对照 + res0 = cv2.matchTemplate(big, bgr, cv2.TM_CCOEFF_NORMED) + _, s0, _, l0 = cv2.minMaxLoc(res0) + s1, l1, _ = match_masked(big, bgr, alpha, 1.0) + + # 多尺度扫描 + scores, locs = [], [] + for sc in SCALES: + sco, loc, _ = match_masked(big, bgr, alpha, float(sc)) + scores.append(sco) + locs.append(loc) + curve = np.array(scores) + k = int(curve.argmax()) + best_scale = float(SCALES[k]) + best_score, best_loc = float(curve[k]), locs[k] + masked = curve.copy() + masked[max(0, k - 1): k + 2] = -1.0 + second = float(masked.max()) + + print(f"{name[:12]} 裸1.0={s0:.3f}@{l0} 蒙版1.0={s1:.3f}@{l1} " + f"最佳尺度={best_scale:.2f} 分数={best_score:.3f} 次峰={second:.3f} " + f"峰谷差={float(curve.max() - curve.min()):.3f}") + + # 可视化：匹配框 + 最佳尺度响应热力图 + _, _, res_best = match_masked(big, bgr, alpha, best_scale) + vis = big.copy() + w = max(8, round(bgr.shape[1] * best_scale)) + h = max(8, round(bgr.shape[0] * best_scale)) + cv2.rectangle(vis, best_loc, (best_loc[0] + w, best_loc[1] + h), (0, 0, 255), 2) + cv2.putText(vis, f"s={best_scale:.2f} {best_score:.2f}", + (best_loc[0], max(12, best_loc[1] - 4)), + cv2.FONT_HERSHEY_SIMPLEX, 0.45, (0, 0, 255), 1) + cv2.imwrite(str(OUT / f"{name[:12]}-match.png"), vis) + if res_best is not None: + cv2.imwrite(str(OUT / f"{name[:12]}-heat.png"), heatmap_png(res_best)) + except Exception as e: + print(f"{name[:12]} 处理失败: {e}") + + +def main(): + pairs = sorted({p.name.replace("-mark.png", "") for p in CAP.glob("*-mark.png")}) + for name in pairs: + process_pair(name) + print(f"\n结果图已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/method_b_sift.py b/try/method_b_sift.py new file mode 100644 index 0000000..8c77e72 --- /dev/null +++ b/try/method_b_sift.py @@ -0,0 +1,193 @@ +"""方法B：SIFT 特征匹配 + RANSAC 相似变换聚类。 + +思路：mark(带 alpha 蒙版)与大图各提 SIFT 特征，ratio test 后用 +estimateAffinePartial2D(RANSAC) 迭代提取多个变换簇 —— 每簇对应大图中一个 +相似区域（目标或干扰项）。按 AGENTS.md 断言：目标 = 旋转≈0 的簇。 + +输出： +- 每簇：scale / rotation / 平移(大图坐标系下 mark 左上角) / 内点数 +- 判定：|rotation| 最小的簇为候选目标 +- 可视化：变换后 mark 包围盒画到大图 → try/out/B/ +- 汇总 → try/out/B/summary.txt +""" +from pathlib import Path +import math +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "B" +OUT.mkdir(parents=True, exist_ok=True) + +RATIO = 0.90 +RANSAC_TH = 3.0 +MAX_CLUSTERS = 3 +MIN_MATCHES = 6 + + +def load_mark(path): + """返回 (灰度图, alpha蒙版uint8) 或 (None, None)。""" + try: + m = cv2.imread(str(path), cv2.IMREAD_UNCHANGED) + if m is None or m.ndim < 2: + return None, None + if m.ndim == 3 and m.shape[2] == 4: + gray = cv2.cvtColor(m[..., :3], cv2.COLOR_BGR2GRAY) + alpha = m[..., 3] + else: + gray = cv2.cvtColor(m, cv2.COLOR_BGR2GRAY) + alpha = np.full(m.shape[:2], 255, np.uint8) + return gray, alpha + except Exception as e: + print(f" mark 读取失败: {e}") + return None, None + + +def good_matches(de_m, de_b): + """BF knn + ratio test。返回 (queryIdx, trainIdx) 对列表。""" + try: + matcher = cv2.BFMatcher(cv2.NORM_L2) + knn = matcher.knnMatch(de_m, de_b, k=2) + return [(m1.queryIdx, m1.trainIdx) + for m1, m2 in knn if m1.distance < RATIO * m2.distance] + except Exception as e: + print(f" 匹配失败: {e}") + return [] + + +def extract_clusters(kp_m, kp_b, good): + """迭代 RANSAC，最多 MAX_CLUSTERS 个变换簇。返回 [(M, 内点数), ...]。""" + clusters = [] + try: + pts_m = np.array([kp.pt for kp in kp_m], dtype=np.float32).reshape(-1, 1, 2) + pts_b = np.array([kp.pt for kp in kp_b], dtype=np.float32).reshape(-1, 1, 2) + remaining = np.arange(len(good)) + for _ in range(MAX_CLUSTERS): + if len(remaining) < MIN_MATCHES: + break + idx_pairs = np.array(good, dtype=np.int64)[remaining] + src = pts_m[idx_pairs[:, 0]] + dst = pts_b[idx_pairs[:, 1]] + M, inl = cv2.estimateAffinePartial2D( + src, dst, method=cv2.RANSAC, ransacReprojThreshold=RANSAC_TH, + maxIters=5000) + if M is None or inl is None: + break + n_inl = int(inl.sum()) + if n_inl < 4: + break + clusters.append((M.copy(), n_inl)) + inlier_pos = np.where(inl.ravel() > 0)[0] + used = remaining[inlier_pos] + keep = np.array([i for i, g in enumerate(remaining) + if not np.any(used == g)], dtype=np.int64) + remaining = remaining[keep] + except Exception as e: + print(f" 聚类失败: {e}") + return clusters + + +def describe(M): + """2x3 相似变换 → (scale, rot_deg, tx, ty)；失败返回 (0,0,0,0)。""" + try: + a, b = float(M[0, 0]), float(M[0, 1]) + tx, ty = float(M[0, 2]), float(M[1, 2]) + scale = math.hypot(a, b) + rot = math.degrees(math.atan2(b, a)) + while rot <= -180: + rot += 360 + while rot > 180: + rot -= 360 + return scale, rot, tx, ty + except Exception as e: + print(f" 变换解析失败: {e}") + return 0.0, 0.0, 0.0, 0.0 + + +def draw_cluster(big, M, mark_shape, color): + """把 mark 四角经 M 映射后画到大图上。""" + try: + h, w = mark_shape[:2] + corners = np.array([[0, 0], [w, 0], [w, h], [0, h]], dtype=np.float32) + proj = cv2.transform(corners.reshape(-1, 1, 2), M) + pts = proj.reshape(-1, 2).astype(np.int32) + cv2.polylines(big, [pts], True, color, 2) + scale, rot, _, _ = describe(M) + cv2.putText(big, f"s={scale:.2f} r={rot:.0f}d", + (int(pts[:, 0].min()), max(12, int(pts[:, 1].min()) - 4)), + cv2.FONT_HERSHEY_SIMPLEX, 0.4, color, 1) + except cv2.error as e: + print(f" 绘制失败: {e}") + + +def process_pair(name, sift, lines): + try: + big = cv2.imread(str(CAP / f"{name}.jpeg"), cv2.IMREAD_COLOR) + if big is None: + lines.append(f"{name[:12]}\tSKIP\t大图读取失败") + return + gray_m, alpha_m = load_mark(CAP / f"{name}-mark.png") + if gray_m is None or alpha_m is None: + lines.append(f"{name[:12]}\tSKIP\tmark 读取失败") + return + gray_b = cv2.cvtColor(big, cv2.COLOR_BGR2GRAY) + + kp_m, de_m = sift.detectAndCompute(gray_m, alpha_m) + kp_b, de_b = sift.detectAndCompute(gray_b, None) + n_kp_m = 0 if kp_m is None else len(kp_m) + if de_m is None or de_b is None or kp_m is None or kp_b is None or n_kp_m < MIN_MATCHES: + lines.append(f"{name[:12]}\tSKIP\t特征不足 mk={n_kp_m}") + return + + good = good_matches(de_m, de_b) + if len(good) < MIN_MATCHES: + lines.append(f"{name[:12]}\tSKIP\tratio后匹配不足 {len(good)}") + return + + clusters = extract_clusters(kp_m, kp_b, good) + if not clusters: + lines.append(f"{name[:12]}\tSKIP\t无变换簇") + return + + # 目标候选：排除退化簇（scale 过小/过大），在余下中选 |rotation| 最小者 + def cluster_scale(i): + return describe(clusters[i][0])[0] + + valid = [i for i in range(len(clusters)) + if 0.2 <= cluster_scale(i) <= 3.0] + if not valid: + lines.append(f"{name[:12]}\tSKIP\t仅退化簇") + return + best_idx = min(valid, key=lambda i: abs(describe(clusters[i][0])[1])) + vis = big.copy() + for i, (M, n_inl) in enumerate(clusters): + scale, rot, tx, ty = describe(M) + is_target = (i == best_idx) + color = (0, 0, 255) if is_target else (255, 0, 0) + draw_cluster(vis, M, gray_m.shape, color) + tag = "<==目标候选" if is_target else "" + print(f"{name[:12]} 簇{i}: s={scale:.3f} r={rot:+7.1f}° " + f"t=({tx:6.1f},{ty:6.1f}) 内点={n_inl} {tag}") + lines.append(f"{name[:12]}\t簇{i}\tscale={scale:.3f}\trot={rot:+.1f}\t" + f"t=({tx:.1f},{ty:.1f})\tinliers={n_inl}\t{tag}") + cv2.imwrite(str(OUT / f"{name[:12]}-clusters.png"), vis) + except Exception as e: + print(f"{name[:12]} 处理失败: {e}") + lines.append(f"{name[:12]}\tERROR\t{e}") + + +def main(): + # cartoon 平面图形对比度低，放低对比度阈值、提高特征上限 + sift = cv2.SIFT_create(nfeatures=2000, contrastThreshold=0.02) + lines = [] + pairs = sorted({p.name.replace("-mark.png", "") for p in CAP.glob("*-mark.png")}) + for name in pairs: + process_pair(name, sift, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + n_ok = sum(1 for l in lines if "目标候选" in l) + print(f"\n{n_ok}/{len(pairs)} 对给出目标候选；结果已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/method_c_rotation.py b/try/method_c_rotation.py new file mode 100644 index 0000000..2d3e788 --- /dev/null +++ b/try/method_c_rotation.py @@ -0,0 +1,247 @@ +"""方法C：两阶段旋转感知模板匹配（纯 OpenCV，离线）。 + +阶段1（定位候选）：rot=0 的尺度扫描 0.40~1.20，蒙版 CCOEFF_NORMED， +每尺度取显著度 top-3 峰，跨尺度 NMS 去重，按显著度保留 top-15。 + +阶段2（旋转消歧）：对每个候选位置，把 mark 旋转 -180°~180°（步长 15°） +后与该位置 patch 做 NCC → 得到旋转响应曲线。真目标的最优旋转角应≈0° +（AGENTS.md：目标无缩放无旋转差异；干扰项带旋转）。 +在 |rot*| ≤ ZERO_TOL 的候选中取 NCC 最高者为目标。 + +输出：try/out/C/{*-match.png,*-rot.png,summary.txt} +""" +from pathlib import Path +import math +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "C" +OUT.mkdir(parents=True, exist_ok=True) + +SCALES = np.arange(0.40, 1.21, 0.05) +ANGLES = np.arange(-180.0, 180.0, 15.0) +PEAKS_PER_SCALE = 3 +CAND_KEEP = 15 +ZERO_TOL = 10.0 + + +def load_mark(path): + """返回 (BGR float32, alpha float32 0~1) 或 (None, None)。""" + try: + m = cv2.imread(str(path), cv2.IMREAD_UNCHANGED) + if m is None or m.ndim < 2: + return None, None + if m.ndim == 3 and m.shape[2] == 4: + return m[..., :3].astype(np.float32), m[..., 3].astype(np.float32) / 255.0 + return m.astype(np.float32), np.ones(m.shape[:2], np.float32) + except Exception as e: + print(f" mark 读取失败: {e}") + return None, None + + +def template_at(bgr, alpha, scale, angle): + """缩放(+旋转)模板与蒙版。返回 (tpl, mask, w, h) 或 (None, None, 0, 0)。""" + try: + w = max(12, round(bgr.shape[1] * scale)) + h = max(12, round(bgr.shape[0] * scale)) + if w < 12 or h < 12: + return None, None, 0, 0 + tpl = cv2.resize(bgr, (w, h), interpolation=cv2.INTER_AREA).astype(np.float32) + mask = cv2.resize(alpha, (w, h), interpolation=cv2.INTER_AREA).astype(np.float32) + if abs(angle) > 1e-6: + M = cv2.getRotationMatrix2D((w / 2.0, h / 2.0), angle, 1.0) + cos, sin = abs(float(M[0, 0])), abs(float(M[0, 1])) + nw, nh = int(h * sin + w * cos) + 1, int(h * cos + w * sin) + 1 + M[0, 2] += nw / 2.0 - w / 2.0 + M[1, 2] += nh / 2.0 - h / 2.0 + tpl = cv2.warpAffine(tpl, M, (nw, nh), flags=cv2.INTER_LINEAR, + borderValue=(0, 0, 0)).astype(np.float32) + mask = cv2.warpAffine(mask, M, (nw, nh), flags=cv2.INTER_LINEAR, + borderValue=0).astype(np.float32) + return tpl, mask, int(tpl.shape[1]), int(tpl.shape[0]) + except cv2.error as e: + print(f" 模板变换失败: {e}") + return None, None, 0, 0 + + +def full_match(big_f, tpl, mask): + """整图蒙版 CCOEFF_NORMED。返回响应图或 None。""" + try: + res = cv2.matchTemplate(big_f, tpl, cv2.TM_CCOEFF_NORMED, mask=mask) + return np.nan_to_num(res, nan=-1.0, posinf=-1.0, neginf=-1.0).astype(np.float32) + except cv2.error as e: + print(f" 整图匹配失败: {e}") + return None + + +def topk_peaks(res, tw, th, k): + """NMS 取前 k 峰，返回 [(score, x, y)]。""" + peaks = [] + try: + r = res.copy() + for _ in range(k): + _, score, _, loc = cv2.minMaxLoc(r) + if score <= -1.0: + break + peaks.append((float(score), int(loc[0]), int(loc[1]))) + x0, y0 = max(0, loc[0] - int(tw * 0.7)), max(0, loc[1] - int(th * 0.7)) + x1, y1 = min(r.shape[1], loc[0] + int(tw * 0.7)), min(r.shape[0], loc[1] + int(th * 0.7)) + r[y0:y1, x0:x1] = -1.0 + except Exception as e: + print(f" 峰提取失败: {e}") + return peaks + + +def patch_ncc(big_f, cx, cy, tpl, mask): + """在以 (cx,cy) 为中心、与旋转后模板同尺寸的 patch 上算 NCC。失败返回 -1。""" + try: + h, w = tpl.shape[:2] + x0 = int(round(cx - w / 2.0)) + y0 = int(round(cy - h / 2.0)) + if x0 < 0 or y0 < 0 or x0 + w > big_f.shape[1] or y0 + h > big_f.shape[0]: + return -1.0 + patch = big_f[y0:y0 + h, x0:x0 + w] + res = cv2.matchTemplate(patch, tpl, cv2.TM_CCOEFF_NORMED, mask=mask) + v = float(np.nan_to_num(res[0, 0], nan=-1.0)) + return v + except cv2.error: + return -1.0 + + +def rotation_curve(big_f, cx, cy, bgr, alpha, scale, angles): + """候选位置的旋转响应曲线。返回 (best_angle, best_ncc, ncc_at_0, samples)。""" + samples = [] + try: + for ang in angles: + tpl, mask, _, _ = template_at(bgr, alpha, scale, float(ang)) + if tpl is None: + samples.append(-1.0) + continue + samples.append(patch_ncc(big_f, cx, cy, tpl, mask)) + arr = np.array(samples) + k = int(arr.argmax()) + i0 = int(np.where(np.isclose(angles, 0.0))[0][0]) if any(np.isclose(angles, 0.0)) else -1 + ncc0 = float(arr[i0]) if i0 >= 0 else -1.0 + return float(angles[k]), float(arr[k]), ncc0, arr + except Exception as e: + print(f" 旋转曲线失败: {e}") + return 0.0, -1.0, -1.0, np.array(samples) + + +def process_pair(name, lines): + try: + big = cv2.imread(str(CAP / f"{name}.jpeg"), cv2.IMREAD_COLOR) + if big is None: + lines.append(f"{name[:12]}\tSKIP\t大图读取失败") + return + big_f = big.astype(np.float32) + bgr, alpha = load_mark(CAP / f"{name}-mark.png") + if bgr is None or alpha is None: + lines.append(f"{name[:12]}\tSKIP\tmark 读取失败") + return + + # 阶段1：rot=0 尺度扫描 + cands = [] # (prom, scale, w, h, x, y, score0) + for sc in SCALES: + tpl, mask, tw, th = template_at(bgr, alpha, float(sc), 0.0) + if tpl is None or tw >= big.shape[1] or th >= big.shape[0]: + continue + res = full_match(big_f, tpl, mask) + if res is None: + continue + mx, med, sd = float(res.max()), float(np.median(res)), float(res.std()) + prom = (mx - med) / sd if sd > 1e-6 else 0.0 + for score, x, y in topk_peaks(res, tw, th, PEAKS_PER_SCALE): + cands.append((prom, float(sc), tw, th, x, y, score)) + if not cands: + lines.append(f"{name[:12]}\tSKIP\t阶段1无候选") + return + + # 跨尺度 NMS：中心距离 < 0.6*模板尺寸 视为同点 + cands.sort(key=lambda c: (-c[0], -c[6])) + kept = [] + for c in cands: + cx, cy = c[4] + c[2] / 2.0, c[5] + c[3] / 2.0 + dup = False + for k in kept: + kcx, kcy = k[4] + k[2] / 2.0, k[5] + k[3] / 2.0 + if math.hypot(cx - kcx, cy - kcy) < 0.6 * max(c[2], k[2]): + dup = True + break + if not dup: + kept.append(c) + if len(kept) >= CAND_KEEP: + break + + # 阶段2：旋转消歧 + results = [] # (rot*, ncc*, ncc0, prom, scale, w, h, x, y) + for prom, sc, tw, th, x, y, score0 in kept: + cx, cy = x + tw / 2.0, y + th / 2.0 + rot_b, ncc_b, ncc0, _ = rotation_curve(big_f, cx, cy, bgr, alpha, sc, ANGLES) + results.append((rot_b, ncc_b, ncc0, prom, sc, tw, th, x, y)) + + ok = [r for r in results if abs(r[0]) <= ZERO_TOL and r[1] > 0.0] + ok.sort(key=lambda r: (-r[1], abs(r[0]))) + results.sort(key=lambda r: -r[1]) + if ok: + t = ok[0] + rot_b, ncc_b, ncc0, prom, sc, tw, th, x, y = t + print(f"{name[:12]} 目标: scale={sc:.2f} rot*={rot_b:+.0f}° " + f"ncc*={ncc_b:.3f} ncc(0°)={ncc0:.3f} offset=({x},{y}) " + f"prom={prom:.2f} | 合规候选={len(ok)}") + lines.append(f"{name[:12]}\t目标\tscale={sc:.2f}\trot*={rot_b:+.0f}\t" + f"ncc*={ncc_b:.3f}\tncc0={ncc0:.3f}\toffset=({x},{y})\t" + f"prom={prom:.2f}\t合规={len(ok)}") + else: + r = results[0] + print(f"{name[:12]} 无零旋转候选，最优旋转响应: scale={r[4]:.2f} " + f"rot*={r[0]:+.0f}° ncc*={r[1]:.3f}") + lines.append(f"{name[:12]}\t无零旋转候选\tbest_rot*={r[0]:+.0f}\t" + f"ncc*={r[1]:.3f}\tscale={r[4]:.2f}") + + # 可视化：目标框 + 旋转曲线图 + vis = big.copy() + for i, r in enumerate(results[:3]): + rot_b, ncc_b, _, _, sc, tw, th, x, y = r + color = (0, 0, 255) if (ok and r is ok[0]) else (255, 0, 0) + cv2.rectangle(vis, (x, y), (x + tw, y + th), color, 2) + cv2.putText(vis, f"s={sc:.2f} r={rot_b:.0f} {ncc_b:.2f}", + (x, max(12, y - 4)), cv2.FONT_HERSHEY_SIMPLEX, 0.4, color, 1) + cv2.imwrite(str(OUT / f"{name[:12]}-match.png"), vis) + if ok: + _, _, _, _, sc, _, _, _, _ = ok[0] + cx, cy = ok[0][7] + ok[0][5] / 2.0, ok[0][8] + ok[0][6] / 2.0 + _, _, _, curve = rotation_curve(big_f, cx, cy, bgr, alpha, sc, ANGLES) + img = np.full((160, 480, 3), 255, np.uint8) + cmax = max(float(curve.max()), 1e-3) + cmin = min(float(curve.min()), 0.0) + for j in range(1, len(curve)): + x0 = int((j - 1) / (len(curve) - 1) * 460) + 10 + x1 = int(j / (len(curve) - 1) * 460) + 10 + y0 = int(150 - (curve[j - 1] - cmin) / (cmax - cmin) * 130) + y1 = int(150 - (curve[j] - cmin) / (cmax - cmin) * 130) + cv2.line(img, (x0, y0), (x1, y1), (255, 0, 0), 1) + xi = int((0.0 - ANGLES[0]) / (ANGLES[-1] - ANGLES[0]) * 460) + 10 + cv2.line(img, (xi, 10), (xi, 150), (0, 200, 0), 1) + cv2.putText(img, f"{name[:12]} scale={sc:.2f}", (10, 14), + cv2.FONT_HERSHEY_SIMPLEX, 0.4, (0, 0, 0), 1) + cv2.imwrite(str(OUT / f"{name[:12]}-rot.png"), img) + except Exception as e: + print(f"{name[:12]} 处理失败: {e}") + lines.append(f"{name[:12]}\tERROR\t{e}") + + +def main(): + lines = [] + pairs = sorted({p.name.replace("-mark.png", "") for p in CAP.glob("*-mark.png")}) + for name in pairs: + process_pair(name, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + n_ok = sum(1 for l in lines if "\t目标\t" in l) + print(f"\n{n_ok}/{len(pairs)} 对定位到零旋转目标；结果已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/method_d_shape.py b/try/method_d_shape.py new file mode 100644 index 0000000..e9ad3e7 --- /dev/null +++ b/try/method_d_shape.py @@ -0,0 +1,172 @@ +"""方法D：基于轮廓/边缘的缩放旋转搜索。 + +mark 的 RGB 颜色可能与大图中的目标渲染颜色不同，因此不再匹配填充颜色， +而只使用 mark alpha 的外轮廓；背景使用 Canny 边缘图，采用距离变换做 +Chamfer matching。分数越高表示模板轮廓落在背景边缘上的平均距离越小。 + +输出：try/out/D/{*-match.png,summary.txt} +""" +from pathlib import Path +import math +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "D" +OUT.mkdir(parents=True, exist_ok=True) + +SCALES = np.arange(0.30, 1.51, 0.05) +ANGLES = np.arange(-180.0, 180.0, 15.0) +TOP_K = 5 + + +def files_for(prefix): + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None, None + big = bigs[0] + return big, CAP / f"{big.stem}-mark.png" + + +def load_shapes(mark_path, big_path): + try: + mark = cv2.imread(str(mark_path), cv2.IMREAD_UNCHANGED) + big = cv2.imread(str(big_path), cv2.IMREAD_GRAYSCALE) + if mark is None or big is None: + return None + alpha = mark[..., 3] if mark.ndim == 3 and mark.shape[2] == 4 else np.full(mark.shape[:2], 255, np.uint8) + shape = (alpha > 128).astype(np.uint8) * 255 + # 轮廓用细边界，避免大面积填充主导匹配 + edge = cv2.morphologyEx(shape, cv2.MORPH_GRADIENT, np.ones((3, 3), np.uint8)) + return big, shape, edge + except Exception as e: + print(f" 图像读取失败: {e}") + return None + + +def transformed_edge(edge, scale, angle): + try: + h0, w0 = edge.shape + w, h = max(12, round(w0 * scale)), max(12, round(h0 * scale)) + small = cv2.resize(edge, (w, h), interpolation=cv2.INTER_NEAREST) + if abs(angle) < 1e-6: + return small + m = cv2.getRotationMatrix2D((w / 2.0, h / 2.0), angle, 1.0) + cos, sin = abs(float(m[0, 0])), abs(float(m[0, 1])) + nw, nh = int(h * sin + w * cos) + 1, int(h * cos + w * sin) + 1 + m[0, 2] += nw / 2.0 - w / 2.0 + m[1, 2] += nh / 2.0 - h / 2.0 + return cv2.warpAffine(small, m, (nw, nh), flags=cv2.INTER_NEAREST, borderValue=0) + except cv2.error: + return None + + +def chamfer_response(bg_edges, tpl_edge): + """边缘模板的 Chamfer 响应图，值越大越好。""" + try: + h, w = tpl_edge.shape + if h >= bg_edges.shape[0] or w >= bg_edges.shape[1]: + return None + points = (tpl_edge > 0).astype(np.float32) + n_points = float(points.sum()) + if n_points < 10: + return None + # 用模板边缘作为权重，计算背景边缘距离的局部平均值。 + distance = cv2.distanceTransform((255 - bg_edges).astype(np.uint8), cv2.DIST_L2, 3) + cost = cv2.matchTemplate(distance.astype(np.float32), points, + cv2.TM_CCORR) + avg_distance = cost / n_points + return 1.0 / (1.0 + np.maximum(avg_distance, 0.0)) + except cv2.error: + return None + + +def peaks(response, tw, th): + out = [] + try: + r = response.copy() + for _ in range(TOP_K): + _, score, _, loc = cv2.minMaxLoc(r) + if score <= 0: + break + out.append((float(score), int(loc[0]), int(loc[1]))) + x0 = max(0, loc[0] - int(tw * 0.65)) + y0 = max(0, loc[1] - int(th * 0.65)) + x1 = min(r.shape[1], loc[0] + int(tw * 0.65)) + y1 = min(r.shape[0], loc[1] + int(th * 0.65)) + r[y0:y1, x0:x1] = 0 + except Exception as e: + print(f" 峰提取失败: {e}") + return out + + +def process(prefix, lines): + try: + big_path, mark_path = files_for(prefix) + if big_path is None: + lines.append(f"{prefix}\tSKIP\t文件缺失") + return + loaded = load_shapes(mark_path, big_path) + if loaded is None: + lines.append(f"{prefix}\tSKIP\t读取失败") + return + big, _, edge = loaded + bg_edges = cv2.Canny(big, 50, 150) + candidates = [] + for scale in SCALES: + for angle in ANGLES: + tpl = transformed_edge(edge, float(scale), float(angle)) + if tpl is None: + continue + response = chamfer_response(bg_edges, tpl) + if response is None: + continue + for score, x, y in peaks(response, tpl.shape[1], tpl.shape[0]): + candidates.append((score, float(scale), float(angle), x, y, + tpl.shape[1], tpl.shape[0])) + if not candidates: + lines.append(f"{prefix}\tSKIP\t无候选") + return + # 跨尺度/角度 NMS + candidates.sort(reverse=True) + kept = [] + for item in candidates: + score, scale, angle, x, y, w, h = item + cx, cy = x + w / 2, y + h / 2 + if all(math.hypot(cx - (q[3] + q[5] / 2), cy - (q[4] + q[6] / 2)) > 0.6 * max(w, h) + for q in kept): + kept.append(item) + if len(kept) >= TOP_K: + break + best = kept[0] + score, scale, angle, x, y, w, h = best + print(f"{prefix} best score={score:.3f} scale={scale:.2f} angle={angle:+.0f} " + f"offset=({x},{y}) top={len(kept)}") + lines.append(f"{prefix}\tscore={score:.3f}\tscale={scale:.2f}\tangle={angle:+.0f}\t" + f"offset=({x},{y})\ttop={len(kept)}") + vis = cv2.cvtColor(big, cv2.COLOR_GRAY2BGR) + colors = [(0, 0, 255), (0, 165, 255), (255, 0, 0), (0, 255, 0), (255, 0, 255)] + for i, item in enumerate(kept): + _, _, _, px, py, pw, ph = item + cv2.rectangle(vis, (px, py), (px + pw, py + ph), colors[i], 2) + cv2.putText(vis, str(i + 1), (px + 2, py + 16), cv2.FONT_HERSHEY_SIMPLEX, + 0.55, colors[i], 2) + cv2.imwrite(str(OUT / f"{prefix}-match.png"), vis) + except Exception as e: + print(f"{prefix} 处理失败: {e}") + lines.append(f"{prefix}\tERROR\t{e}") + + +def main(): + lines = [] + prefixes = sorted({p.name.split("~", 1)[0] for p in CAP.glob("*.jpeg") + if "-mark" not in p.stem}) + for prefix in prefixes: + process(prefix, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\n结果已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/method_e_shape_coverage.py b/try/method_e_shape_coverage.py new file mode 100644 index 0000000..d795b22 --- /dev/null +++ b/try/method_e_shape_coverage.py @@ -0,0 +1,168 @@ +"""方法E：轮廓边缘覆盖率匹配。 + +与方法D不同，不直接比较边缘距离的原始平均值（小模板容易偶然得高分）， +而统计模板轮廓点落在背景边缘容差范围内的比例，并以平均距离作为次级指标。 +候选按 (coverage, -mean_distance) 排序。 + +输出：try/out/E/{*-match.png,summary.txt} +""" +from pathlib import Path +import math +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "E" +OUT.mkdir(parents=True, exist_ok=True) + +SCALES = np.arange(0.40, 1.31, 0.05) +ANGLES = np.arange(-180.0, 180.0, 15.0) +TOP_K = 3 +EDGE_TOL = 2.5 + + +def files_for(prefix): + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None, None + big = bigs[0] + return big, CAP / f"{big.stem}-mark.png" + + +def load_shape(mark_path, big_path): + try: + mark = cv2.imread(str(mark_path), cv2.IMREAD_UNCHANGED) + big = cv2.imread(str(big_path), cv2.IMREAD_GRAYSCALE) + if mark is None or big is None: + return None + alpha = mark[..., 3] if mark.ndim == 3 and mark.shape[2] == 4 else np.full(mark.shape[:2], 255, np.uint8) + mask = (alpha > 128).astype(np.uint8) * 255 + edge = cv2.morphologyEx(mask, cv2.MORPH_GRADIENT, np.ones((3, 3), np.uint8)) + return big, edge + except Exception as e: + print(f" 读取失败: {e}") + return None + + +def transform_edge(edge, scale, angle): + try: + h0, w0 = edge.shape + w, h = max(16, round(w0 * scale)), max(16, round(h0 * scale)) + small = cv2.resize(edge, (w, h), interpolation=cv2.INTER_NEAREST) + if abs(angle) < 1e-6: + return small + m = cv2.getRotationMatrix2D((w / 2.0, h / 2.0), angle, 1.0) + c, s = abs(float(m[0, 0])), abs(float(m[0, 1])) + nw, nh = int(h * s + w * c) + 1, int(h * c + w * s) + 1 + m[0, 2] += nw / 2.0 - w / 2.0 + m[1, 2] += nh / 2.0 - h / 2.0 + return cv2.warpAffine(small, m, (nw, nh), flags=cv2.INTER_NEAREST, borderValue=0) + except cv2.error: + return None + + +def response_maps(bg_edges): + try: + dist = cv2.distanceTransform((255 - bg_edges).astype(np.uint8), cv2.DIST_L2, 3) + near = (dist <= EDGE_TOL).astype(np.float32) + return dist.astype(np.float32), near + except cv2.error: + return None, None + + +def candidates_for_template(dist, near, tpl): + try: + points = (tpl > 0).astype(np.float32) + n = float(points.sum()) + h, w = tpl.shape + if n < 20 or h >= dist.shape[0] or w >= dist.shape[1]: + return [] + hit = cv2.matchTemplate(near, points, cv2.TM_CCORR) / n + cost = cv2.matchTemplate(dist, points, cv2.TM_CCORR) / n + out = [] + for _ in range(TOP_K): + _, cov, _, loc = cv2.minMaxLoc(hit) + mean_dist = float(cost[loc[1], loc[0]]) + if cov <= 0: + break + out.append((float(cov), -mean_dist, int(loc[0]), int(loc[1]))) + x0 = max(0, loc[0] - int(w * 0.7)) + y0 = max(0, loc[1] - int(h * 0.7)) + x1 = min(hit.shape[1], loc[0] + int(w * 0.7)) + y1 = min(hit.shape[0], loc[1] + int(h * 0.7)) + hit[y0:y1, x0:x1] = 0 + return out + except cv2.error: + return [] + + +def process(prefix, lines): + try: + big_path, mark_path = files_for(prefix) + if big_path is None: + lines.append(f"{prefix}\tSKIP\t文件缺失") + return + loaded = load_shape(mark_path, big_path) + if loaded is None: + lines.append(f"{prefix}\tSKIP\t读取失败") + return + big, edge = loaded + bg_edges = cv2.Canny(big, 50, 150) + dist, near = response_maps(bg_edges) + if dist is None or near is None: + lines.append(f"{prefix}\tSKIP\t无法生成距离图") + return + all_candidates = [] + for scale in SCALES: + for angle in ANGLES: + tpl = transform_edge(edge, float(scale), float(angle)) + if tpl is None: + continue + for cov, neg_dist, x, y in candidates_for_template(dist, near, tpl): + all_candidates.append((cov, neg_dist, float(scale), float(angle), x, y, + tpl.shape[1], tpl.shape[0])) + if not all_candidates: + lines.append(f"{prefix}\tSKIP\t无候选") + return + all_candidates.sort(key=lambda x: (-x[0], -x[1])) + kept = [] + for item in all_candidates: + cov, neg_dist, scale, angle, x, y, w, h = item + cx, cy = x + w / 2.0, y + h / 2.0 + if all(math.hypot(cx - (q[4] + q[6] / 2), cy - (q[5] + q[7] / 2)) > 0.6 * max(w, h) + for q in kept): + kept.append(item) + if len(kept) >= TOP_K: + break + best = kept[0] + cov, neg_dist, scale, angle, x, y, w, h = best + print(f"{prefix} coverage={cov:.3f} meanDist={-neg_dist:.2f} " + f"scale={scale:.2f} angle={angle:+.0f} offset=({x},{y})") + lines.append(f"{prefix}\tcoverage={cov:.3f}\tmeanDist={-neg_dist:.2f}\t" + f"scale={scale:.2f}\tangle={angle:+.0f}\toffset=({x},{y})") + vis = cv2.cvtColor(big, cv2.COLOR_GRAY2BGR) + colors = [(0, 0, 255), (0, 165, 255), (255, 0, 0)] + for i, item in enumerate(kept): + _, _, _, _, px, py, pw, ph = item + cv2.rectangle(vis, (px, py), (px + pw, py + ph), colors[i], 2) + cv2.putText(vis, str(i + 1), (px + 2, py + 16), cv2.FONT_HERSHEY_SIMPLEX, + 0.55, colors[i], 2) + cv2.imwrite(str(OUT / f"{prefix}-match.png"), vis) + except Exception as e: + print(f"{prefix} 处理失败: {e}") + lines.append(f"{prefix}\tERROR\t{e}") + + +def main(): + lines = [] + prefixes = sorted({p.name.split("~", 1)[0] for p in CAP.glob("*.jpeg") + if "-mark" not in p.stem}) + for prefix in prefixes: + process(prefix, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\n结果已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/method_f_contour.py b/try/method_f_contour.py new file mode 100644 index 0000000..298bf88 --- /dev/null +++ b/try/method_f_contour.py @@ -0,0 +1,140 @@ +"""方法F：轮廓候选匹配。 + +适用于 mark 的 RGB 颜色与大图目标不同、但 alpha 轮廓保持一致的样本。 +流程： +1. 从大图生成边缘并提取外部轮廓； +2. 对 mark alpha 轮廓做缩放/旋转模板； +3. 用轮廓形状相似度 + 面积/边界覆盖率筛选候选。 + +输出：try/out/F/{*-match.png,summary.txt} +""" +from pathlib import Path +import math +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "F" +OUT.mkdir(parents=True, exist_ok=True) + +SCALES = np.arange(0.35, 1.31, 0.05) +ANGLES = np.arange(-180.0, 180.0, 15.0) +TOP_K = 5 + + +def files_for(prefix): + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None, None + big = bigs[0] + return big, CAP / f"{big.stem}-mark.png" + + +def read_shapes(mark_path, big_path): + try: + mark = cv2.imread(str(mark_path), cv2.IMREAD_UNCHANGED) + big = cv2.imread(str(big_path), cv2.IMREAD_COLOR) + if mark is None or big is None: + return None + alpha = mark[..., 3] if mark.ndim == 3 and mark.shape[2] == 4 else np.full(mark.shape[:2], 255, np.uint8) + mask = (alpha > 128).astype(np.uint8) * 255 + contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) + if not contours: + return None + mark_contour = max(contours, key=cv2.contourArea) + gray = cv2.cvtColor(big, cv2.COLOR_BGR2GRAY) + # 目标通常与局部背景有明显亮度差；保留边缘并闭合轮廓。 + edges = cv2.Canny(gray, 40, 120) + edges = cv2.morphologyEx(edges, cv2.MORPH_CLOSE, np.ones((3, 3), np.uint8), iterations=2) + return big, mark_contour, edges + except Exception as e: + print(f" 读取失败: {e}") + return None + + +def transform_contour(contour, scale, angle): + try: + pts = contour.reshape(-1, 2).astype(np.float32) + center = pts.mean(axis=0) + pts = (pts - center) * scale + theta = math.radians(angle) + rot = np.array([[math.cos(theta), -math.sin(theta)], + [math.sin(theta), math.cos(theta)]], dtype=np.float32) + return (pts @ rot.T).astype(np.float32) + except Exception: + return None + + +def contour_score(mark_contour, candidate): + try: + # matchShapes 对平移/尺度基本不敏感，旋转也较稳定；用于形状初筛。 + return float(cv2.matchShapes(mark_contour, candidate, cv2.CONTOURS_MATCH_I1, 0.0)) + except cv2.error: + return 1e9 + + +def process(prefix, lines): + try: + big_path, mark_path = files_for(prefix) + if big_path is None: + lines.append(f"{prefix}\tSKIP\t文件缺失") + return + loaded = read_shapes(mark_path, big_path) + if loaded is None: + lines.append(f"{prefix}\tSKIP\t读取失败") + return + big, mark_contour, edges = loaded + contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) + mark_area = max(cv2.contourArea(mark_contour), 1.0) + candidates = [] + for contour in contours: + area = cv2.contourArea(contour) + if area < mark_area * 0.08 or area > mark_area * 20: + continue + score = contour_score(mark_contour, contour) + x, y, w, h = cv2.boundingRect(contour) + candidates.append((score, area, x, y, w, h)) + candidates.sort(key=lambda item: item[0]) + kept = [] + for item in candidates: + _, _, x, y, w, h = item + cx, cy = x + w / 2.0, y + h / 2.0 + if all(math.hypot(cx - (q[2] + q[4] / 2), cy - (q[3] + q[5] / 2)) > 0.5 * max(w, h) + for q in kept): + kept.append(item) + if len(kept) >= TOP_K: + break + if not kept: + lines.append(f"{prefix}\tSKIP\t无轮廓候选") + return + best = kept[0] + score, area, x, y, w, h = best + print(f"{prefix} shape={score:.4f} area={area:.0f} offset=({x},{y}) size=({w},{h}) top={len(kept)}") + lines.append(f"{prefix}\tshape={score:.4f}\tarea={area:.0f}\toffset=({x},{y})\t" + f"size=({w},{h})\ttop={len(kept)}") + vis = big.copy() + for i, item in enumerate(kept): + _, _, px, py, pw, ph = item + color = (0, 0, 255) if i == 0 else (255, 0, 0) + cv2.rectangle(vis, (px, py), (px + pw, py + ph), color, 2) + cv2.putText(vis, str(i + 1), (px + 2, py + 16), cv2.FONT_HERSHEY_SIMPLEX, + 0.55, color, 2) + cv2.imwrite(str(OUT / f"{prefix}-match.png"), vis) + except Exception as e: + print(f"{prefix} 处理失败: {e}") + lines.append(f"{prefix}\tERROR\t{e}") + + +def main(): + lines = [] + prefixes = sorted({p.name.split("~", 1)[0] for p in CAP.glob("*.jpeg") + if "-mark" not in p.stem}) + for prefix in prefixes: + process(prefix, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\n结果已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/method_g_component.py b/try/method_g_component.py new file mode 100644 index 0000000..5ab3b43 --- /dev/null +++ b/try/method_g_component.py @@ -0,0 +1,72 @@ +"""方法G：暗色图形的连通域 + alpha 轮廓形状匹配实验。""" +from pathlib import Path +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "G" +OUT.mkdir(parents=True, exist_ok=True) + + +def files_for(prefix): + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None, None + big = bigs[0] + return big, CAP / f"{big.stem}-mark.png" + + +def detect_components(gray): + """用局部暗目标检测提取候选组件。""" + blur = cv2.GaussianBlur(gray, (0, 0), 15) + dark = np.clip(blur.astype(np.int16) - gray.astype(np.int16), 0, 255).astype(np.uint8) + mask = (dark > 12).astype(np.uint8) * 255 + mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, np.ones((5, 5), np.uint8), iterations=2) + mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((3, 3), np.uint8)) + return mask + + +def process(prefix): + big_path, mark_path = files_for(prefix) + if big_path is None: + return + big = cv2.imread(str(big_path), cv2.IMREAD_COLOR) + mark = cv2.imread(str(mark_path), cv2.IMREAD_UNCHANGED) + if big is None or mark is None: + return + alpha = mark[..., 3] if mark.ndim == 3 and mark.shape[2] == 4 else np.full(mark.shape[:2], 255, np.uint8) + m = (alpha > 128).astype(np.uint8) * 255 + mc, _ = cv2.findContours(m, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) + mc = max(mc, key=cv2.contourArea) + ma = max(cv2.contourArea(mc), 1) + gray = cv2.cvtColor(big, cv2.COLOR_BGR2GRAY) + mask = detect_components(gray) + contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) + vis = big.copy() + found = [] + for c in contours: + area = cv2.contourArea(c) + x, y, w, h = cv2.boundingRect(c) + if area < ma * 0.08 or area > ma * 3 or min(w, h) < 20 or max(w, h) > 180: + continue + score = cv2.matchShapes(mc, c, cv2.CONTOURS_MATCH_I1, 0) + found.append((score, area, x, y, w, h)) + found.sort(key=lambda z: z[0]) + for i, (_, _, x, y, w, h) in enumerate(found[:5]): + color = (0, 0, 255) if i == 0 else (255, 0, 0) + cv2.rectangle(vis, (x, y), (x + w, y + h), color, 2) + cv2.putText(vis, f"{i+1}", (x, max(14, y - 3)), cv2.FONT_HERSHEY_SIMPLEX, .5, color, 2) + cv2.imwrite(str(OUT / f"{prefix}-components.png"), vis) + line = " ".join(f"#{i+1}:{s:.3f}@({x},{y},{w},{h})" for i,(s,_,x,y,w,h) in enumerate(found[:5])) + print(prefix, line or "无候选") + + +def main(): + prefixes = sorted({p.name.split("~", 1)[0] for p in CAP.glob("*.jpeg") if "-mark" not in p.stem}) + for prefix in prefixes: + process(prefix) + + +if __name__ == "__main__": + main() diff --git a/try/method_h_silhouette.py b/try/method_h_silhouette.py new file mode 100644 index 0000000..e8d5b6b --- /dev/null +++ b/try/method_h_silhouette.py @@ -0,0 +1,191 @@ +"""方法H：受尺度约束的 alpha 轮廓 + 局部对比度匹配。 + +前面的 C/D/E 实验暴露出两个系统性误差： +- 小模板会被背景纹理误选； +- mark 的 RGB 颜色不是大图目标的真实颜色。 + +本方法只使用 mark 的 alpha 轮廓，并把尺度限制在 0.75~1.25（样本中目标 +物体与 110px mark 近似同尺寸）。候选分数由两部分组成： +1. alpha 轮廓在大图边缘距离图上的贴合度； +2. alpha 内部与外部环带之间的局部对比度。 + +输出：try/out/H/{*-match.png,summary.txt} +""" +from pathlib import Path +import math +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "H" +OUT.mkdir(parents=True, exist_ok=True) + +SCALES = np.arange(0.75, 1.26, 0.025) +TOP_K = 5 +RING = 5 + + +def files_for(prefix): + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None, None + big = bigs[0] + return big, CAP / f"{big.stem}-mark.png" + + +def load_data(big_path, mark_path): + try: + big = cv2.imread(str(big_path), cv2.IMREAD_GRAYSCALE) + mark = cv2.imread(str(mark_path), cv2.IMREAD_UNCHANGED) + if big is None or mark is None: + return None + alpha = mark[..., 3] if mark.ndim == 3 and mark.shape[2] == 4 else np.full(mark.shape[:2], 255, np.uint8) + shape = (alpha > 128).astype(np.uint8) + edge = cv2.morphologyEx(shape * 255, cv2.MORPH_GRADIENT, np.ones((3, 3), np.uint8)) > 0 + outer = cv2.dilate(shape, np.ones((RING * 2 + 1, RING * 2 + 1), np.uint8)) > 0 + ring = outer & ~shape + return big, shape, edge, ring + except Exception as e: + print(f" 读取失败: {e}") + return None + + +def resize_binary(mask, scale): + try: + h, w = mask.shape + size = (max(12, round(w * scale)), max(12, round(h * scale))) + return cv2.resize(mask.astype(np.uint8), size, interpolation=cv2.INTER_NEAREST) > 0 + except cv2.error: + return None + + +def distance_fit(distance, edge): + try: + pts = edge.astype(np.float32) + n = float(pts.sum()) + h, w = edge.shape + if n < 10 or h >= distance.shape[0] or w >= distance.shape[1]: + return None + # 每个候选框内，模板边缘点到背景边缘的平均距离 + cost = cv2.matchTemplate(distance.astype(np.float32), pts, cv2.TM_CCORR) / n + return 1.0 / (1.0 + cost) + except cv2.error: + return None + + +def contrast_map(gray, shape, ring): + try: + inside = shape.astype(np.float32) + outside = ring.astype(np.float32) + ni, no = float(inside.sum()), float(outside.sum()) + if ni < 10 or no < 10: + return None + mean_i = cv2.matchTemplate(gray.astype(np.float32), inside, cv2.TM_CCORR) / ni + mean_o = cv2.matchTemplate(gray.astype(np.float32), outside, cv2.TM_CCORR) / no + return np.abs(mean_i - mean_o) + except cv2.error: + return None + + +def normalize_map(values): + try: + lo, hi = np.percentile(values, (10, 99)) + if hi <= lo: + return np.zeros_like(values, dtype=np.float32) + return np.clip((values - lo) / (hi - lo), 0.0, 1.0).astype(np.float32) + except Exception: + return np.zeros_like(values, dtype=np.float32) + + +def peaks(score_map, w, h): + result = [] + try: + r = score_map.copy() + for _ in range(TOP_K): + _, score, _, loc = cv2.minMaxLoc(r) + if score <= 0: + break + result.append((float(score), int(loc[0]), int(loc[1]))) + x0 = max(0, loc[0] - int(w * 0.65)) + y0 = max(0, loc[1] - int(h * 0.65)) + x1 = min(r.shape[1], loc[0] + int(w * 0.65)) + y1 = min(r.shape[0], loc[1] + int(h * 0.65)) + r[y0:y1, x0:x1] = 0 + except Exception as e: + print(f" 峰提取失败: {e}") + return result + + +def process(prefix, lines): + try: + big_path, mark_path = files_for(prefix) + if big_path is None: + lines.append(f"{prefix}\tSKIP\t文件缺失") + return + loaded = load_data(big_path, mark_path) + if loaded is None: + lines.append(f"{prefix}\tSKIP\t读取失败") + return + big, shape0, edge0, ring0 = loaded + bg_edges = cv2.Canny(big, 80, 180) + distance = cv2.distanceTransform((255 - bg_edges).astype(np.uint8), cv2.DIST_L2, 3) + candidates = [] + for scale in SCALES: + shape = resize_binary(shape0, float(scale)) + edge = resize_binary(edge0, float(scale)) + ring = resize_binary(ring0, float(scale)) + if shape is None or edge is None or ring is None: + continue + fit = distance_fit(distance, edge) + contrast = contrast_map(big, shape, ring) + if fit is None or contrast is None or fit.shape != contrast.shape: + continue + score_map = 0.55 * normalize_map(fit) + 0.45 * normalize_map(contrast) + for score, x, y in peaks(score_map, shape.shape[1], shape.shape[0]): + candidates.append((score, float(scale), x, y, shape.shape[1], shape.shape[0], + float(fit[y, x]), float(contrast[y, x]))) + if not candidates: + lines.append(f"{prefix}\tSKIP\t无候选") + return + candidates.sort(key=lambda c: -c[0]) + kept = [] + for c in candidates: + _, _, x, y, w, h, _, _ = c + cx, cy = x + w / 2.0, y + h / 2.0 + if all(math.hypot(cx - (q[2] + q[4] / 2), cy - (q[3] + q[5] / 2)) > 0.6 * max(w, h) + for q in kept): + kept.append(c) + if len(kept) >= TOP_K: + break + best = kept[0] + score, scale, x, y, w, h, fit_score, contrast_score = best + print(f"{prefix} score={score:.3f} scale={scale:.3f} offset=({x},{y}) " + f"edgeFit={fit_score:.2f} contrast={contrast_score:.2f}") + lines.append(f"{prefix}\tscore={score:.3f}\tscale={scale:.3f}\toffset=({x},{y})\t" + f"edgeFit={fit_score:.2f}\tcontrast={contrast_score:.2f}") + vis = cv2.cvtColor(big, cv2.COLOR_GRAY2BGR) + colors = [(0, 0, 255), (0, 165, 255), (255, 0, 0), (0, 255, 0), (255, 0, 255)] + for i, item in enumerate(kept): + _, _, px, py, pw, ph, _, _ = item + cv2.rectangle(vis, (px, py), (px + pw, py + ph), colors[i], 2) + cv2.putText(vis, str(i + 1), (px + 2, py + 16), cv2.FONT_HERSHEY_SIMPLEX, + 0.55, colors[i], 2) + cv2.imwrite(str(OUT / f"{prefix}-match.png"), vis) + except Exception as e: + print(f"{prefix} 处理失败: {e}") + lines.append(f"{prefix}\tERROR\t{e}") + + +def main(): + lines = [] + prefixes = sorted({p.name.split("~", 1)[0] for p in CAP.glob("*.jpeg") + if "-mark" not in p.stem}) + for prefix in prefixes: + process(prefix, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\n结果已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/method_i_adaptive_components.py b/try/method_i_adaptive_components.py new file mode 100644 index 0000000..fbb3163 --- /dev/null +++ b/try/method_i_adaptive_components.py @@ -0,0 +1,152 @@ +"""方法I：背景自适应的目标连通域检测 + alpha 轮廓评分。 + +与 G 的固定暗度阈值不同：对每张图扫描多个局部对比度阈值，收集尺寸合理的 +暗色/高对比度连通域；通过候选轮廓的形状、填充率和局部边缘完整度综合评分。 +该方法的目标是先得到可解释的物体级候选，而不是在整张照片纹理上做 NCC。 + +输出：try/out/I/{*-match.png,summary.txt} +""" +from pathlib import Path +import math +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "I" +OUT.mkdir(parents=True, exist_ok=True) + +THRESHOLDS = (10, 15, 20, 25, 30, 40, 50, 60) +MIN_SIZE, MAX_SIZE = 25, 180 +TOP_K = 5 + + +def files_for(prefix): + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None, None + big = bigs[0] + return big, CAP / f"{big.stem}-mark.png" + + +def mark_info(path): + try: + mark = cv2.imread(str(path), cv2.IMREAD_UNCHANGED) + if mark is None: + return None + alpha = mark[..., 3] if mark.ndim == 3 and mark.shape[2] == 4 else np.full(mark.shape[:2], 255, np.uint8) + mask = (alpha > 128).astype(np.uint8) + cs, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) + if not cs: + return None + c = max(cs, key=cv2.contourArea) + x, y, w, h = cv2.boundingRect(c) + return mask, c, max(float(cv2.contourArea(c)), 1.0), (x, y, w, h) + except Exception as e: + print(f" mark 读取失败: {e}") + return None + + +def candidate_mask(gray, threshold): + try: + blur = cv2.GaussianBlur(gray, (0, 0), 21) + delta = np.clip(blur.astype(np.int16) - gray.astype(np.int16), 0, 255).astype(np.uint8) + m = (delta > threshold).astype(np.uint8) * 255 + m = cv2.morphologyEx(m, cv2.MORPH_CLOSE, np.ones((7, 7), np.uint8), iterations=2) + return cv2.morphologyEx(m, cv2.MORPH_OPEN, np.ones((3, 3), np.uint8)) + except cv2.error: + return None + + +def overlap(a, b): + try: + x1, y1, w1, h1 = a + x2, y2, w2, h2 = b + ix = max(0, min(x1 + w1, x2 + w2) - max(x1, x2)) + iy = max(0, min(y1 + h1, y2 + h2) - max(y1, y2)) + inter = ix * iy + union = w1 * h1 + w2 * h2 - inter + return inter / union if union else 0.0 + except Exception: + return 0.0 + + +def score_contour(mark_c, mark_area, contour, gray): + try: + x, y, w, h = cv2.boundingRect(contour) + area = float(cv2.contourArea(contour)) + shape_score = float(cv2.matchShapes(mark_c, contour, cv2.CONTOURS_MATCH_I1, 0.0)) + perimeter = max(float(cv2.arcLength(contour, True)), 1.0) + compact = min(1.0, 4.0 * math.pi * area / (perimeter * perimeter)) + contrast = float(gray[y:y + h, x:x + w].std()) + # 越小越好；外部排序转换为负分 + return shape_score, area, compact, contrast, (x, y, w, h) + except cv2.error: + return None + + +def process(prefix, lines): + try: + big_path, mark_path = files_for(prefix) + if big_path is None: + return + big = cv2.imread(str(big_path), cv2.IMREAD_COLOR) + info = mark_info(mark_path) + if big is None or info is None: + lines.append(f"{prefix}\tSKIP\t读取失败") + return + _, mark_c, mark_area, mark_box = info + gray = cv2.cvtColor(big, cv2.COLOR_BGR2GRAY) + candidates = [] + for threshold in THRESHOLDS: + mask = candidate_mask(gray, threshold) + if mask is None: + continue + contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) + for contour in contours: + x, y, w, h = cv2.boundingRect(contour) + area = cv2.contourArea(contour) + if min(w, h) < MIN_SIZE or max(w, h) > MAX_SIZE or area < mark_area * 0.05: + continue + item = score_contour(mark_c, mark_area, contour, gray) + if item is None: + continue + shape_score, area, compact, contrast, box = item + if any(overlap(box, old[5]) > 0.6 for old in candidates): + continue + candidates.append((shape_score, -compact, -contrast, threshold, area, box)) + candidates.sort(key=lambda z: (z[0], z[1], z[2])) + candidates = candidates[:TOP_K] + if not candidates: + lines.append(f"{prefix}\tSKIP\t无候选") + return + best = candidates[0] + shape_score, neg_compact, neg_contrast, threshold, area, (x, y, w, h) = best + print(f"{prefix} shape={shape_score:.4f} compact={-neg_compact:.3f} " + f"contrast={-neg_contrast:.1f} threshold={threshold} offset=({x},{y}) size=({w},{h})") + lines.append(f"{prefix}\tshape={shape_score:.4f}\tcompact={-neg_compact:.3f}\t" + f"contrast={-neg_contrast:.1f}\tthreshold={threshold}\toffset=({x},{y})\t" + f"size=({w},{h})") + vis = big.copy() + for i, item in enumerate(candidates): + _, _, _, _, _, (px, py, pw, ph) = item + color = (0, 0, 255) if i == 0 else (255, 0, 0) + cv2.rectangle(vis, (px, py), (px + pw, py + ph), color, 2) + cv2.putText(vis, str(i + 1), (px + 2, py + 16), cv2.FONT_HERSHEY_SIMPLEX, .55, color, 2) + cv2.imwrite(str(OUT / f"{prefix}-match.png"), vis) + except Exception as e: + print(f"{prefix} 处理失败: {e}") + lines.append(f"{prefix}\tERROR\t{e}") + + +def main(): + lines = [] + prefixes = sorted({p.name.split("~", 1)[0] for p in CAP.glob("*.jpeg") if "-mark" not in p.stem}) + for prefix in prefixes: + process(prefix, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\n结果已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/method_j_objects.py b/try/method_j_objects.py new file mode 100644 index 0000000..d999723 --- /dev/null +++ b/try/method_j_objects.py @@ -0,0 +1,159 @@ +"""方法J：低饱和度暗色几何对象检测 + alpha 轮廓匹配。 + +样本中的 mark 不是用于 RGB 像素复制，而是一个透明几何图形；背景中对应 +对象通常是低饱和度、较暗的半透明图形。先用多组局部暗度/饱和度阈值提取 +物体级连通区域，再用 mark 的 alpha 外轮廓进行形状筛选。 +""" +from pathlib import Path +import math +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "J" +OUT.mkdir(parents=True, exist_ok=True) + +DARK_THRESHOLDS = (8, 12, 16, 20, 25, 30, 40, 50) +SAT_THRESHOLDS = (50, 70, 90, 110, 140) +TOP_K = 5 +MIN_SIDE, MAX_SIDE = 25, 220 + + +def files_for(prefix): + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None, None + big = bigs[0] + return big, CAP / f"{big.stem}-mark.png" + + +def mark_data(path): + try: + mark = cv2.imread(str(path), cv2.IMREAD_UNCHANGED) + if mark is None: + return None + alpha = mark[..., 3] if mark.ndim == 3 and mark.shape[2] == 4 else np.full(mark.shape[:2], 255, np.uint8) + mask = (alpha > 128).astype(np.uint8) * 255 + contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) + if not contours: + return None + contour = max(contours, key=cv2.contourArea) + x, y, w, h = cv2.boundingRect(contour) + area = max(float(cv2.contourArea(contour)), 1.0) + return contour, area, (x, y, w, h) + except Exception as e: + print(f" mark 读取失败: {e}") + return None + + +def segmentation_masks(bgr): + gray = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY) + hsv = cv2.cvtColor(bgr, cv2.COLOR_BGR2HSV) + local = cv2.GaussianBlur(gray, (0, 0), 21) + dark_delta = np.clip(local.astype(np.int16) - gray.astype(np.int16), 0, 255).astype(np.uint8) + for dt in DARK_THRESHOLDS: + for st in SAT_THRESHOLDS: + mask = ((dark_delta >= dt) & (hsv[..., 1] <= st)).astype(np.uint8) * 255 + mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, np.ones((5, 5), np.uint8), iterations=2) + yield mask, dt, st + + +def contour_metrics(mark_contour, mark_area, contour, gray): + try: + x, y, w, h = cv2.boundingRect(contour) + if min(w, h) < MIN_SIDE or max(w, h) > MAX_SIDE: + return None + area = float(cv2.contourArea(contour)) + if area < mark_area * 0.04: + return None + shape = float(cv2.matchShapes(mark_contour, contour, cv2.CONTOURS_MATCH_I1, 0.0)) + perimeter = max(float(cv2.arcLength(contour, True)), 1.0) + compact = min(1.0, 4.0 * math.pi * area / perimeter**2) + hull = cv2.convexHull(contour) + hull_area = max(float(cv2.contourArea(hull)), 1.0) + solidity = min(1.0, area / hull_area) + aspect = min(w, h) / max(w, h) + # 几何图形一般轮廓完整、实心度较高；照片纹理多为细长碎片。 + if compact < 0.08 or solidity < 0.25: + return None + inside = gray[y:y + h, x:x + w] + contrast = float(inside.std()) if inside.size else 0.0 + return shape, -compact, -solidity, -aspect, -contrast, (x, y, w, h) + except cv2.error: + return None + + +def overlap(box1, box2): + try: + x1, y1, w1, h1 = box1 + x2, y2, w2, h2 = box2 + ix = max(0, min(x1 + w1, x2 + w2) - max(x1, x2)) + iy = max(0, min(y1 + h1, y2 + h2) - max(y1, y2)) + inter = ix * iy + union = w1 * h1 + w2 * h2 - inter + return inter / union if union else 0.0 + except Exception: + return 0.0 + + +def process(prefix, lines): + try: + big_path, mark_path = files_for(prefix) + if big_path is None: + return + bgr = cv2.imread(str(big_path), cv2.IMREAD_COLOR) + md = mark_data(mark_path) + if bgr is None or md is None: + lines.append(f"{prefix}\tSKIP\t读取失败") + return + mark_contour, mark_area, _ = md + gray = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY) + candidates = [] + for mask, dt, st in segmentation_masks(bgr): + contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) + for contour in contours: + metric = contour_metrics(mark_contour, mark_area, contour, gray) + if metric is None: + continue + shape, neg_compact, neg_solidity, neg_aspect, neg_contrast, box = metric + if any(overlap(box, old[6]) > 0.6 for old in candidates): + continue + candidates.append((shape, neg_compact, neg_solidity, neg_aspect, + neg_contrast, (dt, st), box)) + candidates.sort(key=lambda c: (c[0], c[1], c[2], c[3])) + candidates = candidates[:TOP_K] + if not candidates: + lines.append(f"{prefix}\tSKIP\t无候选") + return + best = candidates[0] + shape, neg_compact, neg_solidity, neg_aspect, neg_contrast, ts, box = best + x, y, w, h = box + print(f"{prefix} shape={shape:.4f} compact={-neg_compact:.3f} solidity={-neg_solidity:.3f} " + f"aspect={-neg_aspect:.3f} offset=({x},{y}) size=({w},{h}) threshold={ts}") + lines.append(f"{prefix}\tshape={shape:.4f}\tcompact={-neg_compact:.3f}\t" + f"solidity={-neg_solidity:.3f}\taspect={-neg_aspect:.3f}\t" + f"offset=({x},{y})\tsize=({w},{h})\tthreshold={ts}") + vis = bgr.copy() + for i, item in enumerate(candidates): + _, _, _, _, _, _, (px, py, pw, ph) = item + color = (0, 0, 255) if i == 0 else (255, 0, 0) + cv2.rectangle(vis, (px, py), (px + pw, py + ph), color, 2) + cv2.putText(vis, str(i + 1), (px + 2, py + 16), cv2.FONT_HERSHEY_SIMPLEX, .55, color, 2) + cv2.imwrite(str(OUT / f"{prefix}-match.png"), vis) + except Exception as e: + print(f"{prefix} 处理失败: {e}") + lines.append(f"{prefix}\tERROR\t{e}") + + +def main(): + lines = [] + prefixes = sorted({p.name.split("~", 1)[0] for p in CAP.glob("*.jpeg") if "-mark" not in p.stem}) + for prefix in prefixes: + process(prefix, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\n结果已写入 {OUT}/") + + +if __name__ == "__main__": + main() diff --git a/try/method_k_objectness.py b/try/method_k_objectness.py new file mode 100644 index 0000000..e9bd7ad --- /dev/null +++ b/try/method_k_objectness.py @@ -0,0 +1,170 @@ +"""方法K：alpha 形状与暗色覆盖物概率图匹配。 + +不直接使用 mark RGB。根据样本结构，背景目标通常比周围照片更暗、饱和度 +更低、局部纹理更均匀。构造暗色覆盖物概率图后，用 alpha silhouette 做 +多尺度匹配，并通过局部峰值抑制避免小纹理重复命中。 + +输出：try/out/K/{*-match.png,summary.txt} +""" +from pathlib import Path +import math +import cv2 +import numpy as np + +ROOT = Path(__file__).resolve().parent.parent +CAP = ROOT / "captchas" +OUT = ROOT / "try" / "out" / "K" +OUT.mkdir(parents=True, exist_ok=True) + +SCALES = np.arange(0.65, 1.36, 0.025) +DARK_SIGMAS = (15.0, 21.0, 31.0) +SAT_WEIGHTS = (0.25, 0.5, 0.75) +TOP_K = 5 + + +def pair_for(prefix): + bigs = [p for p in CAP.glob(f"{prefix}*.jpeg") if "-mark" not in p.stem] + if not bigs: + return None, None + big = bigs[0] + return big, CAP / f"{big.stem}-mark.png" + + +def read_pair(big_path, mark_path): + try: + big = cv2.imread(str(big_path), cv2.IMREAD_COLOR) + mark = cv2.imread(str(mark_path), cv2.IMREAD_UNCHANGED) + if big is None or mark is None: + return None + alpha = mark[..., 3] if mark.ndim == 3 and mark.shape[2] == 4 else np.full(mark.shape[:2], 255, np.uint8) + shape = (alpha > 128).astype(np.float32) + return big, shape + except Exception as e: + print(f" 读取失败: {e}") + return None + + +def objectness_maps(bgr): + """生成暗色覆盖物概率图的多个版本。""" + gray = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY).astype(np.float32) + hsv = cv2.cvtColor(bgr, cv2.COLOR_BGR2HSV).astype(np.float32) + sat = hsv[..., 1] / 255.0 + maps = [] + for sigma in DARK_SIGMAS: + try: + local = cv2.GaussianBlur(gray, (0, 0), sigma) + dark = np.maximum(local - gray, 0.0) + # 局部变暗 + 低饱和度，抑制花田/山地的彩色纹理 + lo = float(dark.min()) + hi = float(dark.max()) + if hi > lo: + dark = (dark - lo) / (hi - lo) + else: + dark = np.zeros_like(dark, dtype=np.float32) + for sw in SAT_WEIGHTS: + obj = dark * (1.0 - sw * sat) + maps.append(obj.astype(np.float32)) + except cv2.error: + continue + return maps + + +def scaled_shape(shape, scale): + try: + h, w = shape.shape + nw, nh = max(16, round(w * scale)), max(16, round(h * scale)) + return cv2.resize(shape, (nw, nh), interpolation=cv2.INTER_AREA).astype(np.float32) + except cv2.error: + return None + + +def response(objectness, shape): + try: + h, w = shape.shape + if h >= objectness.shape[0] or w >= objectness.shape[1]: + return None + # 只在 alpha 前景内统计暗色覆盖程度 + return cv2.matchTemplate(objectness, shape, cv2.TM_CCORR_NORMED, mask=shape) + except cv2.error: + return None + + +def peaks(res, w, h): + result = [] + try: + r = res.copy() + for _ in range(TOP_K): + _, score, _, loc = cv2.minMaxLoc(r) + if score <= 0: + break + result.append((float(score), int(loc[0]), int(loc[1]))) + x0, y0 = max(0, loc[0] - int(w * 0.7)), max(0, loc[1] - int(h * 0.7)) + x1, y1 = min(r.shape[1], loc[0] + int(w * 0.7)), min(r.shape[0], loc[1] + int(h * 0.7)) + r[y0:y1, x0:x1] = 0 + except Exception as e: + print(f" 峰提取失败: {e}") + return result + + +def process(prefix, lines): + try: + big_path, mark_path = pair_for(prefix) + if big_path is None: + return + loaded = read_pair(big_path, mark_path) + if loaded is None: + lines.append(f"{prefix}\tSKIP\t读取失败") + return + big, shape = loaded + maps = objectness_maps(big) + candidates = [] + for obj in maps: + for scale in SCALES: + tpl = scaled_shape(shape, float(scale)) + if tpl is None: + continue + res = response(obj, tpl) + if res is None: + continue + for score, x, y in peaks(res, tpl.shape[1], tpl.shape[0]): + candidates.append((score, float(scale), x, y, tpl.shape[1], tpl.shape[0])) + if not candidates: + lines.append(f"{prefix}\tSKIP\t无候选") + return + candidates.sort(key=lambda c: -c[0]) + kept = [] + for c in candidates: + score, scale, x, y, w, h = c + cx, cy = x + w / 2.0, y + h / 2.0 + if all(math.hypot(cx - (q[2] + q[4] / 2), cy - (q[3] + q[5] / 2)) > 0.65 * max(w, h) + for q in kept): + kept.append(c) + if len(kept) >= TOP_K: + break + score, scale, x, y, w, h = kept[0] + print(f"{prefix} score={score:.3f} scale={scale:.3f} offset=({x},{y}) size=({w},{h}) top={len(kept)}") + lines.append(f"{prefix}\tscore={score:.3f}\tscale={scale:.3f}\toffset=({x},{y})\t" + f"size=({w},{h})\ttop={len(kept)}") + vis = big.copy() + colors = [(0, 0, 255), (0, 165, 255), (255, 0, 0), (0, 255, 0), (255, 0, 255)] + for i, item in enumerate(kept): + _, _, px, py, pw, ph = item + cv2.rectangle(vis, (px, py), (px + pw, py + ph), colors[i], 2) + cv2.putText(vis, str(i + 1), (px + 2, py + 16), cv2.FONT_HERSHEY_SIMPLEX, .55, colors[i], 2) + cv2.imwrite(str(OUT / f"{prefix}-match.png"), vis) + except Exception as e: + print(f"{prefix} 处理失败: {e}") + lines.append(f"{prefix}\tERROR\t{e}") + + +def main(): + lines = [] + prefixes = sorted({p.name.split("~", 1)[0] for p in CAP.glob("*.jpeg") if "-mark" not in p.stem}) + for prefix in prefixes: + process(prefix, lines) + (OUT / "summary.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\n结果已写入 {OUT}/") + + +if __name__ == "__main__": + main()