feat: expand merged chat records directly from read-only databases

Add db merged with exact account/chat/local-ID selection and shard ambiguity checks. Preserve Int64 composite message types and parse bounded, DTD-free embedded record XML, including duplicate messages and source metadata. Revalidate cached keys against current page-1 HMAC; replace the SQL write probe with sqlite3_db_readonly. Core tests: 129 passed. Windows Session 0 database validation: 5/5; all 16 text/sender/source-ID/timestamp entries matched independent XML parsing. Read-only UI baseline: 3/3; zero sends.
This commit is contained in:
2026-09-06 20:10:04 +08:00
parent e5548bd471
commit 30df7ae433
15 changed files with 775 additions and 24 deletions
@@ -0,0 +1,41 @@
# 数据库合并聊天记录展开 — 2026-09-06
## 调用方式
```text
WxAgent.Host db merged --account <account-fingerprint> --chat filehelper --local-id 149
WxAgent.Host db merged --account <account-fingerprint> --chat filehelper --local-id 149 --include-content
```
可选 `--database <relative-path>` 消除不同消息分库 local ID 重复时的歧义;支持 `--key-file` 与 `--timeout`。只使用既有账户密钥缓存,不调用内存扫描器,也不打开聊天窗口。
默认返回父记录身份、条目路径、类型、脱敏发送人标识、原消息 ID、时间戳和正文长度;正文、显示名称、合并标题和描述为 null。`--include-content` 显式启用正文/名称。源 `hashusername` 不是可确认的真实 wxid,因此不把它伪装为 wxid。
## 实现与安全
- 将数据库 `local_type` 模型和转换从 Int32 修正为 **Int64**;实测合并类型为 `81604378673`。原先直接读取此记录会发生 Int32 溢出。
- 精确按聊天表及 `local_id` 参数查询,逐个消息分库检查;多处命中必须显式选择分库,不返回第一个碰巧匹配的消息。
- 解析 `appmsg/type=19 → recorditem → recordinfo/datalist/dataitem`,保留顺序、重复正文、原始字符串 ID、秒级 Unix 时间、显示时间和发送人名称。嵌套记录拥有独立路径,不把重复 dataid 当作去重依据。
- XML 禁止 DTD/外部实体,限制文档字符数、深度、嵌套记录层数及总条目数。声明数量与实际不符、文本缺失、非法时间等明确报错,不用空结果掩盖损坏。
- Hex/Zstd 解码有尺寸上限,避免压缩消息无限膨胀。
- 每次 SQLCipher 打开前,缓存密钥重新通过**当前目标数据库 page-1 HMAC**校验;连接仍为 `Mode=ReadOnly` 与 `query_only=ON`。
- 移除原来的 `CREATE TABLE` 写拒绝探针,改为读取 `sqlite3_db_readonly`,不尝试任何数据库写入。打开失败时也释放连接。
- 仅输出允许的附件元数据,不导出 CDN 地址/AES 密钥,不下载附件或绕过协议。
## 真机验收
- 微信 `4.1.13.63`,Windows `10.1.1.101`。
- 最终发布:`C:\Users\Rogee\wx-agent\releases\db-merged-20260906T120104`。
- EXE SHA-256:`5DB6A6422F6DAD9AE76C6C4D5A522576C7B7EB0534396434A536190D4E0DEEF8`。
- Linux **129/129 Core 测试通过**,完整 Release build 无警告/错误。
- `Test-DatabaseMerged.ps1` 在 **SSH Session 0** 运行,5/5 通过:默认脱敏输出、显式正文输出、数据库只读状态、非法 ID、缺失 ID。数据库读操作无需微信所在交互桌面;没有在 Session 0 执行 UI 自动化。
- 既有交互会话另行执行 doctor、inspect-ui、`smoke --read-only`,3/3 通过。
- 文件传输助手中父记录 local ID **149** 展开 **16 条文本消息**。
- 用独立 Python XML 解析器逐条对照数据库原始嵌入 XML:正文、发送人、原始消息 ID、时间戳 **各 16/16 完全一致**。
- 全程发送/转发次数 **0**。完整正文仅保留于忽略的私有验证产物,不进入 Git。
脱敏证据:[`evidence/Database-Merged-20260906/`](evidence/Database-Merged-20260906/)。第一次验证脚本误读 `metadata.WritesRejected` 的嵌套层次,修正脚本后同一二进制通过;并非数据库接受了写入。
## 验收范围
真实样本为 16 条文本;嵌套记录、附件元数据、恶意/截断 XML、重复条目和解压上限有离线测试。尚无混合图片/视频/文件、嵌套合并的真机样本;附件正文/原文件下载不属于本次展开结果。字段缺失保留 null,不补造身份或时间。
@@ -0,0 +1,18 @@
{
"Checks": [
{
"Stage": "doctor",
"ExitCode": 0
},
{
"Stage": "inspect-ui",
"ExitCode": 0
},
{
"Stage": "smoke-readonly",
"ExitCode": 0
}
],
"SendsAttempted": 0,
"Success": true
}
@@ -0,0 +1,4 @@
{
"error": "InvalidArgument",
"Message": "--local-id must be a positive 64-bit integer."
}
@@ -0,0 +1,307 @@
{
"account": "a2e8a1eaab7fd5fbd3806525c0d9cce750c28f4efb8e7cfcabc9e3e8660e0277",
"database": "message/message_0.db",
"chatId": "sha256:7F7D698D0AE00B0D",
"recordId": "a2e8a1eaab7fd5fbd3806525c0d9cce750c28f4efb8e7cfcabc9e3e8660e0277:message/message_0.db:149",
"parent": {
"LocalId": 149,
"ServerId": 7498992953238207040,
"Type": 81604378673,
"Timestamp": "2026-09-06T09:18:20+00:00"
},
"record": {
"title": null,
"description": null,
"count": 16,
"messages": [
{
"path": "0",
"dataId": "d7a3cf5d308ef865ededc7cda76cdf06",
"dataType": 1,
"senderId": "sha256:6221F48E5D88469F",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "5158611296847210182",
"timestamp": "2026-08-28T08:13:35+00:00",
"displayTime": "2026-08-28 16:13",
"text": null,
"length": 14,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "1",
"dataId": "a55bf6920207cc484914173ca7ec98ff",
"dataType": 1,
"senderId": "sha256:2F0AF7C1B8E3D684",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "2505042190386030576",
"timestamp": "2026-08-28T08:14:27+00:00",
"displayTime": "2026-08-28 16:14",
"text": null,
"length": 1,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "2",
"dataId": "30831307ce3e4909af357ca3a0d36492",
"dataType": 1,
"senderId": "sha256:2F0AF7C1B8E3D684",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "4314853433147875167",
"timestamp": "2026-08-28T08:14:31+00:00",
"displayTime": "2026-08-28 16:14",
"text": null,
"length": 3,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "3",
"dataId": "835abcae449f3e156d2e4961c42138ff",
"dataType": 1,
"senderId": "sha256:6221F48E5D88469F",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "6850076042613883819",
"timestamp": "2026-08-28T08:15:35+00:00",
"displayTime": "2026-08-28 16:15",
"text": null,
"length": 41,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "4",
"dataId": "fd7fcef26f3408b3e24fcf526f8c101b",
"dataType": 1,
"senderId": "sha256:2F0AF7C1B8E3D684",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "6621524893657137135",
"timestamp": "2026-08-28T08:16:12+00:00",
"displayTime": "2026-08-28 16:16",
"text": null,
"length": 3,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "5",
"dataId": "6fe59e85d75f439e66c9871458e322c1",
"dataType": 1,
"senderId": "sha256:6221F48E5D88469F",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "6108206286854864988",
"timestamp": "2026-08-28T08:16:59+00:00",
"displayTime": "2026-08-28 16:16",
"text": null,
"length": 58,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "6",
"dataId": "b0195dfb34d6552e9697717c85e3244d",
"dataType": 1,
"senderId": "sha256:2F0AF7C1B8E3D684",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "6291818813835930172",
"timestamp": "2026-08-28T08:17:32+00:00",
"displayTime": "2026-08-28 16:17",
"text": null,
"length": 9,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "7",
"dataId": "7865e3c4ef4610909cbda0034897852b",
"dataType": 1,
"senderId": "sha256:6221F48E5D88469F",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "8805680531905882697",
"timestamp": "2026-08-28T08:17:57+00:00",
"displayTime": "2026-08-28 16:17",
"text": null,
"length": 19,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "8",
"dataId": "ec62b6595e5f3391381b82b847d47de3",
"dataType": 1,
"senderId": "sha256:2F0AF7C1B8E3D684",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "7935299570123740393",
"timestamp": "2026-08-28T08:18:07+00:00",
"displayTime": "2026-08-28 16:18",
"text": null,
"length": 3,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "9",
"dataId": "8871d9bfe9c261850b155f737b9e0fc4",
"dataType": 1,
"senderId": "sha256:6221F48E5D88469F",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "5334061395816435285",
"timestamp": "2026-08-28T08:18:11+00:00",
"displayTime": "2026-08-28 16:18",
"text": null,
"length": 8,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "10",
"dataId": "06987bf3be56f93fe26f516ee9fce712",
"dataType": 1,
"senderId": "sha256:6221F48E5D88469F",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "3775467287450770804",
"timestamp": "2026-08-28T08:18:21+00:00",
"displayTime": "2026-08-28 16:18",
"text": null,
"length": 5,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "11",
"dataId": "811bbaf41661bc93de162cb79051d30c",
"dataType": 1,
"senderId": "sha256:6221F48E5D88469F",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "5606979100728674670",
"timestamp": "2026-08-28T08:18:33+00:00",
"displayTime": "2026-08-28 16:18",
"text": null,
"length": 4,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "12",
"dataId": "1b54a0acfa9af3f87612f1ad1d899615",
"dataType": 1,
"senderId": "sha256:2F0AF7C1B8E3D684",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "8768702616063460855",
"timestamp": "2026-08-28T08:18:34+00:00",
"displayTime": "2026-08-28 16:18",
"text": null,
"length": 1,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "13",
"dataId": "8f995c7506723cb9bbac6428de6e48da",
"dataType": 1,
"senderId": "sha256:2F0AF7C1B8E3D684",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "540359944811197130",
"timestamp": "2026-08-28T08:18:39+00:00",
"displayTime": "2026-08-28 16:18",
"text": null,
"length": 2,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "14",
"dataId": "6444ca65742edd89728ec3872ad10bef",
"dataType": 1,
"senderId": "sha256:6221F48E5D88469F",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "3039346887632908543",
"timestamp": "2026-08-28T08:19:05+00:00",
"displayTime": "2026-08-28 16:19",
"text": null,
"length": 9,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
},
{
"path": "15",
"dataId": "2489f0db398df842a99962035db3f82e",
"dataType": 1,
"senderId": "sha256:6221F48E5D88469F",
"senderName": null,
"sourceLocalId": null,
"sourceServerId": "679561649162720019",
"timestamp": "2026-08-28T08:19:16+00:00",
"displayTime": "2026-08-28 16:19",
"text": null,
"length": 8,
"contentAvailable": true,
"title": null,
"format": null,
"sizeBytes": null,
"nestedRecord": null
}
]
}
}
@@ -0,0 +1,4 @@
{
"error": "ControlNotFound",
"Message": "The selected database message was not found."
}
@@ -0,0 +1,3 @@
{
"WritesRejected": true
}
@@ -0,0 +1,34 @@
{
"BinarySha256": "5DB6A6422F6DAD9AE76C6C4D5A522576C7B7EB0534396434A536190D4E0DEEF8",
"SendsAttempted": 0,
"Checks": [
{
"ExpectedExit": 0,
"ExitCode": 0,
"Stage": "merged-default"
},
{
"ExpectedExit": 0,
"ExitCode": 0,
"Stage": "merged-content.private"
},
{
"ExpectedExit": 0,
"ExitCode": 0,
"Stage": "database-readonly"
},
{
"ExpectedExit": 1,
"ExitCode": 1,
"Stage": "invalid-id"
},
{
"ExpectedExit": 1,
"ExitCode": 1,
"Stage": "missing-id"
}
],
"Success": true,
"ProcessSessionId": 0,
"RecordCount": 16
}
@@ -0,0 +1,9 @@
{
"MessageCount": 16,
"ExactTextMatches": 16,
"ExactSenderMatches": 16,
"ExactSourceIdMatches": 16,
"ExactTimestampMatches": 16,
"IndependentParser": "Python xml.etree.ElementTree",
"ContentIncluded": false
}