feat: add document OCR and offline model archives

This commit is contained in:
lofyer
2026-08-11 16:49:51 +08:00
parent 19a4469561
commit e0e7bc573c
46 changed files with 7831 additions and 85 deletions
+23
View File
@@ -513,6 +513,26 @@ GoodBuddy 是可调整窗口大小的桌面应用。响应式设计优先保证
- 自动生效、仅执行即时命令或自行管理编辑流程的分类不显示全局保存操作。窄窗口下操作区可以换行,但保存入口必须保持清晰可见。
- 保存或测试成功统一进入应用通知视口,并按全局规则自动消失,不在分类页头或内容卡片中保留持久成功文案。加载、保存和测试错误显示在分类页头下方,并保留可处理的上下文。
### 13.8 文档解析设置
- 设置中心新增独立的“文档解析”分类,统一管理聊天附件、知识库导入以及后续文档审阅场景使用的提取、转换和 OCR 策略。OCR 不作为普通对话模型出现在“模型连接”中。
- 分类页头说明文档解析的跨场景作用,右侧依次显示“测试解析”和“保存设置”;保存位于最右侧。测试必须选择真实文件并执行实际解析,不能只检查模型文件或接口连通性。
- 页面首先显示原生解析、文档转换和 OCR 的运行状态,并明确当前可处理格式、回退能力与不可用原因。部分能力未配置时使用“部分可用”状态,不得把原生文本解析一并标记为失败。
- “使用场景”分别配置聊天附件和知识库导入。普通用户选择“自动解析”“快速文本”“完整索引”等预设;阈值、并发和超时放入默认折叠的高级设置。
- 本地 OCR 的全平台基线使用同一组 PP-OCRv6 ONNX 模型和 ONNX Runtime WebAssembly,在 Windows、macOS、Linux 的 x64 与 arm64 上保持相同功能。原生 ONNX、WebGPU、DirectML、CoreML 或 CUDA 只能作为可选加速,失败时必须回退到 WASM CPU。
- OCR 模型管理与语音模型保持一致:应用不内置权重,用户可按需从 ModelScope 下载,也可在联网设备导出 ZIP 并在离线或内网设备直接导入。语音和 OCR 模型的下载、取消、ZIP 导入、ZIP 导出、删除与打开受管目录使用同一交互语义;ZIP 操作不得隐式切换当前模型或保存解析设置。
- OCR 模型卡片必须持续显示来源、语言、运行时、体积、安装状态和许可。“打开 ModelScope”直接位于卡片右上角,不再使用“模型详情与手动导入”折叠区。窄窗口下仓库操作换行到模型摘要下方,仍须保持可访问名称和键盘操作。
- PP-OCRv6 提供三个已实现档位:Tiny 约 6 MiB,适合低资源设备;Small 约 30 MiB,官方支持 50 种语言并作为推荐档位;Medium 约 132 MiB,官方支持 50 种语言、质量更高但速度较慢,界面必须提示其更高的内存占用和延迟。
- 本地模型按受管目录和固定清单加载。ModelScope 下载地址必须固定不可变 revision、字节数和 SHA-256;下载先进入临时目录,全部校验成功后再原子安装。识别时不得从网络或可变分支临时加载模型。
- 模型 ZIP 使用版本化的 `goodbuddy-model.json` 清单,声明模型类型、内置目录 ID、文件角色、大小与 SHA-256。导出前重新校验已安装文件;导入时限制压缩包大小、条目数、单文件和总展开大小,拒绝路径穿越、重复、未知、缺失或嵌套条目,并以应用内置目录重新校验后原子安装。ZIP 内的自声明信息不能扩大受信任模型集合。
- PDF 先读取文本层。仅当页面无有效文本、乱码比例过高或用户选择“始终 OCR”时渲染该页并识别;不得因为单页需要 OCR 而丢弃其他页面已经提取的可靠文本。
- DOCX、XLSX、PPTX 优先保留段落、单元格、公式、备注等原生语义。转换为 PDF 用于补充版面、页码、图表和图片理解,不作为唯一中间格式。
- DOC、XLS、PPT 等旧格式通过受控转换 Provider 生成新式 Office 文档和 PDF。转换子进程必须禁用宏和网络,限制输入、输出、内存、超时与临时目录,并在关闭或取消时清理。
- OCR 来源使用“本地模型 / 远程服务”互斥选择。选择本地后显示模型下载、模型下拉选择和本地运行参数;选择远程后显示 MinerU、PaddleOCR-VL 等服务连接配置。未实现的远程服务入口保持可读但禁用,不再增加与来源选择重复的“隐私与云端处理”授权区。
- 用户配置并保存远程 OCR 服务即表示选择该处理路径,不再逐场景重复询问。界面仍须明确显示当前服务名称、处理范围和远程属性,API 密钥只保存在主进程加密设置中,未选中远程服务时不得上传文档。
- 解析结果使用统一文档结构,至少保留文档标题、来源格式、页码或工作表定位、正文块、置信度、处理方式和警告。聊天附件对结果做有界截断,知识库使用完整结果分块和索引。
- 测试结果显示文件类型、页数、实际工作流、提取字数、OCR 页数、耗时和警告。测试文件不得自动进入聊天上下文或知识库。
## 14. 文案规则
- 使用简体中文,动词直接、对象明确。
@@ -562,6 +582,7 @@ GoodBuddy 是可调整窗口大小的桌面应用。响应式设计优先保证
- [ ] 智能心跳迁移到 `dashboard`,统一状态卡片、配置和运行历史层级。
- [ ] 任务迁移到 `standard`,活动记录迁移到 `dashboard`,统一导航、筛选和表格行为。
- [ ] 设置中心使用共享分类定义与 `SettingsCategoryHeader`,将保存与测试操作统一放到分类页头右侧,并把成功反馈接入应用通知。
- [ ] 文档解析设置统一聊天附件与知识库的解析预设、OCR 状态、转换状态、隐私限制和真实文件测试。
### 15.5 验收
@@ -572,6 +593,8 @@ GoodBuddy 是可调整窗口大小的桌面应用。响应式设计优先保证
- [ ] 验证页面范围、对象范围和操作范围在关键流程中始终可见。
- [ ] 验证删除、批量操作、停止运行和清空历史符合风险等级策略。
- [ ] 验证加载中、首次为空、筛选无结果、搜索无结果、失败和只读状态不会互相混用。
- [ ] 在 Windows、macOS、Linux 的 x64 与 arm64 上执行真实本地 OCR,并验证 WASM CPU 回退、取消、超时和离线运行。
- [ ] 在联网设备导出语音与 OCR 模型 ZIP,在离线设备导入后执行真实推理;验证错误模型 ID、篡改文件、路径穿越、未知条目和压缩炸弹均被拒绝。
## 16. 完成标准
+331
View File
@@ -0,0 +1,331 @@
# 文档解析与本地 OCR
## 1. 目标
GoodBuddy 需要用同一条可信文档解析链路服务以下场景:
- 聊天附件问答;
- 知识库导入、同步、分块与来源定位;
- 后续的合同审阅、表格分析、演示文稿理解和文档转换。
文档解析不是对话模型的附属功能。它是主进程管理的独立基础能力,设置入口为“设置中心 / 文档解析”。
## 2. 当前基线
原生解析器已经支持:
- UTF-8 文本、代码、配置、HTML;
- 带文本层的 PDF
- DOCX 正文;
- XLSX 工作表 XML 与共享字符串;
- PPTX 幻灯片文字。
现有局限:
- 纯扫描 PDF 没有文本层时无法提取内容;
- DOC、XLS、PPT 等旧版二进制 Office 格式不支持;
- Office 解析主要提取文字,不能完整保留表格、公式、图表和版面;
- 聊天附件和知识库直接调用底层解析函数,缺少可配置的统一工作流;
- 没有本地 OCR 模型状态、真实解析测试和按场景策略。
## 3. 产品原则
### 3.1 双通道解析
PDF 不是所有文档唯一的中间格式。解析应同时保留:
1. 原生语义通道:标题、段落、单元格、公式、备注和对象关系;
2. 渲染视觉通道:页码、版面、图表、图片和 OCR 结果。
两条通道合并为统一文档结构。转换为 PDF 用于补充视觉信息,不得覆盖更可靠的原生语义结果。
### 3.2 场景工作流
| 场景 | 默认预设 | 行为 |
| --- | --- | --- |
| 聊天附件 | 自动解析 | 优先快速提取,文本不足时按需 OCR,有界截断后加入当前请求 |
| 知识库导入 | 完整索引 | 完整解析、按页或工作表定位、按需 OCR、分块与索引 |
| 扫描文档 | OCR | 页面渲染、文字识别、置信度与定位保留 |
| 表格分析 | 语义优先 | 单元格和值优先,PDF 或图片补充图表与打印布局 |
| 高保真审阅 | 视觉增强 | 原生解析、页面渲染、OCR 或视觉理解合并 |
### 3.3 本地优先
- 文本层和本地 OCR 均在设备上处理;
- 本地处理不因 Ask 或 Execute 模式改变;
- OCR 来源必须在“本地模型 / 远程服务”之间明确选择;
- 配置并保存远程服务即表示用户选择该处理路径,不再增加逐场景授权;
- API 密钥只能保存在主进程加密设置中;
- 测试文件不得自动进入聊天或知识库。
## 4. 设置设计
设置中心新增“文档解析”分类,结构如下:
1. 分类页头:“测试解析”“保存设置”;
2. 运行状态:原生解析、文档转换、本地 OCR;
3. 使用场景:聊天附件、知识库导入;
4. 文档转换;
5. OCR 识别;
6. 高级解析设置;
OCR 模型区沿用语音模型管理模式:
- 应用不内置模型权重;
- 用户按需从 ModelScope 下载,下载完成后离线使用;
- 显示来源、语言、运行时、模型体积、安装与校验状态;
- 联网设备可导出已安装模型 ZIP,离线或内网设备可直接导入;
- 支持下载进度、取消、删除、ZIP 导入导出、打开模型仓库和受管目录;
- “打开 ModelScope”直接显示在 OCR 模型卡片右上角,不使用手动导入折叠区;
- 模型操作即时生效,解析策略仍通过分类页头的“保存设置”提交。
### 4.1 第一阶段字段
- 聊天附件预设:`auto``fast-text``high-fidelity`
- 知识库预设:`complete-index``fast-index``high-fidelity`
- PDF OCR 策略:`auto``always``disabled`
- OCR 来源:第一阶段固定为 `local`,远程服务入口禁用;
- 本地 OCR 模型:`pp-ocrv6-tiny``pp-ocrv6-small``pp-ocrv6-medium`
- 单文档最大页数;
- OCR 并发数;
- 单页超时。
OCR 来源使用互斥选择。本地模型选中后才显示模型下拉列表、按需下载、导入和本地 OCR 参数;远程服务计划接入 MinerU、PaddleOCR-VL 等接口,第一阶段保持可读但禁用。来源选择本身就是用户的明确决策,不再显示额外的“隐私与云端处理”授权区。
## 5. 架构
```text
聊天附件 ─┐
├─ DocumentParsingService
知识库导入 ┘ ├─ NativeDocumentParser
├─ PdfTextQualityEvaluator
├─ PdfPageRenderer
├─ LocalOcrProvider
├─ DocumentConversionProvider
└─ ParsedDocument merger
```
`DocumentParsingService` 是唯一场景入口:
```ts
type DocumentParsingPurpose = 'chat-attachment' | 'knowledge-index'
type DocumentParsingService = {
parse(
name: string,
bytes: Buffer,
purpose: DocumentParsingPurpose,
signal?: AbortSignal
): Promise<ParsedDocument>
}
```
聊天上下文管理器与知识库服务依赖该接口,不直接选择 OCR Provider。
## 6. 统一结果
第一阶段兼容现有 `ParsedDocument`,并逐步扩展:
```ts
type ParsedDocument = {
title: string
sourceFormat: string
content: string
sections: Array<{
locator: string
content: string
method?: 'native' | 'ocr' | 'converted' | 'vision'
confidence?: number
}>
warnings?: string[]
}
```
定位字段必须对使用者有意义:
- PDF`第 3 页`
- XLSX`工作表:预算 / A1:F28`
- PPTX`幻灯片 5`
- DOCX:标题路径或页码;
- 文本:`全文`
## 7. 本地 OCR 基线
### 7.1 模型与运行时
全平台功能基线:
- 模型:PP-OCRv6 ONNX/ORT
- 轻量下载档位:Tiny,约 6 MiB,用于低资源设备和六平台离线链路;
- 推荐下载档位:Small,约 30 MiB,官方支持 50 种语言;
- 高精度下载档位:Medium,约 132 MiB,官方支持 50 种语言,但识别较慢且需要更多内存;
- 运行时:ONNX Runtime WebAssembly
- 处理环境:隔离 Worker
- 加速:WebGPU 或平台原生执行 Provider,仅作为可选层;
- 回退:任何加速失败后使用 WASM CPU。
需要覆盖的发布矩阵:
- Windows x64、Windows arm64
- macOS x64、macOS arm64
- Linux x64、Linux arm64。
模型清单必须固定以下信息:
- 上游仓库和不可变 revision;
- 文件名、字节数和 SHA-256
- 模型族、语言、质量和速度;
- 许可证名称、完整许可证和来源;
- 检测模型、识别模型、字符字典的匹配关系。
运行时不得从 `main``latest` 或其他可变地址加载模型。
### 7.2 下载与安装
Tiny、Small 和 Medium 模型均由 PaddlePaddle 官方 ModelScope 仓库提供。Small 是默认推荐档位;Medium 面向更高识别质量,但具有更高内存占用和延迟。每个档位的检测模型、识别模型与字符字典配置分别使用固定提交,并在应用内记录文件字节数和 SHA-256。
下载流程:
1. 主进程从固定 ModelScope `resolve/<revision>/...` 地址读取文件;
2. 禁用凭据与缓存,限制重定向次数和单文件大小;
3. 写入受管目录下的随机临时安装目录;
4. 边下载边计算 SHA-256,并核对完整字节数;
5. 三个文件全部通过校验后写入安装清单;
6. 原子重命名为正式模型目录;
7. 失败、取消或退出时删除临时文件。
模型只在下载或用户显式打开仓库时访问网络。OCR 推理从受管目录读取已校验文件,不发起网络请求。
### 7.3 离线 ZIP 迁移
语音模型和 OCR 模型使用同一种离线迁移流程:
1. 联网设备完成受信任来源下载和校验;
2. 在模型卡片选择“导出 ZIP”;
3. 将 ZIP 通过组织批准的介质传输到离线或内网设备;
4. 在相同模型的卡片选择“导入 ZIP”;
5. 主进程按当前应用内置目录重新校验,并在全部通过后原子安装。
ZIP 根目录包含模型文件和 `goodbuddy-model.json`。清单格式为 `goodbuddy-model-archive`,当前版本为 `1`,记录:
- 模型类型:`speech``document-ocr`
- 内置模型 ID 和显示名称;
- 文件名、角色、原始字节数和 SHA-256;
- 导出时间。
导出不能直接信任已有安装清单,必须重新读取并校验每个文件。导入不能只信任 ZIP 自声明内容,模型 ID、文件角色、字节数和哈希必须再次与当前应用内置目录完全匹配。导入通过后复用普通本地安装的受控临时目录和原子重命名路径。
归档处理使用有界流式读写,不把大型模型或整个展开结果复制到内存。主进程限制压缩包大小、条目数、清单大小、单文件大小和总展开大小,并拒绝:
- 绝对路径、`..`、目录或嵌套路径;
- 大小写不敏感的重复条目;
- 未声明、缺失或角色不匹配的文件;
- 模型类型或模型 ID 不匹配;
- 解压后大小或 SHA-256 不匹配;
- 超过边界的压缩包和压缩炸弹。
取消文件对话框不会改变安装状态。导入和导出也不会切换当前语音/OCR 模型,不会隐式保存文档解析设置。
### 7.4 PDF 流程
1. 使用 PDF.js 读取每页文本层;
2. 评估有效字符数、乱码率和图片占比;
3. `auto` 模式只渲染文本不足的页面;
4. `always` 模式渲染所有页面;
5. Worker 将页面限制在配置的最大边长内;
6. OCR 返回文字、坐标和置信度;
7. 按页合并原生文本与 OCR,不重复可靠文本;
8. 达到页数、超时、取消或输出限制时停止并返回明确错误。
受密码保护、损坏或超限的 PDF 不得进入 OCR。
## 8. Office 与转换
### 8.1 新格式
- DOCX:正文、标题、表格、批注和图片关系;
- XLSX:工作表、单元格地址、值、公式、合并关系和图表;
- PPTX:幻灯片、文字对象、备注、图片和阅读顺序。
Office 内嵌图片 OCR 属于增强流程,不能替代原生结构解析。
### 8.2 旧格式
DOC、XLS、PPT 通过 `DocumentConversionProvider` 转换:
1. 转换为 DOCX、XLSX 或 PPTX,供语义解析;
2. 转换为 PDF,供页码、版面和视觉解析;
3. 合并结果并记录转换警告。
本地 LibreOffice Provider 必须:
- 在隔离子进程中运行;
- 禁用宏和网络;
- 使用单任务临时目录;
- 限制输入大小、输出大小、内存和超时;
- 在成功、失败、取消和退出时清理;
- 不接受用户提供的任意命令参数。
## 9. 安全边界
- 文件路径解析、读取、大小检查和格式校验在主进程完成;
- OCR Worker 只接收当前任务所需的有界页面图像和只读模型;
- 不向 Worker 暴露文件系统、Electron API、凭据或任意网络访问;
- 文档内容视为不可信数据,不解释其中的提示词为系统指令;
- 模型和转换程序必须固定版本并校验哈希;
- OCR 输出受字符数限制,错误不得包含绝对路径或未脱敏文档内容;
- 取消、超时和应用关闭必须终止待处理页面并释放模型会话。
## 10. 错误与回退
必须区分:
- 不支持的格式;
- 文档损坏或受密码保护;
- 文本层为空但 OCR 未启用;
- OCR 模型不可用;
- OCR 超时或取消;
- 文档页数、大小或输出超限;
- 本地转换服务未配置;
- 所选远程 OCR 服务不可用或配置不完整。
`auto` 工作流可以从 OCR 回退到可靠的原生文本,但不能把空结果标记为成功。知识库导入失败时保留来源和可重试上下文。
## 11. 实施阶段
### 阶段一
- 新增文档解析设置分类和持久化契约;
- 建立 `DocumentParsingService`,供聊天和知识库共用;
- 将无文本 PDF 识别为可触发 OCR 的明确状态;
- 接入 PP-OCRv6 Tiny、Small、Medium 的 ModelScope 下载、校验、ZIP 离线迁移、删除与 WASM Worker
- 实现真实文件测试和六平台验证入口。
### 阶段二
- 增强 DOCX、XLSX、PPTX 语义结构;
- 实现按页混合文本层与 OCR
- 增加版面、表格和阅读顺序。
### 阶段三
- 增加 LibreOffice 和 API 转换 Provider
- 支持 DOC、XLS、PPT
- 增加 MinerU、PaddleOCR-VL 等远程 OCR 服务连接配置;
- 增加高保真工作流和解析结果预览。
## 12. 验收
- 同一份扫描 PDF 可从聊天附件和知识库得到一致的逐页文本;
- 文本型 PDF 在 `auto` 模式下不运行 OCR
- 本地 OCR 在六个平台和两种架构上完全离线运行;
- 模型文件损坏时拒绝加载并显示可恢复错误;
- 未安装模型时扫描文档提示用户前往“文档解析”下载,文本型文档仍可原生解析;
- 下载中可显示文件与总进度并允许取消,失败或取消后不留下已安装状态;
- ModelScope 下载与 ZIP 导入均经过同一大小和 SHA-256 校验;
- 语音和 OCR 模型可在联网设备导出 ZIP,并在离线设备导入后完成真实推理;
- 路径穿越、未知条目、错误模型 ID、篡改文件和超限 ZIP 均被拒绝;
- 超页数、超时、取消和关闭不会留下运行任务;
- 测试解析不会创建聊天消息或知识库文档;
- 选择本地模型时没有任何文档上传;
- 文档中的提示词不会改变系统、模式或工具权限。
+3
View File
@@ -34,6 +34,9 @@ export default defineConfig({
'@shared': resolve('src/shared')
}
},
worker: {
format: 'es'
},
plugins: [react()]
}
})
+182 -3
View File
@@ -20,7 +20,9 @@
"i18next": "^25.10.10",
"json5": "^2.2.3",
"lucide-react": "^1.27.0",
"onnxruntime-web": "^1.23.2",
"pdfjs-dist": "^6.2.108",
"ppu-paddle-ocr": "^6.4.0",
"qrcode": "^1.5.4",
"quill": "^2.0.3",
"react": "^19.2.8",
@@ -2325,7 +2327,6 @@
"resolved": "https://registry.npmjs.org/@napi-rs/canvas/-/canvas-1.0.3.tgz",
"integrity": "sha512-OlI657a5XXvKGFX7kNeIzJ8rO7IXt87Mqu2H8rXE46viAuOfum/JA7ysX7+eBhxNKznT+RCZh418mndlcFX3+w==",
"license": "MIT",
"optional": true,
"workspaces": [
"e2e/*"
],
@@ -2659,6 +2660,63 @@
"node": ">=14.18.0"
}
},
"node_modules/@protobufjs/aspromise": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz",
"integrity": "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/base64": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/base64/-/base64-1.1.2.tgz",
"integrity": "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/codegen": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.5.tgz",
"integrity": "sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/eventemitter": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.1.tgz",
"integrity": "sha512-vW1GmwMZNnL+gMRaovlh9yZX74kc+TTU3FObkkurpMaRtBfLP3ldjS9KQWlwZgraRE0+dheEEoAxdzcJQ8eXZg==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/fetch": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.1.tgz",
"integrity": "sha512-GpptLrs57adMSuHi3VNj0mAF8dwh36LMaYF6XyJ6JMWlVsc+t42tm1HSEDmOs3A8fC9yyeisgLhsTVQokOZ0zw==",
"license": "BSD-3-Clause",
"dependencies": {
"@protobufjs/aspromise": "^1.1.1"
}
},
"node_modules/@protobufjs/float": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/@protobufjs/float/-/float-1.0.2.tgz",
"integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/path": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz",
"integrity": "sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/pool": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@protobufjs/pool/-/pool-1.1.0.tgz",
"integrity": "sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/utf8": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.2.tgz",
"integrity": "sha512-b1UQwcEZ4yCnMCD8DAL1VlbvBJE9/IX4FTIp7BG1xYpf29SLazLSrqUkj4w7Y5y7cCVP6E5tcqqcI0xemPkHug==",
"license": "BSD-3-Clause"
},
"node_modules/@rolldown/pluginutils": {
"version": "1.0.0-rc.3",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.3.tgz",
@@ -3104,6 +3162,12 @@
"node": ">=10"
}
},
"node_modules/@techstark/opencv-js": {
"version": "5.0.0-release.1",
"resolved": "https://registry.npmjs.org/@techstark/opencv-js/-/opencv-js-5.0.0-release.1.tgz",
"integrity": "sha512-PIm+eB0MFtieXoNC2GRao0dv/02sehG+Nv2nSW5D6pQm6J/4WqvDHm0RyoqOmGYQm67jdGiaOdIeTShCY3PIUg==",
"license": "Apache-2.0"
},
"node_modules/@testing-library/dom": {
"version": "10.4.1",
"resolved": "https://registry.npmjs.org/@testing-library/dom/-/dom-10.4.1.tgz",
@@ -3512,7 +3576,6 @@
"version": "26.1.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-26.1.2.tgz",
"integrity": "sha512-Vu4a5UFA9rIIFJ7rB/Vaafh9lrCQszopTCx6KjFboXTGQbPNasehVR5TEiithSDGyd1DEiUByggTZsg8jukeIg==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~8.3.0"
@@ -6794,6 +6857,12 @@
"node": ">=16"
}
},
"node_modules/flatbuffers": {
"version": "25.9.23",
"resolved": "https://registry.npmjs.org/flatbuffers/-/flatbuffers-25.9.23.tgz",
"integrity": "sha512-MI1qs7Lo4Syw0EOzUl0xjs2lsoeqFku44KpngfIduHBYvzm8h2+7K8YMQh1JtVVVrUvhLpNwqVi4DERegUJhPQ==",
"license": "Apache-2.0"
},
"node_modules/flatted": {
"version": "3.4.3",
"resolved": "https://registry.npmjs.org/flatted/-/flatted-3.4.3.tgz",
@@ -7170,6 +7239,12 @@
"lodash": "^4.17.15"
}
},
"node_modules/guid-typescript": {
"version": "1.0.9",
"resolved": "https://registry.npmjs.org/guid-typescript/-/guid-typescript-1.0.9.tgz",
"integrity": "sha512-Y8T4vYhEfwJOTbouREvG+3XDsjr8E3kIr7uf+JZ0BYloFsttiHU0WfvANVsR7TxNUJa/WpCnw/Ino/p+DeBhBQ==",
"license": "ISC"
},
"node_modules/has-flag": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz",
@@ -8044,6 +8119,12 @@
"deprecated": "This package is deprecated. Use require('node:util').isDeepStrictEqual instead.",
"license": "MIT"
},
"node_modules/long": {
"version": "5.3.2",
"resolved": "https://registry.npmjs.org/long/-/long-5.3.2.tgz",
"integrity": "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==",
"license": "Apache-2.0"
},
"node_modules/longest-streak": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/longest-streak/-/longest-streak-3.1.0.tgz",
@@ -9457,6 +9538,26 @@
"wrappy": "1"
}
},
"node_modules/onnxruntime-common": {
"version": "1.23.2",
"resolved": "https://registry.npmjs.org/onnxruntime-common/-/onnxruntime-common-1.23.2.tgz",
"integrity": "sha512-5LFsC9Dukzp2WV6kNHYLNzp8sT6V02IubLCbzw2Xd6X5GOlr65gAX6xiJwyi2URJol/s71gaQLC5F2C25AAR2w==",
"license": "MIT"
},
"node_modules/onnxruntime-web": {
"version": "1.23.2",
"resolved": "https://registry.npmjs.org/onnxruntime-web/-/onnxruntime-web-1.23.2.tgz",
"integrity": "sha512-T09JUtMn+CZLk3mFwqiH0lgQf+4S7+oYHHtk6uhaYAAJI95bTcKi5bOOZYwORXfS/RLZCjDDEXGWIuOCAFlEjg==",
"license": "MIT",
"dependencies": {
"flatbuffers": "^25.1.24",
"guid-typescript": "^1.0.9",
"long": "^5.2.3",
"onnxruntime-common": "1.23.2",
"platform": "^1.3.6",
"protobufjs": "^7.2.4"
}
},
"node_modules/opencode-ai": {
"version": "1.18.9",
"resolved": "https://registry.npmjs.org/opencode-ai/-/opencode-ai-1.18.9.tgz",
@@ -9932,6 +10033,12 @@
"url": "https://paulmillr.com/funding/"
}
},
"node_modules/platform": {
"version": "1.3.6",
"resolved": "https://registry.npmjs.org/platform/-/platform-1.3.6.tgz",
"integrity": "sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg==",
"license": "MIT"
},
"node_modules/plist": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/plist/-/plist-3.1.0.tgz",
@@ -10051,6 +10158,56 @@
"node": "^12.20.0 || >=14"
}
},
"node_modules/ppu-ocv": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/ppu-ocv/-/ppu-ocv-4.0.0.tgz",
"integrity": "sha512-ol+S9/KLZ0aTV0xbC8+ZB0iM+FYux8ujoULeQoYY2IDBZj4LAuujJUkzHzUA2O3KzHs17KuuIzV/w4/htPdn3w==",
"license": "MIT",
"dependencies": {
"@napi-rs/canvas": "^1.0.0",
"@techstark/opencv-js": "^5.0.0-release.1"
},
"peerDependencies": {
"@shopify/react-native-skia": ">=1.0.0"
},
"peerDependenciesMeta": {
"@shopify/react-native-skia": {
"optional": true
}
}
},
"node_modules/ppu-paddle-ocr": {
"version": "6.4.0",
"resolved": "https://registry.npmjs.org/ppu-paddle-ocr/-/ppu-paddle-ocr-6.4.0.tgz",
"integrity": "sha512-Llhlh6zIDbbvgG+6zVKLaoRdi8sQzH8hkh6YB6WewFjK7jhbNKfX8Rel2w2g7EKELhLhgg2zCTaZFM9loeWLVQ==",
"license": "MIT",
"dependencies": {
"ppu-ocv": "^4.0.0"
},
"bin": {
"ppu-paddle-ocr": "cli/index.js"
},
"peerDependencies": {
"@shopify/react-native-skia": ">=1.0.0",
"onnxruntime-node": "^1.23.2",
"onnxruntime-react-native": "^1.23.2",
"onnxruntime-web": "^1.23.2"
},
"peerDependenciesMeta": {
"@shopify/react-native-skia": {
"optional": true
},
"onnxruntime-node": {
"optional": true
},
"onnxruntime-react-native": {
"optional": true
},
"onnxruntime-web": {
"optional": true
}
}
},
"node_modules/prelude-ls": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/prelude-ls/-/prelude-ls-1.2.1.tgz",
@@ -10154,6 +10311,29 @@
"url": "https://github.com/sponsors/wooorm"
}
},
"node_modules/protobufjs": {
"version": "7.6.5",
"resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.5.tgz",
"integrity": "sha512-/FPD0nUc9jH6rfFjji9IBqOz4pcSE3CsT1m7Ep6Mdb0LxSUMj8hgl6GomOvZzpNpAqqGaXA0P3VSrZLFzIhQrw==",
"hasInstallScript": true,
"license": "BSD-3-Clause",
"dependencies": {
"@protobufjs/aspromise": "^1.1.2",
"@protobufjs/base64": "^1.1.2",
"@protobufjs/codegen": "^2.0.5",
"@protobufjs/eventemitter": "^1.1.1",
"@protobufjs/fetch": "^1.1.1",
"@protobufjs/float": "^1.0.2",
"@protobufjs/path": "^1.1.2",
"@protobufjs/pool": "^1.1.0",
"@protobufjs/utf8": "^1.1.1",
"@types/node": ">=13.7.0",
"long": "^5.3.2"
},
"engines": {
"node": ">=12.0.0"
}
},
"node_modules/proxy-addr": {
"version": "2.0.7",
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
@@ -11737,7 +11917,6 @@
"version": "8.3.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.3.0.tgz",
"integrity": "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==",
"dev": true,
"license": "MIT"
},
"node_modules/unicorn-magic": {
+14
View File
@@ -98,6 +98,18 @@
{
"from": "node_modules/@fontsource-variable/noto-sans-sc/LICENSE",
"to": "licenses/noto-sans-sc-OFL-1.1.txt"
},
{
"from": "node_modules/ppu-paddle-ocr/LICENSE",
"to": "licenses/ppu-paddle-ocr-MIT.txt"
},
{
"from": "node_modules/ppu-ocv/LICENSE",
"to": "licenses/ppu-ocv-MIT.txt"
},
{
"from": "node_modules/onnxruntime-web/LICENSE",
"to": "licenses/onnxruntime-web-MIT.txt"
}
],
"win": {
@@ -148,7 +160,9 @@
"i18next": "^25.10.10",
"json5": "^2.2.3",
"lucide-react": "^1.27.0",
"onnxruntime-web": "^1.23.2",
"pdfjs-dist": "^6.2.108",
"ppu-paddle-ocr": "^6.4.0",
"qrcode": "^1.5.4",
"quill": "^2.0.3",
"react": "^19.2.8",
+27 -4
View File
@@ -24,6 +24,7 @@ import type {
} from './agent/runtime'
import { encodeBoundedJpeg } from './bounded-jpeg'
import { parseDocument } from './knowledge/document-parser'
import type { ParsedDocument } from './knowledge/document-parser'
type StoredTextContext = ContextAttachment & {
kind: 'text'
@@ -94,7 +95,7 @@ function truncateUtf8(value: string, maximumBytes: number): string {
}
function formatParsedDocument(
sections: Awaited<ReturnType<typeof parseDocument>>['sections']
sections: ParsedDocument['sections']
): string {
return sections
.map(
@@ -121,6 +122,23 @@ function remoteAttachmentName(value: string): string {
export class ContextManager {
private readonly contexts = new Map<string, StoredContext>()
private totalBytes = 0
private readonly documentParser: (
name: string,
buffer: Buffer,
purpose: 'chat-attachment'
) => Promise<ParsedDocument>
constructor(options?: {
parseDocument?: (
name: string,
buffer: Buffer,
purpose: 'chat-attachment'
) => Promise<ParsedDocument>
}) {
this.documentParser =
options?.parseDocument ??
((name, buffer) => parseDocument(name, buffer))
}
private toPublic(context: StoredContext): ContextAttachment {
return {
@@ -245,7 +263,11 @@ export class ContextManager {
)
}
if (supportedDocumentExtensions.has(extension)) {
const parsed = await parseDocument(name, data)
const parsed = await this.documentParser(
name,
data,
'chat-attachment'
)
return this.storeText(
name,
truncateUtf8(
@@ -340,9 +362,10 @@ export class ContextManager {
) {
throw new Error('PDF 或 Office 文档必须小于 20MB 且不能是目录')
}
const parsed = await parseDocument(
const parsed = await this.documentParser(
basename(canonicalPath),
await handle.readFile()
await handle.readFile(),
'chat-attachment'
)
attachments.push(
this.storeText(
+53
View File
@@ -0,0 +1,53 @@
import { describe, expect, it, vi } from 'vitest'
import { ipcChannels } from '../shared/ipc-channels'
import { DocumentOcrBroker } from './document-ocr-broker'
function request() {
return {
modelId: 'pp-ocrv6-tiny',
fileName: 'scan.pdf',
mimeType: 'application/pdf' as const,
data: new ArrayBuffer(8),
maximumPages: 10,
pageNumbers: [1],
pageTimeoutSeconds: 60
}
}
describe('DocumentOcrBroker', () => {
it('forwards an AbortSignal cancellation to the renderer', async () => {
const send = vi.fn()
const broker = new DocumentOcrBroker({
isDestroyed: vi.fn(() => false),
webContents: { send }
} as never)
const controller = new AbortController()
const result = broker.recognize(request(), controller.signal)
const ocrRequest = send.mock.calls.find(
([channel]) => channel === ipcChannels.documentParsingOcrRequest
)?.[1] as { requestId: string }
controller.abort()
await expect(result).rejects.toThrow('OCR 解析已取消')
expect(send).toHaveBeenCalledWith(
ipcChannels.documentParsingOcrCancel,
ocrRequest.requestId
)
broker.dispose()
})
it('rejects a request that is already cancelled', () => {
const broker = new DocumentOcrBroker({
isDestroyed: vi.fn(() => false),
webContents: { send: vi.fn() }
} as never)
const controller = new AbortController()
controller.abort()
expect(() =>
broker.recognize(request(), controller.signal)
).toThrow('OCR 解析已取消')
broker.dispose()
})
})
+131
View File
@@ -0,0 +1,131 @@
import type { BrowserWindow } from 'electron'
import { ipcChannels } from '../shared/ipc-channels'
import {
documentOcrFailureSchema,
documentOcrRequestSchema,
documentOcrResultSchema,
type DocumentOcrRequest,
type DocumentOcrResult
} from '../shared/document-parsing-contracts'
type PendingRequest = {
resolve: (result: DocumentOcrResult) => void
reject: (error: Error) => void
timer: ReturnType<typeof setTimeout>
detachAbort: () => void
}
const maximumPendingRequests = 4
const maximumTotalTimeoutMs = 10 * 60 * 1_000
export class DocumentOcrBroker {
private readonly pending = new Map<string, PendingRequest>()
private disposed = false
constructor(private readonly window: BrowserWindow) {}
recognize(
input: Omit<DocumentOcrRequest, 'requestId'>,
signal?: AbortSignal
): Promise<DocumentOcrResult> {
if (this.disposed || this.window.isDestroyed()) {
throw new Error('OCR 渲染服务不可用')
}
if (this.pending.size >= maximumPendingRequests) {
throw new Error('OCR 任务过多,请稍后重试')
}
const request = documentOcrRequestSchema.parse({
...input,
requestId: crypto.randomUUID()
})
if (signal?.aborted) {
throw new Error('OCR 解析已取消')
}
const timeoutMs = Math.min(
maximumTotalTimeoutMs,
Math.max(
request.pageTimeoutSeconds * 1_000,
request.pageTimeoutSeconds *
request.maximumPages *
1_000
)
)
return new Promise<DocumentOcrResult>((resolve, reject) => {
const cancel = (message: string): void => {
const pending = this.pending.get(request.requestId)
if (!pending) {
return
}
clearTimeout(pending.timer)
pending.detachAbort()
this.pending.delete(request.requestId)
this.window.webContents.send(
ipcChannels.documentParsingOcrCancel,
request.requestId
)
reject(new Error(message))
}
const timer = setTimeout(() => {
cancel('OCR 解析超时')
}, timeoutMs)
const onAbort = (): void => cancel('OCR 解析已取消')
signal?.addEventListener('abort', onAbort, { once: true })
this.pending.set(request.requestId, {
resolve,
reject,
timer,
detachAbort: () =>
signal?.removeEventListener('abort', onAbort)
})
if (signal?.aborted) {
cancel('OCR 解析已取消')
return
}
this.window.webContents.send(
ipcChannels.documentParsingOcrRequest,
request
)
})
}
respond(input: unknown): void {
const result = documentOcrResultSchema.safeParse(input)
const failure = result.success
? undefined
: documentOcrFailureSchema.safeParse(input)
const requestId = result.success
? result.data.requestId
: failure?.success
? failure.data.requestId
: undefined
if (!requestId) {
throw new Error('OCR 响应无效')
}
const pending = this.pending.get(requestId)
if (!pending) {
return
}
clearTimeout(pending.timer)
pending.detachAbort()
this.pending.delete(requestId)
if (result.success) {
pending.resolve(result.data)
} else {
if (!failure?.success) {
pending.reject(new Error('OCR 响应无效'))
return
}
pending.reject(new Error(failure.data.error))
}
}
dispose(): void {
this.disposed = true
for (const pending of this.pending.values()) {
clearTimeout(pending.timer)
pending.detachAbort()
pending.reject(new Error('OCR 解析已取消'))
}
this.pending.clear()
}
}
+204
View File
@@ -0,0 +1,204 @@
import {
documentOcrModelCatalogEntrySchema,
type DocumentOcrModelCatalogEntry
} from '../shared/document-parsing-contracts'
const detectionRevision =
'7d7f5d128d9309ebf6de4f21f404dd583afdbae3'
const recognitionRevision =
'afba04b618200c5f4824531c6e42c957c6439d9a'
const smallDetectionRevision =
'956a0b620a4017cc04056c692be1703b0025d028'
const smallRecognitionRevision =
'296d43bc0ebced0fd9c605174aa5962e49810ab6'
const mediumDetectionRevision =
'c317b40325be40bfaaff58c8dcece2a075294f8a'
const mediumRecognitionRevision =
'db5d610d492a14e3c34dc1fd4e9339bd369f79e6'
export const DOCUMENT_OCR_MODEL_CATALOG: readonly DocumentOcrModelCatalogEntry[] =
documentOcrModelCatalogEntrySchema.array().parse([
{
id: 'pp-ocrv6-tiny',
displayName: 'PP-OCRv6 Tiny',
description:
'PaddleOCR 官方轻量中文 OCR 模型,适合扫描 PDF 和图片的本地 CPU 识别。',
languages: ['中文', '英语'],
runtime: 'onnxruntime-web-wasm',
quality: 'basic',
speed: 'fast',
recommended: false,
repositoryUrl:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_tiny_rec_onnx',
license: {
name: 'Apache License 2.0',
notice:
'检测与识别模型由 PaddlePaddle 在 ModelScope 发布,使用前请阅读模型仓库及 PaddleOCR 的许可证说明。',
url: 'https://github.com/PaddlePaddle/PaddleOCR/blob/main/LICENSE'
},
files: [
{
name: 'detection.onnx',
role: 'detection',
download: {
url:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_tiny_det_onnx/resolve/' +
`${detectionRevision}/inference.onnx`,
size: 1_780_590,
sha256:
'193bab7a04fca699a6c82e6abb5b81bdb28177f0abd4062552b04908dafb19f8'
}
},
{
name: 'recognition.onnx',
role: 'recognition',
download: {
url:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_tiny_rec_onnx/resolve/' +
`${recognitionRevision}/inference.onnx`,
size: 4_462_639,
sha256:
'9ef676d6ed3c88256a2d92c640c44f25b0c40947e111b14b8be8f594091563e6'
}
},
{
name: 'dictionary.yml',
role: 'dictionary',
download: {
url:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_tiny_rec_onnx/resolve/' +
`${recognitionRevision}/inference.yml`,
size: 55_571,
sha256:
'66170210bad538e83fff3c4a3867e547d6bf20b50d64b20347c4b913f3034ea1'
}
}
]
},
{
id: 'pp-ocrv6-small',
displayName: 'PP-OCRv6 Small',
description:
'PaddleOCR 官方 50 语言 OCR 模型,在识别质量、速度和本地资源占用之间取得平衡。',
languages: ['50 种语言'],
runtime: 'onnxruntime-web-wasm',
quality: 'balanced',
speed: 'balanced',
recommended: true,
repositoryUrl:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_small_rec_onnx',
license: {
name: 'Apache License 2.0',
notice:
'检测与识别模型由 PaddlePaddle 在 ModelScope 发布,使用前请阅读模型仓库及 PaddleOCR 的许可证说明。',
url: 'https://github.com/PaddlePaddle/PaddleOCR/blob/main/LICENSE'
},
files: [
{
name: 'detection.onnx',
role: 'detection',
download: {
url:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_small_det_onnx/resolve/' +
`${smallDetectionRevision}/inference.onnx`,
size: 9_880_512,
sha256:
'd73e0058b7a8086bbd57f3d10b8bcd4ff95363f67e06e2762b5e814fe9c9410e'
}
},
{
name: 'recognition.onnx',
role: 'recognition',
download: {
url:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_small_rec_onnx/resolve/' +
`${smallRecognitionRevision}/inference.onnx`,
size: 21_159_378,
sha256:
'5435fd747c9e0efe15a96d0b378d5bd157e9492ed8fd80edf08f30d02fa24634'
}
},
{
name: 'dictionary.yml',
role: 'dictionary',
download: {
url:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_small_rec_onnx/resolve/' +
`${smallRecognitionRevision}/inference.yml`,
size: 150_579,
sha256:
'ab078671bb49f06228eadccd34f1bb501e157f7a047095ffb943ba81512c77d1'
}
}
]
},
{
id: 'pp-ocrv6-medium',
displayName: 'PP-OCRv6 Medium',
description:
'PaddleOCR 官方 50 语言高质量 OCR 模型,识别较慢,并需要更多内存且具有更高延迟。',
languages: ['50 种语言'],
runtime: 'onnxruntime-web-wasm',
quality: 'high',
speed: 'slow',
recommended: false,
repositoryUrl:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_medium_rec_onnx',
license: {
name: 'Apache License 2.0',
notice:
'检测与识别模型由 PaddlePaddle 在 ModelScope 发布,使用前请阅读模型仓库及 PaddleOCR 的许可证说明。',
url: 'https://github.com/PaddlePaddle/PaddleOCR/blob/main/LICENSE'
},
files: [
{
name: 'detection.onnx',
role: 'detection',
download: {
url:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_medium_det_onnx/resolve/' +
`${mediumDetectionRevision}/inference.onnx`,
size: 62_032_837,
sha256:
'eb13b44b25bb36f89528b68720af8a61d9cf381176107f465db1757b65d086e1'
}
},
{
name: 'recognition.onnx',
role: 'recognition',
download: {
url:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_medium_rec_onnx/resolve/' +
`${mediumRecognitionRevision}/inference.onnx`,
size: 76_554_979,
sha256:
'9c09abf0957f7968c7586464b7397b84ad2387a0497a351af40e9acc71b673ba'
}
},
{
name: 'dictionary.yml',
role: 'dictionary',
download: {
url:
'https://modelscope.cn/models/PaddlePaddle/' +
'PP-OCRv6_medium_rec_onnx/resolve/' +
`${mediumRecognitionRevision}/inference.yml`,
size: 150_580,
sha256:
'991b700facf5b50a7de193468207d5f4255b538dde0d312ae3b7c7a9b6873129'
}
}
]
}
])
+341
View File
@@ -0,0 +1,341 @@
import { createHash } from 'node:crypto'
import {
mkdtemp,
mkdir,
rm,
writeFile
} from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { DocumentOcrModelCatalogEntry } from '../shared/document-parsing-contracts'
import { DOCUMENT_OCR_MODEL_CATALOG } from './document-ocr-model-catalog'
import {
DocumentOcrModelManager,
extractPaddleCharacterDictionary
} from './document-ocr-model-manager'
const temporaryDirectories: string[] = []
function sha256(value: Uint8Array): string {
return createHash('sha256').update(value).digest('hex')
}
function dictionaryYaml(): Uint8Array {
const characters = [
"'!'",
"'\"'",
"''''",
...Array.from({ length: 120 }, (_, index) =>
String.fromCodePoint(0x4e00 + index)
)
]
return Buffer.from(
`PostProcess:\n name: CTCLabelDecode\n character_dict:\n${characters
.map((character) => ` - ${character}`)
.join('\n')}\n`,
'utf8'
)
}
function catalog(
detection: Uint8Array,
recognition: Uint8Array,
dictionary: Uint8Array
): readonly DocumentOcrModelCatalogEntry[] {
const files = [
{
name: 'detection.onnx',
role: 'detection' as const,
bytes: detection
},
{
name: 'recognition.onnx',
role: 'recognition' as const,
bytes: recognition
},
{
name: 'dictionary.yml',
role: 'dictionary' as const,
bytes: dictionary
}
]
return [
{
id: 'pp-ocrv6-tiny',
displayName: 'PP-OCRv6 Tiny',
description: 'Test OCR model catalog entry.',
languages: ['中文', '英语'],
runtime: 'onnxruntime-web-wasm',
quality: 'balanced',
speed: 'fast',
recommended: true,
repositoryUrl:
'https://modelscope.cn/models/PaddlePaddle/PP-OCRv6_tiny_rec_onnx',
license: {
name: 'Apache License 2.0',
notice: 'Test license notice.',
url: 'https://example.com/license'
},
files: files.map((file) => ({
name: file.name,
role: file.role,
download: {
url: `https://modelscope.cn/models/example/resolve/revision/${file.name}`,
size: file.bytes.byteLength,
sha256: sha256(file.bytes)
}
}))
}
]
}
async function createManager(
bytes?: {
detection: Uint8Array
recognition: Uint8Array
dictionary: Uint8Array
}
): Promise<{
directory: string
manager: DocumentOcrModelManager
modelBytes: {
detection: Uint8Array
recognition: Uint8Array
dictionary: Uint8Array
}
}> {
const directory = await mkdtemp(
join(tmpdir(), 'goodbuddy-document-ocr-model-')
)
temporaryDirectories.push(directory)
const modelBytes = bytes ?? {
detection: Buffer.from('detection model'),
recognition: Buffer.from('recognition model'),
dictionary: dictionaryYaml()
}
const testCatalog = catalog(
modelBytes.detection,
modelBytes.recognition,
modelBytes.dictionary
)
const entry = testCatalog[0]
if (!entry) {
throw new Error('Test OCR catalog is empty')
}
const files = new Map(
entry.files.map((file) => [
file.download.url,
modelBytes[file.role]
])
)
const transport = vi.fn(async (input: string | URL | Request) => {
const url =
input instanceof Request ? input.url : input.toString()
const body = files.get(url)
if (!body) {
return new Response(null, { status: 404 })
}
return new Response(body, {
status: 200,
headers: {
'content-length': String(body.byteLength)
}
})
}) as unknown as typeof fetch
return {
directory,
manager: new DocumentOcrModelManager({
userDataDirectory: directory,
fetch: transport,
catalog: testCatalog
}),
modelBytes
}
}
afterEach(async () => {
await Promise.all(
temporaryDirectories.splice(0).map((directory) =>
rm(directory, { recursive: true, force: true })
)
)
})
describe('DocumentOcrModelManager', () => {
it('uses immutable SHA-256 verified ModelScope catalog files', () => {
expect(DOCUMENT_OCR_MODEL_CATALOG).toHaveLength(3)
expect(
new Set(DOCUMENT_OCR_MODEL_CATALOG.map((entry) => entry.id)).size
).toBe(3)
expect(
DOCUMENT_OCR_MODEL_CATALOG.filter((entry) => entry.recommended).map(
(entry) => entry.id
)
).toEqual(['pp-ocrv6-small'])
for (const entry of DOCUMENT_OCR_MODEL_CATALOG) {
for (const file of entry.files) {
expect(file.download.url).toMatch(
/^https:\/\/modelscope\.cn\/models\/PaddlePaddle\/[^/]+\/resolve\/[a-f0-9]{40}\/[^/]+$/u
)
expect(file.download.sha256).toMatch(/^[a-f0-9]{64}$/u)
expect(file.download.size).toBeGreaterThan(0)
}
}
expect(
DOCUMENT_OCR_MODEL_CATALOG.find(
(entry) => entry.id === 'pp-ocrv6-small'
)
).toMatchObject({
languages: ['50 种语言'],
quality: 'balanced',
speed: 'balanced',
recommended: true,
files: [
{
role: 'detection',
download: {
url: 'https://modelscope.cn/models/PaddlePaddle/PP-OCRv6_small_det_onnx/resolve/956a0b620a4017cc04056c692be1703b0025d028/inference.onnx',
size: 9_880_512,
sha256:
'd73e0058b7a8086bbd57f3d10b8bcd4ff95363f67e06e2762b5e814fe9c9410e'
}
},
{
role: 'recognition',
download: {
url: 'https://modelscope.cn/models/PaddlePaddle/PP-OCRv6_small_rec_onnx/resolve/296d43bc0ebced0fd9c605174aa5962e49810ab6/inference.onnx',
size: 21_159_378,
sha256:
'5435fd747c9e0efe15a96d0b378d5bd157e9492ed8fd80edf08f30d02fa24634'
}
},
{
role: 'dictionary',
download: {
url: 'https://modelscope.cn/models/PaddlePaddle/PP-OCRv6_small_rec_onnx/resolve/296d43bc0ebced0fd9c605174aa5962e49810ab6/inference.yml',
size: 150_579,
sha256:
'ab078671bb49f06228eadccd34f1bb501e157f7a047095ffb943ba81512c77d1'
}
}
]
})
expect(
DOCUMENT_OCR_MODEL_CATALOG.find(
(entry) => entry.id === 'pp-ocrv6-medium'
)
).toMatchObject({
languages: ['50 种语言'],
quality: 'high',
speed: 'slow',
recommended: false,
files: [
{
role: 'detection',
download: {
url: 'https://modelscope.cn/models/PaddlePaddle/PP-OCRv6_medium_det_onnx/resolve/c317b40325be40bfaaff58c8dcece2a075294f8a/inference.onnx',
size: 62_032_837,
sha256:
'eb13b44b25bb36f89528b68720af8a61d9cf381176107f465db1757b65d086e1'
}
},
{
role: 'recognition',
download: {
url: 'https://modelscope.cn/models/PaddlePaddle/PP-OCRv6_medium_rec_onnx/resolve/db5d610d492a14e3c34dc1fd4e9339bd369f79e6/inference.onnx',
size: 76_554_979,
sha256:
'9c09abf0957f7968c7586464b7397b84ad2387a0497a351af40e9acc71b673ba'
}
},
{
role: 'dictionary',
download: {
url: 'https://modelscope.cn/models/PaddlePaddle/PP-OCRv6_medium_rec_onnx/resolve/db5d610d492a14e3c34dc1fd4e9339bd369f79e6/inference.yml',
size: 150_580,
sha256:
'991b700facf5b50a7de193468207d5f4255b538dde0d312ae3b7c7a9b6873129'
}
}
]
})
})
it('downloads, verifies, and loads OCR assets', async () => {
const { manager, modelBytes } = await createManager()
await expect(manager.install('pp-ocrv6-tiny')).resolves.toMatchObject({
id: 'pp-ocrv6-tiny',
source: 'download'
})
await expect(manager.getStatus('pp-ocrv6-tiny')).resolves.toMatchObject({
available: true,
verified: true
})
const assets = await manager.getAssets('pp-ocrv6-tiny')
expect(new Uint8Array(assets.detection)).toEqual(
Uint8Array.from(modelBytes.detection)
)
expect(new Uint8Array(assets.recognition)).toEqual(
Uint8Array.from(modelBytes.recognition)
)
expect(new TextDecoder().decode(assets.dictionary)).toContain(
"!\n\"\n'\n"
)
})
it('rejects an imported model whose hash does not match', async () => {
const { directory, manager, modelBytes } = await createManager()
const source = join(directory, 'manual-model')
await mkdir(source)
await Promise.all([
writeFile(join(source, 'detection.onnx'), modelBytes.detection),
writeFile(join(source, 'recognition.onnx'), modelBytes.recognition),
writeFile(join(source, 'dictionary.yml'), 'tampered')
])
await expect(
manager.registerLocalDirectory('pp-ocrv6-tiny', source)
).rejects.toThrow('校验失败')
await expect(manager.getSnapshot()).resolves.toMatchObject({
installed: [],
operations: []
})
})
it('round-trips a verified OCR model through an offline ZIP archive', async () => {
const { directory, manager, modelBytes } = await createManager()
const archive = join(directory, 'ocr-model.zip')
await manager.install('pp-ocrv6-tiny')
await manager.exportArchive('pp-ocrv6-tiny', archive)
await manager.remove('pp-ocrv6-tiny')
await expect(
manager.importArchive('pp-ocrv6-tiny', archive)
).resolves.toMatchObject({
id: 'pp-ocrv6-tiny',
source: 'local'
})
const assets = await manager.getAssets('pp-ocrv6-tiny')
expect(new Uint8Array(assets.detection)).toEqual(
Uint8Array.from(modelBytes.detection)
)
expect(new Uint8Array(assets.recognition)).toEqual(
Uint8Array.from(modelBytes.recognition)
)
})
})
describe('extractPaddleCharacterDictionary', () => {
it('converts Paddle YAML scalars into the line dictionary used by OCR', () => {
const dictionary = extractPaddleCharacterDictionary(
new TextDecoder().decode(dictionaryYaml())
)
expect(dictionary.startsWith("!\n\"\n'\n")).toBe(true)
expect(dictionary.split('\n')).toHaveLength(124)
})
})
+899
View File
@@ -0,0 +1,899 @@
import { createHash, randomUUID } from 'node:crypto'
import {
copyFile,
lstat,
mkdir,
open,
readFile,
readdir,
rename,
rm,
stat,
writeFile
} from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import {
documentOcrAssetsSchema,
documentOcrModelCatalogEntrySchema,
documentOcrModelSnapshotSchema,
documentParsingModelStatusSchema,
installedDocumentOcrModelSchema,
localOcrModelIdSchema,
type DocumentOcrAssets,
type DocumentOcrModelCatalogEntry,
type DocumentOcrModelFile,
type DocumentOcrModelOperation,
type DocumentOcrModelSnapshot,
type InstalledDocumentOcrModel
} from '../shared/document-parsing-contracts'
import { DOCUMENT_OCR_MODEL_CATALOG } from './document-ocr-model-catalog'
import {
exportModelArchive,
extractModelArchive
} from './model-archive'
const DEFAULT_MAX_FILE_BYTES = 96 * 1024 * 1024
const MANIFEST_FILE_NAME = 'manifest.json'
const MAX_REDIRECTS = 3
const PARTIAL_SUFFIX = '.partial'
const MAXIMUM_ARCHIVE_BYTES = 512 * 1024 * 1024
const ARCHIVE_OVERHEAD_BYTES = 1024 * 1024
const executableExtensionPattern =
/\.(?:app|bat|bin|cmd|com|cpl|dll|dmg|exe|hta|inf|ins|iso|jar|js|jse|lnk|msi|msp|mst|pif|ps1|reg|scr|sh|sys|vb|vbe|vbs|ws|wsc|wsf|wsh)$/iu
type ActiveOperation = {
controller: AbortController
progress: DocumentOcrModelOperation
}
export type DocumentOcrModelManagerOptions = {
userDataDirectory: string
fetch: typeof fetch
catalog?: readonly DocumentOcrModelCatalogEntry[]
maxFileBytes?: number
}
function abortError(): DOMException {
return new DOMException('The operation was aborted', 'AbortError')
}
function ensureNotAborted(signal: AbortSignal): void {
if (signal.aborted) {
throw abortError()
}
}
function cloneCatalogEntry(
entry: DocumentOcrModelCatalogEntry
): DocumentOcrModelCatalogEntry {
return documentOcrModelCatalogEntrySchema.parse(entry)
}
function safeChild(parent: string, name: string): string {
const child = resolve(parent, name)
if (dirname(child) !== resolve(parent)) {
throw new Error('OCR 模型路径超出受管目录')
}
return child
}
function validateDownloadUrl(value: string): URL {
const url = new URL(value)
if (url.protocol !== 'http:' && url.protocol !== 'https:') {
throw new Error('OCR 模型下载地址必须使用 HTTP 或 HTTPS')
}
return url
}
function toArrayBuffer(buffer: Buffer): ArrayBuffer {
return Uint8Array.from(buffer).buffer
}
async function hashFile(
path: string,
signal?: AbortSignal
): Promise<{ size: number; sha256: string }> {
const handle = await open(path, 'r')
const hash = createHash('sha256')
const buffer = Buffer.allocUnsafe(64 * 1024)
let size = 0
try {
while (true) {
if (signal) {
ensureNotAborted(signal)
}
const { bytesRead } = await handle.read(buffer, 0, buffer.length)
if (bytesRead === 0) {
break
}
hash.update(buffer.subarray(0, bytesRead))
size += bytesRead
}
} finally {
await handle.close()
}
return { size, sha256: hash.digest('hex') }
}
function parseYamlScalar(value: string): string {
if (value.startsWith("'") && value.endsWith("'")) {
return value.slice(1, -1).replace(/''/gu, "'")
}
if (value.startsWith('"') && value.endsWith('"')) {
return JSON.parse(value) as string
}
return value
}
export function extractPaddleCharacterDictionary(source: string): string {
const characters: string[] = []
let readingDictionary = false
for (const line of source.replace(/\r/gu, '').split('\n')) {
if (line === ' character_dict:') {
readingDictionary = true
continue
}
if (!readingDictionary) {
continue
}
const match = /^ {2}- (.*)$/u.exec(line)
if (!match) {
break
}
const character = parseYamlScalar(match[1]!)
if (!character) {
throw new Error('OCR 字符字典包含空条目')
}
characters.push(character)
}
if (characters.length < 100) {
throw new Error('OCR 字符字典格式无效')
}
return `${characters.join('\n')}\n`
}
export class DocumentOcrModelManager {
readonly rootDirectory: string
private readonly transport: typeof fetch
private readonly catalog: DocumentOcrModelCatalogEntry[]
private readonly maxFileBytes: number
private readonly operations = new Map<string, ActiveOperation>()
private readonly verifiedModels = new Map<string, Promise<void>>()
constructor(options: DocumentOcrModelManagerOptions) {
if (!options.userDataDirectory.trim()) {
throw new Error('userDataDirectory is required')
}
this.rootDirectory = resolve(
options.userDataDirectory,
'models',
'document-ocr'
)
this.transport = options.fetch
this.catalog = (options.catalog ?? DOCUMENT_OCR_MODEL_CATALOG).map(
cloneCatalogEntry
)
if (
new Set(this.catalog.map((entry) => entry.id)).size !==
this.catalog.length
) {
throw new Error('OCR 模型目录包含重复 ID')
}
this.maxFileBytes = options.maxFileBytes ?? DEFAULT_MAX_FILE_BYTES
if (
!Number.isSafeInteger(this.maxFileBytes) ||
this.maxFileBytes <= 0 ||
this.maxFileBytes > 512 * 1024 * 1024
) {
throw new RangeError('maxFileBytes must be a positive safe integer')
}
}
async getSnapshot(): Promise<DocumentOcrModelSnapshot> {
await this.ensureRoot()
return documentOcrModelSnapshotSchema.parse({
rootDirectory: this.rootDirectory,
catalog: this.catalog.map(cloneCatalogEntry),
installed: await this.readInstalled(),
operations: [...this.operations.values()].map((operation) => ({
...operation.progress
}))
})
}
async getStatus(
modelId: string
): Promise<ReturnType<typeof documentParsingModelStatusSchema.parse>> {
const entry = this.requireCatalogEntry(modelId)
try {
await this.getVerifiedStatus(entry)
return documentParsingModelStatusSchema.parse({
id: entry.id,
displayName: entry.displayName,
available: true,
verified: true,
runtime: entry.runtime,
detail: '模型已安装并通过 SHA-256 校验,可离线使用'
})
} catch {
return documentParsingModelStatusSchema.parse({
id: entry.id,
displayName: entry.displayName,
available: false,
verified: false,
runtime: entry.runtime,
detail: '模型尚未安装或校验失败,请从 ModelScope 下载'
})
}
}
getAssets(modelId: string): Promise<DocumentOcrAssets> {
return this.loadVerifiedAssets(this.requireCatalogEntry(modelId))
}
async install(
modelId: string,
externalSignal?: AbortSignal
): Promise<InstalledDocumentOcrModel> {
const entry = this.requireCatalogEntry(modelId)
const totalBytes = entry.files.reduce(
(total, file) => total + file.download.size,
0
)
if (!Number.isSafeInteger(totalBytes)) {
throw new RangeError('OCR 模型总大小超出安全范围')
}
const operation = this.beginOperation(entry.id, 'download', totalBytes)
const detachAbort = this.attachExternalSignal(
externalSignal,
operation.controller
)
let stagingDirectory: string | undefined
try {
await this.ensureRoot()
await this.assertNotInstalled(entry.id)
stagingDirectory = await this.createStagingDirectory(entry.id)
for (const file of entry.files) {
ensureNotAborted(operation.controller.signal)
operation.progress.phase = 'transferring'
operation.progress.currentFile = file.name
await this.downloadFile(
file,
safeChild(stagingDirectory, file.name),
operation,
operation.controller.signal
)
}
operation.progress.phase = 'installing'
operation.progress.currentFile = null
const installed = await this.createInstalledManifest(
entry,
'download',
stagingDirectory,
operation.controller.signal
)
ensureNotAborted(operation.controller.signal)
await rename(stagingDirectory, this.modelDirectory(entry.id))
stagingDirectory = undefined
this.verifiedModels.delete(entry.id)
return installed
} finally {
detachAbort()
this.operations.delete(entry.id)
if (stagingDirectory) {
await rm(stagingDirectory, { recursive: true, force: true })
}
}
}
async registerLocalDirectory(
modelId: string,
sourceDirectory: string,
externalSignal?: AbortSignal
): Promise<InstalledDocumentOcrModel> {
const entry = this.requireCatalogEntry(modelId)
const source = resolve(sourceDirectory)
const operation = this.beginOperation(entry.id, 'import', null)
const detachAbort = this.attachExternalSignal(
externalSignal,
operation.controller
)
let stagingDirectory: string | undefined
try {
await this.ensureRoot()
await this.assertNotInstalled(entry.id)
await this.validateLocalDirectory(
source,
entry,
operation.controller.signal
)
stagingDirectory = await this.createStagingDirectory(entry.id)
operation.progress.phase = 'transferring'
for (const file of entry.files) {
ensureNotAborted(operation.controller.signal)
operation.progress.currentFile = file.name
const sourceFile = safeChild(source, file.name)
const destination = safeChild(stagingDirectory, file.name)
await copyFile(sourceFile, destination)
operation.progress.completedBytes +=
(await stat(destination)).size
}
operation.progress.totalBytes =
operation.progress.completedBytes
operation.progress.phase = 'installing'
operation.progress.currentFile = null
const installed = await this.createInstalledManifest(
entry,
'local',
stagingDirectory,
operation.controller.signal
)
ensureNotAborted(operation.controller.signal)
await rename(stagingDirectory, this.modelDirectory(entry.id))
stagingDirectory = undefined
this.verifiedModels.delete(entry.id)
return installed
} finally {
detachAbort()
this.operations.delete(entry.id)
if (stagingDirectory) {
await rm(stagingDirectory, { recursive: true, force: true })
}
}
}
async exportArchive(
modelId: string,
destinationPath: string
): Promise<void> {
const entry = this.requireCatalogEntry(modelId)
await this.ensureRoot()
const installed = (await this.readInstalled()).find(
(model) => model.id === entry.id
)
if (!installed) {
throw new Error('只能导出已安装的 OCR 模型')
}
const directory = this.modelDirectory(entry.id)
const files = []
for (const expected of entry.files) {
const recorded = installed.files.find(
(file) =>
file.name === expected.name &&
file.role === expected.role
)
if (
!recorded ||
recorded.size !== expected.download.size ||
recorded.sha256 !== expected.download.sha256
) {
throw new Error(`OCR 模型文件校验失败:${expected.name}`)
}
files.push({
name: expected.name,
role: expected.role,
size: recorded.size,
sha256: recorded.sha256
})
}
await exportModelArchive({
destinationPath,
sourceDirectory: directory,
descriptor: {
kind: 'document-ocr',
modelId: entry.id,
displayName: entry.displayName,
files
}
})
}
async importArchive(
modelId: string,
archivePath: string
): Promise<InstalledDocumentOcrModel> {
const entry = this.requireCatalogEntry(modelId)
const expectedTotal = entry.files.reduce(
(total, file) => total + file.download.size,
0
)
const operation = this.beginOperation(
entry.id,
'import',
expectedTotal
)
let stagingDirectory: string | undefined
try {
await this.ensureRoot()
await this.assertNotInstalled(entry.id)
stagingDirectory = await this.createStagingDirectory(entry.id)
operation.progress.phase = 'transferring'
const descriptor = await extractModelArchive({
archivePath,
destinationDirectory: stagingDirectory,
expectedKind: 'document-ocr',
expectedModelId: entry.id,
expectedFiles: entry.files.map((file) => ({
name: file.name,
role: file.role
})),
maximumArchiveBytes: Math.min(
MAXIMUM_ARCHIVE_BYTES,
expectedTotal + ARCHIVE_OVERHEAD_BYTES
),
maximumFileBytes: this.maxFileBytes,
maximumTotalBytes: expectedTotal + ARCHIVE_OVERHEAD_BYTES,
signal: operation.controller.signal,
onProgress: (completedBytes) => {
operation.progress.completedBytes = completedBytes
}
})
for (const expected of entry.files) {
const archived = descriptor.files.find(
(file) =>
file.name === expected.name &&
file.role === expected.role
)
if (
!archived ||
archived.size !== expected.download.size ||
archived.sha256 !== expected.download.sha256
) {
throw new Error(
`OCR 模型 ZIP 与当前模型目录不匹配:${expected.name}`
)
}
}
operation.progress.phase = 'installing'
operation.progress.currentFile = null
const installed = installedDocumentOcrModelSchema.parse({
id: entry.id,
displayName: entry.displayName,
source: 'local',
installedAt: new Date().toISOString(),
files: descriptor.files
})
await writeFile(
safeChild(stagingDirectory, MANIFEST_FILE_NAME),
`${JSON.stringify(installed, null, 2)}\n`,
{ encoding: 'utf8', flag: 'wx' }
)
ensureNotAborted(operation.controller.signal)
await rename(stagingDirectory, this.modelDirectory(entry.id))
stagingDirectory = undefined
this.verifiedModels.delete(entry.id)
return installed
} finally {
this.operations.delete(entry.id)
if (stagingDirectory) {
await rm(stagingDirectory, { recursive: true, force: true })
}
}
}
cancel(modelId: string): boolean {
const id = localOcrModelIdSchema.parse(modelId)
const operation = this.operations.get(id)
if (!operation) {
return false
}
operation.controller.abort()
return true
}
async remove(modelId: string): Promise<void> {
const id = localOcrModelIdSchema.parse(modelId)
this.cancel(id)
this.verifiedModels.delete(id)
await rm(this.modelDirectory(id), {
recursive: true,
force: true
})
}
dispose(): void {
for (const operation of this.operations.values()) {
operation.controller.abort()
}
this.operations.clear()
this.verifiedModels.clear()
}
private async ensureRoot(): Promise<void> {
await mkdir(this.rootDirectory, { recursive: true })
}
private modelDirectory(modelId: string): string {
return safeChild(
this.rootDirectory,
localOcrModelIdSchema.parse(modelId)
)
}
private requireCatalogEntry(
modelId: string
): DocumentOcrModelCatalogEntry {
const id = localOcrModelIdSchema.parse(modelId)
const entry = this.catalog.find((candidate) => candidate.id === id)
if (!entry) {
throw new Error('未知的 OCR 模型')
}
return entry
}
private beginOperation(
modelId: string,
kind: DocumentOcrModelOperation['kind'],
totalBytes: number | null
): ActiveOperation {
if (this.operations.has(modelId)) {
throw new Error('该 OCR 模型已有进行中的操作')
}
const operation: ActiveOperation = {
controller: new AbortController(),
progress: {
modelId: localOcrModelIdSchema.parse(modelId),
kind,
phase: 'preparing',
currentFile: null,
completedBytes: 0,
totalBytes
}
}
this.operations.set(modelId, operation)
return operation
}
private attachExternalSignal(
signal: AbortSignal | undefined,
controller: AbortController
): () => void {
if (!signal) {
return () => undefined
}
const abort = (): void => controller.abort()
if (signal.aborted) {
controller.abort()
} else {
signal.addEventListener('abort', abort, { once: true })
}
return () => signal.removeEventListener('abort', abort)
}
private async assertNotInstalled(modelId: string): Promise<void> {
try {
await lstat(this.modelDirectory(modelId))
throw new Error('OCR 模型已安装')
} catch (error) {
if (
error instanceof Error &&
'code' in error &&
error.code === 'ENOENT'
) {
return
}
throw error
}
}
private async createStagingDirectory(modelId: string): Promise<string> {
const directory = safeChild(
this.rootDirectory,
`.install-${modelId}-${randomUUID()}`
)
await mkdir(directory, { recursive: false })
return directory
}
private async fetchFollowingRedirects(
initialUrl: string,
signal: AbortSignal
): Promise<Response> {
let url = validateDownloadUrl(initialUrl)
for (let redirectCount = 0; ; redirectCount += 1) {
ensureNotAborted(signal)
const response = await this.transport(url, {
method: 'GET',
redirect: 'manual',
credentials: 'omit',
cache: 'no-store',
signal
})
if ([301, 302, 303, 307, 308].includes(response.status)) {
if (redirectCount >= MAX_REDIRECTS) {
await response.body?.cancel().catch(() => undefined)
throw new Error('OCR 模型下载重定向次数过多')
}
const location = response.headers.get('location')
await response.body?.cancel().catch(() => undefined)
if (!location) {
throw new Error('OCR 模型下载重定向缺少地址')
}
url = validateDownloadUrl(new URL(location, url).toString())
continue
}
return response
}
}
private async downloadFile(
file: DocumentOcrModelFile,
destination: string,
operation: ActiveOperation,
signal: AbortSignal
): Promise<void> {
if (file.download.size > this.maxFileBytes) {
throw new RangeError(`OCR 模型文件过大:${file.name}`)
}
const response = await this.fetchFollowingRedirects(
file.download.url,
signal
)
if (!response.ok) {
await response.body?.cancel().catch(() => undefined)
throw new Error(`OCR 模型下载失败:HTTP ${response.status}`)
}
if (!response.body) {
throw new Error('OCR 模型下载响应没有内容')
}
const declaredLength = response.headers.get('content-length')
if (
declaredLength !== null &&
Number(declaredLength) !== file.download.size
) {
await response.body.cancel().catch(() => undefined)
throw new Error(`OCR 模型文件大小不匹配:${file.name}`)
}
const partialPath = `${destination}${PARTIAL_SUFFIX}`
const handle = await open(partialPath, 'wx')
const reader = response.body.getReader()
const hash = createHash('sha256')
let written = 0
try {
while (true) {
ensureNotAborted(signal)
const result = await reader.read()
if (result.done) {
break
}
written += result.value.byteLength
if (
written > file.download.size ||
written > this.maxFileBytes
) {
await reader.cancel()
throw new RangeError(`OCR 模型文件过大:${file.name}`)
}
await handle.write(result.value)
hash.update(result.value)
operation.progress.completedBytes += result.value.byteLength
}
} catch (error) {
await reader.cancel().catch(() => undefined)
throw error
} finally {
await handle.close()
}
if (
written !== file.download.size ||
hash.digest('hex') !== file.download.sha256
) {
throw new Error(`OCR 模型文件校验失败:${file.name}`)
}
await rename(partialPath, destination)
}
private async validateLocalDirectory(
sourceDirectory: string,
entry: DocumentOcrModelCatalogEntry,
signal: AbortSignal
): Promise<void> {
const sourceInfo = await lstat(sourceDirectory)
if (!sourceInfo.isDirectory() || sourceInfo.isSymbolicLink()) {
throw new Error('本地 OCR 模型来源必须是普通目录')
}
const entries = await readdir(sourceDirectory, { withFileTypes: true })
for (const localEntry of entries) {
ensureNotAborted(signal)
if (
localEntry.isSymbolicLink() ||
executableExtensionPattern.test(localEntry.name)
) {
throw new Error('本地 OCR 模型目录包含不安全文件')
}
}
for (const file of entry.files) {
ensureNotAborted(signal)
const path = safeChild(sourceDirectory, file.name)
const info = await lstat(path)
if (!info.isFile() || info.isSymbolicLink()) {
throw new Error(`OCR 模型文件必须是普通文件:${file.name}`)
}
const actual = await hashFile(path, signal)
if (
actual.size !== file.download.size ||
actual.sha256 !== file.download.sha256
) {
throw new Error(`本地 OCR 模型文件校验失败:${file.name}`)
}
}
}
private async createInstalledManifest(
entry: DocumentOcrModelCatalogEntry,
source: InstalledDocumentOcrModel['source'],
stagingDirectory: string,
signal: AbortSignal
): Promise<InstalledDocumentOcrModel> {
const files = []
for (const file of entry.files) {
ensureNotAborted(signal)
files.push({
name: file.name,
role: file.role,
...(await hashFile(
safeChild(stagingDirectory, file.name),
signal
))
})
}
const manifest = installedDocumentOcrModelSchema.parse({
id: entry.id,
displayName: entry.displayName,
source,
installedAt: new Date().toISOString(),
files
})
await writeFile(
safeChild(stagingDirectory, MANIFEST_FILE_NAME),
`${JSON.stringify(manifest, null, 2)}\n`,
{ encoding: 'utf8', flag: 'wx' }
)
return manifest
}
private async readInstalled(): Promise<InstalledDocumentOcrModel[]> {
const entries = await readdir(this.rootDirectory, {
withFileTypes: true
})
const installed: InstalledDocumentOcrModel[] = []
for (const entry of entries) {
if (
!entry.isDirectory() ||
entry.name.startsWith('.install-') ||
!localOcrModelIdSchema.safeParse(entry.name).success
) {
continue
}
try {
const manifest = installedDocumentOcrModelSchema.parse(
JSON.parse(
await readFile(
safeChild(
this.modelDirectory(entry.name),
MANIFEST_FILE_NAME
),
'utf8'
)
) as unknown
)
if (manifest.id === entry.name) {
installed.push(manifest)
}
} catch {
// Ignore incomplete or externally modified model directories.
}
}
return installed
}
private async readInstalledManifest(
entry: DocumentOcrModelCatalogEntry
): Promise<InstalledDocumentOcrModel> {
const directory = this.modelDirectory(entry.id)
const manifest = installedDocumentOcrModelSchema.parse(
JSON.parse(
await readFile(
safeChild(directory, MANIFEST_FILE_NAME),
'utf8'
)
) as unknown
)
if (manifest.id !== entry.id) {
throw new Error('OCR 模型清单 ID 不匹配')
}
return manifest
}
private async verifyInstalledModel(
entry: DocumentOcrModelCatalogEntry
): Promise<void> {
const directory = this.modelDirectory(entry.id)
const manifest = await this.readInstalledManifest(entry)
for (const file of entry.files) {
const installed = manifest.files.find(
(candidate) =>
candidate.name === file.name &&
candidate.role === file.role
)
const actual = await hashFile(safeChild(directory, file.name))
if (
!installed ||
actual.size !== file.download.size ||
actual.sha256 !== file.download.sha256 ||
actual.size !== installed.size ||
actual.sha256 !== installed.sha256
) {
throw new Error(`OCR 模型文件校验失败:${file.name}`)
}
}
}
private getVerifiedStatus(
entry: DocumentOcrModelCatalogEntry
): Promise<void> {
let verification = this.verifiedModels.get(entry.id)
if (!verification) {
verification = this.verifyInstalledModel(entry).catch((error) => {
this.verifiedModels.delete(entry.id)
throw error
})
this.verifiedModels.set(entry.id, verification)
}
return verification
}
private async loadVerifiedAssets(
entry: DocumentOcrModelCatalogEntry
): Promise<DocumentOcrAssets> {
const directory = this.modelDirectory(entry.id)
const manifest = await this.readInstalledManifest(entry)
const loaded = new Map<
DocumentOcrModelFile['role'],
ArrayBuffer
>()
for (const file of entry.files) {
const installed = manifest.files.find(
(candidate) =>
candidate.name === file.name &&
candidate.role === file.role
)
const path = safeChild(directory, file.name)
const contents = await readFile(path)
const actual = {
size: contents.byteLength,
sha256: createHash('sha256').update(contents).digest('hex')
}
if (
!installed ||
actual.size !== file.download.size ||
actual.sha256 !== file.download.sha256 ||
actual.size !== installed.size ||
actual.sha256 !== installed.sha256
) {
throw new Error(`OCR 模型文件校验失败:${file.name}`)
}
loaded.set(
file.role,
file.role === 'dictionary'
? toArrayBuffer(
Buffer.from(
extractPaddleCharacterDictionary(
contents.toString('utf8')
),
'utf8'
)
)
: toArrayBuffer(contents)
)
}
return documentOcrAssetsSchema.parse({
modelId: entry.id,
detection: loaded.get('detection'),
recognition: loaded.get('recognition'),
dictionary: loaded.get('dictionary')
})
}
}
+130
View File
@@ -0,0 +1,130 @@
import { describe, expect, it, vi } from 'vitest'
import {
defaultDocumentParsingSettings
} from './document-parsing-settings-store'
import { DocumentParsingService } from './document-parsing-service'
function createPdfFixture(text: string): Buffer {
const stream = `BT /F1 18 Tf 50 100 Td (${text}) Tj ET`
const objects = [
'<< /Type /Catalog /Pages 2 0 R >>',
'<< /Type /Pages /Kids [3 0 R] /Count 1 >>',
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 200] /Resources << /Font << /F1 4 0 R >> >> /Contents 5 0 R >>',
'<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>',
`<< /Length ${Buffer.byteLength(stream)} >>\nstream\n${stream}\nendstream`
]
let content = '%PDF-1.4\n'
const offsets = [0]
for (const [index, object] of objects.entries()) {
offsets.push(Buffer.byteLength(content))
content += `${index + 1} 0 obj\n${object}\nendobj\n`
}
const xrefOffset = Buffer.byteLength(content)
content += `xref\n0 ${objects.length + 1}\n`
content += '0000000000 65535 f \n'
content += offsets
.slice(1)
.map((offset) => `${String(offset).padStart(10, '0')} 00000 n \n`)
.join('')
content += `trailer\n<< /Size ${objects.length + 1} /Root 1 0 R >>\n`
content += `startxref\n${xrefOffset}\n%%EOF\n`
return Buffer.from(content)
}
function createService(overrides?: {
settings?: Partial<typeof defaultDocumentParsingSettings>
}) {
const settings = {
...defaultDocumentParsingSettings,
...overrides?.settings
}
const recognize = vi.fn(async () => ({
requestId: crypto.randomUUID(),
sections: [
{
locator: '第 1 页',
content: '扫描件识别正文',
confidence: 0.93
}
],
pageCount: 1,
warnings: []
}))
const service = new DocumentParsingService(
{
get: vi.fn(async () => settings),
update: vi.fn(async () => settings)
} as never,
{
getStatus: vi.fn(async () => ({
id: 'pp-ocrv6-tiny',
displayName: 'PP-OCRv6 Tiny',
available: true,
verified: true,
runtime: 'onnxruntime-web-wasm',
detail: '可用'
}))
} as never,
{ recognize } as never
)
return { recognize, service }
}
describe('DocumentParsingService', () => {
it('keeps useful PDF text local without invoking OCR', async () => {
const { recognize, service } = createService()
const parsed = await service.parse(
'native.pdf',
createPdfFixture('Native PDF body text'),
'knowledge-index'
)
expect(parsed.content).toContain('Native PDF body text')
expect(parsed.sections[0]?.method).toBe('native')
expect(recognize).not.toHaveBeenCalled()
})
it('uses OCR for a PDF without useful text', async () => {
const { recognize, service } = createService()
const parsed = await service.parse(
'scan.pdf',
createPdfFixture(''),
'chat-attachment'
)
expect(parsed.content).toBe('扫描件识别正文')
expect(parsed.sections).toEqual([
{
locator: '第 1 页',
content: '扫描件识别正文',
method: 'ocr',
confidence: 0.93
}
])
expect(recognize).toHaveBeenCalledWith(
expect.objectContaining({
fileName: 'scan.pdf',
modelId: 'pp-ocrv6-tiny',
mimeType: 'application/pdf',
pageNumbers: [1]
})
)
})
it('does not use OCR in a fast-text workflow', async () => {
const { recognize, service } = createService({
settings: { chatWorkflow: 'fast-text' }
})
await expect(
service.parse(
'scan.pdf',
createPdfFixture(''),
'chat-attachment'
)
).rejects.toThrow('未启用 OCR')
expect(recognize).not.toHaveBeenCalled()
})
})
+270
View File
@@ -0,0 +1,270 @@
import { extname } from 'node:path'
import {
documentParsingDiagnosticSchema,
documentParsingSnapshotSchema,
type DocumentParsingDiagnostic,
type DocumentParsingPurpose,
type DocumentParsingSettings,
type DocumentParsingSnapshot
} from '../shared/document-parsing-contracts'
import type { DocumentOcrBroker } from './document-ocr-broker'
import type { DocumentOcrModelManager } from './document-ocr-model-manager'
import type { DocumentParsingSettingsStore } from './document-parsing-settings-store'
import {
DocumentTextUnavailableError,
extractPdfTextPages,
parseDocument,
type ParsedDocument,
type ParsedSection,
type PdfTextPage
} from './knowledge/document-parser'
const minimumUsefulPdfCharacters = 12
const maximumReplacementCharacterRatio = 0.08
export type ParseDocumentForPurpose = (
name: string,
buffer: Buffer,
purpose: DocumentParsingPurpose,
signal?: AbortSignal
) => Promise<ParsedDocument>
function ensureNotAborted(signal?: AbortSignal): void {
if (signal?.aborted) {
throw signal.reason instanceof Error
? signal.reason
: new Error('文档解析已取消')
}
}
function hasUsefulText(content: string): boolean {
const compact = content.replace(/\s+/gu, '')
if (compact.length < minimumUsefulPdfCharacters) {
return false
}
const replacementCount = [...compact].filter(
(character) => character === '\uFFFD'
).length
return replacementCount / compact.length <=
maximumReplacementCharacterRatio
}
function effectiveOcrMode(
settings: DocumentParsingSettings,
purpose: DocumentParsingPurpose
): DocumentParsingSettings['pdfOcrMode'] {
if (
(purpose === 'chat-attachment' &&
settings.chatWorkflow === 'fast-text') ||
(purpose === 'knowledge-index' &&
settings.knowledgeWorkflow === 'fast-index')
) {
return 'disabled'
}
if (
(purpose === 'chat-attachment' &&
settings.chatWorkflow === 'high-fidelity') ||
(purpose === 'knowledge-index' &&
settings.knowledgeWorkflow === 'high-fidelity')
) {
return 'always'
}
return settings.pdfOcrMode
}
function buildPdfDocument(
name: string,
sections: ParsedSection[],
pageCount: number,
warnings: string[] = []
): ParsedDocument {
const content = sections
.map((section) => section.content)
.join('\n\n')
.slice(0, 5_000_000)
if (!content) {
throw new DocumentTextUnavailableError()
}
return {
title: name.replace(/\.[^.]+$/u, ''),
sourceFormat: '.pdf',
content,
sections,
pageCount,
warnings
}
}
function nativePdfSections(pages: PdfTextPage[]): ParsedSection[] {
return pages
.filter((page) => page.content.length > 0)
.map((page) => ({
locator: `${page.pageNumber}`,
content: page.content,
method: 'native' as const
}))
}
export class DocumentParsingService {
constructor(
private readonly settingsStore: DocumentParsingSettingsStore,
private readonly modelManager: DocumentOcrModelManager,
private readonly ocrBroker: DocumentOcrBroker
) {}
async snapshot(): Promise<DocumentParsingSnapshot> {
const settings = await this.settingsStore.get()
const [localOcr, ocrModels] = await Promise.all([
this.modelManager.getStatus(settings.localOcrModelId),
this.modelManager.getSnapshot()
])
return documentParsingSnapshotSchema.parse({
settings,
status: {
nativeParsingAvailable: true,
conversionAvailable: false,
localOcr
},
ocrModels
})
}
async update(input: unknown): Promise<DocumentParsingSnapshot> {
await this.settingsStore.update(input)
return this.snapshot()
}
parse: ParseDocumentForPurpose = async (
name,
buffer,
purpose,
signal
) => {
ensureNotAborted(signal)
if (extname(name).toLowerCase() !== '.pdf') {
return parseDocument(name, buffer)
}
const settings = await this.settingsStore.get()
const pages = await extractPdfTextPages(buffer)
ensureNotAborted(signal)
const mode = effectiveOcrMode(settings, purpose)
const pagesWithoutUsefulText = pages
.filter((page) => !hasUsefulText(page.content))
.map((page) => page.pageNumber)
const ocrPageNumbers =
mode === 'always'
? pages.map((page) => page.pageNumber)
: mode === 'auto'
? pagesWithoutUsefulText
: []
if (mode === 'disabled' || !settings.localOcrEnabled) {
const native = nativePdfSections(pages)
if (native.length > 0) {
return buildPdfDocument(
name,
native,
pages.length,
pagesWithoutUsefulText.length > 0
? ['部分页面没有有效文本,当前工作流未启用 OCR']
: []
)
}
throw new DocumentTextUnavailableError(
'PDF 没有可用文本层,当前工作流未启用 OCR'
)
}
if (ocrPageNumbers.length === 0) {
return buildPdfDocument(
name,
nativePdfSections(pages),
pages.length
)
}
if (pages.length > settings.maximumPages) {
throw new Error(
`PDF 共 ${pages.length} 页,超过本地 OCR 的 ${settings.maximumPages} 页限制`
)
}
const modelStatus = await this.modelManager.getStatus(
settings.localOcrModelId
)
if (!modelStatus.available || !modelStatus.verified) {
throw new Error(modelStatus.detail)
}
const ocrRequest = {
modelId: settings.localOcrModelId,
fileName: name,
mimeType: 'application/pdf' as const,
data: Uint8Array.from(buffer).buffer,
maximumPages: settings.maximumPages,
pageNumbers: ocrPageNumbers,
pageTimeoutSeconds: settings.pageTimeoutSeconds
}
const ocr = await (signal
? this.ocrBroker.recognize(ocrRequest, signal)
: this.ocrBroker.recognize(ocrRequest))
ensureNotAborted(signal)
const ocrByLocator = new Map(
ocr.sections.map((section) => [section.locator, section])
)
const merged = pages.flatMap((page): ParsedSection[] => {
const locator = `${page.pageNumber}`
const recognized = ocrByLocator.get(locator)
if (
recognized &&
(mode === 'always' || !hasUsefulText(page.content))
) {
return [
{
locator,
content: recognized.content,
method: 'ocr',
confidence: recognized.confidence
}
]
}
return page.content
? [{ locator, content: page.content, method: 'native' }]
: []
})
return buildPdfDocument(name, merged, pages.length, ocr.warnings)
}
async diagnose(
name: string,
buffer: Buffer,
purpose: DocumentParsingPurpose = 'diagnostic'
): Promise<DocumentParsingDiagnostic> {
const startedAt = Date.now()
const parsed = await this.parse(name, buffer, purpose)
const ocrPageCount = parsed.sections.filter(
(section) => section.method === 'ocr'
).length
const nativePageCount = parsed.sections.filter(
(section) => section.method !== 'ocr'
).length
return documentParsingDiagnosticSchema.parse({
fileName: name,
sourceFormat:
parsed.sourceFormat.replace(/^\./u, '').toUpperCase() || 'UNKNOWN',
pageCount:
parsed.sourceFormat === '.pdf'
? (parsed.pageCount ?? parsed.sections.length)
: 0,
ocrPageCount,
characterCount: parsed.content.length,
method:
ocrPageCount > 0 && nativePageCount > 0
? 'mixed'
: ocrPageCount > 0
? 'ocr'
: 'native',
durationMs: Date.now() - startedAt,
preview: parsed.content.slice(0, 2_000),
warnings: parsed.warnings
})
}
}
@@ -0,0 +1,120 @@
import {
mkdtemp,
readFile,
readdir,
rm,
writeFile
} from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest'
import {
defaultDocumentParsingSettings,
DocumentParsingSettingsStore
} from './document-parsing-settings-store'
const temporaryDirectories: string[] = []
async function createStore(): Promise<{
directory: string
filePath: string
store: DocumentParsingSettingsStore
}> {
const directory = await mkdtemp(
join(tmpdir(), 'goodbuddy-document-parsing-settings-')
)
temporaryDirectories.push(directory)
const filePath = join(directory, 'document-parsing-settings.json')
return {
directory,
filePath,
store: new DocumentParsingSettingsStore(filePath)
}
}
afterEach(async () => {
await Promise.all(
temporaryDirectories.splice(0).map((directory) =>
rm(directory, { recursive: true, force: true })
)
)
})
describe('DocumentParsingSettingsStore', () => {
it('returns local-first defaults without creating a file', async () => {
const { directory, store } = await createStore()
await expect(store.get()).resolves.toEqual(
defaultDocumentParsingSettings
)
await expect(readdir(directory)).resolves.toEqual([])
})
it('persists a complete versioned settings document', async () => {
const { filePath, store } = await createStore()
const settings = {
...defaultDocumentParsingSettings,
chatWorkflow: 'fast-text' as const,
maximumPages: 42
}
await expect(store.update(settings)).resolves.toEqual(settings)
expect(JSON.parse(await readFile(filePath, 'utf8'))).toEqual({
version: 2,
...settings
})
await expect(
new DocumentParsingSettingsStore(filePath).get()
).resolves.toEqual(settings)
})
it('migrates legacy cloud permissions to the local OCR provider', async () => {
const { filePath, store } = await createStore()
const {
ocrProvider: _ocrProvider,
...legacySettings
} = defaultDocumentParsingSettings
void _ocrProvider
await writeFile(
filePath,
JSON.stringify({
version: 1,
...legacySettings,
chatCloudPermission: 'always',
knowledgeCloudPermission: 'never'
}),
'utf8'
)
await expect(store.get()).resolves.toEqual(
defaultDocumentParsingSettings
)
})
it('rejects incomplete or out-of-range settings', async () => {
const { directory, store } = await createStore()
await expect(store.update({})).rejects.toThrow()
await expect(
store.update({
...defaultDocumentParsingSettings,
maximumPages: 0
})
).rejects.toThrow()
await expect(readdir(directory)).resolves.toEqual([])
})
it('isolates corrupt settings and restores defaults', async () => {
const { directory, filePath, store } = await createStore()
await writeFile(filePath, '{not-json', 'utf8')
await expect(store.get()).resolves.toEqual(
defaultDocumentParsingSettings
)
const entries = await readdir(directory)
expect(entries).toHaveLength(1)
expect(entries[0]).toMatch(
/^document-parsing-settings\.json\.corrupt-\d+-[a-f0-9]{12}$/u
)
})
})
+181
View File
@@ -0,0 +1,181 @@
import { randomBytes } from 'node:crypto'
import {
mkdir,
readFile,
rename,
rm,
writeFile
} from 'node:fs/promises'
import { dirname } from 'node:path'
import { z } from 'zod'
import {
documentParsingSettingsSchema,
documentParsingSettingsUpdateSchema,
type DocumentParsingSettings
} from '../shared/document-parsing-contracts'
const CURRENT_SETTINGS_VERSION = 2
const storedDocumentParsingSettingsSchema =
documentParsingSettingsSchema
.extend({
version: z.literal(CURRENT_SETTINGS_VERSION)
})
.strict()
type StoredDocumentParsingSettings = z.infer<
typeof storedDocumentParsingSettingsSchema
>
const legacyDocumentParsingSettingsSchema =
documentParsingSettingsSchema
.omit({ ocrProvider: true })
.extend({
version: z.literal(1),
chatCloudPermission: z.enum(['ask', 'always', 'never']),
knowledgeCloudPermission: z.enum(['ask', 'always', 'never'])
})
.strict()
export const defaultDocumentParsingSettings: DocumentParsingSettings = {
chatWorkflow: 'auto',
knowledgeWorkflow: 'complete-index',
pdfOcrMode: 'auto',
ocrProvider: 'local',
localOcrEnabled: true,
localOcrModelId: 'pp-ocrv6-tiny',
maximumPages: 100,
ocrConcurrency: 1,
pageTimeoutSeconds: 60
}
function isMissingFile(error: unknown): boolean {
return (
error !== null &&
typeof error === 'object' &&
'code' in error &&
error.code === 'ENOENT'
)
}
export class DocumentParsingSettingsStore {
private settings?: StoredDocumentParsingSettings
private updateQueue: Promise<void> = Promise.resolve()
constructor(private readonly filePath: string) {}
private async isolateCorruptFile(): Promise<void> {
const isolatedPath =
`${this.filePath}.corrupt-${Date.now()}-` +
randomBytes(6).toString('hex')
try {
await rename(this.filePath, isolatedPath)
} catch (error) {
if (!isMissingFile(error)) {
throw new Error('文档解析设置损坏且无法隔离', {
cause: error
})
}
}
}
private async loadStored(): Promise<StoredDocumentParsingSettings> {
if (this.settings) {
return this.settings
}
try {
const contents = await readFile(this.filePath, 'utf8')
let parsed: unknown
try {
parsed = JSON.parse(contents) as unknown
} catch {
await this.isolateCorruptFile()
this.settings = {
version: CURRENT_SETTINGS_VERSION,
...defaultDocumentParsingSettings
}
return this.settings
}
const result =
storedDocumentParsingSettingsSchema.safeParse(parsed)
if (!result.success) {
const legacy =
legacyDocumentParsingSettingsSchema.safeParse(parsed)
if (legacy.success) {
const {
version: _version,
chatCloudPermission: _chatCloudPermission,
knowledgeCloudPermission: _knowledgeCloudPermission,
...settings
} = legacy.data
void _version
void _chatCloudPermission
void _knowledgeCloudPermission
this.settings = {
version: CURRENT_SETTINGS_VERSION,
ocrProvider: 'local',
...settings
}
return this.settings
}
await this.isolateCorruptFile()
this.settings = {
version: CURRENT_SETTINGS_VERSION,
...defaultDocumentParsingSettings
}
return this.settings
}
this.settings = result.data
} catch (error) {
if (!isMissingFile(error)) {
throw new Error('无法读取文档解析设置', { cause: error })
}
this.settings = {
version: CURRENT_SETTINGS_VERSION,
...defaultDocumentParsingSettings
}
}
return this.settings
}
async get(): Promise<DocumentParsingSettings> {
const { version: _version, ...settings } = await this.loadStored()
void _version
return documentParsingSettingsSchema.parse(settings)
}
update(input: unknown): Promise<DocumentParsingSettings> {
const operation = this.updateQueue.then(async () => {
const updates = documentParsingSettingsUpdateSchema.parse(input)
const next: StoredDocumentParsingSettings = {
version: CURRENT_SETTINGS_VERSION,
...updates
}
await mkdir(dirname(this.filePath), { recursive: true })
const temporaryPath =
`${this.filePath}.${process.pid}.` +
`${randomBytes(6).toString('hex')}.tmp`
try {
await writeFile(
temporaryPath,
`${JSON.stringify(next, null, 2)}\n`,
{
encoding: 'utf8',
mode: 0o600,
flag: 'wx'
}
)
await rename(temporaryPath, this.filePath)
} finally {
await rm(temporaryPath, { force: true })
}
this.settings = next
return this.get()
})
this.updateQueue = operation.then(
() => undefined,
() => undefined
)
return operation
}
}
+32 -4
View File
@@ -67,6 +67,10 @@ import { KnowledgeEmbeddingIndexRepository } from './knowledge/knowledge-embeddi
import { GlobalTlsPolicy } from './global-tls-policy'
import type { AgentRuntimeSelection } from '../shared/runtime-selection-contracts'
import { waitForCleanup } from './shutdown'
import { DocumentParsingSettingsStore } from './document-parsing-settings-store'
import { DocumentOcrModelManager } from './document-ocr-model-manager'
import { DocumentOcrBroker } from './document-ocr-broker'
import { DocumentParsingService } from './document-parsing-service'
const shortcut = 'CommandOrControl+Shift+Space'
const mainModuleDirectory = dirname(fileURLToPath(import.meta.url))
@@ -98,6 +102,8 @@ let knowledgeGateway: KnowledgeMcpGateway | undefined
let assistantDatabase: AssistantDatabase | undefined
let browserService: BrowserService | undefined
let globalTlsPolicy: GlobalTlsPolicy | undefined
let documentOcrBroker: DocumentOcrBroker | undefined
let documentOcrModelManager: DocumentOcrModelManager | undefined
function createEmbeddingProvider(
settings: ResolvedRuntimeSettings
@@ -340,6 +346,20 @@ if (hasSingleInstanceLock) {
const applicationSettingsStore = new ApplicationSettingsStore(
join(app.getPath('userData'), 'application-settings.json')
)
const documentParsingSettingsStore =
new DocumentParsingSettingsStore(
join(app.getPath('userData'), 'document-parsing-settings.json')
)
documentOcrModelManager = new DocumentOcrModelManager({
userDataDirectory: app.getPath('userData'),
fetch: globalThis.fetch
})
documentOcrBroker = new DocumentOcrBroker(mainWindow)
const documentParsingService = new DocumentParsingService(
documentParsingSettingsStore,
documentOcrModelManager,
documentOcrBroker
)
const versionChecker = new VersionChecker({
fetch: globalThis.fetch,
currentVersion: app.getVersion(),
@@ -362,7 +382,8 @@ if (hasSingleInstanceLock) {
knowledgeService = new KnowledgeService({
databasePath: join(app.getPath('userData'), 'knowledge.sqlite'),
managedRoot: join(app.getPath('userData'), 'knowledge'),
extractStructured: createModelGraphExtractor(settingsStore)
extractStructured: createModelGraphExtractor(settingsStore),
parseDocument: documentParsingService.parse
})
await knowledgeService.initialize()
const embeddingIndexCoordinator = new EmbeddingIndexCoordinator(
@@ -459,7 +480,9 @@ if (hasSingleInstanceLock) {
selectedRuntimeManager = new SelectedRuntimeManager(
createSelectedRuntime
)
const contextManager = new ContextManager()
const contextManager = new ContextManager({
parseDocument: documentParsingService.parse
})
const approvalBroker = new ToolApprovalBroker()
const shortcutRegistered = globalShortcut.register(shortcut, () => {
@@ -510,7 +533,10 @@ if (hasSingleInstanceLock) {
selectedRuntimeManager,
speechTranscriptionService,
knowledgeGateway,
launchWechatSidecar
launchWechatSidecar,
documentParsingService,
documentOcrModelManager,
documentOcrBroker
)
loadMainWindow(mainWindow)
@@ -550,7 +576,9 @@ app.on('before-quit', (event) => {
Promise.resolve().then(() => knowledgeGateway?.dispose()),
Promise.resolve().then(() => knowledgeService?.dispose()),
Promise.resolve().then(() => browserService?.dispose()),
Promise.resolve().then(() => globalTlsPolicy?.dispose())
Promise.resolve().then(() => globalTlsPolicy?.dispose()),
Promise.resolve().then(() => documentOcrModelManager?.dispose()),
Promise.resolve().then(() => documentOcrBroker?.dispose())
])
globalShortcut.unregisterAll()
tray?.destroy()
+180 -1
View File
@@ -24,6 +24,10 @@ const electronMocks = vi.hoisted(() => {
canceled: true,
filePaths: [] as string[]
})),
showSaveDialog: vi.fn(async () => ({
canceled: true,
filePath: undefined as string | undefined
})),
openPath: vi.fn(async () => ''),
showItemInFolder: vi.fn(),
openExternal: vi.fn(async () => undefined)
@@ -248,7 +252,8 @@ vi.mock('electron', () => ({
},
BrowserWindow: class {},
dialog: {
showOpenDialog: electronMocks.showOpenDialog
showOpenDialog: electronMocks.showOpenDialog,
showSaveDialog: electronMocks.showSaveDialog
},
ipcMain: {
handle: electronMocks.handle,
@@ -290,6 +295,180 @@ vi.mock('./channels/channel-env', () => ({
)
}))
describe('registerIpcHandlers model ZIP dialogs', () => {
afterEach(() => {
electronMocks.handlers.clear()
vi.clearAllMocks()
})
it('imports and exports speech and OCR ZIPs through trusted dialogs', async () => {
const webContents = {
mainFrame: { url: 'file:///goodbuddy/index.html' },
getURL: vi.fn(() => 'file:///goodbuddy/index.html'),
send: vi.fn()
}
const window = {
webContents,
isDestroyed: vi.fn(() => false),
isMaximized: vi.fn(() => false),
on: vi.fn(),
removeListener: vi.fn()
}
const event = {
sender: webContents,
senderFrame: webContents.mainFrame
}
const speechSnapshot = {
catalog: [],
installed: [],
operations: []
}
const speechModelManager = {
rootDirectory: 'C:\\models\\speech',
importArchive: vi.fn(async () => speechSnapshot),
exportArchive: vi.fn(async () => undefined),
getSnapshot: vi.fn(async () => speechSnapshot),
cancel: vi.fn()
}
const ocrSnapshot = {
settings: {},
models: {
catalog: [],
installed: [],
operations: []
}
}
const documentParsingService = {
snapshot: vi.fn(async () => ocrSnapshot)
}
const documentOcrModelManager = {
importArchive: vi.fn(async () => undefined),
exportArchive: vi.fn(async () => undefined)
}
const dispose = registerIpcHandlers(
window as never,
{ capability: 'text' } as never,
'CommandOrControl+Shift+Space',
{} as never,
{} as never,
{ clear: vi.fn() } as never,
{} as never,
{ claimDueSchedules: vi.fn(() => []) } as never,
{ clear: vi.fn() } as never,
{} as never,
vi.fn(async () => undefined),
undefined,
undefined,
undefined,
undefined,
undefined,
undefined,
speechModelManager as never,
undefined,
undefined,
undefined,
undefined,
undefined,
documentParsingService as never,
documentOcrModelManager as never
)
await expect(
electronMocks.handlers.get(
ipcChannels.speechModelsImportArchive
)?.(event, { modelId: 'speech-model' })
).resolves.toBeUndefined()
expect(speechModelManager.importArchive).not.toHaveBeenCalled()
electronMocks.showOpenDialog.mockResolvedValueOnce({
canceled: false,
filePaths: ['C:\\transfer\\speech.zip']
})
await expect(
electronMocks.handlers.get(
ipcChannels.speechModelsImportArchive
)?.(event, { modelId: 'speech-model' })
).resolves.toBe(speechSnapshot)
expect(electronMocks.showOpenDialog).toHaveBeenLastCalledWith(
window,
expect.objectContaining({
properties: ['openFile'],
filters: [
{
name: 'GoodBuddy 模型 ZIP',
extensions: ['zip']
}
]
})
)
expect(speechModelManager.importArchive).toHaveBeenCalledWith(
'speech-model',
'C:\\transfer\\speech.zip'
)
electronMocks.showSaveDialog.mockResolvedValueOnce({
canceled: false,
filePath: 'C:\\transfer\\speech-model'
})
await expect(
electronMocks.handlers.get(
ipcChannels.speechModelsExportArchive
)?.(event, { modelId: 'speech-model' })
).resolves.toBe(speechSnapshot)
expect(speechModelManager.exportArchive).toHaveBeenCalledWith(
'speech-model',
'C:\\transfer\\speech-model.zip'
)
electronMocks.showOpenDialog.mockResolvedValueOnce({
canceled: false,
filePaths: ['C:\\transfer\\ocr.zip']
})
await expect(
electronMocks.handlers.get(
ipcChannels.documentOcrModelsImportArchive
)?.(event, { modelId: 'ocr-model' })
).resolves.toBe(ocrSnapshot)
expect(documentOcrModelManager.importArchive).toHaveBeenCalledWith(
'ocr-model',
'C:\\transfer\\ocr.zip'
)
electronMocks.showSaveDialog.mockResolvedValueOnce({
canceled: false,
filePath: 'C:\\transfer\\ocr-model.ZIP'
})
await expect(
electronMocks.handlers.get(
ipcChannels.documentOcrModelsExportArchive
)?.(event, { modelId: 'ocr-model' })
).resolves.toBe(ocrSnapshot)
expect(documentOcrModelManager.exportArchive).toHaveBeenCalledWith(
'ocr-model',
'C:\\transfer\\ocr-model.ZIP'
)
await expect(
electronMocks.handlers.get(
ipcChannels.speechModelsExportArchive
)?.(
{
sender: {},
senderFrame: webContents.mainFrame
},
{ modelId: 'speech-model' }
)
).rejects.toThrow('拒绝来自未知窗口的 IPC 请求')
await expect(
electronMocks.handlers.get(
ipcChannels.documentOcrModelsImportArchive
)?.(event, {})
).rejects.toThrow()
await dispose()
})
})
describe('registerIpcHandlers connection tests', () => {
afterEach(() => {
electronMocks.handlers.clear()
+280 -6
View File
@@ -73,6 +73,12 @@ import {
embeddingIndexJobRequestSchema,
embeddingSettingsSnapshotSchema
} from '../shared/embedding-contracts'
import {
documentOcrModelActionInputSchema,
documentOcrFailureSchema,
documentOcrResultSchema,
documentParsingSettingsUpdateSchema
} from '../shared/document-parsing-contracts'
import {
agentRuntimeSelectionSchema,
type AgentRuntimeSelection
@@ -178,6 +184,9 @@ import type { VersionChecker } from './version-checker'
import type { SpeechModelManager } from './speech/speech-model-manager'
import type { SpeechTranscriptionService } from './speech/speech-transcription-service'
import type { EmbeddingIndexCoordinator } from './knowledge/embedding-index-coordinator'
import type { DocumentParsingService } from './document-parsing-service'
import type { DocumentOcrModelManager } from './document-ocr-model-manager'
import type { DocumentOcrBroker } from './document-ocr-broker'
import { OpenAIEmbeddingClient } from './knowledge/openai-embedding-client'
import {
magicNotePlainText,
@@ -321,6 +330,17 @@ const taskStatusRequestSchema = z
status: z.enum(['completed', 'cancelled'])
})
.strict()
const modelArchiveDialogFilters = [
{
name: 'GoodBuddy 模型 ZIP',
extensions: ['zip']
}
]
function ensureZipExtension(path: string): string {
return extname(path).toLowerCase() === '.zip' ? path : `${path}.zip`
}
const expertUpdateRequestSchema = z
.object({
expertId: assistantIdSchema,
@@ -575,7 +595,10 @@ export function registerIpcHandlers(
selectedRuntimes?: SelectedRuntimeResolver,
speechTranscriptionService?: SpeechTranscriptionService,
knowledgeGateway?: KnowledgeMcpGateway,
launchWechatSidecar?: WechatSidecarLauncher
launchWechatSidecar?: WechatSidecarLauncher,
documentParsingService?: DocumentParsingService,
documentOcrModelManager?: DocumentOcrModelManager,
documentOcrBroker?: DocumentOcrBroker
): () => Promise<void> {
const activeRequests = new Map<string, AbortController>()
const pendingAgentQuestions = new Map<
@@ -2463,6 +2486,233 @@ export function registerIpcHandlers(
}
)
ipcMain.handle(ipcChannels.documentParsingGet, (event) => {
assertTrustedSender(event, window)
if (!documentParsingService) {
throw new Error('文档解析设置服务不可用')
}
return documentParsingService.snapshot()
})
ipcMain.handle(
ipcChannels.documentParsingUpdate,
(event, input: unknown) => {
assertTrustedSender(event, window)
if (!documentParsingService) {
throw new Error('文档解析设置服务不可用')
}
return documentParsingService.update(
documentParsingSettingsUpdateSchema.parse(input)
)
}
)
ipcMain.handle(
ipcChannels.documentParsingTest,
async (event) => {
assertTrustedSender(event, window)
if (!documentParsingService) {
throw new Error('文档解析设置服务不可用')
}
const result = await dialog.showOpenDialog(window, {
title: '选择测试文档',
properties: ['openFile'],
filters: [
{
name: '支持的文档',
extensions: supportedDocumentExtensions.map((extension) =>
extension.slice(1)
)
}
]
})
const selectedPath = result.filePaths[0]
if (result.canceled || !selectedPath) {
return undefined
}
try {
const canonicalPath = await realpath(selectedPath)
const fileStat = await stat(canonicalPath)
if (!fileStat.isFile() || fileStat.size > 20 * 1024 * 1024) {
throw new Error('测试文档必须小于 20MB 且不能是目录')
}
return documentParsingService.diagnose(
basename(canonicalPath),
await readFile(canonicalPath)
)
} catch (error) {
if (error instanceof Error && !('code' in error)) {
throw error
}
throw new Error('无法读取测试文档,请检查文件权限和状态', {
cause: error
})
}
}
)
ipcMain.handle(
ipcChannels.documentOcrModelsInstall,
(event, input: unknown) => {
assertTrustedSender(event, window)
if (!documentOcrModelManager || !documentParsingService) {
throw new Error('本地 OCR 模型服务不可用')
}
const { modelId } =
documentOcrModelActionInputSchema.parse(input)
return trackExecution(
documentOcrModelManager
.install(modelId)
.then(() => documentParsingService.snapshot())
)
}
)
ipcMain.handle(
ipcChannels.documentOcrModelsCancel,
(event, input: unknown) => {
assertTrustedSender(event, window)
if (!documentOcrModelManager) {
throw new Error('本地 OCR 模型服务不可用')
}
const { modelId } =
documentOcrModelActionInputSchema.parse(input)
return documentOcrModelManager.cancel(modelId)
}
)
ipcMain.handle(
ipcChannels.documentOcrModelsRemove,
async (event, input: unknown) => {
assertTrustedSender(event, window)
if (!documentOcrModelManager || !documentParsingService) {
throw new Error('本地 OCR 模型服务不可用')
}
const { modelId } =
documentOcrModelActionInputSchema.parse(input)
await documentOcrModelManager.remove(modelId)
return documentParsingService.snapshot()
}
)
ipcMain.handle(
ipcChannels.documentOcrModelsImportArchive,
async (event, input: unknown) => {
assertTrustedSender(event, window)
if (!documentOcrModelManager || !documentParsingService) {
throw new Error('本地 OCR 模型服务不可用')
}
const { modelId } =
documentOcrModelActionInputSchema.parse(input)
const result = await dialog.showOpenDialog(window, {
title: '导入 OCR 模型 ZIP',
properties: ['openFile'],
filters: modelArchiveDialogFilters
})
const archivePath = result.filePaths[0]
if (result.canceled || !archivePath) {
return undefined
}
return trackExecution(
documentOcrModelManager
.importArchive(modelId, archivePath)
.then(() => documentParsingService.snapshot())
)
}
)
ipcMain.handle(
ipcChannels.documentOcrModelsExportArchive,
async (event, input: unknown) => {
assertTrustedSender(event, window)
if (!documentOcrModelManager || !documentParsingService) {
throw new Error('本地 OCR 模型服务不可用')
}
const { modelId } =
documentOcrModelActionInputSchema.parse(input)
const result = await dialog.showSaveDialog(window, {
title: '导出 OCR 模型 ZIP',
defaultPath: `${modelId}.zip`,
filters: modelArchiveDialogFilters
})
if (result.canceled || !result.filePath) {
return undefined
}
const destination = ensureZipExtension(result.filePath)
await documentOcrModelManager.exportArchive(
modelId,
destination
)
return documentParsingService.snapshot()
}
)
ipcMain.handle(
ipcChannels.documentOcrModelsOpenRepository,
async (event, input: unknown) => {
assertTrustedSender(event, window)
if (!documentOcrModelManager) {
throw new Error('本地 OCR 模型服务不可用')
}
const { modelId } =
documentOcrModelActionInputSchema.parse(input)
const snapshot = await documentOcrModelManager.getSnapshot()
const entry = snapshot.catalog.find(
(candidate) => candidate.id === modelId
)
if (!entry) {
throw new Error('未知的 OCR 模型')
}
await shell.openExternal(entry.repositoryUrl)
}
)
ipcMain.handle(
ipcChannels.documentOcrModelsOpenDirectory,
async (event) => {
assertTrustedSender(event, window)
if (!documentOcrModelManager) {
throw new Error('本地 OCR 模型服务不可用')
}
await documentOcrModelManager.getSnapshot()
const error = await shell.openPath(
documentOcrModelManager.rootDirectory
)
if (error) {
throw new Error('无法打开 OCR 模型目录')
}
}
)
ipcMain.handle(
ipcChannels.documentParsingOcrAssets,
(event, input: unknown) => {
assertTrustedSender(event, window)
if (!documentOcrModelManager) {
throw new Error('本地 OCR 模型服务不可用')
}
const { modelId } =
documentOcrModelActionInputSchema.parse(input)
return documentOcrModelManager.getAssets(modelId)
}
)
ipcMain.handle(
ipcChannels.documentParsingOcrRespond,
(event, input: unknown) => {
assertTrustedSender(event, window)
if (!documentOcrBroker) {
throw new Error('本地 OCR 任务服务不可用')
}
const result = documentOcrResultSchema.safeParse(input)
documentOcrBroker.respond(
result.success
? result.data
: documentOcrFailureSchema.parse(input)
)
}
)
ipcMain.handle(ipcChannels.versionCheck, async (event) => {
assertTrustedSender(event, window)
if (!versionChecker) {
@@ -2610,7 +2860,7 @@ export function registerIpcHandlers(
)
ipcMain.handle(
ipcChannels.speechModelsImportLocal,
ipcChannels.speechModelsImportArchive,
async (event, input: unknown) => {
assertTrustedSender(event, window)
if (!speechModelManager) {
@@ -2618,20 +2868,44 @@ export function registerIpcHandlers(
}
const { modelId } = speechModelActionInputSchema.parse(input)
const result = await dialog.showOpenDialog(window, {
properties: ['openDirectory']
title: '导入语音模型 ZIP',
properties: ['openFile'],
filters: modelArchiveDialogFilters
})
const directory = result.filePaths[0]
if (result.canceled || !directory) {
const archivePath = result.filePaths[0]
if (result.canceled || !archivePath) {
return undefined
}
return trackExecution(
speechModelManager
.registerLocalDirectory(modelId, directory)
.importArchive(modelId, archivePath)
.then(() => speechModelManager.getSnapshot())
)
}
)
ipcMain.handle(
ipcChannels.speechModelsExportArchive,
async (event, input: unknown) => {
assertTrustedSender(event, window)
if (!speechModelManager) {
throw new Error('语音模型服务不可用')
}
const { modelId } = speechModelActionInputSchema.parse(input)
const result = await dialog.showSaveDialog(window, {
title: '导出语音模型 ZIP',
defaultPath: `${modelId}.zip`,
filters: modelArchiveDialogFilters
})
if (result.canceled || !result.filePath) {
return undefined
}
const destination = ensureZipExtension(result.filePath)
await speechModelManager.exportArchive(modelId, destination)
return speechModelManager.getSnapshot()
}
)
ipcMain.handle(
ipcChannels.speechModelsOpenRepository,
async (event, input: unknown) => {
@@ -0,0 +1,50 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
const getDocument = vi.hoisted(() => vi.fn())
vi.mock('pdfjs-dist/legacy/build/pdf.mjs', () => ({
getDocument
}))
import { extractPdfTextPages } from './document-parser'
describe('PDF extraction in Electron main', () => {
beforeEach(() => {
getDocument.mockReset()
})
it('disables PDF.js DOM factories for headless text extraction', async () => {
const cleanup = vi.fn()
const destroy = vi.fn(async () => undefined)
getDocument.mockReturnValue({
promise: Promise.resolve({
numPages: 1,
getPage: vi.fn(async () => ({
getTextContent: vi.fn(async () => ({
items: [{ str: 'PDF body text' }]
})),
cleanup
}))
}),
destroy
})
await expect(
extractPdfTextPages(Buffer.from('synthetic PDF'))
).resolves.toEqual([
{
pageNumber: 1,
content: 'PDF body text'
}
])
expect(getDocument).toHaveBeenCalledWith({
data: expect.any(Uint8Array),
disableFontFace: true,
isOffscreenCanvasSupported: false,
useSystemFonts: false,
useWorkerFetch: false
})
expect(cleanup).toHaveBeenCalledOnce()
expect(destroy).toHaveBeenCalledOnce()
})
})
+44 -11
View File
@@ -5,12 +5,17 @@ import { extname } from 'node:path'
export type ParsedSection = {
locator: string
content: string
method?: 'native' | 'ocr' | 'converted' | 'vision'
confidence?: number
}
export type ParsedDocument = {
title: string
sourceFormat: string
content: string
sections: ParsedSection[]
warnings: string[]
pageCount?: number
}
export type DocumentChunk = {
@@ -160,12 +165,43 @@ function parseOfficeArchive(
}
async function parsePdf(buffer: Buffer): Promise<ParsedSection[]> {
const pages = await extractPdfTextPages(buffer)
return pages
.filter((page) => page.content.length > 0)
.map((page) => ({
locator: `${page.pageNumber}`,
content: page.content
}))
}
export type PdfTextPage = {
pageNumber: number
content: string
}
export class DocumentTextUnavailableError extends Error {
constructor(message = '文档中没有可索引的文本内容') {
super(message)
this.name = 'DocumentTextUnavailableError'
}
}
export async function extractPdfTextPages(
buffer: Buffer
): Promise<PdfTextPage[]> {
const pdfjs = await import('pdfjs-dist/legacy/build/pdf.mjs')
const loadingTask = pdfjs.getDocument({
data: new Uint8Array(buffer)
data: new Uint8Array(buffer),
// Electron's main process identifies itself as process.type ===
// "browser", so PDF.js otherwise selects DOM font factories even
// though no document exists there.
disableFontFace: true,
isOffscreenCanvasSupported: false,
useSystemFonts: false,
useWorkerFetch: false
})
const document = await loadingTask.promise
const sections: ParsedSection[] = []
const pages: PdfTextPage[] = []
try {
for (let pageNumber = 1; pageNumber <= document.numPages; pageNumber += 1) {
const page = await document.getPage(pageNumber)
@@ -175,18 +211,13 @@ async function parsePdf(buffer: Buffer): Promise<ParsedSection[]> {
.join(' ')
.replace(/\s+/g, ' ')
.trim()
if (content) {
sections.push({
locator: `${pageNumber}`,
content
})
}
pages.push({ pageNumber, content })
page.cleanup()
}
} finally {
await loadingTask.destroy()
}
return sections
return pages
}
export async function parseDocument(
@@ -227,12 +258,14 @@ export async function parseDocument(
.join('\n\n')
.slice(0, maximumExtractedCharacters)
if (!content) {
throw new Error('文档中没有可索引的文本内容')
throw new DocumentTextUnavailableError()
}
return {
title: name.replace(/\.[^.]+$/, ''),
sourceFormat: extension || 'unknown',
content,
sections
sections,
warnings: []
}
}
+22 -3
View File
@@ -18,7 +18,12 @@ import {
relative,
resolve
} from 'node:path'
import { chunkDocument, parseDocument, supportedDocumentExtensions } from './document-parser'
import {
chunkDocument,
parseDocument,
supportedDocumentExtensions,
type ParsedDocument
} from './document-parser'
import { classifyEmbeddingError } from './embedding-errors'
import {
extractKnowledgeGraph,
@@ -99,6 +104,12 @@ export type KnowledgeServiceOptions = {
urlImporter?: UrlImporter
embeddingProvider?: EmbeddingProvider
embeddingBatchSize?: number
parseDocument?: (
name: string,
buffer: Buffer,
purpose: 'knowledge-index',
signal?: AbortSignal
) => Promise<ParsedDocument>
}
const supportedExtensions = new Set<string>(supportedDocumentExtensions)
@@ -118,6 +129,9 @@ export class KnowledgeService {
private readonly managedRoot: string
private readonly extractStructured?: ExtractStructured
private readonly urlImporter: UrlImporter
private readonly documentParser: NonNullable<
KnowledgeServiceOptions['parseDocument']
>
private embeddingProvider?: EmbeddingProvider
private readonly embeddingBatchSize: number
private readonly watchers = new Map<string, FSWatcher>()
@@ -131,6 +145,9 @@ export class KnowledgeService {
this.managedRoot = resolve(options.managedRoot)
this.extractStructured = options.extractStructured
this.urlImporter = options.urlImporter ?? new UrlImporter()
this.documentParser =
options.parseDocument ??
((name, buffer) => parseDocument(name, buffer))
this.embeddingProvider = options.embeddingProvider
const embeddingBatchSize = options.embeddingBatchSize ?? 16
if (
@@ -839,9 +856,11 @@ export class KnowledgeService {
})
continue
}
const parsed = await parseDocument(
const parsed = await this.documentParser(
basename(file.absolutePath),
buffer
buffer,
'knowledge-index',
this.lifecycleController.signal
)
this.updateKnowledgeTask(parsingTask.id, {
progress: 75,
+211
View File
@@ -0,0 +1,211 @@
import { createHash } from 'node:crypto'
import {
mkdtemp,
mkdir,
readFile,
rm,
writeFile
} from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { zipSync } from 'fflate'
import { afterEach, describe, expect, it } from 'vitest'
import {
exportModelArchive,
extractModelArchive
} from './model-archive'
const temporaryDirectories: string[] = []
async function temporaryDirectory(): Promise<string> {
const directory = await mkdtemp(
join(tmpdir(), 'goodbuddy-model-archive-')
)
temporaryDirectories.push(directory)
return directory
}
function sha256(value: Uint8Array): string {
return createHash('sha256').update(value).digest('hex')
}
afterEach(async () => {
await Promise.all(
temporaryDirectories.splice(0).map((directory) =>
rm(directory, { recursive: true, force: true })
)
)
})
describe('model archive', () => {
it('exports and extracts only declared verified model files', async () => {
const directory = await temporaryDirectory()
const source = join(directory, 'source')
const extracted = join(directory, 'extracted')
const archive = join(directory, 'model.zip')
await Promise.all([mkdir(source), mkdir(extracted)])
const model = Buffer.from('verified model bytes')
const tokens = Buffer.from('verified tokens')
await Promise.all([
writeFile(join(source, 'model.onnx'), model),
writeFile(join(source, 'tokens.txt'), tokens),
writeFile(join(source, 'ignored.txt'), 'not exported'),
writeFile(archive, 'archive selected for replacement')
])
await exportModelArchive({
destinationPath: archive,
sourceDirectory: source,
descriptor: {
kind: 'speech',
modelId: 'test-model',
displayName: 'Test model',
files: [
{
name: 'model.onnx',
role: 'model',
size: model.byteLength,
sha256: sha256(model)
},
{
name: 'tokens.txt',
role: 'tokens',
size: tokens.byteLength,
sha256: sha256(tokens)
}
]
}
})
await expect(
extractModelArchive({
archivePath: archive,
destinationDirectory: extracted,
expectedKind: 'speech',
expectedModelId: 'test-model',
expectedFiles: [
{ name: 'model.onnx', role: 'model' },
{ name: 'tokens.txt', role: 'tokens' }
],
maximumArchiveBytes: 1024 * 1024,
maximumFileBytes: 1024,
maximumTotalBytes: 2048
})
).resolves.toMatchObject({
kind: 'speech',
modelId: 'test-model'
})
await expect(readFile(join(extracted, 'model.onnx'))).resolves.toEqual(
model
)
await expect(readFile(join(extracted, 'tokens.txt'))).resolves.toEqual(
tokens
)
})
it('preserves an existing archive when source verification fails', async () => {
const directory = await temporaryDirectory()
const source = join(directory, 'source')
const archive = join(directory, 'model.zip')
await mkdir(source)
const model = Buffer.from('changed model')
await Promise.all([
writeFile(join(source, 'model.onnx'), model),
writeFile(archive, 'existing archive')
])
await expect(
exportModelArchive({
destinationPath: archive,
sourceDirectory: source,
descriptor: {
kind: 'speech',
modelId: 'test-model',
displayName: 'Test model',
files: [
{
name: 'model.onnx',
role: 'model',
size: model.byteLength,
sha256: 'a'.repeat(64)
}
]
}
})
).rejects.toThrow('模型文件校验失败')
await expect(readFile(archive, 'utf8')).resolves.toBe(
'existing archive'
)
})
it('rejects path traversal and undeclared archive entries', async () => {
const directory = await temporaryDirectory()
const archive = join(directory, 'unsafe.zip')
const extracted = join(directory, 'extracted')
await mkdir(extracted)
await writeFile(
archive,
zipSync({
'../model.onnx': Buffer.from('unsafe')
})
)
await expect(
extractModelArchive({
archivePath: archive,
destinationDirectory: extracted,
expectedKind: 'speech',
expectedModelId: 'test-model',
expectedFiles: [{ name: 'model.onnx', role: 'model' }],
maximumArchiveBytes: 1024 * 1024,
maximumFileBytes: 1024,
maximumTotalBytes: 1024
})
).rejects.toThrow()
})
it('rejects an archive whose manifest model ID does not match', async () => {
const directory = await temporaryDirectory()
const archive = join(directory, 'mismatch.zip')
const extracted = join(directory, 'extracted')
await mkdir(extracted)
const model = Buffer.from('model')
await writeFile(
archive,
zipSync({
'goodbuddy-model.json': Buffer.from(
JSON.stringify({
format: 'goodbuddy-model-archive',
version: 1,
kind: 'speech',
modelId: 'other-model',
displayName: 'Other model',
exportedAt: '2026-08-11T00:00:00.000Z',
files: [
{
name: 'model.onnx',
role: 'model',
size: model.byteLength,
sha256: sha256(model)
}
]
})
),
'model.onnx': model
})
)
await expect(
extractModelArchive({
archivePath: archive,
destinationDirectory: extracted,
expectedKind: 'speech',
expectedModelId: 'test-model',
expectedFiles: [{ name: 'model.onnx', role: 'model' }],
maximumArchiveBytes: 1024 * 1024,
maximumFileBytes: 1024,
maximumTotalBytes: 1024
})
).rejects.toThrow('模型 ID 不匹配')
})
})
+617
View File
@@ -0,0 +1,617 @@
import { createHash, randomUUID } from 'node:crypto'
import {
lstat,
open,
readFile,
rename,
rm,
type FileHandle
} from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import {
Unzip,
UnzipInflate,
UnzipPassThrough,
Zip,
ZipPassThrough
} from 'fflate'
import { z } from 'zod'
const ARCHIVE_MANIFEST_NAME = 'goodbuddy-model.json'
const ARCHIVE_FORMAT = 'goodbuddy-model-archive'
const ARCHIVE_VERSION = 1
const MAXIMUM_ARCHIVE_ENTRIES = 40
const MAXIMUM_MANIFEST_BYTES = 256 * 1024
const archiveFileNameSchema = z
.string()
.min(1)
.max(255)
.regex(/^[^/\\:\0]+$/u)
const modelArchiveFileSchema = z
.object({
name: archiveFileNameSchema,
role: z.string().trim().min(1).max(64),
size: z.number().int().positive().safe(),
sha256: z.string().regex(/^[a-f0-9]{64}$/u)
})
.strict()
const modelArchiveDescriptorSchema = z
.object({
kind: z.enum(['speech', 'document-ocr']),
modelId: z
.string()
.min(1)
.max(96)
.regex(/^[a-z0-9]+(?:-[a-z0-9]+)*$/u),
displayName: z.string().trim().min(1).max(120),
files: z.array(modelArchiveFileSchema).min(1).max(32)
})
.strict()
const modelArchiveManifestSchema = modelArchiveDescriptorSchema
.extend({
format: z.literal(ARCHIVE_FORMAT),
version: z.literal(ARCHIVE_VERSION),
exportedAt: z.string().datetime()
})
.strict()
.superRefine((manifest, context) => {
if (
new Set(manifest.files.map((file) => file.name.toLowerCase()))
.size !== manifest.files.length
) {
context.addIssue({
code: 'custom',
path: ['files'],
message: '模型 ZIP 清单包含重复文件'
})
}
})
export type ModelArchiveKind = z.infer<
typeof modelArchiveManifestSchema
>['kind']
export type ModelArchiveFile = z.infer<typeof modelArchiveFileSchema>
export type ModelArchiveDescriptor = {
kind: ModelArchiveKind
modelId: string
displayName: string
files: ModelArchiveFile[]
}
export type ModelArchiveExpectedFile = {
name: string
role: string
}
type ExportModelArchiveOptions = {
destinationPath: string
sourceDirectory: string
descriptor: ModelArchiveDescriptor
}
type ExtractModelArchiveOptions = {
archivePath: string
destinationDirectory: string
expectedKind: ModelArchiveKind
expectedModelId: string
expectedFiles: ModelArchiveExpectedFile[]
maximumArchiveBytes: number
maximumFileBytes: number
maximumTotalBytes: number
signal?: AbortSignal
onProgress?: (completedBytes: number) => void
}
function safeChild(parent: string, name: string): string {
const child = resolve(parent, name)
if (dirname(child) !== resolve(parent)) {
throw new Error('模型 ZIP 路径超出临时目录')
}
return child
}
function ensureArchiveName(name: string): string {
return archiveFileNameSchema.parse(name)
}
function ensureUniqueFiles(files: ModelArchiveExpectedFile[]): void {
const names = files.map((file) => ensureArchiveName(file.name))
if (new Set(names.map((name) => name.toLowerCase())).size !== names.length) {
throw new Error('模型目录包含重复文件名')
}
}
async function hashFile(path: string): Promise<ModelArchiveFile['sha256']> {
const handle = await open(path, 'r')
const hash = createHash('sha256')
const buffer = Buffer.allocUnsafe(64 * 1024)
try {
while (true) {
const { bytesRead } = await handle.read(buffer, 0, buffer.length)
if (bytesRead === 0) {
break
}
hash.update(buffer.subarray(0, bytesRead))
}
} finally {
await handle.close()
}
return hash.digest('hex')
}
function checkedLimit(value: number, label: string): number {
if (!Number.isSafeInteger(value) || value <= 0) {
throw new RangeError(`${label}无效`)
}
return value
}
function ensureNotAborted(signal?: AbortSignal): void {
if (signal?.aborted) {
throw signal.reason instanceof Error
? signal.reason
: new Error('模型 ZIP 导入已取消')
}
}
async function pushFileIntoArchive(
archive: Zip,
file: ModelArchiveFile,
sourcePath: string,
waitForOutput: () => Promise<void>
): Promise<void> {
const input = new ZipPassThrough(ensureArchiveName(file.name))
archive.add(input)
const sourceInfo = await lstat(sourcePath)
if (!sourceInfo.isFile() || sourceInfo.isSymbolicLink()) {
throw new Error(`模型文件不可导出:${file.name}`)
}
const handle = await open(sourcePath, 'r')
const buffer = Buffer.allocUnsafe(64 * 1024)
const hash = createHash('sha256')
let size = 0
try {
const openedInfo = await handle.stat()
if (
!openedInfo.isFile() ||
openedInfo.dev !== sourceInfo.dev ||
openedInfo.ino !== sourceInfo.ino
) {
throw new Error(`模型文件在打开前已发生变化:${file.name}`)
}
while (true) {
const { bytesRead } = await handle.read(buffer, 0, buffer.length)
if (bytesRead === 0) {
break
}
const chunk = buffer.subarray(0, bytesRead)
hash.update(chunk)
size += bytesRead
input.push(Uint8Array.from(chunk))
await waitForOutput()
}
if (size !== file.size || hash.digest('hex') !== file.sha256) {
throw new Error(`模型文件校验失败:${file.name}`)
}
input.push(new Uint8Array(), true)
await waitForOutput()
} finally {
await handle.close()
}
}
async function pushBytesIntoArchive(
archive: Zip,
name: string,
value: Uint8Array,
waitForOutput: () => Promise<void>
): Promise<void> {
const input = new ZipPassThrough(ensureArchiveName(name))
archive.add(input)
input.push(value, true)
await waitForOutput()
}
async function replaceArchiveFile(
partialPath: string,
destinationPath: string
): Promise<void> {
const backupPath = `${destinationPath}.${randomUUID()}.backup`
let movedExistingFile = false
try {
try {
await rename(destinationPath, backupPath)
movedExistingFile = true
const existingInfo = await lstat(backupPath)
if (!existingInfo.isFile() || existingInfo.isSymbolicLink()) {
throw new Error('模型 ZIP 导出目标必须是普通文件')
}
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') {
throw error
}
}
await rename(partialPath, destinationPath)
if (movedExistingFile) {
await rm(backupPath, { force: true }).catch(() => undefined)
}
} catch (error) {
if (movedExistingFile) {
await rm(destinationPath, { force: true }).catch(() => undefined)
await rename(backupPath, destinationPath).catch(() => undefined)
}
throw error
}
}
export async function exportModelArchive(
options: ExportModelArchiveOptions
): Promise<void> {
const descriptor = modelArchiveDescriptorSchema.parse(
options.descriptor
)
ensureUniqueFiles(descriptor.files)
const sourceDirectory = resolve(options.sourceDirectory)
const destinationPath = resolve(options.destinationPath)
const partialPath = `${destinationPath}.${randomUUID()}.partial`
const output = await open(partialPath, 'wx')
let writeChain = Promise.resolve()
let archiveError: Error | undefined
let resolveFinished: (() => void) | undefined
let rejectFinished: ((error: Error) => void) | undefined
const finished = new Promise<void>((resolvePromise, rejectPromise) => {
resolveFinished = resolvePromise
rejectFinished = rejectPromise
})
const archive = new Zip((error, data, final) => {
if (error) {
archiveError = error
rejectFinished?.(error)
return
}
writeChain = writeChain.then(async () => {
if (data.byteLength > 0) {
await output.write(data)
}
})
if (final) {
void writeChain.then(resolveFinished, rejectFinished)
}
})
const waitForOutput = async (): Promise<void> => {
await writeChain
if (archiveError) {
throw archiveError
}
}
try {
const manifest = modelArchiveManifestSchema.parse({
format: ARCHIVE_FORMAT,
version: ARCHIVE_VERSION,
kind: descriptor.kind,
modelId: descriptor.modelId,
displayName: descriptor.displayName,
exportedAt: new Date().toISOString(),
files: descriptor.files
})
await pushBytesIntoArchive(
archive,
ARCHIVE_MANIFEST_NAME,
Buffer.from(`${JSON.stringify(manifest, null, 2)}\n`, 'utf8'),
waitForOutput
)
for (const file of descriptor.files) {
await pushFileIntoArchive(
archive,
file,
safeChild(sourceDirectory, file.name),
waitForOutput
)
}
archive.end()
await finished
await output.sync()
await output.close()
await replaceArchiveFile(partialPath, destinationPath)
} catch (error) {
archive.terminate()
await output.close().catch(() => undefined)
await rm(partialPath, { force: true })
throw error
}
}
function closeHandle(handle: FileHandle): Promise<void> {
return handle.close().catch(() => undefined)
}
export async function extractModelArchive(
options: ExtractModelArchiveOptions
): Promise<ModelArchiveDescriptor> {
ensureNotAborted(options.signal)
const maximumArchiveBytes = checkedLimit(
options.maximumArchiveBytes,
'模型 ZIP 大小限制'
)
const maximumFileBytes = checkedLimit(
options.maximumFileBytes,
'模型文件大小限制'
)
const maximumTotalBytes = checkedLimit(
options.maximumTotalBytes,
'模型展开大小限制'
)
const expectedFiles = options.expectedFiles.map((file) => ({
name: ensureArchiveName(file.name),
role: file.role
}))
ensureUniqueFiles(expectedFiles)
const allowedNames = new Set([
ARCHIVE_MANIFEST_NAME,
...expectedFiles.map((file) => file.name)
])
const source = resolve(options.archivePath)
let sourceInfo
try {
sourceInfo = await lstat(source)
} catch (error) {
throw new Error('无法读取模型 ZIP', { cause: error })
}
if (
!sourceInfo.isFile() ||
sourceInfo.isSymbolicLink() ||
sourceInfo.size <= 0 ||
sourceInfo.size > maximumArchiveBytes
) {
throw new Error('模型 ZIP 必须是大小合规的普通文件')
}
let input: FileHandle | undefined
try {
input = await open(source, 'r')
const openedInfo = await input.stat()
if (
!openedInfo.isFile() ||
openedInfo.size !== sourceInfo.size ||
openedInfo.dev !== sourceInfo.dev ||
openedInfo.ino !== sourceInfo.ino
) {
await input.close()
throw new Error('模型 ZIP 在打开前已发生变化')
}
} catch (error) {
await input?.close().catch(() => undefined)
if (error instanceof Error && error.message.startsWith('模型 ZIP')) {
throw error
}
throw new Error('无法读取模型 ZIP', { cause: error })
}
if (!input) {
throw new Error('无法读取模型 ZIP')
}
const destination = resolve(options.destinationDirectory)
const seenNames = new Set<string>()
const openHandles = new Set<FileHandle>()
const completions: Promise<void>[] = []
const pendingWrites = new Set<Promise<void>>()
let entryCount = 0
let totalBytes = 0
let completedModelBytes = 0
let fatalError: Error | undefined
const fail = (error: unknown): Error => {
const resolvedError =
error instanceof Error ? error : new Error('模型 ZIP 已损坏')
fatalError ??= resolvedError
return resolvedError
}
const unzip = new Unzip((file) => {
try {
entryCount += 1
if (
entryCount > MAXIMUM_ARCHIVE_ENTRIES ||
entryCount > allowedNames.size
) {
throw new Error('模型 ZIP 包含过多条目')
}
const name = ensureArchiveName(file.name)
const key = name.toLowerCase()
if (seenNames.has(key)) {
throw new Error('模型 ZIP 包含重复条目')
}
seenNames.add(key)
if (!allowedNames.has(name)) {
throw new Error(`模型 ZIP 包含未声明文件:${name}`)
}
const entryMaximum =
name === ARCHIVE_MANIFEST_NAME
? MAXIMUM_MANIFEST_BYTES
: maximumFileBytes
if (
file.originalSize !== undefined &&
(file.originalSize <= 0 ||
file.originalSize > entryMaximum ||
totalBytes + file.originalSize > maximumTotalBytes)
) {
throw new Error(`模型 ZIP 条目大小超出限制:${name}`)
}
const handlePromise = open(
safeChild(destination, name),
'wx'
).then((handle) => {
openHandles.add(handle)
return handle
})
let written = 0
let writeChain = Promise.resolve()
let resolveEntry: (() => void) | undefined
let rejectEntry: ((error: Error) => void) | undefined
const completion = new Promise<void>((resolveEntryPromise, rejectEntryPromise) => {
resolveEntry = resolveEntryPromise
rejectEntry = rejectEntryPromise
})
completions.push(completion)
file.ondata = (error, data, final) => {
if (error) {
rejectEntry?.(fail(error))
return
}
if (fatalError) {
file.terminate()
rejectEntry?.(fatalError)
return
}
if (options.signal?.aborted) {
file.terminate()
rejectEntry?.(
fail(
options.signal.reason instanceof Error
? options.signal.reason
: new Error('模型 ZIP 导入已取消')
)
)
return
}
written += data.byteLength
totalBytes += data.byteLength
if (name !== ARCHIVE_MANIFEST_NAME) {
completedModelBytes += data.byteLength
options.onProgress?.(completedModelBytes)
}
if (
written > entryMaximum ||
totalBytes > maximumTotalBytes
) {
file.terminate()
rejectEntry?.(
fail(new Error(`模型 ZIP 条目大小超出限制:${name}`))
)
return
}
writeChain = writeChain.then(async () => {
const handle = await handlePromise
if (data.byteLength > 0) {
await handle.write(data)
}
})
const pendingWrite = writeChain
pendingWrites.add(pendingWrite)
void pendingWrite.then(
() => pendingWrites.delete(pendingWrite),
() => pendingWrites.delete(pendingWrite)
)
if (final) {
void writeChain.then(async () => {
const handle = await handlePromise
openHandles.delete(handle)
await closeHandle(handle)
resolveEntry?.()
}, (writeError: unknown) => {
rejectEntry?.(fail(writeError))
})
}
}
file.start()
} catch (error) {
file.terminate()
fail(error)
}
})
unzip.register(UnzipPassThrough)
unzip.register(UnzipInflate)
const buffer = Buffer.allocUnsafe(16 * 1024)
try {
while (true) {
ensureNotAborted(options.signal)
if (fatalError) {
throw fatalError
}
const { bytesRead } = await input.read(buffer, 0, buffer.length)
if (bytesRead === 0) {
unzip.push(new Uint8Array(), true)
break
}
unzip.push(
Uint8Array.from(buffer.subarray(0, bytesRead)),
false
)
await Promise.all([...pendingWrites])
}
await Promise.all(completions)
if (fatalError) {
throw fatalError
}
} catch (error) {
throw fail(error)
} finally {
await input.close()
await Promise.all(
[...openHandles].map((handle) => closeHandle(handle))
)
}
if (
seenNames.size !== allowedNames.size ||
[...allowedNames].some(
(name) => !seenNames.has(name.toLowerCase())
)
) {
throw new Error('模型 ZIP 缺少必需文件')
}
let manifest
try {
manifest = modelArchiveManifestSchema.parse(
JSON.parse(
await readFile(
safeChild(destination, ARCHIVE_MANIFEST_NAME),
'utf8'
)
) as unknown
)
} catch {
throw new Error('模型 ZIP 清单无效')
}
if (
manifest.kind !== options.expectedKind ||
manifest.modelId !== options.expectedModelId
) {
throw new Error('模型 ZIP 类型或模型 ID 不匹配')
}
if (
manifest.files.length !== expectedFiles.length ||
expectedFiles.some((expected) => {
const archived = manifest.files.find(
(file) => file.name === expected.name
)
return !archived || archived.role !== expected.role
})
) {
throw new Error('模型 ZIP 清单与当前模型目录不匹配')
}
for (const archived of manifest.files) {
const path = safeChild(destination, archived.name)
const metadata = await lstat(path)
if (
!metadata.isFile() ||
metadata.isSymbolicLink() ||
metadata.size !== archived.size ||
(await hashFile(path)) !== archived.sha256
) {
throw new Error(`模型 ZIP 文件校验失败:${archived.name}`)
}
}
return {
kind: manifest.kind,
modelId: manifest.modelId,
displayName: manifest.displayName,
files: manifest.files
}
}
@@ -373,6 +373,47 @@ describe('SpeechModelManager downloads', () => {
operations: []
})
})
it('round-trips a verified model through an offline ZIP archive', async () => {
const userData = await temporaryDirectory()
const modelBytes = new TextEncoder().encode('verified model bytes')
const tokenBytes = new TextEncoder().encode('verified tokens')
const catalog = downloadableCatalog(modelBytes, tokenBytes)
const manager = new SpeechModelManager({
userDataDirectory: userData,
catalog,
fetch: vi.fn<typeof fetch>(async (input) => {
const bytes = String(input).endsWith('model.onnx')
? modelBytes
: tokenBytes
return new Response(bytes, {
headers: { 'content-length': String(bytes.byteLength) }
})
})
})
const archive = join(userData, 'speech-model.zip')
await manager.install('download-test-model')
await manager.exportArchive('download-test-model', archive)
await manager.remove('download-test-model')
await expect(
manager.importArchive('download-test-model', archive)
).resolves.toMatchObject({
id: 'download-test-model',
source: 'local',
files: [
{
name: 'model.onnx',
sha256: sha256(modelBytes)
},
{
name: 'tokens.txt',
sha256: sha256(tokenBytes)
}
]
})
})
})
describe('SpeechModelManager local import', () => {
+143
View File
@@ -25,12 +25,18 @@ import {
type SpeechModelSnapshot
} from '../../shared/speech-model-contracts'
import { SPEECH_MODEL_CATALOG } from './speech-model-catalog'
import {
exportModelArchive,
extractModelArchive
} from '../model-archive'
const DEFAULT_MAX_FILE_BYTES = 2 * 1024 * 1024 * 1024
const MAX_REDIRECTS = 3
const MANIFEST_FILE_NAME = 'manifest.json'
const SELECTION_FILE_NAME = '.selection.json'
const PARTIAL_SUFFIX = '.partial'
const MAXIMUM_ARCHIVE_BYTES = 4 * 1024 * 1024 * 1024 - 1
const ARCHIVE_OVERHEAD_BYTES = 1024 * 1024
const selectionSchema = z
.object({
@@ -380,6 +386,143 @@ export class SpeechModelManager {
}
}
async exportArchive(
modelId: string,
destinationPath: string
): Promise<void> {
const entry = this.requireCatalogEntry(modelId)
await this.ensureRoot()
const installed = (await this.readInstalled()).find(
(model) => model.id === entry.id
)
if (!installed) {
throw new Error('只能导出已安装的语音模型')
}
const directory = this.modelDirectory(entry.id)
const files = []
for (const expected of entry.files) {
const recorded = installed.files.find(
(file) =>
file.name === expected.name && file.role === expected.role
)
if (
!recorded ||
recorded.size <= 0 ||
recorded.size > this.maxFileBytes ||
(expected.download &&
(recorded.size !== expected.download.size ||
recorded.sha256 !== expected.download.sha256))
) {
throw new Error(`语音模型文件不可导出:${expected.name}`)
}
files.push({
name: expected.name,
role: expected.role,
size: recorded.size,
sha256: recorded.sha256
})
}
await exportModelArchive({
destinationPath,
sourceDirectory: directory,
descriptor: {
kind: 'speech',
modelId: entry.id,
displayName: entry.displayName,
files
}
})
}
async importArchive(
modelId: string,
archivePath: string
): Promise<InstalledSpeechModel> {
const entry = this.requireCatalogEntry(modelId)
const expectedTotal = entry.files.reduce(
(total, file) =>
total + (file.download?.size ?? this.maxFileBytes),
0
)
const maximumTotalBytes = Math.min(
MAXIMUM_ARCHIVE_BYTES,
expectedTotal + ARCHIVE_OVERHEAD_BYTES
)
const operation = this.beginOperation(
entry.id,
'import',
expectedTotal
)
let stagingDirectory: string | undefined
try {
await this.ensureRoot()
await this.assertNotInstalled(entry.id)
stagingDirectory = await this.createStagingDirectory(entry.id)
operation.progress.phase = 'transferring'
const descriptor = await extractModelArchive({
archivePath,
destinationDirectory: stagingDirectory,
expectedKind: 'speech',
expectedModelId: entry.id,
expectedFiles: entry.files.map((file) => ({
name: file.name,
role: file.role
})),
maximumArchiveBytes: Math.min(
MAXIMUM_ARCHIVE_BYTES,
maximumTotalBytes + ARCHIVE_OVERHEAD_BYTES
),
maximumFileBytes: this.maxFileBytes,
maximumTotalBytes,
signal: operation.controller.signal,
onProgress: (completedBytes) => {
operation.progress.completedBytes = completedBytes
}
})
for (const expected of entry.files) {
const archived = descriptor.files.find(
(file) =>
file.name === expected.name &&
file.role === expected.role
)
if (
!archived ||
archived.size > this.maxFileBytes ||
(expected.download &&
(archived.size !== expected.download.size ||
archived.sha256 !== expected.download.sha256))
) {
throw new Error(
`语音模型 ZIP 与当前模型目录不匹配:${expected.name}`
)
}
}
operation.progress.phase = 'installing'
operation.progress.currentFile = null
const installed = installedSpeechModelSchema.parse({
id: entry.id,
displayName: entry.displayName,
source: 'local',
installedAt: new Date().toISOString(),
files: descriptor.files
})
await writeFile(
safeChild(stagingDirectory, MANIFEST_FILE_NAME),
`${JSON.stringify(installed, null, 2)}\n`,
{ encoding: 'utf8', flag: 'wx' }
)
ensureNotAborted(operation.controller.signal)
await rename(stagingDirectory, this.modelDirectory(entry.id))
stagingDirectory = undefined
return installed
} finally {
this.operations.delete(entry.id)
if (stagingDirectory) {
await rm(stagingDirectory, { recursive: true, force: true })
}
}
}
private async ensureRoot(): Promise<void> {
await mkdir(this.rootDirectory, { recursive: true })
}
+104 -2
View File
@@ -75,6 +75,15 @@ import type {
EmbeddingIndexStatus,
EmbeddingSettingsSnapshot
} from '../shared/embedding-contracts'
import type {
DocumentOcrAssets,
DocumentOcrFailure,
DocumentOcrRequest,
DocumentOcrResult,
DocumentParsingDiagnostic,
DocumentParsingSettings,
DocumentParsingSnapshot
} from '../shared/document-parsing-contracts'
import type { AgentRuntimeSelection } from '../shared/runtime-selection-contracts'
import type { WeixinBindingSnapshot } from '../shared/weixin-channel-contracts'
import type { RemoteChannelActivity } from '../shared/remote-channel-contracts'
@@ -345,9 +354,14 @@ const desktopApi: DesktopApi = {
ipcChannels.speechModelsSelect,
{ modelId }
) as Promise<SpeechModelSnapshot>,
importLocalDirectory: (modelId: string) =>
importArchive: (modelId: string) =>
ipcRenderer.invoke(
ipcChannels.speechModelsImportLocal,
ipcChannels.speechModelsImportArchive,
{ modelId }
) as Promise<SpeechModelSnapshot | undefined>,
exportArchive: (modelId: string) =>
ipcRenderer.invoke(
ipcChannels.speechModelsExportArchive,
{ modelId }
) as Promise<SpeechModelSnapshot | undefined>,
openRepository: async (modelId: string) => {
@@ -403,6 +417,94 @@ const desktopApi: DesktopApi = {
)
}
},
documentParsing: {
getSnapshot: () =>
ipcRenderer.invoke(
ipcChannels.documentParsingGet
) as Promise<DocumentParsingSnapshot>,
update: (input: DocumentParsingSettings) =>
ipcRenderer.invoke(
ipcChannels.documentParsingUpdate,
input
) as Promise<DocumentParsingSnapshot>,
test: () =>
ipcRenderer.invoke(
ipcChannels.documentParsingTest
) as Promise<DocumentParsingDiagnostic | undefined>,
installOcrModel: (modelId: string) =>
ipcRenderer.invoke(
ipcChannels.documentOcrModelsInstall,
{ modelId }
) as Promise<DocumentParsingSnapshot>,
cancelOcrModelOperation: (modelId: string) =>
ipcRenderer.invoke(
ipcChannels.documentOcrModelsCancel,
{ modelId }
) as Promise<boolean>,
removeOcrModel: (modelId: string) =>
ipcRenderer.invoke(
ipcChannels.documentOcrModelsRemove,
{ modelId }
) as Promise<DocumentParsingSnapshot>,
importOcrModelArchive: (modelId: string) =>
ipcRenderer.invoke(
ipcChannels.documentOcrModelsImportArchive,
{ modelId }
) as Promise<DocumentParsingSnapshot | undefined>,
exportOcrModelArchive: (modelId: string) =>
ipcRenderer.invoke(
ipcChannels.documentOcrModelsExportArchive,
{ modelId }
) as Promise<DocumentParsingSnapshot | undefined>,
openOcrModelRepository: async (modelId: string) => {
await ipcRenderer.invoke(
ipcChannels.documentOcrModelsOpenRepository,
{ modelId }
)
},
openOcrModelsDirectory: async () => {
await ipcRenderer.invoke(
ipcChannels.documentOcrModelsOpenDirectory
)
},
getOcrAssets: (modelId: string) =>
ipcRenderer.invoke(
ipcChannels.documentParsingOcrAssets,
{ modelId }
) as Promise<DocumentOcrAssets>,
respondOcr: async (
response: DocumentOcrResult | DocumentOcrFailure
) => {
await ipcRenderer.invoke(
ipcChannels.documentParsingOcrRespond,
response
)
},
onOcrRequest: (listener) => {
const handler = (
_event: Electron.IpcRendererEvent,
request: DocumentOcrRequest
): void => listener(request)
ipcRenderer.on(ipcChannels.documentParsingOcrRequest, handler)
return () =>
ipcRenderer.removeListener(
ipcChannels.documentParsingOcrRequest,
handler
)
},
onOcrCancel: (listener) => {
const handler = (
_event: Electron.IpcRendererEvent,
requestId: string
): void => listener(requestId)
ipcRenderer.on(ipcChannels.documentParsingOcrCancel, handler)
return () =>
ipcRenderer.removeListener(
ipcChannels.documentParsingOcrCancel,
handler
)
}
},
projects: {
list: (includeArchived = false) =>
ipcRenderer.invoke(
+13
View File
@@ -36,4 +36,17 @@ describe('sandboxed preload', () => {
/(?:setComputerCapability|BrowserProfile).{0,80}(?:executablePath|command|env|args)/su
)
})
it('exposes model ZIP dialogs without renderer-controlled paths', () => {
const source = readFileSync(
join(process.cwd(), 'src', 'preload', 'index.ts'),
'utf8'
)
expect(source).toContain('importArchive: (modelId: string)')
expect(source).toContain('exportArchive: (modelId: string)')
expect(source).toContain('importOcrModelArchive: (modelId: string)')
expect(source).toContain('exportOcrModelArchive: (modelId: string)')
expect(source).not.toContain('importLocalDirectory:')
expect(source).not.toContain('importOcrModel:')
})
})
@@ -0,0 +1,377 @@
import {
cleanup,
fireEvent,
render,
screen,
waitFor
} from '@testing-library/react'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import type {
DocumentParsingSettings,
DocumentParsingSnapshot
} from '../../shared/document-parsing-contracts'
import { changeUiLocale } from './i18n'
import { DocumentParsingSettingsSection } from './DocumentParsingSettingsSection'
const settings: DocumentParsingSettings = {
chatWorkflow: 'auto',
knowledgeWorkflow: 'complete-index',
pdfOcrMode: 'auto',
ocrProvider: 'local',
localOcrEnabled: true,
localOcrModelId: 'pp-ocrv6-tiny',
maximumPages: 100,
ocrConcurrency: 1,
pageTimeoutSeconds: 60
}
const modelEntry = {
id: 'pp-ocrv6-tiny' as const,
displayName: 'PP-OCRv6 Tiny',
description: '轻量中文 OCR 模型',
languages: ['中文', '英语'],
runtime: 'onnxruntime-web-wasm' as const,
quality: 'basic' as const,
speed: 'fast' as const,
recommended: false,
repositoryUrl:
'https://modelscope.cn/models/PaddlePaddle/PP-OCRv6_tiny_rec_onnx',
license: {
name: 'Apache License 2.0',
notice: '使用前请阅读模型许可证。',
url: 'https://example.com/license'
},
files: [
{
name: 'detection.onnx',
role: 'detection' as const,
download: {
url: 'https://modelscope.cn/models/example/detection.onnx',
size: 1_000,
sha256: 'a'.repeat(64)
}
},
{
name: 'recognition.onnx',
role: 'recognition' as const,
download: {
url: 'https://modelscope.cn/models/example/recognition.onnx',
size: 2_000,
sha256: 'b'.repeat(64)
}
},
{
name: 'dictionary.yml',
role: 'dictionary' as const,
download: {
url: 'https://modelscope.cn/models/example/dictionary.yml',
size: 500,
sha256: 'c'.repeat(64)
}
}
]
}
const secondModelEntry = {
...modelEntry,
id: 'pp-ocrv6-small',
displayName: 'PP-OCRv6 Small',
quality: 'balanced' as const,
speed: 'balanced' as const,
recommended: true
}
const thirdModelEntry = {
...modelEntry,
id: 'pp-ocrv6-medium',
displayName: 'PP-OCRv6 Medium',
quality: 'high' as const,
speed: 'slow' as const,
recommended: false
}
const snapshot: DocumentParsingSnapshot = {
settings,
status: {
nativeParsingAvailable: true,
conversionAvailable: false,
localOcr: {
id: 'pp-ocrv6-tiny',
displayName: 'PP-OCRv6 Tiny',
available: false,
verified: false,
runtime: 'onnxruntime-web-wasm',
detail: '模型尚未安装'
}
},
ocrModels: {
rootDirectory: 'C:\\Users\\test\\models\\document-ocr',
catalog: [modelEntry, secondModelEntry, thirdModelEntry],
installed: [
{
id: 'pp-ocrv6-small',
displayName: 'PP-OCRv6 Small',
source: 'download',
installedAt: '2026-08-11T00:00:00.000Z',
files: secondModelEntry.files.map((file) => ({
name: file.name,
role: file.role,
size: file.download.size,
sha256: file.download.sha256
}))
}
],
operations: []
}
}
const getSnapshot = vi.fn(async () => snapshot)
const update = vi.fn(async (input: DocumentParsingSettings) => ({
...snapshot,
settings: input
}))
const test = vi.fn(async () => ({
fileName: 'scan.pdf',
sourceFormat: 'PDF',
pageCount: 2,
ocrPageCount: 2,
characterCount: 120,
method: 'ocr' as const,
durationMs: 1_250,
preview: '扫描件识别正文',
warnings: []
}))
const installOcrModel = vi.fn(async () => ({
...snapshot,
status: {
...snapshot.status,
localOcr: {
...snapshot.status.localOcr,
available: true,
verified: true,
detail: '模型已安装并校验'
}
},
ocrModels: {
...snapshot.ocrModels,
installed: [
{
id: 'pp-ocrv6-tiny' as const,
displayName: 'PP-OCRv6 Tiny',
source: 'download' as const,
installedAt: '2026-08-11T00:00:00.000Z',
files: modelEntry.files.map((file) => ({
name: file.name,
role: file.role,
size: file.download.size,
sha256: file.download.sha256
}))
}
]
}
}))
const importOcrModelArchive = vi.fn(async () => snapshot)
const exportOcrModelArchive = vi.fn(async () => snapshot)
const openOcrModelRepository = vi.fn(async () => undefined)
describe('DocumentParsingSettingsSection', () => {
beforeEach(async () => {
await changeUiLocale('zh-CN')
vi.clearAllMocks()
Object.defineProperty(window, 'goodbuddy', {
configurable: true,
value: {
documentParsing: {
getSnapshot,
update,
test,
installOcrModel,
cancelOcrModelOperation: vi.fn(async () => true),
removeOcrModel: vi.fn(async () => snapshot),
importOcrModelArchive,
exportOcrModelArchive,
openOcrModelRepository,
openOcrModelsDirectory: vi.fn(),
getOcrAssets: vi.fn(),
respondOcr: vi.fn(),
onOcrRequest: vi.fn(() => () => undefined),
onOcrCancel: vi.fn(() => () => undefined)
}
}
})
})
afterEach(() => cleanup())
it('shows actual capability status and saves workflow settings', async () => {
const onNotify = vi.fn()
render(
<DocumentParsingSettingsSection onNotify={onNotify} />
)
expect(await screen.findByText('PP-OCRv6 Tiny')).toBeInTheDocument()
expect(screen.getByText('ModelScope')).toBeInTheDocument()
expect(screen.getByText('质量:基础')).toBeInTheDocument()
expect(screen.getByText('速度:快')).toBeInTheDocument()
expect(screen.getByText('旧版 Office 转换')).toBeInTheDocument()
expect(
screen.getByRole('button', { name: '本地模型' })
).toHaveAttribute('aria-pressed', 'true')
expect(
screen.getByRole('button', {
name: '远程服务(即将支持)'
})
).toBeDisabled()
expect(screen.queryByText('隐私与云端处理')).not.toBeInTheDocument()
expect(
screen.queryByText('模型详情与手动导入')
).not.toBeInTheDocument()
fireEvent.click(
screen.getByRole('button', {
name: '打开 PP-OCRv6 Tiny 的 ModelScope 页面'
})
)
expect(openOcrModelRepository).toHaveBeenCalledWith('pp-ocrv6-tiny')
fireEvent.change(screen.getByLabelText('聊天附件'), {
target: { value: 'fast-text' }
})
fireEvent.click(
screen.getByRole('button', { name: '保存设置' })
)
await waitFor(() =>
expect(update).toHaveBeenCalledWith(
expect.objectContaining({ chatWorkflow: 'fast-text' })
)
)
expect(onNotify).toHaveBeenCalledWith(
expect.objectContaining({
message: '文档解析设置已保存'
})
)
})
it('downloads the verified OCR model from the model catalog', async () => {
const onNotify = vi.fn()
render(
<DocumentParsingSettingsSection onNotify={onNotify} />
)
fireEvent.click(
await screen.findByRole('button', {
name: '下载 PP-OCRv6 Tiny'
})
)
await waitFor(() =>
expect(installOcrModel).toHaveBeenCalledWith('pp-ocrv6-tiny')
)
expect(onNotify).toHaveBeenCalledWith(
expect.objectContaining({
message: 'PP-OCRv6 Tiny 已安装'
})
)
})
it('imports and exports verified OCR model ZIP archives', async () => {
const onNotify = vi.fn()
render(
<DocumentParsingSettingsSection onNotify={onNotify} />
)
fireEvent.click(
await screen.findByRole('button', {
name: '从 ZIP 导入 PP-OCRv6 Tiny'
})
)
await waitFor(() =>
expect(importOcrModelArchive).toHaveBeenCalledWith(
'pp-ocrv6-tiny'
)
)
expect(onNotify).toHaveBeenCalledWith(
expect.objectContaining({
message: 'PP-OCRv6 Tiny 已从 ZIP 导入'
})
)
fireEvent.change(screen.getByLabelText('当前 OCR 模型'), {
target: { value: 'pp-ocrv6-small' }
})
fireEvent.click(
screen.getByRole('button', {
name: '将 PP-OCRv6 Small 导出为 ZIP'
})
)
await waitFor(() =>
expect(exportOcrModelArchive).toHaveBeenCalledWith(
'pp-ocrv6-small'
)
)
expect(update).not.toHaveBeenCalled()
expect(onNotify).toHaveBeenCalledWith(
expect.objectContaining({
message: 'PP-OCRv6 Small 已导出为 ZIP'
})
)
})
it('switches the selected OCR model only when settings are saved', async () => {
render(<DocumentParsingSettingsSection />)
const selector = await screen.findByLabelText('当前 OCR 模型')
expect(
screen.getByRole('option', {
name: 'PP-OCRv6 Tiny · 可下载'
})
).toBeInTheDocument()
expect(
screen.getByRole('option', {
name: 'PP-OCRv6 Small · 已安装'
})
).toBeInTheDocument()
expect(
screen.getByRole('option', {
name: 'PP-OCRv6 Medium · 可下载'
})
).toBeInTheDocument()
fireEvent.change(selector, {
target: { value: 'pp-ocrv6-small' }
})
expect(
screen.getByText('模型选择尚未生效,点击“保存设置”后切换。')
).toBeInTheDocument()
expect(screen.getByText('PP-OCRv6 Small')).toBeInTheDocument()
expect(screen.getByText('质量:均衡')).toBeInTheDocument()
expect(screen.getByText('速度:均衡')).toBeInTheDocument()
expect(update).not.toHaveBeenCalled()
fireEvent.click(
screen.getByRole('button', { name: '保存设置' })
)
await waitFor(() =>
expect(update).toHaveBeenCalledWith(
expect.objectContaining({
localOcrModelId: 'pp-ocrv6-small'
})
)
)
})
it('runs a real-file diagnostic flow and displays its result', async () => {
render(<DocumentParsingSettingsSection />)
await screen.findByText('PP-OCRv6 Tiny')
fireEvent.click(
screen.getByRole('button', { name: '测试解析' })
)
expect(
await screen.findByRole('dialog', {
name: '解析测试结果'
})
).toHaveTextContent('扫描件识别正文')
expect(test).toHaveBeenCalledOnce()
})
})
File diff suppressed because it is too large Load Diff
+2 -1
View File
@@ -478,7 +478,8 @@ describe('SettingsPanel runtime files', () => {
cancel: vi.fn(async () => true),
remove: vi.fn(),
select: selectSpeechModel,
importLocalDirectory: vi.fn(),
importArchive: vi.fn(),
exportArchive: vi.fn(),
openRepository: vi.fn(),
openModelsDirectory: vi.fn()
},
+6
View File
@@ -38,6 +38,7 @@ import { UpdateSettingsSection } from './UpdateSettingsSection'
import { PlatformFeaturesSettingsSection } from './PlatformFeaturesSettingsSection'
import { SpeechModelSettingsSection } from './SpeechModelSettingsSection'
import { EmbeddingSettingsSection } from './EmbeddingSettingsSection'
import { DocumentParsingSettingsSection } from './DocumentParsingSettingsSection'
import { PageHeader, SegmentedControl } from './WorkspacePrimitives'
import { SettingsCategoryHeader } from './SettingsPrimitives'
import {
@@ -346,6 +347,7 @@ export function SettingsPanel({
activeTab === 'roles'
const categoryRendersOwnHeader =
activeTab === 'platform-features' ||
activeTab === 'document-parsing' ||
activeTab === 'channels' ||
activeTab === 'skills' ||
activeTab === 'mcp' ||
@@ -2258,6 +2260,10 @@ export function SettingsPanel({
</>
)}
{activeTab === 'document-parsing' && (
<DocumentParsingSettingsSection onNotify={onNotify} />
)}
{activeTab === 'security' && (
<>
<div className="settings-section">
@@ -75,7 +75,8 @@ describe('SpeechModelSettingsSection', () => {
cancel: vi.fn(async () => true),
remove: vi.fn(),
select: vi.fn(),
importLocalDirectory: vi.fn(),
importArchive: vi.fn(),
exportArchive: vi.fn(),
openRepository: vi.fn(),
openModelsDirectory: vi.fn()
}
@@ -128,7 +129,8 @@ describe('SpeechModelSettingsSection', () => {
cancel: vi.fn(async () => true),
remove: vi.fn(),
select: vi.fn(),
importLocalDirectory: vi.fn(),
importArchive: vi.fn(),
exportArchive: vi.fn(),
openRepository: vi.fn(),
openModelsDirectory: vi.fn()
}
@@ -181,7 +183,8 @@ describe('SpeechModelSettingsSection', () => {
cancel: vi.fn(async () => true),
remove: vi.fn(),
select: vi.fn(),
importLocalDirectory: vi.fn(),
importArchive: vi.fn(),
exportArchive: vi.fn(),
openRepository: vi.fn(),
openModelsDirectory: vi.fn()
}
@@ -200,6 +203,86 @@ describe('SpeechModelSettingsSection', () => {
)
})
it('imports and exports verified speech model ZIP archives', async () => {
const installedSnapshot: SpeechModelSnapshot = {
...snapshot,
installed: [
{
id: entry.id,
displayName: entry.displayName,
source: 'local',
installedAt: '2026-08-11T00:00:00.000Z',
files: [
{
name: 'model.int8.onnx',
role: 'model',
size: 1_000,
sha256: 'a'.repeat(64)
}
]
}
]
}
const importArchive = vi.fn(async () => installedSnapshot)
const exportArchive = vi.fn(async () => installedSnapshot)
const select = vi.fn()
const onNotify = vi.fn()
const getSnapshot = vi
.fn<() => Promise<SpeechModelSnapshot>>()
.mockResolvedValueOnce(snapshot)
.mockResolvedValue(installedSnapshot)
Object.defineProperty(window, 'goodbuddy', {
configurable: true,
value: {
speechModels: {
getSnapshot,
install: vi.fn(),
cancel: vi.fn(async () => true),
remove: vi.fn(),
select,
importArchive,
exportArchive,
openRepository: vi.fn(),
openModelsDirectory: vi.fn()
}
} as unknown as DesktopApi
})
render(<SpeechModelSettingsSection onNotify={onNotify} />)
fireEvent.click(
await screen.findByRole('button', {
name: '从 ZIP 导入 SenseVoiceSmall INT8'
})
)
await waitFor(() =>
expect(importArchive).toHaveBeenCalledWith(
'sensevoice-small-int8'
)
)
expect(onNotify).toHaveBeenCalledWith(
expect.objectContaining({
message: 'SenseVoiceSmall INT8 已从 ZIP 导入'
})
)
fireEvent.click(
await screen.findByRole('button', {
name: '将 SenseVoiceSmall INT8 导出为 ZIP'
})
)
await waitFor(() =>
expect(exportArchive).toHaveBeenCalledWith(
'sensevoice-small-int8'
)
)
expect(select).not.toHaveBeenCalled()
expect(onNotify).toHaveBeenCalledWith(
expect.objectContaining({
message: 'SenseVoiceSmall INT8 已导出为 ZIP'
})
)
})
it('shows live progress and cancellation for an active download', async () => {
const active: SpeechModelSnapshot = {
...snapshot,
@@ -224,7 +307,8 @@ describe('SpeechModelSettingsSection', () => {
cancel,
remove: vi.fn(),
select: vi.fn(),
importLocalDirectory: vi.fn(),
importArchive: vi.fn(),
exportArchive: vi.fn(),
openRepository: vi.fn(),
openModelsDirectory: vi.fn()
}
@@ -290,7 +374,8 @@ describe('SpeechModelSettingsSection', () => {
cancel: vi.fn(async () => true),
remove: vi.fn(),
select: vi.fn(),
importLocalDirectory: vi.fn(),
importArchive: vi.fn(),
exportArchive: vi.fn(),
openRepository: vi.fn(),
openModelsDirectory: vi.fn()
}
@@ -355,7 +440,8 @@ describe('SpeechModelSettingsSection', () => {
cancel: vi.fn(async () => true),
remove: vi.fn(),
select,
importLocalDirectory: vi.fn(),
importArchive: vi.fn(),
exportArchive: vi.fn(),
openRepository: vi.fn(),
openModelsDirectory: vi.fn()
}
+50 -24
View File
@@ -6,7 +6,8 @@ import {
FolderOpen,
Mic,
Square,
Trash2
Trash2,
Upload
} from 'lucide-react'
import type { TFunction } from 'i18next'
import { useCallback, useEffect, useRef, useState } from 'react'
@@ -408,24 +409,49 @@ export function SpeechModelSettingsSection({
{t('speech.actions.cancel')}
</button>
) : installed ? (
<button
aria-label={t('speech.accessibility.deleteModel', {
name: displayName
})}
className={
confirmingRemove === entry.id
? 'danger-button'
: 'danger-ghost'
}
disabled={busyModelId === entry.id}
onClick={() => void remove(entry.id)}
type="button"
>
<Trash2 aria-hidden="true" size={12} />
{confirmingRemove === entry.id
? t('speech.actions.confirmDelete')
: t('speech.actions.delete')}
</button>
<>
<button
aria-label={t(
'speech.accessibility.exportModelZip',
{ name: displayName }
)}
className="secondary-button"
disabled={busyModelId === entry.id}
onClick={() =>
void run(
entry.id,
() =>
window.goodbuddy.speechModels!
.exportArchive(entry.id),
t('speech.notifications.exportedZip', {
name: displayName
})
)
}
type="button"
>
<Download aria-hidden="true" size={13} />
{t('speech.actions.exportZip')}
</button>
<button
aria-label={t('speech.accessibility.deleteModel', {
name: displayName
})}
className={
confirmingRemove === entry.id
? 'danger-button'
: 'danger-ghost'
}
disabled={busyModelId === entry.id}
onClick={() => void remove(entry.id)}
type="button"
>
<Trash2 aria-hidden="true" size={12} />
{confirmingRemove === entry.id
? t('speech.actions.confirmDelete')
: t('speech.actions.delete')}
</button>
</>
) : (
<>
{!entry.manualOnly && (
@@ -455,7 +481,7 @@ export function SpeechModelSettingsSection({
</button>
)}
<button
aria-label={t('speech.accessibility.importModel', {
aria-label={t('speech.accessibility.importModelZip', {
name: displayName
})}
className="secondary-button"
@@ -465,16 +491,16 @@ export function SpeechModelSettingsSection({
entry.id,
() =>
window.goodbuddy.speechModels!
.importLocalDirectory(entry.id),
t('speech.notifications.imported', {
.importArchive(entry.id),
t('speech.notifications.importedZip', {
name: displayName
})
)
}
type="button"
>
<FolderOpen aria-hidden="true" size={13} />
{t('speech.actions.import')}
<Upload aria-hidden="true" size={13} />
{t('speech.actions.importZip')}
</button>
</>
)}
+22 -2
View File
@@ -30,6 +30,7 @@ export type PageTab<T extends string> = {
export type SegmentedOption<T extends string> = {
value: T
label: string
disabled?: boolean
}
function nextControlIndex(
@@ -258,11 +259,11 @@ export function SegmentedControl<T extends string>({
? 'segmented-control__option segmented-control__option--active'
: 'segmented-control__option'
}
disabled={disabled}
disabled={disabled || option.disabled}
key={option.value}
onClick={() => onChange(option.value)}
onKeyDown={(event) => {
const nextIndex = nextControlIndex(
let nextIndex = nextControlIndex(
event,
index,
options.length
@@ -270,6 +271,25 @@ export function SegmentedControl<T extends string>({
if (nextIndex === undefined) {
return
}
const direction =
event.key === 'ArrowLeft' ||
event.key === 'ArrowUp' ||
event.key === 'End'
? -1
: 1
for (
let attempts = 0;
attempts < options.length &&
options[nextIndex]?.disabled;
attempts += 1
) {
nextIndex =
(nextIndex + direction + options.length) %
options.length
}
if (options[nextIndex]?.disabled) {
return
}
event.preventDefault()
onChange(options[nextIndex]!.value)
const controls =
+203
View File
@@ -0,0 +1,203 @@
import type {
DocumentOcrFailure,
DocumentOcrRequest,
DocumentOcrResult
} from '../../shared/document-parsing-contracts'
type WorkerOutput =
| { type: 'ready' }
| { type: 'result'; result: DocumentOcrResult }
| { type: 'error'; requestId?: string; error: string }
type PendingWorkerRequest = {
resolve: (result: DocumentOcrResult) => void
reject: (error: Error) => void
timer: number
}
let worker: Worker | undefined
let workerModelId: string | undefined
let workerReady: Promise<void> | undefined
let resolveWorkerReady: (() => void) | undefined
let rejectWorkerReady: ((error: Error) => void) | undefined
const pending = new Map<string, PendingWorkerRequest>()
const cancelledRequestIds = new Set<string>()
let activeRequestId: string | undefined
let requestQueue: Promise<void> = Promise.resolve()
function safeError(error: unknown): string {
return error instanceof Error
? error.message.slice(0, 1_000)
: '本地 OCR 解析失败'
}
function terminateWorker(error: Error): void {
worker?.terminate()
rejectWorkerReady?.(error)
worker = undefined
workerModelId = undefined
workerReady = undefined
resolveWorkerReady = undefined
rejectWorkerReady = undefined
for (const request of pending.values()) {
window.clearTimeout(request.timer)
request.reject(error)
}
pending.clear()
}
async function ensureWorker(modelId: string): Promise<Worker> {
if (worker && workerReady && workerModelId === modelId) {
await workerReady
return worker
}
if (worker) {
terminateWorker(new Error('本地 OCR 模型已切换'))
}
const api = window.goodbuddy.documentParsing
if (!api) {
throw new Error('文档解析服务不可用')
}
worker = new Worker(
new URL('./document-ocr-worker.ts', import.meta.url),
{ type: 'module', name: 'goodbuddy-document-ocr' }
)
workerReady = new Promise<void>((resolve, reject) => {
resolveWorkerReady = resolve
rejectWorkerReady = reject
})
worker.addEventListener(
'message',
(event: MessageEvent<WorkerOutput>) => {
const output = event.data
if (output.type === 'ready') {
resolveWorkerReady?.()
return
}
if (output.type === 'error' && !output.requestId) {
rejectWorkerReady?.(new Error(output.error))
return
}
const requestId =
output.type === 'result'
? output.result.requestId
: output.requestId
if (!requestId) {
return
}
const request = pending.get(requestId)
if (!request) {
return
}
window.clearTimeout(request.timer)
pending.delete(requestId)
if (output.type === 'result') {
request.resolve(output.result)
} else {
request.reject(new Error(output.error))
}
}
)
worker.addEventListener('error', (event) => {
terminateWorker(
new Error(event.message || '本地 OCR Worker 异常')
)
})
try {
const assets = await api.getOcrAssets(modelId)
if (assets.modelId !== modelId) {
throw new Error('本地 OCR 模型与请求不匹配')
}
workerModelId = modelId
worker.postMessage(
{ type: 'initialize', assets },
[
assets.detection,
assets.recognition,
assets.dictionary
]
)
await workerReady
} catch (error) {
terminateWorker(
error instanceof Error ? error : new Error('本地 OCR 初始化失败')
)
throw error
}
return worker
}
async function recognize(
request: DocumentOcrRequest
): Promise<DocumentOcrResult> {
const activeWorker = await ensureWorker(request.modelId)
if (cancelledRequestIds.has(request.requestId)) {
throw new Error('本地 OCR 解析已取消')
}
const pageCount = request.pageNumbers?.length ?? request.maximumPages
const timeoutMs = Math.min(
10 * 60 * 1_000,
Math.max(
request.pageTimeoutSeconds * 1_000,
request.pageTimeoutSeconds * pageCount * 1_000
)
)
return new Promise<DocumentOcrResult>((resolve, reject) => {
const timer = window.setTimeout(() => {
pending.delete(request.requestId)
terminateWorker(new Error('本地 OCR 解析超时'))
reject(new Error('本地 OCR 解析超时'))
}, timeoutMs)
pending.set(request.requestId, { resolve, reject, timer })
activeWorker.postMessage(
{ type: 'recognize', request },
[request.data]
)
})
}
async function handleRequest(request: DocumentOcrRequest): Promise<void> {
const api = window.goodbuddy.documentParsing
if (!api || cancelledRequestIds.has(request.requestId)) {
cancelledRequestIds.delete(request.requestId)
return
}
activeRequestId = request.requestId
try {
await api.respondOcr(await recognize(request))
} catch (error) {
const failure: DocumentOcrFailure = {
requestId: request.requestId,
error: safeError(error)
}
await api.respondOcr(failure).catch(() => undefined)
} finally {
activeRequestId = undefined
cancelledRequestIds.delete(request.requestId)
}
}
export function installDocumentOcrBridge(): () => void {
const api = window.goodbuddy.documentParsing
if (!api) {
return () => undefined
}
const removeRequestListener = api.onOcrRequest((request) => {
requestQueue = requestQueue
.then(() => handleRequest(request))
.catch(() => undefined)
})
const removeCancelListener = api.onOcrCancel((requestId) => {
cancelledRequestIds.add(requestId)
if (activeRequestId === requestId) {
terminateWorker(new Error('本地 OCR 解析已取消'))
}
})
return () => {
removeRequestListener()
removeCancelListener()
cancelledRequestIds.clear()
activeRequestId = undefined
terminateWorker(new Error('本地 OCR 服务已关闭'))
}
}
+233
View File
@@ -0,0 +1,233 @@
/// <reference lib="webworker" />
import * as ort from 'onnxruntime-web'
import wasmModuleUrl from '../../../node_modules/onnxruntime-web/dist/ort-wasm-simd-threaded.mjs?url'
import wasmBinaryUrl from '../../../node_modules/onnxruntime-web/dist/ort-wasm-simd-threaded.wasm?url'
import pdfWorkerUrl from 'pdfjs-dist/build/pdf.worker.min.mjs?url'
import type { PDFPageProxy } from 'pdfjs-dist/types/src/display/api'
import type {
DocumentOcrAssets,
DocumentOcrRequest,
DocumentOcrResult
} from '../../shared/document-parsing-contracts'
type InitializeMessage = {
type: 'initialize'
assets: DocumentOcrAssets
}
type RecognizeMessage = {
type: 'recognize'
request: DocumentOcrRequest
}
type WorkerInput = InitializeMessage | RecognizeMessage
type WorkerOutput =
| { type: 'ready' }
| { type: 'result'; result: DocumentOcrResult }
| { type: 'error'; requestId?: string; error: string }
type OcrService = InstanceType<
typeof import('ppu-paddle-ocr/web').PaddleOcrService
>
const worker = self as DedicatedWorkerGlobalScope
let service: OcrService | undefined
function absoluteAssetUrl(value: string): string {
return new URL(value, worker.location.href).href
}
ort.env.wasm.numThreads = 1
ort.env.wasm.proxy = false
ort.env.wasm.wasmPaths = {
mjs: absoluteAssetUrl(wasmModuleUrl),
wasm: absoluteAssetUrl(wasmBinaryUrl)
}
function safeWorkerError(error: unknown): string {
return error instanceof Error
? error.message.slice(0, 1_000)
: '本地 OCR 识别失败'
}
async function initialize(assets: DocumentOcrAssets): Promise<void> {
await service?.destroy()
const { PaddleOcrService } = await import('ppu-paddle-ocr/web')
service = new PaddleOcrService({
model: {
detection: assets.detection,
recognition: assets.recognition,
charactersDictionary: assets.dictionary
},
session: {
executionProviders: ['wasm'],
graphOptimizationLevel: 'disabled'
},
processing: {
engine: 'canvas-native'
},
recognition: {
charactersDictionary: [],
minimumConfidence: 0.5,
strategy: 'per-line',
recBatchSize: 4
},
detection: {
maxSideLength: 1920
}
})
await service.initialize()
}
async function recognizeImage(
data: ArrayBuffer,
locator: string
): Promise<DocumentOcrResult['sections'][number] | undefined> {
if (!service?.isInitialized()) {
throw new Error('本地 OCR 模型尚未初始化')
}
const result = await service.recognize(data)
const content = result.text.replace(/\n{3,}/gu, '\n\n').trim()
return content
? {
locator,
content,
confidence: result.confidence
}
: undefined
}
async function renderPdfPage(
page: PDFPageProxy
): Promise<ArrayBuffer> {
const baseViewport = page.getViewport({ scale: 1 })
const scale = Math.min(
2,
2200 / Math.max(baseViewport.width, baseViewport.height, 1)
)
const viewport = page.getViewport({ scale })
const canvas = new OffscreenCanvas(
Math.max(1, Math.ceil(viewport.width)),
Math.max(1, Math.ceil(viewport.height))
)
const context = canvas.getContext('2d', {
alpha: false,
willReadFrequently: true
})
if (!context) {
throw new Error('无法创建 PDF 页面渲染画布')
}
await page.render({
canvas: canvas as unknown as HTMLCanvasElement,
canvasContext: context as unknown as CanvasRenderingContext2D,
viewport
}).promise
const blob = await canvas.convertToBlob({
type: 'image/png'
})
return blob.arrayBuffer()
}
async function recognizePdf(
request: DocumentOcrRequest
): Promise<DocumentOcrResult> {
const pdfjs = await import('pdfjs-dist')
pdfjs.GlobalWorkerOptions.workerSrc = pdfWorkerUrl
const loadingTask = pdfjs.getDocument({
data: new Uint8Array(request.data)
})
const document = await loadingTask.promise
const selectedPages = new Set(
request.pageNumbers ??
Array.from(
{ length: Math.min(document.numPages, request.maximumPages) },
(_, index) => index + 1
)
)
if (document.numPages > request.maximumPages) {
await loadingTask.destroy()
throw new Error(
`PDF 共 ${document.numPages} 页,超过 ${request.maximumPages} 页限制`
)
}
const sections: DocumentOcrResult['sections'] = []
const warnings: string[] = []
try {
for (
let pageNumber = 1;
pageNumber <= document.numPages;
pageNumber += 1
) {
if (!selectedPages.has(pageNumber)) {
continue
}
const page = await document.getPage(pageNumber)
try {
const section = await recognizeImage(
await renderPdfPage(page),
`${pageNumber}`
)
if (section) {
sections.push(section)
} else {
warnings.push(`${pageNumber} 页未识别到文字`)
}
} finally {
page.cleanup()
}
}
} finally {
await loadingTask.destroy()
}
return {
requestId: request.requestId,
sections,
pageCount: document.numPages,
warnings
}
}
async function recognize(request: DocumentOcrRequest): Promise<DocumentOcrResult> {
if (request.mimeType === 'application/pdf') {
return recognizePdf(request)
}
const section = await recognizeImage(request.data, '图片')
return {
requestId: request.requestId,
sections: section ? [section] : [],
pageCount: 1,
warnings: section ? [] : ['图片中未识别到文字']
}
}
worker.addEventListener('message', (event: MessageEvent<WorkerInput>) => {
const input = event.data
if (input.type === 'initialize') {
void initialize(input.assets).then(
() => worker.postMessage({ type: 'ready' } satisfies WorkerOutput),
(error: unknown) =>
worker.postMessage({
type: 'error',
error: safeWorkerError(error)
} satisfies WorkerOutput)
)
return
}
void recognize(input.request).then(
(result) =>
worker.postMessage({
type: 'result',
result
} satisfies WorkerOutput),
(error: unknown) =>
worker.postMessage({
type: 'error',
requestId: input.request.requestId,
error: safeWorkerError(error)
} satisfies WorkerOutput)
)
})
export {}
+174 -3
View File
@@ -26,6 +26,12 @@ export const settings = {
navigationDescription: 'LLMs, embedding models, and credentials',
description: 'LLMs, embedding models, and credentials'
},
documentParsing: {
label: 'Document parsing',
navigationDescription: 'Attachments, knowledge, and local OCR',
description:
'Configure extraction, conversion, and OCR for chat attachments and knowledge imports'
},
runtime: {
label: 'Agent Runtime',
navigationDescription: 'OpenCode, Continue, and workspace settings',
@@ -76,6 +82,8 @@ export const settings = {
saveAndTestRuntime: 'Save and test {{runtime}}',
saving: 'Saving…',
saveSettings: 'Save settings',
testParsing: 'Test parsing',
testingParsing: 'Parsing…',
select: 'Select',
selectFile: 'Select file',
clear: 'Clear',
@@ -115,11 +123,18 @@ export const settings = {
openRuntimeConfig: 'Could not open the Runtime configuration',
selectWorkspace: 'Could not select the workspace folder',
retainModelConnection: 'Keep at least one model connection',
clearLocalData: 'Could not clear local data'
clearLocalData: 'Could not clear local data',
documentParsingUnavailable: 'Document parsing is unavailable',
readDocumentParsing: 'Could not load document parsing settings',
saveDocumentParsing: 'Could not save document parsing settings',
testDocumentParsing: 'Document parsing test failed',
manageDocumentOcrModel: 'OCR model operation failed'
},
notifications: {
settingsSaved: 'Settings saved',
connectionSucceeded: 'Connected: {{label}}'
connectionSucceeded: 'Connected: {{label}}',
documentParsingSaved: 'Document parsing settings saved',
documentParsingTestSucceeded: 'Document parsing test completed'
},
credentials: {
none: 'Not configured',
@@ -212,6 +227,162 @@ export const settings = {
'Continue remains unavailable without a configuration file and will not load a remote default model anonymously.'
}
},
documentParsing: {
status: {
title: 'Runtime status',
description: 'Capabilities currently available on this device',
available: 'Available',
unavailable: 'Unavailable',
verified: 'Verified',
native: 'Native document parsing',
nativeDetail:
'Text, HTML, text PDFs, and modern Office documents',
conversion: 'Legacy Office conversion',
conversionUnavailable:
'Not implemented yet; DOC, XLS, and PPT are currently unavailable',
localOcr: 'Local OCR',
ocrReady:
'The model is installed, SHA-256 verified, and available offline',
ocrUnavailable:
'The model is not installed or failed verification. Download it from ModelScope.',
partialNotice:
'Basic document parsing is available. Legacy Office conversion is not implemented yet; scanned PDFs use local OCR.'
},
workflows: {
title: 'Usage scenarios',
description:
'Choose different parsing depth for chat attachments and knowledge imports',
chat: 'Chat attachments',
chatDescription:
'Controls parsing before an attachment is added to the current request',
knowledge: 'Knowledge imports',
knowledgeDescription:
'Controls parsing before chunking, indexing, and source location',
chatOptions: {
auto: 'Automatic parsing (recommended)',
fastText: 'Fast text',
highFidelity: 'High-fidelity parsing'
},
knowledgeOptions: {
completeIndex: 'Complete indexing (recommended)',
fastIndex: 'Fast indexing',
highFidelity: 'High-fidelity indexing'
}
},
ocr: {
title: 'OCR recognition',
description:
'Install a local model on demand to recognize scanned PDFs on this device',
enabled: 'Enable local OCR',
enabledDescription:
'After installation, the model runs only on this device through ONNX Runtime WebAssembly. Documents are not uploaded for recognition.',
model: 'Local model',
runtime: 'Runtime',
provider: {
title: 'OCR source',
description:
'Choose either a local model or a remote service, then save settings to switch.',
local: 'Local model',
remote: 'Remote service (coming soon)',
remoteDescription:
'Remote integrations will support services such as MinerU and PaddleOCR-VL. They are disabled in this version.'
},
modelSelector: 'Current OCR model',
modelSelectorDescription:
'This saved model is used for chat attachments and knowledge imports.',
pendingSelection:
'This model selection is not active yet. Save settings to switch.',
installedOption: 'Installed',
downloadableOption: 'Available to download',
openModelsDirectory: 'Open model folder',
storagePrefix: 'Models are installed on demand in',
storageSuffix:
' and can be exported as ZIP archives for offline devices.',
recommended: 'Recommended',
quality: {
label: 'Quality: {{value}}',
values: {
basic: 'Basic',
balanced: 'Balanced',
high: 'High'
}
},
speed: {
label: 'Speed: {{value}}',
values: {
fast: 'Fast',
balanced: 'Balanced',
slow: 'Slow'
}
},
installed: 'Installed and verified',
availableToDownload: 'Available from ModelScope',
download: 'Download',
importZip: 'Import ZIP',
exportZip: 'Export ZIP',
delete: 'Delete',
confirmDelete: 'Confirm delete',
cancel: 'Cancel',
openRepository: 'Open ModelScope',
catalogUnavailable:
'No OCR model catalog is available in this version.',
mode: 'PDF OCR strategy',
modes: {
auto: 'Automatic; recognize only pages without useful text',
always: 'Always recognize every page',
disabled: 'Use only the PDF text layer'
},
modelLicense:
'The model uses Apache License 2.0 and is SHA-256 verified before loading.',
operations: {
preparing: 'Preparing model files',
downloading: 'Downloading from ModelScope',
importing: 'Importing model ZIP',
installing: 'Verifying and installing'
},
accessibility: {
downloadModel: 'Download {{name}}',
importModelZip: 'Import {{name}} from a ZIP archive',
exportModelZip: 'Export {{name}} as a ZIP archive',
deleteModel: 'Delete {{name}}',
cancelOperation: 'Cancel {{name}} operation',
downloadProgress: '{{name}} download progress',
openRepository: 'Open the ModelScope page for {{name}}'
},
notifications: {
installed: '{{name}} installed',
importedZip: '{{name}} imported from ZIP',
exportedZip: '{{name}} exported as ZIP',
removed: 'OCR model deleted'
}
},
advanced: {
title: 'Advanced parsing settings',
maximumPages: 'Maximum OCR pages per document',
concurrency: 'OCR concurrency',
timeout: 'OCR time budget per page (seconds)',
concurrencyHint:
'The WASM baseline currently processes pages serially; this value is reserved for batching and hardware acceleration.'
},
diagnostic: {
title: 'Parsing test result',
file: 'File',
format: 'Format',
method: 'Method',
pages: 'Pages',
ocrPages: 'OCR pages',
characters: 'Extracted characters',
duration: 'Duration',
preview: 'Text preview',
warnings: 'Warnings',
methods: {
native: 'Native parsing',
ocr: 'Local OCR',
mixed: 'Native parsing and OCR'
},
close: 'Close result'
}
},
model: {
typeAriaLabel: 'Model type',
types: {
@@ -228,7 +399,7 @@ export const settings = {
speech: {
label: 'Speech model',
description:
'Select an installed model and save Settings to apply it; models can be downloaded or imported from a local folder.'
'Select an installed model and save Settings to apply it; models can be downloaded or moved offline with ZIP archives.'
}
},
profile: {
@@ -7,11 +7,11 @@ export const settingsSections = {
speech: {
title: 'Speech models',
description:
'Model weights are not bundled. Download them as needed or import them from a local directory.',
'Model weights are not bundled. Download them as needed or move them offline with ZIP archives.',
openModelsDirectory: 'Open models directory',
storagePrefix: 'Models are stored in',
storageSuffix:
'. Automatic downloads pin the source revision and verify file sizes and SHA-256 hashes. You can also download models from their repositories and import them.',
'. Automatic downloads pin the source revision and verify SHA-256 hashes. Export a ZIP on an online device and import it directly on an offline device.',
availableModels: 'Available speech models',
loading: 'Loading speech models…',
errors: {
@@ -59,7 +59,8 @@ export const settingsSections = {
delete: 'Delete',
confirmDelete: 'Confirm delete',
download: 'Download',
import: 'Import',
importZip: 'Import ZIP',
exportZip: 'Export ZIP',
modelDetails: 'Model details',
openRepository: 'Open model repository'
},
@@ -69,13 +70,15 @@ export const settingsSections = {
cancelOperation: 'Cancel the {{name}} operation',
deleteModel: 'Delete {{name}}',
downloadModel: 'Download {{name}}',
importModel: 'Import {{name}} from a local directory',
importModelZip: 'Import {{name}} from a ZIP archive',
exportModelZip: 'Export {{name}} as a ZIP archive',
downloadProgress: '{{name}} download progress',
openRepository: 'Open the {{name}} model repository'
},
notifications: {
installed: '{{name}} installed',
imported: '{{name}} imported from a local directory',
importedZip: '{{name}} imported from ZIP',
exportedZip: '{{name}} exported as ZIP',
removed: 'Speech model deleted'
},
details: {
+160 -3
View File
@@ -22,6 +22,11 @@ export const settings = {
navigationDescription: 'LLM、向量模型与凭据',
description: 'LLM、向量模型与凭据'
},
documentParsing: {
label: '文档解析',
navigationDescription: '附件、知识库与本地 OCR',
description: '统一配置聊天附件和知识库使用的提取、转换与 OCR 策略'
},
runtime: {
label: 'Agent Runtime',
navigationDescription: 'OpenCode、Continue 与工作区',
@@ -69,6 +74,8 @@ export const settings = {
saveAndTestRuntime: '保存并测试 {{runtime}}',
saving: '保存中…',
saveSettings: '保存设置',
testParsing: '测试解析',
testingParsing: '正在解析…',
select: '选择',
selectFile: '选择文件',
clear: '清除',
@@ -105,11 +112,18 @@ export const settings = {
openRuntimeConfig: '打开 Runtime 配置失败',
selectWorkspace: '选择工作区目录失败',
retainModelConnection: '请至少保留一个模型连接',
clearLocalData: '本地数据清除失败'
clearLocalData: '本地数据清除失败',
documentParsingUnavailable: '文档解析服务不可用',
readDocumentParsing: '读取文档解析设置失败',
saveDocumentParsing: '保存文档解析设置失败',
testDocumentParsing: '测试文档解析失败',
manageDocumentOcrModel: 'OCR 模型操作失败'
},
notifications: {
settingsSaved: '设置已保存',
connectionSucceeded: '连接成功:{{label}}'
connectionSucceeded: '连接成功:{{label}}',
documentParsingSaved: '文档解析设置已保存',
documentParsingTestSucceeded: '文档解析测试完成'
},
credentials: {
none: '尚未配置',
@@ -195,6 +209,149 @@ export const settings = {
'未指定配置文件时 Continue 将保持不可用,不会匿名加载远程默认模型。'
}
},
documentParsing: {
status: {
title: '运行状态',
description: '显示当前设备实际可用的解析能力',
available: '可用',
unavailable: '不可用',
verified: '已校验',
native: '原生文档解析',
nativeDetail: '文本、HTML、文本型 PDF 和新式 Office 文档',
conversion: '旧版 Office 转换',
conversionUnavailable: '尚未实现,DOC、XLS、PPT 暂不可用',
localOcr: '本地 OCR',
ocrReady: '模型已安装并通过 SHA-256 校验,可离线使用',
ocrUnavailable: '模型尚未安装或校验失败,请从 ModelScope 下载',
partialNotice:
'基础文档解析可用。旧版 Office 转换尚未实现;扫描 PDF 使用本地 OCR。'
},
workflows: {
title: '使用场景',
description: '为聊天附件和知识库选择不同的解析深度',
chat: '聊天附件',
chatDescription: '控制附件加入当前请求前的解析方式',
knowledge: '知识库导入',
knowledgeDescription: '控制文档分块、索引和来源定位前的解析方式',
chatOptions: {
auto: '自动解析(推荐)',
fastText: '快速文本',
highFidelity: '高保真解析'
},
knowledgeOptions: {
completeIndex: '完整索引(推荐)',
fastIndex: '快速索引',
highFidelity: '高保真索引'
}
},
ocr: {
title: 'OCR 识别',
description: '按需安装本地模型,在设备上识别扫描 PDF',
enabled: '启用本地 OCR',
enabledDescription:
'模型安装后仅在本机通过 ONNX Runtime WebAssembly 运行,识别时不会上传文档。',
model: '本地模型',
runtime: '运行时',
provider: {
title: 'OCR 来源',
description: '本地模型与远程服务二选一,切换后保存设置生效。',
local: '本地模型',
remote: '远程服务(即将支持)',
remoteDescription:
'远程服务将支持 MinerU、PaddleOCR-VL 等接口,当前版本暂不可选。'
},
modelSelector: '当前 OCR 模型',
modelSelectorDescription: '选择已保存,聊天附件和知识库将使用此模型。',
pendingSelection: '模型选择尚未生效,点击“保存设置”后切换。',
installedOption: '已安装',
downloadableOption: '可下载',
openModelsDirectory: '打开模型目录',
storagePrefix: '模型按需安装到',
storageSuffix: '。可导出 ZIP,并在内网设备直接导入。',
recommended: '推荐',
quality: {
label: '质量:{{value}}',
values: {
basic: '基础',
balanced: '均衡',
high: '高'
}
},
speed: {
label: '速度:{{value}}',
values: {
fast: '快',
balanced: '均衡',
slow: '慢'
}
},
installed: '已安装并校验',
availableToDownload: '可从 ModelScope 下载',
download: '下载',
importZip: '导入 ZIP',
exportZip: '导出 ZIP',
delete: '删除',
confirmDelete: '确认删除',
cancel: '取消',
openRepository: '打开 ModelScope',
catalogUnavailable: '当前版本没有可用的 OCR 模型目录。',
mode: 'PDF OCR 策略',
modes: {
auto: '自动,仅识别无有效文本的页面',
always: '始终识别所有页面',
disabled: '仅使用 PDF 文本层'
},
modelLicense:
'模型采用 Apache License 2.0,并在加载前校验 SHA-256。',
operations: {
preparing: '正在准备模型文件',
downloading: '正在从 ModelScope 下载',
importing: '正在导入模型 ZIP',
installing: '正在校验并安装'
},
accessibility: {
downloadModel: '下载 {{name}}',
importModelZip: '从 ZIP 导入 {{name}}',
exportModelZip: '将 {{name}} 导出为 ZIP',
deleteModel: '删除 {{name}}',
cancelOperation: '取消 {{name}} 操作',
downloadProgress: '{{name}} 下载进度',
openRepository: '打开 {{name}} 的 ModelScope 页面'
},
notifications: {
installed: '{{name}} 已安装',
importedZip: '{{name}} 已从 ZIP 导入',
exportedZip: '{{name}} 已导出为 ZIP',
removed: 'OCR 模型已删除'
}
},
advanced: {
title: '高级解析设置',
maximumPages: '单文档最大 OCR 页数',
concurrency: 'OCR 并发数',
timeout: '每页 OCR 时间预算(秒)',
concurrencyHint:
'当前 WASM 基线按页串行执行;该值为后续批处理和硬件加速保留。'
},
diagnostic: {
title: '解析测试结果',
file: '文件',
format: '格式',
method: '处理方式',
pages: '页数',
ocrPages: 'OCR 页数',
characters: '提取字符',
duration: '耗时',
preview: '文本预览',
warnings: '警告',
methods: {
native: '原生解析',
ocr: '本地 OCR',
mixed: '原生解析与 OCR'
},
close: '关闭结果'
}
},
model: {
typeAriaLabel: '模型类型',
types: {
@@ -209,7 +366,7 @@ export const settings = {
speech: {
label: '语音模型',
description:
'选择已安装模型后保存设置生效;模型可按需下载或从本地目录导入。'
'选择已安装模型后保存设置生效;模型可按需下载或通过 ZIP 离线迁移。'
}
},
profile: {
@@ -1,11 +1,11 @@
export const settingsSections = {
speech: {
title: '语音模型',
description: '应用不内置模型权重,按需下载或从本地目录导入',
description: '应用不内置模型权重,按需下载或通过 ZIP 离线迁移',
openModelsDirectory: '打开模型目录',
storagePrefix: '模型保存在',
storageSuffix:
'。自动下载会固定来源版本并校验文件大小和 SHA-256也可以从模型仓库手动下载后导入。',
'。自动下载会固定来源版本并校验 SHA-256;外网设备可导出 ZIP,内网设备可直接导入。',
availableModels: '可用语音模型',
loading: '正在读取语音模型…',
errors: {
@@ -52,7 +52,8 @@ export const settingsSections = {
delete: '删除',
confirmDelete: '确认删除',
download: '下载',
import: '导入',
importZip: '导入 ZIP',
exportZip: '导出 ZIP',
modelDetails: '模型详情',
openRepository: '打开模型仓库'
},
@@ -62,13 +63,15 @@ export const settingsSections = {
cancelOperation: '取消 {{name}} 操作',
deleteModel: '删除 {{name}}',
downloadModel: '下载 {{name}}',
importModel: '从本地目录导入 {{name}}',
importModelZip: '从 ZIP 导入 {{name}}',
exportModelZip: '将 {{name}} 导出为 ZIP',
downloadProgress: '{{name}}下载进度',
openRepository: '打开 {{name}} 模型仓库'
},
notifications: {
installed: '{{name}} 已安装',
imported: '{{name}} 已从本地目录导入',
importedZip: '{{name}} 已从 ZIP 导入',
exportedZip: '{{name}} 已导出为 ZIP',
removed: '语音模型已删除'
},
details: {
+3
View File
@@ -17,6 +17,7 @@ import {
loadAppearanceTheme,
resolveAppearanceTheme
} from './theme'
import { installDocumentOcrBridge } from './document-ocr-bridge'
import './styles.css'
const root = document.getElementById('root')
@@ -43,6 +44,8 @@ applyAppearanceTheme(
)
)
installDocumentOcrBridge()
createRoot(root).render(
<StrictMode>
<UiLocaleProvider
+4
View File
@@ -13,6 +13,10 @@ export const settingsCategoryList = [
id: 'model',
translationKey: 'model'
},
{
id: 'document-parsing',
translationKey: 'documentParsing'
},
{
id: 'runtime',
translationKey: 'runtime'
+451
View File
@@ -5093,6 +5093,457 @@ details.settings-section > :not(summary) + :not(summary) {
}
}
.document-parsing-status__list {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: var(--space-3);
}
.document-parsing-status__row {
display: grid;
min-width: 0;
align-items: center;
padding: var(--space-3);
border: 1px solid var(--border-default);
border-radius: var(--radius-card);
background: var(--surface-raised);
grid-template-columns: auto minmax(0, 1fr);
gap: var(--space-2) var(--space-3);
}
.document-parsing-status__row > svg {
color: var(--text-muted);
}
.document-parsing-status__row:has(
.document-parsing-status__badge--available
)
> svg {
color: var(--success);
}
.document-parsing-status__row > span:nth-child(2),
.document-parsing-model > span:first-child {
display: grid;
min-width: 0;
gap: var(--space-1);
}
.document-parsing-status__row strong,
.document-parsing-model strong {
color: var(--text-primary);
font-size: var(--font-body);
}
.document-parsing-status__row small,
.document-parsing-model small {
color: var(--text-muted);
font-size: var(--font-caption);
line-height: 1.5;
}
.document-parsing-status__badge {
width: fit-content;
padding: var(--space-1) var(--space-2);
border-radius: var(--radius-control);
background: var(--surface-muted);
color: var(--text-muted);
font-size: var(--font-caption);
grid-column: 2;
white-space: nowrap;
}
.document-parsing-status__badge--available {
background: var(--success-subtle);
color: var(--success);
}
.document-parsing-grid {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: var(--space-4);
}
.document-parsing-grid .field > small {
color: var(--text-muted);
font-size: var(--font-caption);
line-height: 1.5;
}
.document-parsing-workflows .field {
padding: var(--space-3);
border: 1px solid var(--border-default);
border-radius: var(--radius-card);
background: var(--surface-raised);
}
.document-ocr-settings .settings-section__title--actions > button,
.document-ocr-model__actions,
.document-ocr-model__actions button,
.document-ocr-model__repository {
display: flex;
align-items: center;
}
.document-ocr-settings .settings-section__title--actions > button,
.document-ocr-model__actions button,
.document-ocr-model__repository {
gap: var(--space-2);
}
.document-ocr-settings__storage {
overflow: hidden;
}
.document-ocr-settings__storage code {
overflow-wrap: anywhere;
}
.document-ocr-provider {
display: grid;
align-items: center;
padding: var(--space-3);
border: 1px solid var(--border-default);
border-radius: var(--radius-card);
background: var(--surface-raised);
grid-template-columns: minmax(0, 1fr) auto;
gap: var(--space-2) var(--space-4);
}
.document-ocr-provider > div:first-child {
display: grid;
gap: var(--space-1);
}
.document-ocr-provider strong {
color: var(--text-primary);
font-size: var(--font-body);
}
.document-ocr-provider small {
color: var(--text-muted);
font-size: var(--font-caption);
line-height: 1.5;
}
.document-ocr-provider > small {
grid-column: 1 / -1;
}
.document-ocr-model-selector {
padding: var(--space-3);
border: 1px solid var(--border-default);
border-radius: var(--radius-card);
background: var(--surface-raised);
}
.document-ocr-model-selector > small {
color: var(--text-muted);
font-size: var(--font-caption);
line-height: 1.5;
}
.document-ocr-model {
display: grid;
min-width: 0;
align-items: start;
padding: var(--space-3);
border: 1px solid var(--border-default);
border-radius: var(--radius-card);
background: var(--surface-raised);
grid-template-columns: minmax(0, 1fr) minmax(150px, auto);
gap: var(--space-2) var(--space-4);
}
.document-ocr-model__summary {
display: grid;
min-width: 0;
gap: var(--space-1);
}
.document-ocr-model__header {
display: flex;
min-width: 0;
align-items: flex-start;
justify-content: space-between;
grid-column: 1 / -1;
gap: var(--space-4);
}
.document-ocr-model__repository {
min-height: 30px;
flex: 0 0 auto;
white-space: nowrap;
}
.document-ocr-model__name,
.document-ocr-model__tags,
.document-ocr-model__status {
display: flex;
align-items: center;
}
.document-ocr-model__name,
.document-ocr-model__tags {
flex-wrap: wrap;
gap: var(--space-2);
}
.document-ocr-model__summary strong {
color: var(--text-primary);
font-size: var(--font-body);
}
.document-ocr-model__summary p {
margin: 0;
color: var(--text-secondary);
font-size: var(--font-caption);
line-height: 1.55;
}
.document-ocr-model__state {
display: flex;
justify-content: flex-start;
}
.document-ocr-model__status {
color: var(--text-muted);
font-size: var(--font-caption);
font-weight: 650;
gap: var(--space-1);
white-space: nowrap;
}
.document-ocr-model__status--installed {
color: var(--success);
}
.document-ocr-model__actions {
justify-content: flex-end;
flex-wrap: wrap;
grid-column: 2;
gap: var(--space-2);
}
.document-ocr-model__actions button {
min-height: 30px;
white-space: nowrap;
}
.document-ocr-model__actions .danger-ghost {
display: flex;
align-items: center;
padding: 0 var(--space-2);
border: 1px solid transparent;
border-radius: var(--radius-control);
background: transparent;
color: var(--danger);
font: inherit;
font-size: var(--font-caption);
gap: var(--space-1);
}
.document-ocr-model__actions .danger-ghost:hover {
border-color: var(--danger-border);
background: var(--danger-subtle);
}
.document-ocr-model__operation {
display: grid;
grid-column: 1 / -1;
gap: var(--space-1);
}
.document-ocr-model__operation progress {
width: 100%;
accent-color: var(--accent-solid);
}
.document-ocr-model__operation small {
color: var(--text-muted);
font-size: var(--font-caption);
}
.document-ocr-settings__options {
display: grid;
align-items: start;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: var(--space-4);
}
.document-ocr-settings__options > * {
min-height: 100%;
padding: var(--space-3);
border: 1px solid var(--border-default);
border-radius: var(--radius-card);
background: var(--surface-raised);
}
.document-ocr-settings__options .settings-checkbox {
display: grid;
align-items: start;
grid-template-columns: auto minmax(0, 1fr);
gap: var(--space-3);
}
.document-ocr-settings__options .settings-checkbox > input {
width: 16px;
height: 16px;
margin: 2px 0 0;
accent-color: var(--accent-solid);
}
.document-ocr-settings__options .settings-checkbox > span {
display: grid;
gap: var(--space-1);
}
.document-ocr-settings__options .settings-checkbox strong {
color: var(--text-primary);
font-size: var(--font-body);
}
.document-ocr-settings__options .settings-checkbox small {
color: var(--text-muted);
font-size: var(--font-caption);
line-height: 1.5;
}
.document-parsing-diagnostic-backdrop {
position: fixed;
z-index: var(--z-dialog);
display: grid;
padding: var(--space-4);
background: rgb(5 12 24 / 58%);
inset: 0;
place-items: center;
}
.document-parsing-diagnostic {
display: grid;
width: min(620px, 100%);
max-height: min(760px, calc(100vh - 32px));
overflow-y: auto;
padding: var(--space-4);
border: 1px solid var(--border-default);
border-radius: var(--radius-card);
background: var(--surface-raised);
box-shadow: var(--shadow-dialog);
gap: var(--space-4);
}
.document-parsing-diagnostic > header {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: var(--space-4);
}
.document-parsing-diagnostic > header > div {
display: grid;
gap: var(--space-1);
}
.document-parsing-diagnostic > header strong {
color: var(--text-primary);
font-size: var(--font-section-title);
}
.document-parsing-diagnostic > header small {
color: var(--text-muted);
font-size: var(--font-caption);
}
.document-parsing-diagnostic dl {
display: grid;
margin: 0;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: var(--space-3);
}
.document-parsing-diagnostic dl > div {
display: grid;
padding: var(--space-3);
border-radius: var(--radius-control);
background: var(--surface-subtle);
gap: var(--space-1);
}
.document-parsing-diagnostic dt {
color: var(--text-muted);
font-size: var(--font-caption);
}
.document-parsing-diagnostic dd {
margin: 0;
color: var(--text-primary);
font-size: var(--font-body);
font-weight: 650;
}
.document-parsing-diagnostic__preview {
display: grid;
min-width: 0;
gap: var(--space-2);
}
.document-parsing-diagnostic__preview pre {
max-height: 260px;
overflow: auto;
margin: 0;
padding: var(--space-3);
border: 1px solid var(--border-subtle);
border-radius: var(--radius-control);
background: var(--surface-subtle);
color: var(--text-secondary);
font-family: var(--font-family-mono);
font-size: var(--font-caption);
line-height: 1.55;
white-space: pre-wrap;
word-break: break-word;
}
@media (max-width: 720px) {
.document-parsing-status__list,
.document-parsing-grid,
.document-parsing-diagnostic dl,
.document-ocr-settings__options {
grid-template-columns: minmax(0, 1fr);
}
.document-ocr-settings .settings-section__title--actions {
align-items: flex-start;
flex-wrap: wrap;
}
.document-ocr-provider {
grid-template-columns: minmax(0, 1fr);
}
.document-ocr-provider > small {
grid-column: 1;
}
.document-ocr-model {
grid-template-columns: minmax(0, 1fr);
}
.document-ocr-model__header {
align-items: stretch;
flex-direction: column;
}
.document-ocr-model__repository {
width: fit-content;
}
.document-ocr-model__state,
.document-ocr-model__actions {
justify-content: flex-start;
grid-column: 1;
}
}
.role-prompt-empty {
min-height: 180px;
padding: var(--space-6);
+45 -1
View File
@@ -74,6 +74,15 @@ import type {
EmbeddingIndexStatus,
EmbeddingSettingsSnapshot
} from './embedding-contracts'
import type {
DocumentOcrAssets,
DocumentOcrFailure,
DocumentOcrRequest,
DocumentOcrResult,
DocumentParsingDiagnostic,
DocumentParsingSettings,
DocumentParsingSnapshot
} from './document-parsing-contracts'
import type { WeixinBindingSnapshot } from './weixin-channel-contracts'
import type { RemoteChannelActivity } from './remote-channel-contracts'
import {
@@ -1038,7 +1047,10 @@ export type DesktopApi = {
cancel: (modelId: string) => Promise<boolean>
remove: (modelId: string) => Promise<SpeechModelSnapshot>
select: (modelId: string | null) => Promise<SpeechModelSnapshot>
importLocalDirectory: (
importArchive: (
modelId: string
) => Promise<SpeechModelSnapshot | undefined>
exportArchive: (
modelId: string
) => Promise<SpeechModelSnapshot | undefined>
openRepository: (modelId: string) => Promise<void>
@@ -1059,6 +1071,38 @@ export type DesktopApi = {
listener: (status: EmbeddingIndexStatus) => void
) => () => void
}
documentParsing?: {
getSnapshot: () => Promise<DocumentParsingSnapshot>
update: (
input: DocumentParsingSettings
) => Promise<DocumentParsingSnapshot>
test: () => Promise<DocumentParsingDiagnostic | undefined>
installOcrModel: (
modelId: string
) => Promise<DocumentParsingSnapshot>
cancelOcrModelOperation: (modelId: string) => Promise<boolean>
removeOcrModel: (
modelId: string
) => Promise<DocumentParsingSnapshot>
importOcrModelArchive: (
modelId: string
) => Promise<DocumentParsingSnapshot | undefined>
exportOcrModelArchive: (
modelId: string
) => Promise<DocumentParsingSnapshot | undefined>
openOcrModelRepository: (modelId: string) => Promise<void>
openOcrModelsDirectory: () => Promise<void>
getOcrAssets: (modelId: string) => Promise<DocumentOcrAssets>
respondOcr: (
response: DocumentOcrResult | DocumentOcrFailure
) => Promise<void>
onOcrRequest: (
listener: (request: DocumentOcrRequest) => void
) => () => void
onOcrCancel: (
listener: (requestId: string) => void
) => () => void
}
projects: {
list: (includeArchived?: boolean) => Promise<AssistantProject[]>
create: (input: ProjectCreateInput) => Promise<AssistantProject>
+310
View File
@@ -0,0 +1,310 @@
import { z } from 'zod'
export const documentParsingPurposeSchema = z.enum([
'chat-attachment',
'knowledge-index',
'diagnostic'
])
export const chatDocumentWorkflowSchema = z.enum([
'auto',
'fast-text',
'high-fidelity'
])
export const knowledgeDocumentWorkflowSchema = z.enum([
'complete-index',
'fast-index',
'high-fidelity'
])
export const pdfOcrModeSchema = z.enum([
'auto',
'always',
'disabled'
])
export const documentOcrProviderSchema = z.literal('local')
export const localOcrModelIdSchema = z
.string()
.min(1)
.max(96)
.regex(/^[a-z0-9]+(?:-[a-z0-9]+)*$/u)
const documentOcrSha256Schema = z
.string()
.regex(/^[a-f0-9]{64}$/u)
export const documentOcrModelFileRoleSchema = z.enum([
'detection',
'recognition',
'dictionary'
])
export const documentOcrModelDownloadSchema = z
.object({
url: z.url().max(2_048),
size: z.number().int().positive().safe(),
sha256: documentOcrSha256Schema
})
.strict()
export const documentOcrModelFileSchema = z
.object({
name: z
.string()
.min(1)
.max(255)
.regex(/^[^/\\\0]+$/u),
role: documentOcrModelFileRoleSchema,
download: documentOcrModelDownloadSchema
})
.strict()
export const documentOcrModelCatalogEntrySchema = z
.object({
id: localOcrModelIdSchema,
displayName: z.string().trim().min(1).max(120),
description: z.string().trim().min(1).max(500),
languages: z.array(z.string().trim().min(1).max(32)).min(1).max(32),
runtime: z.literal('onnxruntime-web-wasm'),
quality: z.enum(['basic', 'balanced', 'high']),
speed: z.enum(['fast', 'balanced', 'slow']),
recommended: z.boolean(),
repositoryUrl: z.url().max(2_048),
license: z
.object({
name: z.string().trim().min(1).max(120),
notice: z.string().trim().min(1).max(1_000),
url: z.url().max(2_048)
})
.strict(),
files: z.array(documentOcrModelFileSchema).length(3)
})
.strict()
.superRefine((entry, context) => {
if (
new Set(entry.files.map((file) => file.name)).size !==
entry.files.length
) {
context.addIssue({
code: 'custom',
path: ['files'],
message: 'OCR 模型文件名不能重复'
})
}
if (
new Set(entry.files.map((file) => file.role)).size !==
entry.files.length
) {
context.addIssue({
code: 'custom',
path: ['files'],
message: 'OCR 模型文件角色不能重复'
})
}
})
export const installedDocumentOcrModelSchema = z
.object({
id: localOcrModelIdSchema,
displayName: z.string().trim().min(1).max(120),
source: z.enum(['download', 'local']),
installedAt: z.string().datetime(),
files: z
.array(
z
.object({
name: z.string().min(1).max(255).regex(/^[^/\\\0]+$/u),
role: documentOcrModelFileRoleSchema,
size: z.number().int().positive().safe(),
sha256: documentOcrSha256Schema
})
.strict()
)
.length(3)
})
.strict()
export const documentOcrModelOperationSchema = z
.object({
modelId: localOcrModelIdSchema,
kind: z.enum(['download', 'import']),
phase: z.enum(['preparing', 'transferring', 'installing']),
currentFile: z.string().min(1).max(255).nullable(),
completedBytes: z.number().int().nonnegative().safe(),
totalBytes: z.number().int().nonnegative().safe().nullable()
})
.strict()
export const documentOcrModelSnapshotSchema = z
.object({
rootDirectory: z.string().min(1).max(32_768),
catalog: z.array(documentOcrModelCatalogEntrySchema).max(16),
installed: z.array(installedDocumentOcrModelSchema).max(16),
operations: z.array(documentOcrModelOperationSchema).max(8)
})
.strict()
export const documentOcrModelActionInputSchema = z
.object({
modelId: localOcrModelIdSchema
})
.strict()
export const documentParsingSettingsSchema = z
.object({
chatWorkflow: chatDocumentWorkflowSchema,
knowledgeWorkflow: knowledgeDocumentWorkflowSchema,
pdfOcrMode: pdfOcrModeSchema,
ocrProvider: documentOcrProviderSchema,
localOcrEnabled: z.boolean(),
localOcrModelId: localOcrModelIdSchema,
maximumPages: z.number().int().min(1).max(500),
ocrConcurrency: z.number().int().min(1).max(4),
pageTimeoutSeconds: z.number().int().min(10).max(300)
})
.strict()
export const documentParsingSettingsUpdateSchema =
documentParsingSettingsSchema
export const documentParsingModelStatusSchema = z
.object({
id: localOcrModelIdSchema,
displayName: z.string().trim().min(1).max(120),
available: z.boolean(),
verified: z.boolean(),
runtime: z.literal('onnxruntime-web-wasm'),
detail: z.string().trim().min(1).max(500)
})
.strict()
export const documentParsingStatusSchema = z
.object({
nativeParsingAvailable: z.literal(true),
conversionAvailable: z.boolean(),
localOcr: documentParsingModelStatusSchema
})
.strict()
export const documentParsingSnapshotSchema = z
.object({
settings: documentParsingSettingsSchema,
status: documentParsingStatusSchema,
ocrModels: documentOcrModelSnapshotSchema
})
.strict()
export const documentParsingTestInputSchema = z
.object({
purpose: documentParsingPurposeSchema.default('diagnostic')
})
.strict()
export const documentParsingDiagnosticSchema = z
.object({
fileName: z.string().trim().min(1).max(500),
sourceFormat: z.string().trim().min(1).max(32),
pageCount: z.number().int().nonnegative().max(10_000),
ocrPageCount: z.number().int().nonnegative().max(10_000),
characterCount: z.number().int().nonnegative().safe(),
method: z.enum(['native', 'ocr', 'mixed']),
durationMs: z.number().int().nonnegative().safe(),
preview: z.string().max(2_000),
warnings: z.array(z.string().trim().min(1).max(500)).max(20)
})
.strict()
export const documentOcrAssetsSchema = z
.object({
modelId: localOcrModelIdSchema,
detection: z.instanceof(ArrayBuffer),
recognition: z.instanceof(ArrayBuffer),
dictionary: z.instanceof(ArrayBuffer)
})
.strict()
export const documentOcrRequestSchema = z
.object({
requestId: z.string().uuid(),
modelId: localOcrModelIdSchema,
fileName: z.string().trim().min(1).max(500),
mimeType: z.enum([
'application/pdf',
'image/jpeg',
'image/png',
'image/webp'
]),
data: z
.instanceof(ArrayBuffer)
.refine(
(value) => value.byteLength > 0 && value.byteLength <= 20 * 1024 * 1024,
'OCR 输入必须介于 1 字节和 20MB 之间'
),
maximumPages: z.number().int().min(1).max(500),
pageNumbers: z
.array(z.number().int().min(1).max(10_000))
.min(1)
.max(500)
.optional(),
pageTimeoutSeconds: z.number().int().min(10).max(300)
})
.strict()
export const documentOcrSectionSchema = z
.object({
locator: z.string().trim().min(1).max(500),
content: z.string().trim().min(1).max(1_000_000),
confidence: z.number().min(0).max(1)
})
.strict()
export const documentOcrResultSchema = z
.object({
requestId: z.string().uuid(),
sections: z.array(documentOcrSectionSchema).max(500),
pageCount: z.number().int().nonnegative().max(10_000),
warnings: z.array(z.string().trim().min(1).max(500)).max(20)
})
.strict()
export const documentOcrFailureSchema = z
.object({
requestId: z.string().uuid(),
error: z.string().trim().min(1).max(1_000)
})
.strict()
export type DocumentParsingPurpose = z.infer<
typeof documentParsingPurposeSchema
>
export type DocumentParsingSettings = z.infer<
typeof documentParsingSettingsSchema
>
export type DocumentParsingSnapshot = z.infer<
typeof documentParsingSnapshotSchema
>
export type DocumentOcrModelFile = z.infer<
typeof documentOcrModelFileSchema
>
export type DocumentOcrModelCatalogEntry = z.infer<
typeof documentOcrModelCatalogEntrySchema
>
export type InstalledDocumentOcrModel = z.infer<
typeof installedDocumentOcrModelSchema
>
export type DocumentOcrModelOperation = z.infer<
typeof documentOcrModelOperationSchema
>
export type DocumentOcrModelSnapshot = z.infer<
typeof documentOcrModelSnapshotSchema
>
export type DocumentParsingDiagnostic = z.infer<
typeof documentParsingDiagnosticSchema
>
export type DocumentOcrAssets = z.infer<typeof documentOcrAssetsSchema>
export type DocumentOcrRequest = z.infer<typeof documentOcrRequestSchema>
export type DocumentOcrResult = z.infer<typeof documentOcrResultSchema>
export type DocumentOcrFailure = z.infer<typeof documentOcrFailureSchema>
+20 -1
View File
@@ -48,7 +48,8 @@ export const ipcChannels = {
speechModelsCancel: 'settings:speech-models:cancel',
speechModelsRemove: 'settings:speech-models:remove',
speechModelsSelect: 'settings:speech-models:select',
speechModelsImportLocal: 'settings:speech-models:import-local',
speechModelsImportArchive: 'settings:speech-models:import-archive',
speechModelsExportArchive: 'settings:speech-models:export-archive',
speechModelsOpenRepository: 'settings:speech-models:open-repository',
speechModelsOpenDirectory: 'settings:speech-models:open-directory',
speechTranscribe: 'speech:transcribe',
@@ -58,6 +59,24 @@ export const ipcChannels = {
embeddingIndexRebuild: 'settings:embedding:index:rebuild',
embeddingIndexCancel: 'settings:embedding:index:cancel',
embeddingIndexStatusChanged: 'settings:embedding:index:status-changed',
documentParsingGet: 'settings:document-parsing:get',
documentParsingUpdate: 'settings:document-parsing:update',
documentParsingTest: 'settings:document-parsing:test',
documentOcrModelsInstall: 'settings:document-ocr-models:install',
documentOcrModelsCancel: 'settings:document-ocr-models:cancel',
documentOcrModelsRemove: 'settings:document-ocr-models:remove',
documentOcrModelsImportArchive:
'settings:document-ocr-models:import-archive',
documentOcrModelsExportArchive:
'settings:document-ocr-models:export-archive',
documentOcrModelsOpenRepository:
'settings:document-ocr-models:open-repository',
documentOcrModelsOpenDirectory:
'settings:document-ocr-models:open-directory',
documentParsingOcrAssets: 'document-parsing:ocr:assets',
documentParsingOcrRequest: 'document-parsing:ocr:request',
documentParsingOcrRespond: 'document-parsing:ocr:respond',
documentParsingOcrCancel: 'document-parsing:ocr:cancel',
projectsList: 'projects:list',
projectsCreate: 'projects:create',
projectsUpdate: 'projects:update',