diff --git a/BUILD.md b/BUILD.md index 12baa17..65fa7a1 100644 --- a/BUILD.md +++ b/BUILD.md @@ -125,6 +125,7 @@ npm run dist:linux:arm64 - OpenCode 平台二进制来自 `.runtime-resources/`。 - Continue Runtime 来自锁定版本的 `@continuedev/cli`。 - DSH 插件安装使用精确锁定并从 `app.asar` 解包的 npm CLI,通过当前 Electron 的 Node 模式运行;最终用户不需要另装 Node.js 或 npm。 +- DSH 图片输入使用精确锁定的 `@napi-rs/canvas` 完整解码 JPEG/PNG。通用包与当前平台的 Skia 原生包必须从 `app.asar` 解包;发布校验会检查版本、目标架构和 MIT 许可证。 - 打包钩子位于 `build/runtime-hooks.cjs`。 跨架构打包前,确认目标架构的 OpenCode 资源已经准备完成。不要用其他架构的二进制替代目标资源。 @@ -181,7 +182,7 @@ git push github "$tag" 4. 本地知识库导入、检索和知识图谱。 5. Ask、Execute 的权限边界与旧版 Plan 数据兼容。 6. OpenCode 与 Continue 的权限边界、取消和超时。 -7. DeepSeek Harness Ask 拒绝写入和第三方插件工具,可调用 Main 管理的 Web Search/Fetch;Execute 可调用已启用插件工具。 +7. DeepSeek Harness Ask 拒绝写入和第三方插件工具,可调用 Main 管理的 Web Search/Fetch;Execute 可调用已启用插件工具。文本模型在网络调用前拒绝图片,声明图片能力的模型可以实际接收 JPEG/PNG。 8. OpenCode Agent/Command、原生上下文 Compact,以及 Continue Rules/Prompt 预设、结构化提问和 GoodBuddy 手动摘要压缩。 9. Runtime 原生清单把 Tools 与 Commands/LSP/Formatters 分开,显示来源及 Ask/Execute 可用性,不混入 GoodBuddy 分配的 Skills/MCP;外部 OpenCode 只报告连接状态,Continue 明确标记原生 Tools 静态发现不支持;MCP 测试只读取有界 Prompt/Resource 元数据,不读取 Resource 内容。 10. DSH 市场可安装、停用、重新启用和移除插件;启动失败插件不会阻止 Host,并显示为自动停用。 diff --git a/FEATURES.md b/FEATURES.md index 475eb69..e51b500 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -21,7 +21,7 @@ - [x] **直连模型 Runtime**:支持问答、知识总结、受控工具执行和图像生成。 - [x] **OpenCode 与 Continue**:使用隔离子进程、环境变量白名单、统一配置、取消、超时和活动记录。 -- [x] **DeepSeek Harness(预览)**:使用 GoodBuddy 固定 Host 和 OpenAI 兼容模型连接;Ask 只允许调用 Host 中真实注册的 `read`、`skill` 以及 Main 管理的 Web Search/Fetch 代理,拒绝插件同名冒充,Execute 放行全部已启用内置及插件工具,并以当前用户权限运行。 +- [x] **DeepSeek Harness(预览)**:使用 GoodBuddy 固定 Host 和 OpenAI 兼容模型连接;Ask 只允许调用 Host 中真实注册的 `read`、`skill` 以及 Main 管理的 Web Search/Fetch 代理,拒绝插件同名冒充,Execute 放行全部已启用内置及插件工具,并以当前用户权限运行。图像输入跟随所选模型连接的能力声明,文本模型在 Host 或模型调用前拒绝图片,图片模型通过有界内联内容和临时 Attachment Store 接收 JPEG/PNG。 - [x] **DSH npm 插件市场**:市场默认关闭,由用户显式开启后搜索公共 npm 的 `dsh-plugin` 包,使用捆绑 npm 执行精确版本安装和普通 lifecycle scripts,并支持启停、JSON 配置、移除、失败启动自动停用和离线管理已安装插件;关闭市场只隐藏目录与管理界面,不改变已有插件的启停状态,第三方代码不受 Ask 初始化隔离。 - [x] **Ask 与 Execute 工作模式**:Ask 保持只读;Execute 运行已启用且受边界约束的工具。 - [x] **专家与 Subagent**:支持显式专家、团队分析和最多三个只读专家并行分析。 @@ -32,7 +32,8 @@ - [x] **OpenCode Runtime 定制**:GoodBuddy 管理的内置 OpenCode 可发现原生 Agents、Tools、Commands、LSP、Formatters、MCP、Skills、Prompts 与 Resources;Tools 单独显示读取、文件修改、命令、网络、Agent 编排等类型、来源及 Ask/Execute 可用性,并隐藏 OpenCode 内部 `invalid` 与 GoodBuddy 临时 MCP 工具。支持保存默认 Agent、每次请求覆盖 Agent、通过原生 SDK 执行 Command、显示上下文用量并调用原生 Compact;外部 OpenCode Server 只报告连接状态,不宣称原生清单可读。任意插件安装、Session Share、自动 Worktree 和 OpenCode 原生会话持久化仍不开放。 - [x] **Continue Runtime 定制**:提供静态配置中的原生 Rules、Prompt 模板与 MCP 清单,以及可编辑的 GoodBuddy Rules/Prompt 配置预设;聊天可按请求选择预设和填入可继续编辑的 Prompt。当前 Continue Host 没有可信的静态原生 Tool 发现接口,且使用隔离的 `CONTINUE_GLOBAL_DIR`,因此界面明确标记 Tools 不支持静态发现,也不把 Host 实际不会加载的工作区或用户 Skills 冒充原生能力;GoodBuddy 分配的 Skills 仍按请求暂存执行。Continue 临时 Host 不复用原生会话压缩,手动压缩由 GoodBuddy 摘要模型完成并验证持久化摘要覆盖范围;Agent 交互提问转换为统一问答卡片。Resources、Hooks、后台 Job 和 Continue 原生会话管理继续暂缓。 - [x] **Runtime 原生清单语义**:原生能力以 Agents、Tools、Commands、Skills、MCP、Rules、Prompts、Resources、LSP、Formatters 和上下文 11 个页签展示;清单状态独立于 Runtime 连通性,区分完整、部分、不可用、仅连接和不支持。DeepSeek Harness 通过 Host Registry 枚举有界的内置/插件 Tools 与 Skills,显示真实 Ask/Execute 边界,并排除 GoodBuddy 按请求分配的 Skills、Web/MCP 代理。 -- [ ] **可执行 Subagent**(规划中):提供显式 Execute 委派,限制嵌套、并行、Token、时间和工具权限,并保留父子任务审计。 +- [ ] **Runtime 监督侧栏**(规划中):在聊天右侧助手工作栏统一承载 OpenCode、Continue 和 DeepSeek Harness 的 Subagent 控制、后台 Job、Workflow/Hook、长任务与原生会话监督;Composer 只保留对当前消息生效的高频上下文选择。 +- [ ] **可执行 Subagent**(规划中):提供显式 Execute 委派,限制嵌套、并行、Token、时间和工具权限,在右侧 Runtime 监督页签显示父子状态、取消入口和审计归属。 ### Skills、MCP 与知识库 diff --git a/UI-DESIGN.md b/UI-DESIGN.md index c4d97c2..c24421c 100644 --- a/UI-DESIGN.md +++ b/UI-DESIGN.md @@ -472,11 +472,14 @@ GoodBuddy 是可调整窗口大小的桌面应用。响应式设计优先保证 - 对话标题和当前项目范围位于 `PageHeader` 或对话上下文区,不在消息流中重复。 - 模式、模型或工具权限属于上下文控制,不与页面导航页签混用。 - 模型、专家角色、工作模式、OpenCode Agent、Continue 预设和 Runtime 快捷操作使用统一的上下文单选菜单,并保持菜单互斥、键盘可达和选中状态明确。 +- 输入区第一行工具栏只承载附件、语音、知识范围、专家角色、工作模式、Runtime 选择和发送等通用操作。OpenCode Agent、Continue 预设及 Runtime 快捷操作必须放入其下方独立的 Runtime 专属功能行,通过可见分组名称、顶部边界和差异化表面与通用操作分层;该行只承载对当前消息生效的高频选择,当前 Runtime 没有可选专属功能时不保留空行。 +- OpenCode、Continue 和 DeepSeek Harness 后续的 Subagent 层级与取消、后台 Job 队列/进度/结果、Workflow/Hook 运行、长任务暂停/恢复/终止及原生会话监督统一进入右侧助手工作栏的“Runtime”页签,不加入 Composer。侧栏按当前会话和 Runtime 能力动态显示区块,不为未支持能力渲染空卡片或成排禁用按钮;切换会话或 Runtime 时必须同步清理上一归属的监督状态。 +- 设置中心只管理持久 Runtime 配置、默认值和能力清单;右侧 Runtime 页签只管理当前活动会话的生命周期。两处不得复制同一实时操作,侧栏中的高风险操作仍须就地确认并保留取消、权限、用量和活动审计。 - Runtime Prompt 快捷操作只把模板填入输入草稿,用户可以继续编辑;OpenCode Command 由 Runtime 原生 API 执行,输入框只承载可选参数,不以普通斜杠文本冒充执行。 - Agent 回复进行中锁定模型、专家角色、工作模式和 Runtime 定制选择器,并关闭已打开的上下文菜单;回复结束或停止后再恢复选择,避免界面状态与本次运行实际使用的上下文不一致。 -- 支持上下文状态的 Runtime 在输入区下方复用同一紧凑用量条;文案必须区分“本次模型调用”和“压缩后对话估算”。手动压缩仅在当前 Runtime 明确支持且没有活动回复时显示,进行中禁用重复操作,结果通过应用通知反馈。 +- 支持上下文状态的 Runtime 在输入区下方复用同一紧凑用量条;文案必须区分“本次模型调用”和“压缩后对话估算”。手动压缩仅在当前 Runtime 明确支持且没有活动回复时显示,作为元信息区左下角的浮动次操作,不参与输入区高度计算;元信息区始终预留稳定高度,切换 Runtime 不得让输入框上下位移。元信息区与窗口底部只保留紧凑安全留白,不形成额外空白区。进行中禁用重复操作,结果通过应用通知反馈。 - 已选择的工作模式在触发按钮中只显示 `Ask` 或 `Execute`;完整中文含义和说明保留在菜单选项、可访问名称及输入区下方的模式说明中。 -- 宽度大于 `700px` 时,添加内容、知识范围、专家、模式和模型控件保持同一行;仅在窄输入区中换行,不能因为允许换行而让所有窗口都固定显示两行。 +- 宽度大于 `700px` 时,通用工具栏内的添加内容、知识范围、专家、模式和 Runtime 选择保持同一行,Runtime 专属功能在自己的下一行横向排列。窄输入区中两行分别换行,专属选择器以至少 `220px` 的基准宽度换行而不是被挤压;不能把专属控件重新塞回通用工具栏。 - 输入框原生支持 `Ctrl+V`:文本直接进入草稿,图片转换为本次消息附件。文件选择由上传按钮承担,不再提供独立“读取剪贴板”按钮;默认工具栏也不提供“截取当前屏幕”和“选择应用窗口”入口,避免与系统粘贴、文件选择和后续工具执行重复。 - “Enter 发送 · Shift+Enter 换行 · Ctrl+V 粘贴图片或文本”等输入操作提示放在空输入框内部,作为主占位文案的次级行;不得在输入框下方单独占用第二行。输入框下方只保留一行当前模式、安全边界或全局快捷键说明。 - 输入操作提示不能替代表单的可访问名称,输入框始终保留持久的程序化标签。 diff --git a/build/build-release.cjs b/build/build-release.cjs index 8136a90..5bbdccc 100644 --- a/build/build-release.cjs +++ b/build/build-release.cjs @@ -52,6 +52,7 @@ const harnessHostEntry = const harnessBundleManifest = 'out/main/package.json' const harnessPackageVersions = { '@deepseek-ai/dsh-agent': '0.1.0-rc.6', + '@napi-rs/canvas': '1.0.3', 'node-pty': '1.1.0' } const koffiVersion = '3.1.4' @@ -60,6 +61,7 @@ const harnessLicenseFiles = [ 'deepseek-cordis-MIT.txt', 'deepseek-harness-MIT.txt', 'koffi-MIT.txt', + 'napi-rs-canvas-MIT.txt', 'node-pty-MIT.txt' ] const portableRequiredFiles = [ @@ -505,7 +507,14 @@ function targetHarnessPaths(options) { macos: `darwin_${options.arch}/koffi.node`, linux: `linux_${options.arch}/koffi.node` }[options.platform] + const canvasTarget = { + windows: `win32-${options.arch}-msvc`, + macos: `darwin-${options.arch}`, + linux: `linux-${options.arch}-gnu` + }[options.platform] return { + canvasPackage: `@napi-rs/canvas-${canvasTarget}`, + canvasBinary: `skia.${canvasTarget}.node`, koffiPackage, koffiBinary, nodePtyBinary: @@ -860,6 +869,18 @@ function verifyHarnessPackage( `${target.koffiPackage} 版本错误:期望 ${koffiVersion},实际 ${String(targetKoffiManifest.version)}` ) } + const targetCanvasManifest = readJson( + `node_modules/${target.canvasPackage}/package.json`, + `${target.canvasPackage} 元数据` + ) + if ( + targetCanvasManifest.version !== + harnessPackageVersions['@napi-rs/canvas'] + ) { + throw new Error( + `${target.canvasPackage} 版本错误:期望 ${harnessPackageVersions['@napi-rs/canvas']},实际 ${String(targetCanvasManifest.version)}` + ) + } const ptyBinary = join( unpackedRoot, @@ -873,6 +894,12 @@ function verifyHarnessPackage( ...target.koffiPackage.split('/'), ...target.koffiBinary.split('/') ) + const canvasBinary = join( + unpackedRoot, + 'node_modules', + ...target.canvasPackage.split('/'), + target.canvasBinary + ) assertBinaryArchitecture( ptyBinary, options.arch, @@ -892,9 +919,17 @@ function verifyHarnessPackage( 'DeepSeek Harness Koffi 元数据', statAsarFile ) + const canvasMetadata = asarEntryMetadata( + asarPath, + entries, + `node_modules/${target.canvasPackage}/${target.canvasBinary}`, + 'DeepSeek Harness Canvas 元数据', + statAsarFile + ) for (const [metadata, description] of [ [nodePtyMetadata, 'DeepSeek Harness node-pty'], - [koffiMetadata, 'DeepSeek Harness Koffi'] + [koffiMetadata, 'DeepSeek Harness Koffi'], + [canvasMetadata, 'DeepSeek Harness Canvas'] ]) { if (!('unpacked' in metadata) || !metadata.unpacked) { throw new Error(`${description}未从 ASAR 解包`) @@ -905,6 +940,11 @@ function verifyHarnessPackage( options.arch, 'DeepSeek Harness Koffi' ) + assertBinaryArchitecture( + canvasBinary, + options.arch, + 'DeepSeek Harness Canvas' + ) if (options.platform === 'darwin') { const helper = join( diff --git a/build/deepseek-harness-utility-smoke.cjs b/build/deepseek-harness-utility-smoke.cjs index 271599d..1f9e304 100644 --- a/build/deepseek-harness-utility-smoke.cjs +++ b/build/deepseek-harness-utility-smoke.cjs @@ -17,8 +17,9 @@ const { const { app, utilityProcess } = require('electron/main') const protocol = 'goodbuddy.deepseek-harness.control' -const version = 1 +const controlVersion = 2 const byteProtocol = 'goodbuddy.deepseek-harness.byte-stream' +const byteProtocolVersion = 1 const configuredHostPath = process.env.GOODBUDDY_HARNESS_SMOKE_HOST const hostPath = configuredHostPath @@ -132,12 +133,12 @@ async function run() { child.on('message', (message) => { if ( message?.protocol === protocol && - message.version === version && + message.version === controlVersion && message.type === 'ready' ) { child.postMessage({ protocol: byteProtocol, - version, + version: byteProtocolVersion, type: 'data', stream: 'stdin', seq: 0, @@ -147,7 +148,7 @@ async function run() { } if ( message?.protocol === byteProtocol && - message.version === version && + message.version === byteProtocolVersion && message.type === 'ack' && message.stream === 'stdin' && message.seq === 0 @@ -158,7 +159,7 @@ async function run() { } if ( message?.protocol === protocol && - message.version === version && + message.version === controlVersion && message.type === 'fatal' ) { finish('fatal', String(message.code)) @@ -172,7 +173,7 @@ async function run() { }) child.postMessage({ protocol, - version, + version: controlVersion, type: 'start', config: { workspace, @@ -181,11 +182,12 @@ async function run() { api: 'openai-completions', provider: 'goodbuddy', model: 'qwen-plus', + supportsImageInput: false, harnessVersion: '0.1.0-rc.6', credentialRefs: ['GOODBUDDY_HARNESS_MODEL_API_KEY'], skillPackages: [], extensionPackages: [], - maxFrameBytes: 1024 * 1024 + maxFrameBytes: 8 * 1024 * 1024 } }) diff --git a/docs/deepseek-harness-runtime-design.md b/docs/deepseek-harness-runtime-design.md index 79d8311..4bf0e11 100644 --- a/docs/deepseek-harness-runtime-design.md +++ b/docs/deepseek-harness-runtime-design.md @@ -75,6 +75,7 @@ GoodBuddy 自己的 Runtime 和控制面不包装成标准 DSH 插件,也不 - 增加 `deepseek-harness` Runtime,并在设置、聊天和消息通道中可选择。 - 使用 GoodBuddy 管理的模型连接,不在 Renderer 或持久化 Harness 配置中写入 API Key。 +- 当所选模型连接明确声明支持图像输入时,允许向 DeepSeek Harness 发送有界的 JPEG/PNG;文本模型在启动 Host 或调用模型前拒绝图片。 - Ask 模式在 Runtime 工具分发边界强制只读,阻止 Shell、写入和编辑工具。 - 在 Web Search 能力启用时,通过 Main 代理向 Ask 与 Execute 提供有界的 `web_search` 和 `web_fetch`,Harness Utility 不持有服务凭据。 - Execute 模式使用 DSH 本地 Provider,以当前用户权限执行文件与命令工具;工作区是默认工作目录,不是 OS 权限边界。 @@ -97,7 +98,7 @@ GoodBuddy 自己的 Runtime 和控制面不包装成标准 DSH 插件,也不 - 不提供 Runtime OS 沙箱模式或相关持久设置。 - 不向 Utility 暴露 MCP 凭据或建立直连 MCP Client。只有用户明确分配给 Harness 的 MCP 工具可以通过 Main 代理调用。 - 不在首版向 Harness 暴露 GoodBuddy 浏览器控制、知识库或 Magic Notes。 -- 不在首版支持图像输入、会话恢复、Harness Subagent、后台 Job、Hook、浏览器控制或 Workflow;Web Search/Fetch 只通过 Main 代理提供,不加载 Harness 自有网页服务。 +- 不在首版支持会话恢复、Harness Subagent、后台 Job、Hook、浏览器控制或 Workflow;Web Search/Fetch 只通过 Main 代理提供,不加载 Harness 自有网页服务。上述长生命周期能力未来统一进入右侧 Runtime 监督栏,不进入 Composer 工具栏。 - 不发布独立 npm 包,也不创建上游 PR。 - 不为第三方插件增加权限矩阵、风险等级、逐工具审批、沙箱档位、回滚代际或兼容性背书。 @@ -158,6 +159,7 @@ Electron utilityProcess ├─ 固定 Cordis 组合 ├─ GoodBuddy Harness Control Plane(内部组件) ├─ DSH Agent 与 LLM seam + ├─ 按模型能力挂载的有界内存图片存储 ├─ 本地 Shell / Filesystem Provider ├─ 最小工具集与 Main 代理 MCP └─ Main 明确启用的第三方 Cordis 插件 @@ -190,6 +192,7 @@ Harness 子进程崩溃、输出异常、拒绝协议或加载错误时,Main - 在 Prompt 前应用 GoodBuddy 指定的 Ask/Execute 权限。 - 将 DSH Session 事件转换为有界的 GoodBuddy 事件。 - 将 LLM 用量转换为稳定的模型用量事件。 +- 根据 Main 传入的模型能力声明 ACP 图片能力,验证内联图片并转换为 DSH 的不可变 Attachment 引用。 - 在 dispose 时先取消 Agent,再等待子 Agent 和工具清理。 - 保证 stdout 只包含协议帧,诊断只写 stderr。 @@ -223,10 +226,11 @@ GoodBuddy 不扫描任意目录、不读取用户 profile 插件清单,也不 - stdout 不得出现日志、Banner、进度条或调试输出。 - stderr 只允许有界诊断,不得包含 Prompt、工具完整输出或凭据。 - 每一帧、每一字段和每个请求累计输出都必须在解析前或接收时限流。 +- 图片只允许作为 ACP 内联 base64 内容传入;拒绝远程 URI,Host 不替用户获取图片 URL。 ### 8.2 标准 ACP 方法 -首版保留 ACP 的初始化、`session/new`、`session/prompt` 和 `session/cancel` 语义。标准 ACP 客户端可以使用只读默认行为,但只有完成 GoodBuddy 能力握手的客户端才能启用 Execute。 +首版保留 ACP 的初始化、`session/new`、`session/prompt` 和 `session/cancel` 语义。`promptCapabilities.image` 必须与所选模型连接的 `supportsImageInput` 完全一致,不能仅根据 Provider 或模型名称猜测。标准 ACP 客户端可以使用只读默认行为,但只有完成 GoodBuddy 能力握手的客户端才能启用 Execute。 ### 8.3 GoodBuddy 扩展 @@ -244,7 +248,7 @@ GoodBuddy 不扫描任意目录、不读取用户 profile 插件清单,也不 | `goodbuddy/native/snapshot` | Main → Control Plane | 从无 Agent scope 的 Host Registry 读取有界的原生 Tool/Skill 元数据,排除 GoodBuddy 分配项与请求级代理 | | `goodbuddy/shutdown` | Main → Control Plane | 停止接收新请求并有序清理 | -扩展版本独立于 ACP 版本。握手响应至少包含: +Utility 启动控制协议使用版本 2,严格携带 `supportsImageInput` 与固定 8 MiB 帧上限;版本 1 或缺少该字段的启动消息失败关闭,不能让 Host 自行猜测模型能力。扩展版本独立于 ACP 版本。握手响应至少包含: ```ts type GoodBuddyHarnessCapabilities = { @@ -386,6 +390,7 @@ GoodBuddy conversationId -> Harness sessionId + process generation - Agent、Session、LLM 和 Tool Registry 基础服务。 - GoodBuddy Harness Control Plane。 - OpenAI 兼容 Chat Completions LLM 适配器。 +- 仅在所选模型声明图片能力时挂载的进程内 Attachment Store;它完整解码图片、校验格式/尺寸/摘要,以内容寻址引用保存,并随 Session 或 Host 释放。 - DSH 本地 Subprocess、Filesystem 和平台 Shell Provider。 - Token Meter 和必要的上下文压缩。 - 有界的读取、写入、编辑和 Shell 工具。 @@ -418,6 +423,7 @@ DeepSeek Harness 首版只使用符合下列边界的 GoodBuddy 模型连接: - 服务地址可以使用自定义主机、端口和部署路径,但不得包含用户名、密码、查询参数或片段。 - 模型名称不限制为 DeepSeek 品牌,由所选 OpenAI 兼容服务决定。 - 模型名称和服务地址由 Main 传入受控 Host。 +- 图片能力只读取所选 GoodBuddy 模型连接的 `supportsImageInput`;Main、Utility 启动配置、ACP 能力和 Pi-AI 模型输入模态必须使用同一个布尔值。 - API Key 继续保存在 GoodBuddy 加密设置中。 - 启动环境提供的部署连接只由 Main 自动解析,不在 Renderer 中显示为可选来源。 @@ -429,8 +435,12 @@ DeepSeek Harness 首版只使用符合下列边界的 GoodBuddy 模型连接: ### 12.3 输入限制 -- 首版只支持文本。 -- 图片输入应在发起网络调用前返回明确错误。 +- 文本始终可用;图片是否可用完全取决于所选模型连接是否显式声明 `supportsImageInput: true`。 +- 文本模型收到图片时必须在启动 Host 或发起模型网络调用前返回明确错误,不能静默丢弃图片。 +- 图片模型只接受内联 JPEG/PNG,不接受 URL、文件路径、ACP `uri` 或其他媒体类型。 +- Main 已通过 `nativeImage` 解码用户选择的图片并生成有界模型输入;Utility 仍须独立执行严格 base64、签名、容器结构、CRC(PNG)、完整解码、尺寸和摘要校验,不能把 Main 校验当作跨进程信任替代。 +- 每条消息最多 8 张图,单图编码后最多 1 MiB,图片合计最多 2 MiB,单图最多 1,600 万像素,累计解码像素最多 3,200 万(重复引用也计入预算)。进程内 Store 另设 32 MiB、256 个唯一对象的总上限。 +- Attachment Store 只服务当前非持久 Harness Session;引用按 Session 释放,Host 退出时清空,不写入磁盘或 GoodBuddy 第二份会话日志。 - GoodBuddy 历史、Prompt、系统指令分别保持不同信任层。 - 任何用户文本都不能进入 Cordis 配置表达式或模块名。 @@ -440,7 +450,10 @@ DeepSeek Harness 首版只使用符合下列边界的 GoodBuddy 模型连接: | 项目 | 默认上限 | | --- | --- | -| 单个 JSON-RPC 帧 | 1 MiB | +| 单个 JSON-RPC 帧 | 8 MiB | +| 单图 / 单条消息图片 | 1 MiB / 8 张且合计 2 MiB | +| 单图 / 单条消息解码像素 | 1,600 万 / 3,200 万 | +| Host 临时图片存储 | 32 MiB 且最多 256 个唯一对象 | | 单个文本或推理事件 | 64 KiB | | 单次请求累计协议输出 | 4 MiB | | 工具输入摘要 | 4,000 字符 | @@ -495,9 +508,25 @@ Host 始终由当前 GoodBuddy 版本提供,不存在自定义 Host 入口。 - 已安装插件使用共享 Switch 启停,并提供 JSON 配置、明确移除确认和启动失败信息。 - npm 目录离线时仍显示并允许管理已安装插件;目录错误就地显示并可重试。 - 安装、启停、配置和移除的短期结果通过应用通知显示,不重复保留页内成功提示。 +- DSH 不提供独立的“允许图片”开关。Runtime 连接选择只引用“模型连接”中维护的图片能力声明,避免同一模型出现两份冲突配置。 聊天顶栏只显示简短 Runtime 状态,不显示文件路径和版本。完整诊断只在设置页展示。 +### 14.3 Agent Runtime 交互表面归属 + +OpenCode、Continue 和 DeepSeek Harness 的后续能力按操作生命周期放置,不按上游产品分别堆叠入口: + +| 表面 | 负责内容 | 不负责内容 | +| --- | --- | --- | +| Composer 通用行 | 附件、语音、知识范围、专家、Ask/Execute、Runtime 和发送 | Session 监督、后台进度、历史任务管理 | +| Composer Runtime 专属行 | 仅对当前消息生效且需要高频选择的 Agent、预设、Prompt/Command 快捷操作 | Subagent 树、后台 Job、Workflow/Hook 生命周期 | +| 右侧助手工作栏的未来“Runtime”页签 | 当前会话的 Runtime 状态、Subagent 层级与取消、后台 Job 队列/进度/结果、Workflow/Hook 运行、长任务暂停/恢复/终止和会话监督 | 持久模型、程序路径、默认 Agent/预设配置 | +| 设置 > Agent Runtime | 持久 Runtime 配置、默认值、插件管理、能力清单和连接诊断 | 某次活动会话的实时控制 | + +右侧 Runtime 页签采用统一监督模型,再按当前 Runtime 能力显示 OpenCode、Continue 或 DSH 的具体区块。未支持的能力不渲染空卡片或一排禁用按钮;只有用户需要理解缺口时才显示简短说明。切换 Runtime 或会话时,侧栏必须明确更新归属,不能把上一 Runtime 的 Job/Subagent 状态留在当前会话中。 + +所有未来的 Subagent、Job、Workflow、Hook 和会话操作仍须经过 Main 的 Runtime 边界,保留取消、超时、权限、父子任务关系、用量和活动审计。高风险动作在侧栏就地确认,运行结果进入活动与成果记录,不以 Composer 按钮代替监督面板。DeepSeek Harness 首版仍不加载这些服务,本节只确定未来跨 Runtime 的产品位置和协议归属。 + ## 15. IPC 与共享契约 共享 schema 需要覆盖: @@ -506,6 +535,7 @@ Host 始终由当前 GoodBuddy 版本提供,不存在自定义 Host 入口。 - Runtime 选择中的 `deepseekHarness` 分支。 - 检测结果中的路径、版本、详情和主机执行模式。 - GoodBuddy 模型连接选择。 +- 从所选模型连接解析并传到 Host 的 `supportsImageInput`,以及 ACP 图片能力的一致性。 - DeepSeek Harness 模型用量归属。 - Skill 与 MCP 对 `deepseek-harness` 的显式分配。 - 插件市场总开关、目录、已安装状态、启停状态、JSON 配置和有界启动错误。 @@ -549,6 +579,7 @@ Renderer 只接收公开 npm 元数据和受管插件状态。任何凭据、完 - `node-pty`,用于受管理的工具子进程。 - `koffi`,用于本地 Filesystem 在 Windows 上保持文件 ACL 和原子替换。 +- `@napi-rs/canvas` 及当前平台二进制,用于在 Utility 内完整解码并复核 JPEG/PNG;原生模块必须从 ASAR 解包并通过目标架构校验。 构建 GoodBuddy 自身时不得广泛批准依赖安装脚本;只允许生产组合实际需要、来源已审查、版本已锁定的脚本。这与用户确认后由市场插件执行自身 lifecycle scripts 是两个不同阶段。六个平台的构建必须验证: @@ -606,6 +637,8 @@ Renderer 只接收公开 npm 元数据和受管插件状态。任何凭据、完 - 未知授权结果失败关闭。 - 超时、取消、迟到帧和进程意外退出。 - 协议帧、事件队列、工具摘要和 stderr 上限。 +- 文本模型在 Host 启动前拒绝图片;图片模型的能力声明、ACP 图片块、Pi-AI 模态和 Attachment Store 保持一致。 +- 图片 base64、格式签名、PNG CRC、完整解码、尺寸、单图/单消息/Store 上限、内容摘要、Session 释放和 Host 清空。 - release 和 dispose 的幂等性。 - 状态卡中的状态、路径、版本和当前用户执行权限。 - 插件 action 与目录 schema 接受严格的市场总开关并拒绝权限、回滚、任意路径和非精确版本等未支持字段。 @@ -690,7 +723,8 @@ npm run build - DeepSeek Harness 底层库当前是 RC,但 GoodBuddy 不自动跟随升级;每次升级都可能要求同步修改内部控制面。 - Harness 文件和命令工具没有 Runtime OS 隔离,会继承 GoodBuddy 客户端当前用户能够访问的主机资源。 - 首版不恢复 Harness 原生 Session,Runtime 重启后由 GoodBuddy 历史重建。 -- 首版不支持图片、知识库、浏览器控制和 Harness Subagent;Web Search/Fetch 仅使用 Main 代理,MCP 仅支持用户分配、Main 代理和 Execute 自动单次授权路径。 +- 图片输入仅在所选模型连接明确声明支持时可用;首版仍不支持知识库、浏览器控制和 Harness Subagent。Web Search/Fetch 仅使用 Main 代理,MCP 仅支持用户分配、Main 代理和 Execute 自动单次授权路径。 +- Harness Subagent、后台 Job、Workflow、Hook 和原生会话监督尚未实现;未来入口固定在右侧 Runtime 监督栏,不扩张 Composer 工具栏。 - 推理、工具和用量扩展属于 GoodBuddy 协议,不是标准 ACP 保证。 - 市场来自公共 npm 关键字搜索,不是精选目录;包的质量、兼容性和维护状态由发布者负责。 - 插件安装、初始化、后台生命周期和 Execute 工具使用当前用户权限,不受 Runtime OS 沙箱保护;Ask 只控制模型工具调用。 diff --git a/package-lock.json b/package-lock.json index 85c61e6..a85aa0a 100644 --- a/package-lock.json +++ b/package-lock.json @@ -35,6 +35,7 @@ "@deepseek-ai/dsh-tools": "0.1.0-rc.6", "@deepseek-ai/dsh-user-approval": "0.1.0-rc.6", "@modelcontextprotocol/sdk": "^1.30.0", + "@napi-rs/canvas": "1.0.3", "@opencode-ai/sdk": "^1.18.9", "@wecom/aibot-node-sdk": "^1.0.6", "cross-spawn": "^7.0.6", diff --git a/package.json b/package.json index d076249..3520ae7 100644 --- a/package.json +++ b/package.json @@ -60,7 +60,9 @@ "node_modules/node-pty/prebuilds/**/*", "node_modules/node-pty/build/Release/**/*", "node_modules/koffi/**/*", - "node_modules/@koromix/koffi-*/**/*" + "node_modules/@koromix/koffi-*/**/*", + "node_modules/@napi-rs/canvas{,/**/*}", + "node_modules/@napi-rs/canvas-*/**/*" ], "npmRebuild": false, "compression": "maximum", @@ -132,6 +134,10 @@ "from": "node_modules/koffi/LICENSE.txt", "to": "licenses/koffi-MIT.txt" }, + { + "from": "node_modules/@napi-rs/canvas/LICENSE", + "to": "licenses/napi-rs-canvas-MIT.txt" + }, { "from": "node_modules/@continuedev/cli", "to": "runtimes/continue", @@ -246,6 +252,7 @@ "@deepseek-ai/dsh-tools": "0.1.0-rc.6", "@deepseek-ai/dsh-user-approval": "0.1.0-rc.6", "@modelcontextprotocol/sdk": "^1.30.0", + "@napi-rs/canvas": "1.0.3", "@opencode-ai/sdk": "^1.18.9", "@wecom/aibot-node-sdk": "^1.0.6", "cross-spawn": "^7.0.6", diff --git a/src/main/agent/create-runtime.test.ts b/src/main/agent/create-runtime.test.ts index 6304462..d4f2dd9 100644 --- a/src/main/agent/create-runtime.test.ts +++ b/src/main/agent/create-runtime.test.ts @@ -103,7 +103,7 @@ describe('createAgentRuntime model compatibility', () => { ) }) - it('creates DeepSeek Harness with a compatible HTTPS gateway profile', async () => { + it('forwards a compatible gateway profile to DeepSeek Harness', async () => { const profile = { id: '00000000-0000-4000-8000-000000000006', name: 'OpenAI-compatible gateway', @@ -111,9 +111,13 @@ describe('createAgentRuntime model compatibility', () => { modelName: 'qwen-plus', protocol: 'openai-chat-completions' as const, authentication: 'api-key' as const, + supportsImageInput: true, imageGenerationQuality: 'auto' as const, apiKey: 'gateway-key' } + const deepseekHarnessLauncher = vi + .fn() + .mockRejectedValue(new Error('stop after launch options')) const runtime = createAgentRuntime( process.cwd(), settings({ @@ -122,10 +126,16 @@ describe('createAgentRuntime model compatibility', () => { defaultModelProfileId: profile.id, deepseekHarnessModelProfile: profile }), - { deepseekHarnessLauncher: vi.fn() } + { deepseekHarnessLauncher } ) expect(runtime.runtimeId).toBe('deepseek-harness') + await expect(runtime.getStatus()).resolves.toMatchObject({ + available: false + }) + expect(deepseekHarnessLauncher).toHaveBeenCalledWith( + expect.objectContaining({ supportsImageInput: true }) + ) await runtime.dispose() }) diff --git a/src/main/agent/create-runtime.ts b/src/main/agent/create-runtime.ts index 9a6eee7..e34b041 100644 --- a/src/main/agent/create-runtime.ts +++ b/src/main/agent/create-runtime.ts @@ -171,6 +171,7 @@ export function createAgentRuntime( defaultWorkspace: workspace, baseUrl: profile.baseUrl, model: profile.modelName, + supportsImageInput: profile.supportsImageInput === true, launch: capabilities.deepseekHarnessLauncher, credentialRefs: { GOODBUDDY_HARNESS_MODEL_API_KEY: profile.apiKey diff --git a/src/main/agent/deepseek-harness-acp-e2e.test.ts b/src/main/agent/deepseek-harness-acp-e2e.test.ts index 3307943..28ef2b0 100644 --- a/src/main/agent/deepseek-harness-acp-e2e.test.ts +++ b/src/main/agent/deepseek-harness-acp-e2e.test.ts @@ -8,6 +8,7 @@ import { import { tmpdir } from 'node:os' import { join, resolve } from 'node:path' import { describe, expect, it, vi } from 'vitest' +import { createCanvas } from '@napi-rs/canvas' import { CallId, type GenerateOptions, @@ -31,8 +32,8 @@ import { } from './deepseek-harness-runtime' import { GOODBUDDY_HARNESS_MAX_STEP_TOKENS } from './goodbuddy-harness-control-plane' import { DshNpmExtensionInstaller } from './dsh-extension-marketplace' +import { DEEPSEEK_HARNESS_MAX_FRAME_BYTES } from './deepseek-harness-control-protocol' -const MAX_FRAME_BYTES = 1024 * 1024 const CREDENTIAL_REF = 'GOODBUDDY_HARNESS_MODEL_API_KEY' const SKILL_CALL_ID = 'e2e-skill-call' const MCP_CALL_ID = 'e2e-mcp-call' @@ -327,6 +328,7 @@ function createInProcessLaunch( api: 'openai-completions', provider: 'goodbuddy', model: options.model, + supportsImageInput: options.supportsImageInput, harnessVersion: '0.1.0-rc.6', credentialRefs: options.credentialRefs, skillPackages: options.skillPackages, @@ -334,7 +336,7 @@ function createInProcessLaunch( stream: createBoundedNdJsonStream( hostToClient.writable, clientToHost.readable, - MAX_FRAME_BYTES + DEEPSEEK_HARNESS_MAX_FRAME_BYTES ) }) hosts.push(host) @@ -378,6 +380,88 @@ function createInProcessLaunch( } describe('DeepSeek Harness real ACP control-plane E2E', () => { + it('delivers bounded inline images to an image-capable Harness model', async () => { + const root = await realpath( + await mkdtemp(join(tmpdir(), 'goodbuddy-harness-acp-image-')) + ) + const workspace = join(root, 'workspace') + const dshHome = join(root, 'dsh-home') + await Promise.all([mkdir(workspace), mkdir(dshHome)]) + let observedRequest: GenerateOptions | undefined + const inProcess = createInProcessLaunch(dshHome, { + stream(options) { + observedRequest = options + return textResponse('Image received.') + } + }) + const runtime = new DeepSeekHarnessRuntime({ + defaultWorkspace: workspace, + baseUrl: 'https://api.deepseek.com', + model: 'vision-test', + supportsImageInput: true, + launch: (options) => inProcess.launch(options), + credentialRefs: { + [CREDENTIAL_REF]: 'unused-in-memory-model-credential' + }, + initializationTimeoutMs: 20_000, + promptTimeoutMs: 20_000, + shutdownTimeoutMs: 5_000 + }) + const png = createCanvas(1, 1).toBuffer('image/png') + + try { + const events = await collect( + runtime.run( + { + requestId: 'request-acp-image', + conversationId: 'acp-image', + prompt: 'Describe this image.', + workMode: 'ask', + images: [ + { + name: 'reference.png', + mediaType: 'image/png', + data: png.toString('base64') + } + ] + }, + new AbortController().signal + ) + ) + const image = observedRequest?.messages + .flatMap((message) => message.content) + .find( + ( + block + ): block is Extract< + GenerateOptions['messages'][number]['content'][number], + { type: 'image' } + > => block.type === 'image' + ) + + expect(events).toContainEqual( + expect.objectContaining({ type: 'done' }) + ) + expect(image?.attachment).toMatchObject({ + mediaType: 'image/png', + bytes: png.byteLength, + width: 1, + height: 1 + }) + const stored = + await inProcess.hosts[0]!.context.attachments.readImage( + image!.attachment + ) + expect(Buffer.from(stored.data).equals(png)).toBe(true) + } finally { + await runtime.dispose() + await Promise.allSettled( + inProcess.hosts.map((host) => host.dispose()) + ) + await rm(root, { recursive: true, force: true }) + } + }) + it( 'coalesces micro reasoning deltas without losing content and caps each model step', async () => { diff --git a/src/main/agent/deepseek-harness-control-protocol.ts b/src/main/agent/deepseek-harness-control-protocol.ts index ac58e4e..d71c37c 100644 --- a/src/main/agent/deepseek-harness-control-protocol.ts +++ b/src/main/agent/deepseek-harness-control-protocol.ts @@ -8,10 +8,12 @@ import { export const DEEPSEEK_HARNESS_CONTROL_PROTOCOL = 'goodbuddy.deepseek-harness.control' -export const DEEPSEEK_HARNESS_CONTROL_VERSION = 1 +export const DEEPSEEK_HARNESS_CONTROL_VERSION = 2 export const DEEPSEEK_HARNESS_HOST_VERSION = '0.1.0-rc.6' export const DEEPSEEK_HARNESS_CREDENTIAL_REF = 'GOODBUDDY_HARNESS_MODEL_API_KEY' +export const DEEPSEEK_HARNESS_MAX_FRAME_BYTES = + 8 * 1024 * 1024 const skillPackageSchema = z .object({ @@ -43,13 +45,14 @@ export const controlledHarnessHostConfigSchema = z api: z.literal('openai-completions'), provider: z.literal('goodbuddy'), model: z.string().min(1).max(128), + supportsImageInput: z.boolean(), harnessVersion: z.literal(DEEPSEEK_HARNESS_HOST_VERSION), credentialRefs: z .tuple([z.literal(DEEPSEEK_HARNESS_CREDENTIAL_REF)]) .readonly(), skillPackages: z.array(skillPackageSchema).max(64), extensionPackages: z.array(extensionPackageSchema).max(64), - maxFrameBytes: z.literal(1024 * 1024) + maxFrameBytes: z.literal(DEEPSEEK_HARNESS_MAX_FRAME_BYTES) }) .strict() diff --git a/src/main/agent/deepseek-harness-runtime.test.ts b/src/main/agent/deepseek-harness-runtime.test.ts index 2cfb5b8..1c50331 100644 --- a/src/main/agent/deepseek-harness-runtime.test.ts +++ b/src/main/agent/deepseek-harness-runtime.test.ts @@ -45,6 +45,8 @@ function setup( promptTimeoutMs?: number maxEventCharacters?: number maxRequestOutputCharacters?: number + supportsImageInput?: boolean + advertisedImageInput?: boolean } = {} ) { const exit = deferred<{ @@ -95,7 +97,13 @@ function setup( if (method === 'initialize') { return { protocolVersion: 1, - agentCapabilities: {} + agentCapabilities: { + promptCapabilities: { + image: + options.advertisedImageInput ?? + (options.supportsImageInput === true) + } + } } } if (method === 'session/new') { @@ -212,6 +220,7 @@ function setup( defaultWorkspace: 'C:\\workspace', baseUrl: 'https://api.deepseek.com', model: 'deepseek-test', + supportsImageInput: options.supportsImageInput, launch, loadAcpSdk: async () => sdk, initializationTimeoutMs: 100, @@ -417,6 +426,86 @@ describe('DeepSeekHarnessRuntime', () => { ).toBeInstanceOf(RequestError) }) + it('rejects images before launch when the selected model is text-only', async () => { + const harness = setup() + + await expect( + collect( + harness.runtime.run( + { + ...request('text-only-image'), + images: [ + { + name: 'reference.png', + mediaType: 'image/png', + data: 'aW1hZ2U=' + } + ] + }, + new AbortController().signal + ) + ) + ).rejects.toThrow('未启用图像输入') + expect(harness.launch).not.toHaveBeenCalled() + }) + + it('forwards inline images when the selected model supports them', async () => { + const harness = setup({ supportsImageInput: true }) + const running = collect( + harness.runtime.run( + { + ...request('vision'), + images: [ + { + name: 'reference.png', + mediaType: 'image/png', + data: 'aW1hZ2U=' + } + ] + }, + new AbortController().signal + ) + ) + await vi.waitFor(() => + expect(harness.promptGates).toHaveLength(1) + ) + + expect(harness.launch).toHaveBeenCalledWith( + expect.objectContaining({ supportsImageInput: true }) + ) + expect( + harness.requests.find( + (entry) => entry.method === 'session/prompt' + )?.params + ).toMatchObject({ + prompt: [ + { type: 'text', text: 'hello' }, + { + type: 'image', + mimeType: 'image/png', + data: 'aW1hZ2U=' + } + ] + }) + harness.promptGates[0]!.resolve({ stopReason: 'end_turn' }) + await expect(running).resolves.toContainEqual( + expect.objectContaining({ type: 'done' }) + ) + }) + + it('fails closed when Host image capability disagrees with the model', async () => { + const harness = setup({ + supportsImageInput: true, + advertisedImageInput: false + }) + + await expect(harness.runtime.getStatus()).resolves.toMatchObject({ + available: false, + detail: expect.stringContaining('图片能力') + }) + expect(harness.child.terminate).toHaveBeenCalled() + }) + it('uses ACP stdio, maps conversations to sessions, and streams text', async () => { const harness = setup() const first = collect( @@ -456,6 +545,7 @@ describe('DeepSeekHarnessRuntime', () => { signal: expect.any(AbortSignal), baseUrl: 'https://api.deepseek.com', model: 'deepseek-test', + supportsImageInput: false, credentialRefs: [], skillPackages: [], extensionPackages: [] diff --git a/src/main/agent/deepseek-harness-runtime.ts b/src/main/agent/deepseek-harness-runtime.ts index 07244f0..afad1e9 100644 --- a/src/main/agent/deepseek-harness-runtime.ts +++ b/src/main/agent/deepseek-harness-runtime.ts @@ -171,6 +171,7 @@ export type DeepSeekHarnessLaunchOptions = { signal: AbortSignal baseUrl: string model: string + supportsImageInput: boolean credentialRefs: readonly string[] skillPackages: readonly RuntimeSkillPackage[] extensionPackages: readonly ControlledHarnessExtensionPackage[] @@ -180,6 +181,7 @@ export type DeepSeekHarnessRuntimeOptions = { defaultWorkspace: string baseUrl: string model: string + supportsImageInput?: boolean launch: ( options: DeepSeekHarnessLaunchOptions ) => Promise @@ -812,6 +814,8 @@ export class DeepSeekHarnessRuntime implements AgentRuntime { signal: launchController.signal, baseUrl: this.options.baseUrl, model: this.options.model, + supportsImageInput: + this.options.supportsImageInput === true, credentialRefs: Object.keys( this.options.credentialRefs ?? {} ), @@ -1022,7 +1026,7 @@ export class DeepSeekHarnessRuntime implements AgentRuntime { () => this.fail(new Error('DeepSeek Harness ACP 连接异常关闭')) ) - await withTimeout( + const initialization = await withTimeout( stateWithoutCapabilities.agent.initialize({ protocolVersion: sdk.PROTOCOL_VERSION, clientCapabilities: {}, @@ -1034,6 +1038,26 @@ export class DeepSeekHarnessRuntime implements AgentRuntime { this.initializationTimeoutMs, 'ACP 初始化' ) + const advertisedImageInput = + Boolean( + initialization && + typeof initialization === 'object' && + ( + initialization as { + agentCapabilities?: { + promptCapabilities?: { image?: unknown } + } + } + ).agentCapabilities?.promptCapabilities?.image === true + ) + if ( + advertisedImageInput !== + (this.options.supportsImageInput === true) + ) { + throw new Error( + 'DeepSeek Harness Host 图片能力与所选模型连接不一致' + ) + } const capabilities = this.parseCapabilities( await withTimeout( stateWithoutCapabilities.agent.extMethod( @@ -1408,8 +1432,11 @@ export class DeepSeekHarnessRuntime implements AgentRuntime { authorize?: RuntimeAuthorizer ): AsyncGenerator { signal.throwIfAborted() - if (request.images?.length) { - throw new Error('DeepSeek Harness Runtime 暂不支持图像输入') + if ( + request.images?.length && + this.options.supportsImageInput !== true + ) { + throw new Error('当前 DeepSeek Harness 模型连接未启用图像输入') } const release = await this.acquireConversation( request.conversationId, @@ -1477,7 +1504,12 @@ export class DeepSeekHarnessRuntime implements AgentRuntime { { type: 'text', text: flattenPrompt(request) - } + }, + ...(request.images ?? []).map((image) => ({ + type: 'image' as const, + data: image.data, + mimeType: image.mediaType + })) ] }), this.promptTimeoutMs, diff --git a/src/main/agent/deepseek-harness-utility-launcher.test.ts b/src/main/agent/deepseek-harness-utility-launcher.test.ts index ec13d62..12e6836 100644 --- a/src/main/agent/deepseek-harness-utility-launcher.test.ts +++ b/src/main/agent/deepseek-harness-utility-launcher.test.ts @@ -52,6 +52,7 @@ async function fixture() { signal: new AbortController().signal, baseUrl: 'https://gateway.example/openai/v1', model: 'qwen-plus', + supportsImageInput: false, credentialRefs: [DEEPSEEK_HARNESS_CREDENTIAL_REF], skillPackages: [], extensionPackages: [] @@ -103,6 +104,7 @@ describe('DeepSeek Harness utility launcher', () => { config: { baseUrl: 'https://gateway.example/openai/v1', model: 'qwen-plus', + supportsImageInput: false, credentialRefs: [DEEPSEEK_HARNESS_CREDENTIAL_REF] } }) diff --git a/src/main/agent/deepseek-harness-utility-launcher.ts b/src/main/agent/deepseek-harness-utility-launcher.ts index 353851f..f4df056 100644 --- a/src/main/agent/deepseek-harness-utility-launcher.ts +++ b/src/main/agent/deepseek-harness-utility-launcher.ts @@ -14,6 +14,7 @@ import { DEEPSEEK_HARNESS_CONTROL_VERSION, DEEPSEEK_HARNESS_CREDENTIAL_REF, DEEPSEEK_HARNESS_HOST_VERSION, + DEEPSEEK_HARNESS_MAX_FRAME_BYTES, parseHarnessControlMessage, type DeepSeekHarnessControlMessage as HarnessControlMessage } from './deepseek-harness-control-protocol' @@ -24,6 +25,7 @@ export { DEEPSEEK_HARNESS_CONTROL_VERSION, DEEPSEEK_HARNESS_CREDENTIAL_REF, DEEPSEEK_HARNESS_HOST_VERSION, + DEEPSEEK_HARNESS_MAX_FRAME_BYTES, parseHarnessControlMessage } from './deepseek-harness-control-protocol' export type { @@ -274,11 +276,12 @@ export function createDeepSeekHarnessUtilityLauncher( api: 'openai-completions', provider: 'goodbuddy', model: options.model, + supportsImageInput: options.supportsImageInput, harnessVersion: DEEPSEEK_HARNESS_HOST_VERSION, credentialRefs: [DEEPSEEK_HARNESS_CREDENTIAL_REF], skillPackages: canonicalSkillPackages, extensionPackages: canonicalExtensionPackages, - maxFrameBytes: 1024 * 1024 + maxFrameBytes: DEEPSEEK_HARNESS_MAX_FRAME_BYTES }) utility.postMessage({ protocol: DEEPSEEK_HARNESS_CONTROL_PROTOCOL, diff --git a/src/main/agent/deepseek-harness-utility-transport.test.ts b/src/main/agent/deepseek-harness-utility-transport.test.ts index 28a2576..fb22403 100644 --- a/src/main/agent/deepseek-harness-utility-transport.test.ts +++ b/src/main/agent/deepseek-harness-utility-transport.test.ts @@ -1,4 +1,6 @@ import { describe, expect, it, vi } from 'vitest' +import { readFileSync } from 'node:fs' +import { resolve } from 'node:path' import { DEEPSEEK_HARNESS_BYTE_PROTOCOL, DEEPSEEK_HARNESS_BYTE_PROTOCOL_VERSION, @@ -7,6 +9,10 @@ import { createDeepSeekHarnessUtilityChild, type DeepSeekHarnessParentPortLike } from './deepseek-harness-utility-transport' +import { + DEEPSEEK_HARNESS_CONTROL_PROTOCOL, + DEEPSEEK_HARNESS_CONTROL_VERSION +} from './deepseek-harness-control-protocol' type Listener = (value: unknown) => void @@ -120,13 +126,27 @@ function setup() { const tick = () => new Promise((resolve) => queueMicrotask(resolve)) describe('DeepSeek Harness utility byte transport', () => { + it('keeps the Electron smoke protocol versions aligned', () => { + const smokeSource = readFileSync( + resolve('build/deepseek-harness-utility-smoke.cjs'), + 'utf8' + ) + + expect(smokeSource).toContain( + `const controlVersion = ${DEEPSEEK_HARNESS_CONTROL_VERSION}` + ) + expect(smokeSource).toContain( + `const byteProtocolVersion = ${DEEPSEEK_HARNESS_BYTE_PROTOCOL_VERSION}` + ) + }) + it('ignores trusted control-plane messages that share the UtilityProcess port', async () => { const { child, hostPort, utility } = setup() await tick() utility.kill.mockClear() utility.emitMessage({ - protocol: 'goodbuddy.deepseek-harness.control', - version: 1, + protocol: DEEPSEEK_HARNESS_CONTROL_PROTOCOL, + version: DEEPSEEK_HARNESS_CONTROL_VERSION, type: 'ready', failedExtensionIds: [] }) @@ -153,8 +173,8 @@ describe('DeepSeek Harness utility byte transport', () => { const { child, utility } = setup() const reader = child.stdout.getReader() utility.emitMessage({ - protocol: 'goodbuddy.deepseek-harness.control', - version: 1, + protocol: DEEPSEEK_HARNESS_CONTROL_PROTOCOL, + version: DEEPSEEK_HARNESS_CONTROL_VERSION, type: 'ready', failedExtensionIds: [], unexpected: true diff --git a/src/main/agent/goodbuddy-harness-attachment-store.test.ts b/src/main/agent/goodbuddy-harness-attachment-store.test.ts new file mode 100644 index 0000000..ea5a509 --- /dev/null +++ b/src/main/agent/goodbuddy-harness-attachment-store.test.ts @@ -0,0 +1,133 @@ +// @vitest-environment node +import { Context } from '@deepseek-ai/cordis' +import { createCanvas } from '@napi-rs/canvas' +import { describe, expect, it } from 'vitest' +import { GoodBuddyHarnessAttachmentStore } from './goodbuddy-harness-attachment-store' + +const canvas = createCanvas(1, 1) +const transparentPng = canvas.toBuffer('image/png') +const jpeg = canvas.toBuffer('image/jpeg') +const secondPng = createCanvas(2, 1).toBuffer('image/png') + +describe('GoodBuddy Harness attachment store', () => { + it('decodes, stores, verifies, and releases inline images', async () => { + const store = new GoodBuddyHarnessAttachmentStore(new Context()) + const input = { + data: transparentPng, + mediaType: 'image/png' as const, + name: '..\\screenshots\\reference.png' + } + + const first = await store.saveImage(input) + const second = await store.saveImage(input) + + expect(first).toEqual(second) + expect(first).toMatchObject({ + mediaType: 'image/png', + bytes: transparentPng.byteLength, + width: 1, + height: 1, + name: 'reference.png' + }) + const stored = await store.readImage(first) + expect(stored.ref).toBe(first) + expect(Buffer.from(stored.data).equals(transparentPng)).toBe(true) + + store.releaseImage(first) + await expect(store.readImage(first)).resolves.toBeDefined() + store.releaseImage(second) + await expect(store.readImage(first)).rejects.toMatchObject({ + code: 'NOT_FOUND' + }) + + const jpegRef = await store.saveImage({ + data: jpeg, + mediaType: 'image/jpeg' + }) + expect(jpegRef).toMatchObject({ + mediaType: 'image/jpeg', + bytes: jpeg.byteLength, + width: 1, + height: 1 + }) + }) + + it('rejects mismatched, malformed, and over-capacity images', async () => { + const store = new GoodBuddyHarnessAttachmentStore(new Context(), { + maxStoredImages: 1 + }) + + await expect( + store.saveImage({ + data: transparentPng, + mediaType: 'image/jpeg' + }) + ).rejects.toMatchObject({ code: 'INVALID_IMAGE' }) + await expect( + store.saveImage({ + data: Buffer.from([ + 0x89, 0x50, 0x4e, 0x47, + 0x0d, 0x0a, 0x1a, 0x0a + ]), + mediaType: 'image/png' + }) + ).rejects.toMatchObject({ code: 'INVALID_IMAGE' }) + const corruptPng = Buffer.from(transparentPng) + corruptPng[corruptPng.length - 8] = + (corruptPng[corruptPng.length - 8] ?? 0) ^ 1 + await expect( + store.saveImage({ + data: corruptPng, + mediaType: 'image/png' + }) + ).rejects.toMatchObject({ code: 'INVALID_IMAGE' }) + + await store.saveImage({ + data: transparentPng, + mediaType: 'image/png' + }) + await expect( + store.saveImage({ + data: secondPng, + mediaType: 'image/png' + }) + ).rejects.toMatchObject({ code: 'STORAGE_LIMIT' }) + }) + + it('does not retain a partial batch when capacity is exceeded', async () => { + const store = new GoodBuddyHarnessAttachmentStore(new Context(), { + maxStoredImages: 1 + }) + + await expect( + store.saveImages([ + { data: transparentPng, mediaType: 'image/png' }, + { data: jpeg, mediaType: 'image/jpeg' } + ]) + ).rejects.toMatchObject({ code: 'STORAGE_LIMIT' }) + await expect( + store.saveImage({ + data: jpeg, + mediaType: 'image/jpeg' + }) + ).resolves.toMatchObject({ mediaType: 'image/jpeg' }) + }) + + it('bounds aggregate decoded pixels before retaining a batch', async () => { + const store = new GoodBuddyHarnessAttachmentStore(new Context(), { + maxBatchImagePixels: 1 + }) + const input = { + data: transparentPng, + mediaType: 'image/png' as const + } + + await expect( + store.saveImages([input, input]) + ).rejects.toMatchObject({ code: 'INVALID_IMAGE' }) + await expect(store.saveImage(input)).resolves.toMatchObject({ + width: 1, + height: 1 + }) + }) +}) diff --git a/src/main/agent/goodbuddy-harness-attachment-store.ts b/src/main/agent/goodbuddy-harness-attachment-store.ts new file mode 100644 index 0000000..16bc34a --- /dev/null +++ b/src/main/agent/goodbuddy-harness-attachment-store.ts @@ -0,0 +1,449 @@ +import { createHash } from 'node:crypto' +import { basename } from 'node:path' +import { crc32 } from 'node:zlib' +import type { Context } from '@deepseek-ai/cordis' +import { + AttachmentError, + AttachmentId, + AttachmentStore, + type ImageAttachmentLimits, + type ImageAttachmentRef, + type ImageMediaType, + type SaveImageAttachment, + type StoredImageAttachment +} from '@deepseek-ai/dsh-attachment' + +const DEFAULT_MAX_STORE_BYTES = 32 * 1024 * 1024 +const DEFAULT_MAX_STORED_IMAGES = 256 +const DEFAULT_MAX_BATCH_IMAGE_PIXELS = 32_000_000 + +export const GOODBUDDY_HARNESS_IMAGE_LIMITS: ImageAttachmentLimits = + Object.freeze({ + maxImageBytes: 1024 * 1024, + maxImagesPerMessage: 8, + maxMessageImageBytes: 2 * 1024 * 1024, + maxImagePixels: 16_000_000, + mediaTypes: Object.freeze([ + 'image/png', + 'image/jpeg' + ] satisfies ImageMediaType[]) + }) + +type StoredImage = { + ref: ImageAttachmentRef + data: Buffer + references: number +} + +type InspectedImage = { + data: Buffer + width: number + height: number +} + +export type GoodBuddyHarnessAttachmentStoreConfig = { + maxStoreBytes?: number + maxStoredImages?: number + maxBatchImagePixels?: number +} + +function invalidImage(message: string, cause?: unknown): AttachmentError { + return new AttachmentError(message, 'INVALID_IMAGE', { + ...(cause === undefined ? {} : { cause }) + }) +} + +function safeImageName(name: string | undefined): string | undefined { + if (!name) { + return undefined + } + const safe = basename(name.replaceAll('\\', '/')) + .replace(/\p{Cc}/gu, '_') + .trim() + .slice(0, 200) + return safe || undefined +} + +function matchesSignature( + data: Buffer, + mediaType: ImageMediaType +): boolean { + if (mediaType === 'image/png') { + return ( + data.length >= 8 && + data.subarray(0, 8).equals( + Buffer.from([ + 0x89, 0x50, 0x4e, 0x47, + 0x0d, 0x0a, 0x1a, 0x0a + ]) + ) + ) + } + return ( + data.length >= 4 && + data[0] === 0xff && + data[1] === 0xd8 && + data.at(-2) === 0xff && + data.at(-1) === 0xd9 + ) +} + +function pngDimensions(data: Buffer): { + width: number + height: number +} | undefined { + let offset = 8 + let chunks = 0 + let width: number | undefined + let height: number | undefined + let sawImageData = false + while (offset + 12 <= data.length && chunks < 256) { + chunks += 1 + const length = data.readUInt32BE(offset) + const typeStart = offset + 4 + const dataStart = typeStart + 4 + const dataEnd = dataStart + length + const chunkEnd = dataEnd + 4 + if (dataEnd < dataStart || chunkEnd > data.length) { + return undefined + } + const typeBytes = data.subarray(typeStart, dataStart) + const type = typeBytes.toString('ascii') + if (!/^[A-Za-z]{4}$/u.test(type)) { + return undefined + } + if ( + crc32(data.subarray(typeStart, dataEnd)) !== + data.readUInt32BE(dataEnd) + ) { + return undefined + } + if (chunks === 1) { + if (type !== 'IHDR' || length !== 13) { + return undefined + } + width = data.readUInt32BE(dataStart) + height = data.readUInt32BE(dataStart + 4) + } else if (type === 'IHDR') { + return undefined + } + if (type === 'IDAT') { + sawImageData = true + } + if (type === 'IEND') { + return ( + length === 0 && + chunkEnd === data.length && + sawImageData && + width !== undefined && + height !== undefined + ) + ? { width, height } + : undefined + } + offset = chunkEnd + } + return undefined +} + +function jpegDimensions(data: Buffer): { + width: number + height: number +} | undefined { + let offset = 2 + while (offset + 4 <= data.length - 2) { + if (data[offset] !== 0xff) { + return undefined + } + while (data[offset] === 0xff) { + offset += 1 + } + const marker = data[offset] + offset += 1 + if (marker === undefined || marker === 0x00 || marker === 0xd9) { + return undefined + } + if (marker === 0xda) { + return undefined + } + if (marker === 0x01 || (marker >= 0xd0 && marker <= 0xd7)) { + continue + } + if (offset + 2 > data.length - 2) { + return undefined + } + const length = data.readUInt16BE(offset) + if (length < 2 || offset + length > data.length - 2) { + return undefined + } + const isStartOfFrame = + marker >= 0xc0 && + marker <= 0xcf && + marker !== 0xc4 && + marker !== 0xc8 && + marker !== 0xcc + if (isStartOfFrame) { + if (length < 7) { + return undefined + } + return { + height: data.readUInt16BE(offset + 3), + width: data.readUInt16BE(offset + 5) + } + } + offset += length + } + return undefined +} + +async function inspectImage( + input: SaveImageAttachment, + limits: ImageAttachmentLimits +): Promise { + if (!limits.mediaTypes.includes(input.mediaType)) { + throw invalidImage('Image media type is not supported') + } + if ( + input.data.byteLength === 0 || + input.data.byteLength > limits.maxImageBytes + ) { + throw invalidImage('Image exceeds the per-image byte limit') + } + const data = Buffer.from(input.data) + if (!matchesSignature(data, input.mediaType)) { + throw invalidImage('Image media type does not match its bytes') + } + const encodedDimensions = + input.mediaType === 'image/png' + ? pngDimensions(data) + : jpegDimensions(data) + if (!encodedDimensions) { + throw invalidImage('Image container is malformed') + } + if ( + encodedDimensions.width < 1 || + encodedDimensions.height < 1 || + encodedDimensions.width * encodedDimensions.height > + limits.maxImagePixels + ) { + throw invalidImage('Image dimensions exceed the pixel limit') + } + let width: number + let height: number + let loadImage: typeof import('@napi-rs/canvas')['loadImage'] + try { + const canvas = await import('@napi-rs/canvas') + loadImage = canvas.loadImage + } catch (error) { + throw new AttachmentError( + 'Harness image decoder is unavailable', + 'DECODER_UNAVAILABLE', + { cause: error } + ) + } + try { + const image = await loadImage(data) + width = image.naturalWidth || image.width + height = image.naturalHeight || image.height + } catch (error) { + throw invalidImage('Image bytes could not be decoded', error) + } + if ( + !Number.isSafeInteger(width) || + !Number.isSafeInteger(height) || + width < 1 || + height < 1 || + width * height > limits.maxImagePixels || + width !== encodedDimensions.width || + height !== encodedDimensions.height + ) { + throw invalidImage('Image dimensions exceed the pixel limit') + } + return { data, width, height } +} + +/** + * Process-local attachment storage for the non-persistent Harness sessions. + * Images are fully decoded before an immutable content-addressed reference is + * published. The store is bounded independently of per-message admission. + */ +export class GoodBuddyHarnessAttachmentStore extends AttachmentStore { + readonly imageLimits = GOODBUDDY_HARNESS_IMAGE_LIMITS + private readonly images = new Map() + private readonly maxBatchImagePixels: number + private readonly maxStoreBytes: number + private readonly maxStoredImages: number + private storedBytes = 0 + + constructor( + ctx: Context, + config: GoodBuddyHarnessAttachmentStoreConfig = {} + ) { + super(ctx) + this.maxStoreBytes = + config.maxStoreBytes ?? DEFAULT_MAX_STORE_BYTES + this.maxStoredImages = + config.maxStoredImages ?? DEFAULT_MAX_STORED_IMAGES + this.maxBatchImagePixels = + config.maxBatchImagePixels ?? + DEFAULT_MAX_BATCH_IMAGE_PIXELS + if ( + !Number.isSafeInteger(this.maxStoreBytes) || + this.maxStoreBytes < this.imageLimits.maxImageBytes || + !Number.isSafeInteger(this.maxStoredImages) || + this.maxStoredImages < 1 || + !Number.isSafeInteger(this.maxBatchImagePixels) || + this.maxBatchImagePixels < 1 + ) { + throw new TypeError( + 'GoodBuddy Harness attachment-store limits are invalid' + ) + } + } + + async validateImage(input: SaveImageAttachment): Promise { + await inspectImage(input, this.imageLimits) + } + + async saveImage( + input: SaveImageAttachment + ): Promise { + return (await this.saveImages([input]))[0]! + } + + async saveImages( + inputs: readonly SaveImageAttachment[] + ): Promise { + const inspectedByContent = new Map() + const candidates: Array<{ + input: SaveImageAttachment + inspected: InspectedImage + attachmentId: ImageAttachmentRef['attachmentId'] + }> = [] + let batchPixels = 0 + for (const input of inputs) { + if ( + !this.imageLimits.mediaTypes.includes(input.mediaType) || + input.data.byteLength === 0 || + input.data.byteLength > this.imageLimits.maxImageBytes + ) { + throw invalidImage('Image exceeds the attachment limits') + } + const digest = createHash('sha256') + .update(input.data) + .digest('hex') + const contentKey = `${input.mediaType}:${digest}` + let inspected = inspectedByContent.get(contentKey) + if (!inspected) { + inspected = await inspectImage(input, this.imageLimits) + inspectedByContent.set(contentKey, inspected) + } + batchPixels += inspected.width * inspected.height + if (batchPixels > this.maxBatchImagePixels) { + throw invalidImage('Images exceed the batch pixel limit') + } + candidates.push({ + input, + inspected, + attachmentId: AttachmentId(`sha256:${digest}`) + }) + } + const additions = new Map< + ImageAttachmentRef['attachmentId'], + InspectedImage + >() + for (const candidate of candidates) { + if ( + !this.images.has(candidate.attachmentId) && + !additions.has(candidate.attachmentId) + ) { + additions.set(candidate.attachmentId, candidate.inspected) + } + } + const additionalBytes = [...additions.values()].reduce( + (total, inspected) => total + inspected.data.byteLength, + 0 + ) + if ( + this.images.size + additions.size > this.maxStoredImages || + this.storedBytes + additionalBytes > this.maxStoreBytes + ) { + throw new AttachmentError( + 'Harness attachment store is full', + 'STORAGE_LIMIT' + ) + } + return candidates.map(({ input, inspected, attachmentId }) => { + const existing = this.images.get(attachmentId) + if (existing) { + existing.references += 1 + return existing.ref + } + const name = safeImageName(input.name) + const ref = Object.freeze({ + attachmentId, + mediaType: input.mediaType, + bytes: inspected.data.byteLength, + width: inspected.width, + height: inspected.height, + ...(name ? { name } : {}) + }) + this.images.set(attachmentId, { + ref, + data: inspected.data, + references: 1 + }) + this.storedBytes += inspected.data.byteLength + return ref + }) + } + + async readImage( + ref: ImageAttachmentRef, + signal?: AbortSignal + ): Promise { + signal?.throwIfAborted() + const stored = this.images.get(ref.attachmentId) + if (!stored) { + throw new AttachmentError( + 'Harness image attachment was not found', + 'NOT_FOUND' + ) + } + if ( + stored.ref.attachmentId !== ref.attachmentId || + stored.ref.mediaType !== ref.mediaType || + stored.ref.bytes !== ref.bytes || + stored.ref.width !== ref.width || + stored.ref.height !== ref.height || + stored.ref.name !== ref.name + ) { + throw new AttachmentError( + 'Harness image attachment failed integrity validation', + 'INTEGRITY' + ) + } + return { + ref: stored.ref, + data: Uint8Array.from(stored.data) + } + } + + releaseImage(ref: ImageAttachmentRef): void { + const stored = this.images.get(ref.attachmentId) + if (!stored) { + return + } + stored.references -= 1 + if (stored.references > 0) { + return + } + this.images.delete(ref.attachmentId) + this.storedBytes -= stored.data.byteLength + } + + clear(): void { + this.images.clear() + this.storedBytes = 0 + } +} diff --git a/src/main/agent/goodbuddy-harness-control-plane.test.ts b/src/main/agent/goodbuddy-harness-control-plane.test.ts index b6d9bf1..50c1612 100644 --- a/src/main/agent/goodbuddy-harness-control-plane.test.ts +++ b/src/main/agent/goodbuddy-harness-control-plane.test.ts @@ -1,5 +1,6 @@ import { describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' +import { createCanvas } from '@napi-rs/canvas' import type { Stream } from '@agentclientprotocol/sdk' import { resolve } from 'node:path' import { @@ -9,17 +10,24 @@ import { GoodBuddyHarnessControlPlane, createBoundedAcpStream } from './goodbuddy-harness-control-plane' +import { GoodBuddyHarnessAttachmentStore } from './goodbuddy-harness-attachment-store' function controlPlane() { - return new GoodBuddyHarnessControlPlane({} as Context, { - provider: 'goodbuddy', - model: 'deepseek-test', - workspace: resolve('workspace'), - harnessVersion: '0.1.0-rc.6', - execution: { mode: 'host' }, - credentialRefs: ['GOODBUDDY_API_KEY'], - skills: [] - }) + return new GoodBuddyHarnessControlPlane( + { + on: vi.fn(), + get: vi.fn() + } as unknown as Context, + { + provider: 'goodbuddy', + model: 'deepseek-test', + workspace: resolve('workspace'), + harnessVersion: '0.1.0-rc.6', + execution: { mode: 'host' }, + credentialRefs: ['GOODBUDDY_API_KEY'], + skills: [] + } + ) } function stubAgentContext() { @@ -123,6 +131,127 @@ function stubAgentContext() { } describe('GoodBuddy Harness internal control plane', () => { + it('advertises and stores only model-enabled inline image prompts', async () => { + const textOnly = controlPlane() as unknown as { + createAgentApi(): { + initialize(): Promise<{ + agentCapabilities: { + promptCapabilities: { image: boolean } + } + }> + } + storePromptImages( + prompt: Array> + ): Promise + } + await expect(textOnly.createAgentApi().initialize()).resolves.toMatchObject({ + agentCapabilities: { + promptCapabilities: { image: false } + } + }) + await expect( + textOnly.storePromptImages([ + { + type: 'image', + mimeType: 'image/png', + data: 'aW1hZ2U=' + } + ]) + ).rejects.toThrow('does not accept image input') + + const storeContext = new Context() + const store = new GoodBuddyHarnessAttachmentStore(storeContext) + const ctx = { + on: vi.fn(), + get: vi.fn((name: string) => + name === 'attachments' ? store : undefined + ) + } as unknown as Context + const subject = new GoodBuddyHarnessControlPlane(ctx, { + provider: 'goodbuddy', + model: 'vision-test', + supportsImageInput: true, + workspace: resolve('workspace'), + harnessVersion: '0.1.0-rc.6', + execution: { mode: 'host' }, + credentialRefs: ['GOODBUDDY_API_KEY'], + skills: [] + }) as unknown as { + createAgentApi(): { + initialize(): Promise<{ + agentCapabilities: { + promptCapabilities: { image: boolean } + } + }> + } + storePromptImages( + prompt: Array> + ): Promise< + Array< + Parameters[0] + > + > + releaseAttachments( + refs: Array< + Parameters[0] + > + ): void + } + const png = createCanvas(1, 1).toBuffer('image/png') + + await expect(subject.createAgentApi().initialize()).resolves.toMatchObject({ + agentCapabilities: { + promptCapabilities: { image: true } + } + }) + const refs = await subject.storePromptImages([ + { type: 'text', text: 'describe this image' }, + { + type: 'image', + mimeType: 'image/png', + data: png.toString('base64') + } + ]) + expect(refs).toHaveLength(1) + await expect(store.readImage(refs[0]!)).resolves.toMatchObject({ + ref: expect.objectContaining({ + mediaType: 'image/png', + width: 1, + height: 1 + }) + }) + + subject.releaseAttachments(refs) + await expect(store.readImage(refs[0]!)).rejects.toMatchObject({ + code: 'NOT_FOUND' + }) + await expect( + subject.storePromptImages([ + { + type: 'image', + mimeType: 'image/png', + data: png.toString('base64'), + uri: 'https://example.com/reference.png' + } + ]) + ).rejects.toThrow('invalid inline image') + const saveImages = vi.spyOn(store, 'saveImages') + const largeInlineData = Buffer.alloc(800 * 1024).toString( + 'base64' + ) + await expect( + subject.storePromptImages( + Array.from({ length: 3 }, () => ({ + type: 'image' as const, + mimeType: 'image/png', + data: largeInlineData + })) + ) + ).rejects.toThrow('invalid inline image') + expect(saveImages).not.toHaveBeenCalled() + await storeContext.fiber.dispose() + }) + it('requires a versioned handshake before privileged extensions', async () => { const subject = controlPlane() diff --git a/src/main/agent/goodbuddy-harness-control-plane.ts b/src/main/agent/goodbuddy-harness-control-plane.ts index 7b7bf0d..a601688 100644 --- a/src/main/agent/goodbuddy-harness-control-plane.ts +++ b/src/main/agent/goodbuddy-harness-control-plane.ts @@ -6,10 +6,16 @@ import { RequestError, type Agent, type AgentSideConnection as AcpAgentConnection, + type ContentBlock as AcpContentBlock, type Stream } from '@agentclientprotocol/sdk' import type { Context } from '@deepseek-ai/cordis' import type { AgentHandle } from '@deepseek-ai/dsh-agent' +import { + AttachmentError, + type ImageAttachmentRef, + type SaveImageAttachment +} from '@deepseek-ai/dsh-attachment' import { CredentialProvider, type CredentialInfo, @@ -19,6 +25,7 @@ import { import { createUserMessage, errorChain, + type ContentBlock as HarnessContentBlock, type TokenUsage } from '@deepseek-ai/dsh-llm' import { @@ -39,6 +46,7 @@ import { GOODBUDDY_TOOLS_CALL, GOODBUDDY_TOOLS_LIST } from './deepseek-harness-protocol' +import { GoodBuddyHarnessAttachmentStore } from './goodbuddy-harness-attachment-store' export { GOODBUDDY_CONTROL_PROTOCOL_VERSION, GOODBUDDY_CREDENTIAL, @@ -94,6 +102,7 @@ export type GoodBuddyHarnessCapabilities = { export type GoodBuddyHarnessControlConfig = { provider: string model: string + supportsImageInput?: boolean workspace: string harnessVersion: string execution: GoodBuddyHarnessCapabilities['execution'] @@ -117,6 +126,7 @@ type Preparation = { type OwnedSession = { handle: AgentHandle + attachmentRefs: ImageAttachmentRef[] preparation?: Preparation proxyTools: Map< string, @@ -375,24 +385,27 @@ function boundedJson(value: unknown): string | undefined { } } -function promptText( - prompt: readonly { type: string; text?: string }[] -): string { - if ( - prompt.some( - (block) => - block.type !== 'text' && - block.type !== 'resource_link' - ) - ) { - throw RequestError.invalidParams( - undefined, - 'only text and resource_link prompt content is supported' - ) +function promptText(prompt: readonly AcpContentBlock[]): string { + const text: string[] = [] + for (const block of prompt) { + switch (block.type) { + case 'text': + text.push(block.text) + break + case 'image': + case 'resource_link': + break + case 'audio': + case 'resource': + throw RequestError.invalidParams( + undefined, + `unsupported ACP prompt content: ${block.type}` + ) + default: + block satisfies never + } } - return prompt - .map((block) => (block.type === 'text' ? block.text ?? '' : '')) - .join('') + return text.join('') } function turnReason(event: SessionEvent): string | undefined { @@ -957,6 +970,118 @@ export class GoodBuddyHarnessControlPlane { } } + private attachmentStore(): GoodBuddyHarnessAttachmentStore { + const store = this.ctx.get('attachments') + if (!(store instanceof GoodBuddyHarnessAttachmentStore)) { + throw RequestError.internalError( + undefined, + 'Harness image attachment service is unavailable' + ) + } + return store + } + + private releaseAttachments(refs: readonly ImageAttachmentRef[]): void { + if (refs.length === 0) { + return + } + const store = this.attachmentStore() + for (const ref of refs) { + store.releaseImage(ref) + } + } + + private async storePromptImages( + prompt: readonly AcpContentBlock[] + ): Promise { + const imageBlocks = prompt.filter( + ( + block + ): block is Extract => + block.type === 'image' + ) + if (imageBlocks.length === 0) { + return [] + } + if (!this.config.supportsImageInput) { + throw RequestError.invalidParams( + undefined, + 'the selected model does not accept image input' + ) + } + const store = this.attachmentStore() + if (imageBlocks.length > store.imageLimits.maxImagesPerMessage) { + throw RequestError.invalidParams( + undefined, + 'too many images in one prompt' + ) + } + const inputs: SaveImageAttachment[] = [] + let totalBytes = 0 + for (const block of imageBlocks) { + if ( + block.uri != null || + !store.imageLimits.mediaTypes.includes( + block.mimeType as SaveImageAttachment['mediaType'] + ) || + block.data.length === 0 || + block.data.length % 4 !== 0 || + block.data.length > + Math.ceil(store.imageLimits.maxImageBytes / 3) * 4 || + !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/u.test( + block.data + ) + ) { + throw RequestError.invalidParams( + undefined, + 'invalid inline image prompt content' + ) + } + const data = Buffer.from(block.data, 'base64') + totalBytes += data.byteLength + if ( + data.byteLength === 0 || + data.byteLength > store.imageLimits.maxImageBytes || + totalBytes > store.imageLimits.maxMessageImageBytes || + data.toString('base64') !== block.data + ) { + throw RequestError.invalidParams( + undefined, + 'invalid inline image prompt content' + ) + } + inputs.push({ + data, + mediaType: block.mimeType as SaveImageAttachment['mediaType'] + }) + } + try { + return await store.saveImages(inputs) + } catch (error) { + if (error instanceof AttachmentError) { + if (error.code === 'INVALID_IMAGE') { + throw RequestError.invalidParams( + undefined, + 'invalid inline image prompt content' + ) + } + if ( + error.code === 'STORAGE_LIMIT' || + error.code === 'DECODER_UNAVAILABLE' + ) { + throw RequestError.internalError( + undefined, + 'Harness image attachment service is unavailable' + ) + } + } + throw RequestError.internalError( + undefined, + 'Harness image attachment storage failed' + ) + } + } + private createAgentApi(): Agent { this.observeSessions() return { @@ -968,7 +1093,7 @@ export class GoodBuddyHarnessControlPlane { }, agentCapabilities: { promptCapabilities: { - image: false, + image: this.config.supportsImageInput === true, audio: false, embeddedContext: false }, @@ -1050,6 +1175,7 @@ export class GoodBuddyHarnessControlPlane { } this.sessions.set(sessionId, { handle, + attachmentRefs: [], proxyTools: new Map(), askToolDefinitions }) @@ -1091,10 +1217,26 @@ export class GoodBuddyHarnessControlPlane { 'empty prompt' ) } - const message = createUserMessage({ - content: [{ type: 'text', text }], - source: { kind: 'user' } - }) + const attachmentRefs = await this.storePromptImages( + params.prompt + ) + let message: ReturnType + try { + const content: HarnessContentBlock[] = [ + { type: 'text', text }, + ...attachmentRefs.map((attachment) => ({ + type: 'image' as const, + attachment + })) + ] + message = createUserMessage({ + content, + source: { kind: 'user' } + }) + } catch (error) { + this.releaseAttachments(attachmentRefs) + throw error + } const stopReason = await new Promise( (resolve, reject) => { record.inflight = { @@ -1110,9 +1252,11 @@ export class GoodBuddyHarnessControlPlane { record.handle.agent.followup(message) } catch (error) { record.inflight = undefined + this.releaseAttachments(attachmentRefs) reject(error) return } + record.attachmentRefs.push(...attachmentRefs) void record.handle.agent.whenIdle().then(() => { const current = record.inflight if (current?.messageId !== message.id) { @@ -1255,7 +1399,11 @@ export class GoodBuddyHarnessControlPlane { record.inflight.resolve('cancelled') record.inflight = undefined } - await record.handle.dispose() + try { + await record.handle.dispose() + } finally { + this.releaseAttachments(record.attachmentRefs) + } } async dispose(): Promise { @@ -1273,6 +1421,9 @@ export class GoodBuddyHarnessControlPlane { await Promise.allSettled( sessions.map(([, record]) => record.handle.dispose()) ) + for (const [, record] of sessions) { + this.releaseAttachments(record.attachmentRefs) + } })() await this.disposing } diff --git a/src/main/deepseek-harness-host.test.ts b/src/main/deepseek-harness-host.test.ts index e888025..d7c919c 100644 --- a/src/main/deepseek-harness-host.test.ts +++ b/src/main/deepseek-harness-host.test.ts @@ -11,6 +11,7 @@ import type { CreateAgentOptions } from '@deepseek-ai/dsh-agent' import { GOODBUDDY_HARNESS_MAX_STEP_TOKENS } from './agent/goodbuddy-harness-control-plane' +import { GoodBuddyHarnessAttachmentStore } from './agent/goodbuddy-harness-attachment-store' import { tmpdir } from 'node:os' import { basename, join } from 'node:path' @@ -80,6 +81,7 @@ describe('controlled DeepSeek Harness host', () => { api: 'openai-completions', provider: 'goodbuddy', model: 'qwen-plus', + supportsImageInput: true, harnessVersion: '0.1.0-rc.6', credentialRefs: ['GOODBUDDY_API_KEY'], skillPackages: [], @@ -91,6 +93,9 @@ describe('controlled DeepSeek Harness host', () => { expect(host.context.fs.sandboxMode).toBeUndefined() expect(host.context.shell.sandboxMode).toBeUndefined() + expect(host.context.get('attachments')).toBeInstanceOf( + GoodBuddyHarnessAttachmentStore + ) expect( host.context.shell.resolve({ command: 'echo goodbuddy-host-execution' diff --git a/src/main/deepseek-harness-host.ts b/src/main/deepseek-harness-host.ts index 0bc4da1..5c5f8b4 100644 --- a/src/main/deepseek-harness-host.ts +++ b/src/main/deepseek-harness-host.ts @@ -19,6 +19,9 @@ import * as ToolBash from '@deepseek-ai/dsh-tool-bash' import * as ToolFs from '@deepseek-ai/dsh-tool-fs' import * as ToolPwsh from '@deepseek-ai/dsh-tool-pwsh' import * as ShellEnv from '@deepseek-ai/dsh-shell-env' +import { + GoodBuddyHarnessAttachmentStore +} from './agent/goodbuddy-harness-attachment-store' import { GoodBuddyCredentialProvider, GoodBuddyHarnessControlPlane, @@ -31,8 +34,8 @@ import { } from './agent/deepseek-harness-extension-loader' import type { Stream } from '@agentclientprotocol/sdk' import { isDeepSeekHarnessCompatibleBaseUrl } from '../shared/deepseek-harness-compatibility' +import { DEEPSEEK_HARNESS_MAX_FRAME_BYTES } from './agent/deepseek-harness-control-protocol' -const DEFAULT_MAX_FRAME_BYTES = 1024 * 1024 const MAX_DIAGNOSTIC_BYTES = 64 * 1024 export type ControlledHarnessHostConfig = Omit< @@ -204,9 +207,10 @@ async function loadControlledSkills( * Boots a fixed, programmatic Cordis graph. It never imports app-boot, a * profile loader, settings-file, local credentials, persistence, telemetry, * web, HMR, marketplace discovery, direct MCP clients, jobs, subagents, - * hooks, or workflow packages. The control plane registers Main-selected - * Skill snapshots, Main-mediated MCP tool proxies, and explicitly enabled - * extension entrypoints. + * hooks, or workflow packages. When the selected model declares image input, + * the graph adds only a bounded process-local attachment store. The control + * plane registers Main-selected Skill snapshots, Main-mediated MCP tool + * proxies, and explicitly enabled extension entrypoints. */ export async function startControlledDeepSeekHarnessHost( input: ControlledHarnessHostConfig @@ -219,6 +223,9 @@ export async function startControlledDeepSeekHarnessHost( const specs: PluginSpec[] = [ { plugin: LlmRuntime }, { plugin: SessionStore }, + ...(config.supportsImageInput + ? [{ plugin: GoodBuddyHarnessAttachmentStore }] + : []), { plugin: SkillRegistry }, { plugin: SystemPrompt, @@ -242,7 +249,14 @@ export async function startControlledDeepSeekHarnessHost( apiKeyEnv: config.credentialRefs[0], api: config.api, baseURL: config.baseUrl, - models: [{ id: config.model, input: ['text'] }] + models: [ + { + id: config.model, + input: config.supportsImageInput + ? ['text', 'image'] + : ['text'] + } + ] } } } @@ -310,13 +324,22 @@ export async function startControlledDeepSeekHarnessHost( 'Controlled Harness credential provider failed to start' ) } + const attachmentStore = ctx.get('attachments') + if ( + config.supportsImageInput && + !(attachmentStore instanceof GoodBuddyHarnessAttachmentStore) + ) { + throw new Error( + 'Controlled Harness attachment store failed to start' + ) + } startupCode = 'HOST_CONTROL_PLANE_FAILED' const rawStream = config.stream ?? createBoundedNdJsonStream( stdoutStream(), stdinStream(), - config.maxFrameBytes ?? DEFAULT_MAX_FRAME_BYTES + config.maxFrameBytes ?? DEEPSEEK_HARNESS_MAX_FRAME_BYTES ) const controlPlane = new GoodBuddyHarnessControlPlane(ctx, { ...config, @@ -325,7 +348,7 @@ export async function startControlledDeepSeekHarnessHost( execution: { mode: 'host' }, stream: createBoundedAcpStream( rawStream, - config.maxFrameBytes ?? DEFAULT_MAX_FRAME_BYTES + config.maxFrameBytes ?? DEEPSEEK_HARNESS_MAX_FRAME_BYTES ) }) controlPlane.bindCredentialProvider(credentialProvider) @@ -337,6 +360,9 @@ export async function startControlledDeepSeekHarnessHost( extensionFailures: extensions.failures, async dispose() { await controlPlane.dispose() + if (attachmentStore instanceof GoodBuddyHarnessAttachmentStore) { + attachmentStore.clear() + } await ctx.fiber.dispose() } } diff --git a/src/renderer/src/App.test.tsx b/src/renderer/src/App.test.tsx index 7f2677f..d86f334 100644 --- a/src/renderer/src/App.test.tsx +++ b/src/renderer/src/App.test.tsx @@ -886,6 +886,9 @@ describe('App', () => { expect(await screen.findByText('桌面工作区')).toBeInTheDocument() expect(screen.getByText('GOODBUDDY 工作台')).toBeInTheDocument() + expect( + document.querySelector('.composer__runtime-toolbar') + ).not.toBeInTheDocument() }) it('schedules lazy workspace routes for idle preloading', () => { @@ -4555,6 +4558,18 @@ describe('App', () => { const agentPicker = await screen.findByRole('button', { name: /OpenCode Runtime Agent/u }) + const runtimeToolbar = screen.getByRole('group', { + name: 'OpenCode 专属功能' + }) + const universalSettings = screen.getByRole('group', { + name: '对话设置' + }) + expect(runtimeToolbar).toHaveClass('composer__runtime-toolbar') + expect(runtimeToolbar).toContainElement(agentPicker) + expect(universalSettings).not.toContainElement(agentPicker) + expect(universalSettings.closest('.composer__toolbar')).toHaveClass( + 'composer__toolbar--with-runtime-controls' + ) fireEvent.click(agentPicker) fireEvent.click( within( @@ -4566,6 +4581,7 @@ describe('App', () => { const actionPicker = screen.getByRole('button', { name: /Runtime 快捷操作/u }) + expect(runtimeToolbar).toContainElement(actionPicker) fireEvent.click(actionPicker) fireEvent.click( within( @@ -4673,6 +4689,14 @@ describe('App', () => { const presetPicker = await screen.findByRole('button', { name: /Continue 配置预设.*使用设置默认预设/u }) + const runtimeToolbar = screen.getByRole('group', { + name: 'Continue 专属功能' + }) + expect(runtimeToolbar).toHaveClass('composer__runtime-toolbar') + expect(runtimeToolbar).toContainElement(presetPicker) + expect( + screen.getByRole('group', { name: '对话设置' }) + ).not.toContainElement(presetPicker) fireEvent.click(presetPicker) fireEvent.click( within( @@ -4684,6 +4708,7 @@ describe('App', () => { const actionPicker = screen.getByRole('button', { name: /Runtime 快捷操作/u }) + expect(runtimeToolbar).toContainElement(actionPicker) fireEvent.click(actionPicker) fireEvent.click( within( @@ -4801,11 +4826,17 @@ describe('App', () => { render() - fireEvent.click( - await screen.findByRole('button', { - name: '压缩上下文' - }) + const compactContext = await screen.findByRole('button', { + name: '压缩上下文' + }) + expect(compactContext.parentElement).toHaveClass( + 'composer-meta', + 'composer-meta--with-context-compact' ) + expect( + compactContext.parentElement?.firstElementChild + ).toBe(compactContext) + fireEvent.click(compactContext) await waitFor(() => expect(api.agent.compactConversation).toHaveBeenCalledWith({ requestId: expect.any(String), diff --git a/src/renderer/src/App.tsx b/src/renderer/src/App.tsx index ed7962b..784be10 100644 --- a/src/renderer/src/App.tsx +++ b/src/renderer/src/App.tsx @@ -6159,6 +6159,36 @@ function App(): React.JSX.Element { activeConversation?.messages.some( (message) => message.state === 'streaming' ) ?? false + const runtimeAgentControlAvailable = + activeRuntimeSelection?.provider === 'opencode' && + runtimeAgentOptions.length > 1 + const runtimePresetControlAvailable = + activeRuntimeSelection?.provider === 'continue' && + runtimePresetOptions.length > 1 + const runtimeActionControlAvailable = + (activeRuntimeSelection?.provider === 'opencode' || + activeRuntimeSelection?.provider === 'continue') && + runtimeActionOptions.length > 1 + const runtimeControlsAvailable = + runtimeAgentControlAvailable || + runtimePresetControlAvailable || + runtimeActionControlAvailable + const runtimeControlsProvider = runtimeControlsAvailable + ? activeRuntimeSelection?.provider === 'opencode' + ? 'OpenCode' + : activeRuntimeSelection?.provider === 'continue' + ? 'Continue' + : undefined + : undefined + const runtimeControlsLabel = runtimeControlsProvider + ? t('composer.runtimeControls.groupLabel', { + runtime: runtimeControlsProvider + }) + : '' + const runtimeContextCompactAvailable = + (activeRuntimeSelection?.provider === 'opencode' || + activeRuntimeSelection?.provider === 'continue') && + runtimeNativeSnapshot?.context.manualCompact === true const composerContextMetrics = useMemo(() => { if ( @@ -7027,7 +7057,13 @@ function App(): React.JSX.Element { }} /> -
+
- {activeRuntimeSelection?.provider === 'opencode' && - runtimeAgentOptions.length > 1 && ( -
+ {runtimeControlsProvider && ( +
+ + {runtimeControlsLabel} + +
+ {runtimeAgentControlAvailable && ( +
+
+ )}
-
+
+ {runtimeContextCompactAvailable && ( + + )} {composerContextMetrics && (
)} - {(activeRuntimeSelection?.provider === 'opencode' || - activeRuntimeSelection?.provider === 'continue') && - runtimeNativeSnapshot?.context.manualCompact && ( - - )} {contextError && ( {contextError} diff --git a/src/renderer/src/WorkspacePrimitives.test.tsx b/src/renderer/src/WorkspacePrimitives.test.tsx index 8ea26e1..638d77e 100644 --- a/src/renderer/src/WorkspacePrimitives.test.tsx +++ b/src/renderer/src/WorkspacePrimitives.test.tsx @@ -119,6 +119,39 @@ describe('WorkspacePrimitives', () => { ) }) + it('floats Runtime context compaction without shifting the composer', () => { + expect(stylesheet).toMatch( + /\.composer-wrap\s*\{[^}]*var\(--space-2\);[^}]*background:\s*var\(--surface-raised\);/u + ) + expect(stylesheet).toMatch( + /\.composer-meta\s*\{[^}]*position:\s*relative;[^}]*min-height:\s*26px;[^}]*margin:\s*var\(--space-1\) 0 0;/u + ) + expect(stylesheet).toMatch( + /\.composer-meta--with-context-compact\s*\{[^}]*padding-left:\s*calc\(/u + ) + expect(stylesheet).toMatch( + /\.composer-context-compact\s*\{[^}]*position:\s*absolute;[^}]*bottom:\s*0;[^}]*left:\s*0;/u + ) + }) + + it('separates Runtime-specific controls from the main composer toolbar', () => { + expect(stylesheet).toMatch( + /\.composer__toolbar--with-runtime-controls\s*\{[^}]*border-radius:\s*0;/u + ) + expect(stylesheet).toMatch( + /\.composer__runtime-toolbar\s*\{[^}]*display:\s*flex;[^}]*min-height:\s*52px;[^}]*border-top:\s*1px solid var\(--border-default\);[^}]*background:\s*var\(--surface-muted\);/u + ) + expect(stylesheet).toMatch( + /\.composer__runtime-controls\s*\{[^}]*display:\s*flex;[^}]*flex-wrap:\s*wrap;/u + ) + expect(stylesheet).toMatch( + /\.composer-picker--runtime > \.model-button,\s*\.composer-picker--runtime-action > \.model-button\s*\{[^}]*width:\s*220px;/u + ) + expect(stylesheet).toMatch( + /\.composer__runtime-controls \.composer-picker\s*\{[^}]*flex-basis:\s*220px;/u + ) + }) + it('keeps shared controls keyboard and pointer accessible at narrow widths', () => { expect(stylesheet).toMatch( /button\s*>\s*svg,\s*button\s*>\s*svg\s+\*\s*\{[^}]*pointer-events:\s*none;/u diff --git a/src/renderer/src/i18n/locales/en-US/app.ts b/src/renderer/src/i18n/locales/en-US/app.ts index 86b08a8..a66f64f 100644 --- a/src/renderer/src/i18n/locales/en-US/app.ts +++ b/src/renderer/src/i18n/locales/en-US/app.ts @@ -322,6 +322,7 @@ export const app = { expertLabel: 'Expert role', modeLabel: 'Work mode', runtimeControls: { + groupLabel: '{{runtime}} controls', agentLabel: 'OpenCode Runtime Agent', presetLabel: 'Continue configuration preset', actionLabel: 'Runtime shortcut', diff --git a/src/renderer/src/i18n/locales/zh-CN/app.ts b/src/renderer/src/i18n/locales/zh-CN/app.ts index 96dd032..8c01a83 100644 --- a/src/renderer/src/i18n/locales/zh-CN/app.ts +++ b/src/renderer/src/i18n/locales/zh-CN/app.ts @@ -314,6 +314,7 @@ export const app = { expertLabel: '专家角色', modeLabel: '工作模式', runtimeControls: { + groupLabel: '{{runtime}} 专属功能', agentLabel: 'OpenCode Runtime Agent', presetLabel: 'Continue 配置预设', actionLabel: 'Runtime 快捷操作', diff --git a/src/renderer/src/styles.css b/src/renderer/src/styles.css index bfedf6a..e9debd8 100644 --- a/src/renderer/src/styles.css +++ b/src/renderer/src/styles.css @@ -4038,7 +4038,7 @@ button > svg { padding: var(--space-3) max(var(--page-gutter), calc((100% - var(--content-reading)) / 2)) - var(--space-4); + var(--space-2); background: var(--surface-raised); } @@ -4387,6 +4387,42 @@ button > svg { gap: var(--space-3); } +.composer__toolbar--with-runtime-controls { + border-radius: 0; +} + +.composer__runtime-toolbar { + display: flex; + min-width: 0; + min-height: 52px; + padding: var(--space-2) var(--space-3); + align-items: center; + border-top: 1px solid var(--border-default); + border-radius: 0 0 calc(var(--radius-card) - 1px) + calc(var(--radius-card) - 1px); + background: var(--surface-muted); + gap: var(--space-3); +} + +.composer__runtime-toolbar-label { + min-width: 0; + padding-right: var(--space-3); + border-right: 1px solid var(--border-default); + color: var(--text-secondary); + font-size: var(--font-caption); + font-weight: 650; + white-space: nowrap; +} + +.composer__runtime-controls { + display: flex; + min-width: 0; + flex: 1; + align-items: center; + flex-wrap: wrap; + gap: var(--space-2); +} + .composer__controls { display: flex; min-width: 0; @@ -4483,7 +4519,7 @@ button > svg { .composer-picker--runtime > .model-button, .composer-picker--runtime-action > .model-button { - width: 138px; + width: 220px; } .composer-picker--ask svg { @@ -4670,8 +4706,12 @@ button > svg { } .composer-meta { + --composer-context-compact-width: 124px; + + position: relative; display: flex; - margin: var(--space-2) 0 0; + min-height: 26px; + margin: var(--space-1) 0 0; align-items: center; justify-content: flex-end; font-size: var(--font-caption); @@ -4680,6 +4720,12 @@ button > svg { line-height: 1.45; } +.composer-meta--with-context-compact { + padding-left: calc( + var(--composer-context-compact-width) + var(--space-3) + ); +} + .composer-meta__error { margin-right: auto; } @@ -4694,12 +4740,17 @@ button > svg { } .composer-context-compact { + position: absolute; + bottom: 0; + left: 0; display: inline-flex; + width: var(--composer-context-compact-width); min-height: 26px; padding: var(--space-1) var(--space-2); border: 1px solid var(--border-control); border-radius: var(--radius-control); align-items: center; + justify-content: center; background: var(--surface-raised); color: var(--text-secondary); font-size: var(--font-caption); @@ -4786,6 +4837,22 @@ button > svg { flex: 1 1 150px; } + .composer__runtime-toolbar { + align-items: stretch; + flex-direction: column; + } + + .composer__runtime-toolbar-label { + padding-right: 0; + padding-bottom: var(--space-2); + border-right: 0; + border-bottom: 1px solid var(--border-default); + } + + .composer__runtime-controls .composer-picker { + flex-basis: 220px; + } + .composer-picker > .model-button, .model-button { width: 100%; diff --git a/tests/build-release.test.ts b/tests/build-release.test.ts index 32ad4ac..8de20b8 100644 --- a/tests/build-release.test.ts +++ b/tests/build-release.test.ts @@ -330,7 +330,9 @@ describe('release build arguments', () => { 'node_modules/node-pty/prebuilds/**/*', 'node_modules/node-pty/build/Release/**/*', 'node_modules/koffi/**/*', - 'node_modules/@koromix/koffi-*/**/*' + 'node_modules/@koromix/koffi-*/**/*', + 'node_modules/@napi-rs/canvas{,/**/*}', + 'node_modules/@napi-rs/canvas-*/**/*' ]) ) expect(packageJson.build.asarUnpack).not.toContain(