feat: add model-aware DSH image input

DeepSeek Harness previously treated every selected model as text-only. It now carries the model profile's image capability through Main, the Utility Host, ACP, and Pi-AI, rejects unsupported images before model invocation, and validates supported JPEG/PNG data in a bounded process-local attachment store.

The Composer keeps universal controls on its first row, places OpenCode and Continue controls on a dedicated responsive row, and anchors context compaction without shifting the footer. Runtime supervision ownership for future Subagents and Jobs is documented for the right sidebar.

Release note: DeepSeek Harness 现可按所选模型连接的能力安全接收 JPEG/PNG 图片;Composer 同时将 Runtime 专属选择器移到独立一行,并固定上下文压缩入口。
This commit is contained in:
mesalogo
2026-08-16 23:30:54 +08:00
parent 18263a6b23
commit b751c70d75
30 changed files with 1589 additions and 187 deletions
+36 -4
View File
@@ -171,6 +171,7 @@ export type DeepSeekHarnessLaunchOptions = {
signal: AbortSignal
baseUrl: string
model: string
supportsImageInput: boolean
credentialRefs: readonly string[]
skillPackages: readonly RuntimeSkillPackage[]
extensionPackages: readonly ControlledHarnessExtensionPackage[]
@@ -180,6 +181,7 @@ export type DeepSeekHarnessRuntimeOptions = {
defaultWorkspace: string
baseUrl: string
model: string
supportsImageInput?: boolean
launch: (
options: DeepSeekHarnessLaunchOptions
) => Promise<DeepSeekHarnessChild>
@@ -812,6 +814,8 @@ export class DeepSeekHarnessRuntime implements AgentRuntime {
signal: launchController.signal,
baseUrl: this.options.baseUrl,
model: this.options.model,
supportsImageInput:
this.options.supportsImageInput === true,
credentialRefs: Object.keys(
this.options.credentialRefs ?? {}
),
@@ -1022,7 +1026,7 @@ export class DeepSeekHarnessRuntime implements AgentRuntime {
() =>
this.fail(new Error('DeepSeek Harness ACP 连接异常关闭'))
)
await withTimeout(
const initialization = await withTimeout(
stateWithoutCapabilities.agent.initialize({
protocolVersion: sdk.PROTOCOL_VERSION,
clientCapabilities: {},
@@ -1034,6 +1038,26 @@ export class DeepSeekHarnessRuntime implements AgentRuntime {
this.initializationTimeoutMs,
'ACP 初始化'
)
const advertisedImageInput =
Boolean(
initialization &&
typeof initialization === 'object' &&
(
initialization as {
agentCapabilities?: {
promptCapabilities?: { image?: unknown }
}
}
).agentCapabilities?.promptCapabilities?.image === true
)
if (
advertisedImageInput !==
(this.options.supportsImageInput === true)
) {
throw new Error(
'DeepSeek Harness Host 图片能力与所选模型连接不一致'
)
}
const capabilities = this.parseCapabilities(
await withTimeout(
stateWithoutCapabilities.agent.extMethod(
@@ -1408,8 +1432,11 @@ export class DeepSeekHarnessRuntime implements AgentRuntime {
authorize?: RuntimeAuthorizer
): AsyncGenerator<RuntimeEvent, void, void> {
signal.throwIfAborted()
if (request.images?.length) {
throw new Error('DeepSeek Harness Runtime 暂不支持图像输入')
if (
request.images?.length &&
this.options.supportsImageInput !== true
) {
throw new Error('当前 DeepSeek Harness 模型连接未启用图像输入')
}
const release = await this.acquireConversation(
request.conversationId,
@@ -1477,7 +1504,12 @@ export class DeepSeekHarnessRuntime implements AgentRuntime {
{
type: 'text',
text: flattenPrompt(request)
}
},
...(request.images ?? []).map((image) => ({
type: 'image' as const,
data: image.data,
mimeType: image.mediaType
}))
]
}),
this.promptTimeoutMs,