feat(storage): move desktop SQLite ownership into a storage utility process
Main no longer opens desktop business SQLite. A single utility-process
storage owner holds the Assistant, Knowledge, attachment, document-result,
remote-binding and native model-ledger databases and runs whole domain
transactions; Main and Runtime callers use explicit async domain operations.
- Remove the Main readWithFallback path and the supervision worker's
separate business write connection; manifests are scanned read-only and
committed by the sole writer.
- Remote events are ACKed only after commit; process loss reports
unconfirmed writes instead of replaying them.
- Use native utilityProcess messaging with one in-flight bound (32) and
ordered waiting instead of capacity rejection and retries.
- Start the storage process before the window so startup overlaps.
- Keep short tool output in memory and store full output once in owned
storage; paging, branching, restart and deletion reuse that copy.
- Put required runtime launch files under goodbuddy-prefixed owned temp
directories and reclaim confirmed stale ones; stop writing duplicate
parsed.md.
- Add lint guards for SQLite ownership and architecture, concurrency
acceptance fixtures, and maintenance/architecture documentation.
Validation: typecheck and lint pass. Alternating baseline (edf019aa) vs
candidate runs show no regression on single-session, C01, C02, C04 and a
5-minute C06; Main p99 drops from ~21 ms to ~16.7 ms under concurrency.
Fresh portable copy (1.75M events) upgrade content unchanged; real-model
output paging passes with 4 external requests. Full npm test runs had
1-9 different timing-sensitive failures per run that pass when rerun.
Not covered: C03 review publication, C07 source body, app-level C05.
This commit is contained in:
mesalogo
@@ -0,0 +1,64 @@
|
||||
# Assistant Storage Consumers
|
||||
|
||||
The S02 domain consumers use the typed port in
|
||||
`src/main/assistant-storage-port.ts`. This implements the consumer side of
|
||||
[Desktop Storage Foundation](./desktop-storage-foundation.md). Production
|
||||
composition in `src/main/index.ts` supplies the client after storage readiness;
|
||||
the consumer services do not construct an Assistant database.
|
||||
|
||||
## Main Integration
|
||||
|
||||
```ts
|
||||
const storage = new DesktopStorageClient({
|
||||
...paths,
|
||||
onChanged: createAssistantStorageOnChanged({
|
||||
onMagicNotesChanged,
|
||||
onMagicTodosChanged,
|
||||
onModelUsageChanged,
|
||||
onExecutionStatsChanged
|
||||
})
|
||||
})
|
||||
await storage.ready
|
||||
const assistant = createAssistantStoragePort(storage)
|
||||
```
|
||||
|
||||
`createAssistantStoragePort` binds only the explicit Assistant operation allowlist
|
||||
and `supervisionContext`. Its return type is `AsyncAssistantStoragePort`: every
|
||||
operation returns a Promise. There is no SQL, arbitrary property proxy, remote
|
||||
callback or live repository API. `AssistantStoragePort` uses `Awaitable` results
|
||||
so existing synchronous database fixtures can be injected into unit tests.
|
||||
Domain change callbacks run in Main through `DesktopStorageClient.onChanged`.
|
||||
|
||||
The subagent service, native client coordinator, terminal manager, computer-control
|
||||
audit sink, channel dedup/outbox, image service and Magic Notes IPC helpers accept
|
||||
this port. Notes helpers retain sender validation, schema validation, source-message
|
||||
checks and revision checks. Note analysis awaits the model-usage callback before
|
||||
publishing its result. Channel and audit adapters return persistence completion to
|
||||
their existing asynchronous callers.
|
||||
|
||||
Image service `initialize`, `persistUploads`, `getOperation`, `cancel`,
|
||||
`cancelConversation`, `regenerate` and `dispose` return Promises. Await upload
|
||||
persistence before submitting generation. Await cancellation and shutdown writes.
|
||||
`onUsage` may return a Promise and must return the actual persistence operation.
|
||||
Notification-only `onOperation` errors are reported without changing provider
|
||||
outcomes; storage and usage-write failures reject instead of reporting durable
|
||||
success. Shutdown reports retained persistence failures.
|
||||
|
||||
Subagent event and model-usage callbacks may return Promises. The service awaits
|
||||
task creation, events and terminal status writes; shutdown also drains task
|
||||
creation and queued cancellation writes that have not entered a scheduler slot.
|
||||
Native client composition additionally forwards the storage-backed
|
||||
`openModelCallLedger` factory to both browser and terminal clients.
|
||||
|
||||
Stop producers and await service disposal before closing `DesktopStorageClient`.
|
||||
This consumer migration does not establish full desktop performance acceptance or
|
||||
replace the separate Main/IPC composition and concurrent acceptance work.
|
||||
|
||||
## Validation
|
||||
|
||||
The 11 focused suites pass with 94 tests, including asynchronous image storage,
|
||||
commit-before-notification, usage-write rejection, subagent shutdown during task
|
||||
creation, channel delivery, audit persistence and IPC sender/schema checks.
|
||||
Repository-wide type checking remains dependent on the concurrent Main/IPC,
|
||||
Knowledge and runtime-storage integrations. No latency or full concurrent-desktop
|
||||
acceptance claim is made by these unit and loopback tests.
|
||||
@@ -1,75 +1,388 @@
|
||||
# Desktop Storage Direction
|
||||
# Desktop Storage And Concurrent Execution
|
||||
|
||||
## Status and scope
|
||||
## Decision and delivery
|
||||
|
||||
2026-10-05: proposed target for the remaining PERF-15/PERF-16 work. The user has
|
||||
requested documentation first, followed by bounded local fixes and a commit.
|
||||
The full storage-service migration remains a separate discussion after those fixes.
|
||||
Current desktop storage still has Main-owned business writes and selected worker
|
||||
reads; this document does not describe a completed migration.
|
||||
Updated 2026-10-05 against `2be57492`. The user selected one complete architecture
|
||||
delivery for PERF-15/PERF-16 and the necessary performance guards. This document
|
||||
supersedes the earlier proposal to leave full migration for a later discussion.
|
||||
The integrated migration and acceptance are not complete. The implemented storage
|
||||
foundation, production composition and remaining acceptance work are recorded in
|
||||
[Storage Foundation Integration](./desktop-storage-foundation.md).
|
||||
Local fixes in `b90ff50f` remain the measured starting point.
|
||||
|
||||
## Target responsibilities
|
||||
Deliver the integrated design in one candidate, with normal product workflows and
|
||||
concurrent acceptance passing together. Internal implementation dependencies do
|
||||
not create separate partial product deliveries. A worker-only prototype, an async
|
||||
facade over Main SQL, or a dual-write transition does not meet completion criteria.
|
||||
|
||||
- Renderer owns presentation, domain stores and disposable visible-data caches.
|
||||
It consumes summaries, pages and incremental notifications, not full database copies.
|
||||
- Electron Main owns windows, trusted IPC, credentials, OS integration and service
|
||||
lifecycle. Heavy SQL, JSON assembly and document processing execute elsewhere.
|
||||
- A desktop storage utility process would own business database connections and
|
||||
complete domain transactions. Retain the existing database files, schemas and
|
||||
durability distinctions. One write owner per database does not require merging databases.
|
||||
- Existing read-only workers can serve heavy queries under the storage service's
|
||||
lifecycle. Do not place every read and write behind one long-running global job.
|
||||
- Parsing, OCR, model inference and Runtime execution remain separate workloads.
|
||||
Remote Agent databases remain remote; desktop persistence acknowledges remote
|
||||
events only after the required local commit succeeds.
|
||||
This document owns architecture and scope. [Performance principles](./performance-principles.md)
|
||||
own normative budgets and concurrency rules; [concurrent acceptance](../quality/concurrent-desktop-acceptance.md)
|
||||
owns reproducible workloads and evidence. The [roadmap](../roadmap/product-performance-experience-improvement-plan.md)
|
||||
tracks status, and the [local repair record](../review/kiss-local-fixes-2026-10-05.md)
|
||||
preserves earlier measurements. Migration completion requires new acceptance evidence.
|
||||
|
||||
The service boundary consists of domain operations such as publishing a review or
|
||||
committing an event batch. Do not expose arbitrary SQL, transport transaction
|
||||
callbacks, or split an atomic transaction across individual IPC calls. Synchronous
|
||||
SQLite inside its dedicated execution environment can remain appropriate.
|
||||
## Requirements
|
||||
|
||||
## Correctness and lifecycle
|
||||
| ID | Required outcome |
|
||||
| --- | --- |
|
||||
| SA-01 | Desktop Main does not open or query business SQLite, directly or through a helper. |
|
||||
| SA-02 | Complete domain transactions execute under one write owner per database. |
|
||||
| SA-03 | Multiple projects, conversations and subtasks progress without unrelated global waits. |
|
||||
| SA-04 | Requests, event buffers, result payloads and caches have bounded lifetimes and capacity. |
|
||||
| SA-05 | Persistence, ordering, cancellation, recovery and user interaction semantics are preserved. |
|
||||
| SA-06 | Startup, process failure and shutdown have one tested ownership path. |
|
||||
| SA-07 | Static guards and concurrent production-path tests enforce the architecture. |
|
||||
| SA-08 | GoodBuddy eliminates unnecessary intermediate disk writes and duplicate full outputs; necessary temporary resources have explicit ownership and cleanup. |
|
||||
|
||||
Preserve write-before-read ordering, result/checkpoint atomicity, cancellation
|
||||
settlement, event deduplication and commit-before-ACK. A rejected caller Promise
|
||||
does not establish that a writer stopped. Reuse existing operation IDs and unique
|
||||
constraints to resolve uncertain outcomes; do not blindly replay mutations or add
|
||||
a general recovery journal.
|
||||
## Responsibilities
|
||||
|
||||
Open business connections after required upgrades. Stop producers before draining
|
||||
required writes and closing storage. A failed storage process must not silently
|
||||
fall back to Main SQL or an empty persistent-data replacement. Keep bounded requests
|
||||
and explicit failure/retry behavior without adding product approval gates.
|
||||
```mermaid
|
||||
flowchart TB
|
||||
UI[Renderer: domain stores and visible views] <--> MAIN[Main: trusted IPC and lifecycle]
|
||||
MAIN <--> STORAGE[Storage utilityProcess: domain operations and write owners]
|
||||
STORAGE <--> READ[Existing bounded read workers]
|
||||
MAIN <--> RUNTIME[Runtime hosts and compute workers]
|
||||
MAIN <--> AGENT[Remote Agent]
|
||||
```
|
||||
|
||||
## VS Code comparison
|
||||
- Renderer retains presentation, local interaction state and disposable visible-data
|
||||
caches. Domain selectors subscribe narrowly. Hidden views retain required state,
|
||||
not continuous full rendering or whole-history refreshes.
|
||||
- Main retains windows, trusted sender/schema checks, OS APIs, credential decryption,
|
||||
network/runtime orchestration and process lifecycle. SQL, large result assembly,
|
||||
hashing and bulk attachment file work do not execute on its event loop.
|
||||
- One desktop storage utility process owns the existing business database files and
|
||||
exposes typed asynchronous domain operations. Keep synchronous SQLite inside this
|
||||
dedicated environment where it simplifies transaction correctness.
|
||||
- Reuse bounded read-only workers for expensive reads. They open only after upgrade,
|
||||
read committed WAL snapshots, and do not run initialization or recovery. Storage
|
||||
owns their lifecycle; a project or conversation does not create a new worker.
|
||||
- Parsing, OCR, embedding and Runtime execution retain their existing hosts. Network
|
||||
requests and model execution must not run inside storage transactions or hold the
|
||||
storage write queue. This work does not relocate browser-dependent OCR engines.
|
||||
- Remote Agent databases remain remote. Changes to desktop storage must preserve
|
||||
remote replay, controller ownership and commit-before-ACK.
|
||||
|
||||
VS Code is a reference for service separation, asynchronous work, lazy activation,
|
||||
cached UI state and shutdown coordination, not a requirement to copy its process map.
|
||||
Its application/profile/workspace state connections are owned by Electron Main;
|
||||
the backend uses asynchronous `@vscode/sqlite3`, while cached reads are synchronous
|
||||
and writes are batched. GoodBuddy's relational history, source bodies and durable
|
||||
remote-event acknowledgements require different payload and persistence boundaries.
|
||||
One write owner means exclusive business-write authority, not necessarily one OS
|
||||
thread for all reads. Startup upgrades and existing maintenance/checkpoint workers
|
||||
must be explicitly owned and excluded from competing business writes. Keep their
|
||||
current bounded mechanisms where valid; do not create independent writers per domain.
|
||||
|
||||
Sources reviewed on 2026-10-05 (upstream main, not a pinned performance comparison):
|
||||
## Ownership inventory
|
||||
|
||||
Inventory records current owners; destinations below are targets, not implementation facts.
|
||||
|
||||
| Surface | Current entry | Required destination |
|
||||
| --- | --- | --- |
|
||||
| `assistant.sqlite` | `src/main/assistant/assistant-database.ts`, constructed in Main | Storage service: conversations, tasks/events, activity, notes metadata, channels, supervision and timing persistence |
|
||||
| `knowledge.sqlite` | `src/main/knowledge/knowledge-service.ts` and `knowledge-database.ts` | Storage service: schema, writes and external metadata; existing workers for costly reads |
|
||||
| `conversation-attachments.sqlite` and referenced files | `src/main/conversation-attachment-storage.ts` | Storage service: references, adoption, reconciliation and bounded file IO |
|
||||
| Document-result persistent lookup and note bodies | `document-result-storage.ts`, `magic-notes` storage | Async domain access to associated files; preserve original directories and reference semantics |
|
||||
| Local remote-runtime binding DB | `src/main/remote-agent/managed-remote-execution-services.ts` | Storage service: transactional binding transitions and uniqueness |
|
||||
| Native terminal / DS Web model-call ledgers | `native-terminal-client.ts`, `native-dsh-web-client.ts` instantiate `AgentModelCallLedger` | Storage service manages desktop ledger files under the existing client lifecycle; Main retains model/credential routing |
|
||||
| Read, checkpoint and upgrade connections | `readonly-query-worker.ts`, `assistant-storage-worker.ts` | Service-owned steady-state reads/checkpoints; exclusive upgrade execution before readiness |
|
||||
| Supervision manifest writer | `readonly-query-worker.ts` supervision mode and `SupervisionReviewStore` | Reuse scanning and manifest logic, but route its business mutations through the same write owner as publication; remove independent writer authority |
|
||||
| Remote Agent journal, owners, prompts and ledger | `src/agent-daemon/daemon.ts`, `runtime-composition.ts` | Existing remote ownership, unchanged by desktop connection migration |
|
||||
| Encrypted JSON settings and transient runtime caches | Existing settings/runtime services | Keep existing appropriate owners; async settings do not move merely to match a diagram |
|
||||
|
||||
An import scan must also find `openPrivateSqliteDatabase` and transitive constructors:
|
||||
an implementation under `agent-daemon` can still open SQLite when called in desktop
|
||||
Main. Do not weaken shared Agent code or relocate remote databases as a side effect.
|
||||
|
||||
Preserve existing database names, table formats, IDs, paths, scopes and durability
|
||||
settings. Process relocation alone does not require a schema bump. Necessary schema
|
||||
changes follow the [migration guide](../development/database-migrations.md).
|
||||
|
||||
## Existing work and consolidation
|
||||
|
||||
This inventory was checked against production callers on 2026-10-05. Reuse means
|
||||
preserving working algorithms and contracts, not claiming their current transport,
|
||||
cancellation or close semantics already meet the target. Implement the replacement
|
||||
at the existing seam and remove superseded callers in the same delivery.
|
||||
|
||||
| Existing mechanism | Reuse or extend | Retire or avoid |
|
||||
| --- | --- | --- |
|
||||
| [ReadonlyQueryReader](../../src/main/readonly-query-reader.ts), [worker handlers](../../src/main/readonly-query-worker.ts) | Reuse request correlation, named operations, errors, lazy lifecycle and reader handlers; adapt the outer utility transport and strengthen typed operations, readiness, byte bounds and settlement-aware close | No second equivalent storage RPC client or admission queue; remove production infrastructure fallback through `readWithFallback`, not legitimate owner-local transaction reads |
|
||||
| [Startup storage](../../src/main/assistant-storage-startup.ts), [upgrade worker](../../src/main/assistant-storage-worker.ts) | Preserve migration algorithms, progress, retry and cancellation; move SQL preflight into the storage-owned exclusive startup path | Remove Main SQL preflight and competing upgrade launch ownership; do not add a new migration framework |
|
||||
| [Checkpoint and supervision connections](../../src/main/assistant/assistant-database.ts), [review store](../../src/main/assistant/supervision-review-store.ts) | Reuse PASSIVE checkpoint and manifest scan/revision algorithms; make checkpoint exit awaited and manifest writes owner-controlled | No second maintenance service or independent business writer hidden in a read-worker entry |
|
||||
| [EmbeddingIndexRepository](../../src/main/knowledge/embedding-index-coordinator.ts), [binding store interface](../../src/main/agent/runtime-session-binding-store.ts), [activity repository](../../src/main/assistant/activity-history-repository.ts) | Reuse domain interfaces and transaction algorithms; adapt the desktop backing implementation and async callers | Do not add remote object proxies or duplicate repositories; Promise-returning methods are not proof of off-Main execution |
|
||||
| [RemoteEventBatcher](../../src/main/remote-event-batcher.ts) | Extend existing batching with awaited persistence/flush and pending-plus-in-flight bounds; update recovery, background and interactive IPC consumers | No second durable event coalescer, receipt ledger or retry queue; no ACK or forwarding before commit |
|
||||
| [AgentModelCallLedger](../../src/agent-daemon/agent-model-gateway.ts), [terminal client](../../src/main/agent/native-terminal-client.ts), [DS Web client](../../src/main/agent/native-dsh-web-client.ts) | Preserve schema, claim/deduplication and uncertain outcomes; desktop access becomes awaited storage operations | Do not relocate remote Agent ledgers; remove desktop concrete synchronous opens and early directory deletion |
|
||||
| [Utility transport](../../src/main/knowledge/embedding-utility-transport.ts), [Harness readiness](../../src/main/agent/deepseek-harness-utility-launcher.ts), [shutdown helpers](../../src/main/shutdown.ts) | Reuse fork/entry/exit adapters and phased application shutdown; add storage readiness and drain at those seams | No new generic process supervisor, duplicate shutdown path or copied Harness streaming/version protocol |
|
||||
| [Runtime controller](../../src/main/agent/runtime-controller.ts), [selected runtimes](../../src/main/agent/selected-runtime-manager.ts), [remote backend](../../src/agent-daemon/runtime-acp-backend.ts) | Keep request ownership, session reuse and retirement; inspect remaining global open/prepare/close waits with healthy peers | No second Runtime pool or global execution manager; the prior route-cleanup fix does not prove every global wait is gone |
|
||||
| [Subagent scheduler](../../src/main/assistant/subagent-scheduler.ts), [supervision model pool](../../src/main/assistant/supervision-model-pool.ts), [review admission](../../src/main/assistant/supervisor-service.ts) | Retain task slots, review single-flight and per-model-call pool as distinct existing scopes; extend capacity/settlement tests where needed | Do not merge ordinary chat into the supervision pool or create another task state machine/provider scheduler |
|
||||
|
||||
The current ordinary reader removes a cancelled request from `pending` before worker
|
||||
settlement, and `close()` provides no exit barrier. Supervision waits longer and can
|
||||
substitute cancellation for a successful result. Extend these semantics to
|
||||
represent actual execution and committed outcomes; do not copy them unchanged into
|
||||
the storage client and call its capacity or shutdown complete.
|
||||
|
||||
Supervision's temporary scan table belongs to one SQLite connection. Retain the
|
||||
bounded scan, transfer only bounded manifest data to its write owner, and preserve
|
||||
initialization/resume invariants. Existing 200-row commits run in synchronous loops;
|
||||
they do not themselves yield an execution turn. Yield at safe existing batch boundaries
|
||||
without dividing final atomic publication or introducing a general scheduler.
|
||||
|
||||
Renderer consolidation stays within existing domains:
|
||||
|
||||
- [Conversation store](../../src/renderer/src/conversation-store.ts) and
|
||||
[persistence](../../src/renderer/src/conversation-persistence.ts) already own ordered
|
||||
deltas, save serialization, history requests and quit flushing. Extend those paths,
|
||||
not a second `conversation-sync` queue. Its current React-aware integration is not
|
||||
a reason to introduce another store library.
|
||||
- [Activity sync](../../src/renderer/src/activity-sync.ts) owns paging, pending writes
|
||||
and retry; [task sync](../../src/renderer/src/task-sync.ts) owns task/artifact refresh.
|
||||
Tighten projections and retention there. A resolved flush that caught an error and
|
||||
requeued data is not a durable success barrier; extend its explicit completion
|
||||
semantics where dependent reads or shutdown require persistence.
|
||||
- [Live messages](../../src/renderer/src/live-message-store.ts),
|
||||
[public event buffering](../../src/main/agent-event-buffer.ts), and
|
||||
[terminal batching](../../src/renderer/src/terminal-output-batcher.ts) serve distinct
|
||||
display stages. Keep them separate from durable remote batching and avoid an
|
||||
additional universal event bus or a second coalescing delay.
|
||||
- Retain domain cache owners: [keep-alive](../../src/renderer/src/keep-alive-cache.ts),
|
||||
[Markdown](../../src/renderer/src/markdown-render-cache.ts),
|
||||
[channel status](../../src/renderer/src/channel-status-store.ts), and
|
||||
[Supervisor scope reads](../../src/renderer/src/SupervisionStories.tsx).
|
||||
Preserve pinned-pane exceptions and scope/revision guards; strengthen bounds at
|
||||
their existing owners.
|
||||
|
||||
Runtime session behavior remains authoritative in the
|
||||
[reuse design](../features/assistant-workbar/runtime-process-reuse-technical-design.md).
|
||||
[Task timing](../features/task-and-job/technical-design.md) and
|
||||
[supervision model scheduling](../features/conversation-supervision/technical-design.md)
|
||||
retain their existing semantics. Ending an elapsed-time segment on abort does not
|
||||
release an execution slot before cleanup; do not unify these clocks or restore
|
||||
historical timing scans/statistics workers during this migration.
|
||||
|
||||
Completion includes a removal inventory tied to the table above: old Main opens,
|
||||
external live-store factories, production sync fallbacks, obsolete transport callers
|
||||
and independent business writers have no production references. Remove only the
|
||||
superseded implementations, not valid in-owner synchronous algorithms, remote stores,
|
||||
released-data compatibility or historical test evidence. No permanent dual path or
|
||||
unused compatibility wrapper is part of the delivery.
|
||||
|
||||
## Operation boundary
|
||||
|
||||
Reuse repository algorithms behind domain operations: save conversation changes,
|
||||
commit remote events, publish a review, adopt attachment references, or transition
|
||||
a runtime binding. Public renderer IPC can keep its current shape while Main awaits
|
||||
the storage client. No generic SQL endpoint or function serialization is introduced.
|
||||
|
||||
Existing transaction callbacks and returned live repository objects stay inside the
|
||||
owner. Replace their external use with typed inputs/results. In particular,
|
||||
`readSnapshot`, Knowledge publication callbacks and Supervisor store factories must
|
||||
not be transported or accept an asynchronous callback that outlives a sync transaction.
|
||||
|
||||
The channel extends the request/handler and utility-adapter seams identified above,
|
||||
correlates requests/results and carries named operations,
|
||||
serializable arguments, cancellation and readiness/failure. Reuse installed Electron
|
||||
transport patterns; do not add a general RPC dependency, persisted transport sequence,
|
||||
protocol negotiation or a service discovery layer for this bundled child process.
|
||||
Validate actual transport behavior for buffers, Maps and errors; use DTOs. Live classes,
|
||||
subscriptions and handles remain with their owners. Errors retain usable codes and causes
|
||||
without credentials or user content in diagnostic logs.
|
||||
|
||||
Queries return the requested projection. Lists use summaries/pages; source bodies,
|
||||
tool outputs and attachments load through detail paths. Large results must not be
|
||||
JSON-assembled in Main or copied through several caches. Existing source records are
|
||||
not truncated to satisfy a latency check. A read transaction ends before posting its
|
||||
reply so a slow consumer cannot keep a WAL snapshot open.
|
||||
|
||||
## Concurrency and ordering
|
||||
|
||||
Keep ordering at the smallest required scope. A conversation's persisted sequence
|
||||
and a database's commits remain ordered; independent model waits, sessions and
|
||||
read requests do not share a global long-running lock. Runtime process reuse retains
|
||||
per-binding/session/operation identity; cancelling one request must not stop its peers.
|
||||
|
||||
Use existing finite concurrency and batch mechanisms before adding scheduling code.
|
||||
Apply capacity to queued bytes as well as operation count where payload size varies.
|
||||
When a queue fills, wait or report the existing actionable busy/error state; never
|
||||
silently discard user messages, tool results or acknowledged events. Obsolete read
|
||||
refreshes and display progress can coalesce; persisted evidence cannot. Preserve one
|
||||
bounded batch of work before yielding so other admitted producers can progress;
|
||||
change scheduling only where concurrent measurements demonstrate starvation.
|
||||
|
||||
Heavy reads use the read workers and leave necessary writes able to progress. UI/control
|
||||
requests must not wait behind model networking, import parsing or route cleanup.
|
||||
Control processing must not depend on acquiring another execution slot. This is a
|
||||
resource-boundary rule, not a mandate for a new global priority scheduler.
|
||||
|
||||
For long publications, compute immutable data before the write transaction when
|
||||
possible, then validate current database preconditions inside the transaction and
|
||||
commit the complete result/checkpoint atomically. Do not await models, filesystem
|
||||
callbacks or IPC while holding a write transaction. An atomic publication cannot be
|
||||
split into externally visible partial commits to improve timing. Worker relocation
|
||||
does not eliminate the single-writer bottleneck: acceptance measures queue wait and
|
||||
commit time, as well as Main responsiveness.
|
||||
|
||||
## Persistence and cancellation
|
||||
|
||||
| Operation | Required completion meaning |
|
||||
| --- | --- |
|
||||
| Cached UI preference/read | Synchronous memory access is allowed; cache ownership and invalidation are explicit |
|
||||
| Existing coalesced local saves | Preserve current flush semantics and expose the existing flush barrier to dependent reads/exit |
|
||||
| Remote event batch | Persist event/checkpoint under existing durability, then notify/ACK; enqueueing is not commit |
|
||||
| Channel outbox / runtime ledger | Preserve duplicate-dispatch prevention and uncertain-outcome handling; no blind retry after lost replies |
|
||||
| Review publication | Facts, result status and checkpoints commit together or roll back together |
|
||||
|
||||
Abort removes queued work before dispatch. In-flight work reaches a defined safe
|
||||
point and reports its actual outcome; capacity is released only after settlement or
|
||||
confirmed host exit. Caller cancellation, operation termination and resource release
|
||||
are distinct observations, not three new persistent product states.
|
||||
|
||||
`worker_threads` SharedArrayBuffer cancellation must not be assumed to cross
|
||||
`utilityProcess` IPC. Reuse it only within verified thread boundaries. A message
|
||||
cannot interrupt synchronous SQL while the receiver is blocked. Keep SQL/transaction
|
||||
units bounded, check cancellation between safe units, and report a completed commit
|
||||
as committed even if cancellation arrived too late. Do not terminate a shared storage
|
||||
process to cancel one request or promise instantaneous rollback of a committed write.
|
||||
|
||||
Filesystem adoption and SQL references are not automatically one atomic transaction.
|
||||
Preserve existing staging, adoption and reconciliation order with async file APIs.
|
||||
Do not delete originals or release protected in-flight files early; do not introduce
|
||||
a cross-database transaction engine or recovery journal.
|
||||
|
||||
## Lifecycle and failure
|
||||
|
||||
Main starts and supervises one storage process using existing utility-process
|
||||
patterns. Process spawn is not readiness: required migrations, connection opening
|
||||
and startup recovery must finish before storage-dependent operations are accepted.
|
||||
Show the existing startup/upgrade progress and retry UI. Render the application shell
|
||||
when safe without marking unavailable data as an empty successful result.
|
||||
|
||||
After process loss, reject affected pending requests with bounded diagnostics, stop
|
||||
admitting writes, confirm exit and close transport resources before replacement.
|
||||
The existing user retry/reopen lifecycle can re-establish storage readiness. Read-only
|
||||
work can be repeated when requested. For writes without a response, consult existing
|
||||
task/event/result identities and deduplication before claiming failure or retrying;
|
||||
model dispatch marked uncertain must not be sent again automatically. No Main SQL
|
||||
fallback, silent empty database, second durable receipt ledger or unbounded restart loop.
|
||||
|
||||
Shutdown stops producers and new admission, cancels unnecessary reads, drains required
|
||||
writes, closes readers/checkpoints/connections, and observes storage process exit.
|
||||
Extend the existing phased shutdown and deadline, including explicit storage failure
|
||||
reporting; existing settled-error helpers alone do not prove a successful drain. Native client ledger
|
||||
directories are removed only after their connection is closed. Upgrade/maintenance
|
||||
requires exclusive ownership before changing files, following the existing maintenance
|
||||
path; it cannot open an extra Main business connection.
|
||||
|
||||
## Output and temporary resource ownership
|
||||
|
||||
This work is included in the same architecture delivery. The [source investigation](./runtime-storage.md#output-and-temporary-file-investigation)
|
||||
records current producers and persistence gaps. Scope is GoodBuddy-created material;
|
||||
third-party MCP/Runtime internals and user workspaces are not cleanup targets.
|
||||
|
||||
Decide content ownership before choosing a temporary directory. Ordinary requests,
|
||||
responses, concatenation and short outputs stay in bounded memory. Where an existing
|
||||
record or artifact already contains complete content, pass its reference and read pages
|
||||
from that owner. Do not write a temporary copy merely to cross IPC or call MCP.
|
||||
|
||||
Current tool events often contain only previews. Preserve full paging access by adapting
|
||||
the existing `PagedOutputStore` interface to the storage/attachment owner: output that
|
||||
must outlive bounded in-memory capture has one backing content object, with events
|
||||
carrying previews and its reference. Write directly to that backing object; do not spool
|
||||
to Temp and later copy the same bytes to another full-output store. Reuse existing file
|
||||
adoption/reference/release operations and paged reads, without a second output database,
|
||||
generic blob service or durable transport journal. Final answers, previews and full
|
||||
stdout/stderr remain distinct data.
|
||||
|
||||
The storage object's retention follows its consumer. Call-only output is released when
|
||||
its existing owner no longer needs paging; content referenced by persisted history is
|
||||
released through that history's existing reference cleanup. Being stored by the storage
|
||||
service does not make every intermediate output permanent. Keep small call-only output
|
||||
in memory and preserve existing model limits. Avoid adding whole-output copies to events
|
||||
or retaining all concurrent streams in an unbounded buffer. Size alone does not justify
|
||||
an additional temporary copy.
|
||||
|
||||
The replacement must specify the complete-content reference and byte cursor through
|
||||
process, search, subagent and `output_read` paths before removing the old file backend.
|
||||
Inspect remote mapping separately: it does not use the local process spool and cannot
|
||||
recover omitted bytes from already-pruned Agent events. Preserve supported remote
|
||||
content/ACK semantics; this work does not promise new full-history capture from third-party
|
||||
runtimes. Remove the old spool creation and synchronous Main paging only once the shared
|
||||
replacement preserves the existing full-output contract.
|
||||
|
||||
### Necessary temporary material
|
||||
|
||||
External CLI configuration, plugin modules and atomic replacement/install staging may
|
||||
still require files. Reuse environment/in-memory configuration when already supported;
|
||||
otherwise keep the actual external contract. Atomic sibling files and installer staging
|
||||
stay on their required filesystem under their existing owner, not a forced central root.
|
||||
|
||||
For disposable runtime launch material, pass a root from application initialization
|
||||
through existing constructors; create it lazily:
|
||||
|
||||
```text
|
||||
<userData>/temp/goodbuddy-runtime/<appVersion>/<runId>/
|
||||
goodbuddy-runtime-launch/
|
||||
```
|
||||
|
||||
Add purpose directories only for evidenced necessary files. Do not precreate process,
|
||||
search or subagent output trees. User-visible disposable roots retain `goodbuddy-`
|
||||
prefixes. Tests use `<repository>/temp/goodbuddy-<task>-<runId>/`; remote resources use
|
||||
GoodBuddy-owned prefixed roots on the Host. Do not modify global `TEMP` or relocate
|
||||
third-party histories/caches through an indiscriminate environment override.
|
||||
|
||||
The creator removes partial creations on failure. Existing service shutdown closes
|
||||
children/handles and then releases its resources. Startup, including the next upgrade,
|
||||
reclaims obsolete owned run directories after the existing single-instance check and
|
||||
verification that no owning child remains active. Version/run IDs identify candidates;
|
||||
they do not establish inactivity. Reuse process ownership checks, with only the minimal
|
||||
owner information needed to identify surviving children; do not create a lease service.
|
||||
Cleanup runs in bounded background work and cannot delay the first screen with a full
|
||||
filesystem scan. Failed deletion leaves the known directory available to retry.
|
||||
|
||||
Legacy OS-Temp prefixes require an explicit list of layouts, creator and inactivity
|
||||
evidence. Reclaim confirmed obsolete instances, preserve ambiguous/live directories,
|
||||
and report the remaining scope. Never delete arbitrary `goodbuddy-*`, `.tmp` or `.partial`
|
||||
paths by age. Existing installer/reference reconciliation retains its own roots and
|
||||
rules; no global garbage collector, cleanup ledger or user-data migration is introduced.
|
||||
|
||||
## KISS constraints
|
||||
|
||||
- Keep Electron, TypeScript, SQLite and existing provider/runtime integrations.
|
||||
- Add the one storage utility process, not processes per project/session/subtask.
|
||||
- Reuse reader, checkpoint, parser, batching and lifecycle mechanisms where appropriate.
|
||||
- Keep credentials/OS integration in Main; storage receives only required business data.
|
||||
- Do not add general schedulers, distributed transactions, durable RPC journals,
|
||||
compatibility readers for unshipped intermediates, new approval gates or user modes.
|
||||
- Synchronous cache reads and pure calculations need not become RPC calls.
|
||||
- The runtime host, database driver and UI framework are not replaced as incidental work.
|
||||
|
||||
## VS Code reference
|
||||
|
||||
VS Code demonstrates service separation, asynchronous execution, lazy activation,
|
||||
cached state and coordinated close. Its application/profile/workspace storage is
|
||||
owned by Electron Main, uses async `@vscode/sqlite3`, and batches cached key/value
|
||||
writes. GoodBuddy retains `DatabaseSync` in a dedicated execution environment because
|
||||
its relational history, graph queries and durable event acknowledgements differ.
|
||||
Do not copy full-state memory loading or empty-store recovery into business history.
|
||||
|
||||
Sources reviewed on 2026-10-05, upstream main; no same-machine benchmark is claimed:
|
||||
|
||||
- [Connection ownership](https://github.com/microsoft/vscode/blob/main/src/vs/platform/storage/electron-main/storageMain.ts)
|
||||
- [SQLite backend](https://github.com/microsoft/vscode/blob/main/src/vs/base/parts/storage/node/storage.ts)
|
||||
- [Cache and flush lifecycle](https://github.com/microsoft/vscode/blob/main/src/vs/base/parts/storage/common/storage.ts)
|
||||
- [Extension hosts](https://code.visualstudio.com/api/advanced-topics/extension-host)
|
||||
|
||||
## Current implementation boundary
|
||||
## One-delivery completion
|
||||
|
||||
The immediate work reuses existing workers, narrows Supervisor loading and payloads,
|
||||
optimizes measured queries, virtualizes lists, and removes repeated digest/snapshot
|
||||
work. It does not add the utility process, convert every storage interface, or claim
|
||||
exclusive ownership of assistant.sqlite. Read-only workers plus remaining Main writes
|
||||
are an explicit transition state.
|
||||
SA-01 through SA-08 must pass together under the [acceptance matrix](../quality/concurrent-desktop-acceptance.md).
|
||||
All listed desktop owners, normal/background/error/cancel/exit callers, and native
|
||||
client ledger paths must use the final boundary. Remove the superseded Main openings,
|
||||
direct storage access and synchronous fallbacks, including helper-based bypasses.
|
||||
No feature-disable workaround or loss of user interaction counts as completion.
|
||||
|
||||
Full migration should proceed through verified domain operations and converge by
|
||||
database ownership. Do not leave undocumented competing write paths. Completion
|
||||
requires no Main-owned desktop business SQL, transactions wholly inside the owner,
|
||||
responsive reads/writes under heavy work, and validated startup/cancellation/shutdown.
|
||||
Run production UI-to-storage tests, existing full validators, old-data integrity and
|
||||
concurrent before/after measurements; validate affected desktop-to-Agent behavior on
|
||||
the shared Linux x64 Host. Native platform CI verifies process startup, SQLite and
|
||||
packaged entry/resource paths. Follow the release runbook for production builds and
|
||||
packaged-app checks, including measurement runs.
|
||||
|
||||
The [local implementation record](../review/kiss-local-fixes-2026-10-05.md) owns this
|
||||
round's scope and measurements. The [performance principles](./performance-principles.md)
|
||||
remain the acceptance criteria; regression ceilings are not equivalent to those budgets.
|
||||
The acceptance record must report latency, throughput, backlog and resource cleanup,
|
||||
including failures. A responsive UI with stalled background commits is not success.
|
||||
No target implementation, new performance baseline or migration completion is claimed
|
||||
by this documentation update.
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
# Desktop File Storage Integration (S04)
|
||||
|
||||
Implemented domain operations, async consumers and production composition on
|
||||
2026-10-05. The storage owner is registered by the foundation and the current Main
|
||||
composition passes its file facades to ContextManager, document parsing and model
|
||||
output. This does not claim that the full application has passed concurrent or
|
||||
packaged acceptance.
|
||||
The governing constraints are [performance principles](./performance-principles.md)
|
||||
and [storage ownership](./desktop-storage-direction.md).
|
||||
|
||||
## Foundation API
|
||||
|
||||
`src/main/desktop-storage-files.ts` exports the complete integration surface:
|
||||
|
||||
- `DesktopFileStorageDomains`: intersect into `DesktopStorageDomains`. The domains
|
||||
are `attachments` and `documentResults`.
|
||||
- `attachmentStorageMethods` and `documentResultStorageMethods`: explicit allowed
|
||||
operations. `attachments.reconcile` is an additional owner composite, with
|
||||
arguments `[startup?, protectedIds?]`; its callback never crosses IPC.
|
||||
- `openDesktopStorageFiles(userDataRoot, hasOwner, persistentResultLookup?)`:
|
||||
construct once inside the existing utility process, after assistant and knowledge
|
||||
databases open. Pass the existing userData root, not the assistant DB directory.
|
||||
Add that root to foundation options and forward it in the client's open message.
|
||||
- `DesktopStorageFilesOwner.dispatch(domain, method, args, signal)`: dispatch the
|
||||
two domains here from the existing owner's dispatcher. It normalizes transported
|
||||
original bytes, checks operation allowlists, and supplies cancellation to result
|
||||
saves. Keep the existing transport admission and result limits.
|
||||
- Before readiness, run `files.attachments.reconcile(hasOwner, true)`. The callback
|
||||
uses the already-open assistant owner. It checks released references, marks
|
||||
interrupted parsing, removes stale call-only output refs, and reclaims owned
|
||||
orphan UUID directories. Do not run it concurrently with file producers.
|
||||
- After draining admitted requests, `await files.close()` before closing assistant
|
||||
and knowledge. Close drops call-only output refs, preserves history refs, removes
|
||||
temporary document results, and closes the existing attachment database.
|
||||
|
||||
No new database, RPC transport, SQL endpoint, or process is introduced. Existing
|
||||
attachment reference transactions and schema 1 remain in place. New relative paths
|
||||
use slash separators; reads and reconciliation also accept released Windows paths.
|
||||
|
||||
## Main And Knowledge
|
||||
|
||||
After `storage.ready`, production composition uses `createDesktopStorageFiles(storage)` to obtain
|
||||
`{ attachments, results, backingStore }`. Pass attachments to `ContextManager` and
|
||||
results to `DocumentParsingService` and Knowledge's result-access dependency.
|
||||
Do not also construct `ConversationAttachmentStorage` or `DocumentResultStorage`
|
||||
in Main. Facades only call the existing storage client; importing them does not open
|
||||
SQLite. Their result `close()` is intentionally inert: shared result shutdown belongs
|
||||
to `DesktopStorageFilesOwner.close()`, after all producers drain.
|
||||
|
||||
Context operations that now require await: `storePastedImage`, `enrichRequest`,
|
||||
`remove`, `serializeForQueue`, `restoreFromQueue`, `getDraft`, `hasImageInputs`,
|
||||
`validateForSend`, `saveDraft`, `sendOriginal`, and `cancelUnavailableImport`.
|
||||
Previously async import/reparse/image operations now await every storage operation.
|
||||
Conversation/target validators may return Promises. Memory-only `clear` and
|
||||
`activeContextIds` remain synchronous. `ContextManager.dispose()` cancels and drains
|
||||
the active import before collecting unreferenced assets. Stop other context producers
|
||||
and await their operations before disposing. Parsing-service disposal drains all
|
||||
diagnostics, including those without an operation ID.
|
||||
|
||||
IPC must await attachment/result calls, including `results.original(id)` before
|
||||
`shell.openPath`, and array predicates/maps after `get`, `draft`, `pendingParsing`,
|
||||
and `getDraft`. Replace callback-based Main reconciliation with
|
||||
`await attachments.reconcile(false, contextManager.activeContextIds())`.
|
||||
|
||||
Knowledge must accept `DocumentResultStorageAccess`, await `detach`, `original`,
|
||||
and `isTemporaryResult`, and remove Main `setPersistentLookup` registration. Supply
|
||||
that lookup to `openDesktopStorageFiles` instead: use owner-local
|
||||
`knowledge.getDocumentByParsedResultId(id)` and the existing
|
||||
`knowledge-assets/<libraryId>/<documentId>/<resultId>` directory and source location.
|
||||
Keep Knowledge's publish/move/detach/release order. Result release never deletes an
|
||||
external source original. No user-data directory migration is needed.
|
||||
|
||||
Project deletion supplies the project's conversation IDs captured before assistant
|
||||
deletion to `attachments.deleteProject(conversationIds, protectedIds?)`. It removes
|
||||
their refs in one attachment transaction, then collects only unreferenced assets.
|
||||
Shared assets remain available to other conversations. Call this after the assistant
|
||||
deletion commits; startup reconciliation covers an interrupted cleanup. Use
|
||||
`deleteConversation` for single conversations and reconciliation for full clearing.
|
||||
|
||||
## Full Output
|
||||
|
||||
Pass the returned `backingStore` to each existing `PagedOutputStore`. It implements
|
||||
`create/append/finish/read/release` directly in the existing attachment owner.
|
||||
An output handle's UUID identifies its attachment row throughout capture, promotion,
|
||||
history retention, restart, and deletion. Capture writes `original.txt` directly in
|
||||
the owned attachment directory. Promotion renames that directory; no temporary full
|
||||
copy or second backing is created. `request.json` contains only the descriptor.
|
||||
Appends are bounded to 64 KiB; backing reads allow 32 KiB plus UTF-8 lookahead.
|
||||
Small outputs remain in the existing bounded in-memory capture.
|
||||
|
||||
For history retention, await
|
||||
`attachments.outputAdopt(callOwnerId, handle, conversationId, kind, referenceId)`
|
||||
before the runtime releases its call reference. It returns the resource ID and adds
|
||||
the reference without replacing the owner's existing attachments. Persist the
|
||||
complete-output handle/reference in the owning event/message path; a preview alone
|
||||
is not full output. Retain the call until required history adoption/persistence has
|
||||
settled. Failed history publication uses existing release/reconciliation, not a
|
||||
second durable operation log. Incomplete captures cannot be adopted.
|
||||
|
||||
`release` drops only the matching `output-call` ref. Persisted history can page after
|
||||
call disposal or restart with
|
||||
`attachments.outputRead(conversationId, handle, cursor, length)`, reading the same
|
||||
bytes; it must use the owning conversation ID. History deletion uses the existing
|
||||
message/queue/conversation reference cleanup. Runtime cancellation must await writer
|
||||
abort/release. Startup removes stale call refs while retaining valid history refs.
|
||||
The remote Agent path does not use this desktop backing and is unchanged.
|
||||
|
||||
## Validation Boundary
|
||||
|
||||
The focused suites exercise real SQLite/files through owner operations and async
|
||||
facades, using structured-cloned arguments/results to cover Buffer-to-Uint8Array
|
||||
conversion. They cover complete UTF-8 paging, one backing file, adoption, release,
|
||||
abort, restart, project deletion with shared history, existing path separators,
|
||||
temporary-result cleanup, failed adoption retry, async restore, and shutdown drain.
|
||||
New result writes and copies omit `parsed.md`; old manifests and directories remain
|
||||
readable. Result image writes settle sequentially before failure cleanup.
|
||||
|
||||
Local Windows validation: six focused suites passed (83 tests), covering this owner,
|
||||
attachment references, ContextManager, parsing service, paged output, and HTTP OCR
|
||||
integration. Scoped ESLint and diff whitespace checks passed. Node type checking
|
||||
reported no diagnostics in S04 source/tests; the complete checkout still has pending
|
||||
integration errors in Main/IPC and concurrent domain migrations.
|
||||
|
||||
These tests do not replace the foundation's utility-process tests or UI acceptance.
|
||||
The real utility-process coverage for these domains is present in the focused
|
||||
storage fixtures. Concurrent desktop acceptance and packaged validation remain
|
||||
outstanding. No latency or throughput improvement is claimed from these focused
|
||||
tests. Test resources live under `temp/goodbuddy-files-upgrade` and individual
|
||||
fixtures remove their files.
|
||||
@@ -0,0 +1,308 @@
|
||||
# Desktop Storage Foundation Integration
|
||||
|
||||
Implemented 2026-10-05. This is the storage foundation for
|
||||
[the desktop storage direction](./desktop-storage-direction.md). The production Main
|
||||
composition now opens one `DesktopStorageClient` and supplies its typed facades to
|
||||
the migrated domains. Full application acceptance is still incomplete. Do not open
|
||||
the existing Main databases alongside the owner.
|
||||
|
||||
## Client API
|
||||
|
||||
Import `DesktopStorageClient` from `src/main/desktop-storage-client.ts` in Main.
|
||||
Construct it after Electron readiness and the application's single-instance check.
|
||||
The existing database directories must exist. Preserve the current database paths.
|
||||
|
||||
```ts
|
||||
const storage = new DesktopStorageClient({
|
||||
assistantPath,
|
||||
knowledgePath,
|
||||
defaultRootPath,
|
||||
userDataPath,
|
||||
onProgress: progress => publishExistingUpgradeProgress(progress),
|
||||
onChanged: domain => publishExistingDomainNotification(domain),
|
||||
onFailure: error => reportStorageFailure(error)
|
||||
})
|
||||
await storage.ready
|
||||
|
||||
await storage.call('assistant', 'saveLocalConversations', [changes])
|
||||
const conversation = await storage.call(
|
||||
'assistant', 'getConversation', [conversationId], { signal }
|
||||
)
|
||||
const results = await storage.call('knowledge', 'search', [searchOptions], { signal })
|
||||
|
||||
// After stopping producers and awaiting their persistence flushes:
|
||||
await storage.close()
|
||||
```
|
||||
|
||||
`call(domain, method, argumentTuple, { signal? })` infers argument and result types
|
||||
from the explicit allowlists in `desktop-storage-contracts.ts`. All results are
|
||||
Promises. Pass cancellation as the transport option, not inside a DTO. Functions,
|
||||
signals and live class instances are rejected before posting. Maps and Uint8Arrays
|
||||
are supported; Node Buffers arrive as Uint8Arrays across Electron IPC.
|
||||
|
||||
`userDataPath` is the existing application user-data root, not a database directory
|
||||
inferred from a filename. `ready` means upgrade worker exit, database initialization,
|
||||
attachment startup reconciliation and runtime-binding owner initialization completed.
|
||||
Calls before readiness fail without queueing. `retry()` retains the same public
|
||||
client and facades. A live failed startup retries in that process; a failed/exited
|
||||
utility is replaced only after exit confirmation and old-request settlement. The
|
||||
replacement gets fresh request correlation, never the old writes. The confirmed
|
||||
upgrade decision is acknowledged into Main memory before upgrade execution, retained
|
||||
across startup loss, and cleared when that upgrade completes. Wire `retry()` to the
|
||||
existing retry UI. There is no automatic restart or replay of unconfirmed writes.
|
||||
|
||||
The defaults locate `desktop-storage-entry.js`, `readonly-query-worker.js` and
|
||||
`assistant-storage-worker.js` as packaged siblings. `entryPath`, `readerWorkerPath`
|
||||
and `upgradeWorkerPath` overrides support isolated tests and explicit composition.
|
||||
The new entry is included in `electron.vite.config.ts`.
|
||||
|
||||
## Owner-Side Operations
|
||||
|
||||
| Existing call shape | Storage operation |
|
||||
| --- | --- |
|
||||
| Assistant/Knowledge ordinary database methods | `call('assistant' / 'knowledge', method, args)` using the explicit allowlists |
|
||||
| SSH project create/update | `assistant.createSshProject` / `assistant.updateSshProject` accept `Omit<SshProjectWrite, 'assertCurrent'>`; Main checks the live host/runtime before dispatch, and the owner retains cancellation and project-revision checks inside the existing transaction |
|
||||
| `activityHistory()` live repository | Existing `getActivityHistoryPage`, `getActivityHistorySummary`, `updateActivityHistory`, `replaceActivityHistory`, `clearActivityHistory`, `reconcileActivityHistory` operations |
|
||||
| `supervisionReviewStore()` | `call('review', method, args)`; initialization/resume execute on the write owner, not the old independent writer |
|
||||
| `supervisionStories()` | `call('stories', method, args)` |
|
||||
| `supervisionExperiences()` | `call('experiences', method, args)` |
|
||||
| `supervisionSuggestions()` | `call('suggestions', method, args)`; resolution uses `assistant.resolveSupervisionSuggestion` |
|
||||
| Knowledge `externalStore` | `call('external', method, args)` |
|
||||
| `readSnapshot(callback)` for review context | `call('assistant', 'supervisionContext', [request])`; summary/background share one reader snapshot |
|
||||
| Knowledge `publishDocument(..., { afterChunksInserted })` | `call('knowledge', 'publishDocument', [input, chunks, { graph, embeddingReplacement, embeddingError }])` |
|
||||
| Knowledge graph-only replacement | `call('knowledge', 'replaceDocumentGraph', [library, document, graph])`; reuses the existing evidence replacement transaction |
|
||||
| Attachment references and full-output backing | `call('attachments', method, args)`; includes owner-local `reconcile(startup?, protectedIds?)` |
|
||||
| Temporary and persisted document results | `call('documentResults', method, args)` |
|
||||
| Runtime bindings and native model-call ledgers | `call('runtime', method, args)` |
|
||||
|
||||
Repository instances never leave the owner. The publication graph DTO is the
|
||||
existing `GraphExtractionResult`. The owner creates the graph callback and performs
|
||||
entity identity resolution, evidence insertion, chunk/embedding publication and
|
||||
pruning within `KnowledgeDatabase.publishDocument`'s existing transaction. Replace
|
||||
the consumer's publication callback with this DTO; remove its duplicate graph
|
||||
publication path when migrating that consumer. Extraction/model work remains outside
|
||||
storage. Publication and replacement share `storeExtractedGraph` from
|
||||
`desktop-storage-knowledge-operations.ts`. Callback-bearing `replaceEvidenceForDocument`
|
||||
is excluded from the transport allowlist. File adoption/release and task-status
|
||||
orchestration remain outside this SQLite transaction.
|
||||
|
||||
After readiness, `createDesktopStorageFiles(storage)` supplies attachments, results
|
||||
and output backing; `createDesktopRuntimeStorageAdapters((domain, method, args) =>
|
||||
storage.call(domain, method, args))` supplies runtime bindings and ledger access.
|
||||
The facade caller types now share the core DTO types, so these calls need no casts.
|
||||
`StorageData` preserves the existing recursive `JsonValue` as-is rather than
|
||||
recursively rebuilding it; runtime validation still rejects callbacks and handles.
|
||||
|
||||
The owner supplies both file callbacks. Attachment reconciliation checks
|
||||
`assistant.hasAttachmentOwner`. Persistent document lookup reads
|
||||
`knowledge.getDocumentByParsedResultId` and resolves the existing
|
||||
`<userDataPath>/knowledge-assets/<libraryId>/<documentId>/<resultId>` directory and
|
||||
source location. Main must not register another persistent lookup on the results
|
||||
facade. The existing `onProgress`, `onChanged` and `onFailure` APIs are unchanged,
|
||||
including `createAssistantStorageOnChanged`'s four notification mappings.
|
||||
|
||||
Keep these composites intact during integration:
|
||||
|
||||
- Remote task/conversation batches use the existing `appendRemote*Batch` or
|
||||
`appendRemote*Once` methods. Await commit before forwarding or ACK. Never translate
|
||||
a batch into independently acknowledged per-event writes.
|
||||
- Review publication uses `assistant.saveSupervisionResult`, which owns result facts
|
||||
and checkpoint commits. `commitCheckpoints` is deliberately not an external review
|
||||
operation. No callback-based transaction API is exposed.
|
||||
- Story/experience apply and review batch save retain their existing repository
|
||||
transactions. A model request must finish before calling these operations.
|
||||
- Additional multi-statement snapshots need a named owner/reader handler; do not
|
||||
recreate `readSnapshot` as several awaited Main calls.
|
||||
|
||||
## Bounds And Lifecycle
|
||||
|
||||
The utility adapter retains `ReadonlyQueryReader` request correlation, errors and
|
||||
admission. Its existing request limit is 32, with 32 MiB of argument admission
|
||||
credit. This is no longer a content-size restriction: one oversized complete input
|
||||
uses the entire credit budget exclusively. Smaller concurrent inputs still share
|
||||
the budget; saturation returns `STORAGE_CAPACITY` without dropping work.
|
||||
|
||||
`desktop-storage-wire.ts` transfers complete values on the same channel in frames
|
||||
of at most 256 KiB accounted data. Each direction sends one frame and waits for its
|
||||
acknowledgement before constructing the next. Strings are split without assembling
|
||||
a whole JSON/serialized copy; binary frames copy only their slice and fill the
|
||||
receiver's final buffer. The receiver exposes a value only after its last frame.
|
||||
These frame acknowledgements provide transport backpressure, not durable write ACKs.
|
||||
Cancellation bypasses the frame queue and is retained for an incompletely received
|
||||
request until dispatch can reject it safely.
|
||||
|
||||
Known complete-note/history/result reads share one materialization/transfer slot,
|
||||
so queued requests do not build many huge reply objects while another reply drains.
|
||||
Writes and bounded queries retain ordinary admission. A complete user value itself
|
||||
still needs memory proportional to its size; the frame budget is not a claim that
|
||||
total process memory or every supported history fits in 32 MiB. There is no result
|
||||
size rejection or truncation, and no `STORAGE_RESULT_CAPACITY` outcome after a commit.
|
||||
|
||||
Renderer conversation persistence now targets 8 MiB batches as well as the existing
|
||||
100-conversation bound. A single oversized conversation stays complete and uses the
|
||||
framed transport. Quit flushing drains all remaining batches and propagates a failed
|
||||
write. Oversized single histories no longer fail solely because of the transport cap.
|
||||
`STORAGE_UNCONFIRMED` means the process exited without confirming a pending operation;
|
||||
reconcile using existing domain identities, not blind retries.
|
||||
|
||||
Queued cancellation is checked before dispatch. In-flight reader cancellation uses
|
||||
the existing thread SharedArrayBuffer inside the owner; that buffer is never sent
|
||||
over utility-process IPC. Synchronous commits cannot be interrupted by an incoming
|
||||
message, so a completed commit returns success even when cancellation was requested
|
||||
too late. Request capacity remains occupied until settlement.
|
||||
|
||||
`close()` stops client admission, drains admitted requests, awaits file-owner cleanup,
|
||||
closes runtime ledgers/bindings and read workers, awaits the existing PASSIVE
|
||||
checkpoint worker's exit, closes both database owners
|
||||
and observes utility-process exit. An exit without the close acknowledgement rejects
|
||||
close. Apply the application's existing shutdown deadline to the whole operation;
|
||||
the transport separately reports an unconfirmed exit after its 10-second exit wait.
|
||||
Startup failure follows the same release barriers. A failed close attempts the
|
||||
remaining independent closes, reports `STORAGE_CLOSE_FAILED`, and exits unsuccessfully;
|
||||
it does not acknowledge a clean drain or permit a second owner to open in that host.
|
||||
|
||||
## Remaining Integration And Acceptance
|
||||
|
||||
### Supervision And Heartbeat Callers
|
||||
|
||||
`createSupervisionDomainPorts(storage)` in
|
||||
`src/main/assistant/supervision-domain-ports.ts` supplies Main-side RPC facades:
|
||||
|
||||
```ts
|
||||
const ports = createSupervisionDomainPorts(storage)
|
||||
const supervisor = createProductionSupervisorService(ports.supervision, getSettings, resolveRuntime, pool, persistUsage)
|
||||
const phrase = createProductionSuggestionPhraser(ports.supervision, getSettings, resolveRuntime, pool, persistUsage)
|
||||
const heartbeat = new HeartbeatService(ports.heartbeat, actions)
|
||||
await deriveSuggestions(ports.suggestions, phrase, input)
|
||||
```
|
||||
|
||||
The production factory's other arguments are unchanged. If provided, `persistUsage`
|
||||
must return the persistence Promise. All heartbeat methods, including configuration
|
||||
CRUD and history, now return Promises. Review repository factories may return a
|
||||
Promise of the typed facade. The facades contain closures only; no database or live
|
||||
repository is transported. `initializeSupervisionReview` and `resumeSupervisionReview`
|
||||
route to `review.initialize` / `review.resume`, with signals in transport options.
|
||||
The caller does not invoke the old independent supervision writer.
|
||||
|
||||
`story-assignment-service.ts` and `experience-extraction-service.ts` are the sole
|
||||
story/experience model orchestrators, including their prompts, output schemas and
|
||||
model-facing types. Production, tests and live scripts import those services.
|
||||
The repository files retain their SQL methods and transactions, with type-only
|
||||
imports for validated model output. `supervision-suggester.ts` awaits candidate
|
||||
selection and suggestion publication. The heartbeat live script also awaits plan
|
||||
creation before using its ID.
|
||||
|
||||
Callers await leaf/navigation writes, final publication, task usage/status,
|
||||
heartbeat projections, story/experience results and error bookkeeping. Cancellation
|
||||
aborts model work immediately, but review admission remains occupied until the
|
||||
cancel write and active execution settle. A late rejected cancel does not overturn
|
||||
a confirmed publication. Failure to persist a suggestion failure is propagated;
|
||||
a model phrasing failure whose status was saved preserves the completed review.
|
||||
|
||||
Validation includes unchanged behavioral assertions in the existing review/story/
|
||||
experience suites, delayed-commit tests, and `supervision-storage.test.ts`: a real
|
||||
Electron storage utility process runs heartbeat, two review leaves, navigation,
|
||||
stories, experiences, suggestion publication, usage, a no-change heartbeat and
|
||||
cancellation using a local deterministic Runtime. This does not validate final
|
||||
Main/IPC composition or concurrent latency budgets.
|
||||
|
||||
On 2026-10-05, after removing the duplicate orchestrators, the targeted supervision/
|
||||
heartbeat run passed 215 tests across 21 files, including the real utility-process
|
||||
test. Scoped ESLint and Node type checking passed.
|
||||
|
||||
The bounded `scripts/supervision-storage-live.cjs <runtime-settings.json>` probe
|
||||
used the current supervisor profile (`gpt-6-astra`, OpenAI Responses) and production
|
||||
utility storage with three synthetic calibration messages. Exactly four actual
|
||||
HTTP requests completed, all HTTP 200: review, story assignment, experience
|
||||
extraction and suggestion phrasing. It committed one leaf covering all three
|
||||
sources (589 characters), assigned three events to one story, created one experience
|
||||
and published two suggestions. The unchanged review made zero additional requests.
|
||||
Persisted usage was four calls, 3,495 input and 1,251 output tokens. Two initial
|
||||
preflight attempts sent zero requests; the probe was corrected to install the
|
||||
isolated Electron encryption state before readiness. Current settings remained
|
||||
byte-identical; credentials were not logged, and isolated databases, encryption
|
||||
state and bundles were removed. This is functional evidence, not concurrent
|
||||
performance acceptance.
|
||||
|
||||
### Integrated Owner
|
||||
|
||||
The foundation registers assistant, knowledge, external metadata, supervision,
|
||||
attachments, document results and runtime operations without Main SQL fallback. The
|
||||
production composition in `src/main/index.ts` constructs the Assistant, Knowledge,
|
||||
file and Runtime facades after storage readiness and passes them to the relevant
|
||||
Main services. Remote Agent database ownership is unchanged.
|
||||
|
||||
The remaining work is product-level acceptance: complete C01-C07/A01-A10 coverage,
|
||||
packaged cross-platform checks, and removal or verification of any caller that is
|
||||
outside the migrated desktop domains. Focused domain tests do not establish those
|
||||
results.
|
||||
|
||||
The S05 runtime-domain owner, Main adapters, required composition wiring and focused
|
||||
validation are documented in [Runtime Storage Integration](./desktop-storage-runtime-integration.md).
|
||||
|
||||
Supervision now uses `ReadonlyQueryReader.reviewManifest(runId)`. The existing
|
||||
read-only assistant worker scans once under a read snapshot into connection-local
|
||||
temporary metadata, ends the snapshot, and returns at most 200 metadata rows per
|
||||
page. It holds at most one live scan manifest, rejects a competing scan explicitly,
|
||||
and releases that temporary data in the operation's settlement path. It never
|
||||
opens a business-write connection.
|
||||
|
||||
The sole write owner clears/rebuilds disposable manifest rows in bounded batches,
|
||||
commits each insertion batch atomically, and yields between commits. Cancellation
|
||||
and persisted pause/cancel status are checked between batches. `initializing`
|
||||
remains set after interruption; clearing it and deleted-source cleanup share the
|
||||
final transaction. Resume rebuilds partial manifests, or validates existing frozen
|
||||
revisions in read-worker pages of at most 200 rows, yielding between pages. A failed
|
||||
scan never falls back to synchronous owner scanning. Final result/checkpoint
|
||||
publication remains the existing atomic operation.
|
||||
|
||||
The old writable `supervision` worker mode, connection opener and separate reader
|
||||
field were removed. `AssistantDatabase`'s direct initialization/resume methods use
|
||||
the same bounded implementation; in-memory owner-local use retains its synchronous
|
||||
algorithm because a separate read connection cannot share that database. Other
|
||||
non-worker operations still execute synchronously inside storage; no latency
|
||||
improvement or full concurrent performance acceptance is claimed.
|
||||
|
||||
The current focused validation passes 70 tests across eleven suites: core storage,
|
||||
file/runtime domains, assistant facade, read workers, bounded review-store algorithms,
|
||||
existing supervision worker regressions, startup, wire backpressure, renderer batching
|
||||
and real supervision/storage orchestration.
|
||||
Real Electron coverage includes graph replacement rollback, SSH revision conflicts,
|
||||
owner-side persistent document lookup after reopen, full-output retention and crash
|
||||
reconciliation, ledger recovery, runtime startup failure/retry and pending file-save
|
||||
drain before cleanup. Review tests cover 200-row bounds, concurrent owner writes,
|
||||
frozen scan revisions, cancellation, failed-batch rollback, reader loss, marker
|
||||
recovery, deleted sources and final marker/cleanup atomicity.
|
||||
|
||||
Real Electron large-value tests save an explicitly typed 16 MiB MP4 payload, read a
|
||||
complete two-entry note, and round-trip all 25,600,230 characters of an 80-message
|
||||
history. Hashes verify complete video and message bodies. The measured accounted
|
||||
frame high-water marks were 248,332 bytes in Main and 252,578 bytes in storage, below
|
||||
262,144 bytes; argument credit peaked at 33,554,432 bytes. Main RSS peaked at
|
||||
426,049,536 bytes including fixture inputs, complete results and application memory.
|
||||
This is queue-bound evidence, not a whole-process memory or latency budget claim.
|
||||
The real process tests also kill an in-progress write transfer and confirm no replay
|
||||
after retry, and verify startup decision retention/clearing across process loss.
|
||||
|
||||
Full Node, Agent and renderer typechecking passes, as does scoped ESLint. Temporary checks and databases use
|
||||
`temp/goodbuddy-storage-foundation`; tests remove their generated bundles/data.
|
||||
The earlier foundation production-build check predates this domain integration.
|
||||
Packaged cross-platform and full UI-to-storage concurrent acceptance remain outstanding.
|
||||
|
||||
### Reader Ownership Cleanup
|
||||
|
||||
The obsolete `readWithFallback` export, database-owned readonly-reader fields and
|
||||
checkpoint lifecycle methods, and the database async wrappers were removed. Heavy
|
||||
reads are dispatched by `DesktopStorageOwner`; owner-local synchronous reads remain
|
||||
available inside transactions and focused database tests. Read-worker and checkpoint
|
||||
tests now create explicit `ReadonlyQueryReader`/`Worker` owners and await their close.
|
||||
The three perf harnesses create explicit owner-local reader/checkpoint resources under
|
||||
the repository temporary root, report `executionBoundary: owner-local`, and close
|
||||
those resources before deleting their copies. No production or test source reference
|
||||
remains to the removed compatibility APIs.
|
||||
|
||||
The cleanup validation adds termination barriers and failed-frame poisoning: a reader
|
||||
cannot reopen capacity while its prior worker termination is pending, a transport
|
||||
posting failure cannot feed later values into a partial decoder, and close waits for
|
||||
wire drain and owner settlement. The focused cleanup run passes 122 tests; Node and
|
||||
renderer type checks and scoped lint pass.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Knowledge Main Storage Migration (S03)
|
||||
|
||||
Knowledge Main now receives `KnowledgeServiceOptions.database`, a
|
||||
`KnowledgeStoragePort`. Production composition constructs it with
|
||||
`createKnowledgeStoragePort(storageClient)` after `storageClient.ready`.
|
||||
`KnowledgeService` no longer opens, initializes, closes, or starts readers for
|
||||
the business database. The application storage owner retains those duties.
|
||||
|
||||
The adapter sends explicit `knowledge` and `external` domain calls. Search
|
||||
signals travel as transport options, not serialized arguments. The port's
|
||||
`Awaitable` return type permits synchronous owner-local test implementations;
|
||||
the production adapter always returns the storage client's promises.
|
||||
|
||||
Main awaits source, document, task, graph, embedding and external metadata
|
||||
operations. IPC mutations await their writes before returning or refreshing a
|
||||
snapshot. Parsing, models, credentials and source watchers remain in their
|
||||
existing Main services. Embedding batches retain their existing sizes and
|
||||
staging/publication sequence.
|
||||
|
||||
Publication passes the extracted graph through foundation
|
||||
`publishDocument(..., { graph })`. Graph-only replacement calls
|
||||
`replaceDocumentGraph(library, document, result)`. Its implementation in
|
||||
`src/main/desktop-storage-knowledge-operations.ts` runs the existing synchronous
|
||||
graph algorithm inside `replaceEvidenceForDocument`'s transaction. No callback
|
||||
or SQL crosses the storage boundary.
|
||||
|
||||
Shutdown aborts producers immediately, drains tracked work and observes the
|
||||
task-interruption write and embedding status persistence. The service does not
|
||||
close the shared storage client. Embedding allocation returns its replacement
|
||||
ID even if cancellation arrives with the reply, allowing cleanup; a committed
|
||||
finish is not reported as rolled back. Rebuild admission remains occupied
|
||||
until its final status write settles, and status-write failures reach the
|
||||
completion barrier.
|
||||
|
||||
## Current Production Boundary
|
||||
|
||||
The following boundary is implemented in the current production composition:
|
||||
|
||||
- `replaceDocumentGraph(library: KnowledgeBase, document: Document,
|
||||
result: GraphExtractionResult): void` to the knowledge domain contract and
|
||||
dispatch it to the exported owner-local operation, passing the owned database
|
||||
as its first argument. It is not part of the concrete database method
|
||||
allowlist because it is a composite owner operation.
|
||||
- Foundation publication reuses the exported `storeExtractedGraph` helper.
|
||||
- `StorageData` preserves the existing recursive JSON type without rebuilding it
|
||||
in the adapter.
|
||||
- Persistent document lookup is resolved by the storage owner through the
|
||||
Knowledge domain; Main does not register a second lookup.
|
||||
- Application composition injects the port, awaits asynchronous gateway and
|
||||
tool-provider calls, and closes shared storage after Knowledge disposal.
|
||||
|
||||
## Validation
|
||||
|
||||
The focused Knowledge service, external service, embedding coordinator and
|
||||
embedding repository suites use a Promise-backed adapter over a test-owned
|
||||
SQLite connection. Coverage includes publication rollback, embedding batch
|
||||
ordering, late cancellation, replacement cleanup, external request cancellation,
|
||||
credential persistence and restart. The production gateway now awaits its library
|
||||
and search operations. Remaining evidence is limited to the focused suites and
|
||||
does not establish full concurrent desktop acceptance.
|
||||
|
||||
Owned production files pass ESLint. The current worktree typecheck and focused
|
||||
storage validation cover the injected production port and owner dispatch. Global
|
||||
architecture acceptance, packaged checks and concurrent performance acceptance
|
||||
remain with the parent delivery; these tests do not establish a new performance
|
||||
baseline.
|
||||
@@ -0,0 +1,96 @@
|
||||
# Desktop Runtime Storage Integration
|
||||
|
||||
S05 domain implementation and production composition, 2026-10-05. This supplies
|
||||
the runtime domain for the [desktop storage foundation](./desktop-storage-foundation.md).
|
||||
The current composition passes the storage-owned binding store and model-call
|
||||
ledger factory to Main runtime services. Full desktop migration and performance
|
||||
acceptance remain incomplete.
|
||||
|
||||
## Integration API
|
||||
|
||||
`src/main/desktop-storage-runtime-operations.ts` exports:
|
||||
|
||||
- `DesktopStorageRuntimeOperations`: the existing `RuntimeSessionBindingStore`
|
||||
interface plus named ledger lifecycle and transition operations.
|
||||
- `runtimeStorageMethods`: the explicit host dispatch allowlist.
|
||||
- `DesktopStorageRuntimeOwner.open(userDataPath)`: creates the sole binding owner
|
||||
at `remote-runtime-bindings.sqlite`. Await it before publishing storage readiness.
|
||||
Native ledgers open lazily at their existing paths through `openLedger(id, path)`.
|
||||
- `createDesktopRuntimeStorageAdapters(call)`: Main adapters returning
|
||||
`bindingStore` and `openModelCallLedger`. `call` has the same domain/method/tuple
|
||||
shape as `DesktopStorageClient.call`, restricted to the `runtime` domain.
|
||||
- `DesktopStorageRuntimeOwner.close()`: closes remaining native ledgers and the
|
||||
binding database. The outer storage owner must first drain admitted operations.
|
||||
This method is not part of the external domain allowlist.
|
||||
|
||||
The storage-core integration adds `runtime: DesktopStorageRuntimeOperations` to
|
||||
`DesktopStorageDomains`, initializes one runtime owner from the existing user-data
|
||||
directory, and dispatches allowed runtime methods against that owner. Include its
|
||||
close in the storage owner's release path, including startup failure. No new RPC
|
||||
client, SQL endpoint, admission queue or process is needed.
|
||||
|
||||
After registering the domain, Main composition is:
|
||||
|
||||
```ts
|
||||
await storage.ready
|
||||
const runtimeStorage = createDesktopRuntimeStorageAdapters(
|
||||
(domain, method, args) => storage.call(domain, method, args)
|
||||
)
|
||||
|
||||
// ManagedRemoteExecutionServices options:
|
||||
// bindingStore: runtimeStorage.bindingStore
|
||||
// The former userDataPath option is removed; the host owns that path.
|
||||
|
||||
// NativeClientCoordinator, NativeTerminalClient and NativeDshWebClientService options:
|
||||
// openModelCallLedger: runtimeStorage.openModelCallLedger
|
||||
```
|
||||
|
||||
`ManagedRemoteExecutionServices` borrows the binding interface and does not close
|
||||
it on disposal. Native clients stop their producers and proxy, await their ledger
|
||||
close, then delete launch material. DS Web retains native history. A failed or
|
||||
unconfirmed close retains files and propagates the error. A lost open reply also
|
||||
retains files because the host may still own the connection. Shutdown must stop
|
||||
native/remote producers before closing the shared storage client.
|
||||
|
||||
## Ordering And Ownership
|
||||
|
||||
The shared `ModelCallLedger` type allows synchronous or Promise-returning methods;
|
||||
the concrete `AgentModelCallLedger` stays synchronous and remains the remote
|
||||
Agent's owner. Schema, database names, call identities, retention and transaction
|
||||
algorithms are unchanged.
|
||||
|
||||
`AgentModelGateway` awaits claim before provider dispatch, complete before returning
|
||||
the response, delivered before resolving the delivery acknowledgement, and
|
||||
outcome-unknown before reporting provider failure. Cancellation while claim is
|
||||
pending does not dispatch after claim completes. Persistence rejection propagates;
|
||||
the gateway does not replay a provider request after an unconfirmed write.
|
||||
|
||||
## Validation
|
||||
|
||||
The final focused run passed 113 tests across eight files, with 11 existing opt-in
|
||||
cases skipped. It included the official DS Web process using standard Node,
|
||||
binding uniqueness/rotation/cursor persistence, concrete ledger reopen and uncertain
|
||||
outcomes, delayed/rejected persistence, and both native clients' close ordering.
|
||||
|
||||
`desktop-storage-runtime-operations.test.ts` also builds real Electron fixtures.
|
||||
The production native terminal loopback gateway uses Main adapters and the real
|
||||
`DesktopStorageClient`/transport to reach a utility-process runtime owner, then
|
||||
closes its ledger before removing launch files. Production core registration is
|
||||
covered by `src/main/index.ts`; the fixture remains focused on the runtime domain.
|
||||
Its model response is local and deterministic, with no external provider call.
|
||||
|
||||
The shared gateway was separately validated on the pinned Linux x64 test Host via
|
||||
the existing lifecycle harness rebuilt from current source. OpenCode ran a bounded
|
||||
tool command in a dedicated `/root/tmp/gb-lifecycle-*` workspace. Desktop detached,
|
||||
the test Agent was promoted, the controller resumed, and the prompt completed.
|
||||
There were exactly **2 real text-provider calls**, both HTTP 200, **0 image calls**,
|
||||
33 persisted event entries, 17 checkpoints, and successful replay deduplication.
|
||||
The old test daemon retired; final owned process count was zero. The harness removed
|
||||
its remote directories. Local artifacts were isolated in
|
||||
`temp/goodbuddy-ledger-upgrade` and cleaned after validation.
|
||||
|
||||
Scoped ESLint and `git diff --check` passed. The current composition supplies
|
||||
`bindingStore` to `ManagedRemoteExecutionServices` and `openModelCallLedger` to
|
||||
the native coordinator. Whole-application acceptance, packaged cross-platform
|
||||
checks and concurrent performance measurements are not established by these
|
||||
focused tests.
|
||||
@@ -5,11 +5,12 @@
|
||||
- **适用范围：** Desktop（Electron Main、Preload、Renderer）及其后台进程和线程。
|
||||
- **用途：** 新功能设计和代码评审的约束。多个 agent 并行修改代码时，以本文为统一标准。
|
||||
- **关联文档：** `docs/roadmap/product-performance-experience-improvement-plan.md`（测量记录与实施进度）。
|
||||
- **架构与验收：** [桌面存储与并发执行](./desktop-storage-direction.md)定义一次完整交付的职责和范围；[桌面并发验收](../quality/concurrent-desktop-acceptance.md)定义多项目、多会话、多子任务负载及证据。本文件是性能规范的唯一数值来源。
|
||||
- **参照：** VS Code 的进程分工和渲染做法。我们没有在同一台机器上实测过 VS Code，下面的预算按常见的流畅标准设定。
|
||||
|
||||
## 1. 性能预算
|
||||
|
||||
“流畅”用下面的数字判断，不靠感觉。“标准机”指不降速运行 `npm run perf:app`，“低端机”指加上 `GB_PERF_CPU_THROTTLE=4`。
|
||||
“流畅”用下面的数字判断，不靠感觉。“标准机”指记录了硬件环境、不降速运行的参考机；Renderer 降速场景使用 `GB_PERF_CPU_THROTTLE=4`。该降速仅作用于 Renderer，不等于真实低配整机，也不模拟慢 Main 或磁盘。预算适用于单场景及规定的并发负载。
|
||||
|
||||
| 指标 | 标准机 | 低端机 |
|
||||
| --- | --- | --- |
|
||||
@@ -21,7 +22,9 @@
|
||||
| Main 事件循环延迟，p99 | ≤ 30 ms | — |
|
||||
| DOM 节点数 | 不随会话数、消息数增长 | 同左 |
|
||||
|
||||
标准机的阈值由 `npm run perf:check` 检查。低端机的数值目前只做记录，等结果稳定后再加进阈值检查。
|
||||
当前 `npm run perf:check` 使用历史回归上限，部分数值宽于本表，也未覆盖全部并发入口。检查通过不能宣称本表达标。目标交付须在验收报告中分别给出规范预算和回归上限的结果；低端机降速目前只做记录，真实低配和新增自动检查的完成状态另行记录。
|
||||
|
||||
点击反馈、所需内容可用和后台工作完成分别计时。显示加载状态只能证明及时反馈；不能用它替代结果就绪。每个进程分别报告长任务／事件循环和排队耗时，禁止把多个独立探针相加为端到端延迟。
|
||||
|
||||
## 2. 原则
|
||||
|
||||
@@ -32,6 +35,7 @@ Main 负责窗口、生命周期、权限和消息转发。数据库查询、文
|
||||
- 新代码不能在 Main 里使用 `DatabaseSync` 和同步文件 API。lint 已有白名单，名单只能减少。
|
||||
- 暂时必须留在 Main 的同步操作，单次耗时要能证明在 16 ms 以内，并在代码里注明理由。
|
||||
- 耗时可能超过 16 ms 的操作，必须支持取消和进度上报，不能一直占着 Main。
|
||||
- 目标交付后 Main 不再直接或通过 helper 打开桌面业务 SQLite；只读、失败回退、维护及退出均在检查范围内。当前迁移例外不是新增同步工作的许可。存储宿主可继续使用同步 SQLite，事务和计算仍需有界。
|
||||
|
||||
### P2 界面状态局部订阅
|
||||
|
||||
@@ -53,7 +57,8 @@ Main 负责窗口、生命周期、权限和消息转发。数据库查询、文
|
||||
|
||||
- `App.tsx` 只负责布局外壳、路由和把 store 实例通过 context 提供给子树。新增领域状态一律走上表四层。
|
||||
- 每个 store 的单元测试和 store 文件放在一起（`<domain>-store.test.ts`），覆盖顺序、失败重试和边界情况。
|
||||
- Main 侧对应：`<domain>` 的数据库访问集中在一个 repository 类里，IPC handler 只做校验和转发。
|
||||
- 领域数据库访问集中在 repository，目标架构中由存储宿主执行；Main 的 IPC handler 只做校验并等待领域操作。受管只读 Worker 与远端 Agent 按各自明确的连接归属执行，不能按源码目录把它们误判为 Main 访问。
|
||||
- 多项目／多会话流式更新只通知相关领域与实体；后台子任务不得让全部项目视图重渲染。隐藏视图保留草稿、选择和必要执行订阅，停止无关绘制与全量重读，切回时按已有摘要及增量恢复。
|
||||
|
||||
### P3 高频数据走专门通道，按帧合并
|
||||
|
||||
@@ -62,12 +67,16 @@ Main 负责窗口、生命周期、权限和消息转发。数据库查询、文
|
||||
- 多条更新合并后，每帧最多提交一次；同步到全局状态的频率要更低，目前是 250 ms。
|
||||
- 高频路径上不能读取布局（`getBoundingClientRect`、`scrollHeight` 等），布局读取要集中到每帧一次。
|
||||
- 合并更新不能丢失、重复或打乱顺序。每条数据带序号，并有测试覆盖。
|
||||
- 显示合并与持久化分开。可以合并进度或替换过时刷新，不能静默丢弃消息、工具结果或需确认的事件；多生产者缓冲达到容量时等待或明确失败，不无限增加内存。
|
||||
|
||||
### P4 数据只有一份来源，只传变化部分
|
||||
|
||||
- 每类数据只有一个权威来源，由数据所在进程负责持久化。界面不把整份数据写回。
|
||||
- 跨进程的通知直接带上变化的内容，接收方不再“收到通知后重新查全量”。
|
||||
- 写入合并成批次和事务，不按每个事件单独开一个事务。
|
||||
- 一个数据库有一个业务写入所有者，完整事务不跨 IPC 拆成单条 SQL。Main 不保存第二份可写业务状态；依赖写入的读取等待对应提交，远程 ACK 等待所需事件与 checkpoint 提交成功。
|
||||
- 缓存必须说明归属、失效和释放条件，不按浏览过的项目／会话无限积累。查询返回摘要、分页或所需详情，保持完整信息可访问，不用截断历史满足预算。
|
||||
- GoodBuddy 自身的中转数据优先使用有界内存；已有完整记录或文件时复用引用，避免重复落盘。预览、最终回答与工具全文分别核实，不能以“已保存消息”推断全文已保存。需要超出内存保留的内容直接使用所属存储的唯一正文，分页读取同一内容，避免临时全文再复制到正式存储。必要文件及清理范围遵守[输出与临时资源设计](./desktop-storage-direction.md#output-and-temporary-resource-ownership)。
|
||||
|
||||
### P5 工作量和可见内容成正比
|
||||
|
||||
@@ -80,22 +89,26 @@ Main 负责窗口、生命周期、权限和消息转发。数据库查询、文
|
||||
- 首屏之前只做显示应用壳和恢复上次界面所需的工作。
|
||||
- 其他初始化（Provider、Runtime、知识库、通知渠道等）按需执行，或等首屏显示后再空闲执行。
|
||||
- 后台模块在第一次使用时才加载。
|
||||
- 冷启动、首次进入功能和暖返回分别验收；预加载只使用有界空闲工作，不能为每个项目预建进程、连接或整份历史缓存。进程 spawn 不代表服务就绪，首次可用反馈与数据实际可用分开记录。
|
||||
|
||||
### P7 先测量，再修改
|
||||
|
||||
- 先跑测量，拿到数据再改代码；每项改动都要有前后对比，基线用临时 worktree 构建。
|
||||
- 每个版本至少跑 3 轮，交替运行；结果差异小于同一版本多次运行之间的波动时，不能说成“有提升”。
|
||||
- 测量用隔离的用户目录和本地假模型，不发出外部模型请求，结果里不含用户内容。
|
||||
- 必测多项目、多会话、多子任务及导入／检索／回顾发布重叠场景，负载和采样按[并发验收](../quality/concurrent-desktop-acceptance.md)。记录 offered load、实际吞吐、排队和排空、资源高水位；不得降低输入速率、串行化夹具或丢事件来取得流畅结果。
|
||||
- 基准使用固定版本、数据和环境。性能测量与真实模型功能验收分开；受发布构建限制时遵守发布手册，不以性能原则授权额外生产打包。
|
||||
|
||||
### P8 正确性优先于速度
|
||||
|
||||
- 事件顺序、持久化、取消、崩溃恢复和显示结果都不能因为性能而改变。
|
||||
- 异步改造要先确认“先写入、后读取”的顺序，没有测试覆盖就不合并。
|
||||
- 风险高、收益小，或者会改变现有功能的优化先不做，在计划文档里记录理由。
|
||||
- 取消请求、执行结算和容量释放不是同一时刻；调用方停止等待不能提前释放仍运行的工作。取消一个会话或子任务不得杀死仍服务其他任务的共享宿主。已提交事务如实报告成功，不伪装成回滚；未确认写入不盲目重放。
|
||||
|
||||
### P10 体验不倒退
|
||||
|
||||
性能优化只能让体验变好，不能用体验换速度。“行为不变”不仅指数据和结果，也包括用户能感知到的交互：
|
||||
性能优化必须保留数据、结果和用户交互语义，不能用体验换速度：
|
||||
|
||||
- **动画和过渡保持原样。** 平滑滚动（如“到底部”按钮）、展开收起、悬停、淡入淡出都不能因为优化而消失。任何瞬间写入 `scrollTop`/`scrollTo` 的代码都会打断正在进行的平滑滚动，必须在平滑滚动期间暂停。
|
||||
- **只渲染可见部分（P5）时，用户状态不能丢。** 行被卸载后，它的局部状态也会消失。以下行必须保持挂载，或者把状态存到行外：有焦点的行、用户展开过 `<details>` 或输入过内容的行、选中文本的两端、待回答的问题和待审批项、打开菜单或正在重命名的行。
|
||||
@@ -109,6 +122,24 @@ Main 负责窗口、生命周期、权限和消息转发。数据库查询、文
|
||||
- lint 规则和性能阈值是合并前的硬性条件，不靠人工记得。
|
||||
- 规则只能收紧，白名单和上限只能往下调。
|
||||
- 每个 agent 只提交自己改的文件和片段，不把别人未提交的改动带进自己的提交。
|
||||
- 检查直接 SQL 导入及已知间接打开函数／构造点，按实际宿主区分测试、存储与远端代码。白名单和上限需要与已提交基线比较，不能只靠注释约定；全局同步回退同样要被发现。
|
||||
- 普通 CI 与性能执行环境分别提供功能和预算证据；新增并发与首次进入场景纳入候选必需检查，缺少运行环境或结果时保留验收缺口，不由单测通过推断达标。
|
||||
|
||||
### P11 并发隔离与有界工作
|
||||
|
||||
同一会话的事件顺序和同一数据库的提交顺序必须保持；不同项目、会话、子任务的模型等待、重查询与清理不因此共用一把长时间全局锁。
|
||||
|
||||
- Main 全局控制队列不得等待模型、网络、原生启动或耗时清理。必要登记和状态提交可短暂串行，等待在所属操作范围内完成。
|
||||
- 重查询复用有界 Worker；必要写入不排在整个导入解析或模型请求之后。长事务先减少事务内无关计算，再测写入排队，不把阻塞简单搬进唯一存储队列。
|
||||
- 复用现有池、批处理和队列，记录在途数、排队数及字节上限。有限任务在持续竞争下仍须完成，繁忙会话不能无限抢占下一批；有实测饥饿才引入最小调度调整，不预建通用优先级框架。
|
||||
- 控制和取消不等待新的执行槽位；饱和时按业务语义等待或返回可恢复错误。禁止静默丢失、无限缓冲、无限重启或自动扩增进程。
|
||||
- 每个异步响应携带已有项目／会话／任务／操作归属；迟到响应不得覆盖新选择或新操作。归属检查先于改变共享请求代次、缓存或队列状态。
|
||||
|
||||
### P12 架构完整性与 KISS
|
||||
|
||||
一次性交付按最终职责边界验收，不以全面 Promise 化或局部热点迁出代替。目标范围内正常、后台、失败、重试、取消和退出路径均使用同一权威实现，清理原直接入口和回退。
|
||||
|
||||
保持现有技术栈、数据库文件及业务合同，复用已有资源机制。不得顺带新增分布式事务、持久 RPC receipt、通用恢复框架、每项目独立服务或产品审批模式。同步内存缓存读取可以保留，不把纯函数和每个 getter 远程化。设计新增机制必须指出当前失败、现有机制的不足与对应验证，不以“将来可能并发”代替证据。
|
||||
|
||||
## 3. 评审清单
|
||||
|
||||
@@ -125,21 +156,30 @@ Main 负责窗口、生命周期、权限和消息转发。数据库查询、文
|
||||
9. 动画、平滑滚动、过渡还在吗？有没有新增会打断平滑滚动的瞬间滚动写入？
|
||||
10. 只渲染可见部分的列表，滚走再滚回来，展开状态、输入内容、选区、焦点还在吗？
|
||||
11. 合并或延迟处理后，第一次响应（打字回显、点击反馈）有没有变慢？复制、导出读到的是不是屏幕上的最新内容？
|
||||
12. 多项目／多会话／多子任务同时运行时，取消或清理一个请求是否拖住其他任务？队列和缓存是否持续增长？
|
||||
13. 存储提交、远程 ACK、调用方返回和资源释放的顺序是否有真实路径及故障注入验证？
|
||||
14. 是否删除目标范围内的旧直接访问和同步回退？新增进程或调度机制是否复用现有设施并有实际必要性？
|
||||
15. 请求或输出是否产生非必要临时文件、重复全文？是否核实完整内容与预览的区别，并验证分页、引用释放、必要文件的创建失败清理及升级后的残留回收？
|
||||
|
||||
## 4. 当前差距
|
||||
|
||||
截至 2026-10-04（第一期结项，见 `docs/review/performance-phase-1-report-2026-10-04.md`），按原则对照的主要差距：
|
||||
截至 2026-10-05，以下状态结合[存储升级实施记录](../quality/storage-upgrade-implementation-2026-10-05.md)和[并发验收规范](../quality/concurrent-desktop-acceptance.md)核对。表中“已实现”只表示代码或定向测试已有证据，不表示 PERF-15、PERF-16 或 C01-C07 已完成。
|
||||
|
||||
| 原则 | 差距 |
|
||||
| --- | --- |
|
||||
| P1 | 同步数据库调用仍在 Main，没有独立数据进程；但热点读已在 worker，高频写单次 0.1–0.3 ms，WAL checkpoint 在后台线程，启动恢复按索引读取。文档解析、分块和大文件哈希已移到 worker。消息附加数据冗余（子任务工具输出、`tools` 与 `blocks` 重复），大会话读取约 170 ms |
|
||||
| P1 | Assistant、Knowledge、文件和 Runtime 生产域已由 `DesktopStorageClient` 接入 utility process；剩余缺口是全量间接访问扫描、异常/维护/退出路径核验，以及 Main 阻塞和存储排队的并发测量。文档解析、分块和大文件哈希仍在各自 worker。 |
|
||||
| P2 | `App.tsx` 约 7,900 行。输入区、隐藏路由、任务和结果事件、切换会话已不再整体重渲染 App；当前会话的流式吸收（每 250 ms）、队列和活动记录仍经过 App |
|
||||
| P3 | 已基本满足：流式输出和终端输出都已按帧合并 |
|
||||
| P4 | 活动记录已改为增量保存、按页读取（界面只保留已加载的页，计数和分组来自 Main 的汇总）；部分 `*Changed` 通知之后还会重新查全量 |
|
||||
| P5 | 已基本满足：消息、会话、笔记、文件树、知识库文档表格都只渲染可见部分 |
|
||||
| P6 | 空配置启动到输入框可用约 0.96 秒；真实数据（2.4 GB）约 1.36 秒。本地工具环境已后台初始化，低频模块按需加载 |
|
||||
| P9 | 性能阈值还没有接入 CI，也没有真实低端机的基线 |
|
||||
| P3 | 流式输出和终端输出已有按帧合并；C01/C02/C06 的长期负载仍需验证是否出现排队增长、长帧或内容丢失。 |
|
||||
| P4 | 存储域已统一到各自 owner，输出 backing 和附件引用已接入；部分 `*Changed` 通知仍会重新读取全量，重复缓存和跨域通知的并发影响待测。 |
|
||||
| P5 | 消息、会话、笔记、文件树、知识库文档表格已有窗口化；C07 仍需记录冷/暖内容可用时刻和切换后的状态保留。 |
|
||||
| P6 | 空配置和真实数据启动有历史测量；当前 C07 尚未完成监督概览、图谱、历史选择和来源正文的完整冷/暖验收。 |
|
||||
| P9 | 已有 lint 和历史回归上限；间接 SQLite 打开、白名单单调收紧、并发指标缺失和 CI/性能环境接入仍待补齐。 |
|
||||
| P10 | 已有局部输入、刷新和项目切换修复；需要在 C01/C02/C06 中继续验证动画、焦点、滚动、草稿和取消语义没有因降载或合并而退化。 |
|
||||
| P11 | 存储请求和运行时路径已有有界容量与按域归属；C02、C04、C05、C06 尚未证明健康 peer 隔离、饱和恢复、队列排空和有限后台任务完成。 |
|
||||
| P12 | 目标边界和 KISS 约束已写入架构文档；S01-S07 已有生产组合，但删除旧直接入口、异常路径和全候选统一验收仍未完成。 |
|
||||
|
||||
2026-10-04 补充：监督回顾的来源初始化和继续前版本校验已移入现有 Worker 入口的独立连接，扫描不持有写锁，清单按 200 条提交，生产失败不回退 Main 同步扫描。最终发布事务仍在 Main；实现与实测边界见[监督者进度](../features/conversation-supervision/progress.md#2026-10-04-回顾初始化阻塞修复)。
|
||||
|
||||
2026-10-05 补充：监督者 UI 的概览、图谱、故事读取也已迁到现有只读 Worker；概览按需加载、列表窗口化及摘要缓存见[KISS 局部修复](../review/kiss-local-fixes-2026-10-05.md)。这与工具侧 Story Graph 读取是不同入口。Main 最终发布及部分大响应仍有超预算样本，不能把局部完成视为全项达标。存储服务目标另见[桌面存储方向](./desktop-storage-direction.md)，本轮不实施全面迁移。
|
||||
2026-10-05 局部修复记录：监督者 UI 的概览、图谱、故事读取也已迁到现有只读 Worker；概览按需加载、列表窗口化及摘要缓存见[KISS 局部修复](../review/kiss-local-fixes-2026-10-05.md)。这与工具侧 Story Graph 读取是不同入口。Main 最终发布及部分大响应仍有超预算样本，不能把局部完成视为全项达标。
|
||||
|
||||
随后用户确定按[完整存储与并发架构](./desktop-storage-direction.md)一次性交付。当前已完成存储 owner、主要生产组合和部分输出迁移，但没有完成全面架构验收。P11／P12 与[并发验收矩阵](../quality/concurrent-desktop-acceptance.md)约束当前候选的剩余验证，不能作为已有实现或实测结果；上述第一期历史状态也不替代当前验收。
|
||||
|
||||
@@ -1,8 +1,18 @@
|
||||
# Runtime Storage And Background Reads
|
||||
|
||||
## Current Implementation
|
||||
|
||||
This file records current runtime timing, reader behavior and temporary-resource
|
||||
ownership. The production runtime receives the storage-owned output backing store and
|
||||
output adoption function during Main startup. Main also computes one per-version,
|
||||
per-run temporary root and passes it to the local runtime cache paths and native
|
||||
client coordinator.
|
||||
|
||||
SQLite owns persisted task, event and conversation evidence. The desktop business
|
||||
connection is long-lived. WAL permits readers alongside a writer; synchronous SQL
|
||||
still blocks the thread that executes it. Memory caches are disposable derived data.
|
||||
Persisted evidence is not necessarily full tool output; the inventory below records
|
||||
the current distinction. The target ownership rules are in the architecture document.
|
||||
|
||||
Startup storage upgrades run in `assistant-storage-worker` before the business
|
||||
connection opens. Runtime readers must not invoke initialization, migrations or
|
||||
@@ -16,6 +26,17 @@ or result cache. Main owns current segments independently of renderer visibility
|
||||
The [duration design](../features/task-and-job/technical-design.md) defines the
|
||||
state boundaries, restart handling and incomplete history.
|
||||
|
||||
## Planned Architecture
|
||||
|
||||
The [complete desktop architecture](./desktop-storage-direction.md) defines the
|
||||
remaining target, including desktop binding databases and native-client model ledgers.
|
||||
Its migration and [concurrent acceptance](../quality/concurrent-desktop-acceptance.md)
|
||||
are one delivery; remote Agent database ownership remains unchanged. Main will still
|
||||
observe runtime transitions while the storage service persists them in order; the
|
||||
indexed timing representation and semantics remain.
|
||||
|
||||
## Runtime Read Lifecycle
|
||||
|
||||
Read transactions end before replies are posted. Long-lived read transactions
|
||||
would prevent WAL checkpoints from reclaiming old pages. Each reader has its own
|
||||
SQLite page cache; adding workers increases memory usage and requires evidence of
|
||||
@@ -34,3 +55,193 @@ reopening. Results remain scoped; superseded responses are discarded.
|
||||
The [task statistics design](../features/task-and-job/technical-design.md) owns the
|
||||
snapshot contract and timing rules. Tests and timings demonstrate only their fixture
|
||||
conditions; they do not establish a latency guarantee for arbitrary retained history.
|
||||
|
||||
## Output And Temporary File Investigation
|
||||
|
||||
Source review on 2026-10-05. The inventory records the current production flow. No
|
||||
user Temp directory was scanned or deleted.
|
||||
|
||||
| Producer | Current content flow | Retention and consequence |
|
||||
| --- | --- | --- |
|
||||
| Local process and ripgrep | [Process capture](../../src/main/agent/direct-model-process-service.ts) writes stdout/stderr through [PagedOutputStore](../../src/main/agent/paged-output-store.ts); model receives a JSON-bounded preview, up to 96 KiB per stream, and a reference when truncated | Every stream opens a file, including empty stderr. Small files are deleted on finish; retained files and their in-memory handles belong to the conversation/service |
|
||||
| Direct-model subagent | [Subagent service](../../src/main/assistant/direct-model-subagent-service.ts) captures child text through the same store, returning up to 192 KiB and an optional reference | [Provider event bridge](../../src/main/agent/model-tool-provider.ts) forwards preview/status but drops the output reference from the subagent event; child text is not an independent full parent-history record |
|
||||
| Tool events and conversation history | [Model runtime](../../src/main/agent/model-runtime.ts) emits at most 16,000 characters of serialized tool result as the event preview; [IPC](../../src/main/ipc.ts) and [database](../../src/main/assistant/assistant-database.ts) persist the supplied event | Tool events still carry bounded previews, while output history adopts finished handles through attachment storage. Even `output_read` responses are previewed in tool events. Persisted event history does not by itself guarantee a complete stdout/stderr copy |
|
||||
| Remote ACP | [Agent transcript](../../src/agent-daemon/semantic-prompt-store.ts) retains received notifications pending ACK; [desktop adapter](../../src/main/agent/acp-remote-runtime.ts) bounds mapped events before persistence | ACK prunes Agent transcript rows. Notifications themselves may already be bounded by the Runtime. Do not treat this journal as a permanent full-output source or remove commit-before-ACK |
|
||||
| MCP calls | [MCP transport](../../src/main/capabilities/mcp-client-transport.ts), provider and [gateway](../../src/main/agent/knowledge-mcp-gateway.ts) pass objects through stdio/HTTP/SSE | No per-call request/response scratch-file creation was found in these paths. Calling a tool over MCP does not itself require a payload file; third-party server internals are outside this inventory |
|
||||
|
||||
`PagedOutputStore` keeps short and empty output in memory and delegates larger
|
||||
or JSON-preview-truncated content to an injected `PagedOutputBackingStore`. Capture
|
||||
has a shared 8 MiB budget, at most 1,024 pending chunks per writer and 4,096 active or
|
||||
retained handles per store; backing writes are at most 64 KiB. Producers must await
|
||||
append, or receive an explicit capacity error. Local process pipes pause until each
|
||||
append settles. The store does not create a temporary file backend or another DB.
|
||||
UTF-8 byte cursors, JSON-escaped preview bounds, owner checks and failed/cancelled
|
||||
output remain covered by focused tests using real async files and local children.
|
||||
|
||||
The production owner exposes output create, append, finish, read, adopt and release
|
||||
through the existing storage transport. Main passes its `backingStore` to the runtime
|
||||
and passes `outputAdopt` to the tool provider. When output history is enabled, finished
|
||||
stdout/stderr references are adopted into the conversation attachment reference for
|
||||
the target message; the bounded preview and output reference are returned together.
|
||||
Missing backing configuration still explicitly fails capture that requires full
|
||||
storage. A model's final answer is not an equivalent copy of its tool output.
|
||||
|
||||
### Planned Architecture And Remaining Acceptance
|
||||
|
||||
The planned architecture still requires full end-to-end history and restart coverage,
|
||||
resource reconciliation and the remaining storage-owner migration. The file-backed
|
||||
test fixture alone does not establish that architecture or its acceptance.
|
||||
|
||||
Tool previews can also appear in both conversation `tools` and `blocks`. This is a
|
||||
separate persisted-preview duplication issue, not evidence that full paged output is
|
||||
already stored. Existing [subagent progress compaction](../../src/main/assistant/subagent-progress-storage.ts)
|
||||
already handles repeated progress snapshots; retain that implementation.
|
||||
|
||||
### Other Owned Files
|
||||
|
||||
| Existing owner | Source finding | Disposition for the upgrade |
|
||||
| --- | --- | --- |
|
||||
| [DocumentResultStorage](../../src/main/document-result-storage.ts) and [attachment storage](../../src/main/conversation-attachment-storage.ts) | `parsed.md` duplicates `manifest.json.content`; normal reads use the manifest and `copyResult` copies the extra file | Remove the redundant new write/copy. Preserve original files, extracted images, manifests and ownership transfer; old directories follow existing reference cleanup |
|
||||
| [OpenCode runtime](../../src/main/agent/opencode-runtime.ts) | Configuration already uses `OPENCODE_CONFIG_CONTENT`; scratch supplies isolated XDG state and skill registration | Keep required isolation, reuse immutable skill caches. Do not invent a config-file optimization for a path already using memory/environment |
|
||||
| [Continue adapter](../../src/main/agent/continue-host-adapter.ts) | CLI uses `--config` plus an isolated global directory; creation can fail before the outer owner receives the path | Retain the external-file interface and clean partial creation at the creating function, including skill-staging failure; preserve pre-existing files on `EEXIST` |
|
||||
| [Continue helper](../../src/agent-daemon/continue-acp-helper.ts) and [model bridge helper](../../src/agent-daemon/model-bridge-helper.ts) | Temporary config/plugin resources use helper finalization; setup before the protected scope and termination need lifecycle coverage | Extend existing ownership and shutdown, verify child exit before deletion. Plugin file URLs and Unix sockets remain necessary interfaces under current contracts |
|
||||
| [Managed Python](../../src/main/local-tool-environment/managed-python-install.ts), [model packages](../../src/main/model-package-utils.ts), [Agent packages](../../src/main/remote-agent/agent-package-manager.ts) | Install staging, partial downloads and backups support validation/publication; existing cleanup has owner-specific patterns | Extend those cleanup functions for demonstrated leftovers; preserve active staging and verified reusable installations. No global installer sweep |
|
||||
| [Magic note storage](../../src/main/magic-notes/magic-note-storage.ts) | Sibling atomic-write `.tmp` files protect replacement; memory-database fixtures have a separate temporary root | Keep atomic replacement and normal failure cleanup. Test scratch follows repository rules; persistent note directories are user data |
|
||||
|
||||
Document results may move from temporary ownership into attachment references, including
|
||||
shared branch references. A directory under `temp/document-parsing` can still be owned
|
||||
by persisted data. Its location alone does not authorize deletion. Native client history,
|
||||
model caches, database WAL/SHM and exported user files are likewise not disposable scratch.
|
||||
|
||||
Some tests/build helpers share production-looking prefixes, including `goodbuddy-process-*`
|
||||
and `goodbuddy-opencode-*`. Prefix and age alone cannot establish who owns a legacy directory.
|
||||
The [architecture rules](./desktop-storage-direction.md#output-and-temporary-resource-ownership)
|
||||
define the replacement and cleanup scope; the [acceptance matrix](../quality/concurrent-desktop-acceptance.md#输出与临时资源验收)
|
||||
defines evidence required before claiming improvement.
|
||||
|
||||
### Creator Cleanup Validation (2026-10-05)
|
||||
|
||||
Continue now removes its partially written config and failed skill-staging directory
|
||||
at the creator, before returning ownership to the run. Exclusive creation failures
|
||||
leave existing files/directories intact. Run finalization preserves prepared caches
|
||||
and external configuration. Both Agent helpers protect setup with finalization and
|
||||
handle SIGTERM/SIGINT; Continue stops prompt admission, aborts and drains active
|
||||
runs, while the OpenCode helper waits for child exit before deleting its plugin.
|
||||
Managed Python's existing pre-install cleanup also removes regular download files
|
||||
with the exact UUID layout, preserving directories, unknown names, caches and history.
|
||||
|
||||
Validation in the concurrent working tree:
|
||||
|
||||
- Five focused suites (`continue-host-adapter`, `continue-acp-helper`, `model-bridge`,
|
||||
`managed-python-install`, `local-tool-environment-service`): 81 passed, 10 Unix-only
|
||||
cases skipped on Windows. Failure injection covers partial creation, EEXIST,
|
||||
failed skill staging, setup-time signals and shutdown cleanup.
|
||||
- Final `npm run typecheck` passed; ESLint passed for all eight owned source/test files.
|
||||
Repository lint passed with `npm run lint -- --ignore-pattern temp`; unfiltered
|
||||
lint encountered generated validation bundles in ignored task/release directories.
|
||||
Task-owned `temp/goodbuddy-scratch-upgrade` was subsequently removed and its absence
|
||||
verified; unrelated release scratch was preserved.
|
||||
- Full Vitest run: 517 suites passed, 7 failed, 15 skipped; 6,244 tests passed,
|
||||
6 failed, 94 skipped. Failures outside this cleanup scope included OpenCode config
|
||||
preparation (`spawnSync npm.cmd EINVAL`), the MCPVault Electron fixture, and
|
||||
message-expansion and workspace-import Electron assertions. These results do not
|
||||
establish full architecture acceptance.
|
||||
- Reused the existing `a01-runtime-host-build.cjs`/`a01-runtime-host-run.cjs` harness
|
||||
logic, rebuilding Agent, daemon and Desktop driver from current source under
|
||||
`temp/goodbuddy-scratch-upgrade`. Connected to Linux x64 over the LAN using
|
||||
GoodBuddy encrypted credentials and the pinned Host identity. Both routes were
|
||||
reachable; only the LAN route was used for execution.
|
||||
- Both OpenCode and Continue passed real tool execution, startup cancellation,
|
||||
disconnect/reconnect during execution, checkpoint persistence/replay deduplication,
|
||||
and retirement. Final-source cancellation took 217 ms and 233 ms respectively;
|
||||
each run reported zero owned processes after retirement and shutdown. There were
|
||||
eight real model requests across the initial and final runs, four per runtime.
|
||||
Remote artifacts stayed in dedicated `/root/tmp/gb-lifecycle-*` directories and
|
||||
followed the harness cleanup. These timings are scenario observations, not a
|
||||
comparative performance claim.
|
||||
|
||||
### Version And Run Scratch Follow-up (2026-10-05)
|
||||
|
||||
`runtimeTemporaryRoot(userDataPath, appVersion)` computes the lazy
|
||||
`temp/goodbuddy-runtime/<version>/<runId>/goodbuddy-runtime-launch` path.
|
||||
Main passes that same value through existing `continueHostCacheRoot` and
|
||||
`opencodeSharedCacheRoot` options. The native client coordinator receives it as
|
||||
`temporaryRoot` and passes it to both native clients. The existing constructors need
|
||||
no new parameter.
|
||||
OpenCode creates its isolated launch directory beneath the supplied shared root;
|
||||
Continue puts generated configuration and isolated globals in one per-call directory.
|
||||
Prepared bundles and immutable Skill/config caches remain reusable within the run.
|
||||
The helpers on a remote Host use `goodbuddy-runtime-launch` beside their existing
|
||||
model-bridge socket, under that process owner's scratch directory. Global `TEMP`
|
||||
is unchanged.
|
||||
|
||||
Each local launch has a sibling owner file containing creator and child PIDs. Keeping
|
||||
metadata outside the deleted tree preserves retry evidence after partial deletion.
|
||||
The launch root also records its creator for cache-only reclamation. After the
|
||||
single-instance check and first screen, call `cleanupRuntimeTemporaryDirectories`
|
||||
in the existing delayed background startup work. It examines at most 256 directory
|
||||
entries per invocation, sequentially, and accepts the startup shutdown signal.
|
||||
Service disposal still stops children before deleting its own launch material;
|
||||
an unconfirmed child exit retains the material. Startup preserves live/reused PIDs,
|
||||
permission-denied process checks, malformed ownership and the ambiguous spawn-to-PID
|
||||
recording window. Empty version/run parents are removed after owned material.
|
||||
This is not a new process owner or an installer collector.
|
||||
|
||||
Legacy candidates are limited to the known OS-Temp layouts `goodbuddy-opencode-*`
|
||||
(desktop isolated state), `goodbuddy-continue-*` (Continue helper), and
|
||||
`goodbuddy-opencode-plugin-*` (model-bridge plugin). Deletion requires valid owner
|
||||
metadata and confirmed inactive creator/child PIDs. Releases that wrote no such
|
||||
metadata cannot be identified as obsolete from their names, contents or age alone;
|
||||
those directories are retained and reported. A bounded pass can leave candidates
|
||||
unvisited. Native-client histories, existing version-only caches and unrelated
|
||||
OS-Temp files are outside this reclamation contract. The old version/layout-only
|
||||
cache sweep is not evidence that a surviving child has exited.
|
||||
|
||||
Remote environment preparation now retains its existing pending-operation record
|
||||
when connection acquisition or staging cleanup fails. Because that store has one
|
||||
record per Host, a replacement prepare retries cleanup before overwriting it;
|
||||
confirmed cleanup or `operation-unavailable` clears it. Already adopted environments
|
||||
remain usable. Agent package delayed removals retain their pending entry until
|
||||
deletion succeeds, retry on inventory access, and rediscover verified obsolete
|
||||
versions through the existing package scan after restart. Active archive leases
|
||||
still prevent deletion.
|
||||
|
||||
Continue bundle staging now waits for all parallel writes to settle before failure
|
||||
cleanup. Existing settings atomic replacement and selected-runtime diagnostic
|
||||
teardown already use `finally`; no global `.tmp` sweep or new diagnostic owner was
|
||||
added. Native clients now accept an optional `temporaryRoot` for the same application
|
||||
version/run launch root. `NativeTerminalClient` keeps histories and reusable caches
|
||||
under its existing `rootDirectory`; only per-launch material moves to scratch.
|
||||
DS Web keeps `DSH_HOME`, sessions and user preferences in place and supplies its
|
||||
generated configuration through `web --patch <scratch-file>`. Its ledger also lives
|
||||
in scratch. Both reuse the existing sibling PID metadata and await ledger close
|
||||
before releasing material; unconfirmed child exit retains it. ConPTY registration
|
||||
waits for the actual child PID, which can be unavailable immediately after spawn.
|
||||
The collector recognizes `goodbuddy-native-dsh-*` alongside the existing prefixes.
|
||||
The production wiring propagates `temporaryRoot` through the native coordinator to
|
||||
both constructors without relocating the retained-history root.
|
||||
Focused verification passed 60 tests, including a real Windows PTY, official DS Web
|
||||
session persistence across restart, and delayed ledger-close cleanup. Typechecking
|
||||
and scoped lint passed; this native follow-up made zero real model calls.
|
||||
|
||||
Final-source verification for this follow-up is separate from the earlier eight
|
||||
real requests above: two real provider requests total, one per runtime. Current-source
|
||||
Continue/OpenCode helpers ran on Linux x64 over pinned SSH, each executed its native
|
||||
command tool, returned the real model answer, and removed launch files after exit.
|
||||
A separate active-request shutdown probe passed for both helpers with zero real
|
||||
provider requests. Four successful scenarios used deterministic command responses;
|
||||
probe setup retries before these results did not reach a provider. A previous-process
|
||||
version/run directory was reclaimed using actual dead PIDs. After the final
|
||||
enumeration-error guard, a separate Host probe confirmed that the current collector
|
||||
retains a real live child and reclaims that same run after its exit, with zero model
|
||||
requests. The timed-out probe's ledger was audited and contained zero calls; all
|
||||
task probe artifacts were removed locally and remotely. These probes cover the
|
||||
changed helper and scratch lifecycle, not full Desktop UI or installation acceptance.
|
||||
|
||||
The seven primary suites passed 267 tests with 10 platform skips; related runtime,
|
||||
native-terminal and diagnostic-owner suites also passed. Agent and web typechecks
|
||||
passed. Full desktop typecheck and repository lint remain blocked by concurrent
|
||||
`portable-storage-upgrade-fixture.ts` typing and `portable-storage-upgrade.mjs` lint
|
||||
errors; unfiltered lint also visits generated files under `temp`. Focused lint for
|
||||
the edited source/tests passed. The C01-C07 concurrent acceptance matrix remains
|
||||
incomplete; these results do not establish full storage-architecture acceptance.
|
||||
|
||||
Reference in New Issue
Block a user