Files
goodbuddy/docs/architecture/desktop-storage-knowledge.md
mesalogo d80d42b032 feat(storage): move desktop SQLite ownership into a storage utility process
Main no longer opens desktop business SQLite. A single utility-process
storage owner holds the Assistant, Knowledge, attachment, document-result,
remote-binding and native model-ledger databases and runs whole domain
transactions; Main and Runtime callers use explicit async domain operations.

- Remove the Main readWithFallback path and the supervision worker's
  separate business write connection; manifests are scanned read-only and
  committed by the sole writer.
- Remote events are ACKed only after commit; process loss reports
  unconfirmed writes instead of replaying them.
- Use native utilityProcess messaging with one in-flight bound (32) and
  ordered waiting instead of capacity rejection and retries.
- Start the storage process before the window so startup overlaps.
- Keep short tool output in memory and store full output once in owned
  storage; paging, branching, restart and deletion reuse that copy.
- Put required runtime launch files under goodbuddy-prefixed owned temp
  directories and reclaim confirmed stale ones; stop writing duplicate
  parsed.md.
- Add lint guards for SQLite ownership and architecture, concurrency
  acceptance fixtures, and maintenance/architecture documentation.

Validation: typecheck and lint pass. Alternating baseline (edf019aa) vs
candidate runs show no regression on single-session, C01, C02, C04 and a
5-minute C06; Main p99 drops from ~21 ms to ~16.7 ms under concurrency.
Fresh portable copy (1.75M events) upgrade content unchanged; real-model
output paging passes with 4 external requests. Full npm test runs had
1-9 different timing-sensitive failures per run that pass when rerun.
Not covered: C03 review publication, C07 source body, app-level C05.
2026-10-06 15:40:18 +08:00

3.6 KiB

Knowledge Main Storage Migration (S03)

Knowledge Main now receives KnowledgeServiceOptions.database, a KnowledgeStoragePort. Production composition constructs it with createKnowledgeStoragePort(storageClient) after storageClient.ready. KnowledgeService no longer opens, initializes, closes, or starts readers for the business database. The application storage owner retains those duties.

The adapter sends explicit knowledge and external domain calls. Search signals travel as transport options, not serialized arguments. The port's Awaitable return type permits synchronous owner-local test implementations; the production adapter always returns the storage client's promises.

Main awaits source, document, task, graph, embedding and external metadata operations. IPC mutations await their writes before returning or refreshing a snapshot. Parsing, models, credentials and source watchers remain in their existing Main services. Embedding batches retain their existing sizes and staging/publication sequence.

Publication passes the extracted graph through foundation publishDocument(..., { graph }). Graph-only replacement calls replaceDocumentGraph(library, document, result). Its implementation in src/main/desktop-storage-knowledge-operations.ts runs the existing synchronous graph algorithm inside replaceEvidenceForDocument's transaction. No callback or SQL crosses the storage boundary.

Shutdown aborts producers immediately, drains tracked work and observes the task-interruption write and embedding status persistence. The service does not close the shared storage client. Embedding allocation returns its replacement ID even if cancellation arrives with the reply, allowing cleanup; a committed finish is not reported as rolled back. Rebuild admission remains occupied until its final status write settles, and status-write failures reach the completion barrier.

Current Production Boundary

The following boundary is implemented in the current production composition:

  • replaceDocumentGraph(library: KnowledgeBase, document: Document, result: GraphExtractionResult): void to the knowledge domain contract and dispatch it to the exported owner-local operation, passing the owned database as its first argument. It is not part of the concrete database method allowlist because it is a composite owner operation.
  • Foundation publication reuses the exported storeExtractedGraph helper.
  • StorageData preserves the existing recursive JSON type without rebuilding it in the adapter.
  • Persistent document lookup is resolved by the storage owner through the Knowledge domain; Main does not register a second lookup.
  • Application composition injects the port, awaits asynchronous gateway and tool-provider calls, and closes shared storage after Knowledge disposal.

Validation

The focused Knowledge service, external service, embedding coordinator and embedding repository suites use a Promise-backed adapter over a test-owned SQLite connection. Coverage includes publication rollback, embedding batch ordering, late cancellation, replacement cleanup, external request cancellation, credential persistence and restart. The production gateway now awaits its library and search operations. Remaining evidence is limited to the focused suites and does not establish full concurrent desktop acceptance.

Owned production files pass ESLint. The current worktree typecheck and focused storage validation cover the injected production port and owner dispatch. Global architecture acceptance, packaged checks and concurrent performance acceptance remain with the parent delivery; these tests do not establish a new performance baseline.