feat(storage): move desktop SQLite ownership into a storage utility process
Main no longer opens desktop business SQLite. A single utility-process
storage owner holds the Assistant, Knowledge, attachment, document-result,
remote-binding and native model-ledger databases and runs whole domain
transactions; Main and Runtime callers use explicit async domain operations.
- Remove the Main readWithFallback path and the supervision worker's
separate business write connection; manifests are scanned read-only and
committed by the sole writer.
- Remote events are ACKed only after commit; process loss reports
unconfirmed writes instead of replaying them.
- Use native utilityProcess messaging with one in-flight bound (32) and
ordered waiting instead of capacity rejection and retries.
- Start the storage process before the window so startup overlaps.
- Keep short tool output in memory and store full output once in owned
storage; paging, branching, restart and deletion reuse that copy.
- Put required runtime launch files under goodbuddy-prefixed owned temp
directories and reclaim confirmed stale ones; stop writing duplicate
parsed.md.
- Add lint guards for SQLite ownership and architecture, concurrency
acceptance fixtures, and maintenance/architecture documentation.
Validation: typecheck and lint pass. Alternating baseline (edf019aa) vs
candidate runs show no regression on single-session, C01, C02, C04 and a
5-minute C06; Main p99 drops from ~21 ms to ~16.7 ms under concurrency.
Fresh portable copy (1.75M events) upgrade content unchanged; real-model
output paging passes with 4 external requests. Full npm test runs had
1-9 different timing-sensitive failures per run that pass when rerun.
Not covered: C03 review publication, C07 source body, app-level C05.
This commit is contained in:
@@ -0,0 +1,66 @@
|
||||
# Knowledge Main Storage Migration (S03)
|
||||
|
||||
Knowledge Main now receives `KnowledgeServiceOptions.database`, a
|
||||
`KnowledgeStoragePort`. Production composition constructs it with
|
||||
`createKnowledgeStoragePort(storageClient)` after `storageClient.ready`.
|
||||
`KnowledgeService` no longer opens, initializes, closes, or starts readers for
|
||||
the business database. The application storage owner retains those duties.
|
||||
|
||||
The adapter sends explicit `knowledge` and `external` domain calls. Search
|
||||
signals travel as transport options, not serialized arguments. The port's
|
||||
`Awaitable` return type permits synchronous owner-local test implementations;
|
||||
the production adapter always returns the storage client's promises.
|
||||
|
||||
Main awaits source, document, task, graph, embedding and external metadata
|
||||
operations. IPC mutations await their writes before returning or refreshing a
|
||||
snapshot. Parsing, models, credentials and source watchers remain in their
|
||||
existing Main services. Embedding batches retain their existing sizes and
|
||||
staging/publication sequence.
|
||||
|
||||
Publication passes the extracted graph through foundation
|
||||
`publishDocument(..., { graph })`. Graph-only replacement calls
|
||||
`replaceDocumentGraph(library, document, result)`. Its implementation in
|
||||
`src/main/desktop-storage-knowledge-operations.ts` runs the existing synchronous
|
||||
graph algorithm inside `replaceEvidenceForDocument`'s transaction. No callback
|
||||
or SQL crosses the storage boundary.
|
||||
|
||||
Shutdown aborts producers immediately, drains tracked work and observes the
|
||||
task-interruption write and embedding status persistence. The service does not
|
||||
close the shared storage client. Embedding allocation returns its replacement
|
||||
ID even if cancellation arrives with the reply, allowing cleanup; a committed
|
||||
finish is not reported as rolled back. Rebuild admission remains occupied
|
||||
until its final status write settles, and status-write failures reach the
|
||||
completion barrier.
|
||||
|
||||
## Current Production Boundary
|
||||
|
||||
The following boundary is implemented in the current production composition:
|
||||
|
||||
- `replaceDocumentGraph(library: KnowledgeBase, document: Document,
|
||||
result: GraphExtractionResult): void` to the knowledge domain contract and
|
||||
dispatch it to the exported owner-local operation, passing the owned database
|
||||
as its first argument. It is not part of the concrete database method
|
||||
allowlist because it is a composite owner operation.
|
||||
- Foundation publication reuses the exported `storeExtractedGraph` helper.
|
||||
- `StorageData` preserves the existing recursive JSON type without rebuilding it
|
||||
in the adapter.
|
||||
- Persistent document lookup is resolved by the storage owner through the
|
||||
Knowledge domain; Main does not register a second lookup.
|
||||
- Application composition injects the port, awaits asynchronous gateway and
|
||||
tool-provider calls, and closes shared storage after Knowledge disposal.
|
||||
|
||||
## Validation
|
||||
|
||||
The focused Knowledge service, external service, embedding coordinator and
|
||||
embedding repository suites use a Promise-backed adapter over a test-owned
|
||||
SQLite connection. Coverage includes publication rollback, embedding batch
|
||||
ordering, late cancellation, replacement cleanup, external request cancellation,
|
||||
credential persistence and restart. The production gateway now awaits its library
|
||||
and search operations. Remaining evidence is limited to the focused suites and
|
||||
does not establish full concurrent desktop acceptance.
|
||||
|
||||
Owned production files pass ESLint. The current worktree typecheck and focused
|
||||
storage validation cover the injected production port and owner dispatch. Global
|
||||
architecture acceptance, packaged checks and concurrent performance acceptance
|
||||
remain with the parent delivery; these tests do not establish a new performance
|
||||
baseline.
|
||||
Reference in New Issue
Block a user