Skip to content

fix(core): keep usage queries off the engine event loop - #406

Open
jcs130 wants to merge 1 commit into
Pal-AI-Lab:mainfrom
jcs130:codex/usage-query-offthread-20261112
Open

jcs130 wants to merge 1 commit into
Pal-AI-Lab:mainfrom
jcs130:codex/usage-query-offthread-20261112

Conversation

@jcs130

@jcs130 jcs130 commented Nov 12, 2026

Copy link
Copy Markdown

Querying usage or opening the storage list synchronously scanned the whole ledger on the engine thread. Large ledgers could therefore stall event delivery and unrelated HTTP requests.

Run usage scans and cost aggregation in a worker, serialize different queries, share identical concurrent queries, and cancel outstanding scans at engine shutdown. Storage listing reads file size. Accounting, pending appends, duplicate IDs, and corrupt-line diagnostics are preserved; read failures are returned as failures. The packaged runtime declares and includes its worker loader.

Validation: 5,211 tests passed, 4 skipped; typecheck; 14 focused packaging/query/WebUI tests; an independently extracted package completed the worker query. An isolated 65.8 MB synthetic ledger returned identical totals; maximum main-thread timer gap was 123 ms before and 12 ms after. Total query time was 113 ms before and 207 ms after, so this demonstrates responsiveness rather than faster aggregation.

Large ledger reads and cost aggregation ran synchronously inside WebUI requests, blocking event delivery and other HTTP routes. Run scans in a serialized, cancellable worker queue, coalesce concurrent identical queries, and show byte size without rescanning in storage listings. Preserve accounting, pending records and corrupt-line diagnostics. Full suite: 5211 passed, 4 skipped; typecheck and packed-package query passed.
@Phantivia

Copy link
Copy Markdown
Member

Thanks @jcs130 for the measurements.

Two questions before we decide:

  1. On how large a real ledger, and in which deployment, did you see a stall, and what did it look like in use? The numbers in the description come from a 65.8 MB synthetic ledger (longest main-thread gap 123 ms before, 12 ms after).
  2. Why does tsx have to enter PUBLISHED_DEPENDENCIES in scripts/publish-package.ts? It is a devDependency in the root package.json today, and this would make it a declared dependency of the published cortico package.

Two concerns on our side:

  • The storage list's usage row would show only the file size and lose the record count. That removes something users see and needs its own decision; the description mentions it only as "Storage listing reads file size".
  • Each query starts a new worker thread that loads usage-log.ts and cost.ts through tsImport. That adds a worker module, a published dependency, query queueing and shutdown cancellation, in exchange for a stall of about 100 ms on the 65.8 MB ledger. The change is wide for that gain.

The PR stays open until we hear back.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants