Repository navigation
Conversation
Large ledger reads and cost aggregation ran synchronously inside WebUI requests, blocking event delivery and other HTTP routes. Run scans in a serialized, cancellable worker queue, coalesce concurrent identical queries, and show byte size without rescanning in storage listings. Preserve accounting, pending records and corrupt-line diagnostics. Full suite: 5211 passed, 4 skipped; typecheck and packed-package query passed.
Member
|
Thanks @jcs130 for the measurements. Two questions before we decide:
Two concerns on our side:
The PR stays open until we hear back. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Querying usage or opening the storage list synchronously scanned the whole ledger on the engine thread. Large ledgers could therefore stall event delivery and unrelated HTTP requests.
Run usage scans and cost aggregation in a worker, serialize different queries, share identical concurrent queries, and cancel outstanding scans at engine shutdown. Storage listing reads file size. Accounting, pending appends, duplicate IDs, and corrupt-line diagnostics are preserved; read failures are returned as failures. The packaged runtime declares and includes its worker loader.
Validation: 5,211 tests passed, 4 skipped; typecheck; 14 focused packaging/query/WebUI tests; an independently extracted package completed the worker query. An isolated 65.8 MB synthetic ledger returned identical totals; maximum main-thread timer gap was 123 ms before and 12 ms after. Total query time was 113 ms before and 207 ms after, so this demonstrates responsiveness rather than faster aggregation.