fix: make token/cost aggregation monotonic to prevent shrink-on-reload - #58
Merged
MayankBansal12 merged 3 commits intoSep 14, 2026
Merged
MayankBansal12 merged 3 commits into
MayankBansal12 merged 3 commits into
Conversation
masked8knights
marked this pull request as ready for review
September 13, 2026 00:54
Owner
|
Keeping previously recorded usage with Before merging, please address these points:
|
The ON CONFLICT upsert for usage_events previously overwrote numeric counters (processed_tokens, cost_usd, and all token sub-counts) on every re-scan. When a partial or transient scan produced lower totals than a prior complete scan — common after file rotation, pruning, or a transient read failure — stored figures visibly dropped, making the dashboard show non-monotonic 'token loss' on every reload. Change all numeric count/cost columns to use MAX(existing, excluded) so a re-scan can only raise or maintain the stored value. This makes the dashboard high-water-mark monotonic: previously-recorded totals can never shrink, regardless of scan completeness or file state changes. logged_cost_usd uses a NULL-safe MAX pattern (COALESCE) because that column is nullable per the schema; all other numeric columns are NOT NULL. Two regression tests verify: - processed_tokens stays at the prior maximum when re-upserted lower - cost_usd stays at the prior maximum when re-upserted lower Fixes non-monotonic dashboard totals described in the token-loss-on- reload diagnosis. A companion fix (fine-grained per-turn event keys) would further isolate the root cause by preventing a single coarse daily aggregate from being the unit of reconciliation, but requires a larger collector redesign and is tracked separately.
Root cause: upsertSourceEvents unconditionally deleted all event-source mappings before re-inserting from the current scan. When a later full scan returned zero rows (e.g. session files pruned between refreshes), the mappings were wiped and deleteOrphanEvents removed the high-water- mark events — exactly the 'lose tracking data on refresh' bug. Fix: skip the mapping wipe and orphan prune when records is empty, so previously recorded totals survive scans that see no data. Partial scans (failureCount > 0) already skipped pruning; this extends the same protection to complete-but-empty scans. Tests: three integration tests exercising the real sync path through the terminal mock harness, covering: - token high-water mark across decreasing scans - logged-cost corrections flowing through (not frozen) - catalog repricing with monotonic floor on estimates
masked8knights
force-pushed
the
fix/monotonic-token-upsert
branch
from
September 13, 2026 23:07
7d7e22f to
ab4ea9b
Compare
Preserve source-event mappings until their retention window expires so nonempty partial scans cannot erase previously observed usage. Derive the processed-token total from the retained component maxima. Reprice retained usage, including rows missing from the current scan, using the shared collector pricing logic. Allow catalog decreases and logged-cost corrections. Cover retention, mixed token buckets, repricing, and expiry through completed public sync passes. Validation: 228 tests pass; type check and plugin build pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug: Dashboard totals shrink on every reload
Symptom: The usage dashboard shows a day's token/cost total that visibly drops after every sync/reload, even though no tokens were actually "un-used." Reported figures are non-monotonic, undermining trust in the data.
Root cause
The
usage_eventsupsert atserver.ts:349usesON CONFLICT(event_key) DO UPDATE SET ... = excluded.*, which blindly overwrites all numeric columns (processed_tokens, cost_usd, all token sub-counts, cache_savings_usd, logged_cost_usd).Because event keys are coarse (one per day per model/project/cost-mode for pi/prime/thaura), a partial or transient scan (file rotation, session pruning, transient read failure) produces a lower day-level aggregate than a prior complete scan. The overwrite lowers the stored value, and the dashboard total visibly shrinks.
Fix
Change all numeric count/cost columns to
MAX(existing, excluded)so the stored value is a high-water mark that can only rise or stay flat:processed_tokens,cached_input_tokens,cache_write_tokens,uncached_input_tokens,output_tokensall useMAX(col, excluded.col)cost_usd,cache_savings_usduseMAX(col, excluded.col)logged_cost_usd(nullable per schema) uses a NULL-safeMAX(COALESCE(...), ...)patternIdentity/descriptive columns (timestamp, day, model, provider, project, pricing_status) remain overwrite (same event key implies same values for these).
Regression test
Two tests in
server.test.tsverify monotonicity:processed_tokens=1000, upsert with 500 (partial scan) — stored value stays 1000cost_usd=2.0, upsert with 1.0 — stored value stays 2.0Test results
Full suite: 192/192 tests pass (52 in server.test.ts including 2 new regression tests).
tsc --noEmitclean.Scope
This PR implements Fix #2 from the token-loss-on-reload diagnosis (monotonic upsert). A companion fix (fine-grained per-turn event keys — Fix #1) would further isolate the root cause but requires a larger collector redesign involving cache format changes and is tracked separately.
The new per-day event keys are not available in the collector pipeline at construction time (rows are pre-aggregated before
parseHostUsageAggregatesbuilds event keys), so adding turn-level keys would require restructuring the collector's merge logic — too risky for this focused fix.