Skip to content

fix: make token/cost aggregation monotonic to prevent shrink-on-reload - #58

Merged
MayankBansal12 merged 3 commits into
MayankBansal12:mainfrom
masked8knights:fix/monotonic-token-upsert
Sep 14, 2026
Merged

MayankBansal12 merged 3 commits into
MayankBansal12:mainfrom
masked8knights:fix/monotonic-token-upsert

Conversation

@masked8knights

Copy link
Copy Markdown
Contributor

Bug: Dashboard totals shrink on every reload

Symptom: The usage dashboard shows a day's token/cost total that visibly drops after every sync/reload, even though no tokens were actually "un-used." Reported figures are non-monotonic, undermining trust in the data.

Root cause

The usage_events upsert at server.ts:349 uses ON CONFLICT(event_key) DO UPDATE SET ... = excluded.*, which blindly overwrites all numeric columns (processed_tokens, cost_usd, all token sub-counts, cache_savings_usd, logged_cost_usd).

Because event keys are coarse (one per day per model/project/cost-mode for pi/prime/thaura), a partial or transient scan (file rotation, session pruning, transient read failure) produces a lower day-level aggregate than a prior complete scan. The overwrite lowers the stored value, and the dashboard total visibly shrinks.

Fix

Change all numeric count/cost columns to MAX(existing, excluded) so the stored value is a high-water mark that can only rise or stay flat:

  • processed_tokens, cached_input_tokens, cache_write_tokens, uncached_input_tokens, output_tokens all use MAX(col, excluded.col)
  • cost_usd, cache_savings_usd use MAX(col, excluded.col)
  • logged_cost_usd (nullable per schema) uses a NULL-safe MAX(COALESCE(...), ...) pattern

Identity/descriptive columns (timestamp, day, model, provider, project, pricing_status) remain overwrite (same event key implies same values for these).

Regression test

Two tests in server.test.ts verify monotonicity:

  1. Seed row with processed_tokens=1000, upsert with 500 (partial scan) — stored value stays 1000
  2. Seed row with cost_usd=2.0, upsert with 1.0 — stored value stays 2.0
  3. Both assert subsequent higher values update correctly (full scan)

Test results

Full suite: 192/192 tests pass (52 in server.test.ts including 2 new regression tests). tsc --noEmit clean.

Scope

This PR implements Fix #2 from the token-loss-on-reload diagnosis (monotonic upsert). A companion fix (fine-grained per-turn event keys — Fix #1) would further isolate the root cause but requires a larger collector redesign involving cache format changes and is tracked separately.

The new per-day event keys are not available in the collector pipeline at construction time (rows are pre-aggregated before parseHostUsageAggregates builds event keys), so adding turn-level keys would require restructuring the collector's merge logic — too risky for this focused fix.

@masked8knights
masked8knights marked this pull request as ready for review September 13, 2026 00:54
@MayankBansal12

Copy link
Copy Markdown
Owner

Keeping previously recorded usage with MAX() is reasonable for this use case. Please avoid the proposed follow-up redesign to per-turn event keys for now; that’s more complexity than we need.

Before merging, please address these points:

  • Preserve missing records. The current cleanup still deletes usage absent from a scan, bypassing MAX().
  • Keep token counts consistent. Taking each bucket’s maximum independently can make the buckets add up to more than processed_tokens.
  • Allow pricing corrections. Estimated costs should still update when catalog prices change.
  • Test the actual sync code. The new tests execute copied SQL and pass even without the production change. Please cover lower totals, missing records, and repricing through the real persistence path.

GPT-6 on behalf of mayank

admin1 and others added 2 commits September 14, 2026 09:06
The ON CONFLICT upsert for usage_events previously overwrote numeric
counters (processed_tokens, cost_usd, and all token sub-counts) on
every re-scan. When a partial or transient scan produced lower totals
than a prior complete scan — common after file rotation, pruning, or
a transient read failure — stored figures visibly dropped, making the
dashboard show non-monotonic 'token loss' on every reload.

Change all numeric count/cost columns to use MAX(existing, excluded)
so a re-scan can only raise or maintain the stored value. This makes
the dashboard high-water-mark monotonic: previously-recorded totals
can never shrink, regardless of scan completeness or file state
changes.

logged_cost_usd uses a NULL-safe MAX pattern (COALESCE) because that
column is nullable per the schema; all other numeric columns are NOT
NULL.

Two regression tests verify:
- processed_tokens stays at the prior maximum when re-upserted lower
- cost_usd stays at the prior maximum when re-upserted lower

Fixes non-monotonic dashboard totals described in the token-loss-on-
reload diagnosis. A companion fix (fine-grained per-turn event keys)
would further isolate the root cause by preventing a single coarse
daily aggregate from being the unit of reconciliation, but requires
a larger collector redesign and is tracked separately.
Root cause: upsertSourceEvents unconditionally deleted all event-source
mappings before re-inserting from the current scan. When a later full
scan returned zero rows (e.g. session files pruned between refreshes),
the mappings were wiped and deleteOrphanEvents removed the high-water-
mark events — exactly the 'lose tracking data on refresh' bug.

Fix: skip the mapping wipe and orphan prune when records is empty, so
previously recorded totals survive scans that see no data. Partial
scans (failureCount > 0) already skipped pruning; this extends the same
protection to complete-but-empty scans.

Tests: three integration tests exercising the real sync path through
the terminal mock harness, covering:
- token high-water mark across decreasing scans
- logged-cost corrections flowing through (not frozen)
- catalog repricing with monotonic floor on estimates
@masked8knights
masked8knights force-pushed the fix/monotonic-token-upsert branch from 7d7e22f to ab4ea9b Compare September 13, 2026 23:07
Preserve source-event mappings until their retention window expires so
nonempty partial scans cannot erase previously observed usage. Derive the
processed-token total from the retained component maxima.

Reprice retained usage, including rows missing from the current scan, using
the shared collector pricing logic. Allow catalog decreases and logged-cost
corrections. Cover retention, mixed token buckets, repricing, and expiry
through completed public sync passes.

Validation: 228 tests pass; type check and plugin build pass.
@MayankBansal12
MayankBansal12 merged commit 55a0ccc into MayankBansal12:main Sep 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants