Skip to content

fix: report measured AI usage and stale snapshots accurately - #446

Merged
devSejung merged 1 commit into
mainfrom
fix/usage-report-record-provenance
Nov 5, 2026
Merged

devSejung merged 1 commit into
mainfrom
fix/usage-report-record-provenance

Conversation

@devSejung

Copy link
Copy Markdown
Owner

Closes #445. Follow-up to #435.

What Problem This Solves

Fixes an issue where employee and model usage reports could count display-only transcript notices as model calls, display unmeasured token usage as a measured zero, or report an expired snapshot as complete. A deterministic reproduction against main returned 31 calls and zero tokens for 31 display notices.

Why This Change Was Made

Classify usage where the transcripts are written and interpreted, then rebuild the derived SQLite rollups under a new rollup version. Count eligible retained usage records, distinguish explicit zero from missing telemetry, and mark a retained Gateway snapshot stale until refresh completes. The existing report routes, query fields, and SQLite schema are unchanged.

User Impact

  • A display-only record does not contribute a model call. A real retained reply with missing telemetry contributes a call and reports null tokens with UNKNOWN_TOKEN_USAGE. An explicitly measured zero remains numeric zero.
  • Individual and model queries flag expired cached data with USAGE_CACHE_STALE; complete data becomes available after a successful refresh. CSV still requires complete data.
  • The rollout rebuilds derived caches automatically. It cannot recover provider telemetry or history that was never retained.

AI-assisted change.

Evidence

Validated in task-owned Ubuntu Linux Docker with Node 24.18.0 and pnpm 11.15.1. The checked source is commit 1b97734 on base 3cd6c8b; its full-index patch SHA-256 is 24a04cfad23919c2262ca64b221b6ca4bd7033617ac47e5321408ccdde8bb4eb.

  • Baseline and fixed SQLite/Gateway/HTTP reproduction: 31 display notices went from 31 calls / 0 tokens / complete to no model calls; measured 29 and measured zero stayed numeric; missing ACP/CLI telemetry returned null and partial; an expired snapshot returned partial and then complete after refresh. Partial data was not exported as CSV.
  • Focused Vitest coverage included usage reports, Gateway cache, transcript accounting, CLI/ACP producers, and Codex projection. The final affected runs passed 352 tests and the final projector lint cleanup passed 42 sibling tests. OpenAPI validation passed 20 examples and 25 references, plus 3 Markdown JSON examples.
  • Fresh autoreview on the exact committed patch was scoped-clean through P2. Codex completion provenance was checked against upstream ad020f29aeda36ceac4f7c75fc6b5c7e00ab6f7b: codex-rs/core/src/session/turn.rs:2299, core/src/session/mod.rs:3692, app-server/src/bespoke_event_handling.rs:1054.
  • PlatformClaw quick wrapper: OPENCLAW_CHECK_CHANGED_REMOTE_CHILD=1 node scripts/platformclaw-check.mjs --changed --quick, exit 0.
  • Production build: node scripts/build-all.mjs, exit 0 in 4m05.9s; corepack pnpm --dir packages/platformclaw-coding-agent-contract build and corepack pnpm --dir packages/platformclaw-control-plane build both exited 0.
  • Built-process HTTP smoke: separate built Gateway and Control Plane processes using production WebSocket service identity pairing, fresh SQLite state, and synthetic records. Exit 0. Verified no-token employee lookup, exact provider/model filtering, sum conservation across models, Korean calendar month boundaries, CSV, and preserved private Gateway authentication. Display notices were excluded; explicitly measured zero remained numeric and unknown telemetry stayed null. With a real 31-second wait, the report served the expired 25-token snapshot as partial with USAGE_CACHE_STALE, then returned complete after refresh. Partial reports returned CSV 503.
  • PlatformClaw full changed-surface wrapper: OPENCLAW_CHECK_CHANGED_REMOTE_CHILD=1 node scripts/platformclaw-check.mjs --changed, exit 0. Includes core and plugin typechecks for production and tests, lint, formatting, dependency and SDK checks, and native schema/storage/runtime guards.
  • Net production growth: 75 lines to carry measured/unknown provenance across runtime writers, interpret retained records once, and project cache freshness. Test growth: 247 lines; documentation growth: 6 lines.

The corporate deployment and its historical data were not accessed. The deterministic fixture proves the code behavior, not the cause of any individual production incident.

@devSejung
devSejung marked this pull request as ready for review November 5, 2026 09:12
@devSejung
devSejung merged commit 0a5f031 into main Nov 5, 2026
106 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: AI usage reports can show unmeasured or stale data as complete

1 participant