Skip to content

feat(usage): say why a stored usage row has no cost (usage unpriced) - #348

Open
mikehasa wants to merge 2 commits into
mainfrom
fix/usage-unpriced-report
Open

mikehasa wants to merge 2 commits into
mainfrom
fix/usage-unpriced-report

Conversation

@mikehasa

Copy link
Copy Markdown
Owner

Problem

unpriced was a dead end. When a stored usage row has no cost, nothing said why — whether the local catalog can still price it, whether no catalog row covers the model at all, or whether the row is excluded from pricing on purpose.

A real incident made that expensive: after the dsh pricing fix (#344, released in 0.12.7), a store still showed dsh / deepseek-flash rows as unpriced. The root cause was a running recorder older than the fix — its importer wrote new dsh rows without cost, while the installed CLI priced the same rows fine. Nothing in the product pointed at that; the same 8 rows became estimated_from_tokens the moment a current importer ran (usage import-local --refresh --estimate-costs), and the store's own scan receipts even carried an advisory watcher_version_mismatch ("Restart usage watch to load the current importer") that nobody connects to unpriced.

Change

agentacct usage unpriced [--store-dir] [--client] [--json] — a read-only inventory of stored rows with no cost, grouped by client, provider and model, with one provable reason per group:

  • Repairable — the local catalog covers the model. The report proves it by running the same stored-row reprice path the repair command uses (build_stored_unknown_cost_reprice_batch, nothing excluded) and only calls a row repairable when that path would really price it. It names the catalog row (priceable as deepseek/deepseek-flash for a dsh row reported under the client's own deepseek-official route) and prints the exact per-client command (agentacct usage import-local --refresh --estimate-costs --client dsh).
  • No catalog row — no price row covers the reported (provider, model) pair even through the provider aliases, so the rows stay unknown rather than guessed.
  • Excluded by design — with the blocking fact the row itself carries: non-additive usage, a client that reported no input/output split, a redacted value, or an ambiguous/conflicted identity.

Read-only: nothing is imported, repriced or written, and the store's existing pricing snapshot is used as-is (no network). Rows that already hold an amount are never listed.

Tests

  • tests/test_usage_unpriced.py (new, 6 tests): a dsh row whose client route resolves through the alias is reported repairable with the exact command, and the named repair command really prices it (end-to-end, then the report is empty); a model no catalog row covers is reported as such with no repair offered; excluded rows report their blocking fact (input_output_split_not_reported, non_additive); already-priced rows never appear; --client filters; a store with nothing unpriced says so.
  • Full suite: pytest -q → 4904 passed.

Notes

  • Docs: docs/usage-truth-table.md (the why/repair section under Pricing catalog) and docs/reference.md.
  • Follow-up, not in this PR: the macOS app renders unpriced from daemon-served data, so surfacing this reason in the Usage pane needs a daemon field/route first; usage unpriced --json is the shape that would feed it.

"unpriced" was a dead end: it never said whether the local catalog could still
price a row, whether no catalog row covers the model at all, or whether the row
is excluded from pricing on purpose. A running importer that predates a pricing
fix makes this worse — the rows it writes stay cost-unknown while the installed
CLI can price them, and nothing in the product pointed at the repair command.

`agentacct usage unpriced` groups stored unknown-cost rows by client, provider
and model and gives each group one provable answer: repairable, with the exact
per-client `usage import-local --refresh --estimate-costs` command and the
catalog row it would price from (`deepseek/deepseek-flash` for a dsh row
reported under the client's own `deepseek-official` route); no catalog row
covers the reported pair even through the provider aliases, so the rows stay
unknown rather than guessed; or excluded by design with the blocking fact
(non-additive usage, an unreported input/output split, a redacted value, or an
ambiguous identity).

Repairability is proved, not guessed: the report runs the same stored-row
reprice path the repair command uses, with nothing excluded, and only calls a
row repairable when that path would really price it. Read-only — nothing is
imported, repriced or written, and the store's existing pricing snapshot is
used as-is.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant