Skip to content

Savings ledger: show what keepalive pings cost and save (0.7.0) - #28

Closed
cris005 wants to merge 27 commits into
mainfrom
feature/savings-ledger
Closed

cris005 wants to merge 27 commits into
mainfrom
feature/savings-ledger

Conversation

@cris005

@cris005 cris005 commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Adds a per-session savings ledger to the sidebar. It shows what keepalive pings cost and what they saved, so you can see the plugin paying for itself. Version 0.7.0.

With a price                          Without a price
  pings     2/5 ✓ hit ($0.05)           pings     2/5 ✓ hit
  saved     $0.42 · 2 rewrites          saved     2 rewrites · 282k
  return    9× ping cost                return    9× ping cost

What counts as a save

An idle stretch counts as one avoided cache rewrite when all three hold:

  • at least one ping in the stretch was a hit;
  • the next real primary/compaction request is sent more than 30 s after the cache would have expired without pings;
  • that response is ok and reads at least 90% of the prior context from cache.

Every ambiguous case counts less:

  • hybrid: only the short-lived part counts. The prefix is the largest 1h write on Claude, or the first primary context on GPT-5.6+. If the prefix size is unknown, no save is counted.
  • Return turn: a failed or overtaken return turn drops the save.
  • Restart: a restart mid-stretch never produces a save.
  • Unpriced events: one unpriced event switches the session to unit-free for good.

Prices

  • Catalog first: OpenCode's model catalog through ctx.model.list(), which includes a cost you declare in opencode.jsonc. The size-based tiers are honoured.
  • models.dev fallback: only with the new modelsDevPrices option, default off. The plugin makes no network calls of its own while it's off. With it on, prices are fetched once a day, cached in plugin storage, pruned to prices only, and fetched with a timeout.
  • Unit-free: used when neither source has a price.
  • Off the request path: nothing on the request path waits for a price. A catalog load that hangs counts as a failed load after 10 s. A failed catalog load makes events unpriced; models.dev doesn't stand in for it.

Changes

  • Output tokens: CacheUsage.outputTokens is parsed on all three wires.
  • New savings setting (default on): in the picker (Session and Global), as /cache-manager savings on|off, and as a plugin option. Off hides the rows; the ledger keeps counting.
  • New code: services/savings-math.ts, model-prices.ts, models-dev.ts, ledger-store.ts, savings-ledger.ts and fetch-json.ts, plus models/savings.ts and constants/savings.ts.
  • Keepalive engine:
    • It tracks idle stretches and detects saves, and reports them to the ledger fire-and-forget.
    • Snapshots gain showSavings and savings.
    • Separate fix (d4d3044): a stale implicit skip reason made the first ping after a learned alias go out unaligned. Wiring the ledger exposed it.
  • Storage: totals persist under ledger/<sessionID>. A single startup pass sweeps both session/ and ledger/ keys of deleted sessions.
  • Sidebar: saved / return rows and the ($…) stretch cost.
  • Images:
    • New sidebar states.
    • picker.png (a screenshot) is replaced by a generated picker.svg. It's built from the real settingsOptions(), with dimensions and colours taken from OpenCode v2.2.1's dialog-select and theme source.
  • Docs:
    • README Savings section, settings and commands.
    • UPGRADE "From 0.6.x".
    • A commented cost example in opencode.openai-compatible.jsonc.
    • CONTRIBUTING and CLAUDE.md layout updates.
    • OmniRoute caveat: the prompt_cache_breakpoint drop is fixed upstream (OmniRoute #18254 / PR #18255, merged 2026-10-27) but not in a release yet.

Before release

  • Compare docs/images/picker.svg with a real screenshot of the picker (dimensions, colours).
  • Confirm at runtime that a cost declared on a custom provider's model in opencode.jsonc comes back from ctx.model.list().
  • Merging bumps main to 0.7.0, which the release workflow tags and publishes.

Testing

bun run fmt:check, bun run lint, bun run typecheck and bun test all pass: 1122 tests, 0 failures.

  • Property tests: pricing math and ledger totals (net = avoided − pings, totals never decrease, unpriced never shows dollars, concurrent events give the same totals as sequential ones).
  • Engine tests: every rescue edge case.
  • Server tests: restart mid-stretch, a catalog that never loads, models.dev on/off, and the deleted-session sweep.

…s a TTL

requestSent committed a snapshot that still carried the previous turn's
implicit skip reason, so the server dropped the primary messages it had
just stored and the first ping after a learned alias went out unaligned.
The hero sample now carries a priced ledger, so sidebar.svg shows the
`($0.05)` suffix and the `saved` and `return` rows. sidebar-states.svg
gains four captioned states: savings in dollars, without a model price,
nothing saved yet, and savings turned off. The older states keep no
savings readout so each still shows only what it is about.

docs/images/picker.png, a manual screenshot, is replaced by picker.svg,
drawn by a capture-only lookalike of OpenCode's dialog.select
(scripts/capture/picker-dialog.tsx). Its rows come from the real
settingsOptions() with built-in defaults and no session override. Every
dimension and colour is taken from OpenCode v2.2.1 (tag v2.2.1, commit
3ebcf9a), legacy layout with the "new-designs" experiment off:
ui/dialog.tsx and ui/dialog-select.tsx for the 60-column width and
insets, and theme/assets/v2/opencode.json resolved for the dialog
surface for the colours.

freeze drops SGR bold, so the script now restores every bold run it
finds in the screen's ANSI rows (restoreBold), not just the sidebar's
section header.

Compared with the old screenshot: same rows, order, highlighted first
Session row and bold section headers, plus the new Savings rows. The
colours differ because the screenshot used another theme: the image
uses OpenCode's default dark theme (dialog #141414, selected row
#fab283 with #0a0a0a bold text, section headers #9d7cd8, muted text
#808080). The cursor is drawn as a #fab283 block over the first
placeholder letter (the real one is the terminal's). The dialog sits
on freeze's #0a0a0a padding rather than the dimmed session behind it.
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@cris005

cris005 commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Split into a stack for review: #29 → #30 → #31 → #32 → #33 → #34. The top of the stack has the same tree as this branch, plus fixes for the two SonarCloud findings.

@cris005 cris005 closed this Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant