Skip to content

Savings ledger 4/6: savings setting, stretch tracking and server wiring - #32

Merged
cris005 merged 1 commit into
mainfrom
savings/04-engine
Nov 2, 2026
Merged

cris005 merged 1 commit into
mainfrom
savings/04-engine

Conversation

@cris005

@cris005 cris005 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Part 4 of 6. This is where the ledger starts counting. The sidebar rows come in #33.

The savings setting

  • Where it's set: default on. In the picker (Session and Global), as /cache-manager savings on|off, and as a plugin option. Turning it off hides the rows, but the ledger keeps counting.
  • modelsDevPrices option: new, default off. It turns on the models.dev price fallback.

What counts as a save

An idle stretch counts as one avoided cache rewrite when all three hold:

  • at least one ping in the stretch was a hit;
  • the next real primary or compaction request is sent more than 30 s after the cache would have expired without pings;
  • that response is ok and reads at least 90% of the prior context from cache.

Every ambiguous case counts less:

  • hybrid: only the part past the long-lived prefix counts. The prefix is the largest 1h write on Claude, or the first primary context on GPT-5.6+. If the prefix size is unknown, no save is counted.
  • Return turn: a failed or overtaken return turn drops the save.
  • Restart: a restart mid-stretch never produces a save.
  • Ping charges: a ping is charged at the TTL it was sent on. A ping answered after its stretch ended still counts against that stretch.

Wiring

  • Engine: reports to the ledger fire-and-forget. Snapshots gain showSavings and savings. The readout loads only for managed, non-child sessions with savings shown.
  • Middleware: passes split, model and reportedModel to the engine.
  • Server:
    • Builds the PriceBook, models.dev fallback and ledger.
    • Prefetches prices with void prices.prefetch(), so nothing on the request path waits for a price.
    • Ledger changes refresh the snapshot.
    • One startup pass sweeps the session/ and ledger/ keys of deleted sessions.
  • Capture sample: the sample snapshot gains the two new fields.

Testing

  • Engine tests: every rescue edge case.
  • Server tests:
    • restart mid-stretch;
    • a catalog that never loads;
    • models.dev on and off;
    • the sweep.

Before release

  • Confirm at runtime that a cost declared on a custom provider's model in opencode.jsonc comes back from ctx.model.list().

Stack created with GitHub Stacks CLI • Give Feedback 💬

@cris005
cris005 added this pull request to stack #35 October 31, 2026 13:27
@cris005 cris005 changed the title feat(savings): savings setting, engine stretch tracking and server wiring Savings ledger 4/6: savings setting, stretch tracking and server wiring Oct 31, 2026
@cris005
cris005 force-pushed the savings/04-engine branch 2 times, most recently from bdbc692 to 03de60c Compare November 2, 2026 03:59
Base automatically changed from savings/03-ledger to main November 2, 2026 03:59
…ring

Adds the `savings` setting (default on; picker, `/cache-manager savings
on|off`, plugin option) and the `modelsDevPrices` option. The keepalive
engine tracks idle stretches, charges each ping at the TTL it fired on,
and reports a rescue when a return sent past the original expiry plus a
30 s margin reads the cache warm; on `hybrid` only the part past the
prefix counts. Snapshots carry `showSavings` and the ledger readout. The
server builds the PriceBook, models.dev fallback and ledger, prefetches
prices off the request path, and sweeps settings and ledger keys of
deleted sessions in one pass.
@cris005
cris005 merged commit 1c4910f into main Nov 2, 2026
2 checks passed
@sonarqubecloud

sonarqubecloud Bot commented Nov 2, 2026

Copy link
Copy Markdown

@cris005
cris005 deleted the savings/04-engine branch November 2, 2026 05:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant