Repository navigation
Description
Problem Statement
When LCM summarizes a session, it can ask the provider for a fractional output budget. A provider that validates max_tokens as an unsigned integer rejects the call outright, and the ladder then falls to the deterministic truncate, so the user's memory of that span is clipped text rather than a summary.
Observed with the shipped defaults (leafChunkTokens: 3000), no unusual configuration, in one week of real Pi sessions (2026-10-30 to 2026-11-06, 191 sessions, 4,830 summarizer calls):
- 588
summarizer-errorrows, of which 540 are this class:deepseek/deepseek-flash: 422: Failed to deserialize the JSON body into the target type: max_tokens: invalid type: floating point 5543.5, expected u32. - 5,358 summaries stored that week: 3,881 at level 1, 162 at level 2, and 1,315 at level 3 (24.5%). Level 3 is the deterministic truncate, so a quarter of the user's memory was mechanically clipped.
- Quality tiers confirm it: 1,198 leaves stored
mechanicaland 381clipped, against 2,707clean.
The user never sees a failed summary. They see a session that keeps working but recalls less, which is the exact failure mode LCM exists to prevent.
The fractional value comes from the leaf-chunk path dividing the chunk budget in half: leafChunkBudget / 2. The chunk budget is a calibrated integer (scaledBudget(leafChunkTokens, charsPerToken)), and the calibration is per session (observed 3.9 to 5.22 chars per token), so the halved target is fractional whenever the calibrated budget is odd. A leaf target of 1447.5 becomes 1447.5 + 4096 (reasoning headroom) = 5543.5 in the request.
It is route dependent, which is why it stayed hidden: openrouter coerced the same fractional budget into 415 calls with zero rejections, while the direct DeepSeek route rejects every one. Any provider that validates the field strictly will fail the same way, and a reasoning model adds an integer headroom, so the fraction survives all the way to the wire.
Solution
Every budget handed to a provider is an integer. The conversion happens once, at the single seam where a completion request is built, so no call site (leaf, condensed, tiled chunk, heal, manual pass, either adapter) can put a fractional or non-finite value on the wire. A provider that still fails for its own reasons keeps the current behavior: the ladder falls to the deterministic truncate and the metric row records it.
User Stories
- As a Pi user on a DeepSeek model, I want every summarizer call to be accepted by the provider, so that my session memory is summarized rather than truncated.
- As a Pi user, I want a summary that LCM pays for to be stored as a summary, so that the tokens I spent produce memory instead of clipped text.
- As a Pi user on a reasoning model, I want the reasoning headroom to never turn a valid budget into an invalid request, so that the headroom stays an implementation detail.
- As a Pi user whose session calibrates chars per token to an unusual value, I want the budget math to be robust to any calibration, so that quality does not silently depend on a rounding coincidence.
- As a Pi user on a provider that coerces malformed fields, I want the same behavior as on a provider that rejects them, so that switching routes does not change my memory quality.
- As an OpenCode user, I want the same guarantee through the shared engine, so that the two adapters cannot drift on this.
- As a Pi user running
/lcm heal --apply, I want heal calls to carry integer budgets too, so that a repair pass does not fail the same way. - As a Pi user whose compaction runs while I am not looking, I want the async pass to be silent and correct, so that I never have to diagnose a background failure.
- As a Pi user with a large session, I want the condensed tier to inherit the same guarantee, so that deeper nodes are not the remaining hole.
- As a maintainer, I want one seam that owns the provider contract, so that a future budget producer cannot reintroduce the bug.
- As a maintainer, I want the seam to round rather than floor, so that a budget is not silently shrunk below what the caller asked for.
- As a maintainer, I want a zero or negative budget never to be sent, so that a malformed config cannot produce an empty or invalid request.
- As a maintainer, I want a regression test at the adapter boundary, so that the assertion is about what a provider receives, not about how a target was computed.
- As a maintainer, I want a pass-level test with an odd chunk budget, so that the end-to-end path is covered and not only the helper.
- As a maintainer, I want the failure to reproduce red against a strict host double before the fix, so that the test proves the bug rather than the implementation.
- As a maintainer, I want the metric rows unchanged, so that existing dashboards and
/lcm reporttotals stay comparable. - As a maintainer, I want a changeset for the fix, so that the patch reaches users through the release PR.
- As a person reading the changelog, I want the entry to say what a user loses today, so that the release note is a user-facing fact instead of an internal one.
Implementation Decisions
- One seam: the request builder in the core summarizer port. Every summarizer call from every adapter passes through it, including the heal wrapper and the OpenCode pass. Coerce the budget there rather than at each division site, because the division sites are semantically correct (half a chunk is the intended target) and the provider contract is what actually needs enforcing.
- Rounding, not flooring. Round to nearest, because the caller asked for that many tokens and flooring a small budget loses meaningful capacity. Clamp to at least 1 so a zero or negative budget cannot become an empty request.
- Non-finite budgets are not passed through. A budget that is not a finite number is coerced to the same floor, so no
nullorInfinitycan be serialized into a request. - The reasoning headroom stays where it is. It is already an integer, so rounding before or after adding it produces the same result; keeping it after the coercion makes the invariant obvious in one line.
- No change to the chunking math, the ladder, or the quality tiers. A genuinely dead summarizer must still fall to level 3 and be stored
mechanical; that behavior is correct and the current metrics depend on it. - No new metric row kind and no field changes to the existing rows. The
threwvariant ofsummarizer-erroralready carries the provider text, which names the offending value, so no schema change is needed to diagnose this class next time. - No config surface change. Nothing becomes tunable, nothing becomes deprecated, and no migration is needed. Defaults keep the same meaning.
- The adapters are not touched. Both reach the seam through the core port, and the OpenCode pass builds no request of its own.
- A changeset is required, because this is user-visible behavior under the repo's own release policy.
Testing Decisions
- A good test asserts what a provider receives, not how a target was computed. The observable contract is: every
CompletionRequestthat leaves the port carries a non-negative integermaxTokens. - Seam test, highest available: drive the summarizer port with a host double that records requests (the existing fake host already records
model,promptandmaxTokens), and assertNumber.isIntegeron what it recorded. Prior art: the port test that asserts an exactmaxTokensfor a plain call. - Red first: a strict host double that rejects a non-integer budget the way the provider did. The new test must fail against the current code before the coercion exists, which is what proves the reproduction.
- Pass-level regression: run a compaction pass over a store seeded with enough messages for at least two leaves, with an odd chunk budget, and assert every recorded request is an integer. Prior art: the pass-level tests that already pass a small
leafChunkTokensthrough the stored-pass helper. - Ladder coverage: leaf, tiled chunk, condensed and heal all reach the same seam, so one parameterized test over those stages covers the surface without duplicating the assertion.
- Boundary values: zero, one, an odd integer, a fractional value ending in
.5, and a non-finite value. These are the values a config or a calibration can realistically produce. - No test asserts on an internal variable or a specific call site, so a future refactor of the chunking math does not have to rewrite the suite.
Out of Scope
- Retry, backoff or fallback policy for a provider that rejects a call for an unrelated reason.
- Provider-specific quirks beyond integer validation, including the other failure categories seen in the same window (insufficient funds, auth, missing session id, stale context).
- The mechanical tier for noisy tools and the recall upgrades that are already in the unreleased 0.1.3 release PR.
- Changing any default budget, including the chunk budget whose parity triggers the bug.
- Making the calibrated chunk budget always even. That would fix the symptom at one call site and leave every other producer exposed.
- Any change to how chars per token is calibrated.
Further Notes
Reproduction
- Seed a store with enough messages for at least two leaf chunks.
- Pass an odd
leafChunkTokens(for example2895, the calibration-adjusted form of the default3000at roughly 3.86 chars per token, or any value whose scaled budget is odd). - Record what the host receives. The leaf target is half of the chunk budget, so the request carries
leafChunkBudget / 2 + 4096, and the observed wire values were5543.5,5453.5,5487.5,5663.5. The.5is the whole bug. - Repeat against a host double that validates integer-only budgets the way the provider does, and observe the throw, the ladder falling to the truncate, and the stored
mechanicalleaf.
Wire evidence, one week, 191 sessions
- 540 of 588 summarizer errors:
422 ... max_tokens: invalid type: floating point 5543.5, expected u32, all on the direct DeepSeek route. - The same week on the OpenRouter route: 415 calls, zero rejections of this class, which is why the failure looked intermittent.
- Stored levels that week: 3,881 level 1, 162 level 2, 1,315 level 3.
Where the seam sits today
- The request builder in the core summarizer port, which every adapter and the heal wrapper reach. The
maxTokensit emits is the only value that reaches a host. - The two division sites that produce the fractional input are both in the leaf-chunk path of the compaction engine.
- Current line numbers, for orientation only: request builder in the core LLM port, division sites in the compaction engine. The tests above do not depend on either location.
Release state
The open release PR (version packages, pi-lossless 0.1.3) does not carry this fix. It is still present on main.
Activity
pullfrog commentedon Nov 6, 2026
No duplicates found. Searched open issues for summarizer, max_tokens, truncate, compaction and LCM terms, and listed all open issues. The only matches are #16, #17 and #18, which each name #15 as Parent and split this fix into slices. They are related work, not duplicates.
Related context: #16 owns the seam fix, #17 owns the pass level regression, #18 owns the changeset. This issue stays as the parent tracker.
Plan verified against code:
packages/core/src/llm.ts:435requestOf, the single builder every adapter and heal reaches. ReplacemaxTokens + headroomwithcoerced + headroomwherecoerced = !Number.isFinite(maxTokens) ? 1 : Math.max(1, Math.round(maxTokens)). This coverspackages/core/src/compaction-engine.ts:324and:416(leafChunkBudget / 2) plus condensed, tiled chunk and heal without touching chunking math, ladder, tiers, metrics or config.REASONING_HEADROOMstays in place.makeLlmwith a recording host double plus a strict double that throws on non-integermaxTokens. AssertNumber.isIntegeron every recordedCompletionRequest(packages/core/src/host.ts:41). Cover fractional.5, zero, negative,NaNandInfinity, reasoning on and off, and confirm integer inputs are unchanged.leafChunkTokens, assert all recorded requests are integers across leaf, condensed and heal stages.vp check,bun run test,bun run typecheckandbun run pack.muse-spark-1.3-contributor| 𝕏SutuSebastian commentedon Nov 6, 2026
Implemented on
mainas0c70748, not pushed yet.leafChunkTokensof 2895. Reproduced independently: 10 of 12 pass-level tests fail with the source change stashed..changeset/whole-token-budgets.md, patch on all three packages, so it rides the open release PR.Gates on the committed revision: 901 tests across 68 files, typecheck clean,
vp checkclean,bun run packbuilds. Independent review bycodex/gpt-6.1-solreturned APPROVE with no findings.Staying open until a published version carries the fix, since the user-visible symptom lasts until then.
fix: whole token budgets at the summarizer port
SutuSebastian commentedon Nov 6, 2026
0.1.3 carries the fix. The published tarballs for lossless-core, pi-lossless and opencode-lossless resolve whole provider budgets at the single
requestOfseam, and the pass-level regression tests cover leaf, tiled, condensed and heal requests. npmlatestis 0.1.3 for all three; tags and GitHub releases were created from 9dedf1f.