Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion packages/redrob/src/session/overflow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,13 @@ export function usable(input: {
* either way and the threshold applies to what is left.
*/
const output = ProviderTransform.maxOutputTokens(input.model, input.outputTokenMax)
const headroom = Math.max(0, budget - Math.min(COMPACTION_BUFFER, output))
/*
* At most a quarter of the window. A 32,768-token model with the full 20,000 reserve was left
* 8,937 tokens, less than the system prompt and tool list alone, so every turn overflowed: a
* one-line question was summarised and auto-continued five times in one `run`.
*/
const reserve = Math.min(COMPACTION_BUFFER, output, Math.floor(budget / 4))
const headroom = Math.max(0, budget - reserve)
const windowLimit = Math.floor((headroom * clamped) / 100)
/*
* The cheaper of the two limits wins. `reserved` above returns before this on purpose: a caller who set
Expand Down
6 changes: 4 additions & 2 deletions packages/redrob/test/session/threshold-and-cost.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,10 @@ describe("compaction threshold", () => {

it("keeps room for the reply, whatever the threshold", () => {
// A threshold of 100 must not mean "compact once the window is full", which is the failure the
// reserve existed to avoid.
expect(source).toContain("budget - Math.min(COMPACTION_BUFFER, output)")
// reserve existed to avoid. The reserve is capped at a quarter of the window so a 32k model is not
// left with less room than its own system prompt.
expect(source).toContain("Math.min(COMPACTION_BUFFER, output, Math.floor(budget / 4))")
expect(source).toContain("budget - reserve")
})

it("treats a published input cap of 0 as absent, not as a cap of zero", () => {
Expand Down
Loading