Repository navigation
Cache-stable requests, step 2: ContextSift keeps the prefix - #133
Merged
Merged
Conversation
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…udget Once what cannot be evicted reached the budget, every call evicted the step that had just finished, so the prompt prefix changed on every call (the 32K-window thrash). A pass now evicts only when at least a quarter of the budget can go, down to max(target, floor + slack). The system prompt is no longer part of the sifted size; it is taken off the budget instead, never below a quarter, so system + messages keep the same ceiling. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The receipt ledger lived in the system message, so every eviction batch changed the start of the request and the whole conversation missed the prompt cache. Each evicted step is now one assistant message in its own place (its text, then a bracketed receipt per call); a batch changes nothing before its first evicted step. The 20-line ledger cap is gone (ledgerEntries is kept, unused). Sifted tool results keep their images. A completed result is bounded (preview + artifact reference) only as part of a batch, after its first evicted step, or when it was already bounded while active: crossing into sifting alone no longer truncates a result the normal path sent whole. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ceipt, not ledger Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The child manager dropped readIdentityParams (so sim_ui/mac_ui reads were never protected), evictionTargetFraction and eagerPersistMinChars. childManager() copies every setting onto a fresh manager over the same store. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The stored history was trimmed to 80% of the window before ContextSift ran, which slid the window on every call once the stored history passed it; the token trim after each step deleted history for good. With a ContextManager the agent now sends from the whole history, falls back to the trimmed one only when the sifted request itself, tool definitions counted, passes the same 80% bound, and reports every trim (.historyTrimmed). The fallback counts removals against the stored history and keeps a per-call note (the progress nudge) after the trimmed history, so a nudge turn neither reports a phantom trim nor loses its nudge. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The net trimmed the RAW stored history to fit, so one breach dropped about twice the steps it needed, and it recomputed the cut on every later call, so the prefix moved by about a step per call. It now leaves out the fewest oldest whole steps that bring the SIFTED request (tool definitions counted) to a low watermark (overflowTargetFraction, 0.7 of the bound), remembers the cut by message id, and holds it until the request passes the bound again: consecutive calls send the same prefix. The step the model is waiting on, the task and the system prompt are never left out. Compaction clears the cut. The bound is ContextManager.overflowFraction (default 0.8, as before), so an app with its own compaction trigger can set it above that trigger. Every .historyTrimmed now carries a reason (fitToWindow, messageCap, tokenBudget, overflow); .overflow is the app's signal to compact. A net that cannot get under its bound logs a warning. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s commit nothing The net pinned the latest user message as "the task", so any mid-run nudge (loop, repair, verifier, plan, reasoning) made the real task the first unit dropped, and a nudge after the tool results unpinned the step the model was answering. The agent now records the task's id at run start and pins it, and trailing user-role nudges count as part of the current step. The cut search's trial sifts now restore ContextManager's sticky state (stickyEvicted, boundedCallIDs) after each trial; only the chosen cut's state is committed, under any eviction fractions. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
It joined the system message, so the turns that carried it changed the prompt prefix — and on Anthropic without a ContextManager it replaced the whole system prompt. It is now appended after ContextSift as the call's last message. beforeModel no longer sees it among the messages. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ame-role messages) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The task-14 review flagged a real HTTP 400 risk: a progress note or a stored stop-reasoning nudge landing after a tool-result turn produced consecutive user-mapped entries that AnthropicProvider's old merge helper (tool-result blocks only) didn't catch. LLMProviderKit 0.1.0-alpha.41 (already required by HEAD) fixes this generically in the adapter via mergingConsecutiveRoles, but nothing exercised it through the real wire-building path. - Add AnthropicRoleAlternationTests: runs an agent through a tool-using turn, a reasoning-timeout turn (stores the "stop reasoning, act now" nudge), and a progress-note turn that lands on the same call as that stored nudge — the exact collision the review named. Builds the request through the real AnthropicProvider.prepareRequest (not ScriptedProvider) and asserts the wire body's messages roles strictly alternate and the note is the last text block of the last user entry. - Add LLMProviderKitAnthropic to the SwiftAgentKitTests target so that test can build real Anthropic request bodies next to ProgressNoteTests / CacheStableRequestTests. - Restore the coverage StreamedReasoningTests had neutralized: aCallThatOnlyThinksPastTheLimitIsStoppedAndToldToAct now keeps progress nudges ON (the default fractions) instead of disabling them, and asserts the stop-reasoning nudge is found among the turn's user messages rather than assuming it is the literal last one (which, with nudges on, can be the trailing progress note instead). swift test --filter "ProgressNote|StreamedReasoning|CacheStable|HistoryTrim|AnthropicRoleAlternation": 29 tests passed. Full swift test: 642 tests in 48 suites passed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Also checks the streaming request body (stream: true) holds the same append-only promise, and pins that a progress note changes only the tail. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 2 of Naseem's cache-stable requests design. Between eviction batches, the request a conversation sends is byte-identical to the previous call plus the new content; a batch changes only the region where receipts replaced evicted steps.
minMessageBudgetFraction).overflowFraction, default 0.8;overflowTargetFraction) measures the sifted request, removes the fewest oldest whole steps, keeps the cut until the bound is breached again, never drops the task or the step the model is waiting on, and emits.historyTrimmedwith a reason (.overflow,.fitToWindow,.messageCap,.tokenBudget). Apps with their own compaction setoverflowFractionabove their trigger and compact on.overflow.childManager()with every parent setting (readIdentityParams included).Requires LLMProviderKit 0.1.0-alpha.41 (Gemini and Anthropic merge consecutive same-role messages).
Breaking:
AgentEvent.historyTrimmedgained areason(SwiftAgentKitUI'sAgentEventViewneeds a third_).Tests: 644 pass (7 skipped).
🤖 Generated with Claude Code