Repository navigation
Cache-stable requests, step 3: frozen run context - #134
Merged
Merged
Conversation
…lt once Opt-in (default off). The system prompt's dynamic parts are built at the first run and kept as a Codable FrozenRunContext, reused while the engine's own base (configured prompt, tool line, group index) has the same digest. invalidateRunContext() and a compaction clear it; restoreRunContext(_:) installs a saved one. The prompt is assembled exactly as before. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review found that invalidateRunContext() during runContextParts' awaits (memory store, run-context provider, skill index) was silently overwritten by the stale build. Add a generation counter, bumped by invalidate, restore and compaction, captured before the first await; the build only freezes its result if the generation is unchanged, otherwise the stale parts still serve the in-flight run but the next run rebuilds. Also make restoreRunContext a no-op when freezeRunContext is off (nothing ever reads the stored value in that mode), and add tests pinning the FrozenRunContext Codable wire keys and the load_tools-mid-chat invariant. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… a relaunch Two frozen runs with a fact filed between them and one ContextSift batch, then a relaunch from the JSON-saved FrozenRunContext, on the OpenAI wire (streaming and not): one system text, one batch, identical tool definitions; the fact reaches the prompt only after invalidateRunContext(). Known issue pinned with withKnownIssue: after a relaunch the messages are append-only only up to the first evicted step. Receipts are rebuilt with new random artifact ids and the evicted set is recomputed (one more step). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ing them again An artifact saved for a tool call now gets its id from the call id and the output (artifact- + 12 hex of an unseeded FNV-1a digest), and both built-in stores return the existing artifact for that id instead of storing it again. After a relaunch, ContextSift's empty receipt cache no longer writes a duplicate artifact per evicted output to the FileArtifactStore, and every receipt is the same bytes as before. The output is part of the key so a provider that reuses call ids across runs never gets a stale artifact; with no call id the id stays random. The relaunch regression test now shares a FileArtifactStore folder and checks receipts and the prefix through every receipt; its known issue is narrowed to the one remaining cause, the evicted set rebuilt from scratch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…int) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rth cache breakpoint (LLMProviderKit alpha.42) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 3 of Naseem's cache-stable requests design.
AgentConfig.freezeRunContext(default false): the memory block, the app's per-run context block and the skill index are built on a run's first request and kept byte-for-byte for every later run.FrozenRunContext(Codable) can be read and restored, so an app can persist it with a chat and reuse it after a relaunch; a restored context from a different base prompt or tool set is rejected by digest and rebuilt.invalidateRunContext()marks it stale; a generation counter makes an invalidation that lands mid-build win.Requires LLMProviderKit 0.1.0-alpha.42 (Anthropic's fourth cache breakpoint on the previous turn).
Tests: 659 pass (1 known issue), XCTest 182 (7 skipped).
🤖 Generated with Claude Code