Repository navigation
Cache-stable requests, step 1: canonical replay arguments, sorted tools, ordered registrations - #132
Merged
Merged
Conversation
…thropic usage) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The arguments of every replayed call were re-encoded with an unsorted encoder on each model call, so the history could change bytes between calls and the prompt cache stopped there. AnyCodable also matched Bool before NSNumber: an integer 1 was replayed as true, 0 as false, 1.0 as true. Calls already stored keep what was written; new calls are right. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…igest ToolRegistry listed tools in dictionary order, which differs per instance: every engine sent its tools in another order. Every piece of registration work (register, registerAll, skills, setSkillStore's load, tool context, autonomous mode, init's) now waits for the one queued before it, so an app's override of a framework tool always wins (the learn_skill race). The agent also exposes firstRequestSystemDigest (stable FNV-1a 64) and registeredTools(). A wire test pins that consecutive calls of one chat are append-only on the OpenAI body. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Provider prompt caches reuse the longest byte-identical start of a request. Three things in the framework changed that start between consecutive calls of one conversation:
JSONEncoderon every call. They now go out canonically (sorted keys,canonicalArguments). The same encoder turned an integer 1 intotrue(and 0 intofalse, 1.0 intotrue, nested values too) and persisted it; an NSNumber is now typed before the Bool check.learn_skillwin over the app's override. Every tracked registration is chained now; a later registration of the same name wins.Also:
Agent.registeredTools(),PromptDigestandfirstRequestSystemDigest(a digest of the exact system prompt of a run's first request, reset per run) for the app's cache measurement, and a consecutive-call wire test through the realOpenAIProvider.prepareRequestasserting the second request's bytes start with the first's prefix.Requires LLMProviderKit 0.1.0-alpha.39 (sorted bodies; Anthropic whole-prompt usage).
Tests:
CacheStableRequestTests(8); full suite: Swift Testing 606 pass, XCTest 182 pass / 7 skipped.🤖 Generated with Claude Code