fix: token accounting (cachedWriteTokens & cacheWriteInputTokens) - #510
Open
joshlarsen wants to merge 1 commit into
Open
fix: token accounting (cachedWriteTokens & cacheWriteInputTokens)#510joshlarsen wants to merge 1 commit into
joshlarsen wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix cache-write token accounting by separating cache writes from ordinary input and exposing them through ACP's
cachedWriteTokensfield.Fixes #509. The
cachedWriteTokensomission is also mentioned in #447.The change also:
cacheWriteInputTokensin quota metadata;/status;CODEX_PATH;Token calculation
Codex reports
inputTokensinclusive of cache reads and writes. The normalized categories are therefore:Reasoning tokens are already included in
outputTokens, soreasoningOutputTokensis exposed as ACPthoughtTokensbut is not added again.For example:
The mapping is:
inputTokens - cachedInputTokens - cacheWriteInputTokensinputTokenscachedInputTokenscachedReadTokenscacheWriteInputTokenscachedWriteTokensoutputTokensoutputTokensreasoningOutputTokensthoughtTokenstotalTokenstotalTokensCompatibility
cacheWriteInputTokensis required by the current generated app-server type. At runtime, a missing value from an olderCODEX_PATHbinary is normalized to zero. Invalid negative partitions are not clamped, so inconsistent upstream counters remain observable.Validation
npm run typechecknpm test— 609 tests passednpm run build3ordinary input +11,099cache-write +5output =11,107total.OpenAI references
cached_tokens,cache_write_tokens, and the ordinary-input subtraction formula.