Repository navigation
fix(adapters)!: Anthropic and Bedrock input_tokens now inclusive of cache tokens - #22
Merged
Merged
Conversation
…ache tokens BREAKING CHANGE: inputTokens changes value for any Anthropic or Bedrock response that reports cache tokens. Anthropic and AWS both document input_tokens/inputTokens as excluding cache tokens (total = cache_read + cache_creation + input_tokens, per each vendor's own prompt-caching docs). This package's own README and types.ts already claimed cache tokens are "already included in inputTokens" for every provider, true for OpenAI, false for Anthropic and Bedrock, whose input_tokens was passed straight through unchanged. The OpenTelemetry GenAI semantic conventions this package targets also say the total input count MAY be published regardless of caching. Every 0.2.x version undercounted inputTokens on cache-heavy Anthropic and Bedrock traffic. See CHANGELOG for the full note. - adapters.ts: anthropic and bedrock extract() now add cache_read/cache_creation (Anthropic) and cacheReadInputTokens/ cacheWriteInputTokens (Bedrock) into inputTokens. openai untouched, already correct. gemini untouched, candidatesTokenCount/ thoughtsTokenCount inclusion is not stated in Google's docs and is gated on a live API check not performed in this branch. - types.ts: cacheWriteTokens doc comment corrected, Bedrock also has cache-write, not Anthropic-only. - test/normalize.test.ts: replaced the input_tokens:20/cache_read:15 fixture (15+5=20 made the wrong reading numerically indistinguishable from the right one) with Anthropic's own doc-example shape, added a golden-invariant test per provider, doubled fixtures/responses.jsonl with cache/reasoning-bearing records so normalizeMany exercises the previously-untested path. - README, CHANGELOG: documented the vendor asymmetry and the fix. - package.json: 0.3.0 (not tagged, not published). Verified locally: npm ci, lint, typecheck, coverage (25/25 tests), build, demo, audit --audit-level=high all pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
7 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Branch 1 of the review's fix plan. Breaking, targets 0.3.0.
Anthropic and AWS both document
input_tokens/inputTokensas excluding cache tokens:total_input_tokens = cache_read_input_tokens + cache_creation_input_tokens + input_tokens(Anthropic prompt-caching docs; matching formula in AWS's Bedrock Converse prompt-caching docs). This package's owntypes.tsand README already claimed cache tokens are "already included ininputTokens" for every provider, true for OpenAI, false for Anthropic and Bedrock, whoseinput_tokenswas passed straight through unmodified. The OpenTelemetry GenAI semantic conventions this package targets also specify the total input count MAY be published regardless of caching.Every 0.2.x version undercounted
inputTokenson any cache-heavy Anthropic or Bedrock request. Full note in CHANGELOG.adapters.ts: anthropic and bedrockextract()now add cache tokens intoinputTokens.openaiuntouched (already correct).geminiuntouched,candidatesTokenCount/thoughtsTokenCountinclusion isn't stated in Google's docs, gated on a live API check not performed here, reported Blocked per the implementation brief rather than inferred from docs.types.ts: corrected thecacheWriteTokensdoc comment, Bedrock has cache-write too, not Anthropic-only.test/normalize.test.ts: replaced theinput_tokens: 20, cache_read: 15, cache_creation: 5fixture, 15+5=20 made the exclusive and inclusive readings numerically indistinguishable, which is exactly why 98%+ coverage never caught this. Added a golden-invariant test per provider (inputTokens >= cacheReadTokens + cacheWriteTokens,outputTokens >= reasoningTokens) and doubledfixtures/responses.jsonlwith cache/reasoning-bearing records sonormalizeManyexercises the previously-untested path.README.md,CHANGELOG.md: documented the vendor asymmetry and the breaking change plainly.package.json: version bumped to0.3.0. Not tagged, not published, per the implementation brief's boundaries.Test plan
npm cinpm run lintnpm run typechecknpm run coverage— 25/25 testsnpm run buildnpm run demo— output confirms the fix mechanically (Anthropic cache-heavy record:inputTokens: 100050= 50 + 100000 + 0; Bedrock:50034= 30 + 50000 + 4)npm audit --audit-level=high🤖 Generated with Claude Code