Skip to content

fix(adapters)!: Anthropic and Bedrock input_tokens now inclusive of cache tokens - #22

Merged
mizcausevic-dev merged 1 commit into
mainfrom
fix/token-inclusion-semantics
Sep 13, 2026
Merged

mizcausevic-dev merged 1 commit into
mainfrom
fix/token-inclusion-semantics

Conversation

@mizcausevic-dev

Copy link
Copy Markdown
Owner

Summary

Branch 1 of the review's fix plan. Breaking, targets 0.3.0.

Anthropic and AWS both document input_tokens/inputTokens as excluding cache tokens: total_input_tokens = cache_read_input_tokens + cache_creation_input_tokens + input_tokens (Anthropic prompt-caching docs; matching formula in AWS's Bedrock Converse prompt-caching docs). This package's own types.ts and README already claimed cache tokens are "already included in inputTokens" for every provider, true for OpenAI, false for Anthropic and Bedrock, whose input_tokens was passed straight through unmodified. The OpenTelemetry GenAI semantic conventions this package targets also specify the total input count MAY be published regardless of caching.

Every 0.2.x version undercounted inputTokens on any cache-heavy Anthropic or Bedrock request. Full note in CHANGELOG.

  • adapters.ts: anthropic and bedrock extract() now add cache tokens into inputTokens. openai untouched (already correct). gemini untouched, candidatesTokenCount/thoughtsTokenCount inclusion isn't stated in Google's docs, gated on a live API check not performed here, reported Blocked per the implementation brief rather than inferred from docs.
  • types.ts: corrected the cacheWriteTokens doc comment, Bedrock has cache-write too, not Anthropic-only.
  • test/normalize.test.ts: replaced the input_tokens: 20, cache_read: 15, cache_creation: 5 fixture, 15+5=20 made the exclusive and inclusive readings numerically indistinguishable, which is exactly why 98%+ coverage never caught this. Added a golden-invariant test per provider (inputTokens >= cacheReadTokens + cacheWriteTokens, outputTokens >= reasoningTokens) and doubled fixtures/responses.jsonl with cache/reasoning-bearing records so normalizeMany exercises the previously-untested path.
  • README.md, CHANGELOG.md: documented the vendor asymmetry and the breaking change plainly.
  • package.json: version bumped to 0.3.0. Not tagged, not published, per the implementation brief's boundaries.

Test plan

  • npm ci
  • npm run lint
  • npm run typecheck
  • npm run coverage — 25/25 tests
  • npm run build
  • npm run demo — output confirms the fix mechanically (Anthropic cache-heavy record: inputTokens: 100050 = 50 + 100000 + 0; Bedrock: 50034 = 30 + 50000 + 4)
  • npm audit --audit-level=high
  • Gemini live API check — not performed, Blocked. Gemini adapter intentionally untouched.

🤖 Generated with Claude Code

…ache tokens

BREAKING CHANGE: inputTokens changes value for any Anthropic or Bedrock
response that reports cache tokens.

Anthropic and AWS both document input_tokens/inputTokens as excluding
cache tokens (total = cache_read + cache_creation + input_tokens, per
each vendor's own prompt-caching docs). This package's own README and
types.ts already claimed cache tokens are "already included in
inputTokens" for every provider, true for OpenAI, false for Anthropic
and Bedrock, whose input_tokens was passed straight through unchanged.
The OpenTelemetry GenAI semantic conventions this package targets also
say the total input count MAY be published regardless of caching.

Every 0.2.x version undercounted inputTokens on cache-heavy Anthropic
and Bedrock traffic. See CHANGELOG for the full note.

- adapters.ts: anthropic and bedrock extract() now add
  cache_read/cache_creation (Anthropic) and cacheReadInputTokens/
  cacheWriteInputTokens (Bedrock) into inputTokens. openai untouched,
  already correct. gemini untouched, candidatesTokenCount/
  thoughtsTokenCount inclusion is not stated in Google's docs and is
  gated on a live API check not performed in this branch.
- types.ts: cacheWriteTokens doc comment corrected, Bedrock also has
  cache-write, not Anthropic-only.
- test/normalize.test.ts: replaced the input_tokens:20/cache_read:15
  fixture (15+5=20 made the wrong reading numerically indistinguishable
  from the right one) with Anthropic's own doc-example shape, added a
  golden-invariant test per provider, doubled fixtures/responses.jsonl
  with cache/reasoning-bearing records so normalizeMany exercises the
  previously-untested path.
- README, CHANGELOG: documented the vendor asymmetry and the fix.
- package.json: 0.3.0 (not tagged, not published).

Verified locally: npm ci, lint, typecheck, coverage (25/25 tests),
build, demo, audit --audit-level=high all pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mizcausevic-dev
mizcausevic-dev merged commit 70f7824 into main Sep 13, 2026
4 checks passed
@mizcausevic-dev
mizcausevic-dev deleted the fix/token-inclusion-semantics branch September 13, 2026 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant