fix(server-utils): Include Gemini reasoning tokens in Vercel AI token usage - #24066
Merged
Merged
Conversation
… usage Forward port of #23433, which landed on `v10` because the Vercel AI OTel span processing it originally targeted was removed here in #23384. Gemini reports its reasoning ("thoughts") tokens separately from the candidate output count, so on `ai` v4/v5 the SDK's `outputTokens` covers only the visible answer and the reasoning count reaches us solely through `providerMetadata.google.usageMetadata`. The conventions define `gen_ai.usage.output_tokens` as reasoning-inclusive, so those spans were under-reporting rather than merely missing a breakdown. Read the `google`/`vertex` `usageMetadata` and derive output from `candidatesTokenCount + thoughtsTokenCount`, the total from `totalTokenCount`, and the reasoning breakdown alongside them. An absent `candidatesTokenCount` counts as zero: Gemini omits it when the response is truncated during thinking, which means no candidate tokens were produced, and skipping the recompute there would report zero output for a call that spent its whole budget reasoning. None of the three land on `gen_ai.invoke_agent` spans. Those carry usage summed across every step while `providerMetadata` describes the last step alone, so writing from it would replace an aggregate with one step's figures. Reasoning is gated with them because it is a subset of an output that span never recomputes, and nothing sums it across steps. Unlike `v10` this branch has a single caller, so the rule lives directly in `enrichSpanOnEnd` and there is no `addProviderMetadataToAttributes` equivalent. Ref #23993 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018TR1cvQA7t6T2saCrHwwUh
RulaKhaled
marked this pull request as ready for review
September 4, 2026 08:52
RulaKhaled
requested review from
stephanie-anderson
and removed request for
a team
September 4, 2026 08:52
Contributor
size-limit report 📦
|
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Gating `gen_ai.usage.reasoning.output_tokens` unconditionally dropped it from `invoke_agent` spans for every provider, not just Google. OpenAI writes the same attribute from `providerMetadata.openai.reasoningTokens`, so a `generateText` call against a reasoning model silently lost a count it used to report. The reason to drop it only applies where a recomputed output is dropped with it: that leaves the span's own output reasoning-exclusive, so the documented subset relationship would not hold. A provider reporting reasoning against an already-inclusive output keeps it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018TR1cvQA7t6T2saCrHwwUh
andreiborza
approved these changes
Sep 4, 2026
andreiborza
left a comment
andreiborza
left a comment
Member
There was a problem hiding this comment.
Thanks for taking care of this!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Forward port of #23433, which landed on
v10because the Vercel AI OTel span processing it originally targeted was removed here in #23384. Without this, upgrading v10 → v11 loses the fix.Gemini reports its reasoning ("thoughts") tokens separately from the candidate output count, so on
aiv4/v5 the SDK'soutputTokenscovers only the visible answer and the reasoning count reaches us solely throughproviderMetadata.google.usageMetadata.getProviderMetadataAttributes()handled OpenAI, Anthropic, Bedrock and DeepSeek but never looked at the Google/Vertex block. The conventions definegen_ai.usage.output_tokensas reasoning-inclusive, so these spans were under-reporting rather than merely missing a breakdown — a real Gemini response of{promptTokenCount: 14, candidatesTokenCount: 1, thoughtsTokenCount: 100, totalTokenCount: 115}emittedoutput 1 / total 15instead ofoutput 101 / total 115.Known limitations, tracked in #23993:
invoke_agentspans carry no reasoning count at all, and nothing sums it from their children the wayapplyAccumulatedTokensdoes for input and output. Separately,enrichSpanOnEndnever readsusage.outputTokenDetails.reasoningTokens, whichaiv6+ supplies directly — reading it would populate the reasoning breakdown for Gemini, OpenAI and Anthropic at once, and is the more valuable change for anyone on a current SDK version. This PR only helps v4/v5 users.