Skip to content

Map OpenAI cached tokens in Java SDK#137

Open
Alex Jean-Baptiste (justcodebruh) wants to merge 1 commit into
mainfrom
ajb-cached-token-fix
Open

Map OpenAI cached tokens in Java SDK#137
Alex Jean-Baptiste (justcodebruh) wants to merge 1 commit into
mainfrom
ajb-cached-token-fix

Conversation

@justcodebruh

Copy link
Copy Markdown

Summary

  • Map OpenAI cached-token usage into Braintrust prompt_cached_tokens metrics.
  • Support both Chat Completions usage.prompt_tokens_details.cached_tokens and
    Responses API usage.input_tokens_details.cached_tokens.
  • Add focused InstrumentationSemConv tests for both payload shapes.

Root Cause

The Java SDK preserved the raw OpenAI usage object, including cached_tokens,
but InstrumentationSemConv only mapped prompt, completion, total, and
reasoning token fields into Braintrust span metrics.

As a result, OpenAI responses could contain:

usage.prompt_tokens_details.cached_tokens

but the exported Braintrust span omitted:

metrics.prompt_cached_tokens

Validation

  • Ran focused SDK tests:
./gradlew :braintrust-sdk:test --tests
dev.braintrust.instrumentation.InstrumentationSemConvTest --console=plain

- Built and published a local 0.3.15-DIRTY SDK artifact.
- Reran the LangChain4j OpenAiStreamingChatModel GPT-5.2 repro locally against a private Braintrust instance.

- Verified the exported Braintrust span now includes:   "prompt_cached_tokens": 9984

@realark Andrew Kent (realark) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to add spec tests for these metrics to make sure all the openai integrations are actually mapping metrics to the semconv tool. I'll knock this out in the next day or two. Thanks for getting us started, Alex!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants