Repository navigation
feat(protocol): Usage gains prompt-cache counts and an includes-cache flag, and the proxy reports its turns' model usage - #791
Merged
earakely-scale merged 8 commits intoNov 19, 2026
Conversation
… flag, and the proxy reports its turns' model usage Usage takes cache_read_tokens, cache_write_tokens and input_includes_cache, validated like its other fields, and Usage.from_dict reads what any protocol version wrote. TaskResult.failure carries usage. The proxy sums the usage of the model calls its driver records into each turn's Usage, OpenAI- and Anthropic-shaped alike, with input_tokens counting the cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e cache Anthropic's keys beside it name A real LiteLLM proxy answers Claude through /chat/completions with prompt_tokens counting neither cache count it sends beside it, while OpenAI and a LiteLLM client say what prompt_tokens counts through prompt_tokens_details.cached_tokens. The normalizer, now Usage.from_model_response so agents can read their own gateway calls the same way, adds the cache to prompt_tokens and total_tokens unless prompt_tokens_details says it's already there. The fixtures are the recorded responses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… mapping, empty ones too; the usage episode test abandons its episode Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…9-p7-usage-cache-fields # Conflicts: # tst/unit/a2a_agent/protocol_test.py
victorlee-scale
approved these changes
Nov 19, 2026
… and a sum short of some calls says so A chat span took its token counts from the raw usage while the turn's Usage read it with from_model_response, so a LiteLLM proxy's Claude call showed 9 input tokens in the trajectory and 7,818 in the usage. Both now read the call once. The turn's provider_details give model_calls and, for a count only some calls gave, how many left it out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l_response decides once whether a response's input counts the cache from_dict, is_empty, to_dict and the proxy's sums read Usage's fields from the dataclass, so a count added to Usage reaches them all. from_model_response makes the one decision its docstring describes and applies it to the input and the total together. Behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fault reader knows the Responses shape Whoever makes a model call knows what its counts hold. record_model_call takes a Usage, sent to the proxy under REPORTED_USAGE_KEY inside the record's usage, so a proxy on an earlier protocol still takes the record, and taken as given; one that isn't a Usage gets 400 and leaves the turn as it was. Each call's input is made to count the cache before its span and its turn read it, a call whose flag is unknown while it has cache counts leaving the turn's flag unknown, and the turn sums cost_usd too. Usage.from_model_response, the default for a response's own usage, reads OpenAI's Responses shape and the cache_write_tokens OpenAI's details now give, keeps each provider's split of its cache writes by how long they're kept (priced apart) in provider_details, and marks a usage in no shape it knows as unrecognized. The fixtures are responses recorded through a LiteLLM proxy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ere it holds counts A driver's Usage may carry any provider_details, so a split with a None count no longer fails the turn when it ends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale
deleted the
edgararakelyan/fd-5979-p7-usage-cache-fields
branch
November 19, 2026 05:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The protocol's
Usagecan now say how many prompt tokens came from the provider's prompt cache, and whetherinput_tokensalready counts them. Without that, a price can't tell cached input from fresh input.Whoever makes a model call knows what its counts hold, so they report usage themselves; the framework only guesses when nobody tells it. This is the protocol and host half of FD-5979 (P7 of FD-5974). The five agents that count cache tokens move to the new fields in a second PR, once a protocol release has them.
What changes
Usagegains three fields (allNoneby default):cache_read_tokensandcache_write_tokens, validated like the other counters;input_includes_cache: bool | None, strictly a bool.Truemeansinput_tokenscounts both cache counts,Falsemeans it counts neither, andNonemeans unknown.Usage.from_dictreads whatto_dictwrote under any protocol version. A field it leaves out isNone, a key this version doesn't know is ignored, and a value a field can't hold raises.TaskResult.failure(..., usage=), so a failure carries usage assuccessdoes.record_model_call(message, usage=Usage(...))sends theUsageto the proxy, which takes it as given.REPORTED_USAGE_KEY("agentenv_usage") inside the record's existingusagefield, not as a new field. Extension requests refuse unknown fields, so a proxy on an older protocol still takes the record.Usagegets 400 and leaves the turn as it was.Usage.from_model_response(usage)is the default reader for a model response's own usage. It handles OpenAI's Chat Completions (prompt_tokens) and Responses (input_tokens) shapes and Anthropic's, always returninginput_tokenscounting the cache:prompt_tokens_detailsorinput_tokens_detailslistscached_tokens, the input already counts the cache. That's OpenAI and a LiteLLM client, andcache_write_tokensthere is counted too.cache_read_input_tokensandcache_creation_input_tokensare added to the input and the total. That covers Anthropic itself, and a LiteLLM proxy answering for Claude, which sends them beside aprompt_tokensthat leaves them out. They're also added whenever the input is smaller than the cache it would hold.cache_creation(5 minutes, 1 hour) and OpenAI'scache_write_tokens_details. They go inprovider_details.unrecognized, and leaves out a value that isn't a count.Usagewas onlytool_call_count.Falsehas its cache added to its input and total. One with no flag but with cache counts stays as given and leaves the turn's flag unknown.chatspan shows the same counts, so the trajectory and the usage agree.cost_usdincluded.provider_detailsgivesmodel_calls;unreported(how many calls left out a count only some gave, so a partial sum reads as the lower bound it is);cache_unsettled;unrecognized; and the cache-write splits summed, counting only real counts.Usage's field list is written once.from_dict,is_empty,to_dictand the proxy's sums read it from the dataclass.Usage, the default reader and the proxy's usage.agents/AGENTS.mdtells agent authors to fill the cache fields and set the flag per call.What a real gateway showed
Calls through the Scale LiteLLM proxy, which are now the unit fixtures:
/chat/completionsprompt_tokens: 9, cache beside it, no details/chat/completionsprompt_tokens: 7823,prompt_tokens_details.cached_tokens: 7680/v1/responsesinput_tokenswithinput_tokens_details/v1/messagesinput_tokens: 9, cache beside itcache_creationSo whether the input counts the cache depends on the route and the model behind it, not on the agent. An adapter sets the flag per call. FD-5979's table gives OpenHands
True, which holds for GPT on this gateway but not for Claude.Mixed versions
None.tool_call_countfrom usage and ignores the rest.TypeError. That's why the adapters wait for the second PR.Tests
test_usage.py(new):provider_detailsthat isn't a mapping, even an empty one, refused;failure(usage=);test_a2a_proxy.py:Falseflag normalized, an unknown flag leaving the turn's flag unknown;tst/unit/a2a_agent/protocol_test.py: the host parses a terminal message whose usage has the new fields, and a failed turn sends its usage.tst/integration/agent/proxy_usage_local_test.py(new, about 1 s, no Docker): an episode's trainer records an OpenAI-shaped and an Anthropic-shaped response and one reportedUsagethroughrecord_model_call. The host stores their sum with the agent's reply.E2E and chaos (Linux VM)
Usagewhose input left the cache out, the host stored the 11 calls summed, the reported one normalized.Next
provider_detailscache keys;from_model_response;Docs (for P13, FD-6263)
Usage's cache fields and the flag, set per call.record_model_call(usage=Usage(...)).from_model_response: the shapes it reads, the cache-write splits, andunrecognized.provider_details.🤖 Generated with Claude Code


Confidence Score: 5/5
The changes appear safe to merge; no outstanding issues were found.Summary
Adds prompt-cache counts to
Usageand reports summed model usage for proxy turns, including failed replies.Reviews (11) · Last reviewed commit: "fix(protocol): a reported Usage's cache-..." · Reviewed by Greptile