feat(openai): enable prompt caching for GPT-5.6 - #679
Conversation
|
We run 5.6 in production on Azure Chat Completions — 92.6% cache-hit across ~6,600 calls, sustained for weeks, sending no On this branch, Stable within a session, random across them — probably worse than sending nothing, since OpenAI's prefix-hash routing is at least deterministic. The test misses it because it passes the same Fix: _make_prompt_cache_key(model_name, getattr(agent, "name", None))5 instances → 1 key. Also: both listed test failures are already fixed on |
c0b88fb to
41c7178
Compare
|
@piercebrookins look at @breedx's comments please. |
41c7178 to
24dcf63
Compare
|
Addressed the latest live-test issue in |
|
Verified the cache-prefix fix in |
|
Hmm the changes in model_factory.py need to be refactored out as much as possible. That file is already way too big. 13 files changed seems excessive. |
Summary
main(09085be5)prompt_cache_breakpointafter the reusable instruction prefix for the official OpenAI providerextra_bodyvalues and expose cache usage with model, agent, API-path, and streaming metadataprompt_cache_breakpoint_enabledcontrols only the explicit markeropenai_prompt_cache.pyThis implements the OpenAI team's attached guidance without sending deprecated
prompt_cache_retentionor overriding implicit cache behavior.Refactor scope
model_factory.pyfrom 129 additions / 16 deletions to 36 additions / 20 deletionsmodel_factory.py; it now delegates through three small integration callsbase_agent.py,subagent_usage_metrics.py,test_model_factory_coverage.py, and the standalone builder cache test fileBreedX feedback addressed
mainLive ChatGPT OAuth validation
ec325fe2,codex-gpt-5.6-solfailed with HTTP 400 because the OAuth Responses backend rejectsprompt_cache_breakpointec325fe2, two identical fresh-process requests completed successfully with 5,749 input tokens eachLive public OpenAI API validation
9fb5cb2b, three fresh-process Responses requests with an explicit breakpoint reported 0/3 hits because the cacheable prefix contained a random per-process agent UUID9fb5cb2b, three identicalgpt-5.6-solResponses requests each contained 5,759 input tokens: the cold request read 0 cached tokens, and both warm requests read 5,756 cached tokensTesting
ruff format --checkandruff checkpass for all changed files