Skip to content

docs: specify server-managed prompt cache state for Messages - #195

Draft
franciscojavierarceo wants to merge 2 commits into
mainfrom
codex/messages-cache-state-rfc
Draft

docs: specify server-managed prompt cache state for Messages#195
franciscojavierarceo wants to merge 2 commits into
mainfrom
codex/messages-cache-state-rfc

Conversation

@franciscojavierarceo

@franciscojavierarceo franciscojavierarceo commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • define private, non-authoritative server-managed cache state for the Messages API
  • preserve the conversation-stateless public Messages contract while coordinating affinity, checkpoints, retention, branches, and provider cache artifacts
  • reconcile the proposal with the existing Claude Code integration design
  • define the ownership boundary: Agentic API derives trusted logical coordinates and lifecycle policy, llm-d owns routed worker selection, and vLLM owns physical KV execution
  • link the reproducible affine KV benchmark evidence and qualify the uncommitted Claude Code capture
  • provide staged rollout, failure semantics, security controls, observability, benchmarks, and acceptance criteria

Test Plan

  • cargo test
  • uv run --with-requirements docs/requirements.txt mkdocs build --strict
  • git diff --check origin/main...HEAD
  • verify linked vLLM, llm-d, affine ADR, and raw benchmark sources resolve

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant