[telemetry] 🤖 feat: measure advisor cache usage and savings - #4205
[telemetry] 🤖 feat: measure advisor cache usage and savings#4205coadler wants to merge 2 commits into
Conversation
## Summary Add an advisor completion event to PostHog. Measure cache usage, estimated savings, timing, and call outcomes without changing cache behavior. ## Background Advisor cache writes can cost more than they save. Existing telemetry cannot measure this tradeoff. ## Implementation - Record successful, failed, and cancelled calls. Exclude calls rejected by the usage limit. - Round token counts, timing, and estimated costs. Preserve unknown values as null. - Compare read savings with the write premium before rounding. - Limit cost estimates to known Anthropic pricing with one known TTL. - Exclude prompt content, cache keys, endpoint URLs, and unresolved custom model names. ## Risks Cache cost estimates use catalog rates, not invoice charges. Mixed TTLs and unsupported pricing remain unknown. Telemetry errors do not change the advisor result. No personal deployment validation is available. --- _Generated with `xum` • Model: `openai:gpt-6-astra` • Thinking: `medium` • Cost: `$13.41`_ <!-- mux-attribution: model=openai:gpt-6-astra thinking=medium costs=13.41 -->
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
This comment has been minimized.
This comment has been minimized.
|
@codex review |
This comment has been minimized.
This comment has been minimized.
|
@codex review |
This comment has been minimized.
This comment has been minimized.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f7853f11ac
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Measure cache policy after transport injection and TTL overrides. --- _Generated with `xum` • Model: `openai:gpt-6-astra` • Thinking: `medium` • Cost: `$26.20`_ <!-- mux-attribution: model=openai:gpt-6-astra thinking=medium costs=26.20 -->
|
@codex review |
|
Codex Review: Didn't find any major issues. Another round soon, please! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
This comment has been minimized.
This comment has been minimized.
1 participant
Summary
Add an advisor completion event to PostHog. Measure cache usage, estimated savings, timing, and call outcomes without changing cache behavior.
Background
Advisor cache writes can cost more than they save. Existing telemetry cannot measure this tradeoff.
Implementation
Risks
Cache cost estimates use catalog rates, not invoice charges. Mixed TTLs and unsupported pricing remain unknown.
Telemetry errors do not change the advisor result. No personal deployment validation is available.
Generated with
xum• Model:openai:gpt-6-astra• Thinking:medium• Cost:$13.41