Add Agent Observability data retention and billing pages - #39676
Draft
gsvigruha wants to merge 23 commits into
Draft
Add Agent Observability data retention and billing pages#39676gsvigruha wants to merge 23 commits into
gsvigruha wants to merge 23 commits into
Conversation
Document retention periods for traces, experiment traces, annotated objects, dataset records, prompts, and metrics on a single page. Also correct the annotation queues retention table, which listed annotation labels as retained indefinitely, and fix a broken link to a data retention anchor that did not exist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 3-year period for records in the current version runs from creation and is not extended by reads. Only the 90-day period for previous versions resets when a version is used. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Experiment retention pairs with the trace retention tier only on plans with pre-purchased capacity. Without it, experiment traces are retained for 15 days rather than 90. Note that custom contracts may not follow the tier pairings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Experiment definitions and aggregate results carry a flat 90-day period that does not scale with the trace retention tier, unlike experiment traces. State annotation label availability in terms of the annotated object rather than asserting label deletion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add a billing page covering how Agent Observability is metered, what counts as an LLM span, usage estimation from provider request counts, and the Sensitive Data Scanner allotment. Rates link to the pricing page rather than being restated. Scope the data retention page to retention periods, moving the commercial detail to the billing page, and correct it against the current plan structure: retention is keyed on plan rather than on the trace retention period, and retention changes apply retroactively to unexpired spans. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Place it after Incident Response at the end of the per-product billing pages, before the cloud integrations, matching its position on the billing index. Renumber the following entries to keep weights contiguous. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Match the name used throughout the docs and parallel the other provider rows, instead of introducing a name used nowhere else. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Point the pricing link at the Agent Observability product anchor, and add the billing page to the retention page's further reading. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The shared pricing page carries a section per product defining its billable unit, but had none for Agent Observability. Add one covering LLM spans, and link to the Agent Observability billing page. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow the APM billing model: a rate table for the free tier and the on-demand span rates, plus worked scenarios showing how a monthly bill is calculated at three usage levels. Note that committed and enterprise pricing is negotiated separately. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sections on the pricing page define each product's billable unit and leave rates to the product billing page, as the APM section does. Drop the duplicated Agent Observability rates so they have a single owner. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gsvigruha
changed the title
State the annotation label period as 90 days rather than deferring to the annotated object, and note that extending retention by annotating is not billed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replace the separate questions about experiments and indexed spans with one section listing everything included and billed only on LLM spans: evaluations, experiments, patterns, annotation queues, datasets, prompt management, and the playground. Tie the Sensitive Data Scanner section to that list, since it is the one inclusion with a usage allotment. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Move the how-to into the Changing your retention period section, where a reader looks for it, and have the traces section link down to it rather than repeating the contact route. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
State that all features are included and nothing is charged extra, rather than enumerating them, and keep the Sensitive Data Scanner allotment as the stated exception. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Evaluations and patterns can run with your own LLM provider key instead of the Datadog-provided model, in which case your provider bills you for that model usage directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Evaluations and patterns are not charged extra when run with the customer's own LLM provider key, rather than BYOK being an alternative to an unmetered default. State annotations and datasets as included, and experiments as metered on LLM spans like production traces. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Evaluations can run without a customer provider key through a free preview with limited capacity, so BYOK is the route for production volume. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Evaluations and patterns are not charged extra either way: they run on a Datadog-provided model by default, with BYOK as an option for production volume. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
State that all features are included rather than listing them, and reduce the evaluations and patterns note to the two model options. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Links: point the metric retention links at the canonical Data Retention Periods page rather than one of its aliases, and add that page to further reading. Grammar: fix a dangling qualifier in the experiment traces intro, an ambiguous pronoun in the annotation labels sentence, and the repeated preposition in the experiment traces table. Say that a free-form note does not extend retention, rather than that no retention applies to it. Conventions: use Trace Explorer, the name used elsewhere in these docs. Deduplication: the billing page defined an LLM span twice, carried the sales contact three times, and stated the Sensitive Data Scanner allotment in three places. Each now appears once. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do? What is the motivation?
Agent Observability had no documentation for data retention or billing. This PR adds both, and corrects existing pages that described retention incorrectly.
New pages
llm_observability/data_retention.md— retention periods for traces and spans, experiment traces, experiment definitions, annotated objects, dataset records, prompts, and metrics, plus what happens to existing data when a retention period changes.account_management/billing/llm_observability.md— how Agent Observability is metered, rate table and worked billing scenarios, estimating usage from provider request counts, and the Sensitive Data Scanner allotment.Changes to existing pages
account_management/billing/pricing.md— adds an Agent Observability section defining the billable unit, following the pattern of the other product sections.account_management/billing/_index.mdand the side nav — index card and nav entries for the new billing page.llm_observability/investigate/annotation_queues.md— the retention table listed annotation labels as retained indefinitely, and did not mention that annotating a trace extends the trace's retention.llm_observability/investigate/metrics.md— fixed two links pointing at a data retention anchor that did not exist.Notes for reviewers
Merge readiness
For Datadog employees:
<name>/<description>convention and include the forward slash (/). If you've already created your PR with an incorrect branch name, please rename your branch and open a fresh PR./reviewto run an automated check that catches common issues before a Documentation team member reviews your PR.AI assistance
Drafted with Claude Code. Retention periods were verified against internal configuration rather than written from memory, and reviewed manually.
Additional notes
The title still needs a Jira ticket key (
[DOCS-XXXXX]), and the description a matchingFixes DOCS-XXXXXline.