Skip to content

Decouple flag eval metrics from CoreMetricCollector - #12666

Open
sarahchen6 wants to merge 5 commits into
masterfrom
sarahchen6/fix-ffe-metrics
Open

sarahchen6 wants to merge 5 commits into
masterfrom
sarahchen6/fix-ffe-metrics

Conversation

@sarahchen6

@sarahchen6 sarahchen6 commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

This PR moves flag evaluation recording out of CoreMetricCollector and into product-owned FlagEvaluationMetrics using :products:metrics:metrics-api’s Accumulator for fixed counters by removing CoreMetricCollector.count() and instead adding a periodic telemetry adapter, preserving metric names, tags, namespace, and type.

Motivation

#11639 has the flag evaluation writer push metrics directly into CoreMetricCollector; however, this breaks the design of CoreMetricCollector which is to drain metrics for delivery (not capture / push metrics). It also coupled product-specific metric recording to a shared internal telemetry component - which we want to avoid.

The changes in this PR restore the ownership pattern for flag evaluation based on existing usages such as by SpanMetricsImpl and BaggageMetrics. Now feature flagging owns its counters and telemetry drains them for delivery.

Additional Notes

With these changes, truncation counts are collectible without waiting for an EVP writer flush, and collection timestamps follow telemetry’s schedule. Queue saturation keeps pending snapshots for later collection, which is similar to StatsDCountReporter. Besides this telemetry collection timing and retention, flag evaluation results and event payloads are unchanged.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@sarahchen6 sarahchen6 added type: bug fix Bug fix comp: metrics Metrics tag: ai generated Largely based on code generated by an AI or LLM labels Sep 28, 2026
@datadog-official

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.09 s 14.00 s [-0.0%; +1.4%] (no difference)
startup:insecure-bank:tracing:Agent 12.95 s 12.96 s [-0.7%; +0.7%] (no difference)
startup:petclinic:appsec:Agent 17.20 s 17.04 s [-0.0%; +1.8%] (no difference)
startup:petclinic:iast:Agent 16.99 s 17.16 s [-1.8%; -0.1%] (maybe better)
startup:petclinic:profiling:Agent 16.68 s 16.61 s [-0.9%; +1.8%] (no difference)
startup:petclinic:sca:Agent 17.13 s 16.95 s [+0.1%; +2.1%] (maybe worse)
startup:petclinic:tracing:Agent 16.25 s 16.22 s [-0.7%; +1.0%] (no difference)

Commit: 4a854c33 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@sarahchen6
sarahchen6 marked this pull request as ready for review October 1, 2026 15:00
@sarahchen6
sarahchen6 requested review from a team as code owners October 1, 2026 15:00
@sarahchen6
sarahchen6 requested review from PerfectSlayer, dougqh, pavlokhrebto and sameerank and removed request for a team October 1, 2026 15:00
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-01T15:05:36.600308Z 236d499 Draft marked ready
🔒 Security Review ✅ Completed 2026-10-01T15:04:41.421280Z 236d499 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-official datadog-official Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Producer-owned flag-evaluation counters preserve existing metric wire metadata and retain pending snapshots when the telemetry queue is full.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 236d499 · @DataDog review to ask questions

@PerfectSlayer PerfectSlayer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 thought: ‏There might be a dependency issue. The FFE bootstrap module now has dependency to the metrics api. (Which might relocated it into the bootstrap? 🤔 )
Same with the telemetry module which gains a dependency to a product (FFE).

I would stick to the original design having a collector in internal-api module (not great but that's the current design pattern for debugger, iast, llmobs, WAF, etc...) to avoid adding such dependencies. WDYT?

@sarahchen6

Copy link
Copy Markdown
Contributor Author

I would stick to the original design having a collector in internal-api module (not great but that's the current design pattern for debugger, iast, llmobs, WAF, etc...) to avoid adding such dependencies. WDYT?

Makes sense! I think this should follow other product collector formats as well as minimize shared module dependencies on products... I'll refactor

@sarahchen6
sarahchen6 force-pushed the sarahchen6/fix-ffe-metrics branch from 236d499 to 4a854c3 Compare October 1, 2026 16:09
@sarahchen6 sarahchen6 added the type: feature Enhancements and improvements label Oct 1, 2026
@bric3
bric3 added this pull request to stack #12733 October 2, 2026 15:53

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: metrics Metrics tag: ai generated Largely based on code generated by an AI or LLM type: bug fix Bug fix type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants