Teach GitHub Codex to follow dd-apm-sdk-review without invoking the skill

[robertomonteromiguel]

What Does This Do

Teaches GitHub Codex (and any reviewer that cannot invoke .agents/skills/) to use the dd-apm-sdk-review lens files as the review spec, instead of trying to run the skill.

Local agents with a skill harness still invoke dd-apm-sdk-review before push.

Motivation

Codex on GitHub reads AGENTS.md but cannot execute the skill. #12364 only tells local agents to invoke it, so Codex reviews against house style instead of the lenses.

Additional Notes

Stacked on #12364 (robertomonteromiguel/dd-apm-sdk-review-core-overrides), not master.

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Once approved, use merge queue to merge the PR

Jira ticket: N/A

[datadog-datadog-prod-us1]

The new sampling predicate returns the opposite of its documented rule. The class has no current caller, so the defect has no active tracer path.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 1e6476d · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[robertomonteromiguel]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1e6476df78

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[pr-commenter]

LLM Validation

LLM Validation Gate — dd-apm-sdk-review

✅ PASS

• Overall quality improved by 16.8 points with no blocking-case regressions.
• Advisory: the candidate introduced a bad signal on 1 case(s) that the baseline did not — surfaced for review (see Cases); it does not block on its own.

Analysis

Changed instruction file(s): .agents/skills/dd-apm-sdk-review/SKILL.md, .agents/skills/dd-apm-sdk-review/reviewers/_common.md, .agents/skills/dd-apm-sdk-review/reviewers/correctness.md, .agents/skills/dd-apm-sdk-review/reviewers/performance.md, .agents/skills/dd-apm-sdk-review/reviewers/report-template.md, .agents/skills/dd-apm-sdk-review/reviewers/coherence.md, .agents/skills/dd-apm-sdk-review/reviewers/security.md, .agents/skills/dd-apm-sdk-review/reviewers/design.md, .agents/skills/dd-apm-sdk-review/reviewers/maintainability.md, .agents/skills/dd-apm-sdk-review/reviewers/conventions.md, .agents/skills/dd-apm-sdk-review/reviewers/cross-sdk.md, .agents/dd-apm-sdk-review-overrides/repo-context.md, .agents/dd-apm-sdk-review-overrides/reviewers/security.md, .agents/dd-apm-sdk-review-overrides/reviewers/performance.md, .agents/dd-apm-sdk-review-overrides/reviewers/design.md, .agents/dd-apm-sdk-review-overrides/reviewers/conventions.md, .agents/dd-apm-sdk-review-overrides/reviewers/maintainability.md.

1 case(s) dipped but stayed within the pass bar.

• java-perf-lens-wrong-collection-001: won 50% of blind comparisons, quality -5.0. ⚠️ Introduced a new bad signal (advisory).
  • Asked: Apply this repo's performance-review criteria (reviewers/performance.md, its .agents/dd-apm-sdk-review-overrides/reviewers/performance.md override, and
  • Tripped 1 bad signal(s):
    • Misses the unbounded / high-cardinality aspect entirely.
  • Baseline answer: Verdict: BLOCK Findings: - P0 | sessionCache field + recordSession (the put) | Unbounded map keyed by high-cardinality sessionId, only ever written, never evicted → heap grows without bound. Reviewer: Performance Why it matters: This is check unbounded-memory / addendum J5 — a Map with no size
  • Candidate answer: Verdict: BLOCK Findings: - P0 | (snippet) sessionCache — unbounded map keyed by per-session ID, no eviction | check: unbounded-memory (SEV-1) Reviewer: Performance Why it matters: sessionCache has no size bound and no byte bound, and recordSession only ever puts — keyed by sessionId (high-cardinality,

Results

• Pairwise win-rate: 89% [81%–97%] — candidate's share of blind comparisons (90% CI; spanning 50% = no clear difference)
• Overall quality: 69.4 → 86.2 (/100, +16.8)
• Bad signals introduced (advisory): 1
• Candidate criteria coverage (advisory): 32/33 (97%) — expected_criteria the candidate met; does not affect the gate
• Candidate bad signals (all, advisory): 1 case(s) — trips on the candidate, including ones the baseline also hit
• Blocking-case regressions: 0

Cases

Case	Mode	Quality Δ	Win-rate (90% CI)	Safety
java-perf-lens-wrong-collection-001	block	-5.0	50% [3%–97%]	⚠️ bad signal
java-perf-pipeline-full-review-002	block	+55.8	100% [100%–100%]	ok
java-security-crash-handler-before-trust	block	+6.7	100% [100%–100%]	ok
java-correctness-capture-before-send	block	+12.5	88% [67%–100%]	ok
java-correctness-sqs-queue-name-incomplete	block	+23.3	100% [100%–100%]	ok
java-maintainability-resource-leak-streams	block	+10.0	88% [67%–100%]	ok
java-correctness-span-events-list-only	block	+20.8	100% [100%–100%]	ok
java-correctness-mapper-state-leak	block	+10.4	88% [67%–100%]	ok

_{Per-dimension scores, token usage, latency, and estimated cost are in the CI job logs.}

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.81 s	14.67 s	[+0.1%; +1.9%] (maybe worse)
startup:insecure-bank:tracing:Agent	13.61 s	13.69 s	[-1.3%; +0.2%] (no difference)
startup:petclinic:appsec:Agent	17.02 s	16.88 s	[+0.0%; +1.7%] (maybe worse)
startup:petclinic:iast:Agent	16.97 s	17.05 s	[-1.3%; +0.4%] (no difference)
startup:petclinic:profiling:Agent	16.85 s	16.92 s	[-1.9%; +1.1%] (no difference)
startup:petclinic:sca:Agent	16.48 s	16.71 s	[-5.5%; +2.8%] (no difference)
startup:petclinic:tracing:Agent	16.15 s	16.12 s	[-0.8%; +1.2%] (no difference)

Commit: 1e6476df · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.