Skip to content

Switch instrumentations over to ContextScope, remove AgentScope - #12557

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 21 commits into
masterfrom
mcculls/remove-agent-scope
Sep 22, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 21 commits into
masterfrom
mcculls/remove-agent-scope

Conversation

@mcculls

@mcculls mcculls commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

After the continuation migration AgentScope was left as a thin bridge interface involving ContextScope + TraceScope + span(). This PR finishes that migration: AgentScope is deleted entirely, and every scope-returning/consuming call site now uses ContextScope directly, with AgentSpan.fromScope(scope) / Java8BytecodeBridge.spanFromScope(scope) helpers replacing scope.span().

Note the AgentSpan.fromContext method used by the helpers is optimized for the common case where the span is the context.

  • Decoupled AgentScope from TraceScope first (isolated, low-risk step) to prove nothing depended on the deprecated async-propagation methods through that interface.
  • Added AgentSpan.fromScope(ContextScope) to extract the active span from a scope's context.
  • Migrated all scope-typed fields, locals, and method signatures from AgentScope to ContextScope across internal-api, dd-trace-core, dd-trace-ot, and all dd-java-agent instrumentation modules (~195 files), plus debugger, LLM Observability, and test modules.
  • Deleted AgentScope.java.
  • Updated documentation (ARCHITECTURE.md, docs/add_new_instrumentation.md, docs/how_instrumentations_work.md) and the apm-integrations skill reference to describe ContextScope instead of AgentScope, including fixing a pre-existing doc issue which mentioned Continuation in AgentScope (we now use ContextContinuation as the continuation type).
  • Cleaned up stray AgentScope mentions in comments (ConfigInversionLinter.kt, DDLLMObsSpanTest.groovy) and renamed AgentScope-named identifiers in StreamTaskContext/KafkaStreamTaskInstrumentation (kafka-streams-0.11) to match. Also corrected the ConfigInversionLinter.kt comment to say Undertow's legacy DD_UNDERTOW_CONTINUATION key stores a ContextContinuation, not a ContextScope.
  • Removed the now-stale NoopAgentScope JaCoCo coverage exclusion in internal-api/build.gradle.kts.

Motivation

AgentScope added a layer of indirection with no remaining unique behaviour. Every implementer is a ContextScope, and the default methods contributed by TraceScope were unused. Switching toContextScope simplifies the scope hierarchy and continues the tracer's move toward the generic Context API

Additional Notes

Performance

Ran several load-test runs using dd-java-agent/benchmark-integration with a real DDAgentWriter serializing via a local trace agent. Comparing this branch against master across workloads from <1ms to 10ms: throughput and latency are statistically indistinguishable between the two (differences within ~0.2%, well under normal run-to-run noise). No measurable performance impact.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@mcculls mcculls added comp: core Tracer core tag: no release notes Changes to exclude from release notes type: refactoring tag: ai generated Largely based on code generated by an AI or LLM labels Sep 18, 2026
@datadog-prod-us1-5

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.02 s 14.05 s [-1.2%; +0.8%] (no difference)
startup:insecure-bank:tracing:Agent 12.92 s 12.98 s [-1.4%; +0.3%] (no difference)
startup:petclinic:appsec:Agent 17.07 s 16.96 s [-0.3%; +1.5%] (no difference)
startup:petclinic:iast:Agent 16.32 s 16.96 s [-7.7%; +0.2%] (no difference)
startup:petclinic:profiling:Agent 16.51 s 16.92 s [-3.4%; -1.4%] (significantly better)
startup:petclinic:sca:Agent 16.98 s 16.76 s [+0.3%; +2.4%] (maybe worse)
startup:petclinic:tracing:Agent 16.14 s 16.08 s [-0.3%; +1.0%] (no difference)

Commit: 7e1a3550 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@pr-commenter

pr-commenter Bot commented Sep 18, 2026

Copy link
Copy Markdown

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1790021549 1790021876
end_time 2026-09-21T20:13:59 2026-09-21T20:19:26
git_branch master mcculls/remove-agent-scope
git_commit_sha a41bd4c 7e1a355
start_time 2026-09-21T20:12:30 2026-09-21T20:17:57
See matching parameters
Baseline Candidate
ci_job_id 2064876778 2064876778
ci_pipeline_id 138997142 138997142
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
git_commit_date 1790020808 1790020808

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 6 unstable metrics.

See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-23.652µs; +19.472µs] or [-7.150%; +5.886%]
unstable
[-34.460µs; +30.521µs] or [-9.509%; +8.422%]
unstable
[-43.464µs; +40.285µs] or [-11.551%; +10.706%]
unstable
[-60.069µs; +133.976µs] or [-5.537%; +12.350%]
same
scenario:basic same same same unstable
[+10.977µs; +207.244µs] or [+1.255%; +23.694%]
unstable
[-166.012op/s; +65.001op/s] or [-7.305%; +2.860%]
scenario:loop same same same same same
Request duration reports for reports
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (362.377 µs) : 334, 391
.   : milestone, 362,
basic (343.524 µs) : 337, 350
.   : milestone, 344,
loop (8.119 ms) : 8055, 8183
.   : milestone, 8119,
section candidate
noprobe (360.408 µs) : 327, 393
.   : milestone, 360,
basic (340.448 µs) : 334, 347
.   : milestone, 340,
loop (8.115 ms) : 8051, 8179
.   : milestone, 8115,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 362.377 µs [333.734 µs, 391.021 µs]
basic 343.524 µs [337.197 µs, 349.851 µs]
loop 8.119 ms [8.055 ms, 8.183 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 360.408 µs [327.491 µs, 393.324 µs]
basic 340.448 µs [333.782 µs, 347.113 µs]
loop 8.115 ms [8.051 ms, 8.179 ms]

@mcculls mcculls changed the title Remove AgentScope Remove AgentScope and switch instrumentations over to ContextScope Sep 18, 2026
@mcculls

mcculls commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

@DataDog review

@pr-commenter

pr-commenter Bot commented Sep 18, 2026

Copy link
Copy Markdown

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/remove-agent-scope
git_commit_date 1789676683 1789726786
git_commit_sha d5f7fcb c1c5a4d
See matching parameters
Baseline Candidate
ci_job_date 1789728325 1789728325
ci_job_id 2057182433 2057182433
ci_pipeline_id 138425420 138425420
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
jdkVersion 11.0.31 11.0.31
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost *.localhost
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-edgipia2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-edgipia2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@datadog-prod-us1-5 datadog-prod-us1-5 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

The new ContextScope path keeps span extraction and scope life cycle behavior.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Datadog Autotest · Commit a002980 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

@mcculls
mcculls requested a lite review from Copilot September 18, 2026 08:33
@mcculls
mcculls requested review from mhlidd and removed request for a team September 18, 2026 10:09

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8a136127df

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@datadog-prod-us1-5 datadog-prod-us1-5 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

The migration keeps scope lifecycle order and gets spans from the scope context. Advice code uses the Java 8 bytecode bridge.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Datadog Autotest · Commit 8a13612 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

@pr-commenter

pr-commenter Bot commented Sep 18, 2026

Copy link
Copy Markdown

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/remove-agent-scope
git_commit_date 1790015524 1790020808
git_commit_sha a41bd4c 7e1a355
See matching parameters
Baseline Candidate
ci_job_date 1790022032 1790022032
ci_job_id 2064876775 2064876775
ci_pipeline_id 138997142 138997142
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
jdkVersion 11.0.31 11.0.31
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost *.localhost
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-3rs8s7mp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-3rs8s7mp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@jpbempel jpbempel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for debugger part

@daniel-mohedano daniel-mohedano left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on CiVis side!

@sarahchen6 sarahchen6 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for LP

@sabrenner sabrenner left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm for mlobs!

@mcculls
mcculls added this pull request to the merge queue Sep 21, 2026
@dd-octo-sts

dd-octo-sts Bot commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Sep 21, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-09-21 21:15:14 UTC ℹ️ Start processing command /merge


2026-09-21 21:15:19 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).


2026-09-21 22:04:55 UTCMergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for 298f4dc:

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

  • Investigate the failures and when ready, re-add your pull request to the queue!
  • If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
  • Any question, go check the FAQ.
Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 21, 2026
@mcculls
mcculls added this pull request to the merge queue Sep 21, 2026
@dd-octo-sts

dd-octo-sts Bot commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Sep 21, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-09-21 22:54:16 UTC ℹ️ Start processing command /merge


2026-09-21 22:54:20 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).


2026-09-22 00:15:39 UTC ℹ️ MergeQueue: This merge request was merged

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 21, 2026
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit dd9d42a into master Sep 22, 2026
617 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the mcculls/remove-agent-scope branch September 22, 2026 00:15
@github-actions github-actions Bot added this to the 1.67.0 milestone Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.