Skip to content

Add native tracing fields to LogEvent to eliminate async MDC overhead#4171

Merged
ramanathan1504 merged 9 commits into
2.xfrom
fix-trace-metadata-1976
Jul 6, 2026
Merged

Add native tracing fields to LogEvent to eliminate async MDC overhead#4171
ramanathan1504 merged 9 commits into
2.xfrom
fix-trace-metadata-1976

Conversation

@ramanathan1504

@ramanathan1504 ramanathan1504 commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Fixes #1976 by introducing a TraceContextProvider SPI and adding
native W3C tracing fields (traceId, spanId, traceFlags) directly to LogEvent.

This allows frameworks like OpenTelemetry to bypass the ThreadContext map
entirely, eliminating the map-cloning memory overhead previously
required during asynchronous logging.

rgoers
rgoers approved these changes Jul 5, 2026
View reviewed changes

@rgoers rgoers left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see no reason not to do this. The implemenation looks pretty simple and clean. However, I don't see an integration with JsonTemplateLayout. Is one needed? Without that how would the data get injected into ElasticSearch?

@ramanathan1504

ramanathan1504 commented Jul 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Hi @rgoers, that is a great point. I hadn't included the JsonTemplateLayout integration yet, but it is definitely needed to support structured logging properly without falling back to intermediate pattern allocations.

I will open a new, follow-up PR with the native JTL resolvers (TraceIdResolver, SpanIdResolver, TraceFlagsResolver) and link it back to this one.

@ramanathan1504
ramanathan1504 merged commit b6ba7d0 into 2.x Jul 6, 2026
10 checks passed
@ramanathan1504
ramanathan1504 deleted the fix-trace-metadata-1976 branch July 6, 2026 18:28
@github-project-automation github-project-automation Bot moved this from Approved to Merged in Log4j pull request tracker Jul 6, 2026
@ramanathan1504 ramanathan1504 mentioned this pull request Jul 6, 2026
Open
@ramanathan1504

ramanathan1504 commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

Hi @rgoers, the follow-up PR for the native JsonTemplateLayout resolvers is now open here: #4175 Thanks again for the guidance!

ramanathan1504 added a commit to ramanathan1504/logging-log4j2 that referenced this pull request Jul 9, 2026
@ramanathan1504
Add native tracing fields to LogEvent to eliminate async MDC overhead (…
1f9a514 
…apache#4171)

* Add tracing fields to RingBufferLogEvent and related classes

* Add W3C trace context support with pattern converters for trace ID, span ID, and trace flags

* Enhance TraceContextProviderService for exception safety and simplify trace ID retrieval

* Add tests for tracing fields serialization and pattern converters

* Refactor tracing metadata documentation and improve code comments for clarity

* Add native W3C tracing fields to LogEvent and introduce TraceContextProvider SPI

* Add benchmark for ContextDataProvider tracing approach

* Enhance TraceContextProviderService to handle SecurityManager restrictions gracefully
This was referenced Jul 17, 2026
Open
Draft
ramanathan1504 added a commit that referenced this pull request Jul 24, 2026
@ramanathan1504 @ppkarwasz
@dependabot @github-actions @SebTardif @bjmi @vy @jethomas-tsi @SunWeb3Sec
Fix NPE in ConfigurationScheduler by ensuring scheduledFuture is chec… (
de17f60 
#4118)

* Fix NPE in ConfigurationScheduler by ensuring scheduledFuture is checked for null before accessing its methods; add test for race condition in scheduling.

* Restrict JUnit dependencies to below version 6.x to maintain compatib… (#4120)

* Restrict JUnit dependencies to below version 6.x to maintain compatibility with Java 8

* Update JUnit dependency restrictions for compatibility with Java 8 on 2.x

* Don't override `apache-rat-plugin` version (#4123)

Remove the version override for `apache-rat-plugin` so it falls back to the version provided by the ASF Parent POM.

Apache RAT `0.18`, for which Dependabot opened an update PR, contains a bug ([RAT-552](https://issues.apache.org/jira/browse/RAT-552)) that effectively disables the `excludeSubProjects` plugin option whenever `<excludes>` is present in the configuration.

Upgrading to `0.18` will require converting our `<excludes>` to `<inputExcludes>`, which depends on a new `logging-parent` release.

* Bump actions/stale from 9.1.0 to 10.2.0 in the dependencies group across 1 directory (#4121)

* Bump actions/stale in the dependencies group across 1 directory

Bumps the dependencies group with 1 update in the / directory: [actions/stale](https://github.com/actions/stale).


Updates `actions/stale` from 9.1.0 to 10.2.0
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](actions/stale@5bef64f...b5d41d4)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-version: 10.2.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* Generate changelog entries for #4121

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix KafkaAppender reporting error after successful retry (#4125)

* Fix KafkaAppender reporting error after successful retry

When retryCount is configured and the initial tryAppend() fails, the
retry loop uses break to exit on success. However, break only exits the
while loop and execution always reaches the error() call afterward. This
causes spurious error notifications for transient Kafka failures that
were successfully recovered by a retry.

Replace break with return so that a successful retry exits append()
without reporting an error. Retry exceptions are now logged at DEBUG
level for diagnostics instead of being silently discarded.

Also remove dead code in Builder.getRetryCount() where Integer.valueOf(int)
was wrapped in a NumberFormatException catch that can never fire.

The bug was introduced in #315.

Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>

* Add changelog entry for KafkaAppender retry fix

Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>

* Use CloseableThreadContext in testRetrySuccessDoesNotReportError

Adopt reviewer suggestion to use try-with-resources CloseableThreadContext
instead of manual ThreadContext.put/clearMap.

---------

Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>

* Revamped `ListAppender` for thread-safety (#4111)

Co-authored-by: Björn Michael <b.michael@gmx.de>
Co-authored-by: Volkan Yazıcı <volkan.yazici@oracle.com>

* Improve `CronExpression` tests to cover daylight saving and scheduling logic (#4081)

Co-authored-by: Piotr P. Karwasz <pkarwasz-github@apache.org>
Co-authored-by: Volkan Yazıcı <volkan@yazi.ci>

* Fix `createOnDemand` for Rolling File Appender (#4072)

Co-authored-by: Piotr P. Karwasz <pkarwasz-github@apache.org>
Co-authored-by: Volkan Yazıcı <volkan@yazi.ci>

* Add support for max compression delay in compression actions (#4071)

Co-authored-by: Volkan Yazıcı <volkan@yazi.ci>

* Improve logging for `LinkageError` scenarios involving the LMAX Disruptor library (#4124)

Co-authored-by: Volkan Yazıcı <volkan@yazi.ci>

* Fix changelog issue and PR references for Disruptor initialization error logging (#2250, #4124, #4134)

* Fix encoding of `MSGID` and `SD-ID` fields of `StructuredDataMessage` to XML (#4136)

* Fix stack trace rendering for exceptions with identity malfunction (#4133)

* Fix circular reference detection for exceptions with colliding equals/hashCode implementations

* Add changelog entry for circular reference detection fix with colliding equals/hashCode exceptions

* Add changelog entry for circular reference detection fix with colliding equals/hashCode exceptions

* Add test for ThrowableProxy serialization with colliding equals/hashCode implementations

* Refactor ThrowableProxy serialization test to use modern Java I/O classes

* Use `TestFriendlyException` to exercise the malfunction

* Update changelog

* Improve comments

* Remove redundant change

---------

Co-authored-by: Volkan Yazıcı <volkan@yazi.ci>

* Remove the `patternFlags` attribute from `RegexFilter` (#4119)

Co-authored-by: Jeff Thomas <jeffrey.thomas@t-systems.com>
Co-authored-by: Volkan Yazıcı <volkan@yazi.ci>

* Fix `DatePatternConverter` locale parsing when timezone is omitted (#4130)

* Harden `readObject(ObjectInputStream)` method argument checks (#4098)

Signed-off-by: SunWeb3Sec <infosecpt@gmail.com>
Co-authored-by: Volkan Yazıcı <volkan@yazi.ci>

* Fix resource leaks in `ConfigurationSource` when loading via URL (#4127)

Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>
Co-authored-by: Ramanathan <ramanathanbscmca@gmail.com>
Co-authored-by: Piotr P. Karwasz <pkarwasz-github@apache.org>
Co-authored-by: Volkan Yazıcı <volkan@yazi.ci>

* Adopt the Dependabot changelog 'draft trick' (#4148)

Applies the consumer-side changes from logging-parent PR #476:

* `build.yaml` and `codeql-analysis.yaml` now subscribe to the
  `ready_for_review` pull request type, so required checks re-run when a
  Dependabot PR is taken out of draft.
* `process-dependabot.yaml` drops the `RECURSIVE_TOKEN` PAT secret, which
  is no longer needed now that the reusable workflow parks the PR in
  draft mode instead of pushing with a privileged token.

Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Add Dependabot config for the 2.26.x maintenance branch (#4149)

Configure a Maven update entry targeting the `2.26.x` branch that only
proposes patch-level upgrades. All dependencies are bundled into a single
`Maven patch updates` group with a 7-day cooldown.

* Fix handling of non-finite numbers while encoding `MapMessage` to JSON (#4163)

* Remove `/log4j-slf4j-impl` Dependabot execution (#4151)

Instead of correcting the target branch, we can safely
remove it: the only dependency version defined in
`log4j-slf4j-impl` is SLF4J 1.x, which is highly unlikely
to see any new releases.

* Upgrade Maven and Maven Wrapper versions to `3.9.16` and `3.3.4`, respectively (#4167)

* Adds `maxRandomDelay` configure to Rolling Appenders (#4165)

* Rework random delay for Rolling Appender action chain

* Stabilize `RollingAppenderDirectCronTest`

* Merge `2.26.1` to `2.x` (#4169)

* Add `2.25.5` release notes (#4174)

* Add native tracing fields to LogEvent to eliminate async MDC overhead (#4171)

* Add tracing fields to RingBufferLogEvent and related classes

* Add W3C trace context support with pattern converters for trace ID, span ID, and trace flags

* Enhance TraceContextProviderService for exception safety and simplify trace ID retrieval

* Add tests for tracing fields serialization and pattern converters

* Refactor tracing metadata documentation and improve code comments for clarity

* Add native W3C tracing fields to LogEvent and introduce TraceContextProvider SPI

* Add benchmark for ContextDataProvider tracing approach

* Enhance TraceContextProviderService to handle SecurityManager restrictions gracefully

* Fix NPE in ConfigurationScheduler by ensuring proper reset of scheduled futures

* Fix NPE in ConfigurationScheduler for frequent cron schedules

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>
Signed-off-by: SunWeb3Sec <infosecpt@gmail.com>
Co-authored-by: Piotr P. Karwasz <pkarwasz-github@apache.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sebastien Tardif <SebTardif@ncf.ca>
Co-authored-by: Björn Michael <b.michael@gmx.de>
Co-authored-by: Volkan Yazıcı <volkan.yazici@oracle.com>
Co-authored-by: Volkan Yazıcı <volkan@yazi.ci>
Co-authored-by: Jeff Thomas <jeffrey.thomas@t-systems.com>
Co-authored-by: SunWeb3Sec <107249780+SunWeb3Sec@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rgoers rgoers rgoers approved these changes

Assignees

No one assigned

Labels

None yet

Projects

Log4j pull request tracker
Status: Merged

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Log events should have trace/span-related metadata

2 participants

@ramanathan1504 @rgoers