Close jar entry streams during Symbol Database scan - #12143
Conversation
SymbolAggregator.parseJarEntry() never closed the jar entry stream, so the JarFile allocated a fresh native Inflater for every class file instead of reusing its cached one. Scanning a large jar could burst hundreds of MB of off-heap memory and OOMKill containers running with tight memory limits. Also closes the Files.walk() stream leaked by the directory scan path.
30aeef0 to
6c397b1
Compare
There was a problem hiding this comment.
More details
The resource-lifetime changes are behavior-preserving for normal JAR and directory scans: entry streams and the directory walk are now closed without changing filtering or parsing order. No diff-only regression was identified; focused execution was blocked because the wrapper could not download Gradle and the available JDKs do not include the required Java 25 toolchain.
🤖 Datadog Autotest · Commit 30aeef0 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
|
🎯 Code Coverage (details) 🔗 Commit SHA: 6c397b1 | Docs | Datadog PR Page | Give us feedback! |
Debugger benchmarksParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 6 unstable metrics.
See unchanged results
Request duration reports for reportsgantt
title reports - request duration [CI 0.99] : candidate=None, baseline=None
dateFormat X
axisFormat %s
section baseline
noprobe (394.855 µs) : 337, 452
. : milestone, 395,
basic (317.42 µs) : 308, 327
. : milestone, 317,
loop (9.002 ms) : 8994, 9011
. : milestone, 9002,
section candidate
noprobe (354.377 µs) : 325, 384
. : milestone, 354,
basic (313.778 µs) : 305, 322
. : milestone, 314,
loop (9.017 ms) : 9010, 9023
. : milestone, 9017,
|
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
0aac80a
into
master
What Does This Do
Closes the JAR entry
InputStreaminSymbolAggregator.parseJarEntry(), and closes theFiles.walk()stream in the directory scan path.Motivation
Symbol Database scans every
.classentry of every loaded JAR. The entry stream was never closed, soJarFilecould not return itsInflaterto the internal cache and allocated a new native zlib context per class file instead. On a large JAR this bursts hundreds of MB of off-heap memory that is invisible to heap metrics and unbounded byMaxDirectMemorySize, which is enough to OOMKill a container with a tight memory limit.parseFileEntry()on the directory path already closed its stream; the JAR path was the outlier.Reported with a reproduction in #12123.
Additional Notes
Added a regression test that asserts every entry stream handed out during a scan is closed.
Jira ticket: DYNIS-66