out of memory and huge cargo-flamegraph.stacks

[mosinnik]

I got simple simple test project with benches that I need to profile (https://github.com/mosinnik/l2_geo_rust)

If run any of follow commands:

cargo flamegraph --profile=bench --bench geo_benchmark -- --bench
cargo flamegraph --bench geo_benchmark -- --bench
cargo flamegraph --bench geo_benchmark -- --bench get_nearest_z_flat

With any of benches durations I got out of memory and huge cargo-flamegraph.stacks. With RUST_BACKTRACE=full :

Error: failed to read dtrace expected output file 'cargo-flamegraph.stacks'

Caused by:
    out of memory

Stack backtrace:
   0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: BaseThreadInitThunk
  11: RtlUserThreadStart

In last try on empty 1tb disk I got 135Gb cargo-flamegraph.stacks file. Last screen before deleted:

windows 10
cargo 1.94.0 (85eff7c80 2026-01-15)
flamegraph-flamegraph 0.6.11
Checked for criterion = "0.5.1" and "0.8"

Main strange thing is that first tries work well - I took flamegrapfs of my 4 benches few times? but after something happen and it broke somehow. Reboots, cleaning, rebuilds, changes dirs - nothing changes

Memory consumption while stack file growing

If I run simple cargo flamegraph --release it work well and I got flamegraph.svg
If I run cargo bench - it work well too

Trying to install dtrace for windows - same thing and almost same error messages:

dtrace: pid 3252 has exited
writing flamegraph to "flamegraph.svg"
Error: unable to generate a flamegraph from the collapsed stack data

Caused by:
    No stack counts found

So as I see problem inside flamegraph

[mosinnik]

main reason is usage as a symbol_name a function with all structure inside blondie.

for example one of symbol_name (it is really only ONE function name):

geo_benchmark-730f6b5d71dfe328.exe`rayon_core::join::join_context::closure$0<rayon::iter::plumbing::bridge_producer_consumer::helper::closure_env$0<rayon::range::IterProducer<usize>,rayon::iter::map_with::MapInitConsumer<rayon::iter::reduce::ReduceConsumer<criterion::stats::univariate::bootstrap::closure_env$3<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)>,criterion::stats::univariate::bootstrap::closure_env$2<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)> >,criterion::stats::univariate::bootstrap::closure_env$0<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)>,criterion::stats::univariate::bootstrap::closure_env$1<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)> > >,rayon::iter::plumbing::bridge_producer_consumer::helper::closure_env$1<rayon::range::IterProducer<usize>,rayon::iter::map_with::MapInitConsumer<rayon::iter::reduce::ReduceConsumer<criterion::stats::univariate::bootstrap::closure_env$3<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)>,criterion::stats::univariate::bootstrap::closure_env$2<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)> >,criterion::stats::univariate::bootstrap::closure_env$0<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)>,criterion::stats::univariate::bootstrap::closure_env$1<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)> > >,tuple$<alloc::vec::Vec<f64,alloc::alloc::Global>,alloc::vec::Vec<f64,alloc::alloc::Global> >,tuple$<alloc::vec::Vec<f64,alloc::alloc::Global>,alloc::vec::Vec<f64,alloc::alloc::Global> > >(rayon_core::join::join_context::closure_env$0<rayon::iter::plumbing::bridge_producer_consumer::helper::closure_env$0<rayon::range::IterProducer<usize>,rayon::iter::map_with::MapInitConsumer<rayon::iter::reduce::ReduceConsumer<criterion::stats::univariate::bootstrap::closure_env$3<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)>,criterion::stats::univariate::bootstrap::closure_env$2<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)> >,criterion::stats::univariate::bootstrap::closure_env$0<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)>,criterion::stats::univariate::bootstrap::closure_env$1<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)> > >,rayon::iter::plumbing::bridge_producer_consumer::helper::closure_env$1<rayon::range::IterProducer<usize>,rayon::iter::map_with::MapInitConsumer<rayon::iter::reduce::ReduceConsumer<criterion::stats::univariate::bootstrap::closure_env$3<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)>,criterion::stats::univariate::bootstrap::closure_env$2<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)> >,criterion::stats::univariate::bootstrap::closure_env$0<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)>,criterion::stats::univariate::bootstrap::closure_env$1<f64,f64,tuple$<f64,f64>,tuple$<f64,f64> (*)(ref$<criterion::stats::univariate::sample::Sample<f64> >,ref$<criterion::stats::univariate::sample::Sample<f64> >)> > >,tuple$<alloc::vec::Vec<f64,alloc::alloc::Global>,alloc::vec::Vec<f64,alloc::alloc::Global> >,tuple$<alloc::vec::Vec<f64,alloc::alloc::Global>,alloc::vec::Vec<f64,alloc::alloc::Global> > >, rayon_core::registry::WorkerThread*, bool)

So it cause for one stack take hundreds kilobytes. as result half million frames took 50Gb text in stack file.

Replacing function name by file + line make it more realistic with few hundred Megabytes. yes it not good too, but I got my flamegraph.

Made just to check:

            for frame in &procedure_frames.frames {
                // symbol_names.push(frame.function.as_deref().unwrap_or("Unknown"));
                let file = frame.file.as_deref().unwrap_or("Unknown");
                let line_str = match frame.line {
                    Some(l) => Box::leak(format!(":{}", l).into_boxed_str()), // leak but work
                    None => "",
                };
                symbol_names.push(format!("{}{}", file, line_str).leak()); 
            }

Fflamegraph with tons of windows benches functions:

[djc]

Unfortunately I won't have time to look into this with any amount of detail any time soon (unless your organization wants to sponsor this work) -- if you have want to propose a design and implement it, I can probably review.