Skip to content

Avoid boxing variables captured by closures - #1026

Merged
maleadt merged 2 commits into
mainfrom
tb/closure-boxes
Oct 13, 2026
Merged

maleadt merged 2 commits into
mainfrom
tb/closure-boxes

Conversation

@maleadt

@maleadt maleadt commented Oct 13, 2026

Copy link
Copy Markdown
Member

The metal/synchronization/stream benchmark (an empty synchronize()) regressed from ~340 to ~640 ns over #995 and the #1001/#1002/#822 stack. Part of that is inherent to the extra locking those PRs need, but part comes from closure boxes: splitting synchronize into two @autoreleasepool blocks made their closures capture queue and submissions, which were reassigned afterwards. Both got boxed, which made the rest of the function type unstable.

Julia 1.14's Test.detect_closure_boxes found 9 more of these, mostly in @autoreleasepool functions rebinding an argument (that macro wraps the function body in a closure), including encode_copy! on the copyto! path. This PR removes all of them without introducing temporaries, and adds a test that runs detect_closure_boxes on Julia versions that have it.

Empty synchronize() on an M1, Julia 1.12, three interleaved runs:

median allocations
main 587–594 ns 5 (112 bytes)
this PR 474–488 ns 2 (48 bytes)

Splitting `synchronize` into two `@autoreleasepool` blocks (#995, #1001) made
their closures capture `queue` and `submissions`, which were later
reassigned. That boxed both, making the rest of the function type unstable,
and raised the latency of an empty `synchronize` from ~310 to ~600 ns. Stop
rebinding them, and release the pool from a function that takes concrete
arguments, which brings it back to ~480 ns with 2 instead of 5 allocations.
Reassigning a variable that a closure captures boxes it. With
`@autoreleasepool function` wrapping the function body in a closure, this
happened to arguments that were rebound after validation, and to the byte
count `encode_copy!` decremented while copying. Test for this using
`Test.detect_closure_boxes`, where available.
@maleadt maleadt added the performance Runtime performance. label Oct 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Metal.jl Benchmarks

Details
Benchmark suite Current: f05014c Previous: fda32ae Ratio
array/accumulate/Float32/1d 394167 ns 394000 ns 1.00
array/accumulate/Float32/dims=1 363833 ns 369166 ns 0.99
array/accumulate/Float32/dims=1L 8871041 ns 8869708 ns 1.00
array/accumulate/Float32/dims=2 431250 ns 434333 ns 0.99
array/accumulate/Float32/dims=2L 2971750 ns 2724458 ns 1.09
array/accumulate/Int64/1d 775416 ns 775291 ns 1.00
array/accumulate/Int64/dims=1 872209 ns 886250 ns 0.98
array/accumulate/Int64/dims=1L 3786708 ns 3729709 ns 1.02
array/accumulate/Int64/dims=2 1162334 ns 1163458 ns 1.00
array/accumulate/Int64/dims=2L 16149083 ns 16005042 ns 1.01
array/broadcast 237917 ns 232584 ns 1.02
array/construct 2375 ns 2292 ns 1.04
array/permutedims/2d 339292 ns 339750 ns 1.00
array/permutedims/3d 461792 ns 461417 ns 1.00
array/permutedims/4d 619958 ns 617417 ns 1.00
array/private/copy 228708 ns 221750 ns 1.03
array/private/copyto!/cpu_to_gpu 224250 ns 224250 ns 1
array/private/copyto!/gpu_to_cpu 222500 ns 221417 ns 1.00
array/private/copyto!/gpu_to_gpu 220584 ns 211959 ns 1.04
array/private/iteration/findall/bool 525458 ns 530292 ns 0.99
array/private/iteration/findall/int 601375 ns 605917 ns 0.99
array/private/iteration/findfirst/bool 375834 ns 374292 ns 1.00
array/private/iteration/findfirst/int 369958 ns 378458 ns 0.98
array/private/iteration/findmin/1d 373792 ns 375417 ns 1.00
array/private/iteration/findmin/2d 1109750 ns 881167 ns 1.26
array/private/iteration/logical 540375 ns 533709 ns 1.01
array/private/iteration/scalar 1384083 ns 1394709 ns 0.99
array/random/rand/Float32 436750 ns 441958 ns 0.99
array/random/rand/Int64 515875 ns 514375 ns 1.00
array/random/rand!/Float32 426833 ns 428250 ns 1.00
array/random/rand!/Int64 448166 ns 449958 ns 1.00
array/random/randn/Float32 350083 ns 366708 ns 0.95
array/random/randn!/Float32 354208 ns 354083 ns 1.00
array/reductions/mapreduce/Float32/1d 234458 ns 223417 ns 1.05
array/reductions/mapreduce/Float32/dims=1 389083 ns 388542 ns 1.00
array/reductions/mapreduce/Float32/dims=1L 584125 ns 586917 ns 1.00
array/reductions/mapreduce/Float32/dims=2 324125 ns 323083 ns 1.00
array/reductions/mapreduce/Float32/dims=2L 457875 ns 451417 ns 1.01
array/reductions/mapreduce/Int64/1d 233708 ns 244208 ns 0.96
array/reductions/mapreduce/Int64/dims=1 406791 ns 406917 ns 1.00
array/reductions/mapreduce/Int64/dims=1L 1000875 ns 1004584 ns 1.00
array/reductions/mapreduce/Int64/dims=2 329625 ns 273167 ns 1.21
array/reductions/mapreduce/Int64/dims=2L 603584 ns 619750 ns 0.97
array/reductions/reduce/Float32/1d 228916 ns 235584 ns 0.97
array/reductions/reduce/Float32/dims=1 388125 ns 388125 ns 1
array/reductions/reduce/Float32/dims=1L 584875 ns 589709 ns 0.99
array/reductions/reduce/Float32/dims=2 301500 ns 316292 ns 0.95
array/reductions/reduce/Float32/dims=2L 456250 ns 442458 ns 1.03
array/reductions/reduce/Int64/1d 248792 ns 237000 ns 1.05
array/reductions/reduce/Int64/dims=1 326917 ns 405750 ns 0.81
array/reductions/reduce/Int64/dims=1L 1000667 ns 1004416 ns 1.00
array/reductions/reduce/Int64/dims=2 335125 ns 331500 ns 1.01
array/reductions/reduce/Int64/dims=2L 604666 ns 620583 ns 0.97
array/shared/copy 128583 ns 128875 ns 1.00
array/shared/copyto!/cpu_to_gpu 37625 ns 37583 ns 1.00
array/shared/copyto!/gpu_to_cpu 36583 ns 38125 ns 0.96
array/shared/copyto!/gpu_to_gpu 36625 ns 38000 ns 0.96
array/shared/iteration/findall/bool 530417 ns 528875 ns 1.00
array/shared/iteration/findall/int 603541 ns 607834 ns 0.99
array/shared/iteration/findfirst/bool 374875 ns 377291 ns 0.99
array/shared/iteration/findfirst/int 379041 ns 378333 ns 1.00
array/shared/iteration/findmin/1d 369417 ns 373375 ns 0.99
array/shared/iteration/findmin/2d 870042 ns 1113333 ns 0.78
array/shared/iteration/logical 536708 ns 547333 ns 0.98
array/shared/iteration/scalar 77.3076923076923 ns 82.34536082474227 ns 0.94
array/sorting/1d 1912167 ns 1919167 ns 1.00
array/sorting/2d 8354166 ns 8351750 ns 1.00
integration/byval/reference 1114875 ns 1121708 ns 0.99
integration/byval/slices=1 1115500 ns 1122125 ns 0.99
integration/byval/slices=2 2025917 ns 2028125 ns 1.00
integration/byval/slices=3 6790792 ns 6844167 ns 0.99
integration/metaldevrt 395875 ns 396500 ns 1.00
kernel/indexing 215625 ns 212083 ns 1.02
kernel/indexing_checked 391333 ns 395250 ns 0.99
kernel/launch 1537.5 ns 1545.9 ns 0.99
kernel/rand 384750 ns 406958 ns 0.95
latency/import 1995882417 ns 2011861958 ns 0.99
latency/precompile 30889583416 ns 30939510500 ns 1.00
latency/ttfp 2236765459 ns 2249344208 ns 0.99
metal/synchronization/context 527.8526315789474 ns 531.3578947368421 ns 0.99
metal/synchronization/stream 574.9071038251366 ns 660.1925465838509 ns 0.87

This comment was automatically generated by workflow using github-action-benchmark.

@codecov

codecov Bot commented Oct 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.77%. Comparing base (fda32ae) to head (f05014c).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1026      +/-   ##
==========================================
+ Coverage   86.75%   86.77%   +0.01%     
==========================================
  Files          89       89              
  Lines        6403     6403              
==========================================
+ Hits         5555     5556       +1     
+ Misses        848      847       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@maleadt
maleadt merged commit c7f2b1d into main Oct 13, 2026
18 checks passed
@maleadt
maleadt deleted the tb/closure-boxes branch October 13, 2026 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Runtime performance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant