You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
#[unroll] over the clone budget leaves an array-indexing loop rolled and moves the array to .local, with no way to raise the budget #1478
When #[unroll] on the loop that selects an element of a local array goes over the 65536-operation clone budget (MAX_CLONED_OPS in mir-transforms/src/unroll.rs), the loop stays rolled. The whole array then moves to .local memory. The only signal is a warning, and there's no way to raise the budget for that loop.
The budget counts the outer loop's body after its inner #[unroll] loops have already been expanded. So a typical tensor-core accumulator pattern goes over budget at modest sizes:
#[kernel]pubfnacc_tiles(a:*constf32,b:*constf32,out:*mutf32){let tid = thread::threadIdx_x()asusize;letmut acc = [[0.0f32;8];32];letmut t = 0usize;#[unroll]while t < 32{letmut j = 0usize;#[unroll]while j < J{letmut n = 0usize;#[unroll]while n < 8{let x = unsafe{*a.add(tid + (j *32 + t)*128)};let y = unsafe{*b.add(tid + (j *8 + n)*128)};
acc[t][n] += x * y;
n += 1;}
j += 1;}
t += 1;}// store acc with the same unrolled t/n loops}
On main (bafffe45), sm_80, ptxas 13.0.88 -v:
J
warning
acc
registers
stack / spill
1
none
registers
72
0 / 0
3
none
registers
168
0 / 0
8
not unrolled (152672 ops)
.local, 792 LDL/STL
96
1024 B / 0
32
not unrolled (609632 ops)
.local, 944 LDL/STL
255
1184 B / 160 B
warning: #[unroll] requested but the loop was not unrolled: unrolling would clone 152672 operations
(32 copies x 4771 operations); the safety limit is 65536 cloned operations
At J = 8, ptxas still has registers to spare, but the 1 KiB accumulator array sits in .local. The inner loops unroll correctly; only the loop that selects acc[t] stays rolled. In speakrs, this decided several kernel shapes (crates/speakrs-cuda-kernels/src/segdense.rs, the spk_segdense_conv1_b32_x3 and spk_segdense_embed_b32_x3 instances). We split work across more warps or kernels so the unrolled body fits, even where a larger tile per warp would have been faster.
Proposed behavior
Either or both:
A per-loop override, such as #[unroll(full)] or #[unroll(max_ops = N)], that unrolls even past the default budget. That's like CUDA C's #pragma unroll, which has no size limit.
A more precise diagnostic: name the local array that will move to .local because the loop stays rolled. Optionally, make it an error when the loop index selects an aggregate local, since the result is rarely what the author wants.
Alternatives considered
Restructuring kernels so the expanded body fits, which is what we do now.
Related: #1346 (for range loops not unrolled, same .local consequence), #1375 and #1468 (loop metadata and #[nounroll]).
Activity
feat(unroll): recognize range `for` loops (#1346)