3-D launches over odd-sized ranges are slower than 1-D ones on an Intel iGPU #814

On an Intel iGPU, a memory-bound kernel launched over a 3-D ndrange with odd extents is about 20% slower than the same kernel launched over the flattened range, though both touch the same memory in the same order:

using oneAPI, KernelAbstractions

@kernel function copy_linear!(B, @Const(A))
    I = @index(Global, Linear)
    @inbounds B[I] = A[I]
end

function gpu_time(f; n = 200)
    f(); oneAPI.synchronize()
    minimum(@elapsed((f(); oneAPI.synchronize())) for _ in 1:n) * 1e3
end

k = copy_linear!(oneAPIBackend())
for dims in ((256, 256, 256), (255, 257, 129))
    A = oneArray(rand(Float32, dims)); B = similar(A)
    t3 = gpu_time(() -> k(B, A; ndrange = size(A)))
    t1 = gpu_time(() -> k(B, A; ndrange = length(A)))
    println("$dims: ndrange=size(A) $(round(t3; digits=2)) ms, ndrange=length(A) $(round(t1; digits=2)) ms")
end

On an Iris Xe (Tiger Lake), over three alternating runs of each version (timings vary by up to 10% between runs):

So it isn't a regression of the N-d launches from #797: KernelAbstractions 0.9, which launched everything on a 1-D grid but computed Cartesian indices with divisions, shows it too. With 0.10, it doesn't depend on the work-group shape or on the index computation either (255×257×129 only):

I don't know the cause. The sub-groups on this GPU are formed linearly (KernelInterface's partial sub-group test passes on it), so it isn't the sub-group layout. What remains is how the hardware or the driver dispatches a 3-D grid of work-groups with such extents. Intel's compute runtime chooses a hardware walk order based on the work-group shape (shared/source/helpers/hw_walk_order.h in intel/compute-runtime), although for Tiger Lake it should be linear.

The question for KernelAbstractions is whether select_launch should prefer a 1-D launch in cases like this, e.g. through a back-end hook, since only the back end can know whether N-d grids cost more. On the GPUs where it was measured for #797, the N-d launch is a clear win (CUDA: 86 → 41 µs for the same copy over 255×257×129), so it can't simply be turned off.

Measured on oneAPI.jl ka-0.10 (JuliaGPU/oneAPI.jl#653) and on the registered oneAPI.jl with KernelAbstractions 0.9.43.