[0z5a]

What

fast_divmod returns wrong quotients for many divisors. fast_divmod(7, 7) gives 0. This PR changes how the multiplier is precomputed so that it is exact for every dividend fast_divmod accepts.

Why

cuda_fastdiv_precompute_m searched for the Hacker's Delight multiplier for the whole uint32 range. For some divisors that multiplier needs 33 bits, and return uint32(m) dropped the top bit. For b = 7 it computed m = 2^32 + 613566757, s = 3, and the kernel ran with m = 613566757.

Of the divisors 1 to 1024, 379 are affected. For example, b = 56 is wrong at a = 56, where it returns 0.

This reaches real kernels. hopper_matmul matmul_v4 to v6 and blackwell_matmul matmul_v4 to v6 divide linear_idx by num_m_blocks * swizzle_size. In hopper_matmul/matmul_v4 (block_m = 128, block_n = 256, swizzle_size = 4), m_size = 1792 and n_size = 4096 give a divisor of 56, and 168 of the 224 tiles get a wrong group_idx from the same multiplier.

How

The dividend of fast_divmod is a non-negative int32: cuda_fastdiv takes a: i32, and the docstring says "int32, must be >= 0". So the multiplier only has to be exact below 2^31, and then it fits in 32 bits:

• p = 31 + ceil(log2(b))
• m = ceil(2^p / b), which is below 2^32
• shift = p - 32

This is the formula in find_divisor in CUTLASS fast_math.h. The runtime (mul.hi.u32 + shr.u32) and the power-of-2 path do not change.

Dividends of 2^31 and above stay out of contract, and the docstrings now say 0 <= a < 2^31. They were never reliable: the power-of-2 path shifts an int32 arithmetically. Every caller in the repo passes a non-negative int32 block index.

Testing

Host tests ran on the Grace CPU of a GH200: GCC 13.3, CUDA 13.2, PyTorch 2.14.1+cu130, tilus main at f23018a. The host-side numbers below run the real cuda_fastdiv_precompute_m/s, built into a cpu_kernel with nvcc, and evaluate mul.hi + shr in Python.

check (host, real precompute code)	main	this PR
Divisors 1 to 1024 with any wrong quotient (all a < 2^20, plus the top of the range)	379	0
Non-power-of-2 divisors below 2^20, wrong at the worst-case dividend (largest k*b - 1 < 2^31)	323313 of 1048555	0
b = 7, 56, 641, 1023, 2^31 - 1, all a in [0, 2^31)	not run	0 wrong
matmul_v4 shape above, tiles with a wrong group_idx	168 of 224	0

tests/instructions/test_fast_divmod.py is new:

test	needs	main	this PR
test_precomputed_multiplier_is_exact (1065 divisors, boundary dividends)	nvcc, no GPU	fails: 399 of 1065 wrong	passes
test_fast_divmod[0-7], [0-56], [2147483392-7]	GPU	3 fail	3 pass
test_fast_divmod[0-3], [0-64] (controls)	GPU	pass	pass

GPU tests ran on GH200 (sm_90, CUDA 13.2). Not run on B300.

The rest of the CPU-runnable suite (tests/lang, tests/ir, tests/drivers, tests/transforms and the top-level tests) gives the same result on main and here: 192 passed, and the same 5 failed because they need a GPU.

ruff check and ruff format --check (0.11.0 from pyproject.toml, 0.12.3 from pre-commit), mypy (1.15.0) and the copyright check pass on the changed files.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yaoyaoding]

/ok to test e54034a