Vendor mlx core v0.32.3 (with a floor_divide fix for unsigned integers)

[CodeAndCanvas728]

Takes the mlx core from v0.32.2 to v0.32.3 (the follow-up proposed in SharpAI/SwiftLM#204). The core PRs it brings in include #4505 and #4487 (lower SDPA memory), #4567 and #4572 (gather_mm / gather_qmm), #4458/#4483 (global scale in qmm), and #4020 (fast gated delta kernels).

What is in it

1. Vendor v0.32.3. Source/Cmlx/mlx is upstream v0.32.3 plus the carried patches (SSD streaming, TurboKV, iOS I/O guards, quieter Metal logging), merged with git rather than copied. The only conflicts were in mlx/io/load.{h,cpp}, where 0.32.3 reworked ParallelFileReader's thread pool (#4408). The upstream design is kept and only the iOS guards are carried on top. mlx-generated and the include dirs are regenerated with tools/update-mlx.sh. mlx-c stays at ebc88f1 (its main is a version bump on top).
2. Build fixes for the new core.
   • Exclude the three new CUDA sources from the Cmlx target in Package.swift.
   • mlx_gather_qmm passes nullopt for the core's new global_scale argument (mlx #4458). The C API does not expose it yet.
   • FP8Tests now expects NaN for the E4M3FN bytes 0x7F/0xFF, which 0.32.3 decodes correctly (#4376).
3. Backport of mlx #4594 (landed after 0.32.3): the gated-delta NAX kernel does not compile with the macOS 27 SDK, which breaks the mlx.metallib build with -DMLX_ENABLE_NAX=1. Drop it when the core moves past #4594.
4. Fork patch: skip the signed floor_divide correction for unsigned integers. See below.

The floor_divide finding

With plain 0.32.3, a request through SwiftLM --stream-experts died with a Metal GPU timeout (kIOGPUCommandBufferCallbackErrorTimeout) on its first decode step, every time, on a 35B-A3B MoE. 0.32.2 never failed the same request.

Bisecting 0.32.2..0.32.3 gives #4515 "Fix floor_divide for integers" as the first bad commit (the commit right before it is good). It makes integer floor_divide correct for negative operands by adding remainder, less, not_equal, logical_and and subtract after the divide. mlx-swift-lm's MoE expert sort calls order.floorDivide(topK) on uint32 sort indices in every layer, so every layer now builds seven ops where it built one. For unsigned dtypes the correction is always zero, so the fork returns the plain quotient there. Signed integers keep upstream's behaviour. FloorDivideTests covers both.

I have not found why the extra ops lead to a timeout rather than just being slower, and standalone floor_divide is correct and equally fast on both cores. Reverting #4515 alone also makes the crash go away. I can write this up for ml-explore/mlx if useful.

Measurements

16 GB M2 Air, Qwen3.5-35B-A3B 4-bit (mixed 4/5/8-bit affine), SwiftLM --stream-experts, SwiftLM's profile_runner.py restricted to the Baseline and TurboQuant KV configs (the MTP configs do not apply to this model). Two alternating runs each, 0.32.2 vs this PR:

Config / context	TTFT 0.32.2	TTFT this PR	Decode tok/s 0.32.2	Decode tok/s this PR
Baseline, 512	12.5 / 11.7 s	11.7 / 12.0 s	5.7 / 5.2	5.2 / 5.1
Baseline, 8,000	57.0 / 54.9 s	56.4 / 63.1 s	5.8 / 5.5	5.6 / 5.5
TurboQuant KV, 512	11.7 / 11.1 s	11.6 / 11.3 s	5.6 / 5.5	5.7 / 5.3
TurboQuant KV, 8,000	60.9 / 57.2 s	59.7 / 57.3 s	1.4 / 1.4	1.3 / 1.5

The 0.32.3 wins that target newer chips (#4505's lower SDPA peak, #4567/#4572 gather_mm) do not show on an M2 with this model: it is parity, not a speedup. Without the floor_divide patch the same build crashed on a code prompt and decoded about 20% slower on the profile prompt. A per-op microbenchmark (quantized matmul at 4/5/8 bit, gather_qmm, SDPA at D=256, router top-k) showed no difference in speed or accuracy between the two cores. On a model that fits in RAM (no --stream-experts) decode was identical (about 50 tok/s on both).

Tests

• xcodebuild test -scheme mlx-swift-Package: 394 tests, 0 failures (2 skipped).
• SwiftLM swift test --filter SwiftLMTests against this branch: 276 tests, 0 failures.
• SwiftLM release build with ./build.sh (Xcode 27.0).
• Output check: the three code-prompt decode runs per core produce sane text.

Notes

• If upstream ml-explore/mlx-swift bumps its core to 0.32.3 or later, the next sync will conflict on Source/Cmlx/mlx.
• The SwiftLM pin moves through the usual update_dependencies.yml flow, not in this PR.

🤖 Generated with Claude Code

[solderzzc]

Thanks for taking on the 0.32.3 sync, @CodeAndCanvas728 — the vendored tree matches upstream v0.32.3 apart from the expected fork patches, and the full local test run is clean.

While testing I found two things our fork CI can't see (it copies mlx.metallib from the pip wheel, and the xcodeproj build only runs upstream), so I pushed small fixes and tests on top of your branch:

1. gather_mm_offsets (new in 0.32.3, used by sorted gatherMM / gatherQuantizedMM) wasn't in KERNEL_LIST, so the SwiftPM bundle metallib aborted with "Unable to load kernel gather_mm_offsets". Fixed, with an exact-arithmetic test (SortedGatherTests) that crashes without the kernel and passes with it.
2. The Xcode project's exclusion list needed the new 0.32.3 files (plus the fork's moe_stream.metal, whose generated copy was also being built), so xcodebuild -scheme MLX failed. Fixed; it now builds and the new tests pass there too.

I also added a small CI workflow that runs MLXTests with no mlx.metallib on disk and builds the xcodeproj, so this kind of gap is caught next time. It hasn't run on a hosted runner yet, so if it needs a tweak I'll follow up.

These are three regular commits on top of yours (d22833d..033d428), no force push, so a plain git pull on your side picks them up. Happy to adjust anything you'd prefer done differently.