Repository navigation
mps: int32 on Metal; transformers continuous batching greedy on mps matches upstream mps (#32) - #100
Merged
Merged
Conversation
…ening (#32, in progress) candle-metal-kernels fork: cast_i32_* / cast_*_i32, copy_i32(_strided), const_set i32; candle-core fork: Metal I32 arms in const_set, to_dtype, copy_strided_src. torch_c: int32 on mps stays I32 storage; claimed ops widen I32 operands to I64 on device, compute, and narrow back (Promote / SelfDtype / Keep modes, _C._shim_mps_int32_widened_ops()); unclaimed ones refuse by name. tests/devices/mps/test_mpsint32.py plus the suites that pinned the old refusal. Not finished: flip, pow(2), exp.
…), maximum and readback ops refused, exp/sqrt records; CB greedy on mps test (#32) - flip widens (SelfDtype); pow.Tensor_Scalar is Promote: its integral path is int64 powi-by-squaring on the device, so narrowing wraps exactly like int32 multiplication (pow 2/3/5 graded with wraparound; a negative exponent raises upstream's own error). - maximum stays refused: it is on MPS_HOST_READBACK_OPS for every dtype. - exp/sqrt answer through the native int32 -> float32 cast and the float kernel; their values are not graded here (Metal float32 exp is 1 ulp off upstream cpu, a float matter). dtypedev int32 mps column: 18 of 23. - clone's encoder check skips by name until PR #98 (#97) lands. - tests/models/test_cbpath.py: test_generate_batch_greedy_tokens_match_ upstream_mps_on_mps, #13's eight configs on mps against upstream mps. Red before (base build): all five test_mpsint32 tests; CB on mps stopped at aten.embedding.default (int32) in all eight cells. Green after: the nine affected suites (542 tests) pass and CB on mps is token-equal to upstream mps in all eight. Nullified: skipping the narrowing makes results int64 where upstream is int32 (two test groups red). Gate (worktree): suites=136/136 ok=2114 FAIL=2, worktree environment only.
Conflict: torchnative/rust/vendor/int8-candle-0.11.0-cpu.patch, a generated file. Both sides edited metal_backend/mod.rs (#98's try_clone copy, this branch's I32 arms), which merged cleanly; the patch was then regenerated from the pristine candle-core 0.11.0 crate against the merged tree (git diff --no-index --no-prefix), and vendor_candle.sh --check reports every tree byte for byte. test_mpsint32: the clone encoder-count skip (#97) is removed; it passes. Rebuilt: test_mpsint32 6/6, test_mpsclone 22/22 (no skips now: #89's strided writes and #98's copy are both in), test_mpscb, test_intmps, test_dtypedev, test_shim ok.
Contributor
Gate on CI (issue #24)Full gate: 137 suites = 129 run on at least one CI runner + 8 local-only. The local gate (
Local-only: verified by no CI runner (8)
Suites that verified nothing, or failed, on a runner
Every suite, every runner
A SKIP is unverified on that runner, never passed. UNVERIFIED means the suite ran and every one of its tests skipped by name. |
b-re-w
added a commit
that referenced
this pull request
Oct 16, 2026
…7.21 - int32 index tensors now cast on the device (PR #100's i32 kernels): the Metal guard calling the deleted refuse_mps_int32_index is removed, and test_mpsindexput's int32 refusal row became an agreement row. - Conflicts: gate_suites.py keeps both new suites; matrix.md keeps develop's section 7.20 (int32) and renumbers this branch's index_put_ section to 7.21, with STATUS.md pointing there. - Writing into an int32 *self* (x_int32[idx] = v) still refuses by name: index_put_ is not in the int32 widened-op table. Rebuilt: test_mpsindexput 6/6, test_mpssetitem, test_mpsindex, test_mpsint32, test_metalplace, test_mpsclone, test_intmps, test_dtypedev, test_shim, test_remeasure2 ok. Gate (worktree): suites=139/139 ok=2146 FAIL=2, worktree environment only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #32 (criterion 3; see notes for what stays out of scope)
Feature added
cast_i32_*,cast_*_i32),copy_i32(_strided),const_seti32; candle-core fork: Metal I32 arms inconst_set,to_dtype,copy_strided_src. Patches regenerated;vendor_candle.sh --checkpasses._C._shim_mps_int32_widened_ops()lists them.#32 end to end: transformers
generate_batchgreedy on mps, #13's 8 configs (qwen3/llama x paged|sdpa/paged|eager x f32/bf16), token-equal to upstream mps in all 8. Before: the shim stopped ataten.embedding.default(int32) in all 8. New testtests/models/test_cbpath.py::test_generate_batch_greedy_tokens_match_upstream_mps_on_mps(skips by name without upstream mps).Test added:
tests/devices/mps/test_mpsint32.py(overflow wraparound vs upstream cpu, Metal encoder counted, 0 downloads, refusals by name); suites that pinned the int32 refusal updated (test_mpscb, test_mpsindex, test_intmps, test_absmps, test_mpsconst, test_mpsinplace, test_dtypedev: int32 mps column 2 -> 18 of 23).Evidence (local, this Mac): red before on the base build (all 5 test_mpsint32 tests; CB 0/8); green after (9 suites, 542 tests; CB 8/8). Nullified: skipping the narrowing returns int64 where upstream is int32 (2 groups red). Gate: 136/136 suites, ok=2114, FAIL=2 (worktree environment only).
Not in this PR: clone's encoder check skips by name until #98 (#97, clone aliasing) lands; f32
exp/sin/cosprecision on Metal (1 ulp vs upstream cpu) from #32's last bullet is a float matter, not measured against upstream mps here; the CB test reads no GPU counters (the host-readback gate is what rules out silent host compute).