Repository navigation
mps index.Tensor / gather on Metal with device-side bounds check (#32, stacked on #84) - #89
Merged
Merged
Conversation
…ses by name (#32) - write_on_device splits a non-contiguous receiver into contiguous runs and slice_sets each on the device; past 65536 runs it refuses by name. - fill_.Tensor accepts a 0-dim cpu value beside an mps receiver (x[..., :k] = 0). - cumsum, any and copy_ into bool stay in their own/opmath dtype off the cpu instead of widening to f64 (candle has no Metal F32->F64 cast). - pow.Tensor_Scalar with an exponent other than 2 on Metal refuses by name: candle fast-math Metal powf gives 3.0 ** 3 = 26.99999237. - int32 compute and index/gather/scatter/sort/multinomial/view.dtype stay refused by name (no int32 Metal kernels; Metal gather/scatter unchecked). Red before: cumsum f32 raised Metal to_dtype F32 F64; zero_ on [:, 0:2] refused. Nullified: dropping the run split, or the pow refusal, turns the suites red.
…ds check (#32) Every transformers forward hits index.Tensor, and on mps it was refused by the readback gate: candle's Metal index_select/gather do not bounds-check. - index_tensor/gather route a cpu self to the host kernels and any other device to candle device ops: negative indices wrap on device, index tensors broadcast, adjacent/separated axes placed as upstream. - Bounds: each index tensor's min/max reduce on the device and one small vector is read back to validate; out of range raises upstream cpu's error type and text (a second readback names the first offender). - bool/uint8 values go through i64 (no is_i64_u8/u32 or gather_i64_u8 Metal kernels). Masks and int32 indices on mps refuse by name. - cpu index.Tensor's error now names the index position, as upstream does. - MPS_HOST_READBACK_OPS 89 -> 87; tests and docs updated. Red before: all six test_mpsindex tests. Nullified: skipping the bounds check returned out-of-range values (index 5 answered) and turned two tests red.
Contributor
Gate on CI (issue #24)Full gate: 135 suites = 127 run on at least one CI runner + 8 local-only. The local gate (
Local-only: verified by no CI runner (8)
Suites that verified nothing, or failed, on a runner
Every suite, every runner
A SKIP is unverified on that runner, never passed. UNVERIFIED means the suite ran and every one of its tests skipped by name. |
Conflicts: the mps host-readback count. develop's #82 (#33) added aten.sort.stable (89 -> 90) and this branch moved index.Tensor and gather onto the device (89 -> 87), so the merged count is 88: MPS_HOST_READBACK_OPS [&str; 88] (the compiler checks the length), README, README_ko, STATUS.md and matrix.md section 7.19 say so. Rebuilt: test_shim (readback derivation), test_mpsindex, test_mpscb, test_sortties, test_remeasure2, test_cuda ok; golden 12515/12515.
4 tasks
b-re-w
added a commit
that referenced
this pull request
Oct 16, 2026
Conflict: torchnative/rust/vendor/int8-candle-0.11.0-cpu.patch, a generated file. Both sides edited metal_backend/mod.rs (#98's try_clone copy, this branch's I32 arms), which merged cleanly; the patch was then regenerated from the pristine candle-core 0.11.0 crate against the merged tree (git diff --no-index --no-prefix), and vendor_candle.sh --check reports every tree byte for byte. test_mpsint32: the clone encoder-count skip (#97) is removed; it passes. Rebuilt: test_mpsint32 6/6, test_mpsclone 22/22 (no skips now: #89's strided writes and #98's copy are both in), test_mpscb, test_intmps, test_dtypedev, test_shim ok.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #32. Stacked on #84 (which is stacked on #83): merge those first.
Feature added:
aten.index.Tensorandaten.gather.defaultcompute on Metal (candle device ops). Negative wrap on device; broadcast index tensors; upstream axis placement. Bounds: per-index-tensor min/max reduced on device, one small vector read back (1 download per call; 0 for an empty broadcast; 2 when it raises). Out of range raises upstream cpu's type and text.Still refused by name: bool/uint8 mask indices (data-dependent shape), int32 indices on mps, int32 values on mps;
scatter.src,sort,multinomial,view.dtype,index_put_stay on the readback list (89 -> 87).Defect fixed: cpu
index.Tensorout-of-range message named the axis; upstream names the index position.Test added:
tests/devices/mps/test_mpsindex.py(6): agreement incl. f32/f16/bf16/i64/bool, out-of-range parity (9 rows, exact download counts), refusals, readback table, router source check. test_shim / test_mpscb / test_mpsgen updated from the old refusal.Evidence (local, this Mac, worktree): red before on all six; green after. Nullified: skipping the bounds check answered
index 5on a 3-row tensor with values from beyond it and turned 2 tests red. Gate:suites=126/126 ok=2027 FAIL=3; the test_cuda FAIL (a secondis_cuda(device)in a label helper) is fixed in this commit and re-run green; the other two are worktree environment.Not measured: whether eager generate on mps now completes; CUDA takes the same device path, untested.