diff --git a/README.md b/README.md index ed3f643e..65df69e2 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,10 @@ whose Asahi boot and encryption tools it reuses. > **M4 Macs (e.g. Mac mini J773g):** step 1 below does not apply; see [M4-MINI-INSTALL.md](mseries/M4-MINI-INSTALL.md); the commands in order are in [INSTALL-COMMANDS.md](mseries/gravity/INSTALL-COMMANDS.md). +**M4 performance tracker:** Follow Gravity, Omarchy and native macOS baselines, +model benchmarks and progress toward outperforming native on matched workloads in +[M4-PERFORMANCE.md](mseries/local-ai/M4-PERFORMANCE.md). + > 🚧 **Work in progress.** Not yet run end to end on a freshly wiped Mac. Back up everything first. [Status](#status) **The whole install, in three commands** (details for each step below): diff --git a/mseries/OS9-HANDOFF.md b/mseries/OS9-HANDOFF.md index 743fbbc1..da865fbf 100644 --- a/mseries/OS9-HANDOFF.md +++ b/mseries/OS9-HANDOFF.md @@ -1,3 +1,11 @@ +## M4 performance tracker — 2026-10-30 + +Added [M4-PERFORMANCE.md](local-ai/M4-PERFORMANCE.md), linked from the project +README and generated engine scoreboard. It tracks matched Gravity, Omarchy +and macOS workloads, including the existing multi-model AI suite, with SHA-1 +and SHA-256 provenance. The first M4 MLX attempt is retained as a failure +receipt; no M4 model speed or native comparison is claimed yet. + ## Warren local AI and eGPU updates — 2026-10-30 Installed `mseries-nvidia-open-source` `595.112.07+apple.1-1` (SHA-256 `a397b68a...b209ab23`) containing the three prepared NVIDIA fixes and license at `/usr/src/mseries-nvidia-open-595.112.07`. This is source only; no NVIDIA module or matching CUDA userspace is installed. The [source package script](nvidia-egpu/package-source.sh) verifies the exact prepared diff before packaging. diff --git a/mseries/ROADMAP.md b/mseries/ROADMAP.md index e3dbed33..43e0f4db 100644 --- a/mseries/ROADMAP.md +++ b/mseries/ROADMAP.md @@ -1,3 +1,13 @@ +## M4 performance — 2026-10-30 + +Track Gravity, Omarchy and native macOS on matched workloads in the +[M4 performance tracker](local-ai/M4-PERFORMANCE.md). Record exact model and +artifact hashes, correctness and per-run metrics; exceeding native is a goal, +not a measured result. No M4 model-speed baseline has been established yet. +The tracker now includes matched Parakeet speech CPU/GPU/ANE and macOS +comparisons. M4 ANE remains unqualified; existing M1 results are not M4 +baselines, and speech accuracy, latency and energy need their own receipts. + ## WD22TB4 and wip4 candidate status — 2026-10-28 Current `dev` is `57b143c4afd78b0a4a0f9905e48e39632d1ecc3c`. Pages deployment diff --git a/mseries/local-ai/ENGINE-SCOREBOARD.md b/mseries/local-ai/ENGINE-SCOREBOARD.md index d9944345..a66b7cef 100644 --- a/mseries/local-ai/ENGINE-SCOREBOARD.md +++ b/mseries/local-ai/ENGINE-SCOREBOARD.md @@ -1,5 +1,9 @@ # Local AI engine scoreboard +For the Mac mini M4 and comparisons with native macOS, see the +[M4 performance tracker](M4-PERFORMANCE.md). The measurements below are M1 Pro +results and are not M4 baselines. + One table for every engine we run or watch on the M1 Pro. ★ marks the most functional setup per model, length and load: the shortest typical reply (first token plus 200 tokens) whose output matched every other engine and whose correctness gate did not fail. Regenerate with `python3 mseries/local-ai/scoreboard.py` after adding a receipt. Tracked in the "Local AI engine scoreboard" issue. ![First token, streaming speed and typical reply time by engine](engine-scoreboard.svg) diff --git a/mseries/local-ai/M4-PERFORMANCE.md b/mseries/local-ai/M4-PERFORMANCE.md new file mode 100644 index 00000000..7789054a --- /dev/null +++ b/mseries/local-ai/M4-PERFORMANCE.md @@ -0,0 +1,76 @@ +# M4 Mac mini performance tracker + +Track the J773g Mac mini across Gravity, the WIP Omarchy port and native macOS. +The goal is to match and, where possible, exceed native performance on the same +workload while retaining correct results and a responsive desktop. A faster +result on one benchmark is a result for that workload, not a claim that the +whole system is faster. + +## Current status + +| Environment | Kernel or OS | Baseline | Status | +| --- | --- | --- | --- | +| Gravity Linux | `7.1.13-509.gravity.fc44.aarch64+16k` | M4 GPU enumerated; no model speed result | October 28 matrix multiply timed out; October 30 tiny add produced an incorrect result and a GPU timeout. [First receipt](receipts/2026-10-28-m4-mini-first-mlx-attempt.md); [October 30 receipt](receipts/2026-10-30-m4-mlx-preflight.md). | +| Omarchy WIP | WIP5 J773g forward port | Not measured | Integration and build in progress; not boot-qualified. | +| macOS | Same Mac mini | Not measured | Matched native runs and their version are needed. | + +The existing [engine scoreboard](ENGINE-SCOREBOARD.md) is from an M1 Pro and +provides workload definitions and reference methods, not M4 performance data. + +## Model baseline matrix + +Use the existing [`engine-compare.py`](engine-compare.py) model pins, prompts, +generation lengths and resource guards where the runtime supports them. +Record each model's exact revision, quantization and weight hashes so an M4 +result can be compared with the same model on native macOS. The table below is +the target set, not a claim that a run completed. + +| Workload | Model and format | Gravity | Omarchy | macOS | +| --- | --- | --- | --- | --- | +| Small dense language model | Granite 4.0 1B, MLX 4-bit | Pending | Pending | Pending | +| Small language model | LFM3 1.7B, MLX 8-bit | Pending | Pending | Pending | +| Language and code | Qwen3.5 2B, MLX 4-bit | Pending | Pending | Pending | +| Larger language model | Qwen3.5 4B, MLX 4-bit | Pending | Pending | Pending | +| Native Vulkan engine comparison | TensorFold, Qwen3.5 2B | Pending | Pending | Pending | +| Speech-to-text | Parakeet TDT 0.6B v3, fixed audio; CPU, GPU and ANE where supported | Pending | Pending | Pending | + +## Speech and Neural Engine + +The requested [speech-encoding article](https://joshuawarren.com/blog/speech-encoding-31-percent-faster-m1-max-neural-engine/) is a lead for optimizing the encoder. Its text was inaccessible during this review, so its title's percentage and exact method are unverified here and are not M4 measurements. The author's [omarchy-mlx](https://github.com/joshuaswarren/omarchy-mlx) supplies a Parakeet path, while [omarchy-ane](https://github.com/joshuaswarren/omarchy-ane) describes an experimental, manually opted-in M4 H16 bring-up module: its README reports no packaged M4 driver or real inference. Do not treat the M1 Max result as M4 support. + +On this M4 mini, the running Gravity system has no `/dev/accel` node; the unbooted Aurora candidate enables the ANE driver as a module but its current match tables have no T8132 entry. Neither observation proves ANE inference works. Separately, our M1 Pro ran a 20-call bit-exact ANE smoke and a Parakeet fixture after a private Mesa correction; those are M1 results, not M4 performance. See [M1 speech research](../dictation/STT-RESEARCH.md). + +For an M4 comparison, use a pinned Parakeet model and public fixed WAV clips with known reference transcripts. Record both SHA-1 and SHA-256 for every model file, clip, transcript and result, plus model revision, preprocessing and exact commands. Compare the same clips and model across Gravity and Omarchy CPU, GPU and ANE when each backend is qualified, and macOS on the same mini; document model conversion, precision and runtime differences. Record word-error rate, encoder-only time and end-to-end transcription latency separately, cold-start and individual warm runs, audio duration and real-time factor, peak memory, CPU load, power and thermals when available. Measure at least 10 warm runs after an explicitly recorded warmup, keep failures, and separate standalone runs from GPU/ANE contention. Do not claim a speedup where correctness or workload equivalence has not been established. + +ANE work is a later qualification track: first establish a safe M4 device/firmware and recovery path and driver binding, then follow the backend's read-only checks and qualified smoke/fixture instructions before running speech inference. Do not load experimental modules, reboot, or run a hardware workload as part of merely adding this plan. + +Track CPU, GPU, memory bandwidth, storage, graphics responsiveness, sustained +power and thermals alongside AI. Add other workloads when a reproducible +fixture and a comparable native run exist. For sound and desktop use, keep +functional correctness separate from raw speed. + +## Recording a run + +Save a dated receipt for each run or attempt, including failures, and link it +from this page. Include: + +- Local date, time and timezone; machine/model, memory and OS version. +- Kernel release and build identity; firmware, graphics driver, runtime and + backend versions; whether the workload ran on CPU, GPU or ANE. +- Model identifier, revision, quantization, exact input/prompt, token count, + settings and run count. Record **SHA-1 and SHA-256** for each model weight + file and for saved benchmark outputs; record hashes of kernel and driver + artifacts where available. Hash files in place instead of making copies. +- Correctness checks, time to first token, decode tokens/s, total time, peak + memory, temperature/power where available, and desktop responsiveness. + Preserve individual runs and report the statistic used. Keep single-request + speed distinct from aggregate throughput under concurrency. +- For a native comparison, use the same model files, prompt, token count and + quantization. State any engine or measurement differences, and calculate + the ratio only for comparable completed runs. + +Use the existing shared worker lock, memory and thermal guards; capture a +failure rather than presenting it as a zero-speed result. Keep downloads and +builds isolated, use one model at a time when space is tight, and remove +temporary copies after recording their hashes and receipts. A benchmark result +does not qualify boot, dual boot, storage durability or the entire GPU stack. diff --git a/mseries/local-ai/receipts/2026-10-28-m4-mini-first-mlx-attempt.md b/mseries/local-ai/receipts/2026-10-28-m4-mini-first-mlx-attempt.md new file mode 100644 index 00000000..9d88fed1 --- /dev/null +++ b/mseries/local-ai/receipts/2026-10-28-m4-mini-first-mlx-attempt.md @@ -0,0 +1,62 @@ +# M4 Mac mini first MLX attempt: GPU detected, first compute job hangs, 2026-10-28 + +Result: **FAIL, not qualified.** MLX enumerates the M4 GPU, but the first compute job never +completes and the kernel GPU driver then wedges until reloaded. No model was run and no +performance number exists. This receipt records the negative result and its evidence. + +## Host + +- Apple Mac mini (M4, 2024), `apple,j773g apple,t8132`, 10 GPU cores, 16 GB RAM. +- Gravity Linux (Fedora 44, KDE), kernel `7.1.13-509.gravity.fc44.aarch64+16k`, + GPU module `asahi_m4` (in-tree). +- Mesa `26.3.0~devel-103.gravity.fc44` (system, Honeykrisp). `vulkaninfo`: `Apple M4 (G16G B1)`, + API 1.4.359, `DRIVER_ID_MESA_HONEYKRISP`, ICD `/usr/share/vulkan/icd.d/asahi_icd.aarch64.json`. +- Python 3.14.8. + +## Install (worked) + +omarchy-mlx `33630f9c1eeefe5fc985a28bed0585432c2c014e` (v0.7.33) with vendor wheels +`omarchy-mlx-vendor-wheels-v0.7.33-cp314-aarch64.tar`, sha256 +`bc9f8c7527e1324d25212922ca0afd2acc2411ec7cdd4b26e2773bffdef32f0e` (verified). Staged with +`install.sh --system --dest-root --vendor ... --lock ... --python /usr/bin/python3.14` +(offline, hash-locked, no root, no pacman). All mlx-lm serve patches applied. The system Mesa was +used; the private `joshuaswarren/mesa-1` compute driver was **not** built. + +## Run + +`mx.default_device()` returned `Device(gpu, 0)`; `mx.device_info()` reported `Apple M4 (G16G B1)`, +driver Honeykrisp, `unified_memory: 1`, `max_recommended_working_set_size` 8220835840. + +First compute job, `a = mx.random.normal((1024, 1024)); b = a @ a; mx.eval(b)`: + +``` +RuntimeError: [omarchy] Vulkan timeline counter failed to advance for 10000 ms +(last observed=0, target=1). The device may be hung; no CPU fallback is available. +``` + +Kernel log at the same time (`journalctl -k`, 2026-10-28 21:07:59): + +``` +asahi_m4: G16: submission pipeline failed: ETIMEDOUT, 1 jobs, 1 issued +asahi_m4: G16: submission pipeline failed: EIO, 1 jobs, 0 issued (22 more, EIO) +``` + +Earlier, at 21:03:12 and before any MLX run: `asahi_m4: UATPageTable::unmap_pages: Page at IOVA +0x2ffff90000 already unmapped` (twice). + +After the hang, a fresh `vulkaninfo` fails with `DRM_IOCTL_ASAHI_VM_BIND failed: Input/output +error` and `BO creation failed`, and only llvmpipe enumerates. A fresh MLX process reports +`no Vulkan physical devices found`. The display keeps working (it uses the DCP driver). + +## Not yet done + +- The smaller-op bisect (tiny add, 1M elementwise, matmul 64/256/1024 f16) did not run: the GPU was + already wedged and a reload needs root. +- The pinned private Honeykrisp (`mesa-1` `7faf04c065`, as in `build-local-ai`) was not tried. +- Not yet known whether the fault is in `asahi_m4`, Mesa, or MLX's Vulkan submission pattern. + +## Next + +1. Reload `asahi_m4` (or reboot), rerun the bisect one process per case. +2. Build the pinned compute driver and compare against system Mesa. +3. Standalone Vulkan compute reproducer, to separate driver bugs from MLX. diff --git a/mseries/local-ai/receipts/2026-10-30-m4-mlx-investigation.md b/mseries/local-ai/receipts/2026-10-30-m4-mlx-investigation.md new file mode 100644 index 00000000..70e2e3ca --- /dev/null +++ b/mseries/local-ai/receipts/2026-10-30-m4-mlx-investigation.md @@ -0,0 +1,46 @@ +# M4 MLX preflight investigation, 2026-10-30 + +The [preflight](2026-10-30-m4-mlx-preflight.md) returned zero for `1 + 1` +and the kernel reported unretired GPU work ten seconds later. Its captured +output has no dispatch trace, so it does not identify which runtime path ran +or prove why the GPU failed to retire work. Git blob comparison confirms the +reviewed `device.cpp`, `fence.cpp` and `encoder.cpp` are identical between +wheel source `33630f9c1eeefe5fc985a28bed0585432c2c014e` and the candidate +branch's base `5dbc64dd`; this does not prove any reviewed path was invoked. + +A source review of the Omarchy MLX backend found two independent completion +hazards. A started GPU event alone does not prove shader completion; candidate +commit `a94172f32385041223df71006bfe9a15b0e15f80` stops the recovery branch +from publishing a completion on that basis. Candidate commit +`6a107cfbf632c5d612cc31fb94d096a59c5129bf` stops a CPU event from advancing +the separate shared GPU completion timeline beyond unfinished work. A host +regression against the actual function fails before and passes after the latter +change. The candidate CPU-disabled MLX static library compiled and linked from +pinned MLX source `9c3d35571ac450a8ecf5c17b4d0e3fac52c08bc8`; this is not a +GPU runtime test. The ordinary upstream MLX graph appears to signal each event +from its producer stream, so it is not established that the second path ran +in the failed smoke. + +Candidate `5276a4a7c9632457fd84223f52e70e397589425c` changes a fence wait +to honor the requested generation instead of the latest one, with a host +regression and a rebuilt CPU-disabled library. Candidate +`1f2ba9efb443c8a21e6d5230d2350253b4dc2bb4` repairs the signal-loss test +hook: it now submits work with signals stripped, rather than dropping the +entire submission. A host regression verifies all three test-hook branches. +Neither candidate has been exercised on a GPU or linked to this smoke's cause. + +None of these changes establishes recovery of the observed M4 GPU timeout. Review also +identified unresolved ordering on a single timeline semaphore shared by CPU and +queued GPU signals, and a retry that can replay a command buffer without proof +its original submission retired. These need a complete synchronization and +lifetime design. No new GPU workload was run for this investigation. + +Candidate `7f657e94` removes the unsafe replay path: absence of a start signal +does not prove the original one-time submission was discarded. The watchdog +now reports the stall instead of attempting recovery by resubmission, trading +automatic recovery for avoiding possible concurrent execution. The pinned +CPU-disabled MLX static library rebuilt successfully with the change. Host +test `881295a1` compiles and exercises the actual callback for several target +values and rounds; it and the three earlier host regressions passed. This +does not demonstrate GPU recovery, identify the smoke failure, or establish +safe teardown of a hung driver. diff --git a/mseries/local-ai/receipts/2026-10-30-m4-mlx-preflight.log b/mseries/local-ai/receipts/2026-10-30-m4-mlx-preflight.log new file mode 100644 index 00000000..8fa53e6f --- /dev/null +++ b/mseries/local-ai/receipts/2026-10-30-m4-mlx-preflight.log @@ -0,0 +1,4 @@ +2026-10-30T20:09:35-04:00 +device Device(gpu, 0) +result array([0], dtype=float32) +exit=0 diff --git a/mseries/local-ai/receipts/2026-10-30-m4-mlx-preflight.md b/mseries/local-ai/receipts/2026-10-30-m4-mlx-preflight.md new file mode 100644 index 00000000..b9d4fbb6 --- /dev/null +++ b/mseries/local-ai/receipts/2026-10-30-m4-mlx-preflight.md @@ -0,0 +1,11 @@ +# M4 MLX baseline preflight, 2026-10-30 + +At 2026-10-30 20:09:35 EDT on Apple Mac mini J773g, Gravity Linux kernel `7.1.13-509.gravity.fc44.aarch64+16k`, Mesa package `mesa-vulkan-drivers-26.3.0~devel-103.gravity.fc44.aarch64`, an isolated MLX v0.7.33 GPU smoke failed. The runtime enumerated `Device(gpu, 0)`. The program evaluated `mx.array([1.0]) + 1`, called `mx.eval`, and printed `array([0], dtype=float32)` rather than the expected `[2]`; process exit status was 0. At 20:09:45 EDT the kernel logged `G16: async Work ... timeout without retirement progress`, `firmware retirement failed: ETIMEDOUT`, and `submission pipeline failed: ETIMEDOUT, 1 jobs, 1 issued`. This is a GPU correctness failure and likely device wedge; no further GPU operations were run. + +No model weights were downloaded or evaluated after the smoke failed, so there are no M4 model SHA-1s or tokens-per-second measurements. The M1 Pro results are not M4 baselines. No system software, kernel, display configuration or boot state was changed. + +- Vendor bundle SHA-256: `bc9f8c7527e1324d25212922ca0afd2acc2411ec7cdd4b26e2773bffdef32f0e`; SHA-1: `7c8b03939ed8448261ea8c04529142a28f9ffefa`. +- MLX wheel `mlx_omarchy-0.32.4.dev202610160529+33630f9c-cp314-cp314-linux_aarch64.whl` SHA-256: `6068ab8370a5e0ffd80778d103114cfb3f42b3153bf4c754465adab036c33d2c`; SHA-1: `7c07b3432c6343321b3537f694fb6a3aa14397b4`. +- [Captured output](2026-10-30-m4-mlx-preflight.log) SHA-256: `559ffaeb36312e86f08a6eb223c5947eabe630633a556a9487872f2afd872e3f`; SHA-1: `35ae6002d35a36ef556eb481e5d20cd9e094a1e8`. + +[Source investigation and candidate changes](2026-10-30-m4-mlx-investigation.md). diff --git a/mseries/local-ai/scoreboard.py b/mseries/local-ai/scoreboard.py index cef0816c..e782744a 100644 --- a/mseries/local-ai/scoreboard.py +++ b/mseries/local-ai/scoreboard.py @@ -193,6 +193,9 @@ def main(): for n in notes: w.writerow({**n, 'best': ''}) md = ['# Local AI engine scoreboard', '', + 'For the Mac mini M4 and comparisons with native macOS, see the', + '[M4 performance tracker](M4-PERFORMANCE.md). The measurements below are M1 Pro', + 'results and are not M4 baselines.', '', 'One table for every engine we run or watch on the M1 Pro. ★ marks the most functional setup per model, length and load: the shortest typical reply ' '(first token plus 200 tokens) whose output matched every other engine and whose correctness gate did not fail. ' 'Regenerate with `python3 mseries/local-ai/scoreboard.py` after adding a receipt. Tracked in the '