SSD-streaming MoE inference engine for consumer hardware. Run 80B parameter models on a 24GB Mac.
-
Updated
Feb 22, 2026 - Rust
#
ssd-streaming
Here are 2 public repositories matching this topic...
Run AI models too large for your Mac's memory — at near-full speed. Intelligent expert caching, speculative execution, and 15+ research techniques for MoE inference on Apple Silicon.
python macos rust machine-learning inference moe quantization mlx speculative-execution mixture-of-experts memory-optimization apple-silicon llm metal-gpu ssd-streaming expert-caching
-
Updated
Apr 1, 2026 - Python
Improve this page
Add a description, image, and links to the ssd-streaming topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the ssd-streaming topic, visit your repo's landing page and select "manage topics."