Milestones
List view
Deferred from November to keep quality bars while meeting the 11-30 usable date: fast paged-attention kernel (torchnative TN-M2), 4-bit weights and GGUF import (TN-M3), several resident models with eviction, remaining Ollama commands (create/copy/push/Modelfile, embeddings), session prefix cache, Qwen3 and larger models, CUDA server path.
Due by February 26, 2027•2/6 issues closedOpenAI-compatible /v1/chat/completions (streaming, tool-call pass-through) and /v1/models; Ollama core endpoints (/api/chat, /api/generate, /api/tags, /api/show, /api/pull, /api/delete, /api/ps) with keep_alive and one resident model at a time; q8_0 weights from Hugging Face. Done: OpenAI and Ollama clients complete a tool-call round trip; an Ollama client pulls, lists, chats with and removes a small model. Moved out: 4-bit, GGUF import, several resident models, the rest of the Ollama commands (M4).
Due by November 30, 2026•20/21 issues closedContinuous batching through transformers generate_batch with the paged KV cache and sdpa_paged / eager_paged (plain torch ops; no new kernel). Done: two concurrent requests equal sequential generation; with the same seed, sampled output is identical whether or not the request was batched; throughput measured and recorded. Depends on torchnative running this path on cpu/mps by 11-16 (torchnative#15). Moved out: the fast paged-attention kernel (M4).
Due by November 23, 2026•3/3 issues closedOne engine on transformers generate (public torch API only; developed on upstream torch, then run on torchnative once TN-M1 lands 10-24). GGUF/BIN/GPTQ removed, Python 3.13, q8_0, WebSocket streaming chat with server-side tools kept, requests serialised, stop on disconnect. Models: Qwen3-0.6B (default) and SmolLM2; automated tests on SmolLM2 until torchnative #23 (Qwen3 matches upstream) lands. Done: streaming-chat test passes on torchnative (cpu and mps); nothing imports llama-cpp or bitsandbytes.
Due by November 16, 2026•5/6 issues closed