deepinfer

Learn DeepInfer through a small, runnable inference engine. Follow the code on CPU, then run real Qwen3.5 models on Ascend 910C.

A readable NumPy inference loop: prefill, prefix recovery, verification budgets and speculative KV rollback

DeepInfer SM86 native Qwen2.5 kernels plus a local OpenAI-compatible productivity gateway

Runnable notes on DeepSeek's DeepInfer release: cache economics, the KV-cache hit policy, and deepseek-recipe's prompt contract

面向 DeepInfer 的可审计 KV Store 适配器 MVP：事务持久化、LRU、张量封装、崩溃恢复与 CPU/CUDA 验证

在 Apple M 系列芯片上跑通 DeepInfer（DeepSeek 开源推理框架）— 用 TileLang 的 Metal 后端生成真 kernel，而非停在 PyTorch/MPS 回退。Getting DeepSeek's DeepInfer actually running on Apple Silicon via TileLang-generated Metal kernels.

Qwen2.5-7B-Instruct DeepInfer SM86 mixed W4/W3, fused qkv/gate-up loader, 256 GQA; weights in v0.1.0 release assets

Qwen2.5-3B-Instruct + maibot-catgirl-v3, DeepInfer SM86 W4A16, fused qkv/gate-up loader, 256 GQA; weights in v0.1.0 release assets

A readable DeepInfer-inspired teaching engine: latent KV, sparse attention, MoE, paged cache, continuous batching and greedy speculation.

An unofficial visual source guide to DeepSeek DeepInfer: architecture, request flow, DSpark, async scheduling, and release boundaries.

Anthropic Messages API server for DeepInfer, with Claude Code as an example client

中文记录 DeepInfer 架构与 HIP/ROCm 迁移学习过程

deepinfer: high-performance, large-scale keyword engineering for deepseek models. nano, mini, awesome, everything, anything. implementation: todo.

Learn AI Infra & DeepSeek inference from scratch. A CPU-only NumPy LLM engine: DeepInfer design, MLA, MoE, paged KV cache, speculative decoding, continuous batching & HTTP serving. English / Chinese.

DeepInfer 可视化导读（非官方）：从推理基础与 vLLM 对照出发，图解 DeepSeek 推理框架 DeepInfer 的设计与源码