Papers

AI

DeepSeek

• DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
• Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures
• DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models

llada

• Large Language Diffusion Models
• Structured Denoising Diffusion Models in Discrete State-Spaces

System

• Efficient Memory Management for Large Language Model Serving with PagedAttention
  • Github
• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• Generating Long Sequences with Sparse Transformers

Articles

WSC

• The Tail at Scale

Books

DRAM

• The Datacenter as a Computer
• Memory Systems