AMD Roadmap 2026 Q4-2027 Q1

[zihaomu]

AMD Roadmap - 2026 Q4 / 2027 Q1

Upstream Integration

upstream/main
└── #535  ROCm 10.0.0 Dockerfile and W7900D CI
    └── #132  ROCm / RDNA runtime foundation
        ├── #133  TVM-FFI index/store HIP JIT
        ├── #134  CUDA-only backend isolation and ROCm routing
        ├── #135  PyTorch distributed to RCCL
        └── #378  CPU/Hybrid MoE graph replay safety
            └── #491  Hybrid decode orchestration refactor

What each PR provides

• #535 — ROCm 10.0.0 Dockerfile and W7900D CI (Open)

  • Adds the shared ROCm development/CI environment and the W7900D validation workflow.
  • Complete ROCm test coverage depends on the follow-up support PRs.

• #132 — ROCm / RDNA runtime foundation (Open)

  • Enables source builds for RDNA 3 and RDNA 4 with both traditional and modular ROCm layouts.
  • Builds the pinned-memory and CPU-MoE native extensions on HIP, enables the shared Triton/runtime paths, and provides ROCm architecture detection.
  • This is the platform foundation required by the follow-up PRs.

• #133 — TVM-FFI index/store HIP JIT (Draft, depends on feat(rocm): add RDNA3 and RDNA4 runtime foundation #132)

  • Makes the generic index and store JIT kernels accept ROCm tensors and use the HIP-compatible launch path.
  • Keeps the existing CUDA PTX path unchanged.

• #134 — CUDA-only backend isolation and ROCm routing (Draft, depends on feat(rocm): add RDNA3 and RDNA4 runtime foundation #132)

  • Prevents ROCm from selecting NVIDIA-only donor backends such as FlashInfer, SGL kernels, Marlin, and OpenAI Triton kernels.
  • Routes ROCm to the supported Triton or PyTorch fallback implementations while preserving CUDA backend selection.

• #135 — RCCL communication path (Draft, deferred)

  • Bypasses the NVIDIA-only custom PyNCCL communicator on ROCm.
  • Uses PyTorch distributed with the nccl backend provided by RCCL; the two-GPU collective path is validated, while full model-level TP remains future work.

• #136CLOSED — Native GGUF Q4_0 ROCm kernels (depends on feat(rocm): add RDNA3 and RDNA4 runtime foundation #132 and fix(rocm): make TVM-FFI index and store JIT kernels portable to HIP #133)

  • HIPifies and builds the native GGUF kernels for wave32 RDNA 3 / RDNA 4 targets.
  • Validates Q4_0 dequantization, int8-dot matvec, and routed MoE numerics on gfx1201; broader architecture and format coverage is not yet claimed.

• #378 — CPU/Hybrid MoE graph replay safety (Draft, depends on feat(rocm): add RDNA3 and RDNA4 runtime foundation #132)

  • Adds a ROCm graph synchronization handshake for CPU/Hybrid MoE and falls back to correct eager execution when the capability probe fails.
  • Preserves the existing CUDA synchronization path and fixes the silent incorrect-output risk reported in [ROCm] CpuMoeExecutor decode is silently wrong under CUDA-graph replay (rel err 1.36, no exception) #350.

• #491 — Hybrid decode orchestration refactor (Draft, needs discussion; depends on fix(rocm): make CPU/Hybrid MoE graph replay safe #378)

  • Extracts the shared CPU/GPU decode schedule from OffloadMoELayer into HybridDecodeExecutor.
  • Keeps LRU policy, expert kernels, numerical behavior, and CUDA/ROCm synchronization unchanged while separating platform-independent orchestration from backend-specific mechanisms.

Recommended merge order

#535 → #132 → { #133, #134, #378 } → { #491 }

Official ROCm Image

assigned to @LZ-QWQ

• 2026 Q4: publish the official AMD ROCm beta image.
• 2027 Q1: publish the release image with a reproducible and maintainable software stack.

ROCm Compatibility & Official AMD install Doc

• Establish ROCm 7.14.x + PyTorch 2.11.x + AMD Triton 3.8.x as the first supported baseline.
• Qualify ROCm 10.0.x + PyTorch 2.11.x + AMD Triton 3.8.x as the next supported stack. ROCm 10 compatibility matrix
• Publish the supported ROCm / PyTorch / AMD Triton / GPU compatibility matrix for each FreeToken AMD release.

CI and Qualification

assigned to @LZ-QWQ

• Add W7900 blocking CI and nightly validation.
• Establish multi-architecture qualification and publish the AMD support matrix.

Model Correctness

• Establish model accuracy and stability gates for AMD platforms, prioritizing:
  • Qwen3.8-Flash-Next accuracy and stability validation.
  • Qwen3.6-35B-A3B accuracy and stability validation.
• Publish model correctness results for each FreeToken AMD release.

Performance Benchmarking

• Establish repeatable performance baselines and regression tracking for priority models.
• Compare FreeToken with llama.cpp using the same models, quantization formats, and workloads on AMD GPUs.
• Compare AMD Radeon and NVIDIA GeForce consumer GPU performance under aligned hardware classes and workloads.
• Publish reproducible benchmark results for each FreeToken AMD release.

Native HIP Kernels

• Move selected performance-critical operators from Triton to native HIP kernels.
• Tune native HIP kernels for Radeon and Instinct architectures.
• Retain Triton as the generic implementation and fallback path.

Hardware Support

• Radeon RDNA 3 and RDNA 4 support.
• Strix Halo UMA support.
• Validate more architectures beyond the W7900 CI machine.

2026 Q4

• Complete the upstream AMD foundation and reviewer-ready PR series.
• Establish the official ROCm image, W7900 CI, and model baselines.
• Complete the first native HIP kernel prototype and multi-architecture bring-up.

2027 Q1

• Deliver release-quality Radeon preview.
• Enable architecture-tuned native HIP kernels for selected hotspots.
• Publish the support matrix, model results, and release artifacts.

Later Roadmap - 2027 Q2+

• Native Windows support on Radeon.
• Model-level Tensor Parallel on RCCL — #135