From c897c866bcf1b4be624dda63add5c037e964d357 Mon Sep 17 00:00:00 2001 From: Debasis Mandal Date: Tue, 28 Jul 2026 00:12:58 -0400 Subject: [PATCH 1/5] Fix ROCm devcontainer torch install: use find-links, not --index-url MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The radeon wheel mirror (repo.radeon.com/rocm/manylinux/rocm-rel-X.Y/) is a flat directory listing, not a PEP 503 simple index — requesting the per-package path (.../torch/) returns 404. With --index-url, pip therefore finds zero torch candidates and fails with "No matching distribution found for torch==2.9.1 (from versions: none)". Switch to -f/--find-links, matching docker/Dockerfile.rocm_ci. This still resolves to AMD's internal ROCm build (torch 2.9.1+rocm7.2.0...) rather than PyPI's CUDA wheel, because PEP 440 ranks the local-version wheel above the plain 2.9.1 on PyPI (verified with pip --dry-run and torch.version.hip in the image). Co-Authored-By: Claude Opus 4.8 --- .devcontainer/rocm/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.devcontainer/rocm/Dockerfile b/.devcontainer/rocm/Dockerfile index 0f0fbe84cd..e19b7569ae 100644 --- a/.devcontainer/rocm/Dockerfile +++ b/.devcontainer/rocm/Dockerfile @@ -83,7 +83,7 @@ RUN \ # Create a new micromamba env and install needed packages for flashinfer development RUN /bin/micromamba create -n ${MAMBA_ENV_NAME} python=${PY_VERSION} gtest gmock bash-completion -c conda-forge && \ /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install --no-cache-dir cmake ninja "scikit-build-core>=0.4.3" "setuptools-scm>=9.2" pre-commit numpy pytest pytest-cov pytest-xdist pytest-rerunfailures pybind11 ruff && \ - /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install --no-cache-dir torch==${TORCH_VERSION} --index-url https://repo.radeon.com/rocm/manylinux/rocm-rel-${ROCM_VERSION}/ && \ + /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install --no-cache-dir torch==${TORCH_VERSION} -f https://repo.radeon.com/rocm/manylinux/rocm-rel-${ROCM_VERSION}/ && \ /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install amd_aiter==${AITER_VERSION} --extra-index-url https://pypi.amd.com/rocm-${AITER_ROCM_VERSION}/simple SHELL ["/usr/local/bin/_dockerfile_shell.sh"] From e2cbe6e365500e1b0e495ffaedcdefcdaeebebe5 Mon Sep 17 00:00:00 2001 From: Debasis Mandal Date: Tue, 28 Jul 2026 00:50:34 -0400 Subject: [PATCH 2/5] Fail fast if torch install is not a ROCm/HIP build Add a post-install assertion after the torch step that errors out when torch.version.hip is None. With -f/--find-links, if the ROCm wheel were ever missing for the pinned version/Python/platform, pip could silently fall back to a PyPI CPU/CUDA wheel and leave the devcontainer broken; this makes that case a hard build failure instead. Co-Authored-By: Claude Opus 4.8 --- .devcontainer/rocm/Dockerfile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.devcontainer/rocm/Dockerfile b/.devcontainer/rocm/Dockerfile index e19b7569ae..4f8db93a96 100644 --- a/.devcontainer/rocm/Dockerfile +++ b/.devcontainer/rocm/Dockerfile @@ -84,6 +84,9 @@ RUN \ RUN /bin/micromamba create -n ${MAMBA_ENV_NAME} python=${PY_VERSION} gtest gmock bash-completion -c conda-forge && \ /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install --no-cache-dir cmake ninja "scikit-build-core>=0.4.3" "setuptools-scm>=9.2" pre-commit numpy pytest pytest-cov pytest-xdist pytest-rerunfailures pybind11 ruff && \ /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install --no-cache-dir torch==${TORCH_VERSION} -f https://repo.radeon.com/rocm/manylinux/rocm-rel-${ROCM_VERSION}/ && \ + # Fail fast if a non-ROCm torch slipped in (e.g. a PyPI CPU/CUDA wheel if the + # ROCm wheel were ever missing for this Python/platform under -f/--find-links). + /bin/micromamba run -n ${MAMBA_ENV_NAME} python -c "import torch, sys; hip = torch.version.hip; print(f'torch {torch.__version__} (hip={hip})'); sys.exit('ERROR: installed torch has no ROCm/HIP build (hip is None)' if hip is None else 0)" && \ /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install amd_aiter==${AITER_VERSION} --extra-index-url https://pypi.amd.com/rocm-${AITER_ROCM_VERSION}/simple SHELL ["/usr/local/bin/_dockerfile_shell.sh"] From b34a8052e372acc3984c082c8a1f248349fe0136 Mon Sep 17 00:00:00 2001 From: Debasis Mandal Date: Tue, 28 Jul 2026 00:56:01 -0400 Subject: [PATCH 3/5] Move HIP-check rationale to column-1 comments above RUN The explanatory comment was indented inside the line-continued RUN chain, where a mid-chain '#' is parsed ambiguously and can comment out the rest of the shell command. Move it to Dockerfile comments above the RUN so the command chain (incl. the HIP fail-fast check) stays intact. Co-Authored-By: Claude Opus 4.8 --- .devcontainer/rocm/Dockerfile | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/.devcontainer/rocm/Dockerfile b/.devcontainer/rocm/Dockerfile index 4f8db93a96..ed82c13f45 100644 --- a/.devcontainer/rocm/Dockerfile +++ b/.devcontainer/rocm/Dockerfile @@ -80,12 +80,13 @@ RUN \ echo 'eval "$(micromamba shell hook --shell bash)"' >> /home/$USERNAME/.bashrc && \ echo "micromamba activate ${MAMBA_ENV_NAME}" >> /home/$USERNAME/.bashrc -# Create a new micromamba env and install needed packages for flashinfer development +# Create a new micromamba env and install needed packages for flashinfer development. +# After installing torch, assert it is a ROCm/HIP build (torch.version.hip is not +# None) so the build fails fast if -f/--find-links ever resolves a PyPI CPU/CUDA +# wheel instead of the radeon ROCm wheel. RUN /bin/micromamba create -n ${MAMBA_ENV_NAME} python=${PY_VERSION} gtest gmock bash-completion -c conda-forge && \ /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install --no-cache-dir cmake ninja "scikit-build-core>=0.4.3" "setuptools-scm>=9.2" pre-commit numpy pytest pytest-cov pytest-xdist pytest-rerunfailures pybind11 ruff && \ /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install --no-cache-dir torch==${TORCH_VERSION} -f https://repo.radeon.com/rocm/manylinux/rocm-rel-${ROCM_VERSION}/ && \ - # Fail fast if a non-ROCm torch slipped in (e.g. a PyPI CPU/CUDA wheel if the - # ROCm wheel were ever missing for this Python/platform under -f/--find-links). /bin/micromamba run -n ${MAMBA_ENV_NAME} python -c "import torch, sys; hip = torch.version.hip; print(f'torch {torch.__version__} (hip={hip})'); sys.exit('ERROR: installed torch has no ROCm/HIP build (hip is None)' if hip is None else 0)" && \ /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install amd_aiter==${AITER_VERSION} --extra-index-url https://pypi.amd.com/rocm-${AITER_ROCM_VERSION}/simple SHELL ["/usr/local/bin/_dockerfile_shell.sh"] From ec7f0ee4de8da6cca1e1d0aa4f77bb64479d5efd Mon Sep 17 00:00:00 2001 From: Debasis Mandal Date: Tue, 28 Jul 2026 01:05:05 -0400 Subject: [PATCH 4/5] Document why torch install uses -f instead of --index-url Record the PEP 503 / flat-directory rationale inline so the -f/--find-links choice isn't accidentally reverted to --index-url (which 404s against the flat radeon wheel repo). Co-Authored-By: Claude Opus 4.8 --- .devcontainer/rocm/Dockerfile | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/.devcontainer/rocm/Dockerfile b/.devcontainer/rocm/Dockerfile index ed82c13f45..392a1fc7e0 100644 --- a/.devcontainer/rocm/Dockerfile +++ b/.devcontainer/rocm/Dockerfile @@ -81,9 +81,12 @@ RUN \ echo "micromamba activate ${MAMBA_ENV_NAME}" >> /home/$USERNAME/.bashrc # Create a new micromamba env and install needed packages for flashinfer development. -# After installing torch, assert it is a ROCm/HIP build (torch.version.hip is not -# None) so the build fails fast if -f/--find-links ever resolves a PyPI CPU/CUDA -# wheel instead of the radeon ROCm wheel. +# torch uses -f/--find-links (not --index-url) because the radeon repo is a flat +# wheel listing, not a PEP 503 simple index — with --index-url pip requests the +# per-package path (.../torch/), which 404s, and the install fails with "No +# matching distribution found". After installing, assert torch is a ROCm/HIP build +# (torch.version.hip is not None) so the build fails fast if -f ever resolves a +# PyPI CPU/CUDA wheel instead of the radeon ROCm wheel. RUN /bin/micromamba create -n ${MAMBA_ENV_NAME} python=${PY_VERSION} gtest gmock bash-completion -c conda-forge && \ /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install --no-cache-dir cmake ninja "scikit-build-core>=0.4.3" "setuptools-scm>=9.2" pre-commit numpy pytest pytest-cov pytest-xdist pytest-rerunfailures pybind11 ruff && \ /bin/micromamba run -n ${MAMBA_ENV_NAME} pip install --no-cache-dir torch==${TORCH_VERSION} -f https://repo.radeon.com/rocm/manylinux/rocm-rel-${ROCM_VERSION}/ && \ From 32bd7ba411e41a7fb63f9dbee1e44f189e741ec1 Mon Sep 17 00:00:00 2001 From: Debasis Mandal Date: Tue, 28 Jul 2026 10:39:19 -0400 Subject: [PATCH 5/5] Fix torch install guidance in README and hip_utils to use find-links README and the hip_utils torch-missing/mismatch messages told users to run `pip install torch ... --index-url https://repo.radeon.com/.../rocm-rel-X.Y/`, which fails ("No matching distribution found") because that repo is a flat wheel listing, not a PEP 503 simple index. Switch the radeon torch commands to `-f`/`--find-links` (matching the devcontainer Dockerfile and Dockerfile.rocm_ci). The pip still resolves the ROCm build over a same-version PyPI wheel because the `+rocm` local version ranks higher; the README NOTE that previously argued for --index-url is updated to explain this and to suggest verifying `torch.version.hip`. The unrelated pypi.amd.com/simple/ index-url lines (a real simple index) are left unchanged. Co-Authored-By: Claude Opus 4.8 --- README.md | 11 +++++++---- flashinfer/hip_utils.py | 6 +++--- 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 1201942203..5b06a92117 100644 --- a/README.md +++ b/README.md @@ -88,7 +88,7 @@ Install the matching ROCm-enabled PyTorch wheel from : ```bash -pip install torch==2.9.1 --index-url https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/ +pip install torch==2.9.1 -f https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/ ``` Other versions may work but have not been tested. Replace `7.2` with the @@ -141,11 +141,14 @@ pip install amd-flashinfer --index-url https://pypi.amd.com/simple/ Install the matching ROCm-enabled torch package from : ```bash -pip install torch==2.9.1 --index-url https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/ +pip install torch==2.9.1 -f https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/ ``` -**NOTE:** Use `--index-url` (not `-f`) so pip cannot silently fall back -to a CPU-only PyPI wheel. +**NOTE:** The radeon repo is a flat wheel listing, not a PEP 503 index, so +`--index-url` fails with "No matching distribution found" — use `-f` +(`--find-links`). pip still prefers the ROCm wheel over a same-version PyPI +wheel because its `+rocm` local version ranks higher. Confirm the result +with `python -c "import torch; assert torch.version.hip, 'not a ROCm build'"`. ### Trying the Examples diff --git a/flashinfer/hip_utils.py b/flashinfer/hip_utils.py index c28aa4f11f..7868377816 100644 --- a/flashinfer/hip_utils.py +++ b/flashinfer/hip_utils.py @@ -536,7 +536,7 @@ def check_torch_rocm_compatibility() -> None: " 1. Uninstall current PyTorch:\n" " pip uninstall torch\n\n" " 2. Install PyTorch for ROCm:\n" - " pip install torch==2.7.1 --index-url https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/\n\n" + " pip install torch==2.7.1 -f https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/\n\n" "See https://github.com/rocm/flashinfer for detailed installation instructions.\n" + "=" * 70 @@ -559,11 +559,11 @@ def check_torch_rocm_compatibility() -> None: f" PyTorch ROCm version: {torch_rocm_major_minor}\n\n" f"This may cause runtime errors or crashes.\n\n" f"To fix, reinstall PyTorch for your ROCm version:\n" - f" pip install torch==2.7.1 --index-url " + f" pip install torch==2.7.1 -f " f"https://repo.radeon.com/rocm/manylinux/rocm-rel-{system_rocm}/\n\n" f"Or if using uv:\n" f" export FLASHINFER_ROCM_VERSION={system_rocm}\n" - f" uv pip install torch==2.7.1 --index-url " + f" uv pip install torch==2.7.1 -f " f"https://repo.radeon.com/rocm/manylinux/rocm-rel-{system_rocm}/\n" f"{'=' * 70}", RuntimeWarning,