build(deps): install flash-attn from prebuilt wheels - #98
Merged
Conversation
PyPI has no flash-attn wheels, so installing the verl-experimental extra compiled it from the sdist: ~1h of nvcc, and it needed CUDA_HOME pointing at a CUDA 13 toolkit to match torch 2.11's cu130 build. Astral's GPU index publishes prebuilt wheels keyed by (cuda, torch). The extras keep a bare `flash-attn==2.8.3` pin: each wheel declares `torch ==2.11.*`, so the variant follows the resolved torch and a torch bump needs no edit here. Both index entries are `explicit` so nothing else resolves from them -- the astral index also carries vllm, which would otherwise silently shift off PyPI. Install drops from ~1h to under two minutes and needs no CUDA toolkit. Validated on 8xB200: clean venv, `uv sync --extra verl-experimental` with CUDA_HOME unset, then examples/math_agent/fsdp_fft_sync_grpo.sh. GSM8K val reward 0.572 -> 0.927 by step 30, matching the README's figure for this config.
lyzustc
approved these changes
Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PyPI has no flash-attn wheels, so installing the
verl-experimentalextra compiledit from the sdist: ~1h of nvcc, and it needed
CUDA_HOMEpointing at a CUDA 13toolkit to match torch 2.11's cu130 build.
Astral's GPU index publishes prebuilt wheels keyed by
(cuda, torch).
Notes
flash-attn==2.8.3pin: each wheel declarestorch ==2.11.*, so the variant follows the resolved torch and a torch bump needsno edit here.
explicitso nothing else resolves from them — the astralindex also carries
vllm, which would otherwise silently shift off PyPI (sameversion string, different build).
Pascal and Volta). Documented in the backend README.
uv.lockcarries only the flash-attn delta; it is not otherwise refreshed.Validation
Clean venv on 8xB200,
uv sync --extra verl-experimentalwithCUDA_HOMEunset,then
examples/math_agent/fsdp_fft_sync_grpo.shagainst a live ACR agent.CUDA_HOME=... uv syncuv syncGSM8K val reward 0.572 → 0.927 by step 30, matching the README's figure for this
config.