Skip to content

nvidia-gpu-compatibility.md

Latest commit

 

History

History
97 lines (62 loc) · 4.35 KB

nvidia-gpu-compatibility.md

File metadata and controls

97 lines (62 loc) · 4.35 KB
Raw

NVIDIA GPU / CUDA Compatibility

This guide explains what must match between your GPU, your NVIDIA driver, and the installed PyTorch build, and what to do when CodeFinetuner reports a problem.

Requirements

1. Your driver must meet the minimum driver version for PyTorch's CUDA major release family.

The "CUDA Version" shown by nvidia-smi is not a ceiling. It only tells you the newest CUDA toolkit that driver shipped with — not the newest one it can run. Example:

| NVIDIA-SMI 550.127.05    Driver Version: 550.127.05    CUDA Version: 12.6 |

This driver can still run a PyTorch build compiled for CUDA 12.8, because NVIDIA guarantees Minor Version Compatibility (MVC): any toolkit build from the same major CUDA family (here, 12.x) works, as long as the driver is at or above that family's minimum required version:

PyTorch CUDA build Minimum driver version (Linux)
CUDA 13.x >= 580
CUDA 12.x >= 525
CUDA 11.x >= 450

Check your installed PyTorch build's CUDA version with:

python3 -c "import torch; print(torch.version.cuda)"

Then check your driver version with nvidia-smi (top-right corner shows Driver Version). What matters is:

  • Major version must match between your driver's CUDA family and PyTorch's CUDA build (e.g. both 12.x).
  • Driver version must be >= the minimum for that family (e.g. >= 525 for CUDA 12.x).

The "CUDA Version" number in nvidia-smi is a convenience label, not a strict cutoff — ignore it when comparing against PyTorch's version, and check the driver version and the table above instead.

If PyTorch's build is from a newer major family than your driver supports (e.g. driver 550, which is CUDA 12.x, but a PyTorch cu130 build), CUDA cannot start at all and training falls back to CPU.

2. Your GPU must be supported by that PyTorch build.

Each PyTorch build includes compiled code only for certain GPUs. If yours is not included, CodeFinetuner stops with an error at startup instead of failing later during training.

3. torch and torchvision must use the same CUDA major version.

torchvision is an indirect dependency (required by unsloth), and it raises an error at import time if its CUDA major version differs from torch's:

RuntimeError: Detected that PyTorch and torchvision were compiled with different
CUDA major versions. PyTorch has CUDA Version=13.0 and torchvision has CUDA Version=12.8.

This means you must always install torch and torchvision together from the same index. Installing only torch leaves torchvision on its previous build and breaks the import.

Diagnosing

Run the diagnostic script to see what is detected on your machine:

python3 scripts/check_cuda.py

It prints your driver status, your GPU, the PyTorch build's CUDA version, and whether the build supports your GPU.

Fixing

Either update your NVIDIA driver, or install a PyTorch build that matches your hardware.

Note: Updating the driver only helps with requirement 1. If the error says your GPU is not supported by the build, a different PyTorch build is required.

Installing a different PyTorch build

  1. Run nvidia-smi and check Driver Version (not the "CUDA Version" label).
  2. Look up the minimum driver version for each CUDA major family in the table above, and pick the newest CUDA family your driver satisfies.
  3. Go to pytorch.org/get-started/locally and pick a PyTorch build from that CUDA family, then copy the index URL it gives you.
  4. Reinstall torch and torchvision with that URL, depending on how you installed CodeFinetuner:
# uv tool install
uv tool install codefinetuner --reinstall --with torch --with torchvision --index <that-url> --index-strategy unsafe-best-match

# uv project / from source
uv add torch torchvision --index pytorch-cuda=<that-url> --index-strategy unsafe-best-match

# pip
pip install torch torchvision --index-url <that-url> --force-reinstall

Then re-run python3 scripts/check_cuda.py to confirm.

Making the pin permanent

To stop torch and torchvision drifting apart on future uv sync runs, pin both to the same index in pyproject.toml:

[tool.uv.sources]
torch = [{ index = "pytorch-cuda", marker = "sys_platform == 'linux'" }]
torchvision = [{ index = "pytorch-cuda", marker = "sys_platform == 'linux'" }]