Repository navigation
fix(cargo-oxide): make doctor reachable without a CUDA toolkit or driver - #174
Merged
Merged
Conversation
libdevice ships in the toolkit's nvvm/ component next to libnvvm.so and is consumed together with libNVVM in the LTOIR pipeline, so the probe belongs in libnvvm-sys. cuda_host::ltoir::find_libdevice becomes a thin delegating wrapper with unchanged error type and message text, and ltoir.rs drops its now-unused cuda_roots/cuda_roots_from_env copy (the same helper already lives in libnvvm-sys). This also lets cargo-oxide's doctor probe libdevice without pulling the cuda-host -> cuda-core -> cuda-bindings chain.
…without the CUDA toolkit cargo-oxide had exactly two real CUDA-coupled call sites: the doctor libdevice probe (cuda_host::ltoir::find_libdevice) and the run-target auto-detect (cuda_core::CudaContext compute_capability). Both pulled the cuda-host -> cuda-core -> cuda-bindings chain, whose build script hard-fails without cuda.h and links libcuda, so on a bare machine 'cargo oxide doctor' died in a build script before main() (issue #87) and a prebuilt binary could not even load without a libcuda.so.1 stub. - doctor's libdevice check now calls libnvvm_sys::find_libdevice() (same probe, no cuda-bindings in the graph) - detect_run_target_arch now derives the compute capability from 'nvidia-smi --query-gpu=compute_cap --format=csv,noheader', the same derivation scripts/smoketest.sh already uses; output is trusted only on exit 0 AND a <digits>.<digits> shaped first line, since nvidia-smi prints failure banners to stdout - precedence is unchanged: --arch and CUDA_OXIDE_TARGET still override, the detected arch stays an advisory CUDA_OXIDE_DEVICE_ARCH hint, and format_sm_arch still emits sm_XYa for cc >= 9.0 Known tradeoff, documented on the helper: nvidia-smi lists GPUs in PCI order while CUDA defaults to fastest-first, so on heterogeneous multi-GPU boxes the hint may describe a different GPU than CUDA device 0. Safe because the hint is advisory and hard overrides remain. cargo tree -p cargo-oxide -e normal,build now contains no cuda-bindings, cuda-core, cuda-host, bindgen, or clang-sys, and the binary's DT_NEEDED no longer carries libcuda.so.1.
… driver Fixes #87. With the cuda-bindings edge gone, doctor finally runs on a bare machine, so make it actually diagnose one: - new fatal 'CUDA headers (cuda.h)' check that mirrors the discovery in cuda-bindings/build.rs (CUDA_TOOLKIT_PATH > CUDA_HOME > /usr/local/cuda; include/ then targets/<x86_64-linux|sbsa-linux>/ include per host arch). On failure it lists every probed path and the env vars to set. The probe is kept in lockstep by hand (build.rs logic is not a library and cuda-bindings must stay out of cargo-oxide's graph); both helpers are pure and unit-tested - new informational 'NVIDIA driver / GPU' check via nvidia-smi (reports name + compute capability), with a /proc/driver/nvidia/ version fallback that distinguishes 'driver loaded but nvidia-smi broken' from 'no driver at all'. Non-fatal: only 'cargo oxide run' needs a driver, and GPU-less CI is a supported workflow - doctor is now build-free: the Doctor arm resolves a side-effect-free context (new resolve_doctor_context + backend::backend_so_candidate) instead of resolve_context, which unconditionally built the backend (and could git-clone in standalone mode) before printing anything - the backend-.so check downgrades to informational '- not built yet (run `cargo oxide setup`)': run/build/pipeline build it on demand, so a fresh healthy clone stays green README and book updated to describe the new checks and that cargo-oxide plus doctor work without the CUDA toolkit.
cargo-oxide no longer pulls cuda-bindings, so its test job needs neither the toolkit install (build-time bindgen) nor the libcuda.so.1 stub shadow. The job now runs on a stock GPU-less, toolkit-less runner, which makes it the exact environment of issue #87; a follow-up step runs the built binary's doctor there and asserts the cuda.h diagnosis is reached (doctor exits 1 by design on that runner, so only the output is asserted).
The shadow existed solely because the cargo-oxide binary carried a DT_NEEDED libcuda.so.1 (its run-mode GPU-arch probe used the driver API). cargo-oxide now probes via nvidia-smi and links no CUDA at all, so the binary loads on a GPU-less runner unaided. Example binaries are still linked against the toolkit's lib64/stubs at link time and are never executed in --compile-only mode, so nothing else on the runner needs the soname shadow. Kept as its own commit so it reverts cleanly if some future job step starts exec-ing a cuda-linked binary.
Doctor gained CUDA headers (cuda.h) and driver / GPU checks, the backend-.so check became informational, and doctor no longer builds the backend before running. Reflect all of that in the checks table and add a short note on which checks are informational.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #87. PR #172 covered the cuda-bindings half (a clean build-script error instead of a bindgen panic when cuda.h is missing); this PR covers the reachability half: on a machine with no CUDA toolkit and no driver, cargo-oxide now builds, loads, and
cargo oxide doctorprints an actionable diagnosis instead of dying in a dependency's build script.What changed
libcuda.so.1in DT_NEEDED, so it builds and runs on a bare machine. libnvvm-sys / nvjitlink-sys stay: they are pure-Rust runtime-dlopen crates with zero build-time CUDA cost.nvidia-smi --query-gpu=compute_cap --format=csv,noheader(the derivation smoketest.sh already uses) instead of the CUDA driver API. Output is trusted only on exit 0 plus a<digits>.<digits>first line, because nvidia-smi prints failure banners to stdout. Precedence is unchanged:--archandCUDA_OXIDE_TARGETstill override, the result stays the advisoryCUDA_OXIDE_DEVICE_ARCHhint, andsm_XYais still emitted for cc >= 9.0.find_libdevice()(libdevice ships in the toolkit's nvvm/ component next to libnvvm.so), removing a duplicatedcuda_roots_from_env.cuda_host::ltoir::find_libdeviceremains as a thin wrapper with identical signature and error text.resolve_doctor_context()(withbackend::backend_so_candidate()) replacesresolve_context(), which used to build the backend unconditionally, or even git-clone in standalone mode, before a single check ran.CUDA_TOOLKIT_PATH>CUDA_HOME>/usr/local/cuda;include/thentargets/<x86_64-linux|sbsa-linux>/includeper host arch). On failure it lists every probed path and the env vars to set. The probe carries a lockstep comment: build.rs logic is not importable and cuda-bindings must stay out of the graph, so the two probes are kept in sync by hand. Both helpers are pure and unit-tested./proc/driver/nvidia/versionfallback that distinguishes "driver loaded but nvidia-smi broken" from "no driver at all". Non-fatal because onlycargo oxide runneeds a driver; GPU-less CI and cross-compiling are supported workflows..socheck downgrades to informational ("not built yet, runcargo oxide setup"): run/build/pipeline build it on demand, so a fresh healthy clone stays green.For users with a toolkit and GPU
cuDeviceGetAttribute.Known tradeoffs (documented in code)
--arch/CUDA_OXIDE_TARGETare hard overrides.cargo oxide setup" line.Verification
cargo tree -p cargo-oxide -e normal,build: no cuda-bindings / cuda-core / cuda-host / bindgen / clang-sys (all five present before).readelf -d cargo-oxide:libcuda.so.1gone from DT_NEEDED (present before; it is why both CI workflows carried the stub shadow).CUDA_TOOLKIT_PATH=/nonexistent CUDA_HOME=/nonexistent cargo build -p cargo-oxidesucceeds: no build script in the graph consults those vars anymore.CUDA_TOOLKIT_PATH=/nonexistentit prints the probed cuda.h paths and exits 1.cargo oxide build vecaddcompletes clean with a fresh vecadd.ptx and no swallowed-error signatures.-D warnings/ doc-D warnings/ tests green on cargo-oxide, libnvvm-sys, cuda-host; the three pre-existing detect_run_target_arch tests pass unmodified.GPU-box confirmation requested:
cargo oxide doctorshould print the GPU name and compute cap on the driver line, andcargo oxide run vecaddshould report a "Detected GPU arch" matchingnvidia-smi --query-gpu=compute_capand pass.Suggested labels: QoL-improvement, cargo-oxide, host-apis, build-related.
GPU-box confirmation (done)
Verified on a 2x RTX 5090 box: doctor prints "NVIDIA driver / GPU... ✓ NVIDIA GeForce RTX 5090 (compute capability 12.0)" and exits green;
cargo oxide run vecaddreports "Detected GPU arch: sm_120a (via nvidia-smi)" and passes with all 1024 elements correct.readelf -don the built binary confirms libcuda.so.1 is gone from DT_NEEDED.