Skip to content

feat(cuda-core): add CudaContext::mem_info for the (free, total) device memory query - #1445

Open
midagedev wants to merge 1 commit into
NVIDIA:mainfrom
midagedev:feat/context-mem-info
Open

midagedev wants to merge 1 commit into
NVIDIA:mainfrom
midagedev:feat/context-mem-info

Conversation

@midagedev

Copy link
Copy Markdown
Contributor

Summary

Recreated from NVlabs/cutile-rs#309 after the move. Closes #1438.

CudaContext::mem_info() returns (free, total) device memory bytes. There was no public way to ask this. The shim has mem_get_info, but it is in a pub(crate) module, so users call cuda_bindings::cuMemGetInfo_v2 directly. Our engine does this to check VRAM before loading a model.

let (free, total) = ctx.mem_info()?;

Changes

  • One method on CudaContext next to compute_capability. It binds the context first, like the other device queries.
  • The leak test vram_returns_to_baseline_after_buffer_cycles now uses the method instead of the raw call.
  • One new test: total equals cuDeviceTotalMem and free <= total.
  • A line under [Unreleased] in cutile-rs/CHANGELOG.md.

I replayed the commit with the "Transplanting a cutile-rs pull request" steps in AGENTS.md. Git put the CHANGELOG line in the 0.4.0 section, so I moved it to [Unreleased].

Testing

On this branch, CUDA 13:

  • cargo test -p cuda-core --test simt_device_buffer_leaks on an RTX 3060 (sm_86): 4 passed.
  • cargo fmt --check and cargo clippy -p cuda-core --all-targets -- -D warnings: clean.
  • I did not run the full just -f cuda-oxide/Justfile check. The change is additive and touches only cuda-core.

Checklist

  • All commits signed off (git commit -s)
  • SPDX headers on new source files (no new files)

I used an AI assistant to draft the patch and this text.

…ry query

cuMemGetInfo had no public wrapper: the shim's mem_get_info sits in a
pub(crate) module, so callers reached for the raw binding. Add the
method on CudaContext next to the other device queries; it binds the
context first. The leak test used the raw call and now uses the method.

Signed-off-by: midagedev <midagedev@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 8, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add CudaContext::mem_info for the (free, total) device memory query

1 participant