Skip to content

cuda-async: non-blocking drop for in-flight device futures (reaper) - #1391

Closed
roivanov wants to merge 1 commit into
mainfrom
feat/cuda-async-drop-reaper
Closed

roivanov wants to merge 1 commit into
mainfrom
feat/cuda-async-drop-reaper

Conversation

@roivanov

@roivanov roivanov commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

On behalf of @elibol. Transplant of NVlabs/cutile-rs#305 onto CUDA Rust after the repository merge. The commit keeps the original author and sign-off. Host-crate paths follow the move from cutile-rs/cuda-async/ to cuda-async/ at the repository root.

The source pull request is still a draft and was not targeted at 0.4.0. The changelog note is under Unreleased.

Summary

Dropping an in-flight DeviceFuture no longer synchronizes the stream on the dropping thread. The result handle is released immediately. The execution context is parked with the completion reactor and released on a dedicated reaper thread once a flag write enqueued behind the abandoned work lands.

Changes

  • Park abandoned execution contexts with the reactor and release them from cuda-async/src/reaper.rs.
  • Fall back to the previous inline wait when the reactor cannot take the context. Faulted and capturing streams still leak owners, quietly when the fault was already delivered.
  • Expose cuda_async::reaper::parked() and reaped_total().
  • Record the behavior change in cutile-rs/CHANGELOG.md: a future drop is no longer a barrier before an unchecked host access through a raw device pointer.

Testing

  • just -f cuda-oxide/Justfile check passes (the local mirror of CI: fmt, clippy, tests, guards, docs)
  • cargo oxide run <example> passes, or cuda-oxide/scripts/smoketest.sh -o '^<example>$'
  • New example added (if applicable)

cargo check -p cuda-async did not run here: no CUDA 13.0+ toolkit is installed on this machine.

Checklist

  • All commits signed off (git commit -s)
  • SPDX headers on new source files

Questions about the review? Ping us in #contributors on Discord.

…nstead of blocking

Dropping a DeviceFuture whose work is still in flight used to synchronize
the stream on the dropping thread, which under select!/timeout is an
executor thread. The result is a handle by the DeviceOp::execute contract
(every device-visible resource is retained by the submission), so it is
released immediately; the execution context is parked with the completion
reactor behind a flag write enqueued after the abandoned work and released
on a dedicated reaper thread when the flag lands. The reactor only hands
off, never releases. If the context cannot be parked it is dropped inline,
which is the previous blocking wait; faulted and capturing streams still
leak the owners with a report (quiet when the fault was already delivered).
reaper::parked() and reaper::reaped_total() expose the counts.

Signed-off-by: Melih Elibol <elibol@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 7, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

elibol pushed a commit that referenced this pull request Oct 7, 2026
## Summary

`AGENTS.md` now records the cutile-rs transplant procedure used for
[NVlabs/cutile-rs#305](NVlabs/cutile-rs#305) /
[#1391](#1391),
so the next replay can follow the same steps.

## Changes

- Keep the original author as author, committer, and the only sign-off.
- Prefix paths with `cutile-rs/`, then cherry-pick so the host-crate
move to the repository root is applied again.
- Leave code conflicts for a developer. A changelog note that belongs
under Unreleased may be resolved.
- Push that feature branch to `NVIDIA/cuda-rust` and cross-link the
source pull request. This is the one exception to the fork-only push
rule.

## Testing

Documentation only. No code or tests to run.

- [ ] `just -f cuda-oxide/Justfile check` passes (the local mirror of
CI: fmt, clippy, tests, guards, docs)
- [ ] `cargo oxide run <example>` passes, or
`cuda-oxide/scripts/smoketest.sh -o '^<example>$'`
- [x] New example added (if applicable)

## Checklist

- [x] All commits signed off (`git commit -s`)
- [x] SPDX headers on new source files

---
> Questions about the review? Ping us in [#contributors on
Discord](https://discord.gg/ZUEr4AhH5C).

Signed-off-by: Roman Ivanov <rivanov@nvidia.com>
@elibol

elibol commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

@roivanov thanks for testing agentic migration! I opened my PR here which I can drive to completion / merge: #1417

Can we close this PR?

@roivanov

roivanov commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator Author

@roivanov thanks for testing agentic migration! I opened my PR here which I can drive to completion / merge: #1417

Can we close this PR?

Sure! Thank you for testing it!

@roivanov roivanov closed this Oct 8, 2026
@roivanov
roivanov deleted the feat/cuda-async-drop-reaper branch October 8, 2026 01:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants