Repository navigation
One host runtime, step 3: cuda-oxide adopts the root cuda-async types and cuda_async::simt goes away #1461
Copy link
Copy link
Open
Labels
cuda-oxideSIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, bookSIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, bookruntimeShared runtime crates: cuda-bindings, cuda-core, cuda-core-derive, cuda-async; Tile and SIMT co-ownShared runtime crates: cuda-bindings, cuda-core, cuda-core-derive, cuda-async; Tile and SIMT co-owntrackingUmbrella tracking issueUmbrella tracking issue
Milestone
Description
Remove the duplicate cuda_async::simt runtime so SIMT launches and Tile operations compose through the same async APIs and share device storage without copies. Coordinate core types with #1460 and host dependencies with #1459.
Requirements
- Share allocation ownership and retain
DeviceBuffer<T>as the typed wrapper. Support Tensor storage, SIMT arguments,DeviceBoxusers, and owned, borrowed, foreign, and sliced allocations. Coordinate subviews with feat(cuda-core): add borrowed DeviceBuffer subviews #1447. - Apply the independent
DTypeandDeviceCopycontracts from One host runtime, step 2: cuda-oxide adopts the root cuda-core types and cuda_core::simt goes away #1460. Use both bounds only where an operation needs both.boolimplements both traits and is excluded from the safe reinterpreting casts by theAnyBitPatternbound from Safe element casts can produce invalidboolvalues (Tensor::reinterpret,DeviceBuffer::cast_elem/cast_chunks) #1506. - Keep one
KernelArgumenttrait with a single blanket implementation overDeviceCopy, replacing the root blanket overDTypeand thesimttrait. The launcher that#[kernel]generates for cutile already bounds its type parameters onDTypeand checksDTYPEat launch, so the shared trait does not need aDTypebound. - Do not implicitly wait for GPU work when buffers, streams, or async work are dropped. Keep allocations, submission arguments, contexts, streams, and other resources valid until their last GPU use completes. Define safe behavior for early drop, cancellation, errors, borrowed submissions, and forgotten futures.
- Use the shared context/stream API, including
Stream, while preserving distinct device and context identities as specified in cuda-core: distinguish CUDA devices and contexts in the shared host API #1504.
Implementation
- Integrate shared typed storage with Tensor and SIMT launch arguments.
- Reconcile
simt::{DeviceOperation, DeviceFuture, launch}with sharedDeviceOp,DeviceFuture, andAsyncKernelLaunch. - Rebase
KernelArgumentonDeviceCopyand reconcilesimt::launch::{KernelArgument, KernelArguments}and theBox<T>escape hatch with the shared launcher. - Implement completion-safe cleanup without synchronization on drop. Early-dropped work must not free allocations or release borrowed host/device memory prematurely.
- Reconcile
device_context,scheduling_policies,reclaim, anderrorwith their shared counterparts, preserving ownership, ordering, and error behavior. - Extend
cutile_inter_kernelto pass one allocation through Tile and SIMT kernels via normal shared launch APIs, checking allocation identity and results without an intermediate device copy. - Cover asynchronous ordering, owned and borrowed resources, early drop/error paths, and CUDA graph capture/replay.
- Migrate cuda-oxide crates, examples, generated-project templates, and documentation; move SIMT tests into shared suites and delete
cuda-async/src/simt/.
Acceptance
Ordinary Tile and SIMT operations share runtime and storage types, zero-copy composition and safe cleanup are validated, and cuda-async/src/simt/ is removed. Document API migration and observable cleanup/lifetime behavior.
Related: #1460 (core runtime), #1503 (element representation), #1504 (device/context model), #1505 (public namespaces), #1506 (bool and the safe casts).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
cuda-oxideSIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, bookSIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, bookruntimeShared runtime crates: cuda-bindings, cuda-core, cuda-core-derive, cuda-async; Tile and SIMT co-ownShared runtime crates: cuda-bindings, cuda-core, cuda-core-derive, cuda-async; Tile and SIMT co-owntrackingUmbrella tracking issueUmbrella tracking issue
Activity
nihalpasham commentedon Oct 9, 2026
@elibol, I think the same staged approach makes sense here. There’s value in sharing the async runtime, but I need time to work through the behavior differences alongside #1460. Let’s keep this migration out of 0.5.0 and fit it around the other priorities.
Is any of this currently blocking
cutile-rs? If something is, let me know and we can look at that piece first.My initial read is that the import and naming changes should be straightforward once we settle the shared APIs. The pieces that need more thought are: