Skip to content

One host runtime, step 3: cuda-oxide adopts the root cuda-async types and cuda_async::simt goes away #1461

Open

Description

@elibol

Remove the duplicate cuda_async::simt runtime so SIMT launches and Tile operations compose through the same async APIs and share device storage without copies. Coordinate core types with #1460 and host dependencies with #1459.

Requirements

Implementation

  • Integrate shared typed storage with Tensor and SIMT launch arguments.
  • Reconcile simt::{DeviceOperation, DeviceFuture, launch} with shared DeviceOp, DeviceFuture, and AsyncKernelLaunch.
  • Rebase KernelArgument on DeviceCopy and reconcile simt::launch::{KernelArgument, KernelArguments} and the Box<T> escape hatch with the shared launcher.
  • Implement completion-safe cleanup without synchronization on drop. Early-dropped work must not free allocations or release borrowed host/device memory prematurely.
  • Reconcile device_context, scheduling_policies, reclaim, and error with their shared counterparts, preserving ownership, ordering, and error behavior.
  • Extend cutile_inter_kernel to pass one allocation through Tile and SIMT kernels via normal shared launch APIs, checking allocation identity and results without an intermediate device copy.
  • Cover asynchronous ordering, owned and borrowed resources, early drop/error paths, and CUDA graph capture/replay.
  • Migrate cuda-oxide crates, examples, generated-project templates, and documentation; move SIMT tests into shared suites and delete cuda-async/src/simt/.

Acceptance

Ordinary Tile and SIMT operations share runtime and storage types, zero-copy composition and safe cleanup are validated, and cuda-async/src/simt/ is removed. Document API migration and observable cleanup/lifetime behavior.

Related: #1460 (core runtime), #1503 (element representation), #1504 (device/context model), #1505 (public namespaces), #1506 (bool and the safe casts).

Activity

  1. added
    trackingUmbrella tracking issue
    cuda-oxideSIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book
    runtimeShared runtime crates: cuda-bindings, cuda-core, cuda-core-derive, cuda-async; Tile and SIMT co-own
    on Oct 8, 2026
  2. nihalpasham commented on Oct 9, 2026

    @nihalpasham
    Collaborator

    @elibol, I think the same staged approach makes sense here. There’s value in sharing the async runtime, but I need time to work through the behavior differences alongside #1460. Let’s keep this migration out of 0.5.0 and fit it around the other priorities.

    Is any of this currently blocking cutile-rs? If something is, let me know and we can look at that piece first.

    My initial read is that the import and naming changes should be straightforward once we settle the shared APIs. The pieces that need more thought are:

    • Shared buffers: We need to agree on who owns the allocation and how Tensor views and SIMT arguments share it safely. The memory needs to remain valid until all GPU work using it has finished.
    • Dropping futures: The root implementation waits for pending work when an in-flight future is dropped, while SIMT normally defers cleanup until completion. We should decide how the shared implementation handles owned buffers and borrowed views without unexpectedly changing application behavior.
    • Tile–SIMT interop: The existing example already shares device memory through a custom launcher. We can build on that by checking how the normal launch APIs compose, including graph capture and replay.
  3. added this to the cuda oxide v0.4.0 milestone on Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

cuda-oxideSIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, bookruntimeShared runtime crates: cuda-bindings, cuda-core, cuda-core-derive, cuda-async; Tile and SIMT co-owntrackingUmbrella tracking issue

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions