[realtime] Pluggable transport providers + per-device device_call sessions by bmhowe23 · Pull Request #4915 · NVIDIA/cuda-quantum

bmhowe23 · 2026-07-12T21:35:27Z

Note: this is to be paired with CUDA-Q QEC PR: NVIDIA/cudaqx#682

~~Note: do not review code changes yet as they are guaranteed to change.~~

Summary

This PR turns the realtime transport layer into a true plug-in boundary and extends the device_call runtime to support one independent ring/dispatcher per device. Transport providers are now plain shared libraries selected by name or path at runtime; the core dispatcher, the device_call runtime, and every consumer (notably the CUDA-QX decoding server, see the companion CUDA-QX PR) speak only the cudaq_bridge_* C API and stay wire-agnostic. An external transport provider -- a partner's NIC stack, a lab interconnect, a simulator -- drops in as a single .so with zero changes to cudaq or to consumers.

Before

The bridge loader knew exactly two providers: the built-in Hololink library (hardwired soname behind CUDAQ_PROVIDER_HOLOLINK) and ONE external library per process, named only through the CUDAQ_REALTIME_BRIDGE_LIB environment variable behind CUDAQ_PROVIDER_EXTERNAL. Two different external transports could not coexist; a second create with a different env value silently reused the first library.
The bridge interface had no capability queries: a consumer could not ask a provider for its live endpoint identity (bound port / QP / rkey) or its ring geometry, so callers re-parsed provider CLI args and duplicated geometry constants, which drifted.
UDP and CPU-RoCE ring transports existed only as internal wrappers wired directly into consumers; the CPU-RoCE rendezvous / HSB-FPGA handshake logic lived in application code (the CUDA-QX decoding server) rather than behind the transport boundary.
The device_call runtime created exactly one dispatch session (device 0) with one channel and one set of channel arguments; device_call(device_id != 0, ...) had no runtime backing, so one-ring-per-consumer topologies were impossible.
The dispatch graph's device-side cudaGraphLaunch of the triggered graph dropped its return code, making device-side trigger failures indistinguishable from a hung triggered graph.

After

Bridge interface v2 (bridge_interface.h): three capability queries appended to the provider struct -- get_endpoint_info (one-line key=value endpoint description, valid as soon as create() returns so servers can publish rendezvous info before a blocking connect()), get_ring_geometry (dispatcher configuration derives from the transport instead of duplicated CLI), and get_cpu_dataplane (rx_poll/tx_publish hooks for the unified host loop). Version negotiation is asymmetric by design: a v2 core loads v1 providers (fields past disconnect are never read), an old core rejects v2 providers loudly at load time, and absent capabilities return the new CUDAQ_ERR_UNSUPPORTED status.
String-keyed provider loader: cudaq_bridge_create_from_library(handle, "libfoo.so" | "/path/foo.so", argc, argv) is the new primary entry point. Loaded libraries are cached per process keyed by that string, so any number of distinct provider libraries coexist, each serving any number of bridge instances. Hololink is no longer special -- it is just the provider whose library is libcudaq-realtime-bridge-hololink.so. The enum-based cudaq_bridge_create survives as a thin compatibility wrapper.
Two in-tree providers join hololink: libcudaq-realtime-bridge-udp.so (wraps the udp ring transceiver; --pinned-rings allocates CUDA pinned+mapped rings so a device-resident dispatch scheduler can poll them directly -- this is what makes the GPU device-graph path testable on any CUDA box without RDMA hardware) and libcudaq-realtime-bridge-cpu-roce.so (absorbs the rendezvous and hsb_fpga QP handshakes that previously lived in consumer code). The hololink provider implements the v2 queries.
Per-device device_call sessions: sessions are keyed by device id and created lazily on first use; CUDAQ_DEVICE_CALL_CHANNEL accepts a spec (<default>[,<id>=<channel>...]) to steer individual devices to different channels; channel arguments accept device-scoped forms (<key>.<id>=<value>, e.g. udp-port.1=48145) so each device can own its own external endpoint -- together these enable the one-ring-per-decoder topology (device_id == decoder_id) the companion CUDA-QX PR builds on.
Trigger-launch diagnostics: cudaq_dispatch_get_trigger_debug() surfaces the device-side launch rc of the triggered graph plus fire/tail-relaunch counters (healthy: rc=0, fires == tails). It uses async copies on a private non-blocking stream, so it is safe to call while the persistent scheduler graph is resident, live or wedged. This is the instrumentation that root-caused a cooperative-launch co-residency deadlock during validation.
Robustness fixes from self-review: provider connect/launch/disconnect are invoked after dropping the loader's map lock (a blocking rendezvous connect previously deadlocked every other bridge call in the process, including the destroy that could cancel it); dlopen handles are closed on every loader error path; cpu_roce disconnect latches disconnected state (a later launch() previously spawned a monitor thread over the closed transceiver) and validates --local-ip; the device_call driver latches a finalized flag so lazy init cannot resurrect sessions after an explicit finalize; flag-style channel arguments (no =) are forwarded instead of dropped; libcudaq-device-call-runtime.so no longer re-exports transport-archive symbols (--exclude-libs; providers previously bound across two copies of the same objects and corrupted teardown).

Breaking changes

Bridge interface version is now 2. The provider struct is extended append-only, so existing v1 provider binaries still load and run against this core. The reverse does not hold: a provider built reporting version 2 is rejected by a pre-PR core ("expected 1, got 2"). Core and providers must upgrade together across this boundary.
cudaq_status_t gains CUDAQ_ERR_UNSUPPORTED (appended, existing values unchanged). Callers with exhaustive switches over the enum need a new case.
The one-external-library-per-process limit is gone, and with it a silent misbehavior: previously, creating a second bridge after changing CUDAQ_REALTIME_BRIDGE_LIB silently reused the FIRST library (the cache was keyed by enum slot). Now the named library is actually loaded. Code that accidentally relied on the old aliasing will observe different (correct) behavior.
cudaq_bridge_create rejects out-of-range provider enums with CUDAQ_ERR_INVALID_ARG instead of treating any non-hololink value as EXTERNAL.
Provider callbacks are no longer serialized by the loader's global lock. connect/launch/disconnect/get_transport_context on different handles now run concurrently. Racing calls on the SAME handle were never supported; providers that implicitly relied on the global lock for cross-handle exclusion must provide their own.
libcudaq-device-call-runtime.so no longer re-exports cpu_udp_* / transport-archive symbols. Anything that linked against those incidental re-exports must link the transport archive directly.
device_call dispatch after finalize now fails with DeviceCallNotInitializedStatus as originally documented, instead of silently re-initializing a fresh session (regression introduced and fixed within this PR's own lazy-init work; net behavior change is only vs. the lazy-init intermediate state, not vs. main).
cpu_roce launch() after disconnect() returns an error instead of starting a monitor thread on a closed transceiver; a disconnected rendezvous bridge must be destroyed and re-created.

Dependencies

Depends on [realtime] Add a unified CPU dispatch loop #4869 (realtime dispatcher-object API). This branch is based on the [realtime] Add a unified CPU dispatch loop #4869 head merged with main; rebase once [realtime] Add a unified CPU dispatch loop #4869 lands so its commits drop out of the diff.
The companion CUDA-QX PR (decoding server as a bridge-provider consumer, one ring per decoder, mixed host+device_graph dispatch) depends on this PR and consumes the interface additions (create_from_library, v2 queries, per-device sessions/scoped args, --pinned-rings, trigger debug).

Validation

device_call unit tests 6/6 (including FinalizeClearsPluginSession, which the intermediate lazy-init state had broken).
End-to-end through the CUDA-QX decoding server on this stack: two-process suite (udp per-decoder rings), and the full mixed host+device_graph flow over pinned-udp rings on a WSL2 laptop -- GPU dispatch graph fired real RelayBP decode graphs (trigger rc=0, fires == tail_relaunches), correct corrections. cpu_roce provider is syntax/loader-validated locally; its RDMA data path needs an HSB rig (no ConnectX/DOCA on the dev box).

🤖 Generated with Claude Code

The host dispatcher only supported the ring model, in which separate transport threads own the TX side of the ring buffer. Some transport providers instead need the daemon to service requests from a single CPU thread that owns both RX and TX. This adds that path and, rather than carrying a second copy of the per-slot dispatch logic, folds both shapes behind one dispatch core selected by a transport policy — so framing, validation, routing, and GRAPH_LAUNCH handling can no longer drift apart between them. To keep that core transport-agnostic, a provider now exposes a CPU data-plane (a device-visible ring plus non-blocking poll/publish hooks) through the bridge interface; providers that don't support the single-thread shape simply omit it. The dispatcher also takes uniform ownership of the worker thread and graph engine for both paths and only stands up the GPU graph engine when the function table actually contains GRAPH_LAUNCH work, which lets the standalone host-dispatcher handle API be retired. Signed-off-by: boschmitt <7152025+boschmitt@users.noreply.github.com>

…der work) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Bump CUDAQ_REALTIME_BRIDGE_INTERFACE_VERSION to 2 and define the compatibility rule the bump implies: the loader accepts providers reporting any version in [1, CURRENT], and fields beyond `disconnect` (get_cpu_dataplane, and the new get_endpoint_info / get_ring_geometry) are only read from providers reporting version >= 2. get_cpu_dataplane was appended without a bump; this formalizes it as a v2 field. New capability queries (each may be NULL in a provider; the API wrapper then returns the new CUDAQ_ERR_UNSUPPORTED status): - cudaq_bridge_get_endpoint_info: one-line key=value description of the live endpoint (port, roce_ip, QP/RKey, ...), valid as soon as create() returns so a server can publish its rendezvous endpoint BEFORE connect() blocks waiting for the peer. Removes the need for every consumer to grow transport-specific readiness printing. - cudaq_bridge_get_ring_geometry: slot count / slot stride, so the dispatcher config is derived from the transport instead of duplicated on every consumer's command line. - cudaq_bridge_get_cpu_dataplane: public wrapper for the v2 vtable field (previously reachable only by poking the struct). Also fixes a pre-existing loader bug: the interface was cached in provider_interface_map before the version check, leaving a rejected provider's interface installed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

Wrap the loopback/Ethernet UDP ring transceiver behind the transport- provider interface so consumers that speak only cudaq_bridge_* / cudaq_dispatcher_* run over UDP with zero transport-specific code. create() binds (endpoint queryable before connect), connect() is a no-op (connectionless), launch() starts the ring threads. Implements the v2 endpoint-info and ring-geometry queries; ring context fills both the device-pointer and host-view fields with the same host addresses. Also reorders lib/ subdirectories (cpu_transport before daemon) so the bridge providers can link and probe the transport targets. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

…e.so) Wrap the CPU RoCE RDMA ring transceiver behind the transport-provider interface, absorbing BOTH queue-pair exchange methods that previously had to live inline in every consumer: - --qp_config=rendezvous: the service end of CpuRoceChannel's TCP QP/rkey rendezvous. The byte-for-byte RendezvousInfo wire struct now lives next to the transceiver and channel it pairs with, instead of being duplicated in downstream repos. create() runs setup() and binds the rendezvous socket, so the endpoint is publishable before connect() blocks in accept(). - --qp_config=hsb_fpga: the Holoscan-Sensor-Bridge FPGA method (hsb_bridge_cpu.cpp precedent): peer QP from --remote-qp, one-shot start() in create(), and connect() prints the canonical '=== Bridge Ready ===' handshake (strict-regex format from hololink_bridge_common.h) with no control-plane traffic. Gated on the CPU RoCE transport target (libibverbs); implements the v2 endpoint-info and ring-geometry queries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

… queries get_endpoint_info reports the RDMA target identity (QP, rkey, buffer address, peer IP) as key=value pairs so consumers can publish it without linking hololink; get_ring_geometry reports num_pages / page_size so dispatcher and scheduler geometry can be derived from the provider. get_cpu_dataplane stays NULL (GPU rings; no host unified shape). Syntax-checked against the realtime headers; full compile/link needs HOLOSCAN_SENSOR_BRIDGE_BUILD_DIR + DOCA (not present on this dev box) and should be exercised in the rig build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

Two changes that make device_call(device_id, ...) a usable data-plane axis (e.g. the QEC one-ring-per-decoder pattern, device_id == decoder_id): - Lazy per-device initialization: acquireFrameForDevice now calls the (idempotent) initializeServiceForDevice for an unseen device id instead of throwing NotInitialized. Eager init only ever covered DefaultDeviceId, so any nonzero-device device_call previously died in lowered code with 'illegal execution of unreachable code'. Builtin channels thus get one ring + one dispatcher per device id on first use. External (non-builtin) channels keep their one-endpoint-per- process behavior: other device ids share DefaultDeviceId's channel (payload-demuxed, exactly the pre-per-device semantics); per-device external endpoints are a follow-up. - Channel SPEC syntax: CUDAQ_DEVICE_CALL_CHANNEL (and the --cudaq-device-call[-channel] argv forms) now accept '<default>[,<id>=<channel>...]', e.g. host_dispatch,1=device_dispatch, so one process can run one device on a host_dispatch (CPU memory) ring and another on a device_dispatch (GPU persistent-kernel) ring simultaneously. Bare values parse exactly as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

…r path cudaq_bridge_create's early-return for an already-loaded provider called the provider's create() but never published the new handle in bridge_handle_interface_map, so every bridge instance after the first failed all subsequent cudaq_bridge_* calls with 'Invalid bridge handle'. Surfaced by the one-ring-per-decoder decoding server, which creates one bridge instance per decoder on the same provider. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

…rgs) External-channel arguments now support device scoping: '<key>.<id>=<value>' applies only to device id (overriding any plain '<key>='), e.g. --cudaq-device-call=udp udp-host=127.0.0.1 udp-port=48144 udp-port.1=48145 A device with scoped arguments gets its OWN external channel (own ring) created lazily on first use; devices without scoped arguments keep the previous behavior of sharing DefaultDeviceId's channel. Together with device_call(device_id, ...) routing this completes the two-process one-ring-per-decoder topology: the decoding server publishes one endpoint per decoder and the caller dials each decoder's session to its own ring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

cpu_udp_create_transceiver_ext lets the caller supply (and own) the four ring buffers, keeping the transport itself CUDA-free while enabling ring memory a GPU consumer can poll. The udp bridge provider grows a --pinned-rings flag that allocates the rings as CUDA pinned+mapped host memory (same pointer under UVA for the device alias) and hands them to the external-rings API -- the local-validation enabler for GPU ring consumers (e.g. the QEC device-graph scheduler) without RDMA hardware: the wire is plain UDP, the ring is GPU-pollable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

…time libcudaq-device-call-runtime.so statically absorbs the udp/cpu_roce transport archives for its channels and re-exported their cpu_udp_*/ cpu_roce_* symbols. A bridge provider .so carries its own copy of the same transceiver code, and the dynamic linker bound the provider's calls to the runtime's copy -- a silent one-definition violation that corrupted teardown whenever the copies diverged (observed: provider created an external-rings transceiver, destroy bound to the runtime's older copy, which freed caller-owned pinned buffers with std::free). --exclude-libs the transport archives so each .so keeps its own copy private. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

…patch graph The dispatch graph's fire-and-forget launch of the triggered (decode) graph dropped its device-side cudaGraphLaunch return code, making a failed trigger indistinguishable from a hung decode graph. Record the rc plus fire and tail-relaunch counters in device globals, and export cudaq_dispatch_get_trigger_debug() to read them -- using async copies on a private non-blocking stream so the reader is safe to call while the persistent scheduler is live or wedged (a legacy default-stream memcpy deadlocks against it). This instrumentation localized a decoding-server pipeline hang to cooperative-launch co-residency starvation (decode graphs captured with reserved_sms=0 cannot co-reside with the resident dispatch graph); a healthy run reads rc=0 with fires == tail_relaunches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

cudaq-realtime serves realtime dispatch generally, not one application. Reword the comments added with the trigger diagnostics and the --pinned-rings option to the library's own vocabulary (triggered graph, cooperative grid, device-resident dispatch scheduler) instead of naming a particular consumer's domain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

Add cudaq_bridge_create_from_library(handle, library, argc, argv): the provider is named directly by any string dlopen accepts, and loaded libraries are cached per process keyed by that string. Any number of distinct provider libraries can now coexist in one process (the previous enum keying allowed exactly one built-in plus one EXTERNAL library), and the built-in Hololink provider is no longer special -- it is just the provider whose library is libcudaq-realtime-bridge-hololink.so. The enum-based cudaq_bridge_create remains as a convenience wrapper (HOLOLINK -> bundled soname, EXTERNAL -> CUDAQ_REALTIME_BRIDGE_LIB), so existing callers are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

…ce_call runtime Bridge loader (bridge_interface_api.cpp): - dlclose the library on every post-dlopen error path (dlsym miss, null interface, version mismatch) instead of leaking the mapping per retry. - Invoke provider connect/launch/disconnect/get_transport_context AFTER dropping the map lock: a provider connect may block indefinitely (e.g. rendezvous accept), and holding the shared lock across it deadlocked every other bridge call in the process, including the destroy that could cancel the stuck bridge. Interface structs are static in never-unloaded provider libraries, so the pointer outlives the lock. - Reject out-of-range provider enums instead of treating them as EXTERNAL; fix an error-message typo. cpu_roce provider: disconnect now clears `connected` (a later launch() previously spawned a monitor thread over the closed transceiver), and a failed inet_pton on --local-ip is an error instead of silently advertising 0.0.0.0 to the peer. device_call runtime (DeviceCallDispatch.cpp): - Fix a lazy-init regression: shutdown() now latches a `finalized` flag so acquireFrameForDevice cannot silently resurrect a session after an explicit finalize (restores DeviceCallNotInitializedStatus; a later explicit initialize re-arms the driver). - Flag-style channel arguments without '=' are forwarded to every device's channel instead of silently dropped by the per-device argument filter. - hasDeviceScopedArguments parses the `.<id>` suffix numerically, matching argumentsForDevice (so `<key>.01=` both scopes and triggers dedicated-channel creation for device 1). - The lazily created per-device external channel path registers the shutdown handler like every other session-creating path. - Document that repeated channel specs merge per-device overrides. Also: declare cudaq_dispatch_get_trigger_debug in cudaq_realtime.h (consumers previously had to hand-declare it), correct the stale synchronous-copy comment on it, and fix the udp provider's CMake header comment about CUDA linkage. device_call unit tests: 6/6 pass (FinalizeClearsPluginSession was failing before this commit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

copy-pr-bot · 2026-07-12T21:35:36Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

bmhowe23 · 2026-07-12T21:56:26Z

/ok to test 171fb7a

Command Bot: Processing...

github-actions · 2026-07-12T22:22:24Z

CI Summary (`push`) — ✅ passed

Run #29969567452 · ✅ 6 · ⏩ 7 · ❌ 0 · ⛔ 0

Top-level jobs (13)

Job	Result
`binaries`	⏩ skipped
`build_and_test`	✅ success
`config_devdeps`	✅ success
`config_source_build`	⏩ skipped
`config_wheeldeps`	✅ success
`devdeps`	✅ success
`docker_image`	⏩ skipped
`gen_code_coverage`	⏩ skipped
`metadata`	✅ success
`python_metapackages`	⏩ skipped
`python_wheels`	⏩ skipped
`source_build`	⏩ skipped
`wheeldeps`	✅ success

⏩ Skipped jobs (7) — intentionally skipped on PR builds; run on merge_group / workflow_dispatch

Job
`binaries`
`config_source_build`
`docker_image`
`gen_code_coverage`
`python_metapackages`
`python_wheels`
`source_build`

All sub-jobs (42) — every matrix leg, with links

Job	Status	Link
Build and test (amd64, gcc12, openmpi) / Dev environment (Debug)	✅ success	view
Build and test (amd64, gcc12, openmpi) / Dev environment (Python)	✅ success	view
Build and test (amd64, llvm, openmpi) / Dev environment (Debug)	✅ success	view
Build and test (amd64, llvm, openmpi) / Dev environment (Python)	✅ success	view
Build and test (arm64, llvm, openmpi) / Dev environment (Debug)	✅ success	view
Build and test (arm64, llvm, openmpi) / Dev environment (Python)	✅ success	view
CI Summary	❔ in_progress	view
Configure build (devdeps)	✅ success	view
Configure build (source_build)	⏩ skipped	view
Configure build (wheeldeps)	✅ success	view
Create CUDA Quantum installer	⏩ skipped	view
Create Docker images	⏩ skipped	view
Create Python metapackages	⏩ skipped	view
Create Python wheels	⏩ skipped	view
Gen code coverage	⏩ skipped	view
Load dependencies (amd64, gcc12) / Caching	✅ success	view
Load dependencies (amd64, gcc12) / Finalize	✅ success	view
Load dependencies (amd64, gcc12) / Metadata	✅ success	view
Load dependencies (amd64, llvm) / Caching	✅ success	view
Load dependencies (amd64, llvm) / Finalize	✅ success	view
Load dependencies (amd64, llvm) / Metadata	✅ success	view
Load dependencies (arm64, gcc12) / Caching	✅ success	view
Load dependencies (arm64, gcc12) / Finalize	✅ success	view
Load dependencies (arm64, gcc12) / Metadata	✅ success	view
Load dependencies (arm64, llvm) / Caching	✅ success	view
Load dependencies (arm64, llvm) / Finalize	✅ success	view
Load dependencies (arm64, llvm) / Metadata	✅ success	view
Load source build cache	⏩ skipped	view
Load wheel dependencies (amd64, 12.6) / Caching	✅ success	view
Load wheel dependencies (amd64, 12.6) / Finalize	✅ success	view
Load wheel dependencies (amd64, 12.6) / Metadata	✅ success	view
Load wheel dependencies (amd64, 13.0) / Caching	✅ success	view
Load wheel dependencies (amd64, 13.0) / Finalize	✅ success	view
Load wheel dependencies (amd64, 13.0) / Metadata	✅ success	view
Load wheel dependencies (arm64, 12.6) / Caching	✅ success	view
Load wheel dependencies (arm64, 12.6) / Finalize	✅ success	view
Load wheel dependencies (arm64, 12.6) / Metadata	✅ success	view
Load wheel dependencies (arm64, 13.0) / Caching	✅ success	view
Load wheel dependencies (arm64, 13.0) / Finalize	✅ success	view
Load wheel dependencies (arm64, 13.0) / Metadata	✅ success	view
Prepare cache clean-up	❔ in_progress	view
Retrieve PR info	✅ success	view

✅ Required checks (6/6) — declared in .github/required-checks.yml for push

Required check	Status	Link
Build and test (amd64, llvm, openmpi) / Dev environment (Debug)	✅ success	view
Build and test (amd64, llvm, openmpi) / Dev environment (Python)	✅ success	view
Build and test (arm64, llvm, openmpi) / Dev environment (Debug)	✅ success	view
Build and test (arm64, llvm, openmpi) / Dev environment (Python)	✅ success	view
Build and test (amd64, gcc12, openmpi) / Dev environment (Debug)	✅ success	view
Build and test (amd64, gcc12, openmpi) / Dev environment (Python)	✅ success	view

…time-bridge-providers

bmhowe23 · 2026-07-15T14:05:26Z

/ok to test 4d5777e

Command Bot: Processing...

…roviders Signed-off-by: Ben Howe <bhowe@nvidia.com> # Conflicts: # realtime/include/cudaq/realtime/daemon/bridge/bridge_interface.h

bmhowe23 · 2026-07-22T22:39:39Z

/ok to test 2694704

Command Bot: Processing...

Signed-off-by: Renaud Kauffmann <rkauffmann@nvidia.com>

Renaud-K · 2026-07-23T00:14:32Z

/ok to test e2f7faa

Command Bot: Processing...

Renaud-K

Thank you.

bmhowe23 · 2026-07-23T00:36:40Z

/ok to test 941bc91

Command Bot: Processing...

bmhowe23 · 2026-07-23T14:01:15Z

Since @Renaud-K approved the core changes, I'm approving for the spell check changes, which he did.

…r decoder, mixed host + device_graph dispatch (#682) Built on the companion cuda-quantum PR NVIDIA/cuda-quantum#4915, **now merged upstream** (squash `2a7911f`, 2026-07-23); `.cudaq_version` pins that commit, so this branch builds against stock `NVIDIA/cuda-quantum` main. ## Summary This PR re-architects the realtime decoding server around the CUDA-Q bridge-provider boundary. The server no longer contains any transport code: the wire is named in the deployment YAML (or a `--transport` fallback), loaded at runtime as a `libcudaq-realtime-bridge-<name>.so` provider, and every decoder gets its own ring buffer and its own consumer -- a host dispatcher thread for `dispatch: host` decoders, or the GPU device-graph scheduler for `dispatch: device_graph` decoders, both side by side in one server process. A partner transport library drops in as a single `.so` with zero decoding-server changes. The guiding litmus test for the layering: cudaq-realtime and its transport providers never say "QEC" or "decoder"; the QEC library code never names a wire (UDP, RoCE, hololink). Wire names appear only where deployments are described: YAML configs, launch scripts, and tests. ## Before (main prior to #670; #670 landed the first step) ``` ┌──────────────────────────────────────────────────────────────────────────┐ │ decoding_server (pre-refactor) │ │ compiled-in wire branches, one per transport: │ │ #ifdef udp path #ifdef cpu_roce path gpu_roce fork │ │ udp wrapper calls RendezvousInfo + hsb_fpga GpuRoceTransceiver = │ │ wired directly QP handshakes INLINE hololink bring-up │ │ (byte-exact copy of the FUSED with the QEC │ │ CpuRoceChannel structs) dispatch engine │ │ hand-rolled std::thread around cudaq_host_ring_dispatch_loop │ │ ONE ring buffer + ONE dispatch loop shared by ALL decoders │ │ two transport knobs that had to agree (--transport + per-decoder YAML) │ └──────────────────────────────────────────────────────────────────────────┘ │ link-time: udp/cpu_roce wrappers; DOCA + hololink + HSB + ibverbs ▼ adding a wire = editing this repo; no partner drop-in possible ``` - The server had per-transport code paths: udp and cpu_roce wired directly against transport wrapper headers, and a separate gpu_roce fork that linked hololink/DOCA/HSB/ibverbs into the QEC libraries (`DT_NEEDED` on the server binary). Adding a wire meant editing the server. - Transport selection was two knobs that had to agree (`--transport` on the CLI AND a per-decoder YAML key), with silent misconfiguration when they disagreed. - One ring buffer served all decoders: every RPC funneled through a single dispatcher, and per-decoder endpoints (the topology the product needs: caller routes with `device_id == decoder_id`) were impossible. - The GPU device-graph path (`gpu_roce`) was hololink-only, named after one wire, and could not coexist with host-dispatch decoders in the same process. - The CPU-RoCE rendezvous / HSB-FPGA QP handshake lived in the server itself instead of behind the transport boundary. - #670 (now on main) took the first step -- the gpu_roce engine moved out of the core library behind a weak factory -- but the component still links hololink/DOCA/HSB directly, serves exactly one decoder, is named for one wire, and the per-decoder `transport:` key remains the selection knob. ## After ``` ┌──────────────────────────────────────────────────────────────────────────┐ │ decoding_server (transport-blind) │ │ YAML `dispatch:` picks the ENGINE YAML `transport:` picks the WIRE │ │ (per decoder) (per deployment; --transport is │ │ a fallback, conflict = error) │ │ ONE RING PER DECODER, each with its own consumer: │ │ host ─► dispatcher object over <name> ► libcudaq-realtime- │ │ its provider ring (4869 API) bridge-<name>.so │ │ device_graph ─► DeviceGraphRing- /path.so ► partner library, │ │ Consumer (GPU scheduler) verbatim, zero changes │ │ geometry + READY ring tokens derived FROM the providers (v2 queries) │ └──────────────────────────────────────────────────────────────────────────┘ │ links │ links (weak factory, WHOLE_ARCHIVE) ┌────────────────────┐ ┌────────────────────────────────────────────────┐ │ CQR plugin │ │ DecodingServer core · device-graph component: │ │ (HOST_CALL table) │ │ DeviceGraphTransceiver = dispatch ENGINE only │ └────────────────────┘ └────────────────────────────────────────────────┘ │ links (the ONLY realtime link dependency) ┌──────────────────────────────────────────────────────────────────────────┐ │ libcudaq-realtime.so — bridge loader (iface v2) + dispatcher object │ └──────────────────────────────────────────────────────────────────────────┘ ◌ dlopen ─────────────── runtime plug-in seam ────────────── ◌ dlopen ┌────────────┬──────────────────┬─────────────────┬────────────────────────┐ │ bridge-udp │ bridge-cpu-roce │ bridge-hololink │ partner transport .so │ │ .so │ .so (rendezvous │ .so (built on │ (out of tree, ~9 C │ │ │ + hsb_fpga) │ the HSB rig) │ functions) │ └────────────┴──────────────────┴─────────────────┴────────────────────────┘ ``` - **Bridge-provider-only server.** `decoding_server` speaks only YAML + the `cudaq_bridge_*` C API. Provider libraries resolve by name next to the cudaq-realtime install (`QEC_BRIDGE_PROVIDER_DIR`) or verbatim by path (partner drop-in). The QEC libraries link no transport libraries; the rendezvous/hsb_fpga handshake moved down into the cpu_roce provider (companion cuda-quantum PR). - **Engine vs. wire, each named once.** Per-decoder YAML `dispatch: host|device_graph` picks HOW RPCs execute; the top-level, shape-keyed `transport:` section (`provider`, `args`, plus a `device_graph:` override for the rings that must be GPU-pollable) picks the wire for the whole deployment. `--transport` is a fallback for configs that intentionally leave the wire unspecified (one YAML reused across wires, selected per launch); a YAML that names a provider combined with an explicit `--transport` is a startup error, never a silent precedence decision. All pre-release aliases (`transport:` as a per-decoder key, `--transport=gpu_roce`, `HOLOLINK_*` env fallbacks) are removed -- this component is new this cycle, so there is no compatibility surface to keep. - **One ring per decoder.** The server opens one bridge instance + one ring + one consumer per YAML decoder; ring geometry and endpoint identity come from the provider's v2 queries instead of re-parsed CLI. Readiness publishes every endpoint in one line (`QEC_DECODING_SERVER_READY port=<p0> ... ring<id>=<port>`) before any blocking connect, and shutdown reports per-ring traffic (`QEC_DECODING_SERVER_RING decoder=<id> dispatched=<n>`). Callers route with `cudaq::device_call(decoder_id, ...)` and per-device endpoint args (`udp-port.<id>=<port>`). - **Mixed dispatch in one process.** `DeviceGraphRingConsumer` runs the CUDA-Q device-graph scheduler (self-relaunching GPU dispatch graph + the decoder's captured decode graph) as a ring consumer over any GPU-pollable ring -- hololink DOCA rings on a rig, or pinned+mapped udp rings (`--pinned-rings`) on any CUDA box. `GpuRoceTransceiver` was renamed to what it actually is (`DeviceGraphTransceiver`, the dispatch engine, transport-blind) and now delegates to the ring consumer; the standalone all-device_graph path remains for the HSB flow. - **Device-graph wedge root-caused and fixed.** The scheduler deadlock after the first decode trigger was cooperative-launch co-residency starvation: the decode graph was captured sized for every SM and could never become co-resident with the persistent dispatch graph, so the device-side fire queued forever at `grid.sync()`. `DecodingSession` now captures with `reserved_sms=1` (`QEC_DEVICE_GRAPH_RESERVED_SMS` to raise it on rigs where hololink RX/TX kernels are also resident). The same bug class affects host-path GPU-cooperative decodes when a scheduler is resident -- documented, with the mixed-deployment guidance of a CPU decoder on host rings until the plugin plumbs reservation into its host path. - **Works with the session stats from main.** `QEC_DECODING_SERVER_STATS=1` prints per-decoder counters in the mixed server too; they are host-session counters, so a `device_graph` ring legitimately reports `decodes=0` -- its execution evidence is the trigger diagnostics (`fires == tail_relaunches`) and the per-ring `dispatched=` line. - **Tests and config schema.** New coverage: transport-section parse/round-trip with a mixed host+device_graph decoder set, a two-process decode where the YAML alone names the wire, CLI/YAML conflict rejection, a per-decoder-rings two-process test asserting traffic on each ring, and `app_examples.surface_code-4-yaml-mixed-dispatch` -- the flagship mixed flow as a gated ctest (registered only when the server links the device-graph component; skips without a GPU or the nv-qldpc plugin) asserting scheduler health via the new trigger diagnostics (rc=0, fires == tail_relaunches > 0). The generated JSON config schema (`decoder_config_json_schema()`) advertises the new per-decoder `dispatch:` key and the top-level `transport:` section (the removed per-decoder `transport:` key is gone from the schema too), with a python `jsonschema` test that validates a populated `dispatch`/`transport` document against it and rejects the removed key. The python config bindings expose the new fields as well -- `decoder_config.dispatch`, `multi_decoder_config.transport`, and the `decoder_dispatch` / `transport_config` / `transport_shape_override` types. ## Sample configurations Three representative deployment shapes (YAML config + exact launch line + matching caller config): **Two host decoders, one udp ring per decoder** -- the YAML says nothing about the wire, so `--transport` (default udp) selects it per launch; the READY line publishes every ring's endpoint. ```yaml decoders: - id: 0 type: pymatching # block_size / syndrome_size / H_sparse / ... - id: 1 type: pymatching # ... ``` ``` decoding_server --config=decoders.yaml # QEC_DECODING_SERVER_READY port=<P0> transport=udp ring0=<P0> ring1=<P1> ``` Caller routes per decoder with device-scoped endpoint args: `--cudaq-device-call=udp udp-host=127.0.0.1 udp-port=<P0> udp-port.1=<P1>`. **Mixed dispatch (host CPU decoder + GPU device-graph decoder), runs on any CUDA box** -- the wire lives in the YAML transport section; the shape-keyed override adds `--pinned-rings` only to the device_graph ring so the GPU scheduler can poll it. No `--transport` on the command line (combining both is a startup error). This is exactly what the new gated ctest runs. ```yaml transport: provider: udp device_graph: args: [--pinned-rings] decoders: - id: 0 type: multi_error_lut # ... - id: 1 type: nv-qldpc-decoder dispatch: device_graph # ... ``` ``` decoding_server --config=decoders.yaml ``` **Partner transport drop-in** -- an out-of-tree provider library is named by path, verbatim; its args are forwarded untouched. Zero decoding-server changes. ```yaml transport: provider: /opt/partner/libpartner_bridge.so args: [--lane=3] decoders: - id: 0 type: pymatching # ... ``` ``` decoding_server --config=decoders.yaml ``` ## Validation - Two-process suite 6/6 (udp; includes per-decoder rings and the YAML-transport-section tests), decoding-server core + YAML suites, per-decoder-rings app example. - Re-validated after merging main with #670 landed (#656/#673/#674/#678/#680/#683/#690/#691/#692): two-process suite, DecoderYAMLTest 18/18, per-decoder-rings + mixed-dispatch app tests, python surface_code-1, and the full mixed E2E all green against an nv-qldpc plugin rebuilt for #674's virtualized decoder API (an ABI break for out-of-tree plugin builds). - Full mixed E2E on a WSL2 laptop, two-process, wire named only by the YAML: multi_error_lut on a host ring + nv-qldpc RelayBP behind the GPU device-graph scheduler on a pinned-udp ring -- decode graph fired for every window (trigger rc=0, fires == tail_relaunches), correct corrections, both rings dispatched, clean teardown. The HSB rig is no longer required to exercise the device-graph dispatch path; rig runs remain the validation for the hololink and cpu_roce RDMA data planes. ## Breaking changes vs. main All of these surfaces are new this release cycle, so no deprecation aliases are kept: - The per-decoder `transport:` YAML key and the `DecoderTransport` enum are replaced by per-decoder `dispatch: host|device_graph` plus the top-level `transport:` section. A config using the old key fails to parse loudly. `cuda_device_id` (#690) is kept unchanged and now also places the device_graph rings and scheduler. - The device-graph transceiver's environment surface is `QEC_DEVICE_GRAPH_*` (was `HOLOLINK_*`), and `--transport=gpu_roce` no longer exists: device_graph is a dispatch shape named in the YAML; `--transport` only ever names a wire provider. The HSB test script is updated to match (injects `dispatch: device_graph` + `cuda_device_id`, sets `QEC_DEVICE_GRAPH_*`, waits for `READY device_graph`). - `CpuRoceTransceiver` (the always-throwing placeholder) is removed; cpu_roce is served by its bridge provider. ## Dependencies - **Built on the companion cuda-quantum PR [#4915](NVIDIA/cuda-quantum#4915 ([realtime] pluggable transport providers / bridge interface v2 + per-device device_call sessions), **merged upstream 2026-07-23** (squash `2a7911f`) and itself built on cuda-quantum #4869. `.cudaq_version` pins that squash commit, so this branch builds against stock main. This PR consumes `cudaq_bridge_create_from_library`, the v2 endpoint/geometry queries, per-device sessions and device-scoped channel args, the udp provider's `--pinned-rings`, and `cudaq_dispatch_get_trigger_debug`. - **Built on main with #670 landed** (origin/main is merged into this branch). Relative to landed #670, this PR completes the follow-up its review called for -- removing the server's transport awareness: `GpuRoce{Transceiver,Factory,LinkCheck}` become `DeviceGraph*` (same weak-factory / optional-component structure, now transport-blind), while #670's post-review hardening (ring-size and host-page-alignment validation, `cuda_device_id`-driven GPU placement threaded through the factory) is preserved in the renamed files. - The nv-qldpc decoder plugin and the proprietary cudevice archive are consumed as prebuilt binaries (`CUDAQ_QEC_REALTIME_CUDEVICE_PROPRIETARY_ARCHIVE`); builds without them still compile and the device-graph paths fail at runtime with a clear not-linked error, and the gated ctest is simply not registered. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Ben Howe <bhowe@nvidia.com> Signed-off-by: Chuck Ketcham <cketcham@nvidia.com> Co-authored-by: Chuck Ketcham <cketcham@nvidia.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

boschmitt and others added 16 commits July 9, 2026 10:52

Merge NVIDIA origin/main into PR 4869 head (baseline for bridge-provi…

c5df403

…der work) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Apply clang-format

171fb7a

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Ben Howe <bhowe@nvidia.com>

bmhowe23 mentioned this pull request Jul 12, 2026

QEC decoding server: transport-agnostic bridge providers, one ring per decoder, mixed host + device_graph dispatch NVIDIA/cudaqx#682

Merged

Merge commit '5f0ee5be3bf5cb33afb544d9f01af77b1d50b26e' into bmh/real…

4d5777e

…time-bridge-providers

cketcham2333 reviewed Jul 20, 2026

View reviewed changes

Comment thread realtime/lib/daemon/bridge/hololink/bridge_impl.cpp

Merge remote-tracking branch 'origin/main' into bmh/realtime-bridge-p…

2694704

…roviders Signed-off-by: Ben Howe <bhowe@nvidia.com> # Conflicts: # realtime/include/cudaq/realtime/daemon/bridge/bridge_interface.h

bmhowe23 marked this pull request as ready for review July 22, 2026 22:44

bmhowe23 requested review from 1tnguyen and Renaud-K July 22, 2026 22:44

bmhowe23 changed the title ~~DRAFT: [realtime] Pluggable transport providers + per-device device_call sessions~~ [realtime] Pluggable transport providers + per-device device_call sessions Jul 22, 2026

Fixed spelling errors

e2f7faa

Signed-off-by: Renaud Kauffmann <rkauffmann@nvidia.com>

Renaud-K requested review from bettinaheim and mitchdz as code owners July 22, 2026 23:42

Renaud-K approved these changes Jul 23, 2026

View reviewed changes

Merge branch 'main' into bmh/realtime-bridge-providers

941bc91

bmhowe23 merged commit 2a7911f into NVIDIA:main Jul 23, 2026
86 checks passed

github-actions Bot pushed a commit that referenced this pull request Jul 23, 2026

Cleaning up docs preview for PR #4915.

22424b5

Navigation Menu

Uh oh!

[realtime] Pluggable transport providers + per-device device_call sessions - #4915

[realtime] Pluggable transport providers + per-device device_call sessions#4915
bmhowe23 merged 20 commits into
NVIDIA:mainfrom
bmhowe23:bmh/realtime-bridge-providers

bmhowe23 commented Jul 12, 2026 •

edited

Loading

Uh oh!

copy-pr-bot Bot commented Jul 12, 2026

Uh oh!

bmhowe23 commented Jul 12, 2026 •

edited by github-actions Bot

Loading

Uh oh!

github-actions Bot commented Jul 12, 2026 •

edited

Loading

Uh oh!

bmhowe23 commented Jul 15, 2026 •

edited by github-actions Bot

Loading

Uh oh!

Uh oh!

bmhowe23 commented Jul 22, 2026 •

edited by github-actions Bot

Loading

Uh oh!

Renaud-K commented Jul 23, 2026 •

edited by github-actions Bot

Loading

Uh oh!

Renaud-K left a comment

Uh oh!

bmhowe23 commented Jul 23, 2026 •

edited by github-actions Bot

Loading

Uh oh!

bmhowe23 commented Jul 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

bmhowe23 commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Before

After

Breaking changes

Dependencies

Validation

Uh oh!

copy-pr-bot Bot commented Jul 12, 2026

Uh oh!

bmhowe23 commented Jul 12, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI Summary (push) — ✅ passed

Uh oh!

bmhowe23 commented Jul 15, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

bmhowe23 commented Jul 22, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Renaud-K commented Jul 23, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Renaud-K left a comment

Choose a reason for hiding this comment

Uh oh!

bmhowe23 commented Jul 23, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bmhowe23 commented Jul 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

bmhowe23 commented Jul 12, 2026 •

edited

Loading

bmhowe23 commented Jul 12, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Jul 12, 2026 •

edited

Loading

CI Summary (`push`) — ✅ passed

bmhowe23 commented Jul 15, 2026 •

edited by github-actions Bot

Loading

bmhowe23 commented Jul 22, 2026 •

edited by github-actions Bot

Loading

Renaud-K commented Jul 23, 2026 •

edited by github-actions Bot

Loading

bmhowe23 commented Jul 23, 2026 •

edited by github-actions Bot

Loading