Add: queue consecutive runs on shared streams

docs/dfx/device-phases.md

@@ -188,7 +188,8 @@ threads, no per-task AICore records, works in `SIMPLER_DFX=0`. See
   `ChipTaskSlotState` in the `TaskAttrs` byte (bit 3 `is_timed` + bits 4-7 the
   0..15 tag), co-located with the other per-task scheduling flags. The 16 slots
   are a fixed `TaskTimingRecord[16]` **tail** appended after the `AicpuPhaseRecord`
-  region in the same device buffer — same base pointer and per-run H2D reset.
+  region in the same per-slot device buffer — one base per in-flight slot and a
+  per-run H2D reset.
   A 16-byte header at the front lets the host skip the tail D2H entirely when
   no task was tagged (saving 1536 bytes on a2a3 and 3584 bytes on a5 per run).
   The tail is a distinct record type (dispatch/finish, not start/end) reduced by

docs/dfx/host-trace.md

@@ -152,7 +152,7 @@ children are TMR-only; HBG orchestration runs on the host and stamps none of
 those phases. All emitted device spans are tagged `clk=dev`. They are not host
 `steady_clock` spans: the AICPU stamps raw sys-counter cycles into a host-allocated buffer
 (whose address rides on `KernelArgs::device_wall_data_base`), the host reads it
-back after stream-sync, converts cycles → ns, and emits the marker. `orch`/
+back after that run's completion event, converts cycles → ns, and emits the marker. `orch`/
 `sched` are the orchestrator/scheduler windows that formerly only appeared as
 device-log lines. A phase that was never stamped
 (0 ns) is skipped — e.g. `so_load` is ~0 on a cached-callable run. See
@@ -161,7 +161,7 @@ device-log lines. A phase that was never stamped
 The phased native-run interface preserves this same marker contract. Prepare
 allocates one `inv` and records the host-wall start; prepare, the child progress
 path's launch/drain lifecycle, and finalize bind that `(inv, hid)` while
-emitting their spans. Finalize releases the runner claim, destroys the per-run
+emitting their spans. Finalize releases the run's submitted-owner slot, destroys the per-run
 state, and then emits the stored `chip.run` wall, so the root includes that
 cleanup tail.
 No trace scope or synthetic nesting remains active between C API calls. For
@@ -343,13 +343,13 @@ tree, and the root span already carries the identity that tells two runs apart.
 | -------- | --------- |
 | successor's preparation | `chip.run.bind` — its arena build + host orchestration |
 | predecessor's device work | `chip.run.runner_run` |
-| when a successor may launch | `chip.run.claim_release` |
+| successor queued launch | overlapping `chip.run.runner_run` spans |
 | which run each belongs to | root `chip.run` attrs, joined by `(pid, inv)` |
 
-Only `claim_release` was added for this: it wraps `release_native_run` inside
-finalize, the point a successor's launch becomes admissible, and no other span
-marks that boundary. `node.post_fence_retirement` covers the L3 orchestrator's
-`release_run` tail for the same reason.
+`claim_release` wraps `release_native_run` inside finalize and records when one
+submitted-owner slot is returned. It is not a launch barrier on a backend that
+queues two runs. `node.post_fence_retirement` covers the L3 orchestrator's
+`release_run` tail.
 
 The identity is `run_id / dispatch_id / run_epoch / slot_id / generation`. Each
 field means one thing: `run_id` and `dispatch_id` are zero on the direct-chip
@@ -363,9 +363,10 @@ python -m simpler_setup.tools.strace_timing path/to/log --assert-native-overlap
 ```
 
 Per adjacent run on one child process, the command requires `bind(N+1)` to
-**overlap** `runner_run(N)` — the intervals intersect — and `runner_run(N+1)` not
-to start before `claim_release(N)`. It exits nonzero on a missing identity, a
-missing span, or an ordering violation.
+**overlap** `runner_run(N)` and `runner_run(N+1)` to begin before
+`runner_run(N)` ends. The latter proves the successor was submitted before the
+predecessor completion fence. It exits nonzero on a missing identity, a missing
+span, or an ordering violation.
 
 Reading `bind` rather than the whole prepare is deliberate: `bind` sits inside
 prepare, so an overlap it reports is one the prepare certainly had.

docs/dynamic-linking.md

@@ -302,7 +302,8 @@ ChipWorker.init(device_id, bins)                       # Python wrapper
            simpler_prepare_run, simpler_launch_run, simpler_poll_run,
            simpler_wait_run, simpler_finalize_run, simpler_run,
            simpler_unregister_callable, get_pipeline_contract,
-           supports_concurrent_native_prepare_ctx,
+           supports_concurrent_native_prepare_ctx, supports_queued_native_launch_ctx,
+           native_run_error_poisons_ctx,
            get_arena_bank_gm_heap_base_ctx, get_retained_temp_addr_ctx,
            finalize_device
     create_device_context() → DeviceContextHandle
@@ -388,9 +389,9 @@ device_worker_main(device_id)
                   launch_aicpu_kernel(Run)     rtsLaunchCpuKernel, cached rtFuncHandle
                   publish acceptance from the completed launch receipt
               simpler_poll_run(...)            nonblocking child progress query
-                DeviceRunner::poll_execution(active) nonblocking stream query
+                DeviceRunner::poll_execution(active) query this run's event pair
               simpler_wait_run(...)
-                DeviceRunner::drain_execution(active) wait on both streams
+                DeviceRunner::drain_execution(active) wait on this run's event pair
               simpler_finalize_run(...)        rtMemcpy results back; destroy state
 
     ChipWorker.finalize()

docs/task-flow.md

@@ -296,9 +296,10 @@ layer's.
 
 Run streams are outside that lease, and there is one pair of them per runner
 rather than one per slot. A slot indexes the resources *preparation* mutates,
-and preparing a run writes nothing to a stream: only launch submits, and launch
-holds the exclusive execution claim, so runs reach the device one at a time and
-the stream orders them. The two streams stay distinct because the AICPU Run
+and preparing a run writes nothing to a stream. On a2a3 onboard, two successful
+launches may enqueue consecutive runs on that pair; per-slot completion events
+identify each run's tail while the stream orders their device work. The two
+streams stay distinct because the AICPU Run
 kernel spins in the handshake waiting for the AICore workers — one queue would
 leave the AICore submission behind a spin that never ends.
 
@@ -307,15 +308,15 @@ content-hash deduplicated GM allocations: simultaneously resident code images
 occupy different allocations, while unregister frees an allocation that a later
 registration may reuse. A dedup miss is the only repeatable path that publishes
 new AICore instruction bytes; after that H2D copy succeeds the pair is marked
-stale, and the next launch destroys the AICore stream and creates a replacement.
+stale, and the next launch after all outstanding runs retire destroys the AICore
+stream and creates a replacement.
 (The `kernel_entry` ELF that `rtRegisterAllKernel` publishes needs no such mark:
 CANN offers no unregister, so it is registered once per runner and released with
 the streams at finalize.) Without a new publication the pair stays warm even
-when two resident callables alternate. Unproven completion still destroys the
-AICore stream conservatively, and only the run that submitted the pair may
-retire it — a prepared successor overlaps its predecessor's execution and must
-leave the live pair alone. The AICPU stream carries no instruction cache state
-and lives for the runner.
+when two resident callables alternate. Proven owners retire in submission order.
+An unproven completion prevents reuse and defers AICore stream destruction until
+the submitted queue is empty. The AICPU stream carries no instruction cache
+state and lives for the runner.
 
 #### Whole-run FIFO admission
 
@@ -414,10 +415,13 @@ Backend and per-run capabilities, rather than the mailbox protocol, select
 between these meanings.
 
 An HBG successor's prepared token remains unlaunched and unaccepted until
-`ACTIVATE`, and activation still cannot launch it until the predecessor has
-polled complete and finalized. The sticky acceptance word therefore remains
-zero throughout preparation. Shutdown, stale activation, and pre-launch
-failure finalize the token exactly once before the frame becomes terminal.
+`ACTIVATE`. On a queued-launch-capable a2a3 backend, activation may submit it
+behind the predecessor before that predecessor finalizes; per-slot events keep
+completion and finalization FIFO. Diagnostic, incompatible, and unsupported
+paths still wait for predecessor finalization. The sticky acceptance word
+therefore remains zero throughout preparation and changes only after launch.
+Shutdown, stale activation, and pre-launch failure finalize the token exactly
+once before the frame becomes terminal.
 
 The scheduler stages only the first eligible single NEXT_LEVEL task from the
 prepared FIFO successor. Tasks from the active run use only the active lane, so
@@ -757,7 +761,7 @@ Step-by-step (one chip worker):
 | 4 | Scheduler thread | pop `slot` from worker 0's FIFO; resolve stable worker ID 0 to WT_chip_0; dispatch |
 | 5 | WT_chip_0 parent side | encode one leased task frame: write `config`, digest prefix, and the args blob; publish `TASK_READY` for the active lane or `PREPARE_READY` for a staged successor |
 | 6 | chip_0 child process | validate the frame and resolve its digest; ordinary HBG with an active predecessor also prepares the leased inactive arena bank before publishing `FRAME_STAGED`, while a frame with no active predecessor, diagnostic HBG, and TMR publish after validation and defer native prepare |
-| 7 | chip_0 native-run path | after activation and the predecessor's finalization fence, launch an already-prepared HBG run or finish deferred native preparation and then launch; poll it to completion and finalize it before another staged frame may launch. Compatibility endpoints perform the equivalent operation through blocking `ChipWorker::run` |
+| 7 | chip_0 native-run path | after activation, a capable a2a3 backend launches the prepared successor onto the predecessor's stream pair; per-slot events preserve FIFO completion and finalization. Diagnostic and incompatible paths wait for predecessor finalization before preparing or launching. Compatibility endpoints perform the equivalent operation through blocking `ChipWorker::run` |
 | 8 | runtime.so | translate host ptrs → device ptrs; dispatch AICPU / AICore; write output into `c`'s shm |
 | 9 | chip_0 child | native finalization returns; write `TASK_DONE` |
 | 10 | WT_chip_0 parent | observe `TASK_DONE`; push success completion |

docs/worker-manager.md

@@ -251,10 +251,11 @@ the same two-frame protocol but defer native prepare because their shared
 diagnostic or device-scratch state cannot be rewritten while another run is
 active.
 
-An HBG token remains unlaunched and unaccepted until activation, and no backend
-launches a successor until the predecessor is polled and finalized. Shutdown,
-stale activation, and pre-launch failure finalize any unlaunched token exactly
-once.
+An HBG token remains unlaunched and unaccepted until activation. A backend with
+queued-launch support may then submit it behind its predecessor; completion and
+finalization remain FIFO. Other backends and diagnostic runs wait for the
+predecessor to finalize. Shutdown, stale activation, and pre-launch failure
+finalize every token exactly once.
 
 Activation is sticky on the parent side: FIFO promotion may be observed before
 the child reaches `FRAME_STAGED`. The endpoint records that permission and

python/bindings/task_interface.cpp

@@ -3088,6 +3088,10 @@ NB_MODULE(_task_interface, m) {
             "supports_concurrent_native_prepare", &ChipWorker::supports_concurrent_native_prepare,
             "Whether non-diagnostic native preparation may overlap one active run in another slot."
         )
+        .def_prop_ro(
+            "supports_queued_native_launch", &ChipWorker::supports_queued_native_launch,
+            "Whether two non-diagnostic native runs may be queued on the device streams."
+        )
         .def_prop_ro(
             "runtime_buffer_addrs", &ChipWorker::runtime_buffer_addrs,
             "Host Runtime staging buffer address of every copy the runtime's "

simpler_setup/tools/strace_timing.py

@@ -384,12 +384,10 @@ def bucket_by_hid(invocations):
     return buckets
 
 
-# The spans one native run contributes to the overlap proof. All three already
-# exist in the `chip.run` tree; only `claim_release` was added for it.
+# The spans one native run contributes to the overlap proof.
 _PREPARE_SPAN = "chip.run.bind"
 _DEVICE_SPAN = "chip.run.runner_run"
-_RELEASE_SPAN = "chip.run.claim_release"
-_NATIVE_REQUIRED_SPANS = (_PREPARE_SPAN, _DEVICE_SPAN, _RELEASE_SPAN)
+_NATIVE_REQUIRED_SPANS = (_PREPARE_SPAN, _DEVICE_SPAN)
 _PIPELINE_IDENTITY_FIELDS = ("run_id", "dispatch_id", "run_epoch", "slot_id", "generation")
 
 
@@ -441,7 +439,8 @@ def assert_native_overlap(spans, *, require_hidden=False):
     * ``bind(N+1)`` overlaps ``runner_run(N)`` — the intervals intersect, which
       is what makes the successor's preparation concurrent with the
       predecessor's device work.
-    * ``runner_run(N+1)`` does not start before ``claim_release(N)``.
+    * ``runner_run(N+1)`` starts before ``runner_run(N)`` ends, proving the
+      successor was submitted before the predecessor's completion fence.
 
     ``bind`` is the successor's own arena build and host orchestration and sits
     inside its prepare, so reading it is conservative: an overlap it reports is
@@ -470,7 +469,6 @@ def assert_native_overlap(spans, *, require_hidden=False):
                     f"launch order is not monotonic: predecessor={predecessor.sequence} successor={successor.sequence}"
                 )
             pred_device = pred_spans[_DEVICE_SPAN]
-            pred_release = pred_spans[_RELEASE_SPAN]
             succ_prepare = succ_spans[_PREPARE_SPAN]
             succ_device = succ_spans[_DEVICE_SPAN]
             pred_device_end = pred_device.ts + pred_device.dur
@@ -486,10 +484,10 @@ def assert_native_overlap(spans, *, require_hidden=False):
                     f"preparation was not fully hidden: sequence={successor.sequence} prepare_end="
                     f"{succ_prepare_end} predecessor_device_end={pred_device_end}"
                 )
-            if succ_device.ts < pred_release.ts:
+            if succ_device.ts >= pred_device_end:
                 raise NativeOverlapError(
-                    f"device execution reordered before the claim release: sequence={successor.sequence} "
-                    f"device_start={succ_device.ts} predecessor_release={pred_release.ts}"
+                    f"successor was not queued before predecessor completion: sequence={successor.sequence} "
+                    f"successor_start={succ_device.ts} predecessor_end={pred_device_end}"
                 )
             checks.append(NativeOverlapCheck(predecessor=predecessor, successor=successor))
     if not checks: