diff --git a/docs/tensormap-and-ringbuffer-a2a3-vs-a5.md b/docs/tensormap-and-ringbuffer-a2a3-vs-a5.md index 53012416c9..db11484b7f 100644 --- a/docs/tensormap-and-ringbuffer-a2a3-vs-a5.md +++ b/docs/tensormap-and-ringbuffer-a2a3-vs-a5.md @@ -4,7 +4,7 @@ This document describes the substantive differences in the current code under `src/{a2a3,a5}/runtime/tensormap_and_ringbuffer/`. > **Maintenance baseline:** The source layout and classifications were verified -> on 2026-08-17. Recompute the counts and update the affected sections whenever +> on 2026-09-03. Recompute the counts and update the affected sections whenever > the files or constants described here change. ## Comparison Boundary and Classification @@ -112,6 +112,7 @@ The functional differences group into the following themes: | URMA completion | A5-specific implementation and product capability gate | Yes, for now | Retain the A5 path; do not claim that URMA is available in the default build | | Next-block prefetch | A2/A3-only performance optimization | No | Retain on A2/A3; validate on A5 before considering a port | | Scheduler progress publication | AICPU topology and measured publication cost | No | Retain A5's 16-task batching; keep per-advance publication on A2/A3, where the portable implementation showed no significant benefit | +| Terminal task release | Measured end-of-run scheduler cost | No | A5 traces show per-task release blocking the tail after task submission has ended, so successful A5 runs elide deferred release after the graph seal; retain incremental release on A2/A3 because no tail release blocking was found there | | Fatal teardown | Software reliability strategy | No | Retain the current implementations; decide whether to converge after measuring the worst-case A5 teardown time | | Scheduler trace attribution | Software diagnostic strategy | No | Preserve the current traces; converge only after comparing generated timelines | @@ -293,6 +294,89 @@ measurements instead showed lower Effective time in all eight workloads, with an unweighted mean reduction of `2.81%`. Full A2/A3 measurements are recorded in the [PR benchmark follow-up](https://github.com/hw-native-sys/simpler/pull/1575#issuecomment-5310909143). +### Terminal Task Release: A5 Seal and Elision + +Task completion and task release are separate scheduler operations. Completion +records the finished AICore work and unlocks dependent tasks. Release later +drops the completed task's retained references, advances the ring's reclaim +head across consumed slots, resets reusable slot state, and publishes reclaim +progress. Both platforms defer this release work in a per-scheduler array with +a capacity of 256 entries. + +A2/A3 preserves the incremental protocol for the whole run. It drains the +array when it becomes full, during idle cleanup, and when dispatch exits. Every +completed task therefore reaches `on_task_release()` before the scheduler +returns. + +A5 follows the same protocol while the Orchestrator can still submit tasks. +After `orchestrator_done_` seals the graph, however, no new task can require a +reclaimed ring slot. At the next existing full-array, idle-drain, or exit-drain +boundary, A5 discards the deferred-release backlog instead of calling +`on_task_release()` once per entry. Shared helper +`drain_or_elide_deferred_releases` owns that decision. The seal is deliberately +not loaded on every scheduler-loop iteration or every completion: completion +and dependency unlocking remain unchanged, and the added acquire loads stay on +boundaries that already perform release bookkeeping. At a sealed capacity +boundary, the overflowing completed slot is also not deferred. + +The seal is stored only after a clean orchestration exit (`orch_error_code == +NONE`). Failed orchestration never sets `orchestrator_done_`, so deferred +release stays exact until emergency teardown. A scheduler or async failure +after a successful seal leaves `orchestrator_done_` true: later idle/exit +drains still elide, and the next-run SM reset closes the lifecycle. + +Skipped incremental release does not need a terminal barrier or bulk slot +closure. The next run clears the entire SM with `memset` and rebuilds flow +control via `init_per_ring` → `fc.init()`, and the orchestrator already +self-cleans each reused slot on submit. There is no functional downstream +reader of successful-exit `CONSUMED` watermarks between runs. An earlier +barrier / `TerminalClose` layer was removed as redundant. + +This optimization is independent of the A5 K=16 progress-publication policy in +the preceding section. K=16 controls how often an already-advanced reclaim +head is copied to shared memory during the run. Terminal release elision avoids +the per-task reference-count and ring-advance work itself after graph sealing; +its deferred-release array still has capacity 256 and does not impose a +16-task release limit. + +| Stage | A2/A3 | A5 | +| ----- | ----- | -- | +| Before graph sealing | Complete tasks, defer release, then incrementally call `on_task_release()` | Same | +| After graph sealing | Continue incremental release | Drop deferred release work at existing release boundaries | +| Successful Scheduler exit | Drain every remaining deferred entry | Drop remaining deferred entries; next run resets SM | +| Orchestration failure (never sealed) | Emergency teardown after the existing error checks | Emergency teardown; `orchestrator_done_` stays false, so deferred release stays exact until teardown | +| Scheduler / async failure after a successful seal | Emergency teardown | `orchestrator_done_` remains true, so exit/idle drains still elide; SM reset on the next run closes the lifecycle | +| Profiling | Release phases | Release phases only when a real drain runs (no `terminal_close`) | + +| File | A5-only terminal-release role | +| ---- | ----------------------------- | +| `runtime/async_wait.h`, `runtime/scheduler/scheduler_completion.cpp` | Sync completion reads the graph seal at deferred-release capacity boundaries; async capacity drains keep exact release | +| `runtime/scheduler/scheduler_dispatch.cpp` | Elide sealed idle/exit backlog drains via shared `drain_or_elide_deferred_releases`; skip DFX `release` when elided | +| `runtime/scheduler/scheduler.h` | Define the shared drain-or-elide helper used by deferred-release sites | +| `runtime/scheduler/scheduler_cold_path.cpp` | Seal `orchestrator_done_` only after a clean orchestration exit | + +The post-removal A/B was run only on the local A5 system against current +`main`. The seven non-Qwen workloads improved by `9.753%` in Effective +geometric mean; Qwen3 changed by `-0.010%`, and no workload regressed by 5% or +more in Effective time. The largest Scheduler gains remain in the +paged-attention-unroll family (`10.550%` to `14.021%` Effective). Full tables +are recorded under the [PR #2070](https://github.com/hw-native-sys/simpler/pull/2070) +benchmark discussion / local `outputs/pr2070_full_bench/` artifacts. + +The platform scope follows the observed bottleneck. On A5, the motivating +timelines contain a visible tail after the Orchestrator has finished submitting +tasks: Schedulers continue executing per-task release work even though no new +task can consume the reclaimed capacity. That release interval extends the +execution critical path, which gives seal-and-elide a direct optimization +target. No tail release blocking was found on A2/A3. Its release work did not +appear as the corresponding post-orchestration critical-path interval, so there +is currently no performance evidence that A2/A3 would benefit from the extra +graph-seal observation. A2/A3 therefore keeps the simpler incremental release +protocol, and this experiment does not run an A2/A3 benchmark or port the +implementation there. This is an evidence-based software decision rather than +an A5 hardware requirement; revisit it if a future A2/A3 timeline exposes the +same tail release blocking. + ### Fatal Teardown The A2/A3 scheduler uses a dedicated fatal latch to elect an owner, broadcasts diff --git a/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler.h b/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler.h index ced25fe2bf..cc38b0c12e 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler.h +++ b/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler.h @@ -1314,6 +1314,29 @@ struct SchedulerState { // Scheduler cold-path API is declared as SchedulerState member functions. // See init()/destroy()/print_stats()/print_queues() below the struct definition. +// Drop deferred releases when release_elided; otherwise drain via on_task_release. +// Callers set release_elided from orchestrator_done_ only at existing release +// boundaries (full buffer / idle / exit); the async path always passes false. +inline void drain_or_elide_deferred_releases( + SchedulerState *sched, ChipTaskSlotState **slots, int32_t &count, bool release_elided +#if SIMPLER_SCHED_PROFILING + , + int32_t thread_idx +#endif +) { + if (release_elided) { + count = 0; + return; + } + while (count > 0) { +#if SIMPLER_SCHED_PROFILING + (void)sched->on_task_release(*slots[--count], thread_idx); +#else + sched->on_task_release(*slots[--count]); +#endif + } +} + // Short-circuit NotDeferred completions seen during drain so they don't grow // entries[]. Mirrors the a2a3 impl; see that mirror for the rationale. inline bool @@ -1323,16 +1346,15 @@ AsyncWaitList::try_inline_complete_locked(AsyncWaitList::DrainCompletionSink &si #else sink.sched->on_task_complete(slot_state); #endif + // Async path keeps exact deferred release (no graph-seal observation here). if (*sink.deferred_release_count >= sink.deferred_release_capacity) { - while (*sink.deferred_release_count > 0) { + drain_or_elide_deferred_releases( + sink.sched, sink.deferred_release_slot_states, *sink.deferred_release_count, /*release_elided=*/false #if SIMPLER_SCHED_PROFILING - (void)sink.sched->on_task_release( - *sink.deferred_release_slot_states[--(*sink.deferred_release_count)], sink.thread_idx - ); -#else - sink.sched->on_task_release(*sink.deferred_release_slot_states[--(*sink.deferred_release_count)]); + , + sink.thread_idx #endif - } + ); } sink.deferred_release_slot_states[(*sink.deferred_release_count)++] = &slot_state; sink.inline_completed++; @@ -1395,13 +1417,13 @@ inline AsyncPollResult AsyncWaitList::poll_and_complete( sched->on_task_complete(*entry.slot_state); #endif if (deferred_release_count >= deferred_release_capacity) { - while (deferred_release_count > 0) { + drain_or_elide_deferred_releases( + sched, deferred_release_slot_states, deferred_release_count, /*release_elided=*/false #if SIMPLER_SCHED_PROFILING - (void)sched->on_task_release(*deferred_release_slot_states[--deferred_release_count], thread_idx); -#else - sched->on_task_release(*deferred_release_slot_states[--deferred_release_count]); + , + thread_idx #endif - } + ); } deferred_release_slot_states[deferred_release_count++] = entry.slot_state; result.completed++; diff --git a/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_cold_path.cpp b/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_cold_path.cpp index 76991f2af0..f3ed3514fb 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_cold_path.cpp +++ b/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_cold_path.cpp @@ -10,8 +10,10 @@ */ #include "scheduler_context.h" +#include #include #include +#include #include "common/unified_log.h" #include "aicpu/aicpu_device_config.h" @@ -1379,9 +1381,10 @@ void SchedulerContext::on_orchestration_done( rt->scheduler.tasks_completed.fetch_add(inline_completed, std::memory_order_relaxed); #endif } - orchestrator_done_.store(true, std::memory_order_release); - // Check for fatal error from orchestration; if so, shut down immediately. + // Seal only on a clean orchestration exit so deferred-release elision never + // discards lifecycle work after an orch error. Failed runs keep the + // unsealed exact-release path until emergency teardown. int32_t orch_err = 0; if (sched_->sm_header) { orch_err = sched_->sm_header->orch_error_code.load(std::memory_order_relaxed); @@ -1390,6 +1393,8 @@ void SchedulerContext::on_orchestration_done( if (!completed_.exchange(true, std::memory_order_acq_rel)) { emergency_shutdown(runtime); } + } else { + orchestrator_done_.store(true, std::memory_order_release); } #if SIMPLER_DFX diff --git a/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_completion.cpp b/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_completion.cpp index c834989455..ec087fab61 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_completion.cpp +++ b/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_completion.cpp @@ -199,16 +199,20 @@ void SchedulerContext::complete_slot_task( } chip_swimlane.phase_complete_count++; #endif - if (deferred_release_count < DEFERRED_RELEASE_CAP) { - deferred_release_slot_states[deferred_release_count++] = &slot_state; - } else { - while (deferred_release_count > 0) { + // At capacity, elide deferred releases (including the overflowing slot) + // when orchestration is done; otherwise drain then push. + bool release_elided = false; + if (deferred_release_count >= DEFERRED_RELEASE_CAP) { + release_elided = orchestrator_done_.load(std::memory_order_acquire); + drain_or_elide_deferred_releases( + sched_, deferred_release_slot_states, deferred_release_count, release_elided #if SIMPLER_SCHED_PROFILING - (void)sched_->on_task_release(*deferred_release_slot_states[--deferred_release_count], thread_idx); -#else - sched_->on_task_release(*deferred_release_slot_states[--deferred_release_count]); + , + thread_idx #endif - } + ); + } + if (!release_elided) { deferred_release_slot_states[deferred_release_count++] = &slot_state; } completed_this_turn++; diff --git a/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp b/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp index 8b22429eea..f2f374440a 100644 --- a/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp +++ b/src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cpp @@ -1174,19 +1174,18 @@ int32_t SchedulerContext::resolve_and_dispatch(Runtime *runtime, int32_t thread_ } #endif // Dummy tasks have no subtasks to retire and no fanout pre-conditions - // beyond their own producers; release self-reference so the slot can - // reach CONSUMED once all consumers drain. + // beyond their own producers. While lifecycle reclamation is active, + // release their self-reference so the slot can reach CONSUMED. deferred_release_slot_states[deferred_release_count++] = &dummy_slot; if (deferred_release_count >= DEFERRED_RELEASE_CAP) { - while (deferred_release_count > 0) { + bool release_elided = orchestrator_done_.load(std::memory_order_acquire); + drain_or_elide_deferred_releases( + sched_, deferred_release_slot_states, deferred_release_count, release_elided #if SIMPLER_SCHED_PROFILING - (void)sched_->on_task_release( - *deferred_release_slot_states[--deferred_release_count], thread_idx - ); -#else - sched_->on_task_release(*deferred_release_slot_states[--deferred_release_count]); + , + thread_idx #endif - } + ); } int32_t prev = completed_tasks_.fetch_add(1, std::memory_order_relaxed); last_progress_count = prev + 1; @@ -1298,15 +1297,18 @@ int32_t SchedulerContext::resolve_and_dispatch(Runtime *runtime, int32_t thread_ 0; uint32_t released_count = static_cast(deferred_release_count); #endif - while (deferred_release_count > 0) { + // Elide remaining deferred releases when orchestration is done; + // otherwise drain via on_task_release. + bool release_elided = deferred_release_count > 0 && orchestrator_done_.load(std::memory_order_acquire); + drain_or_elide_deferred_releases( + sched_, deferred_release_slot_states, deferred_release_count, release_elided #if SIMPLER_SCHED_PROFILING - (void)sched_->on_task_release(*deferred_release_slot_states[--deferred_release_count], thread_idx); -#else - sched_->on_task_release(*deferred_release_slot_states[--deferred_release_count]); + , + thread_idx #endif - } + ); #if SIMPLER_DFX - if (release_t0 != 0) { + if (release_t0 != 0 && !release_elided) { chip_swimlane_aicpu_record_sched_phase( thread_idx, ChipSwimlaneSchedPhaseKind::Release, release_t0, get_sys_cnt_aicpu(), chip_swimlane.sched_loop_count, released_count @@ -1384,18 +1386,17 @@ int32_t SchedulerContext::resolve_and_dispatch(Runtime *runtime, int32_t thread_ } } - // Drain any entries left in the deferred-release batch. The in-loop flush - // only fires on idle iterations and on buffer-full; a loop exit while the - // last iteration made progress can leave entries un-released. Drop them - // here so every consumed producer slot completes its on_task_release - // regardless of which loop-exit path fired. - while (deferred_release_count > 0) { + // Exact release on every exit while orchestration is still running. + // After orchestrator_done_, drop remaining deferred releases — the next + // run memset+reinit clears SM, and reused slots self-clean on submit. + bool release_elided = deferred_release_count > 0 && orchestrator_done_.load(std::memory_order_acquire); + drain_or_elide_deferred_releases( + sched_, deferred_release_slot_states, deferred_release_count, release_elided #if SIMPLER_SCHED_PROFILING - (void)sched_->on_task_release(*deferred_release_slot_states[--deferred_release_count], thread_idx); -#else - sched_->on_task_release(*deferred_release_slot_states[--deferred_release_count]); + , + thread_idx #endif - } + ); #if SIMPLER_DFX // Final-drain: emit any pop_hit / pop_miss accrued since the last diff --git a/tests/st/a5/tensormap_and_ringbuffer/dfx/chip_swimlane/kernels/orchestration/many_adds_orch.cpp b/tests/st/a5/tensormap_and_ringbuffer/dfx/chip_swimlane/kernels/orchestration/many_adds_orch.cpp new file mode 100644 index 0000000000..3497fa142d --- /dev/null +++ b/tests/st/a5/tensormap_and_ringbuffer/dfx/chip_swimlane/kernels/orchestration/many_adds_orch.cpp @@ -0,0 +1,117 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ +/** + * Pre-seal short single-block adds (orch-side wait → idle exact drain → `release` + * phase), then a burst of long multi-block SPMD tasks submitted without waiting + * so orchestration can seal while they still run and their deferred releases + * are elided. + */ +#include + +#include "orchestration_api.h" // NOLINT(build/include_subdir) + +#define FUNC_ADD 0 +#define FUNC_SPMD_SLOW 1 + +constexpr int32_t kShortAddTasks = 16; +constexpr int32_t kLongSpmdTasks = 4; +constexpr int16_t kSpmdBlocks = 8; +// Long enough to outlast orch exit+seal on a5sim; far shorter than the 1s blocker. +constexpr int64_t kSpmdSpinIters = 200000; +constexpr int32_t kFloatsPerCacheLine = 16; + +static TaskId submit_short_add( + const simpler::tmr::Tensor &ext_a, const simpler::tmr::Tensor &ext_b, const TensorCreateInfo &inter_ci, + const simpler::tmr::Tensor *ext_out +) { + CoreTaskArgs params; + params.add_input(ext_a); + params.add_input(ext_b); + if (ext_out != nullptr) { + params.add_output(*ext_out); + } else { + params.add_output(inter_ci); + } + return rt_submit_aiv_task(FUNC_ADD, params).task_id(); +} + +// Wait until every listed producer has completed, then force one more full +// scheduler loop with only dummy work so an idle drain (exact release) runs +// before orchestration seals. +static void wait_preseal_idle_release(const TaskId *producers, int32_t producer_count) { + CoreTaskArgs completion_args; + uint32_t fence_shape[1] = {1}; + TensorCreateInfo fence_info(fence_shape, 1, DataType::INT32); + completion_args.add_output(fence_info); + completion_args.set_dependencies(producers, static_cast(producer_count)); + TaskOutputTensors completion_out = rt_submit_dummy_task(completion_args); + uint32_t index[1] = {0}; + (void)get_tensor_data(completion_out.get_ref(0), 1, index); + + CoreTaskArgs first_args; + TaskId first = rt_submit_dummy_task(first_args).task_id(); + + CoreTaskArgs second_args; + second_args.add_output(fence_info); + TaskId deps[1] = {first}; + second_args.set_dependencies(deps, 1); + TaskOutputTensors loop_out = rt_submit_dummy_task(second_args); + (void)get_tensor_data(loop_out.get_ref(0), 1, index); +} + +static void submit_long_spmd_burst(const simpler::tmr::Tensor &spmd_ws) { + for (int32_t i = 0; i < kLongSpmdTasks; ++i) { + CoreTaskArgs args; + args.add_inout(spmd_ws); + args.add_scalar(static_cast(i) * kSpmdBlocks); // base_cl + args.add_scalar(kSpmdSpinIters); + args.launch_spec.set_core_num(kSpmdBlocks); + (void)rt_submit_aiv_task(FUNC_SPMD_SLOW, args); + } +} + +extern "C" { + +__attribute__((visibility("default"))) OrchestrationConfig aicpu_orchestration_config(const ChipTaskArgs &orch_args) { + (void)orch_args; + return OrchestrationConfig{ + .expected_arg_count = 3, + }; +} + +__attribute__((visibility("default"))) void aicpu_orchestration_entry(const ChipTaskArgs &orch_args) { + const simpler::tmr::Tensor &ext_a = orch_args.tensor(0).ref(); + const simpler::tmr::Tensor &ext_b = orch_args.tensor(1).ref(); + const simpler::tmr::Tensor &ext_out = orch_args.tensor(2).ref(); + + uint32_t size = ext_a.shapes[0]; + uint32_t inter_shapes[1] = {size}; + TensorCreateInfo inter_ci(inter_shapes, 1, DataType::FLOAT32); + + // Phase 1: short single-block adds + orch-side wait → guaranteed pre-seal release. + TaskId short_ids[kShortAddTasks]; + for (int32_t i = 0; i < kShortAddTasks; ++i) { + const bool last_short = (i + 1 == kShortAddTasks); + short_ids[i] = submit_short_add(ext_a, ext_b, inter_ci, last_short ? &ext_out : nullptr); + } + wait_preseal_idle_release(short_ids, kShortAddTasks); + + // Phase 2: long multi-block SPMD burst with no wait — orch returns and seals + // while these still run; their deferred releases are elided. + uint32_t spmd_elems = + static_cast(kLongSpmdTasks) * static_cast(kSpmdBlocks) * kFloatsPerCacheLine; + uint32_t spmd_shape[1] = {spmd_elems}; + TensorCreateInfo spmd_ci(spmd_shape, 1, DataType::FLOAT32); + TaskOutputTensors spmd_buf = alloc_tensors(spmd_ci); + submit_long_spmd_burst(spmd_buf.get_ref(0)); +} + +} // extern "C" diff --git a/tests/st/a5/tensormap_and_ringbuffer/dfx/chip_swimlane/test_chip_swimlane.py b/tests/st/a5/tensormap_and_ringbuffer/dfx/chip_swimlane/test_chip_swimlane.py index fc5d16633b..d6845fc818 100644 --- a/tests/st/a5/tensormap_and_ringbuffer/dfx/chip_swimlane/test_chip_swimlane.py +++ b/tests/st/a5/tensormap_and_ringbuffer/dfx/chip_swimlane/test_chip_swimlane.py @@ -78,7 +78,10 @@ class TestChipSwimlane(SceneTestCase): "platforms": ["a5sim", "a5"], "manual": ["a5sim"], "params": {}, - "required_sched_phases": ("release",), + # Tiny 5-task graphs often seal before an idle drain, so post-seal + # elision may drop every deferred release. Release E2E lives in + # TestChipSwimlaneManyAdds below. + "required_sched_phases": (), }, { "name": "aicpu_threads_2", @@ -86,7 +89,7 @@ class TestChipSwimlane(SceneTestCase): "manual": ["a5sim"], "config": {"aicpu_thread_num": 2}, "params": {}, - "required_sched_phases": ("release",), + "required_sched_phases": (), }, ] @@ -117,5 +120,74 @@ def test_run(self, st_platform, st_worker, request): ) +# Short single-block adds (1 perf row each) + long SPMD (blocks per task). +_SHORT_ADD_TASKS = 16 +_LONG_SPMD_TASKS = 4 +_SPMD_BLOCKS = 8 +_MANY_ADDS_TASK_COUNT = _SHORT_ADD_TASKS + _LONG_SPMD_TASKS * _SPMD_BLOCKS + + +@scene_test(level=2, runtime="tensormap_and_ringbuffer") +class TestChipSwimlaneManyAdds(SceneTestCase): + """Short adds (pre-seal `release`) then long SPMD burst (post-seal elide).""" + + CALLABLE = { + "orchestration": { + "source": "kernels/orchestration/many_adds_orch.cpp", + "function_name": "aicpu_orchestration_entry", + "signature": [D.IN, D.IN, D.OUT], + }, + "incores": [ + { + "func_id": 0, + "source": f"{KERNELS_BASE}/aiv/kernel_add.cpp", + "core_type": "aiv", + "signature": [D.IN, D.IN, D.OUT], + }, + { + "func_id": 1, + "name": "SPMD_WRITE_SLOW_AIV", + "source": "../../spmd_sync_start_early_dispatch/kernels/aiv/kernel_spmd_write_slow.cpp", + "core_type": "aiv", + "signature": [D.INOUT], + }, + ], + } + + CASES = [ + { + "name": "default", + "platforms": ["a5sim", "a5"], + "manual": ["a5sim"], + "params": {}, + "required_sched_phases": ("release",), + }, + ] + + def generate_args(self, params): + SIZE = 128 * 128 + return TaskArgsBuilder( + TensorArg("a", torch.full((SIZE,), 2.0, dtype=torch.float32)), + TensorArg("b", torch.full((SIZE,), 3.0, dtype=torch.float32)), + TensorArg("f", torch.zeros(SIZE, dtype=torch.float32)), + ) + + def compute_golden(self, args, params): + args.f[:] = args.a + args.b + + def test_run(self, st_platform, st_worker, request): + run_marker = int(time.time()) + super().test_run(st_platform, st_worker, request) + if not request.config.getoption("--enable-chip-swimlane", default=0): + return + for case in self._matching_cases(st_platform, request): + validate_perf_artifact( + f"TestChipSwimlaneManyAdds_{case['name']}", + since=run_marker, + expected_task_count=_MANY_ADDS_TASK_COUNT, + required_sched_phases=case["required_sched_phases"], + ) + + if __name__ == "__main__": SceneTestCase.run_module(__name__) diff --git a/tests/ut/cpp/a5/test_scheduler_state.cpp b/tests/ut/cpp/a5/test_scheduler_state.cpp index 2cb54304d0..6d990abb8f 100644 --- a/tests/ut/cpp/a5/test_scheduler_state.cpp +++ b/tests/ut/cpp/a5/test_scheduler_state.cpp @@ -195,6 +195,70 @@ TEST_F(SchedulerStateTest, ConsumedTransition) { EXPECT_EQ(slot.task_state.load(), CHIP_TASK_CONSUMED); } +TEST_F(SchedulerStateTest, DrainOrElideExactDrainReleasesToConsumed) { + alignas(64) ChipTaskSlotState first; + alignas(64) ChipTaskSlotState second; + init_slot(first, CHIP_TASK_COMPLETED, 0, 1); + init_slot(second, CHIP_TASK_COMPLETED, 0, 1); + first.fanout_refcount.store(1); + second.fanout_refcount.store(1); + ChipTaskSlotState *deferred[2] = {&first, &second}; + int32_t deferred_count = 2; + + drain_or_elide_deferred_releases(&sched, deferred, deferred_count, /*release_elided=*/false); + + EXPECT_EQ(deferred_count, 0); + EXPECT_EQ(first.task_state.load(), CHIP_TASK_CONSUMED); + EXPECT_EQ(second.task_state.load(), CHIP_TASK_CONSUMED); +} + +TEST_F(SchedulerStateTest, DrainOrElideElidesWithoutCallingOnTaskRelease) { + alignas(64) ChipTaskSlotState first; + alignas(64) ChipTaskSlotState second; + init_slot(first, CHIP_TASK_COMPLETED, 0, 1); + init_slot(second, CHIP_TASK_COMPLETED, 0, 1); + first.fanout_refcount.store(1); + second.fanout_refcount.store(1); + ChipTaskSlotState *deferred[2] = {&first, &second}; + int32_t deferred_count = 2; + + drain_or_elide_deferred_releases(&sched, deferred, deferred_count, /*release_elided=*/true); + + EXPECT_EQ(deferred_count, 0); + EXPECT_EQ(first.task_state.load(), CHIP_TASK_COMPLETED); + EXPECT_EQ(second.task_state.load(), CHIP_TASK_COMPLETED); +} + +TEST_F(SchedulerStateTest, AsyncInlineCompletionDefersReleaseAndDrainsExactlyAtCapacity) { + alignas(64) ChipTaskSlotState first; + alignas(64) ChipTaskSlotState second; + // fanout_count == fanout_refcount so an exact on_task_release reaches CONSUMED. + init_slot(first, CHIP_TASK_PENDING, 0, 1); + init_slot(second, CHIP_TASK_PENDING, 0, 1); + first.fanout_refcount.store(1); + second.fanout_refcount.store(1); + ChipTaskSlotState *deferred[1]{}; + int32_t deferred_count = 0; + AsyncWaitList::DrainCompletionSink sink{}; + sink.sched = &sched; + sink.deferred_release_slot_states = deferred; + sink.deferred_release_count = &deferred_count; + sink.deferred_release_capacity = 1; + + EXPECT_TRUE(sched.async_wait_list.try_inline_complete_locked(sink, first)); + EXPECT_EQ(first.task_state.load(), CHIP_TASK_COMPLETED); + EXPECT_EQ(deferred_count, 1); + EXPECT_EQ(deferred[0], &first); + + // Async path always passes release_elided=false: capacity drains first via + // on_task_release (CONSUMED), then defers second. + EXPECT_TRUE(sched.async_wait_list.try_inline_complete_locked(sink, second)); + EXPECT_EQ(first.task_state.load(), CHIP_TASK_CONSUMED); + EXPECT_EQ(second.task_state.load(), CHIP_TASK_COMPLETED); + EXPECT_EQ(deferred_count, 1); + EXPECT_EQ(deferred[0], &second); +} + TEST_F(SchedulerStateTest, ConsumedHeadAdvancesAfterContendedAdvanceLock) { constexpr int32_t ring_id = CHIP_MAX_RING_DEPTH - 1; constexpr int32_t head_task_id = 0; diff --git a/tests/ut/py/test_sched_overhead_analysis.py b/tests/ut/py/test_sched_overhead_analysis.py index a4764716e8..f387cec5d2 100644 --- a/tests/ut/py/test_sched_overhead_analysis.py +++ b/tests/ut/py/test_sched_overhead_analysis.py @@ -246,6 +246,29 @@ def test_parse_scheduler_counts_hbg_p_thread_standalone_phases(): assert threads[0]["phases_seen"] == {"resolve", "async_poll", "dummy", "idle"} +def test_parse_scheduler_classifies_release_as_scheduler_work(): + data = { + "aicpu_scheduler_phases": [ + [ + { + "phase": "release", + "start_time_us": 1.0, + "end_time_us": 2.0, + "loop_iter": 7, + "tasks_processed": 5, + }, + {"phase": "resolve", "start_time_us": 2.0, "end_time_us": 3.0, "loop_iter": 8}, + ] + ] + } + + threads = parse_scheduler_from_json_phases(data) + + assert threads[0]["role"] == "scheduler" + assert threads[0]["release_us"] == 1.0 + assert threads[0]["phases_seen"] == {"release", "resolve"} + + def test_parse_scheduler_uses_explicit_hbg_resolve_discriminator_at_parent_boundary(): data = { "aicpu_scheduler_phases": [