Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 61 additions & 3 deletions src/a2a3/runtime/host_build_graph/docs/RUNTIME_LOGIC.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,8 @@ a boundary from which region happens to be reserved first —

**Why the orchestrator is not in the arena at all.** hbg has no device-side
orchestrator, so nothing on the device reads its state: not the `fanin_seen_epoch`
table, not the scope arrays, not the TensorMap (~9.3 MB between them). It is
or `fanin_reach` tables, not the scope arrays, not the TensorMap (~9.4 MB between
them). It is
therefore a plain host object that owns those arrays — `OrchestratorState::init`
allocates them — and `RuntimeContext` reaches it through a pointer that `bind` drops
before the copied zone is uploaded, so no host address crosses the boundary. A
Expand Down Expand Up @@ -275,12 +276,69 @@ TensorMap maps tensor regions to producer task IDs. For every task:
1. INPUT/INOUT regions look up overlapping producers.
2. Explicit and discovered producers are deduplicated into the payload's fanin
region.
3. OUTPUT/INOUT regions register the new task as producer.
4. Each producer tracks its highest consumer local ID for completion metadata.
3. Transitive reduction drops the producers another producer already reaches.
4. OUTPUT/INOUT regions register the new task as producer.
5. Each producer tracks its highest consumer local ID for completion metadata.

There is no fanout adjacency or dependency pool. A per-slot completion flag is
the readiness truth on device.

#### Bounded transitive reduction

Step 3 removes edges the rest of the fanin already orders. When a consumer names
both `P` and `Q`, and `Q` is itself reachable from `P`, the chain
`P -> ... -> Q -> consumer` orders the consumer behind `P` on its own, so the
direct `P` edge decides nothing. Dropping it shortens the region the boot scan
and every wake-list reclassification walk, at the cost of two words of work per
submit on the host.

Each task publishes one 64-bit word of ancestors, indexed by task local id: bit
`i` of task `t` is set when the task `i + 1` ids before `t` reaches it. A submit
folds its producers' words -- shifted by each producer's distance, which is
distance addition -- into its own, then drops every producer whose bit the fold
produced. Because a producer's word is already its own closure, a chain of any
length inside the window collapses in one pass, not one hop at a time.

Two host_build_graph properties keep this to a single word of state. A task id
is its slot index, handed out by a forward-only bump allocator and never
reclaimed, so it doubles as the global submission order and a distance is a
subtraction; and a producer's word, published by its own submit, is never
rewritten, so reading it needs no proof that the slot still holds the task that
wrote it. `FANIN_REACH_WINDOW` bounds how far back a proof can reach: a producer
further back keeps its edge, since neither it nor its ancestors fit the word.

Reduction rewrites readiness only. Buffer lifetime rides
`last_consumer_local_id`, raised when the edge was appended and never lowered, so
a producer whose edge is dropped still waits for that consumer to retire before
the host may overwrite it. A `SIMPLER_DFX` build reports the edges built and the
edges dropped once per orchestration.

#### Inside a recorded Graph body

A Graph body's edges live in the Definition's own fanin CSR, not in a task table,
and they are reduced where they are recorded -- once, before any replay reads
them. `graph_reduce_recorded_fanin` runs on each recorded task as its producers
are settled, and shortens the CSR the same way.

The resolution differs, and the difference is the point. The global path runs on
every submit against a table with no fixed bound, so one word of ancestors is
what it can afford. A body is capped at `MAX_IN_GRAPH_TASKS`, so a row is a fixed
128 B and the whole array 128 KiB of recorder scratch; and the fold is paid once
against however many times that Graph is replayed. The recording path therefore
carries the **exact** closure and has no window: a producer arbitrarily far back
in the body is still reduced.

The two compose without either knowing about the other. A body's tasks are
ordered against everything before the Graph by the outer Graph task's own fanin,
which the global path reduced; a producer outside the recording window contributes
no in-body edge at all. So reducing inside a body cannot change how the body is
ordered against the rest of the run.

Recorded bodies need no retention argument: a body's buffers come out of the
Graph's own heap and are released when the Graph completes, never per task, so a
dropped edge holds no lifetime. Each Definition logs its shipped and reduced edge
counts at DEBUG as it is laid out.

## 6. Boot Classification and Wake Lists

Submit does not push tasks into ready queues. After the graph arrives on device,
Expand Down
64 changes: 61 additions & 3 deletions src/a5/runtime/host_build_graph/docs/RUNTIME_LOGIC.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,8 @@ a boundary from which region happens to be reserved first —

**Why the orchestrator is not in the arena at all.** hbg has no device-side
orchestrator, so nothing on the device reads its state: not the `fanin_seen_epoch`
table, not the scope arrays, not the TensorMap (~9.3 MB between them). It is
or `fanin_reach` tables, not the scope arrays, not the TensorMap (~9.4 MB between
them). It is
therefore a plain host object that owns those arrays — `OrchestratorState::init`
allocates them — and `RuntimeContext` reaches it through a pointer that `bind` drops
before the copied zone is uploaded, so no host address crosses the boundary. A
Expand Down Expand Up @@ -275,12 +276,69 @@ TensorMap maps tensor regions to producer task IDs. For every task:
1. INPUT/INOUT regions look up overlapping producers.
2. Explicit and discovered producers are deduplicated into the payload's fanin
region.
3. OUTPUT/INOUT regions register the new task as producer.
4. Each producer tracks its highest consumer local ID for completion metadata.
3. Transitive reduction drops the producers another producer already reaches.
4. OUTPUT/INOUT regions register the new task as producer.
5. Each producer tracks its highest consumer local ID for completion metadata.

There is no fanout adjacency or dependency pool. A per-slot completion flag is
the readiness truth on device.

#### Bounded transitive reduction

Step 3 removes edges the rest of the fanin already orders. When a consumer names
both `P` and `Q`, and `Q` is itself reachable from `P`, the chain
`P -> ... -> Q -> consumer` orders the consumer behind `P` on its own, so the
direct `P` edge decides nothing. Dropping it shortens the region the boot scan
and every wake-list reclassification walk, at the cost of two words of work per
submit on the host.

Each task publishes one 64-bit word of ancestors, indexed by task local id: bit
`i` of task `t` is set when the task `i + 1` ids before `t` reaches it. A submit
folds its producers' words -- shifted by each producer's distance, which is
distance addition -- into its own, then drops every producer whose bit the fold
produced. Because a producer's word is already its own closure, a chain of any
length inside the window collapses in one pass, not one hop at a time.

Two host_build_graph properties keep this to a single word of state. A task id
is its slot index, handed out by a forward-only bump allocator and never
reclaimed, so it doubles as the global submission order and a distance is a
subtraction; and a producer's word, published by its own submit, is never
rewritten, so reading it needs no proof that the slot still holds the task that
wrote it. `FANIN_REACH_WINDOW` bounds how far back a proof can reach: a producer
further back keeps its edge, since neither it nor its ancestors fit the word.

Reduction rewrites readiness only. Buffer lifetime rides
`last_consumer_local_id`, raised when the edge was appended and never lowered, so
a producer whose edge is dropped still waits for that consumer to retire before
the host may overwrite it. A `SIMPLER_DFX` build reports the edges built and the
edges dropped once per orchestration.

#### Inside a recorded Graph body

A Graph body's edges live in the Definition's own fanin CSR, not in a task table,
and they are reduced where they are recorded -- once, before any replay reads
them. `graph_reduce_recorded_fanin` runs on each recorded task as its producers
are settled, and shortens the CSR the same way.

The resolution differs, and the difference is the point. The global path runs on
every submit against a table with no fixed bound, so one word of ancestors is
what it can afford. A body is capped at `MAX_IN_GRAPH_TASKS`, so a row is a fixed
128 B and the whole array 128 KiB of recorder scratch; and the fold is paid once
against however many times that Graph is replayed. The recording path therefore
carries the **exact** closure and has no window: a producer arbitrarily far back
in the body is still reduced.

The two compose without either knowing about the other. A body's tasks are
ordered against everything before the Graph by the outer Graph task's own fanin,
which the global path reduced; a producer outside the recording window contributes
no in-body edge at all. So reducing inside a body cannot change how the body is
ordered against the rest of the run.

Recorded bodies need no retention argument: a body's buffers come out of the
Graph's own heap and are released when the Graph completes, never per task, so a
dropped edge holds no lifetime. Each Definition logs its shipped and reduced edge
counts at DEBUG as it is laid out.

## 6. Boot Classification and Wake Lists

Submit does not push tasks into ready queues. After the graph arrives on device,
Expand Down
13 changes: 13 additions & 0 deletions src/common/host_build_graph/orchestrator.h
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,14 @@ struct OrchestratorState {
std::unique_ptr<uint32_t[]> fanin_seen_epoch;
uint32_t fanin_seen_current_epoch{1};

// Frozen dependency-ancestor reachability, one word per task, indexed by local
// id. Bit i of entry `t` is set when the task with local id `t - i - 1` reaches
// `t` through a chain of fanin edges. Published once by t's own submit and never
// rewritten: a task id is also its slot index and hbg reclaims neither, so an
// entry describes the same task for the whole run. Orchestrator scratch like
// fanin_seen_epoch — host-only, and nothing on the device reads it.
std::unique_ptr<uint64_t[]> fanin_reach;

// === TENSOR MAP (Private) ===
ChipTensorMap tensor_map; // Producer lookup

Expand Down Expand Up @@ -124,6 +132,11 @@ struct OrchestratorState {
int64_t tasks_submitted;
int64_t buffers_allocated;
int64_t bytes_allocated;
// Fanin edges transitive reduction dropped, and the edges that reached it. Both
// count the whole run and are reported once by mark_done, which is how a change
// to the window or to a workload's shape is read off a run.
int64_t fanin_edges_seen;
int64_t fanin_edges_reduced;
#endif

bool in_manual_scope() const { return scope_stack_top >= manual_begin_depth; }
Expand Down
9 changes: 9 additions & 0 deletions src/common/host_build_graph/runtime_types.h
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,15 @@ inline constexpr uint64_t READY_QUEUE_CAPACITY_LIMIT = 32768;
// fanin of any workload (paged_attention is the densest).
#define CHIP_MAX_FANIN 128

// How far back transitive reduction can prove one fanin edge redundant, measured
// in task local ids. One native word, so a task's whole ancestor set is a single
// uint64 and the shift-merge in reduce_redundant_fanin is one instruction; the
// pass relies on that width, since a shift by FANIN_REACH_WINDOW would be
// undefined and every ancestor of a producer that far back is out of the window
// anyway. Raising it means widening the entry to several words and shifting
// across them, not editing this number.
inline constexpr int32_t FANIN_REACH_WINDOW = 64;

// Alignment of every per-task region inside an argument pool. Each region starts
// and ends on a cache line so TaskPayload::init's round-up scalar memcpy stays
// inside the task's own region — see its comment. simpler::hbg::Tensor is already 2 cache
Expand Down
Loading
Loading