Skip to content

✨ Run spawned document work concurrently with <All> (#855) - #856

Open
taras wants to merge 4 commits into
agent/issue-854-projectionfrom
agent/all-spawns
Open

taras wants to merge 4 commits into
agent/issue-854-projectionfrom
agent/all-spawns

Conversation

@taras

@taras taras commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

Why

A document could not run two pieces of work at the same time. Everything a
document does is sequential, so two <Session> conversations, two long reads or
two independent sub-documents run one after another even when neither needs the
other's result. #827's agent sessions need concurrent work, and #854's REPL needs
two live conversations at once, so this is the structural primitive they are
both waiting on.

What changes

Before:

<Session agent="claude">…</Session>
<Session agent="codex">…</Session>

Two conversations, one after the other. Nothing in the language can say they are
independent.

After:

<All>
  <Spawn><Session agent="claude">…</Session></Spawn>
  <Spawn><Session agent="codex">…</Session></Spawn>
</All>

Both run at the same time. <All> waits for every <Spawn> child and renders
their Markdown in authored order even when they finish in another order, so
the document reads the same however the work interleaved.

Each <Spawn> is its own child: bindings made in one stay in it, and a <Return>
or <Break> inside one cannot target work outside it. A <Spawn> anywhere but
directly inside <All> is refused before the document runs, and so is an <All>
with fewer than two <Spawn> children.

Under a durable run the children are joined through durableAll(), so each child
is a coroutine of its own in the Journal and a replay resumes them the same way.

How it works

<All> → one child per <Spawn> → all()/durableAll() → children rendered in authored order

Two identity mechanisms had to stop being execution-wide for this to work at
all, because concurrent children resolve components at the same time:

  • Import frames (invocation-identity.ts) are now one LIFO stack per
    Effection scope
    rather than one per execution. Two children importing at once
    no longer see each other's frame.
  • Resolution windows (component-resolution.ts) are likewise per scope, and
    a provider's issuance is spent against the window that asked for it rather than
    against whatever window happened to be innermost.

Both were found by measurement: two ordinary <Session> invocations under <All>
failed with ComponentInvocationError until ownership followed the scope.

Review guide

Start with: packages/core/tests/all.test.ts

Then review:

  1. packages/core/src/structural.ts and structural-rules.ts — what the syntax
    is, and what is refused before anything runs
  2. packages/core/src/expand.ts — expandAll(), spawnChild(),
    spawnEnvironment(), and the live/durable join
  3. packages/core/src/invocation-identity.ts and
    components/component-resolution.ts — per-scope ownership
  4. specs/executable-mdx-spec.md and architecture.md

Look carefully at:

  • The join: finishing order and append order are deliberately separate, and the
    rendered output is authored order in both the live and the durable path.
  • Containment: a child's <Return>/<Break> and its bindings stop at its own
    boundary.

What must stay true

  • <All> renders in authored order — enforced by collecting each child's result
    positionally and checked by ALL6 and the <All>-of-<Session> rows.
  • A <Spawn> outside <All> is a document error, not a runtime one — enforced by
    spawnElementViolations()/misplacedSpawnViolations() and checked in
    all.test.ts.
  • Concurrent children resolve their own components — enforced by per-scope frames
    and windows, checked by the concurrent rows in answer-identity.test.ts and
    invocation-identity.test.ts.
  • The complete structural vocabulary stays frozen — syntax-catalog.test.ts SY4c
    and syntax-cli.test.ts SX2b now include All and Spawn.

How to verify it

  • deno task test packages/core/tests/all.test.ts proves the syntax, the
    refusals, isolation, ordering and the durable join, and fails if a child's
    bindings escape or output follows finishing order.
  • The concurrent identity rows prove two children resolving at once each get
    their own answer, and fail if a frame or window is shared — which is exactly
    what happened before this change.

Scope

Included

  • <All>/<Spawn> syntax, validation, expansion and durable join
  • Per-Effection-scope import frames and resolution windows
  • The frozen-vocabulary corrections that follow from adding two constructs
  • test-weights.json from run 36473005680 at b14a2ba1

Intentionally unchanged

  • No concurrency limit, no cancellation policy beyond what a failing child
    already does: a child's failure raises into its owner as any task does.
  • <All> is not an iteration construct; <Loop> is unchanged.

Generated or mechanical changes

  • test-weights.json is the artifact of Measure test weights run
    36473005680 at b14a2ba1, committed unchanged. Nothing was hand-edited, and no
    shard recalibration is included.

Risks and limitations

Scope confirmation

  • Every changed file supports the purpose described above.
  • Unrelated cleanup and formatting changes are excluded.
  • Generated or mechanical changes are clearly identified.
  • The description matches the final diff and test results.

Closes #855

@taras
taras changed the base branch from main to agent/issue-854-projection September 28, 2026 22:15
@taras
taras added this pull request to stack #858 September 28, 2026 22:24
`<All>` runs its direct `<Spawn>` children at the same time and waits for
all of them. Both names are engine structural syntax, so no repository
file, registration, bundle or Plugin can supply either, and they work in
ordinary and admitted generated XMD alike.

One shared pure rule decides the whole structure from source — paired
forms, no props, at least two spawns, blank text only between them, no
`<Spawn>` outside its `<All>`, and no `<Return>` or `<Break>` reaching
past a spawn — and expansion, non-executing validation and the
whole-fragment preflight read the same result, so a malformed construct
starts no child anywhere.

Each child is a durable coroutine of the one that reached the `<All>`,
allocated in source order by the existing `durableAll()` join, with its
own binding environment, live overlay, eval scope, block counter, output
buffer, failure ledger, hide set and expansion path. Nothing merges back.
`<All>` appends the renderings in authored order only after every child
succeeds, so completion order may decide when records append and never
what the document renders. The first failure cancels and joins the rest,
publishes nothing, and fabricates no completion.

Capability-backed identity becomes branch-local, which is what the
Story's own `<All><Spawn><Session>` example needs. Canonical import
frames and provider answer windows were kept as one execution-wide
last-open stack, so two children resolving at once could have one child's
selection land in the other's frame — an ordinary `<Session>` in each
branch then named no durable identity at all. Both stacks are now owned
per Effection scope: nesting inside one scope still hides its parent
until it closes, sibling branches have independent tops, closing or
cancelling one neither clears nor authorizes the other, and a provider
installation may answer in two open windows while a second, different
answer in either still refuses. The scope is a private key and reaches no
document, component or provider.

`All` and `Spawn` join the two frozen complete structural-vocabulary
expectations, which is what failed the earlier weights run.
The artifact of run 36473005680, unchanged. `<All>` and `<Spawn>` add one test
file and change what several others run, and a weight is only true of the runner
that measured it.
`toSorted()` is ES2023 and the Node typecheck targets ES2022, so two rows of the
`<All>` suite failed every `test-node` shard while Deno and Bun passed. `sort()`
on the array `map` just made is the convention this repository already documents
in `scripts/lib/verify.ts` and Git's remote-composition proof.
…ced spawn once

Three corrections to the reviewed `<All>` delivery.

A claim was keyed by the answer object alone, which made the first resolution
part of that object's permanent identity. A provider that owns one immutable
definition and hands it to every import it answers was refused the second time —
punished for not copying itself. The key is now the pair of answer and resolution
window: what an object *is* stays settled by its first claim, so it cannot be
renamed, re-originated or taken by another installation, while which import it
answered is recorded per window. Each window keeps its own retained copy, so
`identify()` still answers only for the window that claimed, and a second
*different* answer in a window already spent still refuses.

Reuse is of the definition, not merely of the reference. The first claim retains
core's copy as the baseline the stated identity describes, and a later window may
answer with that object only while it still describes it — so a provider cannot
claim a definition, edit it, and have the next import record the edited one under
the revision that described the original. An answer core could never retain does
not become claimable because it changed.

A `<Spawn>` written below an `<All>` but not directly inside it was reported
twice during non-executing validation: once by the `<All>`'s own structure walk,
which anchors the diagnostic at the misplaced element, and again by the element's
own rule, whose guard only knew its immediate parent. The element's rule now owns
exactly one case — a `<Spawn>` with no `<All>` above it at all — and the walk
owns every other.

And the vocabulary: work owned by a `<Spawn>` is a spawn, a spawned child,
sibling spawns, spawned concurrency. It was described as branching, which is what
an `<If>` arm and a `<Case>` are. Established meanings elsewhere are untouched.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run spawned document work concurrently with <All>

1 participant