Skip to content

feat: build supervisor-proposed variations side by side and continue the one chosen (#10) - #45

Draft
sam-bretz wants to merge 2 commits into
issue-24-chat-decision-logfrom
issue-10-supervisor-variations
Draft

sam-bretz wants to merge 2 commits into
issue-24-chat-decision-logfrom
issue-10-supervisor-variations

Conversation

@sam-bretz

@sam-bretz sam-bretz commented Sep 16, 2026 •

Copy link
Copy Markdown
Owner

Closes #10.

Stacked on #44. This PR is based on issue-24-chat-decision-log, because #44 removes the Checkpoint tab and this PR's dashboard display lives in the Chat panel that replaces it. Review and merge #44 first, then retarget this PR to main. Its own changes are the top two commits.

What this does

When a stage has several credible approaches, envctl now builds each one and lets you compare them before anything is published, so the choice no longer rests on one agent's judgement.

  1. A stage opts in with variations: N (2 or 3, off by default). Its supervisor may propose that many alternatives while accepting the work, each with a name and a rationale.
  2. Branching happens inside the same update that accepts the stage, so it happens exactly once:
    • the accepted work continues as as proposed;
    • each alternative becomes a sibling revision, restored from before that stage, with the alternative given to its worker as steering.
  3. Isolation. Each variation reruns that stage and everything after it through QA, in its own worktree and VM. Output branches are named per revision, so variations can't write to each other's branches.
  4. Nothing publishes before you choose. ReadyNodes holds back the approved-change stage while a revision is undecided, and publication refuses too, because publishing work nobody chose can't be undone.
  5. Comparing. envctl run variations or v in the dashboard shows, for each variation: its rationale, stage summaries, the size of its change (from retained evidence), the checks it ran and their results, and its token usage. --json gives the structured form. The run summary shows a waiting comparison on every tab.
  6. Choosing. envctl run choose --variation <name> or C continues one to approval and makes it current. The others become not chosen: their VMs are released, their checkpoints stay reviewable, and they're never published.

Three bugs building it turned up

Each of these would have broken the feature on its own.

1. Deadlock with the default settings. limits.vms defaults to 2, but even two proposals make three revisions, and a finished variation kept its VM while waiting to be chosen. So the last sibling could never start. Now:

  • a finished, undecided variation parks (releases its VM) while a sibling is waiting for one;
  • VMCount doesn't count a parked variation (it stays active, so it isn't mistaken for a retired one);
  • choosing a parked variation queues it for a new VM, the same path a rewound revision resumes by.

The end-to-end test reproduces the deadlock exactly when the VMCount fix is removed ("never reached: every variation finishes"), and checks on every tick that the run never holds more VMs than its limit.

2. Steering landed on the wrong variation. Run.Message wrote to the current revision whatever revision was addressed, so a message meant for one variation would silently go to another. It now writes to the addressed revision.

3. Eight current-revision checks. The engine and daemon assumed only the current revision can run, which would have left every sibling queued forever. They now also accept undecided variations. The daemon allows this only for message, ask and choose. I confirmed by mutation that allowing rewind too would let a rewind addressed to a variation rewind the current revision out from under the comparison.

Two fixes to shared code

  • The composer's stale-selection check compared the viewed revision against the current one, which always differ while you're viewing a variation, so it silently refused every message sent there. It now compares against the viewed revision. Nothing tested this guard before, so I added TestAMessageIsNotSentToWhateverTheSelectionBecameWhileTyping and confirmed it fails when the guard is removed. A rewind that lands while you're typing is still refused.
  • Run.Usage now sums a new Revision.Usage, so a run's total and each variation's usage can't disagree.

Acceptance criteria

  • Opt in per workflow or per stage, with a maximum; off by default; existing workflows unchanged
  • Supervisor proposes 2–3 named variations with rationales; recorded durably
  • Each variation is an isolated branch with its own worktree, runtime and downstream stages through QA
  • Each variation's QA records which checks ran and whether they passed, per variation
  • Side-by-side review in envctl ui (v) and on the command line (run variations --json). Variant data is also in run show --json
  • Choosing continues only that variation to the approved-change gate; the others stay reviewable and are never published
  • Variations count toward limits.run_tokens; the configured maximum bounds how many start, and limits.vms bounds how many run at once

Two deliberate choices:

  • Proposals come alongside acceptance, not instead of it (the issue says "instead of a single accept or correction"). This keeps the attempt state machine intact, and "as proposed" is a real candidate in the comparison.
  • The issue's open question (branch at Design or Code, chosen by the person or the supervisor) is answered: the stage that proposes is the branch point, and you pick it by where you set variations. The supervisor picks which alternatives, never where to branch.

Testing

gofmt -l . prints nothing, go vet ./... is clean, and the full go test ./... passes after the rebase. Coverage by layer:

  • workflow: branching and each refusal, holding back approval, choosing and retiring, steering reaching the right variation. The hold-back test was passing vacuously (no readiness evidence in the fixture); I fixed the fixture and confirmed the test fails without the gate.
  • engine: the full lifecycle end to end (branch, build all within the VM limit, park, choose a parked variation, re-provision, approve, exactly one publication, the others retired), plus the deadlock regression.
  • daemon: which actions may address a variation, with the rewind danger shown by mutation.
  • review and CLI: diff stats, the side-by-side view (including an unavailable diff), lookup by name or ID, and run show.
  • dashboard: the waiting banner, proposals and variation banner in Chat, choosing by name, steer and ask versus approve on a viewed variation, the comparison view, the Decision Log, and the stale-selection guard.

Not verified against real VMs. Re-provisioning a parked variation and restoring its source from checkpoints goes through the fixture backend. It's the same path rewinds use, but one real comparison before merge is worth running.

🤖 Generated with Claude Code

sam-bretz and others added 2 commits September 16, 2026 10:42
…ding them

When a stage had several credible approaches, the worker picked one and
the supervisor accepted or corrected it; the alternatives were never
recorded, so the choice between designs was made silently by one agent.

A node can now opt in with `variations: N`, from 2 to 3, off by default.
Its supervisor may then propose that many alternatives alongside
accepting the work, each with a name and a rationale. They are recorded
on the review, shown on the dashboard's Checkpoint and carried in
`envctl run show --json`.

Proposals come with an acceptance, never instead of one, so the attempt
state machine is untouched and an opted-in stage runs exactly as before.
Recording them is safe for two reasons that are now tested: WorkDigest
clears the review, so an approval stays bound to the same work, and an
unset `variations` restores a configuration's digest exactly.

The existing supervisor contract has six callers, including the
connection probe, so it is unchanged; opted-in nodes get a node-aware
schema instead. It refuses a lone variation, which is not a choice;
variations on a rejection, where the correction is the path forward;
repeated names, which a person could not tell apart; and anything from a
node that did not opt in.

This does not build the variations. Running each as its own approach
through QA and choosing one is most of the issue and needs the checkpoint
model to hold competing results for the same stages. The design proposes
sibling revisions for that, answers the issue's open question about
where variations branch, and orders the remaining work into steps that
each ship behind the opt-in. The dashboard labels proposals as considered
and not built, so nothing implies a comparison that does not exist.

Refs #10

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first slice let a supervisor record alternatives but built none of
them, so "which approach is better?" was still one agent's judgement.

An accepted stage's proposals now become variations, in the mutation
that accepts it so they branch exactly once. The accepted work continues
as "as proposed"; each alternative is a sibling revision restored from
before that stage, with the alternative given to its worker as steering,
running that stage and everything after it through QA in its own
worktree and VM. Siblings supersede nothing and leave the current
revision alone. None reaches its approved change: ReadyNodes holds those
back while a revision is undecided, and publication refuses too, since
publishing work nobody chose cannot be undone.

`envctl run variations` and `v` in the dashboard show them side by side:
rationale, stage summaries, the size of each change from retained
evidence, the checks each ran and their results, and token usage.
`envctl run choose` and `C` continue one to approval and make it
current; the rest become not-chosen, releasing their VMs and keeping
their checkpoints, and are never published.

Building it turned up three bugs that would each have broken this.
limits.vms defaults to 2, but two proposals make three revisions and a
finished variation kept its VM while waiting, so the last sibling could
never start; a finished variation now parks while a sibling is queued,
VMCount stops counting it, and choosing it provisions a VM again.
Run.Message wrote to the current revision whatever was addressed, so
steering one variation would have landed on another. And eight
current-revision checks in the engine and daemon would have left every
sibling queued; they now admit undecided variations, the daemon only for
message, ask and choose, because a rewind addressed to a variation would
otherwise rewind the current revision out from under the comparison.

The composer's stale-selection check compared the viewed revision with
the current one, which always differ while viewing a variation, so it
refused every message sent there. It now compares against the viewed
revision, which keeps its protection: a test proves a rewind that lands
while typing is still refused.

Closes #10

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sam-bretz
sam-bretz force-pushed the issue-10-supervisor-variations branch from 4a9a75c to 569afd1 Compare September 16, 2026 17:42
@sam-bretz sam-bretz changed the title feat: let a supervisor record alternative approaches, and design building them (#10, first slice) feat: build supervisor-proposed variations side by side and continue the one chosen (#10) Sep 16, 2026
@sam-bretz
sam-bretz changed the base branch from main to issue-24-chat-decision-log September 16, 2026 17:42
@sam-bretz
sam-bretz added this pull request to stack #46 September 17, 2026 02:59

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant