fix(autopilot): observe a plan cancel at every stage-advance gate - #4811
Conversation
The Cancel control revokes approval to keep orchestrating: it sets tracker.stopped and clears slot._auto_run, and deliberately leaves slot._stopping alone, because that flag means the slot itself is being torn down. _stage_loop's four advancement gates read only _stopping, so none of them observed a cancel at all. The window that matters is _run_chat -- the loop's longest await, and so the likeliest place for a cancel to land. The loop resumed from it and ran the next stage against an approval the user had already revoked. The sub-agent poll and the between-stages re-entry had the same gap. Route all four gates through one _orchestration_stopped(slot, tracker) predicate that reads both channels, so they cannot drift apart again. Fixing it the other way -- having Cancel set slot._stopping -- would hand a plan cancel the teardown semantics that flag carries for paths outside this loop, which is not what cancelling a plan asks for. The all-stages-complete summary still reads _stopping alone: a cancel arriving after the final stage's gate leaves a plan whose stages all genuinely ran, and suppressing a truthful summary there would be worse. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated. Missing sections:
Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle. |
2 similar comments
|
👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated. Missing sections:
Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle. |
|
👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated. Missing sections:
Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle. |
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
First Principles Review (Fable 5, fork) — ✅ PASSPremise-level review of All evidence gathered — the base file confirms the description's claims (all four gates on First-Principles-Verdict: PASS A confirmed defect — cancel ignored at every stage gate — fixed at its cause, with nothing riding along and no duplicate mechanism. What this change shipsIntent: make the plan Cancel control actually stop the orchestration loop instead of letting one more stage run. This is a FIX.
Counts run: [FIRST-PRINCIPLES-REVIEWED] 733572e |
Design Review (Fable 5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS A missed-signal bug fixed at its root: one predicate unifies both revocation channels at every gate, with the flag-conflation alternative correctly rejected. The single-predicate shape is the right one — [DESIGN-REVIEWED] 733572e |
Opus 4.8 Review (fork) — ✅ no blocking findingsReviewed Review detailsThe change is a narrow, well-grounded correctness fix. Verified independently:
The discovery pass produced no candidates, and Step 2 surfaced nothing groundable to the required bar. No findings. [OPUS-REVIEWED] 733572e |
bolichen97
left a comment
There was a problem hiding this comment.
Approved after a description-vs-diff consistency review: every claim in the PR description is backed by the diff, and the diff carries no material change the description leaves unmentioned.
Base branch was modified
…rodotdev#4811) The Cancel control revokes approval to keep orchestrating: it sets tracker.stopped and clears slot._auto_run, and deliberately leaves slot._stopping alone, because that flag means the slot itself is being torn down. _stage_loop's four advancement gates read only _stopping, so none of them observed a cancel at all. The window that matters is _run_chat -- the loop's longest await, and so the likeliest place for a cancel to land. The loop resumed from it and ran the next stage against an approval the user had already revoked. The sub-agent poll and the between-stages re-entry had the same gap. Route all four gates through one _orchestration_stopped(slot, tracker) predicate that reads both channels, so they cannot drift apart again. Fixing it the other way -- having Cancel set slot._stopping -- would hand a plan cancel the teardown semantics that flag carries for paths outside this loop, which is not what cancelling a plan asks for. The all-stages-complete summary still reads _stopping alone: a cancel arriving after the final stage's gate leaves a plan whose stages all genuinely ran, and suppressing a truthful summary there would be worse. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Problem / Motivation
Cancelling a plan does not stop the plan. The user clicks Cancel, the UI posts
🛑 Plan cancelled., and the orchestrator then runs the next stage anyway — against an approval the user has just revoked.Two independent channels revoke an orchestration run, and they mean different things:
slot._stoppingPOST /api/chat/slots/{slot}/stop)tracker.stoppedapi_chat_plan_action,action="cancel") and the typedstop/cancel/abortwords_stage_loophas four advancement gates, and onmainall four readslot._stoppingonly. Cancel never sets that flag, so no gate observes a cancel.grep tracker.stoppedinside the loop onmainreturns zero readers.Why it matters
The window that matters is
_run_chat. It is the loop's longest await, so it is the likeliest place for a cancel to land — and it is the exact point the loop resumes from directly into the next stage.So the failure lands on the user who is trying hardest to stop the agent: someone who sees a plan going wrong mid-stage and hits Cancel gets one more full stage of LLM turns and tool calls executed on their behalf, after the product told them the plan was cancelled. Sub-agent tasks are cancelled and
_auto_runis cleared, which makes the UI look stopped, so the extra stage is not obviously attributable to the ignored cancel.The existing test
test_cancel_clears_auto_runasserts onlyslot._auto_run is False— it never asserted that the loop stops, which is why the gap survived.What changed (motivation → approach → change)
Symptom — a cancel that lands during
_run_chatis followed by the next stage running.Root cause — the four advancement gates read one of the two stop channels. The cancel handler sets
tracker.stoppedandslot._auto_run, and deliberately notslot._stopping; the gates test onlyslot._stopping. The two channels were never wired together.Change — one predicate,
_orchestration_stopped(slot, tracker), reading both channels, called from all four gates (top-of-iteration, post-_run_chat, the sub-agent poll condition, pre-capture). Routing them through a single predicate rather than repeating a two-term condition four times is the point: it is what stops the two channels drifting apart again.Deliberately not the other direction. #4783 offered a second option — have Cancel set
slot._stopping. That flag carries session/ACP teardown semantics for paths outside this loop, and cancelling a plan is not a request to tear the session down. Folding one into the other would trade this bug for a semantics conflation.One gate is deliberately left alone. The all-stages-complete summary still reads
slot._stoppingby itself. A cancel that arrives after the final stage's gate has already passed leaves a plan whose stages all genuinely ran, and suppressing a truthful completion summary there would be the worse trade. This is called out in the spec so it reads as a decision rather than an oversight.Not touched: cancellation-handler semantics,
slot._stopping, and theauto_runapproval gate. With the gates fixed, a cancelled run breaks before reaching that gate, so the parameter-snapshot question #4783 also raises is moot here and stays available for separate work. One production file, one test file, one spec.docs/system-specs/modules/autopilot.mddocumented the old single-flag behavior in two places (the step list, and the Stop-and-Cancel table); both are updated in the same commit, perAGENTS.md.Tests
Three tests added in
test/test_orchestrator_cancel_stops_advance.py, each driving the real HTTP cancel handler concurrently with a real_stage_loop— not a simulated flag flip.test_cancel_during_run_chat_does_not_advance_run_chatstops the loop before stage 2test_cancel_between_stages_blocks_reentrytest_cancel_during_subagent_wait_does_not_advanceFail-before / pass-after, measured on pristine
main(c505a877) by reverting only the production file:Each test also asserts
slot._stopping is False, which pins the fix to reading the tracker rather than to widening what a plan cancel means — the assertion fails if someone later "fixes" this by conflating the flags.Two details that produce a green-for-the-wrong-reason test if got wrong, both handled and commented in the fixture:
running_agents_forreturningNone, breaks the loop on its own. The fixture wires the permissive case (running_agents_for→[]) so the cancel is the only thing that can stop the loop.test_cancel_between_stages_blocks_reentryruns stage 1 for real before cancelling. Cancel no-ops whenslot._orch_trackeris stillNone, so a test that cancels before any stage has run cancels nothing and passes vacuously.The sub-agent test substitutes the poll's sleep via the module reference
chat_orchestratorholds, rather than patchingasyncio.sleepitself — the substitute awaits the aiohttp client to issue the cancel, so a global patch would reach the very call it depends on.Regression run —
test_dashboard_chat.pyplus both existing_stage_loopsuites: 654 passed.Gates:
flake8·isort·scripts/check_black_formatting.py(3 files in scope) ·mypy·scripts/docs_lint.py·scripts/check_brand_name.py— all clean.Manual verification
N/A — unit coverage sufficient: the tests exercise the real
api_chat_plan_actionhandler over a real aiohttpTestClientagainst a real_stage_loop, so the HTTP cancel path and the orchestration loop are both the production code rather than stand-ins. The only mocked component is_run_chat(the LLM turn), which is what makes the cancel's arrival deterministic instead of a race.Related Issues
Closes #4783.
Sibling context: the write-window case of this same failure shape was fixed in #3771; umbrella #1783.
Checklist
fix(autopilot): observe a plan cancel at every stage-advance gate)docs/system-specs/modules/autopilot.md, same commit)Contribution License Agreement
The template carries a placeholder here pending OSPO-supplied wording, so no CLA text is reproduced. Happy to agree to the CLA once it is published in the template.
🤖 Generated with Claude Code