Skip to content

Carry a provider failure out of a container body - #2131

Merged
ppXD merged 1 commit into
mainfrom
fix/carry-a-provider-failure-out-of-a-loop-body
Oct 19, 2026
Merged

ppXD merged 1 commit into
mainfrom
fix/carry-a-provider-failure-out-of-a-loop-body

Conversation

@ppXD

@ppXD ppXD commented Oct 19, 2026

Copy link
Copy Markdown
Owner

Summary

  • A provider failure inside a flow.map branch or a flow.loop pass that ended the container was written only on the body node's record, so the run page said "Node 'map' failed." with no credential, repository or repair. The container's node.failed record now also carries the failure of the cell that ended it, which is that node's own recorded failure, else its last agent run's. It is read through a new ICellFailureReader in the same order and with the same parsers RunFailureReader uses, and a nested container reads the record the level below wrote. RunFailureReader therefore returns the same remedy as for a top-level node.
  • A terminate map whose branches fail together answers for the first failing branch by element order, which is the branch its sentence already names, never whichever finished first. A branch replayed from the ledger while a sibling was parked is still the cell that ended the map. Failures the body handled, a flow.try caught, a sibling met or a continue policy skipped leave nothing on a container that later fails for another reason, and a container that fails on its own (a tripped budget, a non-array binding) carries none.
  • A map-branch rerun replays the failed sibling it did not choose into its fork. The agent run that worked in that branch belongs to the original run, so the fork's map failed again with no remedy. The rerun seeder now re-emits a failed cell with the failure the original run recorded for it (own record, else agent run), so the fork answers as the original did. A failed cell that recorded no failure is re-emitted exactly as before.
  • Container semantics are untouched: fail-fast and collect policies, retries, partial results and the sentences containers write do not change. Only what the failure record carries changes.
  • Out of scope: flow.iterate is a single-expression node, not a container, and flow.subworkflow runs a child run. A failure inside a child run is not carried to the parent node by this change.
  • Non-breaking: no migration, no API or wire change. A record gains the failure key a node record already carries. Rows written before read exactly as they did, a reader that predates the key ignores it, and a rollback is a plain revert.

Test plan

  • Integration through the real engine against GitLabDoorkeeperStub: map, loop, loop-in-map and map-in-loop each end on a dead credential and the run detail remedy names the credential and repository, with every container's sentence untouched.
  • Integration: first failing branch by element order for two credentials; a branch parked and replayed from the ledger; agent-run fallback never reading a sibling's agent run; a handled or flow.try-caught failure followed by a non-provider container failure answers no stale remedy (flow.iterate and a flaky node as the non-provider failure).
  • Integration: CellFailureReader boundary (newest record, then last agent run; nothing from another iteration, node, run, team, or another run of the same team); the real rerun seeder carries a failed cell's failure into the fork for a kept top-level cell and a sibling branch, from the cell's own record and from the original run's agent run, and leaves a failure-less cell's record unchanged.
  • Integration: a map-branch rerun through RerunMapBranchAsync whose fork fails on the replayed failed branch answers with the original run's remedy, never the fresh branch's.
  • HTTP E2E: run detail and the lean remedy read for an Owner, a Member and a Viewer after a map and a loop end on a dead connection; the map-branch rerun fork over POST /api/workflows/runs/{id}/rerun-map-branch; a run recorded before containers carried the failure answers exactly as it did.
  • Mutation checks: dropping the run scoping on the agent-run lookup, and dropping the failure the seeder carries, are each caught.
  • Regression: rerun integration (101), credentials integration (277), engine-area integration (646), credentials and rerun E2E (83), related unit tests (6229), all green; full solution builds.
  • Follow-up for the owner: carry a child run's failure onto a flow.subworkflow parent node, if wanted.

@ppXD
ppXD force-pushed the fix/carry-a-provider-failure-out-of-a-loop-body branch from 12b3b4f to a37d76b Compare October 19, 2026 03:04
@ppXD
ppXD force-pushed the fix/let-a-reauthorization-win-over-the-repair-sweep branch from 90c3126 to 2dcd2be Compare October 19, 2026 03:05
@ppXD
ppXD changed the base branch from fix/let-a-reauthorization-win-over-the-repair-sweep to main October 19, 2026 03:06
A provider failure inside a flow.map branch or a flow.loop pass that
ended the container was written on the body node's own node.failed
record and nowhere the run's remedy could find it. The run's failure is
the one the node that ENDED the run recorded; for such a run that node is
the container, and the container's node.failed carried only its sentence,
so the run page said "Node 'map' failed." with no card, no credential and
no repair.

The container's record now carries the failure of the cell whose
unhandled failure ended it: that node's own recorded failure, else the
agent run that last worked in that very cell, read through a new
ICellFailureReader in the order and with the parsers RunFailureReader
uses for the node that ended a run. A nested container reads the record
the level below wrote, so the failure travels as far as the nesting goes
and the run's remedy is the one a top-level node of the same failure
gets. Fail-fast and collect policies, retries, partial results and the
sentences containers write are untouched; only what the record carries
changes.

The cell is chosen deterministically. A terminate map whose branches
fail concurrently answers for the first failing branch by element order,
which is the branch its sentence already names, never whichever finished
first. A branch replayed from the ledger while a sibling was parked is
still the cell that ended the map and is read back from its own record.
Nothing else is read: a provider failure the body handled with an error
edge, one a try caught, one a sibling branch met, or one a continue
policy skipped leaves nothing on a container that later fails for
another reason, and a container that fails on its own (a tripped budget,
a binding that is not an array) carries none.

A map-branch rerun replays a failed sibling into its fork, and the agent
run that worked in that branch belongs to the original run, so the fork
could never find it: its map failed again on the replayed branch and
answered with no remedy. The rerun seeder now re-emits a failed cell with
the failure the original run recorded for it (the cell's own record, else
its agent run, through the same reader), so the fork's map answers for the
replayed branch as the original's did. A failed cell that recorded no
failure is re-emitted exactly as before.

flow.iterate is a single-expression node, not a container, and
flow.subworkflow runs a child run: a failure inside a child run is not
carried to the parent node by this change.

Additive: a container record gains the failure key a node record already
carries, no migration, no API or wire change. A row written before this
reads exactly as it did, and a reader that predates the key ignores it.
@ppXD
ppXD force-pushed the fix/carry-a-provider-failure-out-of-a-loop-body branch from a37d76b to 85357b9 Compare October 19, 2026 03:06
@ppXD
ppXD merged commit e09fc39 into main Oct 19, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant