ci(fleet): retry transient lane failures and fix the self-repin version SIGPIPE - #543
Merged
Merged
Conversation
…on SIGPIPE The fleet fan-out lanes primary, dependents, and remainder ran with no retry, so a transient GitHub hosted-runner non-acquisition (the job not acquired by a runner, cancelled after fifteen minutes) or a broken pipe hard-failed the whole fleet with no second chance. Set retry-attempts to two on those lanes to match the heavy lane; the dispatch-suite loop already treats a cancelled run as retryable. Also fix an intermittent broken-pipe failure in the fleet-repin self-repin step: it read the installed version with cascade version piped into head, and because cascade version prints three lines from separate writes, head closing the pipe after the first line races the later writes into a broken pipe that pipefail promotes to a step failure. Capture the output first and split it without a pipe. Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Makes the fleet resilient to the two transient flakes that have hard-failed publishing runs.
fleet-e2e.yaml): theprimary,dependents, andremainderlanes ran with no retry, so a transient hosted-runner non-acquisition cancel (or a broken pipe) hard-failed the fleet with no second chance. Setretry-attempts: 2on those lanes (heavy already had it). The dispatch-suite loop already treats a cancelled run as retryable, so a runner-non-acquisition cancel now consumes a retry instead of failing.fleet-repin/action.yaml): the step read the installed version withcascade version | head -n1 | awk.cascade versionprints three lines from separate writes;headclosing the pipe after line one races the later writes into a broken pipe (exit 141) thatpipefailpromotes to a step failure - it hit release-only and single-env. Capture the output first, then split it without a pipe.Verification
go build,go test ./...(2686),golangci-lint, andactionlinton the changed workflow all clean; shellcheck clean on the new snippet. No test pinned the retry values. This is the fleet resilience the goal needs so the publishing run reaches green reliably.