From 6bee4f1697890bf89451a8ba664a4f49bc699cf6 Mon Sep 17 00:00:00 2001 From: Joshua Temple Date: Thu, 9 Jul 2026 17:34:29 -0400 Subject: [PATCH] ci(fleet): serialize the remainder lane to lower peak hosted-runner demand The fleet fans out to twelve example repos, each dispatching several child runs that each need a GitHub hosted runner, and under that peak demand GitHub intermittently fails to allocate runners and cancels lanes. The retry coverage tolerates it; this reduces the cause. Drop the remainder lane max-parallel from two to one so the eight light repos run one at a time instead of two, roughly halving the peak concurrent runner demand during that lane. The remainder lane's wall-clock roughly doubles; total fleet wall-clock grows by less than that since the other lanes are unchanged. Lane sequencing, roster, needs-graph, the aggregate gate, and the retry attempts are unchanged. Signed-off-by: Joshua Temple --- .github/workflows/fleet-e2e.yaml | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/.github/workflows/fleet-e2e.yaml b/.github/workflows/fleet-e2e.yaml index ac87ce91..649bcf03 100644 --- a/.github/workflows/fleet-e2e.yaml +++ b/.github/workflows/fleet-e2e.yaml @@ -439,10 +439,17 @@ jobs: # them re-dispatches once rather than redding the whole fleet. retry-attempts: '2' - # Lane 3: the light remainder, capped at two repos in flight. Sequenced AFTER - # the heavy lane (via needs) so 4env and this capped matrix never overlap; - # `always()` lets it proceed when the heavy lane was filtered out. Still gated - # on a green repin. max-parallel keeps live API pressure at two repos. + # Lane 3: the light remainder, run one repo at a time. Sequenced AFTER + # the heavy lane (via needs) so 4env and this matrix never overlap; + # `always()` lets it proceed when the heavy lane was filtered out. Still + # gated on a green repin. max-parallel is serialized (was 2) to lower the + # fleet's peak hosted-runner demand: each suite dispatches several of its + # own child runs, so two suites in flight at once was enough concurrent + # runner requests that GitHub occasionally could not acquire a hosted + # runner in time and cancelled the lane. Running the remainder repos one at + # a time trades a longer remainder lane for far fewer runner-starvation + # flakes, without touching the already-serial primary/dependents/heavy + # sequencing ahead of it. remainder: name: remainder (${{ matrix.repo }}) needs: [resolve, plan, repin, heavy] @@ -453,7 +460,7 @@ jobs: actions: read strategy: fail-fast: false - max-parallel: 2 + max-parallel: 1 matrix: repo: ${{ fromJSON(needs.plan.outputs.remainder_repos) }} steps: