From 6101089a26f7aaf7234f396542ef468305a9a92e Mon Sep 17 00:00:00 2001 From: fujibee Date: Tue, 18 Aug 2026 08:41:33 -0700 Subject: [PATCH 1/2] ci: cancel a PR run the next push has already replaced Measured over the last 92 PR-branch runs of this workflow: 65 of them (71%) had been overtaken by a later push to the same branch before they finished, and each was still holding four macOS runner slots while it queued. macOS is the scarce resource, and the wait is self-inflicted. Queue time to first macOS job, across the last 40 runs: macOS median 18.6 min max 167.1 min over 20 min in 19 of 40 ubuntu median 0.3 min max 32.5 min over 20 min in 1 of 40 Every one of the thirteen runs created in the busiest two-hour window waited between 20 and 167 minutes; outside it the median is 9.3. One branch alone produced fifteen runs, fourteen of them obsolete on arrival. The group is the PR number, so a push cancels only the run for the commit it replaced. Pushes to main key on the SHA instead: each is its own group, so they are neither cancelled nor serialised behind one another. --- .github/workflows/tests.yml | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 54649be2..fc7fa558 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -51,6 +51,23 @@ on: pull_request: branches: [main, integration/remote] +# A superseded run keeps its macOS jobs queued, and those are the scarce ones. +# Measured over the last 92 PR-branch runs: 65 of them (71%) had already been +# overtaken by a later push to the same branch, and each was still holding four +# macOS runner slots. In the worst two-hour window, thirteen runs piled up that +# way and every one of them waited between 20 and 167 minutes for a macOS +# runner, while ubuntu started in under a minute throughout. The wait is +# self-inflicted: we queue behind our own obsolete runs. +# +# The group key is the PR number, so a new push to the same PR cancels the run +# for the commit it replaced -- results nobody will read. Pushes to `main` fall +# back to the SHA, which makes each of them its own group: they are never +# cancelled (`cancel-in-progress` is false off the pull_request event) and never +# serialise behind each other either, which a shared group would have done. +concurrency: + group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + permissions: contents: read From 6d81b93267f399a045a11c5b8c6ca5bcf28a0314 Mon Sep 17 00:00:00 2001 From: fujibee Date: Wed, 19 Aug 2026 09:34:20 -0700 Subject: [PATCH 2/2] ci: date the queue measurements against #885 The figures in the comment were taken before #885 raised the shard timeout from 25 to 30 minutes and moved the heavy files between shards, and they have not been re-measured since. They are kept, and now say so. What #885 does not change is the mechanism: a run still asks for four macOS jobs, and a superseded run still holds them. The 71% superseded rate and the per-run job count follow from how branches are pushed and how the matrix is shaped, not from the timeout. --- .github/workflows/tests.yml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index fc7fa558..075845ca 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -52,6 +52,14 @@ on: branches: [main, integration/remote] # A superseded run keeps its macOS jobs queued, and those are the scarce ones. +# +# The numbers below are a HISTORICAL BASELINE, taken 2026-08-19 over the runs of +# this workflow that existed then -- before #885 raised the shard timeout from 25 +# to 30 minutes and moved the heavy files between shards. The queue figures have +# not been re-measured since. What #885 does not change is the mechanism this +# block addresses: a run still asks for four macOS jobs, and a superseded run +# still holds them. +# # Measured over the last 92 PR-branch runs: 65 of them (71%) had already been # overtaken by a later push to the same branch, and each was still holding four # macOS runner slots. In the worst two-hour window, thirteen runs piled up that