Record how long CI takes, so a change in it is visible - #838
Merged
Conversation
There was no way to answer "has CI got slower". The web UI shows one run at a time
and reports created-to-finished, which folds in how long the run waited for a
runner; on a busy afternoon that reads as a regression when nothing about the code
changed. Run history is kept for a bounded window and logs for ninety days, so the
question also gets harder to answer the longer it goes unasked -- the January 2026
logs had already expired when this was written, and only the job timestamps
survived to be recovered.
So: dotnet run CiTiming, from Sources/Utils/Utils.
It separates the two durations, because they answer different questions. Exec is
runner-picks-the-job-up to job-finishes, which is what moves when a suite grows.
Queue is run-created to runner-picks-it-up, which moves when several pull requests
are in flight and says nothing about the code.
The baseline is committed rather than derived, for the retention reason above, and
every earlier snapshot is kept when a new one is recorded. Two are on file: today,
and 22 January 2026, recovered from the job timestamps before that window closes.
What it says today, against January:
C# Test / Test (ubuntu-latest) 10.0m +56%
C# Test / Test (windows-latest) 13.5m +52%
C# Test / Test (macos-latest) 10.5m +15%
Against 3129 to 5118 test cases over the same period, +64%. So the growth is
sublinear -- time per case fell about a fifth -- and macOS looks better than its
neighbours because those runners moved to ARM, not because less is run there.
Nothing structural was found to reclaim. The build steps are under a minute, and
the test step is the job.
gh does the talking, so no token lives in this repository.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
There was no way to answer "has CI got slower?". The web UI shows one run at a time and reports created-to-finished, which folds in how long the run waited for a runner — on a busy afternoon that reads as a regression when nothing about the code changed.
It also gets harder the longer it goes unasked: run history is kept for a bounded window and logs for ninety days. The January 2026 logs had already expired when this was written; only the job timestamps survived to be recovered.
Usage
ghdoes the talking, so no token lives in this repository.Two durations, because they answer different questions
What it says today
C# Test / Test (windows-latest)C# Test / Test (macos-latest)C# Test / Test (ubuntu-latest)Test cases over the same period went 3129 → 5118 (+64%), so the growth is sublinear — time per case fell about a fifth. macOS looks better than its neighbours because those runners moved to ARM, not because less is run there.
Aggregate: 76 minutes of runner time per commit; the slowest single job, which is what a contributor actually waits for, is 13.5m.
Nothing structural to reclaim
I went looking for waste and did not find much worth acting on — recording the negatives so nobody re-derives them:
Build Testsstep is 0.2–0.9m, not a hidden second build.parallelizeAssembly: falseis a no-op with a single test assembly; collections already run in parallel.The one genuinely dead setting: ubuntu and macOS pass
/p:CollectCoverage=true, which produces a report that every upload step skips (if: windows-latest). It costs about a second, so this PR leaves it alone rather than touching the workflow.Baseline is committed, and accumulates
Two snapshots are on file — today, and 22 January 2026 recovered before that window closed. Recording a new one appends the old to
historyrather than replacing it, so a slow drift over many small steps stays visible.