Repository navigation
ci(cli): split the four slowest mutation scopes into parallel shards - #1007
Merged
Merged
Conversation
framework, kernel, tools and telemetry took 30 to 45 minutes each in one job, and set the wall time of every pull request. Each now runs as 3 or 4 legs in parallel: a deterministic split by file weight (heaviest first, to the lightest leg, ties by path), one incremental file and sandbox per leg, the full stryker.conf.json otherwise. A scoring job merges the legs' reports and applies the same score and floor as one run; it fails when a leg is missing, mutated other files than the split gives it at this commit, or the legs hold no mutant, and it runs even when a leg crashed. The canary and every other scope stay one job, and running a scope locally is unchanged. Measured on tools-codex, split in two for the proof: one run and two legs give the same 444 detected, 21 undetected and survivors, score 95.5 both ways. The incremental cache keys now end their scope name with --, so tools can never restore tools-codex's file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp
After the first split, the warm run's slowest legs were tools-codex (10.3 min), kernel (9.6) and translate. Their shares now run in parallel too. Kernel's heaviest file alone outweighs a quarter of the scope, so the balance check bounds a shard by a quarter over its fair share or by its heaviest file, whichever is larger. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp
blafourcade
marked this pull request as ready for review
October 12, 2026 15:46
run-mutation.mjs walked src/ on its own, beside the architecture tests' sourceFiles(). Both now use the one in mutation-shards.mjs, so the CI cuts exactly the file list the partition tests check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp
blafourcade
added a commit
that referenced
this pull request
Oct 12, 2026
Brings the sharded mutation CI (#1007). Conflict: the telemetry scope keeps this branch's globs and floor (96) and takes next's 3 shards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 What & why
The CLI CI took 45 minutes, and the mutation jobs were all of it: framework 44.6 min, kernel 35.0, tools 29.6, telemetry 29.5, everything else 15 or less (run 38247560624). Each scope ran in one job with 4 Stryker processes. This splits the four slowest scopes into parallel legs, without changing what is measured.
🛠️ How it works
mutation-scopes.jsongivesframework,toolsandkernel4 shards,telemetry3,tools-codexandtranslate2. A scope's files are cut by weight (line count): heaviest first, each to the lightest leg so far, ties by path. The same commit gives the same split on every machine. The heaviest leg stays within 1.25 of a fair share (tested for each sharded scope).stryker.conf.jsonwith only its ownmutatelist, incremental file, sandbox and report paths (run-mutation.mjs--shard i/n). A leg applies no floor.cli / Mutation (<scope>)downloads every leg's report and manifest, merges them and applies the samescoreOfand floor as one run (--aggregate). It fails when a leg is missing, when a leg mutated other files than the split gives it at this commit, or when the legs hold no mutant. It runsif: always(), so a crashed leg fails the score instead of skipping it, andgateneeds it.pnpm test:mutation:<scope>runs a scope whole, locally, as before.--:mutation-tools-used to matchmutation-tools-codex-…too.🧪 How to verify
cd cli && pnpm exec vitest run tests/architecture(164),node --test 'scripts/__tests__/**/*.test.js'(681).!globs; a leg keeps the whole config; the score fails on a missing, drifted or empty leg.tools-codexsplit in two for the proof: one run and two legs plus the score give the same 444 detected, 21 undetected and the same survivors, score 95.5 both ways.📏 Measured
The second split shortens the cold worst case; warm, it changes nothing measurable (13.8 then 15.2 min, within run-to-run spread): the slowest warm legs are now about 10 to 12 minutes whatever the split, the fixed cost of a leg (checkout, install, a dry run of the whole suite) plus the survivors Stryker reruns every time.
Scores over the shards, cold run: framework 93.0 (floor 93), kernel 96.8 (95), tools 95.2 (94), telemetry 95.4 (93). The cold run's slowest leg was kernel 3/3 (21.4 min): it holds 27 of kernel's 29 mutants that reach the 20 s timeout, which line weight does not see. The timeout is left as it is: lowering it would count a slow but passing test as a kill.
🤖 Generated with Claude Code
https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp