perf(mutation): stop booting a Postgres container for every mutant - #100
Merged
Conversation
Stryker was spending most of its runtime starting databases, not testing mutants. A 9-file diff kept it busy for 30+ minutes on the CI runner, with 18 Postgres containers alive at once and the box swapping. Cause: Stryker runs vitest once per mutant batch across 'concurrency' workers (default: CPU cores - 1, so 9 on the Mac mini). vitest.config.ts declares globalSetup, which boots a testcontainers Postgres AND replays the whole migration chain -- and that fires on every worker. vitest.related already limits which tests run, but globalSetup runs regardless of which tests are selected, so Postgres booted even to run money.test.ts. Measured, one test file with 29ms of tests: 6.66s -> 1.37s. The full 53-file DB-free unit suite now runs in 6.47s with zero containers. Stryker on money.ts: dry run 24s/192 tests -> 3s/111 tests, mutation phase 2m42s -> 1m52s, peak containers 7 -> 0. The gap is far wider on the 16GB mini. Fix: a Stryker-only vitest config via the supported vitest.configFile option, with no globalSetup, excluding tests/ and the five colocated src tests that need a live database. Adds ignoreStatic, which skips mutants that only run at file load -- the ones that force an environment reload. vitest.config.ts is untouched, so pnpm test and CI's test job are unaffected (113 files / 751 tests still pass). Mutation score on money.ts moves 77.32% -> 75.26%, and the composition is better than the number suggests: before: 71 killed, 4 timeout, 22 survived after: 73 killed, 0 timeout, 24 survived It kills more mutants outright. The old run's 4 timeouts -- which Stryker counts as kills -- were mutants timing out on container contention, not tests catching them. Part of the old score was infrastructure noise scored as success. Mutants covered only by DB-backed tests now report NoCoverage. That is honest signal about unit-test strength, and it makes ci.yml's claim that Stryker 'runs just the unit suite' true -- it was not before. Note: coverageAnalysis is kept for documentation only. The vitest runner ignores it and always uses perTest, but ignoreStatic documents perTest as a prerequisite.
This was referenced Aug 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split out from #99 — that PR is pure CI config; this one changes mutation scores and deserves its own review.
Problem
Stryker was spending most of its runtime starting databases, not testing mutants. A 9-file diff kept the mutation job busy for 30+ minutes on the CI runner, with 18 Postgres containers alive simultaneously and the 16GB mini swapping (991k pageouts).
Cause
Three things compound:
concurrencyworkers — default iscpuCores - 1, so 9 on the Mac mini.vitest.config.tsdeclaresglobalSetup, which boots a testcontainers Postgres and replays the entire migration chain. That fires on every worker.vitest.related(defaulttrue) already limits which tests run — butglobalSetupruns regardless of which tests are selected. So Postgres booted even to runmoney.test.ts.Net effect: 8.92s of database bootstrap to run 29ms of tests.
Fix
A Stryker-only vitest config via the supported
vitest.configFileoption, with noglobalSetup, excludingtests/and the five colocatedsrctests that need a live database. PlusignoreStatic: true, which skips mutants that only execute at file load — precisely the ones that force an environment reload.vitest.config.tsis untouched, sopnpm testand CI's test job are unaffected.Measured
money.ts— dry runmoney.ts— mutation phaseMeasured on an M4 Pro (14 cores, 48GB). The gap is far wider on the mini — 10 cores and 16GB with swapping is where 7 containers became 18 and minutes became 30.
The score change, stated plainly
Mutation score on
money.tsmoves 77.32% → 75.26%. The composition is better than the number:It kills more mutants outright. The old run's 4 timeouts — which Stryker counts as kills — were mutants timing out on container contention, not tests catching them. Part of the old score was infrastructure noise scored as success.
Mutants covered only by DB-backed tests now report
NoCoverage. That's honest signal about unit-test strength, and it makesci.yml's claim that Stryker "currently runs just the unit suite" true — it wasn't before.Verification
pnpm typecheck— cleanpnpm lint— 0 errors (1 warning, pre-existing onmain, verified by stashing)pnpm test— 113 files / 751 tests pass in 32smoney.ts, numbers aboveNote
coverageAnalysis: "perTest"is kept for documentation only — the vitest runner ignores it and always uses perTest, butignoreStaticdocuments perTest as a prerequisite.🤖 Generated with Claude Code