ci(api): the integration suite runs in six shards, and the floors read all six - #1322
Conversation
…d all six api-tests was 8m20s, of which the integration suite was 5m16s run serially in one JVM over 121 classes. Split it: unit tests get their own job with no database at all, and the integration suite shards six ways on the same stable FQCN hash :tests:system and the frontend e2e suite already use. Each shard gets its own MariaDB service. The fixtures collide on unique fields, so the suite cannot share one database across parallel writers. The coverage floors read a single integrationTest.exec, and a shard's file covers only its own slice, so no floor holds against one alone. Shards upload their execution data and api-coverage merges the six: -PjacocoExecDir points the report and both floors at that directory and lifts the dependency that would otherwise re-run the suite. The floors stay as strict as they were. The shard filter is applied in doFirst, which Gradle does not hash, so the shard number is declared as a task input. Without it every shard has identical inputs and one shard's result is served to the next FROM-CACHE. ktlint and detekt move out of api-static into their own job. They are source-only and nothing waits on them, so 46s leaves the critical path that every test job sits behind. Refs #1318
…y one refuses `fileTree(dir)` inside :services:api resolves against services/api, so -PjacocoExecDir=services/api/build/jacoco/shards looked under services/api/services/api/ and found nothing. Resolve against the root instead. Finding nothing was not loud. JacocoReport with empty execution data goes NO-SOURCE and JacocoCoverageVerification passes with nothing to measure, so a wrong path reads as a green coverage gate — only the report upload failing on if-no-files-found made it visible. The merged mode now refuses an empty set rather than measuring it. Verified against the six .exec files the first run produced: merged instruction coverage 0.6851 against a floor of 0.40, and the report is written.
|
First run: every shard green, the fan-in caught a real defect in my own wiring.
The unsharded The defect
Fixed in 061c20b by resolving against the root, and by refusing an empty set rather than measuring it: Verified locally against the six Shard countPer-shard overhead is 2–4 minutes and varies more than the test content does — shard 2 took 4m55s to run 31.9s of tests. Six shards may be more than this suite is worth once #1319 removes the 68.5s class that shard 4 is pinned by. Worth revisiting with a second data point rather than now. |
Closes #1318. Part of #1321.
api-testswas 8m20s and, once #1309 lands, the longest thing left invalidate.yml. Measured on run 35332899202:The integration suite ran serially in one JVM — a single
Test worker, nomaxParallelForks, no JUnit parallelism.What this does
Splits the job in two. Unit tests need no MariaDB service at all, so they get their own job and stop sitting in front of the integration suite.
Shards the integration suite six ways, on the same stable FQCN hash
:tests:systemand the frontend e2e suite use. Each shard gets its own MariaDB service — the fixtures collide on unique fields, somaxParallelForksagainst one shared database was never an option.Moves the coverage floors to a fan-in.
jacocoIntegrationTestCoverageVerificationreads a singleintegrationTest.exec, and a shard's file covers only its slice, so no floor holds against one alone. Shards upload their execution data;api-coveragemerges the six.-PjacocoExecDirpoints the report and both floors at that directory, and lifts thedependsOnthat would otherwise re-run the whole suite inside the fan-in job. The floors from #1282 keep their exact numbers.Moves ktlint and detekt out of
api-static. Source-only, nothing waits on them, 46s off the path every test job sits behind.The cache trap this had to avoid
The shard filter is applied in
doFirst, which Gradle does not hash. Left alone, all six shards present identical task inputs, and a cacheableTesttask under--build-cachewithcache-read-only: falsecan serve one shard's result to the next asFROM-CACHE— six green shards where one suite ran. So the shard number is declared::tests:systemhas the same shape and does not declare them. Not touched here; filed as #1320.Expected shard times
Partitioning the 121 classes by the hash, against their measured durations:
Bounded by shard 4, of which
RecoveryControllerEmailITalone is 68.5s for 7 tests. That class is the floor on what sharding can reach here and is filed separately as #1319.Review notes
Validate completegainsapi-lint,api-unit-tests,api-integration-testsandapi-coverage, and stays the only required check, so no branch-protection edit is needed.integrationTestwithoutSHARD_TOTAL/SHARD_INDEXset behaves exactly as before, so a local run is unchanged.-PjacocoExecDiris the only thing that lifts thedependsOn. A developer runningjacocoIntegrationTestCoverageVerificationlocally still gets the suite run for them.api-testsstarted 3m18s after its dependencies finished on the run measured above, so the queue is worth watching — the per-shard times and the queue delay on this PR's own run are the measurement, and the shard count comes down if the parallelism is not being served.Diff breakdown —
█added░removed, scaled to the largest row.