ci(e2e): the frontend suite runs in six shards, and one report is merged from them - #1309
Conversation
|
First sharded run: 13m58s → ~5m15s.
40s of spread across the five green shards, so Both config changes are confirmed working by this run: Shard 6 failed on What this PR changed is that |
…ged from them fe-e2e was the longest job in validate.yml at 13m58s, of which 12m29s was Playwright running 878 tests. Split it six ways with Playwright's own --shard, matching the system-tests matrix already in this workflow. Workers drop from 8 to 4, one per runner vCPU: the oversubscription existed to hide the tail of an 878-test run, and a ~146-test slice has none worth hiding. The single retry went with it, since what it absorbed was assertions starved by 8 workers on 4 vCPUs rather than real flake. trace becomes retain-on-failure so the first failure is still readable with no second attempt to record. A shard only holds part of the coverage picture, so it uploads its raw directory and skips conversion via a new --no-report flag. fe-e2e-coverage merges the six and publishes frontend-e2e-coverage under the name the single job used to. Refs #1306
74eef43 to
5a16e6a
Compare
Closes #1306. Part of #1308.
fe-e2ewas the longest job invalidate.yml. Measured on run 35329092557, a green run:fe-e2etotalplaywright install)Almost all of it is the test run, so it shards cleanly. Six slices, using Playwright's own
--shardand matching thesystem-testsmatrix already in this workflow.--listputs 147/147/146/146/146/146 tests on the six.What else moved, and why
The 8-worker, 1-retry pair in
playwright.config.tsexisted to cope with an 878-test run: oversubscription hid the tail, and the retry absorbed assertions that 8 workers on 4 vCPUs starved past the 5sexpectcap. A ~146-test slice has neither problem, so workers drop to 4 — one per vCPU — and retries to 0, where a retry would only hide a real flake. That leavestrace: on-first-retryunable to ever fire, so it becomesretain-on-failure: the first failure is recorded, a green run still uploads nothing.Coverage needed a fan-in. A shard only holds part of the picture, so it uploads its raw directory and skips the istanbul conversion through a new
--no-reportflag onrun-e2e-with-coverage.mjs. Bareyarn test:e2eis unchanged for local use. The newfe-e2e-coveragejob downloads all six and runsconvert-frontend-coverage.mjsonce, publishingfrontend-e2e-coverageunder the name the single job used to. The converter already walks its raw directory recursively and merges every JSON it finds, so the per-shard subdirectories the download leaves behind need no flattening.What this does not claim
The run as a whole stays around 15 minutes.
api-static(5m9s) →build-imagesapi (4m43s) →system-tests(4m40s) is a 14m49s chain that finished within half a minute offe-e2eon the run above, so it becomes the critical path the moment this lands. #1308 says so; shortening it is separate work.Review notes
Validate completealone, and that job gainsfe-e2e-coveragein itsneeds.--shardbalances by spec file, not duration.module-smoke.spec.tsis two tests at ~35s each and theesports-*specs are heavy, so the shards may come out uneven. The per-shard times on this PR's own run are the measurement — balancing, if it is needed at all, is a follow-up.module-smokeneeds the one on 4174. Same measurement decides whether that is worth separating.Diff breakdown —
█added░removed, scaled to the largest row.