Skip to content

ci(e2e): the frontend suite runs in six shards, and one report is merged from them - #1309

Merged
ExtraToast merged 1 commit into
mainfrom
ci/shard-frontend-e2e
Sep 18, 2026
Merged

ExtraToast merged 1 commit into
mainfrom
ci/shard-frontend-e2e

Conversation

@ExtraToast

@ExtraToast ExtraToast commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Closes #1306. Part of #1308.

fe-e2e was the longest job in validate.yml. Measured on run 35329092557, a green run:

fe-e2e total 13m58s
setup (checkout, node, playwright install) 37s
vite build, preview and dev server boot 38s
Playwright, 878 tests, 8 workers 12m29s
coverage conversion 8s

Almost all of it is the test run, so it shards cleanly. Six slices, using Playwright's own --shard and matching the system-tests matrix already in this workflow. --list puts 147/147/146/146/146/146 tests on the six.

What else moved, and why

The 8-worker, 1-retry pair in playwright.config.ts existed to cope with an 878-test run: oversubscription hid the tail, and the retry absorbed assertions that 8 workers on 4 vCPUs starved past the 5s expect cap. A ~146-test slice has neither problem, so workers drop to 4 — one per vCPU — and retries to 0, where a retry would only hide a real flake. That leaves trace: on-first-retry unable to ever fire, so it becomes retain-on-failure: the first failure is recorded, a green run still uploads nothing.

Coverage needed a fan-in. A shard only holds part of the picture, so it uploads its raw directory and skips the istanbul conversion through a new --no-report flag on run-e2e-with-coverage.mjs. Bare yarn test:e2e is unchanged for local use. The new fe-e2e-coverage job downloads all six and runs convert-frontend-coverage.mjs once, publishing frontend-e2e-coverage under the name the single job used to. The converter already walks its raw directory recursively and merges every JSON it finds, so the per-shard subdirectories the download leaves behind need no flattening.

What this does not claim

The run as a whole stays around 15 minutes. api-static (5m9s) → build-images api (4m43s) → system-tests (4m40s) is a 14m49s chain that finished within half a minute of fe-e2e on the run above, so it becomes the critical path the moment this lands. #1308 says so; shortening it is separate work.

Review notes

  • No branch-protection change is needed: the ruleset requires Validate complete alone, and that job gains fe-e2e-coverage in its needs.
  • --shard balances by spec file, not duration. module-smoke.spec.ts is two tests at ~35s each and the esports-* specs are heavy, so the shards may come out uneven. The per-shard times on this PR's own run are the measurement — balancing, if it is needed at all, is a follow-up.
  • Every shard boots both web servers although only the shard holding module-smoke needs the one on 4174. Same measurement decides whether that is worth separating.

Diff breakdown added removed, scaled to the largest row.

frontend                                           +20    -16    2
  build & config     ████████░░░░░░░               +20    -16    2

ci                                                 +59     -3    1
  build & config     █████████████████████████░    +59     -3    1

──────────────────────────────────────────────────────────────────
total (hand-written)                               +79    -19  3 files

@ExtraToast

Copy link
Copy Markdown
Contributor Author

First sharded run: 13m58s → ~5m15s.

shard attempt 1
1/6 4m34s
2/6 4m37s
3/6 5m14s
4/6 4m35s
5/6 4m45s
6/6 6m30s (failed)
fe-e2e-coverage 36s

40s of spread across the five green shards, so --shard's file-count split is even enough and the balancing follow-up floated in the description is not needed.

Both config changes are confirmed working by this run: fe-e2e-coverage merged six raw artifacts into one report and passed even with a failed shard, and retain-on-failure produced the trace that diagnosed the failure below.

Shard 6 failed on member manager virtualization >> scrolling reaches a member far down the list, which is not caused by this PR. It is the spec from #1173#1186#1203, and the trace shows the #1203 poll dispatching two wheels of deltaY: 10956 each because the first had not moved the scroller when the poll re-read it — 21912px against a maximum scrollTop of roughly 12800, so the list clamps at the bottom and row 250 sits off-window above. Filed as #1310.

What this PR changed is that retries: 0 stops hiding it. That is the point of the setting, so the fix belongs in its own PR off main rather than stacked here.

…ged from them

fe-e2e was the longest job in validate.yml at 13m58s, of which 12m29s was
Playwright running 878 tests. Split it six ways with Playwright's own --shard,
matching the system-tests matrix already in this workflow.

Workers drop from 8 to 4, one per runner vCPU: the oversubscription existed to
hide the tail of an 878-test run, and a ~146-test slice has none worth hiding.
The single retry went with it, since what it absorbed was assertions starved by
8 workers on 4 vCPUs rather than real flake. trace becomes retain-on-failure so
the first failure is still readable with no second attempt to record.

A shard only holds part of the coverage picture, so it uploads its raw directory
and skips conversion via a new --no-report flag. fe-e2e-coverage merges the six
and publishes frontend-e2e-coverage under the name the single job used to.

Refs #1306
@ExtraToast
ExtraToast force-pushed the ci/shard-frontend-e2e branch from 74eef43 to 5a16e6a Compare September 18, 2026 11:02
@ExtraToast
ExtraToast merged commit a0ce1be into main Sep 18, 2026
33 checks passed
@ExtraToast
ExtraToast deleted the ci/shard-frontend-e2e branch September 18, 2026 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

the frontend e2e suite runs in six shards, and one report is merged from them

1 participant