Repository navigation
ci: shard the browser e2e suites across runners - #642
Merged
Merged
Conversation
Rinat S (rsolmano)
requested review from
Olga Lavrichenko (OLavrik),
Rustam Sadykov (SBOne-Kenobi) and
danyaberezun
as code owners
October 5, 2026 19:52
Rinat S (rsolmano)
enabled auto-merge
October 5, 2026 19:54
2 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
CI takes ~22 min on every PR and merge-queue entry (measured over the last successful runs, e.g. run 37349114307). Almost all of it is Playwright execution in three jobs, which run the same ~464 no-agent specs against three hosts:
Setup (checkout, install, Playwright, builds) is only ~1–2 min per job.
Approach
Split each browser suite across several runners and keep the ruleset's required checks stable through aggregate gate jobs.
THINKRAIL_E2E_JOB_SHARD=k/N, and its local lanes subdivide that slice into global Playwright shards(k-1)·L+iofN·L, so the jobs together cover exactly the whole suite.--shard=k/4(the runners already forward argv). The artifact configs now distribute individual tests (fullyParallel: true, stillworkers: 1), so the split is even (116 × 4) instead of file-sized.E2E (no-agent),Binary smoke & e2eandDesktop smoke & e2e. Those names now belong to small gate jobs (needs:the matrix,if: !cancelled()) that pass only when every shard succeeded, so no ruleset change is needed.Trade-off: wall time drops from ~22 to ~7–8 min (estimated from the step timings above), while total runner time grows ~20% (~62 → ~74 min per run), because each shard repeats the ~1–1.5 min setup. Running the full artifact suites only in the merge queue was considered and left out: with sharding it no longer shortens PR wall time; it would only save runner minutes.
Changes
.github/workflows/ci.yml:e2e-shard(2),binary-shard(4) anddesktop-shard(4) matrices withfail-fast: falseand per-shard report artifact names, plus the three gate jobs under the existing required names.e2e/shardPlan.ts,e2e/run.ts:parseJobShard/laneShardArgs. The runner applies the job slice in both serial and lane modes, and rejectsTHINKRAIL_E2E_JOB_SHARDcombined with an explicit--shard.playwright.config.ts:fullyParallelis also on when a job slice is set.e2e/artifactPlaywright.ts:fullyParallel: truefor binary and desktop. No spec usesbeforeAll/afterAll/describe.serial, and the source lanes already run the same specs split per test.e2e/SPEC.md,.github/SPEC.md: job-slice contract, per-test artifact sharding, and the gate-name ↔ ruleset coupling.Screenshots
Not applicable — no frontend changes.
Related issues
None.
Checklist
bun run lint,bun run typecheck,bun run testbun run e2e, orbun run e2e:fullwhen touching agent behavior) — not app-affecting (test-harness + CI only); targeted sharded runs below, and this PR's own CI runs every shardSPEC.md/ top-level specs updated to reflect any boundary, contract, or behavior changeTesting
bun run lint→ cleanbun run typecheck→ 18/18 successfulbun run test→ 18/18 tasks, 0 failbun run check:deps/check:boundaries/check:seams/check:spec-surface→ OKplaywright test e2e/shard-runner.spec.ts→ 8 passed (incl. new job-slice test)bun run e2e --list→ 470;THINKRAIL_E2E_JOB_SHARD=1/2and2/2→ 235 each; combined with--shard=1/2→ rejected (exit 1)playwright test -c playwright.{binary,desktop}.config.ts --list --shard=k/4→ 116 each of 464bun run build:binary && bun run e2e:binary --shard=1/4→ 116 passed (4.8 min)THINKRAIL_E2E_JOB_SHARD=2/2 bun run e2e→ 232 passed (8 local lanes → global shards 9–16 of 16)@action-validator/cli .github/workflows/ci.yml→ valid