Skip to content

ci: cut the Playwright run from six minutes to three and a half #8970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
skjnldsv wants to merge 4 commits into
base: main
Choose a base branch
from
+18 −7
Open

ci: cut the Playwright run from six minutes to three and a half #8970

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
271d80e
ci: split the Playwright run over four shards
skjnldsv Sep 17, 2026
1b812b1
ci: run the Playwright shards in the Playwright image
skjnldsv Sep 17, 2026
25f0466
ci: point HOME at root inside the Playwright image
skjnldsv Sep 17, 2026
96e9be5
test: give the CI run a second worker
skjnldsv Sep 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 13 additions & 5 deletions .github/workflows/playwright.yml
View file
Open in desktop

@susnux susnux Sep 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead we can also just utilize the browser installed in github default runner image just like server does.

Also this should not be changed here but in the template repository.

@skjnldsv skjnldsv Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this should not be changed here but in the template repository.

Yeah, I plan to upstream it if we approve. But it's best to have real live data 👍

@skjnldsv skjnldsv Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead we can also just utilize the browser installed in github default runner image just like server does.

But here we test all three browsers, I'm not against narrowing it down, but that seemed on purpose. Onn server we're only testing against "chrome"

Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,22 @@ jobs:
playwright-tests:
timeout-minutes: 60
runs-on: ubuntu-latest
# Ships the browsers and the system libraries they need, which is the
# `playwright install --with-deps` step this job used to run. The tag
# has to match the `playwright-core` version resolved by the lockfile,
# or Playwright downloads a second set of browsers at runtime.
container:
image: mcr.microsoft.com/playwright:v1.62.1-noble
env:
# The steps run as root while the runner's own /github/home belongs
# to the image's pwuser, and Firefox refuses to launch when $HOME is
# owned by somebody else
HOME: /root
strategy:
fail-fast: false
matrix:
shardIndex: [1, 2]
shardTotal: [2]
shardIndex: [1, 2, 3, 4]
shardTotal: [4]
outputs:
node-version: ${{ steps.versions.outputs.node-version }}
package-manager-version: ${{ steps.versions.outputs.package-manager-version }}
Expand All @@ -50,9 +61,6 @@ jobs:
- name: Install dependencies
run: npm ci

- name: Install Playwright browsers
run: npx playwright install --with-deps

- name: Run Playwright tests
run: npm run test:component -- --shard='${{ matrix.shardIndex }}/${{ matrix.shardTotal }}'

Expand Down
7 changes: 5 additions & 2 deletions playwright.config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,11 @@ export default defineConfig({
forbidOnly: !!process.env.CI,
/* Retry on CI only */
retries: process.env.CI ? 2 : 0,
/* Opt out of parallel tests on CI. */
workers: process.env.CI ? 1 : undefined,
/* Two of the runner's four cores on CI, which is where the suite stops
scaling: measured over the whole suite, one worker takes 6m19, two
take 4m49 and four take 4m22, and four workers is also where a
second test starts flaking. */
workers: process.env.CI ? 2 : undefined,

// On CI we are using the github annotations + blob which will be merged by workflow to a downloadable HTML report (like the one we receive locally)
reporter: process.env.CI ? 'blob' : 'html',
Expand Down
Loading