Repository navigation
test: split browser_computer_test's multi-env tests out and refresh the shard table - #622
Merged
andrewpark-scaleai merged 3 commits intoNov 13, 2026
Merged
Conversation
earakely-scale
approved these changes
Nov 13, 2026
andrewpark-scaleai
force-pushed
the
andrewpark/split-browser-multi-env-tests
branch
from
November 13, 2026 17:55
b4ab685 to
ad38354
Compare
… own browser_computer_test.py took 1035s on main (run 43464622905), and shard 4 of the slow tier, which ran it, took 1142s of the workflow's 1160s. A file's tests run on one worker of one shard, so the file set the tier's time. The ten tests of its multi-env sections (browser_envs, a site reached by its names through the gateway, and a site wrapping an app) move unchanged to browser_multi_env_test.py. Nine of them took 461s in that run; the tenth came later in #656. The 30 that stay took 574s. The new file imports its helpers from browser_computer_test.py, as browser_gateway_test.py does, and deploys the module-scoped browser_computer fixture once more. _browser_op stays in browser_computer_test.py, since a browser recording test there uses it too.
The table had browser_computer_test.py at 304s; it took 1035s. Rewritten with .github/scripts/slow_test_durations.py from every JUnit report of run 43464622905, with the nine moved tests it timed counted under browser_multi_env_test.py. The script rewrites the whole table, so most of the diff is that regeneration, including about 70 keys the old table lacked. Projected from that run's test times, the longest slow shard's pytest time drops from 1035s to 959s (chromium_app_test.py). The projection doesn't count the new file's own build of the browser_computer fixture.
The comment still said its tests run serially; since #600 each runner runs its files on two pytest workers (-n 2 --dist loadfile).
andrewpark-scaleai
force-pushed
the
andrewpark/split-browser-multi-env-tests
branch
from
November 13, 2026 18:00
ad38354 to
7bc4ec0
Compare
andrewpark-scaleai
deleted the
andrewpark/split-browser-multi-env-tests
branch
November 13, 2026 18:23
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The goal is a faster slow tier on the same runners, with no test weakened. On main's run 43464622905, the slow tier's shard 4 took 1142s of the workflow's 1160s. Shard 4 ran
browser_computer_test.py, which took 1035s, whileslow_durations.jsonstill had it at 304s. A file's tests run on one worker of one shard, so that file set the time of the whole workflow.What
browser_computer_test.py's multi-env sections (browser_envs, and a site reached by its names through the gateway) move unchanged tobrowser_multi_env_test.py. That is ten tests, three of them added on main since this PR opened. Nine of them took 461s in that run (the tenth came later, in feat(website): a third-party app wrapped as a site, with ForwardingSite (M8.34, M8.35) #656), and the 30 that stay took 574s. The new file imports its helpers frombrowser_computer_test.py, asbrowser_gateway_test.pyalready does, and builds the module-scopedbrowser_computerfixture once more.slow_durations.jsonis rewritten with.github/scripts/slow_test_durations.pyfrom every JUnit report of run 43464622905, with the nine moved tests counted under the new file. The script rewrites the whole table, so most of the diff is that regeneration.local-backends.ymlthat still said the slow tier runs serially now says it runs two workers per runner (since ci: run the slow tier's tests two at a time on each shard #600).Expected effect
Projected from run 43464622905's test times with the new table, the longest slow shard's pytest time drops from 1035s to 959s.
chromium_app_test.pyalone is that 959s, so it is the next file to split. The projection doesn't count the new file's own fixture build. The run predates #580, which changed how the slow tier gets its computer images.Verification
pytest --collect-onlycollects the same 40 test ids across the two files that main's file had, with no node id twice.browser_gateway_test.py, which imports from the old file, still collects its 3.ruff checkpasses.

Confidence Score: 5/5
The PR appears safe to merge; no blocking issues were found.Summary
Splits the multi-environment browser tests into
browser_multi_env_test.pyso CI can schedule them separately.Reviews (3) · Last reviewed commit: "ci: say the slow tier runs on two worker..." · Reviewed by Greptile