Skip to content

ci(acceptance): Experiment with 4 xdist workers on Arm pull requests (Experiment, do not merge) - #131164

Draft
sentry-junior[bot] wants to merge 1 commit into
masterfrom
ci/experiment-acceptance-4-xdist-workers
Draft

sentry-junior[bot] wants to merge 1 commit into
masterfrom
ci/experiment-acceptance-4-xdist-workers

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Experiment, do not merge.

Raises acceptance XDIST_WORKERS from 3 to 4 on pull requests only (Arm); master stays at 3 on x64.

4 workers were slower than 3 on x64 because the runners were CPU-saturated (#128316, #129390). Since then acceptance reuses one Chrome per worker (#129544) and pull requests moved to Arm with Chrome for Testing (#130088). 4 workers has not been tried on that setup.

via @joshuarli.

--

View Session [Sentry]

Co-Authored-By: Joshua Li <14209781+joshuarli@users.noreply.github.com>
@sentry-junior

sentry-junior Bot commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

Result: no clear gain. 4 workers came out −2.5% on acceptance test-step time vs. 3 workers (attempts: 0.0% and −4.9%). That is about the size of the run-to-run noise, and job time didn't move. Keep 3 unless more samples show otherwise.

Baseline: 20 green 10-shard (frontend-only) Arm PR acceptance runs from today on other PRs. Experiment: run 43537321952, attempts 1 and 2. Shards are compared one to one (shard i vs. the baseline median for shard i). Nearly every runner was Neoverse N2 (part=0xd49), and none of the experiment's 20 jobs landed elsewhere. I normalized the 1 baseline job on part=0xd84 by its measured speed factor (0.65).

baseline (mean ± sd, n=20) 4 workers, attempt 1 4 workers, attempt 2
summed test step 711s ± 14 710s 679s
slowest shard (test step) 92s ± 4 92s 96s
median job 195s ± 12 198s 187s
shard-paired test-time effect 0 (sd 2.1%, range −4.3%..+2.3% across runs) +0.0% (SE 1.9%) −4.9% (SE 2.1%)
CPU busy during tests 79.5% 86.7% 88.4%
CPU pressure (PSI some avg10) during tests 36 49 50
peak memory used 65.0% 68.7% 68.8%

The test step is only about 70s of a roughly 195s job, so even the better attempt saves about 3s per job. The slowest shard didn't improve. The fourth worker mostly turns idle CPU into CPU pressure. Unlike x64, Arm isn't saturated at 3 workers, but the per-shard test work is too small for a fourth worker to pay off.

@linear-code

linear-code Bot commented Nov 14, 2026

Copy link
Copy Markdown

DI-2855

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant