Build: raise the timeout for full-monorepo builds - #51029
Conversation
full builds need ~26 min; the 25 min cap cancels them during archiving
|
Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.
Interested in more tips and information?
|
|
Thank you for your PR! When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:
This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖 Follow this PR Review Process:
If you have questions about anything, reach out in #jetpack-developers for guidance! |
manzoorwanijk
left a comment
There was a problem hiding this comment.
This is fine but the time taken by these builds concerns me a bit. We need to do something about it.
|
Yes, we'll continue with WordPress/gutenberg#79889 |
Yeah we agree, it actually surprised me how much it took compared to the total build time of the projects. |
Proposed changes
Raise
timeout-minuteson thebuild_alljob in.github/workflows/build.ymlfrom 25 to 40. The comment next to it said successful runs take about 15 minutes; that was written in April and is no longer true.Most PRs rebuild only what they touch and never get near the limit. A PR that modifies
pnpm-lock.yamlor a rootpackage.jsonmarks every project as changed, sobuild_allrebuilds all 140 of them, and those runs now finish with seconds to spare, when they finish at all. Measured on #50509 on 2026-08-04:The last successful run spent 73 seconds on setup, 20m30s building and 2m03s creating the archive. The build itself has room; the archive is what pushes the job over the edge, and a 21 second margin is no margin at all: a slower runner or a slightly larger archive cancels the job.
A cancelled build costs more than a re-run. On a PR it also takes down the two TeamCity checks, which report "Build failed" seconds later, so one cause produces three red checks. On trunk it means
Push to mirror reposnever runs, so no mirror repo advances for that commit and everything that releases from the mirrors sits on the previous state until a later build succeeds.build_wpcomandbuild_nonwpcomcarry the same value and the same stale comment, but they run on smaller matrices and show no sign of pressure, so they stay as they are.Follow-up
The critical path is
premium-analytics: 16m06s of the 20m30s build, with the next project at 11m53s. Its 76 widgets and 6 routes fan out into roughly 164 esbuild jobs with no global cap. WordPress/gutenberg#79889 adds that cap upstream and is awaiting review; once it ships, this timeout gets real headroom back. Until then the pressure grows rather than eases: Premium Analytics went from 51 to 76 widgets in a month, and the 2m15s that #50964 saved by routing shared libraries through one externals module was a one-off gain, not a change in slope.Related product discussion/links
Does this pull request change what data or activity we track or use?
No.
Testing instructions
Nothing to test on a site. Confirm the diff touches only the
build_alltimeout in.github/workflows/build.yml, and that CI here is green; this PR runs a partial build, so it never exercises the new value. The value earns its keep on any PR that rebuilds everything, such as a dependency bump touchingpnpm-lock.yaml: under the old cap, anything above roughly 24 minutes gets cancelled.