Skip to content

Build: raise the timeout for full-monorepo builds - #51029

Merged
retrofox merged 1 commit into
trunkfrom
chore/build-timeout-full-builds
Aug 5, 2026
Merged

Build: raise the timeout for full-monorepo builds#51029
retrofox merged 1 commit into
trunkfrom
chore/build-timeout-full-builds

Conversation

@retrofox

@retrofox retrofox commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Proposed changes

Raise timeout-minutes on the build_all job in .github/workflows/build.yml from 25 to 40. The comment next to it said successful runs take about 15 minutes; that was written in April and is no longer true.

Most PRs rebuild only what they touch and never get near the limit. A PR that modifies pnpm-lock.yaml or a root package.json marks every project as changed, so build_all rebuilds all 140 of them, and those runs now finish with seconds to spare, when they finish at all. Measured on #50509 on 2026-08-04:

Run Result Duration Margin to the 25 min cap
30910793192 cancelled 25m18s hit the cap
30919473627 success 26m42s ran under a temporary 40 min cap, would have been cancelled at 25
30923362248 success 24m39s 21s
30927805943 success 23m56s 64s

The last successful run spent 73 seconds on setup, 20m30s building and 2m03s creating the archive. The build itself has room; the archive is what pushes the job over the edge, and a 21 second margin is no margin at all: a slower runner or a slightly larger archive cancels the job.

A cancelled build costs more than a re-run. On a PR it also takes down the two TeamCity checks, which report "Build failed" seconds later, so one cause produces three red checks. On trunk it means Push to mirror repos never runs, so no mirror repo advances for that commit and everything that releases from the mirrors sits on the previous state until a later build succeeds.

build_wpcom and build_nonwpcom carry the same value and the same stale comment, but they run on smaller matrices and show no sign of pressure, so they stay as they are.

Follow-up

The critical path is premium-analytics: 16m06s of the 20m30s build, with the next project at 11m53s. Its 76 widgets and 6 routes fan out into roughly 164 esbuild jobs with no global cap. WordPress/gutenberg#79889 adds that cap upstream and is awaiting review; once it ships, this timeout gets real headroom back. Until then the pressure grows rather than eases: Premium Analytics went from 51 to 76 widgets in a month, and the 2m15s that #50964 saved by routing shared libraries through one externals module was a one-off gain, not a change in slope.

Related product discussion/links

Does this pull request change what data or activity we track or use?

No.

Testing instructions

Nothing to test on a site. Confirm the diff touches only the build_all timeout in .github/workflows/build.yml, and that CI here is green; this PR runs a partial build, so it never exercises the new value. The value earns its keep on any PR that rebuilds everything, such as a dependency bump touching pnpm-lock.yaml: under the old cap, anything above roughly 24 minutes gets cancelled.

full builds need ~26 min; the 25 min cap cancels them during archiving
@retrofox
retrofox requested a review from a team as a code owner August 4, 2026 14:30
@retrofox retrofox added the [Status] Needs Review This PR is ready for review. label Aug 4, 2026
@retrofox retrofox self-assigned this Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

  • To test on WoA, go to the Plugins menu on a WoA dev site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin (Jetpack or WordPress.com Site Helper), and enable the chore/build-timeout-full-builds branch.
  • To test on Simple, run the following command on your sandbox:
bin/jetpack-downloader test jetpack chore/build-timeout-full-builds
bin/jetpack-downloader test jetpack-mu-wpcom-plugin chore/build-timeout-full-builds

Interested in more tips and information?

  • In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
  • Read more about our development workflow here: PCYsg-eg0-p2
  • Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

@github-actions github-actions Bot added the Actions GitHub actions used to automate some of the work around releases and repository management label Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

  • ✅ Include a description of your PR changes.
  • ✅ Add a "[Status]" label (In Progress, Needs Review, ...).
  • ✅ Add testing instructions.
  • ✅ Specify whether this PR includes any changes to data or privacy.
  • ✅ Add changelog entries to affected projects

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖


Follow this PR Review Process:

  1. Ensure all required checks appearing at the bottom of this PR are passing.
  2. Make sure to test your changes on all platforms that it applies to. You're responsible for the quality of the code you ship.
  3. You can use GitHub's Reviewers functionality to request a review.
  4. When it's reviewed and merged, you will be pinged in Slack to deploy the changes to WordPress.com simple once the build is done.

If you have questions about anything, reach out in #jetpack-developers for guidance!

@retrofox
retrofox requested a review from louwie17 August 4, 2026 14:44

@manzoorwanijk manzoorwanijk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine but the time taken by these builds concerns me a bit. We need to do something about it.

@retrofox
retrofox merged commit f463864 into trunk Aug 5, 2026
78 checks passed
@retrofox
retrofox deleted the chore/build-timeout-full-builds branch August 5, 2026 07:09
@github-actions github-actions Bot removed the [Status] Needs Review This PR is ready for review. label Aug 5, 2026
@retrofox

retrofox commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

Yes, we'll continue with WordPress/gutenberg#79889

@louwie17

louwie17 commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

This is fine but the time taken by these builds concerns me a bit. We need to do something about it.

Yeah we agree, it actually surprised me how much it took compared to the total build time of the projects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Actions GitHub actions used to automate some of the work around releases and repository management

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants