Performance: add a My Jetpack scenario (timing + bundle-size) to CodeVitals tracking

[LiamSarsfield]

Fixes FORMS-717

Why: My Jetpack ships the heaviest Jetpack admin bundle, but nothing tracks how long it takes to load or how big it is. This adds a My Jetpack scenario to the tools/performance CodeVitals pipeline so a load-time or bundle-size regression shows up as a trend break instead of a user complaint.

Merge gate: satisfied. The jetpack-production mirror the fixture clones now renders My Jetpack (verified 2026-07-10 against a clean checkout of mirror commit 9ef44a8: full run, 5/5 iterations, all capture guards pass). Earlier revisions held this PR back because the perf runner aborts the whole run on any scenario failure, so merging while My Jetpack rendered blank would also have stopped the already-working Dashboard/Forms posts. That precondition is now met.

Proposed changes

• Add a myJetpack scenario that measures admin.php?page=my-jetpack and posts LCP, TTFB, FCP, and bundle size (decodedBytesKB) to four new production keys (my-jetpack-connection-sim-*).
• Reuse the existing jetpack-connected Docker fixture as a page-targeted scenario, the same way formsResponses does, so no new service is added.
• Flip Jetpack offline mode off in the shared fixture so My Jetpack initializes. The fixture URL is localhost (no dot), which Jetpack reads as a local site and skips init. Two filters are needed because Status::is_offline_mode() falls back to the stored option: jetpack_offline_mode => __return_false and pre_option_jetpack_offline_mode => __return_zero (guards against a reused/dirty fixture DB).
• Add unit coverage and README docs.

Slice 1 of FORMS-717. Settings, Stats, frontend, and control scenarios stay out of scope.

One caveat: that offline flip runs install-wide, so it nudges the existing Dashboard and Forms trends by one small step at the landing commit (+9 resources, +107 KB decoded; timing stays within noise). Those keys should get a CodeVitals annotation at that commit so the step is not misread as a regression.

Related product discussion/links

• Linear: FORMS-717
• Base (merged): Performance: add a bundle-size (decodedBodySize) metric on a Forms wp-build dashboard scenario #50274

Does this pull request change what data or activity we track or use?

Yes, narrowly: it adds four internal performance-telemetry keys (my-jetpack-connection-sim-*: LCP, TTFB, FCP, decodedBytesKB) to CodeVitals. These are aggregate page-load numbers captured from a synthetic Docker fixture, extending the existing Dashboard/Forms perf trends. No real users, no PII, no cookies, no end-user tracking. No Privacy Updates label needed.

Testing instructions

All commands run from tools/performance/.

One-time setup:

cd tools/performance
pnpm install
pnpm exec playwright install chromium --with-deps   # headless browser used to load the page
pnpm calibrate                                       # normalizes CPU speed so timings are comparable across machines

1. Unit tests (no Docker needed, ~seconds):

pnpm test:unit        # expect 110 passing

2. Measure just the My Jetpack page locally, without publishing anything:

SCENARIO=my-jetpack pnpm test -- --skip-codevitals

--skip-codevitals runs the full Docker fixture and browser measurement but does not send results to the CodeVitals server, so no token is required and nothing is published. Drop SCENARIO=my-jetpack to run every scenario.

3. Inspect the payload a real post would send, without posting:

pnpm report:dry       # exits 0, prints the four keys, no token needed

Results from a local 5-iteration run against a clean mirror checkout (`9ef44a8`, 2026-07-10; medians, all in range)

metric	median	min	max	stdDev
lcp	684	656	908	93
ttfb	64	61	103	16
fcp	176	176	228	20
decodedBytesKB	5866	5866	5866	0
resources	92 (all 5 iterations)

minResourceCount is 64 (70% of the steady 92), so a truncated capture fails instead of posting an undercounted bundle size.

Known, not caused by this PR: the formsResponses scenario cannot be measured in the local fixture because its GET /wp/v2/settings request hangs server-side past 60s (with offline on or off, so the flip is not the cause; re-verified 2026-07-10 on the current mirror). Flagged so a local Forms failure is not read as a regression from this change. Fixture realism is tracked in BOOST-456.

[github-actions]

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

• ✅ Include a description of your PR changes.
  
• ✅ Add a "[Status]" label (In Progress, Needs Review, ...).
  
• ✅ Add testing instructions.
  
• ✅ Specify whether this PR includes any changes to data or privacy.
  
• ✅ Add changelog entries to affected projects
  

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

---

Follow this PR Review Process:

1. Ensure all required checks appearing at the bottom of this PR are passing.
2. Make sure to test your changes on all platforms that it applies to. You're responsible for the quality of the code you ship.
3. You can use GitHub's Reviewers functionality to request a review.
4. When it's reviewed and merged, you will be pinged in Slack to deploy the changes to WordPress.com simple once the build is done.

If you have questions about anything, reach out in #jetpack-developers for guidance!

[enejb]

Thanks for this PR.

Tthe tests are thorough (110/110 locally) and the write-up is refreshingly honest about the install-wide side effect and the mirror dependency. Left two inline notes: one worth resolving before merge, and one reminder.

[enejb]

One thing worth resolving before merge — and thank you for documenting the mirror dependency so clearly, it's what made this easy to spot.

The failure here isn't graceful. When this waitForSelector times out (which the README says is the expected state until the mirror carries #50291), measure-lcp.js still ends with process.exit( hasFailures ? 1 : 0 ), and run-performance-tests.js runs it via execFileSync — so the non-zero exit throws and the runner process.exit(1)s before the CodeVitals posting step. Net: Dashboard and Forms get measured fine but never posted — every trend gets a hole for that build, not just My Jetpack. (--allow-codevitals-failure can't rescue it either; it only guards the post step, which is never reached.)

Since HEAD is still "awaiting mirror sync," merging before the mirror updates would blank out posting for all three trends. Two ways out:

• (a) Hold the merge until the mirror carries My Jetpack: register wp-build-polyfills so the app (and Boost) loads without Gutenberg on WP < 7.0 #50291 (the stated plan) — totally fine, but nothing enforces that gate in code, so it'd help to flag it explicitly for whoever merges.
• (b) Make degradation graceful: don't let a non-baseline scenario's measurement failure block posting of the scenarios that did measure. That removes the landmine permanently rather than just for this PR.

The per-scenario try/catch already signals the intent to isolate failures — it's just the terminal process.exit(1) + the sync short-circuit in the runner that undo it for the posting phase.

[LiamSarsfield]

Thanks for the thorough trace! You've read the mechanics exactly right. One mitigating detail: the poster already skips an errored scenario and posts the rest (post-to-codevitals.js), so the runner's short-circuit is the only gap.

Good news on both fronts since your review:

1. The gate condition now holds. jetpack-production trunk picked up My Jetpack: register wp-build-polyfills so the app (and Boost) loads without Gutenberg on WP < 7.0 #50291. I reset my checkout to a clean origin/trunk and re-ran the full pipeline: My Jetpack renders and measures (median LCP 684 ms, 5/5 iterations, all four metrics in range). Forms still hits its pre-existing local hang (/wp/v2/settings stalls past 60 s; unrelated to this PR, measures fine in CI), and the dry-run poster skipped it with a warning while keeping the other keys. I've updated the README and PR description to match.

2. On (b): agreed, and I'd like to do it as a follow-up. If the runner posts survivors and still exits non-zero, a retried red build re-appends them as duplicate trend points (CodeVitals is append-only, dedup off by default). If it exits zero, a scenario failure goes silent. That trade-off deserves its own PR, likely a per-scenario optional flag. I've filed it as FORMS-728 and will pick it up right after this lands.

[enejb]

Just a reminder rather than a change request: since this flip is install-wide, please make sure the CodeVitals annotation lands on the existing wp-admin-dashboard-* / forms-responses-* keys at the landing commit — the +9 resources / +107 KB step will read as a regression later without it. Easy to drop since it's a non-code action.

Also wanted to call out the nice detail here: using __return_zero (not __return_false) for the pre_option_ filter is exactly right — pre_option only short-circuits get_option when the value is !== false, so __return_false would quietly fall through to the stored option. The inline comment explaining that is much appreciated. 🙏

[LiamSarsfield]

Noted, and it won't get dropped: annotating the existing wp-admin-dashboard-* / forms-responses-* keys at the landing commit is on the landing checklist, and the PR description calls it out in the rollout notes for the same reason.

Thanks for the kind words on __return_zero! The pre_option short-circuit was subtle enough to bite silently, so the comment felt worth the lines.

[Copilot]

Pull request overview

Adds My Jetpack page-load performance tracking into the existing tools/performance CodeVitals pipeline so regressions in load timing and overall runtime payload size are visible as trend changes (FORMS-717), reusing the existing simulated-connection WordPress fixture.

Changes:

• Introduces a new myJetpack scenario targeting wp-admin/admin.php?page=my-jetpack, capturing LCP/TTFB/FCP plus decodedBytesKB to new production CodeVitals keys.
• Updates shared safeguards/docs to reflect the new scenario and the broader applicability of the decodedBytesKB sanity range.
• Forces Jetpack offline mode off in the simulated-connection mu-plugin so My Jetpack initializes in the localhost-based fixture; adds unit coverage to pin scenario config + posting payload.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
tools/performance/scripts/scenarios.js	Adds myJetpack scenario configuration and updates decodedBytesKB sanity-range commentary to cover both Forms and My Jetpack.
tools/performance/scripts/post-to-codevitals.test.js	Extends unit tests to cover the new scenario’s metrics wiring, capture guards, and dry-run payload accumulation.
tools/performance/README.md	Documents the new scenario, its prerequisites, and updates safeguard text to include the production-key waiver for myJetpack.
tools/performance/docker/mu-plugins/simulate-wpcom-connection.php	Disables Jetpack offline mode (with option fallback defense) to ensure My Jetpack renders/initializes in the fixture.