Skip to content

perf: reuse Apple runner cache across version bumps#900

Open
thymikee wants to merge 5 commits into
mainfrom
perf/apple-runner-build-time
Open

perf: reuse Apple runner cache across version bumps#900
thymikee wants to merge 5 commits into
mainfrom
perf/apple-runner-build-time

Conversation

@thymikee

@thymikee thymikee commented Jun 27, 2026

Copy link
Copy Markdown
Member

Summary

Reduce Apple runner cold-build and first-use work while keeping cache invalidation reliable.

Shipped changes:

Area Before After
Runtime runner Swift surface Runtime code and Swift unit-test helpers compiled together Unit-test-only Swift is behind AGENT_DEVICE_RUNNER_UNIT_TESTS
Asset catalog Tiny branding assets still ran actool during cold runner builds Assets.xcassets and bundled branding images removed
Runner cache key Invalidated on package version bumps Reused across package version bumps; still keyed by toolchain, SDK, source fingerprint, platform, and build flags
Maintainer/CI script destination generic/platform=iOS Simulator, which can build arm64 + x86_64 Concrete available simulator when possible, with 3s fallback to generic
Swift unit coverage Coupled to build:all Separate CI compile job for the guarded Swift unit-test surface

Measurements

Measured on Xcode 26.2 with fresh DerivedData per run.

Scenario Runs Before median / mean After median / mean Delta
User runner xcodebuild build-for-testing, iPhone 17 Pro Max simulator 5 before + 5 after 8.06s / 7.82s 7.76s / 7.75s -3.7% median
First use: open settings then first snapshot -i, no runner cache 1 before + 1 after 31.5s 21.4s -32% wall time
pnpm build:xcuitest iOS script destination 5 before + 5 after 8.210s / 8.042s 5.767s / 5.825s -29.8% median

Notes:

  • The user runner build-only win is modest because Swift frontend work and Xcode warm caches dominate after the first run, but the first-use CLI path is materially faster.
  • The script destination win applies to maintainer/CI build:xcuitest; runtime/user builds already used concrete simulator destinations.
  • Package dry run stayed small: 161 files, 568 KB packed / 1.97 MB unpacked, with no asset catalog or branding images.

Rejected Experiments

I also tested wholemodule + -num-threads, debug info off / Swift -gnone, explicit modules off, testability off, incremental off, lazy recording compilation, and related combinations. None met the shipping bar for runtime builds: either the median win was noise-level, wall time got worse, or the change added complexity/semantic risk.

Validation

  • pnpm check:quick: passed
  • pnpm exec vitest run src/platforms/ios/__tests__/runner-client.test.ts src/platforms/ios/__tests__/runner-xctestrun.test.ts src/platforms/ios/__tests__/runner-icon.test.ts: 79 tests passed
  • pnpm build:xcuitest: passed for iOS and macOS
  • npm pack --dry-run --ignore-scripts --json --cache /private/tmp/agent-device-npm-cache: verified package contents and asset removal
  • AGENT_DEVICE_XCUITEST_INCLUDE_UNIT_TESTS=1 ... pnpm build:xcuitest:macos: passed earlier and showed -D AGENT_DEVICE_RUNNER_UNIT_TESTS

Latest pushed SHA: 5cce08d8d.

@github-actions

github-actions Bot commented Jun 27, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.4 MB 1.4 MB +167 B
JS gzip 445.5 kB 445.6 kB +52 B
npm tarball 584.7 kB 545.9 kB -38.8 kB
npm unpacked 2.0 MB 1.9 MB -38.9 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 28.6 ms 28.2 ms -0.4 ms
CLI --help 48.7 ms 50.1 ms +1.4 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/9722.js +167 B +52 B

@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstack.github.io/agent-device/pr-preview/pr-900/

Built to branch gh-pages at 2026-06-27 12:53 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@thymikee thymikee force-pushed the perf/apple-runner-build-time branch 2 times, most recently from e2897b2 to 1799d0d Compare June 27, 2026 13:37
@thymikee thymikee force-pushed the perf/apple-runner-build-time branch from b821241 to 10ef1ef Compare June 27, 2026 19:05
@thymikee

Copy link
Copy Markdown
Member Author

Reviewed the latest head, including the new concrete-simulator destination commit.

I do not see a blocker. The default iOS/tvOS script path now prefers an available concrete simulator id, but keeps the generic simulator fallback when simctl lookup fails. Cache metadata still normalizes the destination back to the simulator family, so choosing a specific UDID should not churn runner cache keys. The unit-test Swift flag is now explicitly represented in both the shell build path and metadata comparison, so runtime and unit-test runner variants stay separated. Current CI is green, including Swift Runner Unit Compile, typecheck, unit, integration, smoke, and iOS runner compatibility.

Residual risk: this is still Apple runner build/cache behavior, so I would treat the PR body’s local Xcode measurements and first-use validation as the device-facing evidence rather than relying on fixture tests alone. With that evidence plus green CI, this is ready for maintainer merge judgment.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Jun 28, 2026
@thymikee thymikee force-pushed the perf/apple-runner-build-time branch from bb03273 to 5cce08d Compare June 28, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant