[shivamhwp]

Follows up on Priyansh's idea in #31 (comment). No issue to close.

What this does

Add the bench label to a PR and a Blacksmith macOS runner builds the PR and its base branch, then records with both builds. Nothing runs unless someone with write access adds the label. To run it again after new commits, remove the label and add it back.

It fails if:

• a recording breaks: nonzero exit, no file, can't decode, wrong codec, wrong size, or timestamps out of order
• peak memory grows more than 2% and 1 MB over the base build

It skips fps, CPU, and latency. The runner is a VM, so those numbers only measure the VM. Peak memory moved under 0.25% between two copies of the same binary on it, so memory is the one number worth checking there.

Changes

• bun run bench ci: new suite. It records balanced 1080p HEVC and H.264 for 5 s, 3 times per build, and checks output and memory only.
• .github/workflows/bench.yml: the label-triggered job. It reuses the Screen Recording grant from the probe branch.
• Unit tests for the new gates and the exit code.

Proof

CI runs are linked in the comments below.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
wrec	Ready	Preview	Oct 12, 2026 12:23am UTC

[greptile-apps]

Confidence Score: 2/5

[Medium impact] Adds a new benchmark suite that runs on labeled PRs.

Fix the cancellation behavior and recording checks before merging.

Findings

1.  Other labels stop benchmarks ▶
2.  Slow displays fail healthy recordings ▶
3.  Empty recordings pass ▶

Summary

Adds a label-triggered macOS benchmark that compares PR and base builds.

• Adding the bench label starts a PR benchmark run.
• The ci suite checks PR recordings and memory against the base build.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[PR label added] --> B[Workflow cancellation group]
  B --> C{Label is bench?}
  C -->|No| D[Skip job]
  C -->|Yes| E[Build base and PR]
  E --> F[Grant Screen Recording]
  F --> G[Warm up and record both builds]
  G --> H[Check output and memory]
  H --> I[Print results and set exit code]
  I --> J[Upload results]

Loading

_{Reviews (1) · Last reviewed commit: "bench: add a label-triggered PR check fo..." · Reviewed by Greptile}

[greptile-apps]

Other labels stop benchmarks

Adding an unrelated label while a benchmark runs can cancel it. Every labeled event shares this workflow-level concurrency group. Cancellation happens before the job-level bench check skips the new run, leaving the PR without benchmark results.

Move concurrency into the gated job, or include the label in the group.

[greptile-apps]

Slow displays fail healthy recordings

If the VM display cannot sustain 50 fps, the new ci suite fails even when output and memory pass. Every measured run still uses runStableCapture. Its unconditional unstableMeasurement check adds an error after three attempts below 50 fps, and statusFromGates fails the profile despite passing gates.

Skip this frame-rate prerequisite for ci so the check measures only output and memory.

[greptile-apps]

Empty recordings pass

evaluateCiGates can pass a movie with zero decoded frames. decode.swift can return an empty frames array with the track's codec and dimensions, including when reading stops before yielding a frame. deriveObserved treats the empty timestamp list as ordered and reports no stimulus rate, so retries do not reject it. The CI gates then pass if memory is unchanged, even though no frame decoded.

Require at least one decoded frame and make the decoder reject a failed read.

[shivamhwp]

Proof

This PR, labeled bench: pass. Run 38186282887 took about 4 minutes, and the bench step about 2.

balanced-1080p30-hevc: pass
  pass         pts_monotonic: true
  pass         codec_match: hevc
  pass         dimensions_match_request: 1280x720
  pass         regression_max_rss: 45.70 MB (-0.08 MB, -0.17%)
balanced-1080p30-h264: pass
  pass         pts_monotonic: true
  pass         codec_match: h264
  pass         dimensions_match_request: 1280x720
  pass         regression_max_rss: 44.80 MB (-0.08 MB, -0.17%)
status: pass

Throwaway #33, deliberately broken: fail. Run 38186336705. #33 mapped --codec h264 to HEVC and added an 8 MB buffer to the daemon. Both breaks were caught.

balanced-1080p30-hevc: fail
  fail         regression_max_rss: 53.11 MB (+8.17 MB, 18.18%)
balanced-1080p30-h264: fail
  fail         codec_match: hevc
  fail         regression_max_rss: 53.16 MB (+9.05 MB, 20.51%)
status: fail