perf: investigate faster iOS simulator screenshots

[thymikee]

Context

SimBench compared iOS-simulator agent tools on speed and token cost using agent-device@0.17.6.

Source: https://github.com/rizwankce/SimBench
Results: https://github.com/rizwankce/SimBench/blob/main/RESULTS.md
Agent-device adapter: https://github.com/rizwankce/SimBench/blob/main/harness/adapters/agent-device.sh

In the benchmark, agent-device screenshot <path> --platform ios --udid <udid> had a warm median around 1574ms, while RocketSim reported 117ms and AXe/Mobilewright were roughly 317-350ms on the same simulator/setup.

This is only one external benchmark, but the gap is large enough to justify a focused investigation.

Goal

Reduce screenshot latency for iOS simulator sessions, or document why the current path is expected to be slower.

Investigation Notes

• Check whether iOS simulator screenshots can use a faster xcrun simctl io <udid> screenshot <path> path when no runner-specific capture behavior is required.
• Compare the current screenshot implementation path against direct simctl io screenshot for the same booted simulator and output file location.
• Preserve existing behavior for physical devices and any cases where the runner path carries required semantics.
• Measure cold and warm screenshot latency before/after with a small repeat count, then a larger repeat count if a promising path exists.
• Keep artifact/output semantics unchanged for callers: path handling, JSON output, error normalization, and diagnostics should remain compatible.

Acceptance Criteria

• A local benchmark or test note shows current iOS simulator screenshot timing and the timing of any proposed fast path.
• If a fast path is viable, implement it behind the existing screenshot command behavior without changing the public command contract.
• If not viable, document the blocker with concrete evidence.
• Add or update focused tests for path selection/error behavior where practical.

[thymikee]

This was generated by AI during triage.

Agent Brief

Category: enhancement
Summary: Investigate and, if viable, reduce iOS simulator screenshot latency without changing the public screenshot contract.

Current behavior:
An external SimBench run on agent-device@0.17.6 reports warm iOS simulator screenshot latency around 1574ms for agent-device screenshot <path> --platform ios --udid <udid>. Other tools in the same benchmark report substantially lower latencies. The repo does not currently document whether this gap is required by agent-device semantics or incidental to the current capture path. A narrow codebase search found screenshot dispatch and iOS runner screenshot routing, but no existing documented/direct simulator screenshot fast path.

Desired behavior:
For iOS simulators, measure the existing screenshot path against a direct simulator screenshot path such as xcrun simctl io <udid> screenshot <path> when runner-specific behavior is not required. If the direct path preserves artifact placement, JSON/result shape, diagnostics, and error behavior, use it for the eligible simulator case. If it is not viable, document the blocker with measured evidence.

Key interfaces:

• The public screenshot command behavior across CLI, Node, daemon, and MCP projection must remain compatible.
• Screenshot artifact placement and result payload fields must remain stable for callers.
• Error normalization and diagnostic reporting should continue to identify actionable failures.
• Platform selection must distinguish iOS simulators from physical devices and other Apple targets.

Acceptance criteria:

• Benchmark notes compare current iOS simulator screenshot latency with any candidate direct simulator path, including cold and warm samples.
• Eligible simulator screenshots use the faster path only when existing semantics are preserved.
• Physical devices and cases requiring runner-specific screenshot behavior keep the existing behavior.
• Path handling, JSON output, normalized errors, and diagnostics remain compatible.
• Focused tests cover path selection and at least one error/fallback behavior where practical.

Out of scope:

• Changing the public screenshot command contract.
• Optimizing Android, macOS, physical iOS devices, or video capture.
• Removing screenshot overlay/ref behavior or other evidence semantics.