Detect host sleep interruptions in benchmark evidence

[deverman]

User outcome

Release and reliability benchmark operators immediately learn that host sleep invalidated a run, instead of spending time diagnosing apparent Bridge timeouts after the suite finishes.

Validation impact

performance

User-facing acceptance journey

1. Start a benchmark profile with a recorded host power-state baseline.
2. The Mac sleeps or enters clamshell sleep during a measured phase.
3. The suite records the interruption and fails closed with a direct host-sleep diagnosis.
4. The summary explains that caffeinate cannot override lid-close sleep and states the safe retry requirement.

Observed sanitized evidence

A frozen-candidate release profile reported one Bridge timeout in each tool phase. Power-management logs aligned those failures with clamshell and maintenance sleep, including a roughly 660-second sleep corresponding to a roughly 665-second call. The caffeinate process held system and idle-sleep assertions, but lid-close sleep still occurred.

No OmniFocus content, identifiers, or client transcript is included.

Requirements

• Record enough monotonic and wall-clock timing to detect a suspension gap during every benchmark phase.
• Capture a sanitized pre/post power-state or sleep/wake marker without requiring raw system logs in published evidence.
• Fail closed with a distinct host_sleep_interruption classification when suspension overlaps warmup or measured coverage.
• Preserve the original call failure and timeout evidence; do not relabel a genuine timeout when no suspension overlaps it.
• Explain in the CLI summary and release checklist that caffeinate prevents idle sleep but not clamshell sleep.
• Keep raw artifacts under .build/benchmarks; commit only intentional documentation.

Tests

• Deterministic injected-clock test classifies a suspension gap.
• Ordinary slow Bridge calls without a wall/monotonic discontinuity remain Bridge timeouts.
• Result validation rejects a suite containing a host-sleep interruption.
• Sanitized summaries contain no request payloads or OmniFocus data.

Non-goals

• Preventing macOS clamshell sleep
• Automatically retrying a failed suite
• Weakening fail-closed release criteria
• Changing production Bridge deadlines or request handling

[deverman]

Priority decision:

• Do not add this tooling change to the frozen v0.12.0-beta production candidate.
• Use the existing manual mitigation for the one permitted retry: AC power, lid open, caffeinate, and continuous heartbeat monitoring.
• Schedule Detect host sleep interruptions in benchmark evidence #193 immediately after Release FocusRelay v0.12.0-beta: reliable inbox workflows #188 and before the next product item (Fetch a bounded set of task details by stable ID in one query #172), so the next release cannot waste another long certification run on an unclassified host-sleep interruption.

This remains priority:P2 because it affects release-evidence reliability, but it is not a reason to mix new benchmark code into the already frozen candidate.

[deverman]

Decision: close as not planned

We are choosing a controlled operating procedure instead of adding clock and power-state classification code after one sleep-interrupted release run.

PR #215 updates the release runbook so the 1.5-hour release profile starts under caffeinate -is, on AC power, with a MacBook lid kept open. An overnight unattended window is explicitly supported. Existing validation still fails closed on timeouts, errors, incomplete scenario coverage, and malformed or missing artifacts, and at most one retry is allowed after removing an environmental cause.

This keeps host power-state diagnosis out of the benchmark implementation. Reopen this issue only if repeated controlled runs still produce ambiguous evidence that the existing artifacts cannot diagnose.