Add GPT Live voice with LoopHarness reasoning and tools

[ashbhat]

Adds GPT Live 1 voice conversations on iPhone and Mac, with reasoning and tools delegated to the selected LoopHarness model through the existing guarded dispatcher. A waveform button in an empty composer starts a call. iPhone uses a header orb, compact call controls, and a blue border that responds to spoken output; Mac uses a floating orb. Active iPhone calls support background audio.

Voice transcripts stream into blue-attributed rows. Each delegated request has an amber reasoning and tools card with model attribution, progress, expandable inputs/results, timing, and saved metadata. Read-only work collapses after speech, while actions and errors remain visible. Image-search thumbnails and source links appear in the same cards, and generated images update from placeholders to results or failures. Images remain visible with collapsed details and can be enlarged. Existing workspace images returned by share_file are included as well. Late image completion updates the original conversation after a call ends.

Tapping a reasoning or tool area toggles its disclosure. Manual expansion is retained and animations are disabled. Live results, including images, continue rendering during inspection, with automatic following disabled and the visible message anchored to prevent scroll jumps. Closing a card does not auto-scroll. Streaming reconfigures only changed rows when IDs are stable; a bounded shared image loader retains decoded images and in-flight requests across cell reconstruction to prevent flashing.

Uses the user's OpenAI credential and the Live WebSocket/client-delegation API. Includes a startup timeout, nonfatal slow-task progress, bounded audio queues, opaque delegation ID handling, correction handling, and cancellation guards. Mac capture negotiates matching hardware I/O formats to address audio initialization error -10875 and empty microphone capture. iPhone overlay ownership follows the UIKit controller hierarchy to fix the reported Live startup crashes.

iOS PDF generation stays in the originating Live conversation, avoiding the ordinary chat host that could create a new conversation and end the call. PDF placeholders update in place; completed files persist, late results stay with the original call, and failed renders can be retried. Delegated tasks running beyond two minutes keep the call and original request alive instead of disconnecting or replaying tools.

Live audio mixes with music on iOS and uses speech-triggered medium ducking. Legacy voice-turn notifications no longer pause music during an active Live call; explicit music controls remain available. Ending Live preserves active Apple Music playback while releasing microphone use.

Validation:

• Latest iPhone and Mac builds pass; music update installed and launched on the physical iPhone Air over Wi-Fi.
• Native iOS simulator test passes with concurrent local music playback, voice-processed capture, and music pause/resume. Live session regressions pass. Actual Apple Music plus a spoken Live follow-up remains unverified on hardware.
• Signed iPhone and macOS Debug builds pass; updated iPhone app installed and launched.
• Signed iPad build passes; installed on the paired iPad mini over Wi-Fi. Launch was blocked by the locked device, so spoken NDA generation remains unverified on hardware.
• Added session regressions cover PDF success/failure/retry, late file persistence, same-conversation restart isolation, and slow tasks completing without ending the call or replaying actions.
• Session regression checks cover progressive speech, stable rows, immediate tool activity, in-place results, persistence, image-search URLs/source metadata, generated-image placeholders, late completion after End, and isolation from ordinary chat images.
• Protocol checks cover startup schema, history bounds/order, Unicode result chunks, and delegation IDs. Message envelope round-trip checks preserve model attribution and activity metadata.
• Earlier hardware/API checks received nonzero Mac microphone samples and Live spoken replies; a signed Mac app completed a Bedrock tool round trip. Native hierarchy tests reproduce the former overlay crashes and pass with the fixes.

Full spoken image search/generation on the newly installed phone still needs a device smoke test. See docs/gpt-live.md for setup and validation details.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
loop-harness	Ready	Preview	Sep 15, 2026 1:38am UTC