Repository navigation
feat(codex): add subscription-backed live voice conversations - #1
Open
romil-bijarnia wants to merge 2 commits into
Open
romil-bijarnia wants to merge 2 commits into
romil-bijarnia wants to merge 2 commits into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the review copy of upstream #36195, which is currently closed by the open-PR limit.
The tested base has dictation, but no two-way voice conversation or Aube picker. This adds experimental live voice to saved Codex conversations on web and desktop, using the ChatGPT account already connected to the selected provider. It does not require an API key or a separate voice login.
The server discovers the backend's V3 voice catalogue, owns call authorization and lifecycle, and sends spoken coding requests through normal T3 orchestration. Clients handle microphone selection, playback, mute, stop and captions. Startup reserves session IDs across threads before validation can suspend. Managed connections bind runtime resolution and credentials under one auth admission, reject profile-revision mismatches, and retain that admission for the socket lifetime. Client mute requests persist through microphone selection and permission and are applied before audio tracks are attached. Ending a call does not cancel coding tasks already dispatched. Native mobile dictation is unchanged.
This is an alternative implementation to upstream #29394, not a claim that no related work exists. Unlike that implementation's native-turn interruption/resubmission path, T3 owns the voice delegation before a coding turn starts. A separate repair to the existing implementation is available in jakeleventhal/t3code#7.
Verified on Linux with unmodified Codex 0.167.2 and an existing ChatGPT-backed T3 provider:
gpt-6-lunatask, saved its request andVOICE_HANDOFF_CONFIRMEDreply in the same thread, and spoke the result back. No registry, projection or orchestrator mocks were used. Decoded audio was nonzero, no realtime errors occurred, and stop completed cleanly.vp run check:changed 2d8a843e6be9ca170f3bce98cc92ad76617b00c4 --concurrency 2passed formatting, lint, both Knip passes, GraphQL, 38 affected-package typechecks, 36 tests in seven selected files and three React Compiler checks. This is not a repo-wide suite.Limits: only Sol and Aube received real audio tests; catalogue discovery is not an entitlement guarantee for every profile. Managed SIWC token-sharing voice, native mobile calls, remote/relay/tunnel operation and physical microphone/speaker quality remain unverified. Rejected account routes report errors rather than switching billing, accounts or auth methods.
The before screenshot is current main rendering an isolated copy of the same harmless saved test conversation. All state and account fixtures were separate from the installed nightly and live chats. Screenshots and the credential-free acceptance result are linked below, not committed to source.
Updated credential-free end-to-end result, rerun after the review fixes: real Aube audio, exactly one new same-thread Codex run, saved reply and spoken result; no realtime errors.
Current main, same saved test conversation:
Alternative with Aube selected:
Discovered voice profiles: