Skip to content

feat(voice): start answering before transcription finishes - #448

Merged
johnmatthewtennant merged 19 commits into
mainfrom
jtennant/goose/voice-early-delivery
Oct 30, 2026
Merged

johnmatthewtennant merged 19 commits into
mainfrom
jtennant/goose/voice-early-delivery

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Voice calls currently wait for finalized recognition before the agent can begin answering. Berd now prepares a private draft when the user pauses, enabled by default, without adding visual states. Voice settings includes an “Early delivery (experimental)” switch, on by default; explicitly saved off remains off. Disabling it immediately discards private preparation and leaves final-only delivery unchanged. Preparation uses a session hidden at creation, with the same configured provider/model and backend-enforced chat-only mode, so provisional recognition cannot execute tools or steer an unrelated run.

Only the final transcript enters the normal chat with execution authority. Preparation receives only the provisional transcript, not copied conversation history. A completed draft is assistant-only context, never a substitute for answering or carrying out the final request. Final delivery waits at most 750 ms for preparation; stalled or unsupported preparation is discarded without blocking later speech. Late native finals retain their original recognition IDs even after recognition holds expire.

The change preserves the queue ordering and dispatch requirements in LAWS/CHAT.md and configured-agent requirement in LAWS/AGENTS.md.

Related issue

None found.

Testing

Rendered-app scenarios exercise the production controller and ACP paths with native recognition and the model substituted at their boundaries:

  • Provisional input prompts only a hidden session after chat-only mode succeeds. Mode rejection fails closed; private text never appears in the chat or plays aloud.
  • Unchanged and corrected finals enter the normal chat exactly once, with optional draft context that does not suppress requested actions. The same behavior holds while an unrelated main-session run is active.
  • A stalled private prompt cannot hold final delivery or a later utterance. Cancellation targets only the private session, and late private output remains hidden.
  • Recognition ending without a final discards preparation and permits the next utterance.
  • The Voice settings switch defaults on, persists off across reloads, prevents private preparation while off without blocking final delivery, and permits preparation again after re-enabling.

Mutation checks reject invoking provisional inference without chat-only mode. Native regressions cover hold expiry, a new provisional utterance, then old and new finals retaining their matching IDs.

The isolated-draft implementation has not been exercised with a live macOS microphone.

Screenshots

Voice settings before early delivery

voice-settings-early-on

@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/goose/voice-early-delivery branch from dcd9309 to 9c37091 Compare October 29, 2026 14:45
@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review October 29, 2026 16:45
@johnmatthewtennant
johnmatthewtennant requested a review from a team October 29, 2026 16:45
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/goose/voice-early-delivery branch from af9a1c2 to 137b3ec Compare October 29, 2026 17:01

@morgmart morgmart left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated code review

Engineering found three blocking issues in the exact full PR comparison: corrected speech can block indefinitely behind a speculative run, provisional recognition retains ordinary tool authority before final confirmation, and late finals can be assigned to a newer utterance. The supplied GitHub evidence is structurally valid and all ten captured checks completed successfully, but required checks still independently govern merge readiness. Recommendation: REQUEST_CHANGES.

Deterministic publication result: 3 blocking and 0 non-blocking inline finding(s) publishable; 0 duplicate(s) suppressed; 0 blocking screenshot-evidence requirement(s) in this review body.

Comment thread src-tauri/src/commands/native_voice.rs Resolved
@johnmatthewtennant
johnmatthewtennant marked this pull request as draft October 29, 2026 17:51
@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review October 29, 2026 18:36
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/goose/voice-early-delivery branch from 2eb65ad to bbca4c2 Compare October 29, 2026 22:32
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
…ered app

Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/goose/voice-early-delivery branch from 372deaa to 757b78a Compare October 30, 2026 01:52
Signed-off-by: John Matthew Tennant <jtennant@block.xyz>
@johnmatthewtennant
johnmatthewtennant merged commit 0b179a0 into main Oct 30, 2026
10 checks passed
@johnmatthewtennant
johnmatthewtennant deleted the jtennant/goose/voice-early-delivery branch October 30, 2026 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants