Skip to content

Fix: Eliminate initial flash of default output on realtime session start#13

Merged
AdirAmsalem merged 3 commits intomainfrom
fix/realtime-initial-state
Mar 14, 2026
Merged

Fix: Eliminate initial flash of default output on realtime session start#13
AdirAmsalem merged 3 commits intomainfrom
fix/realtime-initial-state

Conversation

@AdirAmsalem
Copy link
Copy Markdown
Contributor

Problem

When starting a realtime session with a reference image (e.g. Lucy 2 with a Firebase-hosted image), users see ~3 seconds of the default "Einstein" output before the intended reference image takes effect. This is because the SDK was applying the initial prompt/image after the WebRTC connection was already established and generating frames.

Solution

The initial state (prompt, reference image, or passthrough) is now sent and acknowledged before the WebRTC handshake begins. The server knows what to generate before the first frame is ever produced.

This is the iOS port of JS SDK PR #93 and Python SDK PR #32.

What changes for users

  • No more flash of default output — the very first frame matches the configured prompt or reference image.
  • initialPrompt is now optional — omitting it defaults to passthrough mode instead of requiring an empty prompt.
  • setPrompt() now forwards enhance — the enrich flag was previously ignored for non-image prompts.
  • No public API changes required — existing integrations continue to work.

How it works

  1. WebSocket connects and waits for service ready (unchanged)
  2. New: sends initial state and waits for server acknowledgment
    • Reference image → set_image + waits for set_image_ack
    • Prompt only → prompt + waits for prompt_ack
    • No prompt/image → passthrough set_image with explicit nulls + waits for set_image_ack
  3. WebRTC handshake proceeds (offer/answer/ICE)
  4. Connection established — first frame already reflects the intended state

Timeout is 30s with fail-fast on server errors or WebSocket disconnect, matching the JS and Python SDKs.

Files changed

  • DecartRealtimeManager.swift — moved initial state before handshake, added ack waiting with timeout and error propagation
  • RealtimeConfiguration.swiftinitialPrompt defaults to empty (passthrough)
  • SignalingModel.swift — added enhance_prompt to prompt messages, null-preserving encoding for passthrough, expanded ack models
  • WebSocketClient.swiftsend() now throws on failure instead of silently dropping messages

@AdirAmsalem AdirAmsalem merged commit 4942350 into main Mar 14, 2026
1 check passed
@AdirAmsalem AdirAmsalem deleted the fix/realtime-initial-state branch March 14, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants