feat(telnyx): add Telnyx assistant server framework#173
Draft
lucasassisrosa wants to merge 3 commits into
Draft
Conversation
Add Telnyx hosted AI Assistant as a first-class EVA framework alongside
the existing Pipecat pipeline. The Telnyx provider connects to the Telnyx
AI gateway via direct anonymous WebRTC (Verto JSON-RPC over WebSocket),
bridges audio between the EVA user simulator (Twilio mulaw frames) and the
Telnyx assistant (PCM 16 kHz), and enriches the audit log from the Telnyx
Conversations API after the call ends.
New files:
- src/eva/assistant/base_server.py
AbstractAssistantServer: template-method base class that defines the
EVA server contract (FastAPI WebSocket endpoint, audio bridging, tool
execution via ToolExecutor, audit log, audio recording, save_outputs).
Concrete servers implement start() and _shutdown(); the base handles
stop(), output saving, and artifact generation.
- src/eva/assistant/audio_bridge.py
Audio format conversion utilities: mulaw 8 kHz ↔ PCM 16-bit 24 kHz
resampling (libsoxr), PCM mixing, Twilio FrameSerializer protocol
helpers, and framework/metrics log writers.
- src/eva/assistant/telnyx_server.py
TelnyxAssistantServer(AbstractAssistantServer): the main server.
- TelnyxAssistantManager: REST CRUD for benchmark assistants.
- TelnyxDirectSessionConfig: dataclass for WebRTC session params.
- TelnyxVertoJsonRpcClient: Verto JSON-RPC over WebSocket.
- TelnyxWebRTCClient: aiortc RTCPeerConnection + media management.
- TelnyxAssistantServer: bridges EVA Twilio frames to Telnyx WebRTC,
handles conversation events, tool calls, dynamic variables,
audit-log enrichment from Conversations API.
- src/eva/utils/audio_utils.py
save_pcm_as_wav() helper shared by all server implementations.
- tests/unit/assistant/test_telnyx_server.py (20 tests)
- tests/unit/assistant/test_audio_bridge.py (3 tests)
Modified:
- src/eva/models/config.py: Add framework field to RunConfig.
- src/eva/orchestrator/worker.py: Add _get_server_class() dispatch.
- pyproject.toml: Add aiortc, aiohttp, soxr dependencies.
Addresses VSDK-277 review: adds the missing Python EVA wrapper that
subclasses AbstractAssistantServer, registers in worker.py, routes tool
calls through ToolExecutor, and produces all required EVA artifacts.
lucasassisrosa
force-pushed
the
feat/telnyx-framework-provider
branch
from
7376bea to
fdb747f
Compare
Add the TURNS/443 fallback (turns:turn2.telnyx.com:443) to TELNYX_DEFAULT_ICE_SERVERS so the Telnyx EVA server's prod ICE list matches DEFAULT_PROD_ICE_SERVERS in @telnyx/webrtc 2.27.4: STUN + TURN UDP/3478 + TURN TCP/3478 + TURNS on 443. This is the last-resort TLS fallback for networks that block both 3478 transports.
The direct-WebRTC session could establish signaling + ICE + DTLS
(iceConnectionState=completed, connectionState=connected) yet receive
zero remote audio frames, leaving both audio buffers empty and the
conversation silent until the far end idle-disconnected ~90s later — with
no diagnostic trace.
- Log iceConnectionState / connectionState transitions.
- On connectionState failed|closed, set disconnected_event so the session
ends immediately instead of idling.
- In _receive_remote_audio, bound track.recv() with a 10s timeout and warn
("media not flowing", with iceConnectionState + frame count) instead of
blocking silently; log the first received frame and frame count on stop.
Diagnostic/robustness only — does not change the audio bridge behaviour
when media flows.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Telnyx hosted AI Assistant as a first-class EVA framework alongside the existing Pipecat pipeline. The Telnyx provider connects to the Telnyx AI gateway via direct anonymous WebRTC (Verto JSON-RPC over WebSocket), bridges audio between the EVA user simulator (Twilio mulaw frames) and the Telnyx assistant (PCM 16 kHz), and enriches the audit log from the Telnyx Conversations API after the call ends.
New files:
src/eva/assistant/base_server.py AbstractAssistantServer: template-method base class that defines the EVA server contract (FastAPI WebSocket endpoint, audio bridging, tool execution via ToolExecutor, audit log, audio recording, save_outputs). Concrete servers implement start() and _shutdown(); the base handles stop(), output saving, and artifact generation.
src/eva/assistant/audio_bridge.py Audio format conversion utilities: mulaw 8 kHz ↔ PCM 16-bit 24 kHz resampling (libsoxr), PCM mixing, Twilio FrameSerializer protocol helpers, and framework/metrics log writers.
src/eva/assistant/telnyx_server.py TelnyxAssistantServer(AbstractAssistantServer): the main server. - TelnyxAssistantManager: REST CRUD for benchmark assistants. - TelnyxDirectSessionConfig: dataclass for WebRTC session params. - TelnyxVertoJsonRpcClient: Verto JSON-RPC over WebSocket. - TelnyxWebRTCClient: aiortc RTCPeerConnection + media management. - TelnyxAssistantServer: bridges EVA Twilio frames to Telnyx WebRTC, handles conversation events, tool calls, dynamic variables, audit-log enrichment from Conversations API.
src/eva/utils/audio_utils.py save_pcm_as_wav() helper shared by all server implementations.
tests/unit/assistant/test_telnyx_server.py (20 tests)
tests/unit/assistant/test_audio_bridge.py (3 tests)
Modified:
Addresses VSDK-277 review: adds the missing Python EVA wrapper that subclasses AbstractAssistantServer, registers in worker.py, routes tool calls through ToolExecutor, and produces all required EVA artifacts.