perf: stream agent message deltas instead of full snapshots (#375) - #382
Open
Naoki326 wants to merge 1 commit into
Open
perf: stream agent message deltas instead of full snapshots (#375)#382Naoki326 wants to merge 1 commit into
Naoki326 wants to merge 1 commit into
Conversation
The events route forwarded every message_update with the full accumulated message, so a ~15 KB response cost ~2.2 MB over the wire (~150x amplification, O(n^2) — especially painful over VPN/remote). The server now projects message_update onto a light message_delta event (assistantMessageEvent with the partial stripped) and the web client assembles the streamed message locally via a new pure helper (lib/stream-delta.ts). Message snapshots are still sent when a delta is missing or of unknown type, and on SSE (re)connect the current partial message is injected as a snapshot so mid-stream page reloads recover immediately instead of waiting for message_end.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #375 — remote access bandwidth amplification (~150x) caused by the events stream resending the full accumulated message on every
message_update.Root cause
The events route forwarded each SDK
message_update(which carries the complete accumulated message plus anassistantMessageEventdelta) almost verbatim — only stripping the delta field. Since streaming emits a chunk roughly every 50 characters, every chunk re-sent everything generated so far → O(n²) transfer (~15 KB response cost ~2.2 MB over the wire).Change
app/api/agent/[id]/events/route.ts):message_updateis projected onto a lightmessage_deltaevent — only the slim delta fields (contentIndex/delta/content/toolCall) are forwarded;partial(the full accumulated message) is stripped.done/errorare omitted (theirreasonalready arrives inmessage_end's full message). Unknown/missing delta types degrade to the existing full-snapshot fallback. On SSE (re)connect, the current partial message is injected as a snapshot so mid-stream page reloads recover immediately instead of waiting formessage_end.hooks/useAgentSession.ts+ newlib/stream-delta.ts): streamed messages are assembled locally by appending deltas (text/thinking/toolCall), keeping the existing snapshot-overwrite fallback. Rendering is unchanged.toolcall_startcarriescontentIndexonly;id/nameare extracted frompartial(two fields only, not the whole message) so tool cards show the tool name while streaming.Verification
lib/stream-delta.test.mjs) plus projection assertions inapp/api/agent/events-route.test.mjs.