Skip to content

Fix/sse streaming events#203

Open
Antherix wants to merge 4 commits into
avishek0769:mainfrom
Antherix:fix/sse-streaming-events
Open

Fix/sse streaming events#203
Antherix wants to merge 4 commits into
avishek0769:mainfrom
Antherix:fix/sse-streaming-events

Conversation

@Antherix

Copy link
Copy Markdown
Contributor

Description

This PR fixes the message streaming endpoint to emit properly structured SSE frames, enabling the frontend to reliably distinguish between content chunks, token usage data, completion signals, and errors. Previously, the backend wrote raw text to the response stream with no event typing, making structured client-side parsing impossible and blocking future features like per-message token tracking.

Closes #52

Type of Change

  • Bug fix
  • Refactoring

Checklist

  • Code follows project style
  • Tested locally
  • Updated documentation

Changes Made

SSE Helper (Backend)

  • Added writeSSE(res, event, data) helper in chatMessage.controller.js that writes valid SSE frames in the format event: …\ndata: …\n\n
  • Centralises all SSE output through one function, eliminating ad-hoc res.write() calls

Streaming Loop (Backend)

  • Replaced raw res.write(content) with writeSSE(res, "chunk", { content }) for each LLM content delta
  • Added writeSSE(res, "usage", { inputTokens, outputTokens }) when token counts arrive from the model
  • Added writeSSE(res, "done", {}) on successful stream completion
  • Replaced res.end("Stream ended with error.", ...) with writeSSE(res, "error", { message }) on failure

Early-Exit Error Path (Backend)

  • Converted the "no relevant sources" early return from a raw res.write() to a proper event: error SSE frame before res.end()
  • Ensures the frontend receives a typed, parseable error instead of an unstructured string

SSE Frame Parser (Frontend — src/lib/api.ts)

  • Replaced the raw byte-pass-through reader loop in sendMessageStream with a proper SSE frame parser
  • Accumulates incoming bytes into a buffer and splits on \n\n to extract complete frames
  • Parses event: and data: lines from each frame and dispatches by event type
  • chunk events: extracts content and calls onChunk
  • error events: extracts message and calls both onError and onChunk to surface it in the UI
  • usage and done events: silently consumed, ready for future use
  • Fixed JSON.stringify(payload) in the fetch body — previously serialised onChunk, onError, and signal (non-serialisable) into the request body; now explicitly serialises only userPrompt, model, provider, and chatId

Signature Updates (Frontend — src/lib/api.ts)

  • Added signal?: AbortSignal to sendMessageStream payload type and wired it to the fetch call, restoring abort support that ChatPage.tsx already expected
  • Added onError?: (message: string) => void callback for callers that want typed error handling separate from the chunk stream

SSE Contract

Event Payload
chunk { content: string }
usage { inputTokens: number, outputTokens: number }
done {}
error { message: string }

Files Updated

Backend

  • backend/controllers/chatMessage.controller.js

Frontend

  • src/lib/api.ts

How Has This Been Tested?

  • Verified streaming output still renders correctly in the chat UI
  • Verified the Network tab shows proper event: chunk / event: done framed responses on /message/send
  • Verified a "no relevant sources" query surfaces a typed error in the UI instead of a raw string
  • Verified AbortController cancellation works correctly when a message is interrupted
  • Verified non-serialisable fields (onChunk, signal) are no longer sent in the request body
  • Ran frontend build successfully

Acceptance Criteria

  • Backend emits valid SSE frames for all stream events
  • Frontend parses frames and dispatches by event type
  • Errors are distinguishable from content without string matching
  • Token usage data is captured and available for future display
  • Abort/cancel behaviour continues to work
  • No regression in existing streaming chat functionality

Technical Notes

  • The SSE frame format (event: …\ndata: …\n\n) is the standard specified in the WHATWG EventSource spec. Using named events allows the client to branch on event type without inspecting the data payload, which was the core fragility this PR addresses.
  • The frontend buffer-and-split approach handles the case where a single reader.read() call returns multiple frames or a partial frame, which is common under network pressure.
  • onChunk continues to receive error message text as a fallback so the chat bubble always shows something meaningful even if the caller does not supply onError.

@avishek0769

Copy link
Copy Markdown
Owner

@Antherix Please resolve the merge conflicts

@avishek0769 avishek0769 added Hard This is issue is hard to solve SSoC26 Social Summer of Code - 2026 labels Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Hard This is issue is hard to solve SSoC26 Social Summer of Code - 2026

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Convert streaming to real SSE events (chunk/usage/done/error)

2 participants