Skip to content

Stream LLM Responses#18

Open
NeonDaniel wants to merge 11 commits intomainfrom
FEAT_StreamingResponses
Open

Stream LLM Responses#18
NeonDaniel wants to merge 11 commits intomainfrom
FEAT_StreamingResponses

Conversation

@NeonDaniel
Copy link
Member

@NeonDaniel NeonDaniel commented Mar 18, 2026

Description

Implements a chat-stream endpoint for SSE-streamed responses
Refactors front-end to use the streaming endpoint for chat rendering

Issues

Other Notes

  • The original changes by @NeonClary included some additions related to synthesized responses which are removed here as they are unrelated to streaming and have no functional impact in the context of streaming responses

@NeonDaniel NeonDaniel force-pushed the FEAT_StreamingResponses branch from bce7fcd to 6b4ffb6 Compare March 18, 2026 20:51
Copy link
Member Author

@NeonDaniel NeonDaniel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not working; I just have a continuous "Orchestrator is thinking" message when I run this locally

Comment on lines +78 to +84
already = (
session.messages
and session.messages[-1].get("role") == "user"
and session.messages[-1].get("content") == message.user_input
)
if not already:
session.append_message("user", message.user_input)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variable name is unclear here. Also, if this is only used for the conditional, then this doesn't need to be defined as a variable

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, looks like 421162f

@NeonDaniel NeonDaniel force-pushed the FEAT_StreamingResponses branch from bb54916 to 8a766e3 Compare March 25, 2026 22:59
@NeonDaniel NeonDaniel marked this pull request as ready for review March 25, 2026 23:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants