Skip to content

perf(sse): split only the newly decoded text in stream line readers - #10118

Merged
waleedlatif1 merged 1 commit into
simstudioai:stagingfrom
b84330808:perf/sse-split-new-text-only
Nov 17, 2026
Merged

waleedlatif1 merged 1 commit into
simstudioai:stagingfrom
b84330808:perf/sse-split-new-text-only

Conversation

@b84330808

Copy link
Copy Markdown
Contributor

Summary

The SSE and NDJSON line readers append each decoded chunk to the pending line and then split the whole buffer again. When one event is large, for example a block output of a few MB in the run stream (block:completed carries output and display.output, execution:completed carries output and finalBlockLogs), every chunk rescans and re-flattens the whole pending event. Reading it is quadratic in its size.

The pending remainder never contains a newline, so splitting only the new text and prefixing its first piece with the remainder gives exactly the same lines. Same change in readSSELines (lib/core/utils/sse.ts), the two NDJSON readers in the Mothership block handler, and the CLI's readNdjson and sseData.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other: performance

Testing

  • sse.test.ts, and sim-cli workflow-run-follow.test.ts and chat.test.ts pass unchanged (57 tests). Biome is clean on the touched files.
  • I ran a throwaway fuzz with random framings (CRLF, multibyte UTF-8 split across chunks, blank lines, unterminated last line) cut at random byte offsets. The old and new readSSELines gave identical lines in 3000 cases, and the NDJSON loop forms matched in 5000.
  • Benchmark setup: a local HTTP server streams a run with one block whose output is N MB, read with fetch through readSSEEvents. Node 24.15 on Windows 11, chunks arrive at 64 KB, 7 alternating rounds, median (min to max):
block output stream before after
1 MB 4 MB 52 ms (50 to 63) 31 ms (30 to 33)
2 MB 8 MB 134 ms (129 to 151) 59 ms (56 to 60)
4 MB 16 MB 422 ms (393 to 522) 114 ms (108 to 122)
8 MB 32 MB 1448 ms (1371 to 1562) 230 ms (221 to 262)

In Chromium the reader gets bigger chunks, so the gain is smaller. With the server pacing delivery at about 2 to 8 MB/s, main-thread time spent in the reader went from 187 to 31 ms for a 2 MB output and from 320 to 43 ms for 4 MB. Wall time stayed the same because the network bounds it.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

The SSE and NDJSON line readers appended each decoded chunk to the pending
line and re-split the whole buffer, so one large event (a block output of a
few MB in the run stream) was rescanned and re-flattened on every chunk:
quadratic in the event size. The pending remainder never holds a newline, so
splitting just the new text and prefixing the first piece gives the same lines.
@vercel

vercel Bot commented Nov 17, 2026

Copy link
Copy Markdown

@b84330808 is attempting to deploy a commit to the Sim Team on Vercel.

A member of the Team first needs to authorize it.

@greptile-apps

greptile-apps Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge.

Summary

Speeds up five SSE and NDJSON read loops across four files by splitting only newly decoded text.

  • Stream readers split new text without rescanning unfinished events.

Reviews (1) · Last reviewed commit: "perf(sse): split only the newly decoded ..." · Reviewed by Greptile

@waleedlatif1
waleedlatif1 merged commit f7d1210 into simstudioai:staging Nov 17, 2026
78 of 79 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants