Skip to content

fix: decode SGLang streaming responses - #550

Draft
TeoSlayer wants to merge 4 commits into
mlcommons:mainfrom
TeoSlayer:fix/sglang-stream
Draft

TeoSlayer wants to merge 4 commits into
mlcommons:mainfrom
TeoSlayer:fix/sglang-stream

Conversation

@TeoSlayer

@TeoSlayer TeoSlayer commented Oct 28, 2026 •

Copy link
Copy Markdown

Implements @maria-18-git's SGLang metadata diagnosis in #497 and adds regression coverage. Accept nullable completion metadata and the retraction field emitted by SGLang; isolate malformed SSE frames so later valid frames in the same chunk are still decoded. Closes #497.

Schema checked against SGLang 0.5.18's tokenizer manager. All 14 regression cases fail before the fix. Synced with upstream dependency/watchdog fixes, with session startup synchronized in the SIGINT test. Full Linux CI passes; validation uses recorded frames rather than a live GPU-backed SGLang server.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

MLCommons CLA bot:
Thank you very much for your submission; we really appreciate it. Before we can accept your contribution,
we ask that you sign the MLCommons CLA (Apache 2). Please submit your GitHub ID to our onboarding form to initiate
authorization. If you are from a MLCommons member organization, we will request that you be added to the CLA.
If you are not from a member organization, we will email you a CLA to sign. For any questions, please contact
support@mlcommons.org.
1 out of 2 committers have signed the MLCommons CLA.
✅ @TeoSlayer
❌ @teovl
You can retrigger this bot by commenting recheck in this Pull Request

@TeoSlayer

TeoSlayer commented Nov 5, 2026 •

Copy link
Copy Markdown
Author

Verified nullable SGLang metadata and recovery after malformed SSE frames. All CI checks pass.

@codecov-commenter

codecov-commenter commented Nov 5, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@745ac91). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #550   +/-   ##
=======================================
  Coverage        ?   82.66%           
=======================================
  Files           ?      170           
  Lines           ?    24331           
  Branches        ?        0           
=======================================
  Hits            ?    20113           
  Misses          ?     4218           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

# Conflicts:
#	tests/integration/commands/test_sigint.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/normal PR Review Policy: <=500 non-test lines & <=20 files

Projects

None yet

3 participants