fix(client): :http streaming stalls small chunks until 8KB buffer fills#102
Merged
Conversation
The :http backend's streaming read loop fills a fixed 8KB buffer with `readbytes!(io, buf)`. On HTTP.jl 2.x, `readbytes!` on a response body blocks until the whole buffer is filled (or the body ends), so a streamed chunk smaller than 8KB is not forwarded to the consumer until enough later data accumulates. On a quiet long-lived stream — e.g. a Kubernetes watch, where a single event is a few KB and the next event may be minutes away — every event was delivered one flush behind the action that caused it, effectively an indefinite stall. Read with `readavailable` instead: `eof(io)` blocks until data is available, and `readavailable` then returns whatever has arrived without further blocking, on both HTTP.jl 1.x and 2.x (the same pattern HTTP.jl's own `download` uses). The `readbytes!` loop was itself a workaround for `read(io, n)` being unavailable on 2.x streams. Found with a watch reaction-time probe (Kuber.jl) against a live k3s cluster: with the :http backend, MODIFIED/DELETED watch events took 30s+ — delivered only when later events flushed the buffer — while `curl -N` through the same `kubectl proxy` delivered every event in under 100ms, and the :downloads backend reacted in ~5-10ms. With the fix, :http reacts in milliseconds as well. Adds a regression test: a raw-TCP chunked HTTP/1.1 server sends one small chunk, stalls, then sends another; the first chunk must reach the consumer during the stall. Unfixed code fails this on HTTP.jl 2.x (the first chunk arrives only together with the second); HTTP.jl 1.x and the :downloads backend already delivered promptly and keep passing.
Member
Author
|
For the record, this fix was originally proposed in #101 and dropped there due to a confusion worth documenting. In #101 I questioned the "blocks until the buffer is filled" premise by quoting the Both observations were misleading:
So the empirical check in #101 validated 1.x behavior and the conclusion was wrongly extended to 2.x. The regression test added here fails on unfixed code only under HTTP.jl 2.x, matching that split. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
:httpbackend's streaming read loop fills a fixed 8KB buffer withreadbytes!(io, buf). On HTTP.jl 2.x,readbytes!on a response body blocks until the whole buffer is filled (or the body ends), so a streamed chunk smaller than 8KB is not forwarded to the consumer until enough later data accumulates. On a quiet long-lived stream — e.g. a Kubernetes watch, where a single event is a few KB and the next event may be minutes away — every event was delivered one flush behind the action that caused it, effectively an indefinite stall.Found with a watch reaction-time probe (Kuber.jl) against a live k3s cluster: with the
:httpbackend, MODIFIED/DELETED watch events took 30s+ — delivered only when later events flushed the buffer — whilecurl -Nthrough the samekubectl proxydelivered every event in under 100ms, and the:downloadsbackend reacted in ~5-10ms.Fix
Read with
readavailableinstead:eof(io)blocks until data is available, andreadavailablethen returns whatever has arrived without further blocking, on both HTTP.jl 1.x and 2.x (the same pattern HTTP.jl's owndownloaduses). Thereadbytes!loop was itself a workaround forread(io, n)being unavailable on 2.x streams.Test
Adds a regression test (
test/streaming_latency_tests.jl): a raw-TCP chunked HTTP/1.1 server (so the test controls exactly when bytes hit the wire, independent of HTTP.jl server API differences between 1.x and 2.x) sends one small chunk, stalls 4s, then sends another; the first chunk must reach the consumer during the stall. Runs for both:httpand:downloadsbackends, with a warmup request so the timing measures I/O behavior rather than JIT.Verified:
:httpfirst chunk arrives at 4.0s, together with the second chunk); HTTP.jl 1.x and the:downloadsbackend already delivered promptly and keep passing:httpwatch reaction goes from every update/delete event missed (30s timeout) to ~6-9ms median, zero missed events