Skip to content

handshake: close accepted streams — port 444 leaked a goroutine per connection - #34

Merged
TeoSlayer merged 2 commits into
mainfrom
fix/close-handshake-streams
Jul 29, 2026
Merged

handshake: close accepted streams — port 444 leaked a goroutine per connection#34
TeoSlayer merged 2 commits into
mainfrom
fix/close-handshake-streams

Conversation

@TeoSlayer

Copy link
Copy Markdown
Contributor

Close appeared nowhere in handshake.go. Every stream accepted on port 444 was leaked — on both the timeout and the success path.

The timeout branch is the worse half

ch := make(chan readResult, 1)
go func() {
    buf := make([]byte, maxMsgSize)   // 64 KiB
    n, err := stream.Read(buf)
    ...
}()
select {
case res := <-ch: ...
case <-time.After(handshakeRecvTimeout):   // 10s
    slog.Warn("handshake timeout waiting for message", ...)
}

On timeout, handleConnection returned without closing. The child goroutine stayed parked in stream.Read forever — nothing could ever unblock it, because the only thing that would is a Close, and there wasn't one. Its stack retained the 64 KiB buffer, the stream, and the underlying daemon Connection with its RecvBuf/SendBuf channels.

Port 444 accepts from any peer. A peer that connects and sends nothing leaked ~64 KiB + a goroutine stack + a full Connection every 10 seconds, uncapped, with no recovery.

That is the same shape as the dead idle-timeout that OOM-killed the 431-agent service fleet — reached through a different door.

Why closing fixes the parked goroutine

CloseCloseConnectionCloseRecvBuf, so the blocked <-RecvBuf returns !ok and the reader exits with io.EOF. Closing is the only thing that can reclaim it.

The dead constant was the clue

handshakeCloseDelay was declared at line 80 for precisely this — "delay before closing after send to let data flush" — and had no remaining reference anywhere in the package. It was stranded when the close path was removed at some point.

processMessage writes its reply synchronously, so closing without the pause would truncate an in-flight response. That's why the delay existed, and why simply adding a bare defer Close() would have been wrong.

Each handleConnection already runs in its own goroutine (see the accept loop), so the pause delays nothing but its own teardown.

Tests

go build, go vet clean. TestSendRequestDirectFailsRelaySucceeds fails identically with and without this change — verified against a clean stash, so it's pre-existing and unrelated.

Found during a codebase-wide sweep for safety mechanisms that silently fail to engage.

🤖 Generated with Claude Code

teovl and others added 2 commits July 28, 2026 19:13
…onnection

Close appeared NOWHERE in handshake.go. Every stream accepted on port 444
was leaked, on both the timeout and the success path.

The timeout branch is the worse half:

    ch := make(chan readResult, 1)
    go func() {
        buf := make([]byte, maxMsgSize)   // 64 KiB
        n, err := stream.Read(buf)
        ...
    }()
    select {
    case res := <-ch: ...
    case <-time.After(handshakeRecvTimeout):   // 10s
        slog.Warn("handshake timeout waiting for message", ...)
    }

On timeout handleConnection returned without closing. The child goroutine
stayed parked in stream.Read forever — nothing could ever unblock it,
because the only thing that would is a Close and there wasn't one. Its stack
retained the 64 KiB buffer, the stream, and the underlying daemon Connection
with its RecvBuf/SendBuf channels.

Port 444 accepts from any peer. A peer that connects and sends nothing leaked
roughly 64 KiB plus a goroutine stack plus a full Connection every 10s, with
no cap and no recovery — the same shape as the dead idle-timeout that
OOM-killed the service fleet, reached through a different door.

Closing is what frees the parked reader: Close reaches CloseConnection, which
calls CloseRecvBuf, so the blocked receive returns !ok and the goroutine
exits with io.EOF.

The handshakeCloseDelay pause is why that constant exists. It was declared at
line 80 for exactly this purpose ("delay before closing after send to let
data flush") and had NO remaining reference anywhere in the package —
stranded when the close path was removed at some point. processMessage writes
its reply synchronously, so closing without the pause would truncate an
in-flight response. Each handleConnection already runs in its own goroutine
(see the accept loop), so the pause delays nothing but its own teardown.

Pre-existing unrelated failure TestSendRequestDirectFailsRelaySucceeds fails
identically with and without this change; verified against a clean stash.

Found during a codebase-wide sweep for safety mechanisms that silently fail
to engage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closing accepted streams (previous commit) made TestHandleConnection_
TimeoutBranch panic: the test double implemented

    func (s *slowStream) Close() error { close(s.done); return nil }

so the test's own `defer stream.Close()` became a SECOND close once
handleConnection started closing the stream itself, and close of a closed
channel panics.

The real transport is idempotent — *Connection guards teardown with a
sync.Once (CloseRecvBuf → c.CloseOnce.Do), so double Close is safe in
production. The double simply did not model that, so it was less faithful
than the thing it stood in for. Fixed with sync.Once to match.

The alternative — dropping the test's own defer — would have been wrong: it
would leave the test unable to fail if handleConnection ever stopped closing
the stream again, which is precisely the regression this guards.

Pre-existing unrelated failure TestSendRequestDirectFailsRelaySucceeds fails
identically on origin/main; verified by running the suite on a clean checkout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@TeoSlayer
TeoSlayer merged commit 6dbd5d5 into main Jul 29, 2026
4 checks passed
@TeoSlayer
TeoSlayer deleted the fix/close-handshake-streams branch July 29, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants