Skip to content

[REA-6846] Send what a chunked channel queues before closing it - #230

Merged
douglaseel merged 2 commits into
mainfrom
douglas/rea-6846-drain-chunked-channels-before-teardown
Oct 5, 2026
Merged

douglaseel merged 2 commits into
mainfrom
douglas/rea-6846-drain-chunked-channels-before-teardown

Conversation

@douglaseel

@douglaseel douglaseel commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Why

Closing the peer lets go of the connection at once. On a chunked channel, a frame sent just before that can still sit in the channel's queue, above libwebrtc's buffer, and is discarded with the connection. The runner relies on the opposite. It broadcasts the moderation and session-ended notices so they are queued on each ordered channel ahead of its close. With chunking, a large message queued before the notice would take the notice down with it.

What Changed

Teardown now closes each open chunked channel first, which sends what the channel has queued. The wait is bounded at 5 s, shared across both channels, and runs on the thread close() already uses. The bound is that high because the channel is ordered: a notice leaves only after whatever was queued ahead of it, and early in a connection SCTP may move only a few MB a second (a 4 MiB frame did not clear in 2 s on loopback). It stays well inside the 10 s a crashed model's process gets to exit. Plain channels, and a wire that has already gone, are left as they were.

A loopback test sends 2 MiB and closes the peer immediately, then checks the client receives the whole message. It fails without this change. Unit tests cover what is and isn't closed. The integration suite passed 3 out of 3 runs.

🤖 Generated with Claude Code

douglaseel commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor Author

@douglaseel
douglaseel marked this pull request as ready for review October 3, 2026 01:23
@douglaseel
douglaseel requested a review from a team as a code owner October 3, 2026 01:23
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

[codex-review] No issues found. This PR looks good.

Scope: full (af2c6ac..bccf3d2).

View workflow run.

@Dere-Wah Dere-Wah left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The drain skips plain channels and channels that are no longer open, so a client without chunking sees no change at teardown. _send_on returns early once _stop_event is set, so nothing new enters the queue during the drain. The binding releases the GIL in close(), and close() runs _teardown through asyncio.to_thread, so the event loop does not block. CI has the same unpublished 0.19.0 pin as #228 and must be green before merge. One non-blocking note inline.

Comment thread src/reactor_runtime/transport/webrtc/peer.py Outdated
@douglaseel
douglaseel force-pushed the douglas/rea-6846-surface-refused-data-channel-sends branch from af2c6ac to 1f8f017 Compare October 3, 2026 23:57
@douglaseel
douglaseel force-pushed the douglas/rea-6846-drain-chunked-channels-before-teardown branch 2 times, most recently from ec4dd82 to 966889e Compare October 5, 2026 14:50
@douglaseel
douglaseel force-pushed the douglas/rea-6846-surface-refused-data-channel-sends branch from 1f8f017 to 224f688 Compare October 5, 2026 14:50
@douglaseel
douglaseel force-pushed the douglas/rea-6846-drain-chunked-channels-before-teardown branch from 966889e to f9b3886 Compare October 5, 2026 15:59
@douglaseel
douglaseel force-pushed the douglas/rea-6846-surface-refused-data-channel-sends branch from 224f688 to af4954c Compare October 5, 2026 15:59

douglaseel commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Merge activity

  • Oct 5, 4:43 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Oct 5, 4:48 PM UTC: Graphite rebased this pull request as part of a merge.
  • Oct 5, 4:52 PM UTC: @douglaseel merged this pull request with Graphite.

@douglaseel
douglaseel changed the base branch from douglas/rea-6846-surface-refused-data-channel-sends to graphite-base/230 October 5, 2026 16:44
@douglaseel
douglaseel changed the base branch from graphite-base/230 to main October 5, 2026 16:46
douglaseel and others added 2 commits October 5, 2026 16:47
Closing the peer let go of the connection at once. On a chunked channel a
frame sent just before that can still sit in the channel's queue, above
libwebrtc's buffer, and is discarded with it. The runner relies on the
opposite: the moderation and session-ended notices are broadcast so that they
are queued on each ordered channel ahead of its close.

The teardown now closes each open chunked channel first, which sends what it
queues, within 5 s shared across both channels. That leaves room in the 10 s a
crashed model's process gets to exit. Plain channels and a wire that has
already gone are left as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Douglas Ferreira <douglaseel@gmail.com>
A reconnect awaits the replaced connection's close before it builds the new
one, and that close could wait up to 5 s draining a chunked channel whose
client is the one reconnecting. The watchdog's close, for a client that
stopped pinging, had the same wait for nobody. close() takes drain=False for
both; a commanded close still drains.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Douglas Ferreira <douglaseel@gmail.com>
@douglaseel
douglaseel force-pushed the douglas/rea-6846-drain-chunked-channels-before-teardown branch from f9b3886 to 9da7e7b Compare October 5, 2026 16:47
@douglaseel
douglaseel merged commit ba3b3ad into main Oct 5, 2026
10 checks passed
@douglaseel
douglaseel deleted the douglas/rea-6846-drain-chunked-channels-before-teardown branch October 5, 2026 16:52
douglaseel added a commit that referenced this pull request Oct 5, 2026
…#240)

## Why

Releases what has landed since 3.6.1. It adds new public surface and moves nothing on the wire, which makes it a minor release:

- **Data-channel chunking** (#228–#230). `WebRtcConfig.dc_chunking` is on by default. The runtime answers clients that ask for chunked data channels, so a command or a model message of up to 64 MiB passes, instead of being capped at 256 KiB. Refused frames are logged once per channel, and a teardown sends what a chunked channel still has queued. A client that doesn't ask keeps the plain channel. 
- **Step sessions** (#231–#239): a session's starting input and step count on the start body, `complete_step()`, closing a session at its steps, the `runtime.step_results` manifest block, and step output saved as an MP4 or a folder of files, served over HTTP.
- **Client stats** (#201, #224): client-observed WebRTC stats decoded in the transport and journaled as metric self-loops.
- **reactor-webrtc 0.21.0** (from 0.19.1).
  - 0.21.0 keeps per-frame metadata on its own frame when libwebrtc drops a frame before render. Before, metadata was matched by queue order, so after a dropped frame every later frame got its predecessor's metadata.
  - It also keeps a `FrameTransform`'s `replace_data` through the metadata step.
  - 0.20.0 is additive.
  - Neither changes the Python API the runtime uses.
- Dependency bumps (#216–#220).

## What Changed

Two commits:
- `uv version 3.7.0`, which updates the version in `pyproject.toml` and `uv.lock`;
- `reactor-webrtc==0.21.0`, relocked. Only that package moves in `uv.lock`.

Merging tags `v3.7.0` and publishes.

Checked locally with a locked install of the published reactor-webrtc 0.21.0 wheel:
- lint, typecheck and the HTTP spec check pass;
- unit tests pass (1917);
- integration tests pass (12).

And with the release workflow's own gates:
- `mise run http-breaking-release`: "3.6.1 -> 3.7.0 (minor) satisfies the mandated patch";
- `mise run wire-check-release`: the pin matches `proto/`;
- `mise run build`: builds the 3.7.0 wheel and sdist.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants