Skip to content

Uses of the GatewayBroker should not occasionally bypass the InferenceBroker trait - #192

Open
drauschenbach wants to merge 1 commit into
cppalliance:masterfrom
drauschenbach:david/widen-inference-broker-trait
Open

drauschenbach wants to merge 1 commit into
cppalliance:masterfrom
drauschenbach:david/widen-inference-broker-trait

Conversation

@drauschenbach

@drauschenbach drauschenbach commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

The prompt defining the proposed unit of work:

write a new file in the vibe/2026-11/ folder called 2026-11-03-1-widen-inference-broker-trait.md. In it, outline a requirement that code should not bypass the InferenceBroker trait by using the GatewayBroker struct directly. Having an InferenceBroker trait at all means that should be its public interface. The GatewayBroker struct's impl section should only contain the "new" factory.

then:

perform the work defined in 2026-11-03-1-widen-inference-broker-trait.md

What changed:

  • Trait: InferenceBroker (in crates/harness-internal/runner/src/performers.rs) has a new chat_streaming method. It takes the same arguments as chat, including round, plus an on_piece callback. By default it just runs chat and sends no pieces, so the 15 or so existing test brokers didn't need changes. The Harness itself still calls only chat.
  • Piece type: StreamDelta now lives next to the trait and is re-exported from harness. I deleted the Gateway client's copy (wire/delta.rs), and its stream reader now uses the Harness type.
  • GatewayBroker: its own impl block now holds only new. Streaming is a trait method, and the round logic is a private free function.
  • Workshop: I removed WorkshopBroker::chat_streaming and moved it into its trait impl. Workshop now stores the Gateway broker as an Arc and only names GatewayBroker::new. RunBroker now passes round when it streams.
  • Tests: the Gateway and Workshop broker tests now call chat_streaming through the trait. I added one test to crates/harness/tests/suite/broker.rs checking that the default runs chat once, under the round it was given, and sends no pieces.

Checks:

  • Passed: both clippy runs (warnings as errors), fmt, docs, the full suite (4126 passed), build-xtask (254 passed), and the docs-claims scan. The facade (crates/runvil) is unchanged, so I didn't run xtask api --check.
  • Workshop suite: 312 passed and 1 failed:
    realtime_relay::overload::stalled_browser_cleanup_is_bounded_after_gateway_disconnect. It fails the same way with my changes stashed, so it was already broken on master.

@drauschenbach
drauschenbach marked this pull request as ready for review November 3, 2026 20:17
@vinniefalco

Copy link
Copy Markdown
Member

This is an improvement

@awheeler294 awheeler294 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@drauschenbach
drauschenbach force-pushed the david/widen-inference-broker-trait branch from 5a31df1 to 3de6fb2 Compare November 4, 2026 15:04

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants