Skip to content

[Brook K6] Served the Stream service writes and describe through the runtime. - #216

Closed
moetemp wants to merge 29 commits into
moe/brook/review/streams-k4-memory-storefrom
moe/brook/review/streams-k6-service-writes
Closed

moetemp wants to merge 29 commits into
moe/brook/review/streams-k4-memory-storefrom
moe/brook/review/streams-k6-service-writes

Conversation

@moetemp

@moetemp moetemp commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

What changed?

This PR serves the Stream service writes and describe through the runtime object.

  • A bridge builds one StreamRuntime per client and calls it with a method name and proto bytes. Core has no hook for it, and a Workflow makes each call from a Local Activity.
  • The runtime takes any Store through a StoreFactory, which it calls on the first call that needs a Store. So a bridge can build it with no async runtime.
  • The memory runtime is here. The Redis runtime is in the Redis crate (R1, temporal-community/sdk-streams#1).
  • CreateStream is strict. A create of an open Stream fails with stream_already_exists, unless it asks for USE_EXISTING, which keeps the Stream's Retention, or retries the create that made the Stream with the same request id. The Stream id is the id of the create's ref. A create with an owner needs a Workflow Producer and a ref whose owner is its run chain, and the answer's ref has that owner. A Retention past the largest proto Duration fails.
  • The service refuses a Local Activity Producer whose epoch gives no Workflow Task. Describe says EXPIRED for a closed Stream that expired.
  • A close writes no Record and carries no value, and a second close succeeds. Describe's last offset stays at the newest Record.
  • The service checks the request and the Store's capabilities before it calls the Store. No Store declares RUN_FENCE until K7, so a retry run's write fails with stream_unsupported.

Depends on #215 (K4). Review lane is the Rust reviewers.
Part of the Brook review series.

Why?

Every SDK calls the write path first, and it needs no read code. Reads get their own PR (K8).

How did you test it?

Cargo.toml has a dev-dependency on the crate itself with test-support, so the integration tests can use the memory Store's test hooks in a default-features build.

cargo fmt, cargo clippy -- -D warnings (with and without features), cargo test -p temporalio-streams --all-features and cargo doc pass at this commit. The service suite runs on the memory Store here, and on Redis in R1.

@moetemp
moetemp force-pushed the moe/brook/review/streams-k4-memory-store branch from c2df597 to 7fe70de Compare October 27, 2026 07:35
@moetemp
moetemp force-pushed the moe/brook/review/streams-k6-service-writes branch 2 times, most recently from 593a974 to a39dd25 Compare October 27, 2026 08:17
@moetemp
moetemp force-pushed the moe/brook/review/streams-k4-memory-store branch from 7fe70de to 12ed854 Compare October 27, 2026 08:17
@moetemp moetemp added the skip-changelog Changelog entry rides another PR label Oct 27, 2026
@moetemp
moetemp force-pushed the moe/brook/review/streams-k4-memory-store branch from 12ed854 to 2a15176 Compare October 27, 2026 16:55
@moetemp
moetemp force-pushed the moe/brook/review/streams-k6-service-writes branch from a39dd25 to 550c147 Compare October 27, 2026 16:55
@moetemp
moetemp force-pushed the moe/brook/review/streams-k4-memory-store branch from 2a15176 to f2287d8 Compare October 27, 2026 23:46
@moetemp
moetemp force-pushed the moe/brook/review/streams-k6-service-writes branch 2 times, most recently from 6f26a43 to 6b804be Compare October 28, 2026 08:55
@moetemp
moetemp force-pushed the moe/brook/review/streams-k4-memory-store branch 2 times, most recently from 384bf80 to cb14cea Compare October 29, 2026 04:06
@moetemp
moetemp force-pushed the moe/brook/review/streams-k6-service-writes branch 2 times, most recently from 5ccca2b to f4950ec Compare October 29, 2026 08:37
The runtime is a plain object that a bridge builds per client and calls with a
method name and proto bytes. It takes any Store through a `StoreFactory` and
connects on its first call that needs the Store. The service checks each
request and the Store's capabilities before it writes.
@moetemp
moetemp force-pushed the moe/brook/review/streams-k6-service-writes branch from f4950ec to 7a22520 Compare October 29, 2026 09:17
@moetemp
moetemp force-pushed the moe/brook/review/streams-k4-memory-store branch from 0e7433f to 167d0f8 Compare October 29, 2026 09:17
The capabilities test drops default_retention and default_stream_id, which K2 and K3 removed. Its hooks are behind test-support now, so the tests turn the feature on.
No consumer reads StreamRuntime::backend, builds a memory runtime with create, or reads MEMORY_RUNTIME_NAME. StreamRuntime::name stays, since the Python bridge exposes it.
A cancel, or an Activity, child Workflow or Nexus Operation failure whose cause is a cancel, closes
as canceled. An append with sequence zero needs `NEXT_SEQUENCE`, and the Store numbers it.
A Store keeps whole ms, so a shorter Retention expired the Stream at its next write. Many small Records fit under the byte limit but kept a Store busy for long.
A create fails as already_exists unless it uses the Stream or retries the create that made it. The service suite checks the order of the create errors and the close rule of a Stream with no owner.
A close carries no value, so the service builds no Record, no status and no size check. The service suite checks that a close leaves the last offset and that a second close succeeds.
The requests carry a StreamRef in ref, and a create in stream_ref. The memory config doc says StreamRef too.
The Producer and the withdraw_run of a write moved into StreamWriter. A Superseded item gives the writer back, and the suites send one.
The proto is in the protos crate now, so "the crate" no longer says which one sets it.
Each method decodes its request, so a bad one is invalid, and a typo in a method name would make it unknown. The upstream runtime layer has the same tests.
The review asked to stop representing invalid states. The Producer kind is now a oneof with run_id only in the run-based variants, so a client can't carry a run. A write's task facts and withdraw_run move into RunWithdrawal, which only a run writer sets.
@moetemp

moetemp commented Nov 10, 2026

Copy link
Copy Markdown
Owner Author

Closed in favor of #228, the combined branch, which has this change.

@moetemp moetemp closed this Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Changelog entry rides another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant