Skip to content

[Brook] Added an auto close to Streams. - #27

Closed
moetemp wants to merge 4 commits into
moe/brook/v4/s10-reads-without-idlefrom
moe/brook/v4/s11-auto-close
Closed

moetemp wants to merge 4 commits into
moe/brook/v4/s10-reads-without-idlefrom
moe/brook/v4/s11-auto-close

Conversation

@moetemp

@moetemp moetemp commented Oct 22, 2026

Copy link
Copy Markdown
Owner

What changed?

This PR adds auto_close to a Stream: no append for that long closes it with no result.
CreateStreamRequest.auto_close (5) sets it, counted from the create. DescribeStreamResponse.auto_close (4) reports it. The Store keeps ac (the period) and la (the time of the create or of the newest append with Records) in the meta hash.
Nothing runs when the period passes. A new auto_close step in lib.lua runs first in the create, append and close scripts. When the period passed, it writes the close Record (a StreamRecord with only kind = CLOSE, no Producer) in the same Lua call. Then the append is refused with stream_closed, the close answers already, and the create answers closed. A resend still gets its first answer.
The read script stays read only, so it still runs on a replica. A read or a describe that finds the close due runs the new auto_close.lua step and reads again. A long-polling read caps its wait at the time the close comes due, so it ends with the close Record.
The memory Store has the same step. A new capability auto_close guards a create that sets the period, from a client and from a Workflow flush, so a Store without it refuses with stream_unsupported and writes nothing. Redis and memory declare it. A create with another auto close is a conflict (stream_already_exists), as one with another Retention is.
The Workflow path passes auto_close from a buffered create. A Workflow append after an auto close is dropped, as for any close by someone else.

Why?

Decision S73 (and S37). A Stream that nobody closes must not stay open until its safety TTL. It is separate from Retention and the safety TTL.

How did you test it?

cargo fmt --all -- --check, cargo clippy --workspace --tests -- -D warnings, and STREAMS_REQUIRE=redis cargo test --workspace on standalone Redis, a 3-primary Redis cluster and the memory Store.
New Store tests check that each append holds the close off, that the append past the period writes the close Record and is refused, that a resend keeps its answer, that a later close and create see the close, the conflict on another period, and that repeated auto close steps write one close Record. New service tests check that a long poll ends with the close Record soon after the period, that a describe and a read apply a due close, and that an append after it fails with stream_closed. A capability test masks auto_close. A Workflow test creates with an auto close and sees a later flush dropped. A unit test pins the encoded auto close Record against the bytes in lib.lua.

@moetemp

moetemp commented Oct 22, 2026

Copy link
Copy Markdown
Owner Author

Review fixes in 4e4ea4e: the memory Store adds the period with saturating_add, a period below 1 ms is stored as 1 ms, and a read or describe that finds the Stream expired at the auto close step (a lagging replica) reports the expiry instead of failing. A new Store test covers both edges.

@moetemp

moetemp commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

Closing as stale. This layer is folded into the v5 tip, #41, and the upstream review uses a fresh series instead. Plan: https://www.notion.so/4048fc567738814990b7c796ac28fe3f

@moetemp moetemp closed this Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant