Repository navigation
refactor(client): drop the read-cache keywords from connect - #95
Merged
Merged
Conversation
A catch-up reads the gap to its cursor once, never the same files twice, so neither cache has anything to reuse there. The settings stay on the reads that repeat: snapshot, scan, sql, and live's base and rebases. live's own catch-up, at open or after a dropped connection, now reads with litelink's defaults. Unreleased, so nothing breaks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsSDkeb5rVAxA1FSmKKfQi
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes
memory_cache,disk_cache,cache_keyanddisk_cache_volume_limitfromconnect, added in #93. They're unreleased, so this breaks no one.Why
A catch-up is a one-off sequential read: the gap from the subscriber's cursor to the published end. A consumer that restarts reads a different gap each time and never the same files twice, so a disk cache has nothing to reuse, and the memory cache doesn't help a single read. The case where caching helps, several subscribers on one box catching up over the same range at once, is niche, and four extra keywords on the subscriber's main entry point were mostly confusing.
The settings stay where reads repeat:
Stream.snapshot,scanandsql, andStream.live's base and its rebase everyrebase_everyseconds.liveliveusedconnect's keywords to pass its settings to its own catch-up. That catch-up now uses litelink's defaults, by the same argument. It runs only:max_replay);In normal operation a
liveview connects once, and a rebase never reconnects. The cost: a view with non-default settings builds a second DuckDB database for that catch-up, once per process (about a quarter-second, mostlyLOAD iceberg).Changes
connect: the four keywords are gone, andCatcherno longer carries a cache.Live: its catch-up no longer passes its settings; its base and rebases still do.ReadCache.keywords()is removed, sinceLivewas its only user.Stream.snapshot's docstring, API.md (connect's signature, plus a sentence on why it has no cache keywords), the README and the CHANGELOG.test_a_catch_up_reads_with_the_callers_cache_settingsis removed with the keywords it tested.test_the_settings_reach_every_read_of_a_streamstill checkssnapshot, thelivebase and a rebase.Gates clean; the reader tests (
test_catchup,test_live,test_published,test_snapshot) pass, 76 tests.🤖 Generated with Claude Code
https://claude.ai/code/session_01FsSDkeb5rVAxA1FSmKKfQi