Skip to content

Spill cold string values to an append-only SSD log - #63

Merged
GoDeskio merged 1 commit into
unstablefrom
cursor/ssd-tiering-afeb
Nov 5, 2026
Merged

GoDeskio merged 1 commit into
unstablefrom
cursor/ssd-tiering-afeb

Conversation

@GoDeskio

Copy link
Copy Markdown
Owner

Summary

  • Adds optional SSD tiering (tiering-enabled, off by default). Keys stay in RAM. String values larger than tiering-min-value that have been idle for tiering-idle-seconds are appended to tiering-path and replaced by a checksummed pointer.
  • A read promotes the value back into RAM. Compaction rewrites live records and renames the new log into place, so a crash before rename keeps the previous file. A torn tail is dropped the next time the log is opened.
  • RDB save and AOF rewrite read the log, so a restart still has the bytes. INFO reports spill, promotion, and dead-byte counters.
  • The value log layout is original. The shape (append-only value log plus a pointer, keys left in RAM) follows the public description of memcached extstore and KeyDB FLASH, both BSD. Redis on Flash was not used.

Tests

  • ./runtest --single unit/tiering (9 tests): spill and promote, delete, expire, mixed read/write/expire, RDB reload, crash before compaction rename, torn tail, AOF rewrite, and a dataset larger than 5× tiering-max-ram spilling back under the cap. All passed.
  • Full suite on this branch: 5876 passed, 2 failed. One failure was this new compaction test (the shared log was not yet half dead, so compaction did not run). After moving that case onto a fresh log it passes. The other was integration/repl-fullsync-compression ("Failed to start fake primary") while another suite was running; rerun alone: 23 passed, 0 failed.
Open in Web Open in Cursor 

Rebased onto unstable after forkless snapshot (#62); kept CRDT, hybrid flash, and forkless hooks.
@GoDeskio
GoDeskio force-pushed the cursor/ssd-tiering-afeb branch from 887d036 to cc66ffc Compare November 5, 2026 10:52
@GoDeskio
GoDeskio merged commit 84a3bd4 into unstable Nov 5, 2026
29 of 74 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant