feat(metadata): impl Snapshot interface for Mux state machine #2675

krishvishal · 2026-02-04T04:56:39Z

The snapshot system has three layers:

Snapshotable per-state-machine trait. Each StreamsInner, UsersInner, ConsumerGroupsInner implements to_snapshot() / from_snapshot() to convert between in-memory state.

SnapshotContributor visitor trait using the same recursive variadic tuple pattern as StateMachine::update. Walks (Users, (Streams, (ConsumerGroups, ()))) at compile time, collecting each state machine's serialized section into a Vec<SnapshotSection>.

MetadataSnapshot a top-level interface for the generic snapshot type on IggyMetadata. Defines the full lifecycle: create (snapshot from mux) → encode (to bytes) → decode (from bytes) → restore (back to mux).

codecov · 2026-02-05T19:53:16Z

Codecov Report

❌ Patch coverage is 53.78422% with 287 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.46%. Comparing base (0aa1f38) to head (3bd27da).

Files with missing lines	Patch %	Lines
core/metadata/src/stm/stream.rs	13.28%	109 Missing and 2 partials ⚠️
core/metadata/src/stm/consumer_group.rs	39.21%	57 Missing and 5 partials ⚠️
core/metadata/src/stm/user.rs	62.89%	53 Missing and 6 partials ⚠️
core/metadata/src/stats/mod.rs	0.00%	24 Missing ⚠️
core/metadata/src/stm/snapshot.rs	80.80%	18 Missing and 1 partial ⚠️
core/metadata/src/impls/metadata.rs	82.05%	5 Missing and 2 partials ⚠️
core/metadata/src/stm/mux.rs	91.52%	0 Missing and 5 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #2675      +/-   ##
============================================
- Coverage     68.62%   68.46%   -0.17%     
  Complexity      637      637              
============================================
  Files           734      735       +1     
  Lines         59660    60281     +621     
  Branches      56073    56694     +621     
============================================
+ Hits          40943    41269     +326     
- Misses        16717    16988     +271     
- Partials       2000     2024      +24

Flag	Coverage Δ
java	`51.41% <ø> (ø)`
rust	`69.62% <53.78%> (-0.20%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
core/metadata/src/stm/mod.rs	`50.00% <100.00%> (+8.20%)`	⬆️
core/metadata/src/stm/mux.rs	`92.30% <91.52%> (-2.43%)`	⬇️
core/metadata/src/impls/metadata.rs	`11.94% <82.05%> (+11.94%)`	⬆️
core/metadata/src/stm/snapshot.rs	`80.80% <80.80%> (ø)`
core/metadata/src/stats/mod.rs	`0.00% <0.00%> (ø)`
core/metadata/src/stm/user.rs	`35.21% <62.89%> (+35.21%)`	⬆️
core/metadata/src/stm/consumer_group.rs	`17.62% <39.21%> (+17.62%)`	⬆️
core/metadata/src/stm/stream.rs	`4.43% <13.28%> (+4.43%)`	⬆️

... and 9 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

numinnex · 2026-02-06T09:38:27Z

core/metadata/src/impls/metadata.rs

+    /// # Arguments
+    /// * `mux` - The multiplexing state machine containing all sub-state machines
+    /// * `commit_number` - The VSR commit number this snapshot corresponds to
+    fn create<T>(mux: &MuxStateMachine<T>, commit_number: u64) -> Result<Self, Self::Error>


We impl StateMachine for MuxStateMachine, therefor I think we could accept as first argument stm: &mut T

numinnex · 2026-02-06T09:38:59Z

core/metadata/src/impls/metadata.rs

+/// This is the interface that `MetadataHandle::Snapshot` must satisfy.
+/// It provides methods for creating, encoding, decoding, and restoring snapshots.
+#[allow(unused)]
+pub trait MetadataSnapshot: Sized {


Lets rename this to Snapshot trait, if there already exists one (that I created as an stub), remove it.

numinnex · 2026-02-06T09:39:54Z

core/metadata/src/impls/metadata.rs

+    where
+        T: StateMachine + SnapshotContributor;
+
+    /// Get the VSR commit number this snapshot corresponds to.


Maybe we could use an sequence_number instead of commit number, the idea is that the sequence number would be monotonically growing on each snapshot, independent of the commit number from consensus.

numinnex · 2026-02-06T10:53:06Z

core/metadata/src/stm/consumer_group.rs

 }
+
+/// Consumer group member snapshot representation for serialization.
+#[derive(Debug, Clone, Serialize, Deserialize)]


I think we don't need those intermediate structs, we can have the State structs impl Serialize and Deserialize directly.

We can't serialize/deserialize directly because some members of these structs can't be used with serde directly. Thats why I had to use intermediate structs.

numinnex · 2026-02-06T10:53:20Z

core/metadata/src/stm/mux.rs

    }
 }

+/// Recursive case for variadic tuple pattern: (Head, Tail)


Hmm, I am not sure if this is the simplest way to go around it.

I imagine the flow of abstraction to be identical to one of the StateMachine trait,
e.g we would have an Snapshotable trait, that would be implemented for everything that impls StateMachine trait,
since StateMachine trait is implemented both for MuxStateMachine aswell as the variadic!() tuple of state machines, the API would be called this way:

fn snapshot_metadata(&self, ....) { let mut snapshot = MetadataSnapshot::default(); <-- This would impl the `Snapshot` trait. // Here is the magic, `fill_snapshot` impl for `MuxStateMachine` would proxy to the `variadic!()` tuple impl, // impl Snapshotable for variadic!(St, ... Rest) // where St: StateMachine + FillSnapshot, // Rest: Snapshotable, // { // fn fill_snapshot<S: Snapshot>(&self, snapshot: &mut S) { // self.0.do_fill_snapshot(snapshot); // self.1.fill_snapshot(snapshot); // } // fn restore_snapshot(...) { // // ... Here similar code for restoring. // } // } // You can choose whatever trait name for the `FillSnapshot` trait, smth that covers filling/recovering // You can have associated types on the `Snapshot` trait that would be used to represent the currently filled snapshot // (for example a binary blob), and use those as arguments into the `fill_snapshot` and `restore_snapshot` methods. // and the snapshot from which we would restore State. self.mux.fill_snapshot(&mut snapshot); } The idea is to avoid using those `const SECTION:NAME: &'str`, rather rely on the type checking to perform the walk for us.

We could alternatively look into extending the StateMachine trait, by adding extra bounds on it

trait StateMachine + Snapshotable + ... { }

and then have single impl block.

I've added type based recursion and snapshotting. Removed string based version.

numinnex · 2026-02-06T10:54:45Z

core/metadata/src/stm/consumer_group.rs

+    pub topic_name_index: Vec<((String, String), Vec<usize>)>,
+}
+
+impl Snapshotable for ConsumerGroupsInner {


Impl those for ConsumerGroup and Streams and Users, rather for their inners. You can create an metehod on the LeftRight, that exposes the read handle.

numinnex · 2026-02-06T11:04:50Z

core/metadata/src/stm/snapshot.rs

+/// Contains metadata about the snapshot and a collection of sections,
+/// where each section corresponds to one state machine's serialized state.
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct SnapshotEnvelope {


This is a valid approach, to store an type-erased bytes inside of the snapshot, but alternative approach (the one that redpanda did choose), is for the snapshot to store an intermediate representation of all the Snapshotable states and serialize those. So rather than serializing individual states, they serialize the snapshot instead.

I didn't think about it a lot and not sure what is the trade off space between those two approaches (I guess you need more allocations with their approach, as you have to copy the data first and then serialize), where with our approach, we just serialize the state and append it to the snapshot envelope.

Removed SnapshotEnvelope. Now we store each snapshot struct and serialize it in one shot.

- Removed `SnapshotEnvelope`. Now we store each snapshot struct and serialize in one shot

krishvishal · 2026-02-10T10:45:23Z

Fixed merge conflicts.

numinnex requested changes Feb 6, 2026

View reviewed changes

hubcio changed the title ~~feat(metadata): impl Snapshot interface for Mux state machine.~~ feat(metadata): impl Snapshot interface for Mux state machine Feb 9, 2026

krishvishal force-pushed the snapshot-interface branch 2 times, most recently from f195436 to 9591421 Compare February 9, 2026 13:18

krishvishal added 3 commits February 10, 2026 16:06

feat(metadata): impl Snapshot interface for Mux state machine.

81da0a3

fix: type based variadic tuple recursion for snapshot

639074a

- Removed `SnapshotEnvelope`. Now we store each snapshot struct and serialize in one shot

fix: revet state snapshot intermediate structs.

3bd27da

krishvishal force-pushed the snapshot-interface branch from 13f739b to 3bd27da Compare February 10, 2026 10:44

feat(metadata): impl Snapshot interface for Mux state machine #2675

Are you sure you want to change the base?

feat(metadata): impl Snapshot interface for Mux state machine #2675

Uh oh!

Conversation

krishvishal commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

krishvishal commented Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

krishvishal commented Feb 4, 2026 •

edited

Loading

codecov bot commented Feb 5, 2026 •

edited

Loading