Skip to content

refactor: move framework concerns out of StateMachine; implementations only provide engine-bound actions #459

Open

Description

@JoshuaChi

Why

StateMachine and the log StorageEngine are extension points: developers embed their own
engine. Today the trait asks every implementer to remember framework-level duties
(TTL, snapshot metadata, serving gate, flush/persist ordering), and implementations drift:
File and sled close() rely on Drop; File and RocksDB swallow persist errors; File's
apply_chunk has no NotServing check. See #455/#458.

Principle

Move framework concerns out to the framework layer (read layer / writer layer) so an
implementation only provides the actions the contract names.

Test: does the duty depend on the storage engine's internal transaction?

  • Yes -> stays in the implementation (e.g. data and last_applied committed atomically).
  • No -> moves to the framework (e.g. serving gate, error policy, TTL, snapshot metadata).

Contracts are written as observable postconditions, not steps. The shared test suite is the
single source of truth; an implementation is valid iff it passes the suite (ADR-046).

Scope: StateMachine (22 methods)

Verdicts are proposals; audit call sites of each method before removing it.

Keep (engine-bound)

  • start - load durable state
  • close - make state recoverable, release exclusive resources, idempotent; becomes fallible
  • get / get_multi / scan_prefix - reads (scan_prefix, get_multi optional)
  • apply_chunk - Ok implies data and last_applied are atomically recoverable
  • last_applied - read-only
  • apply_snapshot_from_file - atomically updates last_applied too
  • generate_snapshot_data - pinned at last_included
  • reset - optional

Remove or move to the framework

  • update_last_applied / persist_last_applied - folded into apply_chunk atomicity
  • update_last_snapshot_metadata / snapshot_metadata / persist_last_snapshot_metadata - framework-owned snapshot store
  • save_hard_state / flush / flush_async - folded into the close postcondition
  • is_running - serving state owned by the framework
  • len / is_empty - cannot express NotServing, O(n) on RocksDB; demote to diagnostics or remove
  • lease_background_cleanup and TTL hooks - see TTL below

Cross-cutting items

  • close() -> Result<(), Error>: always release, return first error; worker propagates it
  • TTL: leader writes absolute expiry into the value so TTL is applied atomically with
    apply_chunk; expiry stops bypassing Raft; no SM-side TTL file
  • Serving gate (NotServing) implemented once in the framework, not per method per impl;
    builder takes the state machine by value so a retained raw Arc cannot bypass it
  • Suite covers the postconditions: reopen without Drop, NotServing after close
    (reads and writes), last_applied never regresses; suite is public for third parties

Log side (StorageEngine / LogStore / MetaStore)

To verify first

  • RocksDB atomic_flush (data and meta live in different CFs, WAL disabled)
  • Snapshot export pins both CFs at one point under concurrent apply_chunk
  • Where TTL lives after a kill -9 (RocksDB)
  • Framework already has a place to own snapshot metadata

Activity

  1. added this to the v0.2.5 milestone on Oct 11, 2026
  2. added
    component:raft-logRaft log store, WAL, buffered log, flush workers, persistence layer.
    type:refactorCode restructuring without changing external behavior
    component:raft-statemachineState machine semantics, apply pipeline, RocksDB/File state machine implementations.
    on Oct 11, 2026
  3. added theissue type on Oct 11, 2026
  4. modified the milestones: v0.2.5, v0.2.6 on Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    component:raft-logRaft log store, WAL, buffered log, flush workers, persistence layer.component:raft-statemachineState machine semantics, apply pipeline, RocksDB/File state machine implementations.type:refactorCode restructuring without changing external behavior

    Type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions