Skip to content

Reduce node-state serialization overhead - #14

Draft
the-code-learner wants to merge 5 commits into
chatgpt/consensus-performance-labfrom
chatgpt/state-persistence-hotpath
Draft

Reduce node-state serialization overhead#14
the-code-learner wants to merge 5 commits into
chatgpt/consensus-performance-labfrom
chatgpt/state-persistence-hotpath

Conversation

@the-code-learner

Copy link
Copy Markdown
Owner

Scope

First profile-driven Zephyr performance experiment, stacked on the Consensus & Performance Lab PR.

The canonical 7-validator profile shows the current persistence hot path dominates before P-256 verification:

  • ledger.(*Store).writeState: ~58% cumulative CPU
  • encoding/json.MarshalIndent: ~55% cumulative CPU
  • MarshalIndent: ~41% of flat allocations in the profiled process
  • Store.Accept: ~90% cumulative mutex delay

Experiment

Replace machine-only state.json serialization from json.MarshalIndent to compact json.Marshal while preserving:

  • the same JSON semantics and schema
  • the same state normalization
  • the same chain binding
  • the same atomic temp-file/fsync/rename persistence path
  • the same owner-only file permissions
  • the same restart/load behavior

Measurement

The PR temporarily runs baseline and optimized canonical 7-validator benchmarks in the same GitHub-hosted job with the same runner, workload and -benchtime=5x, then verifies go vet ./..., the full Go suite and the Consensus & Performance Lab before committing the experiment.

This is deliberately not the journal/checkpoint redesign yet. It isolates how much of the current bottleneck is purely pretty-printed full-state JSON serialization before we make a larger storage-architecture change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant