Skip to content

Serve historical reads below the FlatKV seed height from memIAVL - #4421

Closed
masih wants to merge 2 commits into
mainfrom
masih/1790896145-flatkv-pre-seed-reads
Closed

masih wants to merge 2 commits into
mainfrom
masih/1790896145-flatkv-pre-seed-reads

Conversation

@masih

@masih masih commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Nodes that switch FlatKV on mid-chain seed it at the height memIAVL had reached, so FlatKV has no history below that height. The composite store's read-only open and its exporter still opened FlatKV at any requested height. For a height below the seed, FlatKV fell back to its empty genesis snapshot and tried to replay from block 1, failing with "WAL starts at block N but replay must start at block 1 ... (data loss or corruption)". On atlantic-2 this broke proved abci_query below the activation height and the state-sync snapshot at 274408000 on archive-0, both on v6.7.0-rc4 and on main. The node's live state was fine; up to the seed height memIAVL alone made up the AppHash. #3928 removed the earliest-version record the read path used to tell these heights apart, and pinned the failure in TestComposite_Auto_ReadOnlyPreFlatKVEraHeightNowFails.

This restores pre-activation reads with a smaller record. After writing the seeded snapshot, SetInitialVersion durably writes the seeded height to flatkv/SEEDED_VERSION, and import removes it. FlatKV's LoadLatest reads that record once, validates it against the store's history and caches it, exposed as SeededVersion() on LiveStateStore. The record must be a positive height no higher than the store's version, no retained non-genesis snapshot may be older than it, and the WAL's first block must come after it. A malformed record, or one the history contradicts, fails the node at open rather than letting it start and then fail historical reads, or worse, serve an in-era height without FlatKV. A record that is too low is accepted, because the heights between it and the real seed just keep failing to load. LoadVersionReadOnly and Exporter leave FlatKV out and serve memIAVL only when the version is at or below the cached seed. The view's write mode is forced to memiavl_only so fixed migration modes work too. Every other height still opens FlatKV, and a real WAL gap after activation still fails loudly.

Stores seeded before this change have no record, so they keep today's behavior until an operator runs seidb flatkv-record-seeded-version --home <home> --version <seed> with the node stopped. For archive-0 on atlantic-2 the seed is 274411998. The tool applies the same validation before writing the record. The NowFails test is turned back into a positive test. New tests pin the exact seed/seed+1 boundary, the operator remediation (no record fails, recorded succeeds), open-time rejection of malformed and too-high records at both the FlatKV and composite level, re-seeding after an aborted import, and cleanup of a leftover SEEDED_VERSION-tmp.

@devin-ai-integration

devin-ai-integration Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access that start with 'Devin'.

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedOct 2, 2026, 2:25 PM

@codecov

codecov Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 69.53125% with 39 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.46%. Comparing base (4c4d935) to head (ea94da5).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
...d/seidb/operations/flatkv_record_seeded_version.go 34.21% 25 Missing ⚠️
sei-db/state_db/sc/flatkv/store_meta.go 84.05% 11 Missing ⚠️
sei-db/state_db/sc/flatkv/snapshot.go 66.66% 1 Missing ⚠️
sei-db/state_db/sc/flatkv/store.go 83.33% 1 Missing ⚠️
sei-db/tools/cmd/seidb/main.go 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4421      +/-   ##
==========================================
+ Coverage   55.68%   56.46%   +0.78%     
==========================================
  Files        2177     2131      -46     
  Lines      169048   166740    -2308     
==========================================
+ Hits        94130    94151      +21     
+ Misses      74913    72584    -2329     
  Partials        5        5              
Flag Coverage Δ
sei-chain 54.63% <33.33%> (+0.80%) ⬆️
sei-db 74.81% <ø> (ø)
sei-db-state-db 78.65% <85.39%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-db/state_db/sc/composite/store.go 79.32% <100.00%> (+0.25%) ⬆️
sei-db/state_db/sc/flatkv/store_constants.go 100.00% <ø> (ø)
sei-db/state_db/sc/flatkv/snapshot.go 75.91% <66.66%> (-0.07%) ⬇️
sei-db/state_db/sc/flatkv/store.go 79.66% <83.33%> (+0.03%) ⬆️
sei-db/tools/cmd/seidb/main.go 0.00% <0.00%> (ø)
sei-db/state_db/sc/flatkv/store_meta.go 81.50% <84.05%> (+0.70%) ⬆️
...d/seidb/operations/flatkv_record_seeded_version.go 34.21% <34.21%> (ø)

... and 92 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@masih masih closed this Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant