store: resume scripthash postings after a pass-1 restart - #873
Merged
Merged
Conversation
Reopen treated a pass-1 MPHF as a sealed durable head, so restart skipped pass 2 and multi-script history never landed. The pack commit is now scripthash.head/NN.packed. Open loads a shard only when that mark exists. A finished unmarked head with no extract in progress is soft-migrated. Schema refuse still uses index occupancy.
A durable head whose include_hwm lagged the tip was marked ready at tip entry, and a cancel before any extract file named scripthash.cold_progress. Electrum stays down until sh_is_tip_ready, then this process binds it once write-behind catches up. The cancel line names cold_progress or scripthash.unsorted only when that path is on disk.
The syncer catches a miner, serves Electrum history, tip-follows, and restarts on write-behind without rewriting the pack mark. A cancelled IBD resumes to the same tip. With the miner down, a partial datadir stays short of that tip and does not open Electrum; the miner coming back lets it finish.
A tip append while pass 1 is unsealed lands on ingest and hides the packed multi-script chain. Packing the shard soft-clears those keys. The end-of-IBD journey freezes after DONE.keys and checks Electrum history across that resume, a later block, and the lying include_hwm.
Regtest retargeting plus min-difficulty makes one miner taller and weaker. The syncer adopts the heavier tip, keeps it across a reopen, and follows the next heavy block.
#862 binds Electrum when --sh-index is off. The pass-1 freeze chapter only needs that catch-up to reach the miner tip before the datadir is copied.
The tip-follow bind still called the seven-argument helper. #866 added the per-connection subscription cap, so that call did not compile.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #867.
A restart during scripthash postings collect treated the pass-1 MPHF as a durable head, entered tip follow, and opened Electrum on an index that has no multi-script history. Pack-complete shards now carry
scripthash.head/NN.packed(orscripthash.head.packedwhen there is one shard). Pass 1 does not write that mark. Open loads sorted main only when the MPHF and the mark both exist, and a restart afterDONE.keysresumes pass 2. A finished unmarked head, with no unsorted extract in progress, is soft-migrated. Schema stays 26.Electrum stays down while
--sh-indexis on and the inclusion floor is behind the tip. The same process binds it once write-behind covers the tip. With--sh-indexoff, Electrum and Esplora still listen (the behavior from #862); address methods fail closed.A tip append that lands on ingest while the shard is unsealed used to hide the packed chain after pass 2. Packing the shard soft-clears ingest keys that packed main now owns.
Tests
end_of_ibd_follow: one miner, one--sh-indexsyncer. IBD leaves initial block download, Electrum history matches, the next block tip-follows, and a restart does not rewrite the pack mark. A cancelled mid-IBD resume finishes the same tip. With the miner down after catch-up, the syncer stays out of IBD. A partial datadir whose miner is down does not open Electrum; the miner coming back finishes that datadir.end_of_ibd_sh_interrupt: catch-up with--sh-indexoff, freeze after pass 1 (DONE.keys, noDONE.post). Resume serves the full Electrum history on the first answer. A block mined after the freeze is in that history. A copy whoseinclude_hwmalready names that later block, with the create sitting on the unsealed head, still serves the same history.end_of_ibd_work_fork: retargeting plus min-difficulty makes one chain taller and weaker. The syncer adopts the heavier tip, keeps it across a reopen, and follows the next heavy block while the tall chain is still ahead.