store: resume scripthash postings after a pass-1 restart

SCHEMA.md

@@ -267,7 +267,7 @@ itself changed.
     scripthash.body                  # 17 file variant: one shared TableFile
     scripthash.body/NN               # 17 dir variant: one TableFile per main shard
     scripthash.ovf/body              # dir variant: ingest + all sealed ovf
-    scripthash.head/NN.mphf + NN.val # Class B sealed MPHF main (8 B pack8; no fuse)
+    scripthash.head/NN.mphf + NN.val + NN.packed # Class B MPHF main; `.packed` is the pack commit (pass-1 BDZ has no mark)
     scripthash.ovf/ingest                                # global OA ingest (key16+pack8, 2^25)
     scripthash.ovf/NNNNNN[.fuse8][.idx]                  # L0 SHSR pack8
     scripthash.ovf/NNNNNN.mphf|.val|.fuse8               # L1 promoted ovf (at most one)
@@ -798,7 +798,10 @@ the directory variant. A leftover file `scripthash.body` **refuses**.
 
 New `Store::create` writes the dir variant. ColdProgress `SHCOLDP1`:
 `next_shard` is the **lowest unsealed** main shard (holes after it
-stay); sealed `scripthash.head/NN.mphf`+`.val` is the per-shard commit. Overflow
+stay). The per-shard pack commit is `scripthash.head/NN.packed` next to
+`.mphf`+`.val`. Pass-1 BDZ writes the MPHF and is not that mark. A complete
+head with no extract in progress and no marks is soft-migrated on open
+(marks written; missing `include_hwm` set from the create count). Overflow
 compact still merges **heads only** — all ovf keys share
 `scripthash.ovf/body`.
 

TESTING.md

@@ -179,7 +179,7 @@ test bytes are RAM.
 | Remining 100-block maturity pads with `confirm_wire_run` | `pad_empty_from` / `build_mature_regtest_with_spend` **once per binary journey** (not once per skinny test) |
 | Wall-time multi-round microbenches in default suite | Deterministic structure / chunk-load asserts; demote wall arms to `#[ignore]` |
 
-**P2P walls:** `two_node_header_and_block_sync`, `three_node_relay_path`, `ibd_two_peers`, `tip_follow_after_ibd`, `tip_follow_getheaders_catches_missed_blocks`, and `node_run_p2p_short` 60s wall (180s under `coverage.sh` / llvm-cov). `serve_after_restart_via_reconstruct` 90s wall (180s under llvm-cov). `p2p_compact_hb_getblocktxn_and_orphan` 30s wall (90s under llvm-cov). `p2p_timeout_getaddr_and_keepalive_ping`, `p2p_feeler_completes_and_closes`, and `p2p_inbound_full_rejects_extra` 20s wall. Live `P2PNode` tests in `integration_multinode` take a process mutex (shared `rbtc-scripts` pool / confirm OS threads); hub-only reorgs do not.
+**P2P walls:** `two_node_header_and_block_sync`, `three_node_relay_path`, `ibd_two_peers`, `tip_follow_after_ibd`, `tip_follow_getheaders_catches_missed_blocks`, and `node_run_p2p_short` 60s wall (180s under `coverage.sh` / llvm-cov). `serve_after_restart_via_reconstruct` and `end_of_ibd_follow` 90s wall (180s under llvm-cov). `end_of_ibd_sh_interrupt` 120s wall (240s under llvm-cov). `end_of_ibd_work_fork` 150s wall (300s under llvm-cov). `p2p_compact_hb_getblocktxn_and_orphan` 30s wall (90s under llvm-cov). `p2p_timeout_getaddr_and_keepalive_ping`, `p2p_feeler_completes_and_closes`, and `p2p_inbound_full_rejects_extra` 20s wall. Live `P2PNode` tests in `integration_multinode` take a process mutex (shared `rbtc-scripts` pool / confirm OS threads); hub-only reorgs do not.
 
 **Speed / reliability (default suite):** prefer `pad_empty_from` / `build_mature_regtest_with_spend` **once per journey** (tx_relay live hub, Electrum protocol, core_analogs assumevalid+mempool) over remine pads; SH run-builder sleeps are 1 ms under `cfg(test)` (40 ms in production). `pin_compose_multi_pack_timed` keeps functional + layout/covered short-circuit gates (multi-ms floor); sticky vs cold assemble is log-only (not a hard timing assert). Schema-13 wire rebuild must stamp create identity from `txid.body` — zero batch identity is treated as missing (regression covered by the spend reconstruct in `consensus_mature_chain_spend_reconstruct_and_scripthash` + multi-vout confirm scenarios). Coverage vs speed: prefer **one** scenario at the real entry over N micro-opens that only paint lines; when adding coverage for reduce/materialize, use a **tiny** target, not production stream depth.
 
@@ -370,6 +370,9 @@ Prefer **one high-level scenario** per behavior cluster. Delete lower-level test
 | `ibd_two_peers` | P2P (**default**) | Dual live seeders, 8-block IBD |
 | `tip_follow_after_ibd` | P2P (**default**) | After IBD, follow + one new tip via inv/headers. With the filter index on, IBD confirm writes no basic filters; `rbtc-idx-wb` materializes them to the tip (its caught-up callback fires once, at the tip), then seals the followed block. With `--sp-tweaks` too: the builder brings both indexes to 5 and is stopped; a followed block with no builder running moves neither (no confirm path writes index data); a new builder seals 6, then follows 7 |
 | `tip_follow_getheaders_catches_missed_blocks` | P2P (**default**) | Blocks mined while disconnected fill via post-connect `getheaders` |
+| `end_of_ibd_follow` | P2P (**default**) | Miner plus `--sh-index` syncer. One mature regtest: coinbases pay script A, one spend pays script B. IBD reaches that tip, leaves IBD, and Electrum history matches. The next block tip-follows. Restart does not rewrite the scripthash pack mark; one more block arrives by write-behind. Cancelling IBD once the height is below the miner, then `run_p2p`, finishes the same tip and history. With the syncer caught up, dropping the miner leaves tip follow (not IBD). A partial datadir whose miner is down does not open Electrum and stays short of that tip; the same miner address coming back lets that datadir finish |
+| `end_of_ibd_sh_interrupt` | P2P (**default**) | Same miner and scripts. The syncer first catches up with `--sh-index` off, then the datadir is frozen after pass 1 (`DONE.keys`, no `DONE.post`). Resume builds the index and Electrum's first answer is the full A/B history; restart does not rewrite the pack mark. A block mined after the freeze is in that history. A copy that already has the block, with `include_hwm` at the new tip and that create appended on the unsealed head, still serves the same history and does not open Electrum early |
+| `end_of_ibd_work_fork` | P2P (**default**) | Two miners on a regtest with retargeting and min-difficulty. One chain retargets harder and stops shorter. The other forks after that retarget, resets to the pow limit, and grows taller with less work. The syncer IBD-adopts the heavier tip, keeps it across a reopen, and follows one more heavy block while the tall chain is still ahead |
 | `node_run_p2p_short` | Node (**default**) | Product `run_p2p` `--blocks-only` `--connect` to a live seeder (`--max-tip-age` so the 3-block pad is not stale IBD); process `getpeerinfo` / `getconnectioncount` / `getnetworkinfo` / `getnettotals` / `ping` while connected (v2 outbound-full-relay; handshake `startingheight` equals the seeder tip; `timeoffset` present; `synced_headers`/`synced_blocks` stay `-1` until the peer announces a header hash (empty getheaders at tip does not copy VERSION height); `servicesnames` present; `getnetworkinfo.timeoffset` present); after catch-up `localrelay` / mempool `relay_enabled` stay false and `sendrawtransaction` is not `relay disabled`; Electrum `broadcast` and Esplora `POST /tx` admit decode/consensus errors (not hub-missing / not `relay disabled`); `addconnection inbound` refuses; `disconnectnode` unknown `nodeid` / empty params error then a real addr drops that row from the next `getpeerinfo`, the seeder sees it go, and a second `disconnectnode` is `-29`; `addnode onetry` reconnects as `manual` and the seeder sees the inbound; seeder inbound `tx` then disconnects. Exit via `stop`. `max_run_secs=0` is `node_listen_and_exit`. Mock-clock `timeoffset` median (odd N, even N upper-middle, inbound-only 0, peer clock behind), connecting dummy `-1`, header-only vs connected `synced_blocks`, query-without-chain, `pingwait` / `NETWORK_LIMITED` / `noban` stay RPC guts |
 
 Removed (covered by the rows above): `confirm_cross_block_prevout_without_tx_head`,

changelog.d/sh-pack-resume.md

@@ -0,0 +1,5 @@
+Fixed
+
+- A scripthash pass-1 MPHF without `scripthash.head/NN.packed` is not a durable head. Restart after `DONE.keys` resumes pass 2 and pack instead of reporting Electrum-ready on an index that has no multi-script history. A finished unmarked head is soft-migrated.
+- Electrum stays down when a durable scripthash head's inclusion floor is behind the tip, and the same process binds it once write-behind catches up. A cancelled extract names `scripthash.cold_progress` or `scripthash.unsorted` only when that path is on disk.
+- A tip append onto an unsealed pass-1 head no longer hides the packed multi-script chain. Packing the shard drops that ingest row once main owns the key.