diff --git a/TESTING.md b/TESTING.md index f2e7101e0..b7f133f8b 100644 --- a/TESTING.md +++ b/TESTING.md @@ -395,7 +395,7 @@ A cell that names a crate-local test describes the witness that exists today. It | `serve_after_restart_via_reconstruct` | P2P (**default**) | Cold serve via reconstruct. Restart RAM body queue is empty. Same-process `rehydrate_block_queue_residue` drops at/below tip, skips empty payloads, keeps above-tip wire, unknown height stays queued. `has_block` / known-archived keep and tip+1 gap `missing` stay `bq_rehydrate_residue_keep_drop_gap_and_unknown` | | `ibd_skips_dead_peer` | P2P (**default**) | Live seeder + `127.0.0.1:1` | | `reorg_to_longer_branch` | P2P/chain (default) | Most-work reorg (hub only — no IBD hang risk). With the filter index on, the disconnect truncates basic filters and the branch's filters replace them. Then 20 side blocks received one at a time beat 19 tip-extends (`mempool_reorg.py`), the loser is a `valid-fork` that reconstructs from Class A (not held), an equal-work received branch waits for `preciousblock`, and invalidate / reconsider move between them without parking the old tip | -| `reorg_same_height_then_multi_block_branch` | P2P/chain (default) | Mature pad + competing-spend reorg (third spend is prevout-spent, not `multi-spender`); same-height rival then multi-block reorg near tip; `getchaintips` `active` vs `valid-fork`; 16 vs 17 equal-work siblings park as `valid-headers` (product held cap 320 does not FIFO at 17); `precious_block` the loser; less work ignored; unknown hash `Block not found` for precious and reconsider. Invalidating the tip picks the first-seen equal-work held sibling, precious of the invalidated hash is a no-op that reconsider does not honor. A lone side block is `SideBlock`, a weaker branch is ignored, and an unknown parent errors on submit but is held from a peer. Held cap FIFO stays `hold_body_caps_fifo` | +| `reorg_same_height_then_multi_block_branch` | P2P/chain (default) | Mature pad + competing-spend reorg (third spend is prevout-spent, not `multi-spender`); same-height rival then multi-block reorg near tip; `getchaintips` `active` vs `valid-fork`; 16 vs 17 equal-work siblings park as `valid-headers` (product held cap 320 does not FIFO at 17); `precious_block` the loser; less work ignored; unknown hash `Block not found` for precious and reconsider. Invalidating the tip with no precious preference picks the first-seen equal-work held sibling. With precious set, invalidate adopts that precious equal-work branch, and more total work still beats precious. Precious of an invalidated hash is a no-op that reconsider does not honor. A lone side block is `SideBlock`, a weaker branch is ignored, and an unknown parent errors on submit but is held from a peer. Held cap FIFO stays `hold_body_caps_fifo` | | `three_node_relay_path` | P2P (**default**) | Leaf IBD-syncs from a mid node that already synced (hop serve) | | `ibd_two_peers` | P2P (**default**) | Dual live seeders, 8-block IBD | | `tip_follow_after_ibd` | P2P (**default**) | After IBD, follow + one new tip via inv/headers. With the filter index on, IBD confirm writes no basic filters; `rbtc-idx-wb` materializes them to the tip (its caught-up callback fires once, at the tip), then seals the followed block. With `--sp-tweaks` too: the builder brings both indexes to 5 and is stopped; a followed block with no builder running moves neither (no confirm path writes index data); a new builder seals 6, then follows 7 | diff --git a/changelog.d/core-functional-closer.md b/changelog.d/core-functional-closer.md index d0d8db5da..524c519ad 100644 --- a/changelog.d/core-functional-closer.md +++ b/changelog.d/core-functional-closer.md @@ -5,12 +5,13 @@ Fixed challenge is exactly `OP_TRUE` (signet bits, empty BIP325 solution). `getblockchaininfo.signet_challenge` is present on signet. A signet solution failure is reported as `bad-signet-blksig`. -- **`getblockfilter` serves a stored block that is not sealed yet.** A - best-chain height already in the index still uses that row. A known - block that is stale or not sealed is rebuilt from the body, and its - header is the BIP157 chain from the last sealed header or from genesis. - An unknown `filtertype` is `-5`. REST `/rest/blockfilter/` stays - watermark-only. +- **`getblockfilter` serves a sealed row, or one block past a sealed + parent.** A best-chain height already in the index uses that row. A + block whose parent filter header is sealed, including a stale branch + walked back to that fork point, is rebuilt from the body. An unsealed + best-chain gap is “Filter not found. Block filters are still in the + process of being indexed.” An unknown `filtertype` is `-5`. REST + `/rest/blockfilter/` stays watermark-only. - **`submitpackage` package-evaluates a child-with-parents remainder.** Members that failed static min relay, the dynamic mempool floor, or missing inputs are retried together, including when the child spends a diff --git a/changelog.d/getblockfilter-unsealed-gap.md b/changelog.d/getblockfilter-unsealed-gap.md new file mode 100644 index 000000000..cc94a21f8 --- /dev/null +++ b/changelog.d/getblockfilter-unsealed-gap.md @@ -0,0 +1,7 @@ +Fixed + +- **`getblockfilter` does not rebuild an unsealed index gap.** A best-chain + block whose parent filter header is not sealed returns “Filter not found. + Block filters are still in the process of being indexed.” A block whose + parent header is already sealed is still built, and a stale branch is + rebuilt only back to that sealed fork point. diff --git a/changelog.d/held-header-before-disconnect.md b/changelog.d/held-header-before-disconnect.md new file mode 100644 index 000000000..2ce1c66f7 --- /dev/null +++ b/changelog.d/held-header-before-disconnect.md @@ -0,0 +1,7 @@ +Fixed + +- **A side branch with a bad header no longer rewinds the tip.** Headers on a + held branch are checked before any disconnect. A later block that claims + enormous work without a valid proof of work is remembered invalid, and a + heavier valid held branch can still become the tip. A store fault during + that check is not cached as an invalid block. diff --git a/changelog.d/invalidate-precious-tie.md b/changelog.d/invalidate-precious-tie.md new file mode 100644 index 000000000..43e1af330 --- /dev/null +++ b/changelog.d/invalidate-precious-tie.md @@ -0,0 +1,5 @@ +Fixed + +- **`invalidateblock` of the tip honors `preciousblock` on an equal-work fork.** + The same tie rule as an ordinary reorg applies: more total work still wins, + then the precious branch, then the earlier held tip. diff --git a/changelog.d/package-trim-once.md b/changelog.d/package-trim-once.md new file mode 100644 index 000000000..2fc7680b1 --- /dev/null +++ b/changelog.d/package-trim-once.md @@ -0,0 +1,7 @@ +Fixed + +- **A package is trimmed once, after every member is in.** A parent that is + under the fee floor alone is not evicted before its paying child commits, + so `submitpackage` does not answer `mempool full` for a package that fits. + A package that does not survive that trim is rolled back, so the conflicts + it replaced stay. diff --git a/changelog.d/proposal-fee-overflow.md b/changelog.d/proposal-fee-overflow.md new file mode 100644 index 000000000..4d3324d86 --- /dev/null +++ b/changelog.d/proposal-fee-overflow.md @@ -0,0 +1,6 @@ +Fixed + +- **A block proposal whose fees do not fit in `u64` is rejected.** Output + totals, input totals, and the fee sum use checked addition. An overflowing + sum is `bad-txns-txouttotal-toolarge`, `bad-txns-inputvalues-outofrange`, + or `bad-txns-fee-outofrange` instead of a successful template check. diff --git a/changelog.d/proposal-script-check.md b/changelog.d/proposal-script-check.md new file mode 100644 index 000000000..d8d3ff6d5 --- /dev/null +++ b/changelog.d/proposal-script-check.md @@ -0,0 +1,6 @@ +Fixed + +- **A block proposal with a failing script is rejected.** After the coinbase + amount check, proposal mode runs the block's script flags on the prevouts + it already resolved. No second parent decode and no UTXO write. P2SH and + witness sigops count toward the 80_000 block limit. diff --git a/changelog.d/pure-rbfr-u128.md b/changelog.d/pure-rbfr-u128.md new file mode 100644 index 000000000..d6f2efbbb --- /dev/null +++ b/changelog.d/pure-rbfr-u128.md @@ -0,0 +1,6 @@ +Fixed + +- **Pure replace-by-fee-rate compares the full product.** A replacement whose + `new_fee * 4 * old_vsize` or `direct_fee * 5 * new_vsize` exceeds `u64` + is held to 1.25× in `u128`, instead of a saturating multiply that can + accept a rate below the rule. diff --git a/crates/rbitcoin-mempool/src/accept.rs b/crates/rbitcoin-mempool/src/accept.rs index 685a00fbc..d158b3ca4 100644 --- a/crates/rbitcoin-mempool/src/accept.rs +++ b/crates/rbitcoin-mempool/src/accept.rs @@ -2111,10 +2111,8 @@ pub fn pure_rbfr_pays(new_fee: u64, new_weight: u64, direct_fee: u64, direct_wei if new_v == 0 || old_v == 0 { return false; } - new_fee.saturating_mul(RBFR_RATIO_DEN).saturating_mul(old_v) - >= direct_fee - .saturating_mul(RBFR_RATIO_NUM) - .saturating_mul(new_v) + u128::from(new_fee) * u128::from(RBFR_RATIO_DEN) * u128::from(old_v) + >= u128::from(direct_fee) * u128::from(RBFR_RATIO_NUM) * u128::from(new_v) } /// Admit replacement if BIP125-style rules **or** pure RBFR (Libre). @@ -3036,6 +3034,11 @@ mod tests { assert!(rbf_allows_replacement( 2_000, 4000, 50_000, 40_000, 1_000, 4000 )); + // TESTING.md True journeys: pure arithmetic. A session cannot reach + // this product without a near-supply fee. Equal fees are not 1.25×. + // Saturating `u64` products both become `u64::MAX` and would pass. + let near = u64::MAX / 4 + 1; + assert!(!pure_rbfr_pays(near, 4000, near, 4000)); } #[test] diff --git a/crates/rbitcoin-net/src/chain.rs b/crates/rbitcoin-net/src/chain.rs index 900fa186b..8ae2f147d 100644 --- a/crates/rbitcoin-net/src/chain.rs +++ b/crates/rbitcoin-net/src/chain.rs @@ -249,6 +249,15 @@ pub(crate) fn reject_is_mutated(reason: &str) -> bool { /// Core logs a contextual header reject (`bad-version`, `time-too-new`) with /// its reason. The returned error keeps the store-facing Display string. +/// A consensus header failure names the block so it can be marked invalid. +/// A store fault stays a store fault: it must not be cached as `BLOCK_FAILED`. +fn connect_failed_for_header(hash: [u8; 32], e: NetError) -> NetError { + match e { + NetError::Consensus(msg) => NetError::ConnectFailed { hash, msg }, + other => other, + } +} + fn header_reject(header: &Header, e: &rbitcoin_consensus::ConsensusError) -> NetError { if let rbitcoin_consensus::ConsensusError::Store(se) = e { return NetError::store(se); @@ -351,6 +360,22 @@ pub struct ChainTipInfo { pub status: &'static str, } +/// Equal total work prefers the precious tip, then the earlier held tip. +/// A precious hash missing from the hold map has sequence `u64::MAX`; the +/// precious bit wins before that comparison. +fn held_branch_beats( + candidate: Work, + candidate_precious: bool, + candidate_seq: u64, + incumbent: Work, + incumbent_precious: bool, + incumbent_seq: u64, +) -> bool { + let tie = !work_better(incumbent, candidate); + work_better(candidate, incumbent) + || (tie && !incumbent_precious && (candidate_precious || candidate_seq < incumbent_seq)) +} + impl ChainHub { pub fn new(query: Query, params: ChainParams, milestone: Milestone) -> Self { let (tip_tx, _) = broadcast::channel(64); @@ -1937,17 +1962,19 @@ impl ChainHub { } /// After invalidate, activate the remaining fork (held or archive) with - /// the most total chain work. Equal work keeps the first-seen held tip. + /// the most total chain work. Equal work prefers precious, then the + /// first-seen held tip. fn try_apply_after_invalidate(&self) -> Result, NetError> { let inv = self.invalidated.set.read().unwrap().clone(); + let precious = *self.precious.read().unwrap(); let mut starts: Vec = self.fork_tips.read().unwrap().iter().copied().collect(); starts.extend(self.held_bodies.read().unwrap().keys()); - if let Some(p) = *self.precious.read().unwrap() { + if let Some(p) = precious { if !starts.contains(&p) { starts.push(p); } } - let mut best: Option<(bitcoin::Work, u64, Vec)> = None; + let mut best: Option<(bitcoin::Work, bool, u64, Vec)> = None; for start in starts { if inv.contains(&start) { continue; @@ -1962,18 +1989,17 @@ impl ChainHub { let Ok(w) = self.branch_chain_work(&branch) else { continue; }; + let is_p = Some(tip) == precious; let seq = self.held_bodies.read().unwrap().seq(tip); let take = match &best { None => true, - Some((bw, bseq, _)) => { - work_better(w, *bw) || (w.to_be_bytes() == bw.to_be_bytes() && seq < *bseq) - } + Some((bw, was_p, bseq, _)) => held_branch_beats(w, is_p, seq, *bw, *was_p, *bseq), }; if take { - best = Some((w, seq, branch)); + best = Some((w, is_p, seq, branch)); } } - let Some((_, _, branch)) = best else { + let Some((_, _, _, branch)) = best else { return Ok(None); }; match self.accept_branch_inner(&branch) { @@ -2263,6 +2289,7 @@ impl ChainHub { if self.accept_branch_weaker(blocks, old_work)? { return Ok((AcceptOutcome::IgnoredWeaker, None)); } + self.accept_branch_check_headers(blocks, fork_height)?; let old_path = self.accept_branch_collect_old(fork_height)?; self.accept_branch_disconnect(fork_height)?; let base = fork_height.map(|h| h + 1).unwrap_or(0); @@ -2405,6 +2432,49 @@ impl ChainHub { Ok(old_path) } + /// Hash and bits for every header before any `disconnect_to`. The first + /// block's parent is on the best chain. Later parents are this branch. + fn accept_branch_check_headers( + &self, + blocks: &[Block], + fork_height: Option, + ) -> Result<(), NetError> { + let Some(fork_h) = fork_height else { + return Ok(()); + }; + let mut batch = HashMap::with_capacity(blocks.len()); + for (i, b) in blocks.iter().enumerate() { + let height = fork_h.saturating_add(1).saturating_add(i as u32); + let checked = if i == 0 { + validate_header(self.query.as_ref(), &self.params, Height(height), &b.header) + .map_err(|e| header_reject(&b.header, &e)) + } else { + let parent = &blocks[i - 1].header; + let mtp = self.mtp_off_tip(parent, &batch); + let expected = self.expected_bits_off_tip( + &b.header, + parent, + height.saturating_sub(1), + &batch, + )?; + validate_header_on_parent(&self.params, Height(height), &b.header, mtp, expected) + .map_err(|e| header_reject(&b.header, &e)) + }; + if let Err(e) = checked { + return Err(connect_failed_for_header(b.block_hash().to_byte_array(), e)); + } + batch.insert( + b.block_hash().to_byte_array(), + HeaderSyncNode { + fk: Fk(0), + header: b.header, + height: Some(height), + }, + ); + } + Ok(()) + } + fn accept_branch_disconnect(&self, fork_height: Option) -> Result<(), NetError> { if let Some(fh) = fork_height { self.disconnect_to(fh)?; @@ -2778,10 +2848,7 @@ impl ChainHub { let seq = self.held_bodies.read().unwrap().seq(tip); let take = match &best { None => true, - Some((bw, was_p, bseq, _)) => { - let tie = !work_better(*bw, w); - work_better(w, *bw) || (tie && !*was_p && (is_p || seq < *bseq)) - } + Some((bw, was_p, bseq, _)) => held_branch_beats(w, is_p, seq, *bw, *was_p, *bseq), }; if take { best = Some((w, is_p, seq, branch)); @@ -3147,28 +3214,60 @@ pub fn check_block_proposal_with( if let Err(e) = rbitcoin_consensus::validate_block_structure(block, &vctx) { return Err(rbitcoin_consensus::block_reject_reason(&e)); } - let fees = proposal_connect(query, block, height, mtp, params.coinbase_maturity())?; + let (fees, prevouts) = proposal_connect(query, block, height, mtp, params.coinbase_maturity())?; let subsidy = rbitcoin_consensus::block_subsidy(height, params) as u64; - let coinbase_out = block.txdata[0] - .output - .iter() - .fold(0u64, |acc, o| acc.saturating_add(o.value.to_sat())); - if coinbase_out > subsidy.saturating_add(fees) { + let mut coinbase_out = 0u64; + for o in &block.txdata[0].output { + coinbase_out = coinbase_out + .checked_add(o.value.to_sat()) + .ok_or("bad-txns-txouttotal-toolarge")?; + } + let allowed = subsidy.checked_add(fees).ok_or("bad-txns-fee-outofrange")?; + if coinbase_out > allowed { return Err("bad-cb-amount".into()); } + let flags = rbitcoin_consensus::ScriptVerifyFlags::for_block( + params, + height, + &block.block_hash().to_byte_array(), + mtp, + ); + // Structure counts legacy sigops only. Prevouts are already in hand, so + // P2SH and witness sigops count here (BIP141 limit, 80_000). + const MAX_BLOCK_SIGOPS_COST: u64 = 80_000; + let mut sigops = rbitcoin_consensus::tx_sigop_cost( + &block.txdata[0], + &[], + flags.bip16_active, + flags.witness_active, + ); + for (tx, ins) in block.txdata.iter().skip(1).zip(prevouts) { + let prev_spks: Vec<&[u8]> = ins.iter().map(|o| o.script_pubkey.as_bytes()).collect(); + sigops = sigops.saturating_add(rbitcoin_consensus::tx_sigop_cost( + tx, + &prev_spks, + flags.bip16_active, + flags.witness_active, + )); + if sigops > MAX_BLOCK_SIGOPS_COST { + return Err("bad-blk-sigops".into()); + } + rbitcoin_consensus::verify_tx_scripts_with_flags(ins, tx.clone(), flags) + .map_err(|e| rbitcoin_consensus::block_reject_reason(&e))?; + } Ok(fees) } /// Resolves every spend against the block and the confirmed chain. `Ok` is -/// the fee total over the non-coinbase txs. Caller has already passed -/// [`rbitcoin_consensus::validate_block_structure`]. +/// the fee total and the prevouts already resolved for each non-coinbase tx. +/// Caller has already passed [`rbitcoin_consensus::validate_block_structure`]. fn proposal_connect( query: &Query, block: &Block, height: u32, mtp: u32, maturity: u32, -) -> Result { +) -> Result<(u64, Vec>), String> { if block.txdata.is_empty() { return Err("bad-blk-length".into()); } @@ -3182,6 +3281,7 @@ fn proposal_connect( // Dropped when the check returns. The fk resolve does not decode the body. let mut parents: HashMap)> = HashMap::new(); let mut fees = 0u64; + let mut prevouts = Vec::with_capacity(block.txdata.len().saturating_sub(1)); for (i, tx) in block.txdata.iter().enumerate() { if !rbitcoin_consensus::is_final_tx(tx, height, mtp.max(block.header.time)) { return Err("bad-txns-nonfinal".into()); @@ -3199,6 +3299,7 @@ fn proposal_connect( continue; } let mut in_val = 0u64; + let mut tx_prevouts = Vec::with_capacity(tx.input.len()); for inp in &tx.input { let op = inp.previous_output; if !spent.insert(op) { @@ -3219,13 +3320,24 @@ fn proposal_connect( } else { return Err("bad-txns-inputs-missingorspent".into()); }; - in_val = in_val.saturating_add(txout.value.to_sat()); + in_val = in_val + .checked_add(txout.value.to_sat()) + .ok_or("bad-txns-inputvalues-outofrange")?; + tx_prevouts.push(txout); + } + let mut out_val = 0u64; + for o in &tx.output { + out_val = out_val + .checked_add(o.value.to_sat()) + .ok_or("bad-txns-txouttotal-toolarge")?; } - let out_val: u64 = tx.output.iter().map(|o| o.value.to_sat()).sum(); if out_val > in_val { return Err("bad-txns-in-belowout".into()); } - fees = fees.saturating_add(in_val - out_val); + fees = fees + .checked_add(in_val - out_val) + .ok_or("bad-txns-fee-outofrange")?; + prevouts.push(tx_prevouts); let tid = tx.compute_txid(); for (vout, o) in tx.output.iter().enumerate() { created.insert( @@ -3237,7 +3349,7 @@ fn proposal_connect( ); } } - Ok(fees) + Ok((fees, prevouts)) } /// `op`'s unspent confirmed output. `parents` holds each parent's create fk @@ -6238,6 +6350,84 @@ mod tests { let _ = std::fs::remove_dir_all(dir); } + #[test] + fn store_fault_during_header_check_is_not_a_failed_block() { + let hash = [0x11; 32]; + let store = connect_failed_for_header(hash, NetError::Store("budget full: SQ".into())); + assert!( + matches!(store, NetError::Store(_)), + "a store fault must not name a block, got {store}" + ); + let bad = connect_failed_for_header(hash, NetError::Consensus("bad-diffbits".into())); + match bad { + NetError::ConnectFailed { hash: h, msg } => { + assert_eq!(h, hash); + assert!(msg.contains("bad-diffbits"), "{msg}"); + } + other => panic!("consensus header failure must name the block, got {other}"), + } + } + + /// Witness sigops are not in the structure walk. Nine P2WSH inputs of + /// 10_000 `OP_CHECKSIG` each are over the 80_000 block limit. + #[test] + fn check_block_proposal_rejects_witness_sigops_over_the_limit() { + let (dir, hub) = tmp_hub(); + let op_true = ScriptBuf::from_bytes(vec![0x51]); + hub.generate_to_script(101, op_true.clone(), vec![]) + .unwrap(); + let ws = ScriptBuf::from_bytes(vec![0xac; 10_000]); + let p2wsh = ScriptBuf::new_p2wsh(&ws.wscript_hash()); + let cb = hub + .query + .reconstruct_block_at_height(Height(1)) + .unwrap() + .txdata[0] + .compute_txid(); + let per = (50_0000_0000u64 - 1_000) / 9; + let fan = Transaction { + version: TxVersion::TWO, + lock_time: LockTime::ZERO, + input: vec![TxIn { + previous_output: OutPoint { txid: cb, vout: 0 }, + script_sig: ScriptBuf::new(), + sequence: Sequence::ENABLE_RBF_NO_LOCKTIME, + witness: Witness::new(), + }], + output: (0..9) + .map(|_| TxOut { + value: Amount::from_sat(per), + script_pubkey: p2wsh.clone(), + }) + .collect(), + }; + let fan_id = fan.compute_txid(); + let spend = Transaction { + version: TxVersion::TWO, + lock_time: LockTime::ZERO, + input: (0..9u32) + .map(|vout| TxIn { + previous_output: OutPoint { txid: fan_id, vout }, + script_sig: ScriptBuf::new(), + sequence: Sequence::ENABLE_RBF_NO_LOCKTIME, + witness: Witness::from_slice(&[ws.as_bytes()]), + }) + .collect(), + output: vec![TxOut { + value: Amount::from_sat(per * 9 - 1_000), + script_pubkey: op_true.clone(), + }], + }; + let block = hub + .assemble_block_to_script(op_true, vec![fan, spend]) + .unwrap(); + assert_eq!( + hub.check_block_proposal(&block).unwrap_err(), + "bad-blk-sigops" + ); + let _ = std::fs::remove_dir_all(dir); + } + /// Core prices the coinbase in ConnectBlock, after CheckBlock: a coinbase /// over subsidy + fees is `bad-cb-amount` once the structure checks pass, /// and a structure reject on the same block still wins. diff --git a/crates/rbitcoin-net/src/tx_relay.rs b/crates/rbitcoin-net/src/tx_relay.rs index 29e300fa7..441660964 100644 --- a/crates/rbitcoin-net/src/tx_relay.rs +++ b/crates/rbitcoin-net/src/tx_relay.rs @@ -2088,7 +2088,7 @@ impl MempoolHub { let sample_p = self.sample_chain_coins(&parent, utxo); let parent_commit = { let mut g = self.lock_write(); - Self::commit_rechecked(&mut g, &parent, prep_p, utxo, false, &sample_p) + Self::commit_rechecked(&mut g, &parent, prep_p, utxo, true, &sample_p) }; let (parent_res, prevouts_p) = match parent_commit.result { Ok(r) => (r, parent_commit.prevouts), @@ -2116,11 +2116,17 @@ impl MempoolHub { let sample_c = self.sample_chain_coins(child, utxo); let child_commit = { let mut g = self.lock_write(); - Self::commit_rechecked(&mut g, child, prep_c, utxo, false, &sample_c) + Self::commit_rechecked(&mut g, child, prep_c, utxo, true, &sample_c) }; self.meter_accept_stages(lock_us, stages); match child_commit.result { Ok(r) => { + self.trim_over_budget(); + if !self.try_contains(&parent_res.txid) || !self.try_contains(&r.txid) { + self.rollback_package_accepted(&[parent_res, r]); + self.publish_fee_floor(); + return None; + } self.publish_admitted(&parent, &parent_res, &prevouts_p, utxo); self.publish_admitted(child, &r, &child_commit.prevouts, utxo); let _ = self.expire_stale(); @@ -2669,7 +2675,7 @@ impl MempoolHub { let committed = { let mut g = self.lock_write(); g.last_accept_stages = stages; - let committed = Self::commit_rechecked(&mut g, tx, prep, &utxo, false, &sample); + let committed = Self::commit_rechecked(&mut g, tx, prep, &utxo, true, &sample); stages = g.last_accept_stages; committed }; @@ -2692,6 +2698,18 @@ impl MempoolHub { } } } + self.trim_over_budget(); + if accepted.iter().any(|r| !self.try_contains(&r.txid)) { + // Trim dropped a member that had already replaced its conflicts. + // Put those conflicts back; the package itself does not stay. + self.rollback_package_accepted(&accepted); + self.publish_fee_floor(); + let us = t0.elapsed().as_micros() as u64; + self.meter_accept_stages(lock_us, stages); + return Err(self + .finish_accept_err(us, AcceptError::Policy("mempool full")) + .unwrap_err()); + } let us = t0.elapsed().as_micros() as u64; self.meter_accept_stages(lock_us, stages); let per = us / (accepted.len().max(1) as u64); @@ -3465,6 +3483,11 @@ impl MempoolHub { self.lock_write().set_bytes_per_sigop(bytes_per_sigop); } + /// Weight budget in WU. + pub fn set_max_weight(&self, wu: u64) { + self.lock_write().max_weight = wu; + } + /// Min-relay overlay (sat/kvB). `0` admits any non-negative fee. pub fn set_min_relay_sat_kvb(&self, sat_kvb: u64) { self.min_relay_sat_kvb.store(sat_kvb, Ordering::Release); @@ -5434,6 +5457,46 @@ mod tests { let _ = std::fs::remove_dir_all(&dir); } + /// The orphan 1p1c path commits both members before the trim. When the + /// trim drops them, the conflict the parent replaced comes back. + #[test] + fn failed_1p1c_trim_restores_the_conflict() { + let (_store, q, cbs) = pad_cbs(3); + let spk = ScriptBuf::from_bytes(vec![0x51]); + let victim = spend_true(cbs[0], 1_000, spk.clone()); + let victim_id = victim.compute_txid(); + let dir = tmp(); + let hub = MempoolHub::open_with_weight(&dir, q, victim.weight().to_wu() + 400).unwrap(); + hub.set_relay_enabled(true); + hub.accept_tx(&victim).expect("victim"); + let parent = spend_true(cbs[0], 20_000, spk.clone()); + let parent_id = parent.compute_txid(); + assert!(hub.try_note_extra_compact(&parent)); + let mut child = spend_vout( + OutPoint { + txid: parent_id, + vout: 0, + }, + parent.output[0].value.to_sat() - 5_000, + ); + child.output[0].script_pubkey = ScriptBuf::from_bytes(vec![0x51; 4_000]); + let err = hub.accept_tx(&child); + assert!( + err.is_err(), + "an over-budget 1p1c must not stay, got {err:?}" + ); + assert!( + !hub.contains(&parent_id), + "trimmed parent must be rolled back" + ); + assert!(!hub.contains(&child.compute_txid())); + assert!( + hub.contains(&victim_id), + "the conflict the parent replaced must be restored" + ); + let _ = std::fs::remove_dir_all(&dir); + } + /// Eviction keeps a prioritisation delta. A later duplicate of the /// survivor does not apply that eviction again. Mining drops the delta. #[test] diff --git a/crates/rbitcoin-query/src/block_filter.rs b/crates/rbitcoin-query/src/block_filter.rs index 8c591f391..7bcc49a5e 100644 --- a/crates/rbitcoin-query/src/block_filter.rs +++ b/crates/rbitcoin-query/src/block_filter.rs @@ -18,6 +18,10 @@ use crate::{Query, QueryError}; use std::sync::atomic::Ordering; use std::sync::{Condvar, Mutex}; +/// Core `getblockfilter` when the index has not reached this block. +const FILTER_STILL_INDEXING: &str = + "Filter not found. Block filters are still in the process of being indexed."; + /// BIP158 basic filter Golomb-Rice parameters (`M`, `P`). const BASIC_FILTER_M: u64 = 784_931; const BASIC_FILTER_P: u8 = 19; @@ -224,10 +228,11 @@ impl Query { /// Filter bytes and header for `hash`. /// - /// Best-chain heights use the sealed index when the watermark has caught - /// up. A hash that is stored but not on the best chain (or not sealed - /// yet) is built from the Class A body: non-`OP_RETURN` output scripts, - /// then each spent prevout script. + /// A sealed best-chain row is that row. Otherwise the parent filter + /// header is resolved first: a sealed best-chain parent, the zero + /// genesis header, or a stale branch walked back to that sealed fork + /// point. An unsealed best-chain gap is refused before the body is + /// rebuilt. pub fn basic_filter_for_hash( &self, hash: &[u8; 32], @@ -236,15 +241,23 @@ impl Query { if let Some(row) = self.basic_filter_at(h.0)? { return Ok(Some(row)); } - let block = self.reconstruct_block_at_height(h)?; - return Ok(Some(self.basic_filter_from_wire_block(&block)?)); } - let Some(block) = self.reconstruct_archived_block(hash)? else { + let Some(prev) = self.header_prev_hash(hash)? else { + return Ok(None); + }; + let parent_header = self.parent_basic_filter_header(&prev)?; + let Some(block) = self.block_body_for_filter(hash)? else { return Ok(None); }; - Ok(Some(self.basic_filter_from_wire_block(&block)?)) + let spent = self.spent_scripts(&block)?; + let filter = self.basic_filter_content(&block, &spent)?; + Ok(Some(( + filter.content.clone(), + filter.filter_header(&parent_header), + ))) } + #[cfg(test)] fn basic_filter_from_wire_block( &self, block: &Block, @@ -271,9 +284,9 @@ impl Query { basic_filter_from_scripts(&hash, outputs, spent_refs) } - /// Header of `prev_hash`. Zeros are only the genesis prev-header. A sealed - /// best-chain row is that row; every other stored block is rebuilt back - /// to the last sealed header, or to genesis, and linked in chain order. + /// Header of `prev_hash`. Zeros are the genesis prev-header. A sealed + /// best-chain block is that row. A stale block is rebuilt back to the + /// best-chain fork point, and only when that fork point is sealed. fn parent_basic_filter_header(&self, prev_hash: &[u8; 32]) -> Result { if *prev_hash == [0u8; 32] { return Ok(FilterHeader::from_byte_array([0u8; 32])); @@ -289,6 +302,7 @@ impl Query { if let Some((_, header)) = self.basic_filter_at(h.0)? { break header; } + return Err(StoreError::Rejected(FILTER_STILL_INDEXING)); } if !seen.insert(cursor) { return Err(StoreError::Corrupt("invariant: blockfilter header cycle")); @@ -298,7 +312,11 @@ impl Query { }; let mut header = anchor; for hash in pending.iter().rev() { - let block = self.block_for_basic_filter(hash)?; + let Some(block) = self.block_body_for_filter(hash)? else { + return Err(StoreError::Corrupt( + "invariant: blockfilter parent body missing", + )); + }; let spent = self.spent_scripts(&block)?; let filter = self.basic_filter_content(&block, &spent)?; header = filter.filter_header(&header); @@ -306,26 +324,27 @@ impl Query { Ok(header) } - fn prev_header_hash(&self, hash: &[u8; 32]) -> Result<[u8; 32], QueryError> { + fn header_prev_hash(&self, hash: &[u8; 32]) -> Result, QueryError> { let Some((_, rec)) = self.get_header_by_hash(hash)? else { - return Err(StoreError::Corrupt( - "invariant: blockfilter parent header missing", - )); + return Ok(None); }; if rec.prev_fk.is_null() { - return Ok([0u8; 32]); + return Ok(Some([0u8; 32])); } - Ok(self.get_header(rec.prev_fk)?.hash) + Ok(Some(self.get_header(rec.prev_fk)?.hash)) + } + + fn prev_header_hash(&self, hash: &[u8; 32]) -> Result<[u8; 32], QueryError> { + self.header_prev_hash(hash)?.ok_or(StoreError::Corrupt( + "invariant: blockfilter parent header missing", + )) } - fn block_for_basic_filter(&self, hash: &[u8; 32]) -> Result { + fn block_body_for_filter(&self, hash: &[u8; 32]) -> Result, QueryError> { if let Some(h) = self.height_of_hash(hash)? { - return self.reconstruct_block_at_height(h); + return Ok(Some(self.reconstruct_block_at_height(h)?)); } - self.reconstruct_archived_block(hash)? - .ok_or(StoreError::Corrupt( - "invariant: blockfilter parent body missing", - )) + self.reconstruct_archived_block(hash) } fn spent_scripts(&self, block: &Block) -> Result>, QueryError> { diff --git a/crates/rbitcoin-rpc/src/methods_tests.rs b/crates/rbitcoin-rpc/src/methods_tests.rs index e40e73741..87d34adcc 100644 --- a/crates/rbitcoin-rpc/src/methods_tests.rs +++ b/crates/rbitcoin-rpc/src/methods_tests.rs @@ -325,14 +325,6 @@ fn getblockfilter_rebuilds_unsealed_and_chains_from_a_sealed_parent() { use bitcoin::bip158::BlockFilter; let (ctx, dir, _hub) = ctx_from_mature_pad(300_000_000); - let (hex, _spend) = spend_generated_coinbase( - &ctx, - 1, - 50_0000_0000 - 1_000, - ScriptBuf::from_bytes(vec![0x51]), - ); - dispatch(&ctx, "sendrawtransaction", vec![json!(hex)]).unwrap(); - let tip = dispatch(&ctx, "generate", vec![json!(1)]).unwrap()[0].clone(); ctx.query.set_block_filter_index(true).unwrap(); assert_eq!(ctx.query.filter_index_next(), Some(0)); @@ -340,53 +332,29 @@ fn getblockfilter_rebuilds_unsealed_and_chains_from_a_sealed_parent() { assert_eq!(missing["code"], ERR_INVALID_ADDRESS_OR_KEY); assert_eq!(missing["message"], "Block not found"); + let tip = dispatch(&ctx, "getbestblockhash", vec![]).unwrap(); + let behind = dispatch(&ctx, "getblockfilter", vec![tip]).unwrap_err(); + assert_eq!(behind["code"], ERR_MISC); + assert_eq!( + behind["message"], + "Filter not found. Block filters are still in the process of being indexed." + ); + let genesis = dispatch(&ctx, "getblockhash", vec![json!(0)]).unwrap(); let height1 = dispatch(&ctx, "getblockhash", vec![json!(1)]).unwrap(); let height2 = dispatch(&ctx, "getblockhash", vec![json!(2)]).unwrap(); let g = dispatch(&ctx, "getblockfilter", vec![genesis.clone()]).unwrap(); - let one_before = dispatch(&ctx, "getblockfilter", vec![height1.clone()]).unwrap(); - let two_before = dispatch(&ctx, "getblockfilter", vec![height2]).unwrap(); - let tip_before = dispatch(&ctx, "getblockfilter", vec![tip.clone()]).unwrap(); - assert!( - tip_before["filter"].as_str().unwrap().len() > 2, - "{tip_before}" - ); let zero = "00".repeat(32); assert_eq!( g["header"], json!(filter_header_hex(g["filter"].as_str().unwrap(), &zero)), "genesis filter header chains from the zero prev-header" ); + let gap = dispatch(&ctx, "getblockfilter", vec![height1.clone()]).unwrap_err(); assert_eq!( - one_before["header"], - json!(filter_header_hex( - one_before["filter"].as_str().unwrap(), - g["header"].as_str().unwrap(), - )), - "height 1 chains from the genesis filter header before anything is sealed, got {one_before}" - ); - assert_eq!( - two_before["header"], - json!(filter_header_hex( - two_before["filter"].as_str().unwrap(), - one_before["header"].as_str().unwrap(), - )), - "height 2 chains through an unsealed parent, got {two_before}" - ); - let tip_block = dispatch(&ctx, "getblock", vec![tip.clone()]).unwrap(); - let parent_before = dispatch( - &ctx, - "getblockfilter", - vec![tip_block["previousblockhash"].clone()], - ) - .unwrap(); - assert_eq!( - tip_before["header"], - json!(filter_header_hex( - tip_before["filter"].as_str().unwrap(), - parent_before["header"].as_str().unwrap(), - )), - "tip chains from its unsealed parent, got {tip_before}" + gap["message"], + "Filter not found. Block filters are still in the process of being indexed.", + "{gap}" ); let body = rbitcoin_primitives::hex_decode(g["filter"].as_str().unwrap()).unwrap(); @@ -403,25 +371,64 @@ fn getblockfilter_rebuilds_unsealed_and_chains_from_a_sealed_parent() { .unwrap(), 1 ); - let g_sealed = dispatch(&ctx, "getblockfilter", vec![genesis]).unwrap(); - assert_eq!(g_sealed, g, "sealed genesis filter must match the rebuild"); - let one = dispatch(&ctx, "getblockfilter", vec![height1]).unwrap(); - assert_eq!(one["filter"], one_before["filter"]); assert_eq!( - one["header"], one_before["header"], - "sealing the parent must not change the child filter header" + one["header"], + json!(filter_header_hex( + one["filter"].as_str().unwrap(), + g["header"].as_str().unwrap(), + )), + "height 1 is built once its parent filter header is sealed, got {one}" + ); + let still = dispatch(&ctx, "getblockfilter", vec![height2.clone()]).unwrap_err(); + assert_eq!( + still["message"], + "Filter not found. Block filters are still in the process of being indexed.", + "{still}" ); - dispatch(&ctx, "invalidateblock", vec![tip.clone()]).unwrap(); - let stale = dispatch(&ctx, "getblockfilter", vec![tip]).unwrap(); + // Height 3's parent is height 2. Invalidating height 2 leaves both stale, + // forked from height 1, whose filter is not sealed yet. + let height3 = dispatch(&ctx, "getblockhash", vec![json!(3)]).unwrap(); + dispatch(&ctx, "invalidateblock", vec![height2.clone()]).unwrap(); + let stale_gap = dispatch(&ctx, "getblockfilter", vec![height3.clone()]).unwrap_err(); assert_eq!( - stale["filter"], tip_before["filter"], - "a stored block off the best chain still has a basic filter" + stale_gap["message"], + "Filter not found. Block filters are still in the process of being indexed.", + "{stale_gap}" ); + + let one_body = rbitcoin_primitives::hex_decode(one["filter"].as_str().unwrap()).unwrap(); + let one_fk = ctx + .query + .store() + .confirmed + .get(Height(1)) + .unwrap() + .expect("height 1 header"); assert_eq!( - stale["header"], tip_before["header"], - "a stale block keeps the same BIP157 header, got {stale}" + ctx.query + .commit_window_filters(1, &[(BlockFilter::new(&one_body), one_fk)]) + .unwrap(), + 1 + ); + let stale_two = dispatch(&ctx, "getblockfilter", vec![height2]).unwrap(); + assert_eq!( + stale_two["header"], + json!(filter_header_hex( + stale_two["filter"].as_str().unwrap(), + one["header"].as_str().unwrap(), + )), + "a stale block builds from its sealed best-chain parent, got {stale_two}" + ); + let stale_three = dispatch(&ctx, "getblockfilter", vec![height3]).unwrap(); + assert_eq!( + stale_three["header"], + json!(filter_header_hex( + stale_three["filter"].as_str().unwrap(), + stale_two["header"].as_str().unwrap(), + )), + "a stale grandchild walks back to the sealed fork point, got {stale_three}" ); let _ = std::fs::remove_dir_all(&dir); } @@ -2635,7 +2642,22 @@ fn mempool_under_pressure() { assert_eq!(v["error"], json!("package-not-validated"), "{sub}"); } + // Height 3's coinbase is one block short of maturity on the 101-block pad. + dispatch(&ctx, "generate", vec![json!(1)]).unwrap(); pressure_admit_then_cluster(&ctx, &spk); + + // Parent is under the relay floor alone. The pool is already heavier than + // the package, and the cap fits the package only. Trimming between the + // two members used to evict the parent and answer `mempool full`. + let (low_hex, low_parent) = spend_generated_coinbase(&ctx, 3, 50_0000_0000 - 1, spk.clone()); + let low_child = child_of(&low_parent, 80_000); + let low_child_hex = hex_encode(serialize(&low_child)); + let mp = ctx.mempool.as_ref().unwrap(); + mp.set_max_weight(low_parent.weight().to_wu() + low_child.weight().to_wu()); + let kept = dispatch(&ctx, "submitpackage", vec![json!([low_hex, low_child_hex])]).unwrap(); + assert_eq!(kept["package_msg"], "success", "{kept}"); + assert!(mp.contains(&low_parent.compute_txid())); + assert!(mp.contains(&low_child.compute_txid())); let _ = std::fs::remove_dir_all(&dir); } @@ -5426,14 +5448,9 @@ fn shipped_rpc_reads_cover_fallback_and_reject_arms() { "getblockfilter", vec![json!(hub.tip_hash().unwrap().to_string())], ) - .unwrap(); + .unwrap_err(); assert!( - tip_filter["filter"].as_str().unwrap_or("").len() > 2, - "an unsealed tip is built on demand: {tip_filter}" - ); - assert_eq!( - tip_filter["header"].as_str().unwrap_or("").len(), - 64, + rpc_message(&tip_filter).contains("still in the process of being indexed"), "{tip_filter}" ); let unknown_filter = dispatch( diff --git a/crates/rbitcoin-rpc/src/regtest_chain_ops_journey.rs b/crates/rbitcoin-rpc/src/regtest_chain_ops_journey.rs index ddea4ba1d..7e50a07c0 100644 --- a/crates/rbitcoin-rpc/src/regtest_chain_ops_journey.rs +++ b/crates/rbitcoin-rpc/src/regtest_chain_ops_journey.rs @@ -1516,6 +1516,14 @@ fn chain_ops_proposal_spends(ctx: &RpcContext, cbs: &mut TrueCoinbases) { let cb_f = generated_coinbase_value(ctx, f); let (hex, spend) = spend_generated_coinbase(ctx, f, cb_f - 1_000, true_spk()); assert!(propose(ctx, &proposal_on_tip(ctx, vec![spend.clone()])).is_null()); + let mut bad_script = spend.clone(); + bad_script.input[0].script_sig = ScriptBuf::from_bytes(vec![0x00, 0x69]); + let got = propose(ctx, &proposal_on_tip(ctx, vec![bad_script])); + let reason = got.as_str().unwrap_or(""); + assert!( + reason.starts_with("block-script-verify-flag-failed"), + "a failing script must not validate: {got}" + ); assert_eq!( propose(ctx, &proposal_on_tip(ctx, vec![spend.clone(), spend.clone()])), "bad-txns-inputs-missingorspent", @@ -1550,10 +1558,30 @@ fn chain_ops_proposal_spends(ctx: &RpcContext, cbs: &mut TrueCoinbases) { dispatch(ctx, "sendrawtransaction", vec![json!(hex)]).unwrap(); dispatch(ctx, "generate", vec![json!(1)]).unwrap(); assert_eq!( - propose(ctx, &proposal_on_tip(ctx, vec![spend])), + propose(ctx, &proposal_on_tip(ctx, vec![spend.clone()])), "bad-txns-inputs-missingorspent", "a coin a block spent is gone from the proposal view" ); + + // Each output is MAX_MONEY, so CheckBlock's per-output gate passes, but + // the sum does not fit in u64. Proposal mode must not return success. + use bitcoin::TxOut; + const MAX_MONEY: u64 = 21_000_000 * 100_000_000; + let mut overflow = spend.clone(); + overflow.output = (0..8785) + .map(|_| TxOut { + value: Amount::from_sat(MAX_MONEY), + script_pubkey: true_spk(), + }) + .collect(); + let got = propose(ctx, &proposal_on_tip(ctx, vec![overflow])); + let reason = got.as_str().unwrap_or(""); + assert!( + reason == "bad-txns-inputvalues-outofrange" + || reason == "bad-txns-txouttotal-toolarge" + || reason == "bad-txns-fee-outofrange", + "overflowing output sum must not validate: {got}" + ); } /// Core reports a repeated txid by where it fails: a repeat that keeps the diff --git a/crates/rbitcoin-test/tests/integration_multinode.rs b/crates/rbitcoin-test/tests/integration_multinode.rs index 7f1275779..b18218f2c 100644 --- a/crates/rbitcoin-test/tests/integration_multinode.rs +++ b/crates/rbitcoin-test/tests/integration_multinode.rs @@ -2509,6 +2509,65 @@ fn pin_precious_held_chaintips(hub: &rbitcoin_net::ChainHub, ext: bitcoin::Block assert!(err.to_string().contains("Block not found"), "{err}"); let err = hub.reconsider_block(miss).unwrap_err(); assert!(err.to_string().contains("Block not found"), "{err}"); + + // Tip has one more block, so the held siblings are weaker and precious + // does not activate yet. After that extra block is invalidated, more work + // still beats precious, and precious then beats the earlier equal sibling. + let top = mine_regtest_block( + sibling.block_hash(), + sibling.header.time.saturating_add(600), + tip_h + 1, + vec![], + ); + match hub.accept_block(top.clone()).unwrap() { + AcceptOutcome::Accepted { height } => assert_eq!(height, tip_h + 1), + other => panic!("expected the extra tip block, got {other:?}"), + } + let later = mine_regtest_block( + p_prev_hash, + p_prev.timestamp.saturating_add(2_500), + tip_h, + vec![], + ); + assert!(matches!( + hub.accept_received_block(later.clone()).unwrap(), + AcceptOutcome::IgnoredWeaker + )); + hub.precious_block(later.block_hash()).unwrap(); + assert_eq!( + hub.tip_hash().unwrap(), + top.block_hash(), + "precious of less work must not activate" + ); + let heavy_1 = mine_regtest_block( + p_prev_hash, + p_prev.timestamp.saturating_add(2_600), + tip_h, + vec![], + ); + let heavy_2 = mine_regtest_block( + heavy_1.block_hash(), + heavy_1.header.time.saturating_add(600), + tip_h + 1, + vec![], + ); + for b in [&heavy_1, &heavy_2] { + hub.accept_received_block(b.clone()) + .unwrap_or_else(|e| panic!("heavier {}: {e}", b.block_hash())); + } + assert_eq!(hub.tip_hash().unwrap(), top.block_hash()); + hub.invalidate_block(top.block_hash()).unwrap(); + assert_eq!( + hub.tip_hash().unwrap(), + heavy_2.block_hash(), + "more work must beat precious after invalidate" + ); + hub.invalidate_block(heavy_1.block_hash()).unwrap(); + assert_eq!( + hub.tip_hash().unwrap(), + later.block_hash(), + "precious equal-work held tip must win after invalidate" + ); } /// A lone side block is not a tip extend, a weaker branch is ignored, and an @@ -2687,6 +2746,85 @@ fn reorg_same_height_then_multi_block_branch() { } pin_precious_held_chaintips(&hub, ext, ext_h); pin_side_weaker_and_unknown_parent(&hub); + pin_fake_work_side_does_not_rewind(&hub); +} + +/// A held branch whose parent is an ancestor of the tip, with enormous claimed +/// work and no valid proof of work on a later header, must not disconnect. +/// A later held branch that really has more work still becomes the tip. +fn pin_fake_work_side_does_not_rewind(hub: &rbitcoin_net::ChainHub) { + use rbitcoin_net::AcceptOutcome; + + let tip_h = hub.tip_height().unwrap(); + let tip = hub.tip_hash().unwrap(); + let ancestor = hub + .query + .header_at_height(Height(tip_h - 3)) + .unwrap() + .unwrap() + .1; + let side = mine_regtest_block( + BlockHash::from_byte_array(ancestor.hash), + ancestor.timestamp.saturating_add(1_100), + tip_h - 2, + vec![], + ); + assert!( + matches!( + hub.accept_received_block(side.clone()).unwrap(), + AcceptOutcome::IgnoredWeaker + ), + "one honest side block is weaker than the tip" + ); + assert_eq!(hub.tip_hash().unwrap(), tip); + + let mut fake = mine_regtest_block( + side.block_hash(), + side.header.time.saturating_add(600), + tip_h - 1, + vec![], + ); + // Claimed work is ~2^256/target. This target is far below regtest, and + // the header was mined for the easy regtest target, so proof of work fails. + // The parent is not on the best chain, so the sibling header check does not run. + fake.header.bits = bitcoin::CompactTarget::from_consensus(0x1d00ffff); + let fake_hash = fake.block_hash(); + let mut seen = 0u64; + let _ = hub.query.take_disconnect(&mut seen); + let submitted = hub.accept_received_block(fake); + assert_eq!( + hub.tip_height(), + Some(tip_h), + "fake work must not move the tip: {submitted:?}" + ); + assert_eq!(hub.tip_hash().unwrap(), tip); + assert!( + hub.is_block_invalid(&fake_hash), + "fake work must be remembered invalid: {submitted:?}" + ); + assert!( + hub.query.take_disconnect(&mut seen).is_none(), + "header failure must not disconnect: {submitted:?}" + ); + + let mut p = side.block_hash(); + let mut t = side.header.time; + let mut heavier = Vec::new(); + for h in (tip_h - 1)..=(tip_h + 1) { + let b = mine_regtest_block(p, t.saturating_add(700 + h), h, vec![]); + p = b.block_hash(); + t = b.header.time; + heavier.push(b); + } + for b in &heavier { + hub.accept_received_block(b.clone()) + .unwrap_or_else(|e| panic!("heavier {}: {e}", b.block_hash())); + } + assert_eq!(hub.tip_height(), Some(tip_h + 1)); + assert_eq!( + hub.tip_hash().unwrap(), + heavier.last().unwrap().block_hash() + ); } /// After catch-up (`initialblockdownload` false; `-maxtipage` so the 2011 diff --git a/docs/operator/operations.md b/docs/operator/operations.md index 2e500088c..fe92d767d 100644 --- a/docs/operator/operations.md +++ b/docs/operator/operations.md @@ -91,7 +91,7 @@ Clean smoke: | `--asmap PATH` | `asmap=` | unset — try `{datadir}/ip_asn.dat` if present; else prefix groups | | `--no-seeds` | `no_seeds=` | seeds on | | `--sh-index` | `sh_index=` | **off** — Class B scripthash (address/history; Electrum/Esplora start without it) | -| `--block-filter-index` | `block_filter_index=` | **off** — BIP158 basic filters. Independent of `--sh-index`. IBD seals them when the flag is on from the start. A later enable still materializes after catch-up: the `rbtc-idx-wb` builder fills the gap from Class A in the background (`index: build from=… to=…`, progress every 10 s with `read=` `build=` `commit=` ms for that interval, `index: build done`). A caught-up chain seals new blocks on the confirm write thread; follow, relay, and Electrum do not wait for it. With `--sp-tweaks` the same pass builds both indexes. Works with `--prune-seqsigwit`. `tip: accept` shows `bf=` and `bf_lag=`. `NODE_COMPACT_FILTERS` is advertised once filters first reach the tip (`blockfilter: caught up …`); peers that connected earlier do not learn the bit. RPC `getblockfilter` serves a stored block even when that height is not sealed (rebuilt from the body, including a stale hash). `/rest/blockfilter/` and P2P serve heights the watermark covers; a stop past the watermark is silence | +| `--block-filter-index` | `block_filter_index=` | **off** — BIP158 basic filters. Independent of `--sh-index`. IBD seals them when the flag is on from the start. A later enable still materializes after catch-up: the `rbtc-idx-wb` builder fills the gap from Class A in the background (`index: build from=… to=…`, progress every 10 s with `read=` `build=` `commit=` ms for that interval, `index: build done`). A caught-up chain seals new blocks on the confirm write thread; follow, relay, and Electrum do not wait for it. With `--sp-tweaks` the same pass builds both indexes. Works with `--prune-seqsigwit`. `tip: accept` shows `bf=` and `bf_lag=`. `NODE_COMPACT_FILTERS` is advertised once filters first reach the tip (`blockfilter: caught up …`); peers that connected earlier do not learn the bit. RPC `getblockfilter` serves a sealed height, one block whose parent filter is already sealed, and a stale branch back to that sealed fork point. An unsealed best-chain gap is refused. `/rest/blockfilter/` and P2P serve heights the watermark covers; a stop past the watermark is silence | | `--prune-seqsigwit` | `prune_seqsigwit=` | **off** — unpruned reads `seqsigwit.body`. On: refuse wire reconstruct below tip−288 **heights**, advertise `NETWORK_LIMITED`, and keep those heights as `store/seqsigwit.window/{height}.bin` plus a RAM cache. Refused with `--sp-tweaks`, and once pruned a datadir serves no tweaks | | `--prune-seqsigwit-ram-threshold-bytes N` | `prune_seqsigwit_ram_threshold_bytes=` | `268435456` (256 MiB). `0` keeps nothing in RAM: every height, including tiny IBD blocks, is read from its file | | `--electrum-max-subs N` | `electrum_max_subs=` | **10000** — `blockchain.scripthash.subscribe` per Electrum connection (>= 1). Wallets subscribe every receive/change address up to their gap limit. | diff --git a/docs/rpc.md b/docs/rpc.md index 9d2dffd4a..339f9e4e5 100644 --- a/docs/rpc.md +++ b/docs/rpc.md @@ -56,7 +56,7 @@ later Core RPC is present, including descriptor-wallet calls from 0.21 | `--rpc-token-file PATH` | `{datadir}/rpc.token` | CSPRNG hex token; TCP `Authorization: Bearer` | | `--rpc-cookie-file PATH` / conf `rpc_cookie_file=` | **off** | Existing Core-format `username:password` file (no trailing newline) accepted as TCP HTTP Basic in addition to Bearer; it is never created or logged | | `--sh-index` | **off** | Class B scripthash (Electrum/Esplora only; RPC by height/hash/txid does not need it) | -| `--block-filter-index` | **off** | BIP158 basic. IBD seals them when the flag is on from the start. A later enable still materializes after catch-up. `NODE_COMPACT_FILTERS` is advertised once filters first reach the tip (`getnetworkinfo` lists `COMPACT_FILTERS`), then for the life of the process. `getblockfilter` serves a stored block even when that height is not sealed yet (rebuilt from the body, including a stale hash). `/rest/blockfilter/` and P2P still serve only heights the watermark covers. Independent of `--sh-index` | +| `--block-filter-index` | **off** | BIP158 basic. IBD seals them when the flag is on from the start. A later enable still materializes after catch-up. `NODE_COMPACT_FILTERS` is advertised once filters first reach the tip (`getnetworkinfo` lists `COMPACT_FILTERS`), then for the life of the process. `getblockfilter` serves a sealed height, one block whose parent filter is already sealed, and a stale branch back to that sealed fork point. An unsealed best-chain gap is refused. `/rest/blockfilter/` and P2P still serve only heights the watermark covers. Independent of `--sh-index` | | `--rpc-work-queue N` | **16** | In-flight HTTP RPC (Core `-rpcworkqueue`). One POST is one slot (a JSON-RPC array is still one slot). Full permit is HTTP **503** `Work queue depth exceeded`. **0** is the default queue of 16. | | `--rest` / conf `rest=` | **off** | Unauthenticated `/rest/` on the RPC listener. Own queue, same depth as `--rpc-work-queue`'s default. The body is read before that permit is taken. Without the flag those paths are 404. | @@ -158,7 +158,7 @@ still wait for durable SH when shindex is on. | `gettxspendingprevout` | All networks. Live mempool spender of each `{txid,vout}`. | | `submitblock` | All networks. Same `ChainHub::accept_received_block` as a P2P `block` message: tip-extend, or hold by hash + most-work `accept_branch`. A consensus reject is a BIP22 reason string. A local store fault, including one reading the parent or tip header or a spent output, is RPC error `-25` (Core `state.IsError()`). A cancelled connect is `inconclusive`. Neither caches the block as invalid. | | `scantxoutset` | All networks. Requires `--sh-index`. Expands output descriptors (`range` default 1000, Core's range errors) and looks each script up on the scripthash index. Refuses more than 10000 derived scripts. Does not store the descriptor. Never returns the genesis coinbase output (Core never adds it to the coins view). `txouts` is always `-1` (no coins DB; the count is not computed). | -| `getblockfilter` | All networks. Requires `--block-filter-index`. `filtertype` `basic` only (anything else is `-5`). Returns `filter` and `header` hex. A sealed best-chain height uses the index. A known block that is not sealed yet, or not on the best chain, is built from the stored body. Its header is the BIP157 chain from the last sealed header, or from the zero genesis prev-header — not a zero stand-in for an unsealed parent. Unknown hash is `-5` “Block not found”. Flag off is “Index is not enabled”. REST `/rest/blockfilter/` stays watermark-only (“still in the process of being indexed” past the watermark). | +| `getblockfilter` | All networks. Requires `--block-filter-index`. `filtertype` `basic` only (anything else is `-5`). Returns `filter` and `header` hex. A sealed best-chain height uses the index. One block whose parent filter header is already sealed (genesis uses the zero prev-header) is built from the stored body. A stale branch is rebuilt back to that sealed fork point. An unsealed best-chain gap is “Filter not found. Block filters are still in the process of being indexed.” Unknown hash is `-5` “Block not found”. Flag off is “Index is not enabled”. REST `/rest/blockfilter/` stays watermark-only (“still in the process of being indexed” past the watermark). | | `gettxout` | All networks. Connected Class A + mempool. Default `include_mempool=true` returns `null` for a confirmed out spent by a live mempool tx. `include_mempool=false` still returns the confirmed coin. A leftover still live in the hub (IBD / `-blocksonly`) uses the connected path, not `confirmations: 0`. A disconnected archive row is `null` (not tip+1 confirmations). A vout at or past the output count is `null`. The genesis coinbase output is `null` (Core never adds it to the coins view); REST `getutxos` follows. | | `getindexinfo` | All networks. Reports `txindex` synced at tip — we reconstruct by txid from Class A (no separate index flag). | | `getchaintips` | All networks. Active + archive `valid-fork` + held `valid-headers` + header-only (`submitheader` / P2P headers). Invalid body after a known header marks that branch `invalid`. | diff --git a/docs/wallets.md b/docs/wallets.md index 737dbb2a9..00982aadb 100644 --- a/docs/wallets.md +++ b/docs/wallets.md @@ -145,8 +145,9 @@ If the endpoint or the credential is wrong, Wasabi keeps loading filters from its own backend and shows the RPC status as disconnected. That fallback is Wasabi's. The node is in use when the status bar says the RPC is connected. Filters are absent until `--block-filter-index` has -sealed the heights Wasabi asks for. `getblockfilter` on a known block -that is not sealed yet is rebuilt from the stored body. REST +sealed the heights Wasabi asks for. `getblockfilter` refuses a +best-chain block whose parent filter is not sealed yet. A stale branch +is rebuilt only back to a sealed fork point. REST `/rest/blockfilter/` and P2P stay watermark-only. ## Core-wallet applications