Skip to content

mempool: decode a confirmed parent once per coin lookup - #969

Merged
reardencode merged 3 commits into
reardencode:masterfrom
average-gary:mempool/prevout-decode-once
Oct 9, 2026
Merged

reardencode merged 3 commits into
reardencode:masterfrom
average-gary:mempool/prevout-decode-once

Conversation

@average-gary

@average-gary average-gary commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #968 (net/proposal-parent-once-followup); only the last commit is new here. Draft until that PR merges.

Why this matters

check_block_proposal is about to become a hot path. The mempool's get_coin runs the parent-output read this commit reshapes for every incoming transaction, and the Stratum V2 job-validation work (sv2-spec discussion #239; rbitcoin branch sv2/job-validation) has a Job Declarator Server ask this node "is this miner's template valid?" for every declared job, dozens of times a minute per pool, through the same read. Both need the read to be cheap (one decode per parent, not one per input), bounded in memory (proportional to the block, not to the parents' total outputs), and right about spent coins. The spentness fix in this stack is also a correctness improvement for every surface that reports whether a coin is spent (gettxout, mempool accept, Electrum, Esplora): after a reorg the spent marker and the spentness probe could consult different copies of the same transaction.

chain_prevout read the parent three times per get_coin: two packed outs decodes (get_tx_by_txid for the meta, tx_output_at_fk for the one output) and a get_tx_full read of input 0 through tx_input_at_fk to tell a coinbase apart (skipped on a seqsigwit cache hit), with a block_tx_fks fallback when the input record was pruned, and then re-read the fence for the create height the resolve had just consulted. The proposal check had grown the same resolve-then-decode pair and the same per-input fence re-read.

  • Query::connected_tx_outputs owns the algorithm for both: a tip-only fk resolve (Store::get_fk_by_txid_tip_height: verifies each head candidate's body txid, one batched fence read, no decode) that returns the create's fence height from the same read, then one packed outs decode. It returns (Fk, height, Vec<OutputRecord>), not a TxRecord no caller read.
  • Query::is_coinbase_create answers the coinbase question from the Class C coinbase table (the rule coinbase_spend_is_immature already used), so the input read, its pruned fallback, and the block_tx_fks meter go away. A height with no Class C coinbase entry answers "not a coinbase": the lenient posture coinbase_spend_is_immature already had, unchanged here.
  • The proposal's parent memo carries the height, so coinbase_spend_is_immature no longer re-reads the fence per input and its no-height arm is gone (a parent is in the memo only if it resolved on the connected chain).
  • Store::resolve_txid loses its last production caller and is deleted; its pins move to get_fk_by_txid / get_fk_by_txid_tip.

Per get_coin on a confirmed parent: master did two packed outs decodes plus a get_tx_full read (skipped on a seqsigwit cache hit) for the parent, one more decode in the spentness probe (removed by #968), and 2 fence reads; now 1 decode and 1 fence read. docs/concurrency.md MempoolHub::inner row updated.

Adversarial review

  • get_coin vout selection unpinned (mutation survivor): the shape test's parent now has two outputs with distinct values (5 000 sat / rest) and the child spends vout 1; the fee is asserted. Against .next() the test fails with non-coinbase chain spend: Policy("negative fee").
  • is_coinbase_create forced false survived (mutation 9c): the shape test now spends the tip coinbase and asserts AcceptError::ImmatureCoinbase. With the mutant applied the spend is accepted (unwrap_err on Ok(AcceptResult { fee_sat: 1000, .. })); restored, it rejects.
  • connected_tx_outputs returns an unread TxRecord; callers re-read the fence: fixed as above.
  • get_fk_by_txid_tip_height rustdoc said "one txid.body verify": it verifies each head candidate's body txid and does one batched fence read; the rustdoc says that now.

Tests

  • get_coin_decodes_once_and_skips_mtp_without_time_lock: one body decode per get_coin, fee from vout 1, tip-coinbase spend is ImmatureCoinbase.
  • cargo test -p rbitcoin-store --lib 26 s, -p rbitcoin-query --lib 84 s, -p rbitcoin-net --lib 264 s (known Mac failure peer::tests::snapshot_omits_peer_after_tcp_fin; ibd_mutated_body_is_refetched_not_blacklisted failed once under the full parallel suite and passes in isolation), -p rbitcoin-rpc --lib 67 s (known unix-socket failures only), clippy -D warnings, fmt, ast-grep, deny green.

🤖 Generated with Claude Code

@average-gary
average-gary force-pushed the mempool/prevout-decode-once branch from c71229c to 6782785 Compare October 9, 2026 13:21
@reardencode
reardencode marked this pull request as ready for review October 9, 2026 13:43
rearden-grok[bot]
rearden-grok Bot previously requested changes Oct 9, 2026

@rearden-grok rearden-grok Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shared lookup is the right shape: one tip resolve that returns the fence height, one packed outs decode, coinbase-ness from the Class C table, and the shape test pins vout 1 plus ImmatureCoinbase.

Requesting a change on the proposal path. Caching that height and dropping the per-input fence read lets a disconnect between the parent pre-pass and the input loop price a no-longer-connected output as valid: spentness is false when get_fk_by_txid_tip misses, and the memo still holds the output. The mempool coinbase read also fail-opens on a table error (unwrap_or(false)), which admits an immature coinbase; the proposal helper already returns that error.

Ok(None) => return Err("bad-txns-inputs-missingorspent".into()),
Err(e) => return Err(e.to_string()),
};
if spend_height >= created_h.saturating_add(maturity) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coinbase_spend_is_immature used to call tx_height_get(create_fk) and return bad-txns-inputs-missingorspent when the fence had no height. created_h is now the height captured in confirmed_parent_outputs, and this early return trusts it with no further fence read.

chain_txout then treats the memo entry as spendable whenever is_outpoint_spent is false. After a disconnect, has_confirmed_strong_spender_at resolves with get_fk_by_txid_tip and returns false when that create is no longer on the best chain, so the stale memo entry is priced as a valid input. This applies to every confirmed parent, including ones already past maturity, because this branch returns before is_coinbase_create.

getblocktemplate proposal mode calls this with no UTXO-view stamp check. Connect and disconnect hold utxo_view while they publish (query/connect.rs, confirm_run/phases.rs); a proposal that overlaps that publish can accept a spend of a coin that has left the best chain.

Restore a fence check on create_fk before trusting the cached height (a missing height stays bad-txns-inputs-missingorspent), or fail the proposal when the view stamp moves between the parent pre-pass and the end of proposal_connect.

Comment thread crates/rbitcoin-net/src/tx_relay.rs Outdated
let is_coinbase = self
.query
.is_coinbase_create(fk, create_height)
.unwrap_or(false);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is_coinbase_create returns Err on a Class C read failure and Ok(false) when the height has no coinbase row. unwrap_or(false) collapses both into "not a coinbase".

check_mempool_structural skips the maturity reject when is_coinbase is false, so a failed coinbase-table read admits a spend of an immature coinbase. The same helper in coinbase_spend_is_immature surfaces that error and the proposal rejects. Spentness just above already fails closed (KnownUnavailable on Err). A coinbase-table error should fail closed the same way. Ok(false) for a height with no Class C row can stay the lenient answer.

@reardencode

Copy link
Copy Markdown
Owner

Grok and I are gonna make those changes and rebase this - quicker connect is on my mind.

average-gary and others added 3 commits October 9, 2026 06:58
`chain_prevout` read the parent three times per `get_coin`: two packed
outs decodes (`get_tx_by_txid` for the meta, `tx_output_at_fk` for the
one output) and a `get_tx_full` read of input 0 through `tx_input_at_fk`
to tell a coinbase apart (skipped on a seqsigwit cache hit), with a
`block_tx_fks` fallback when the input record was pruned, and then read
the fence a second time for the create height the resolve had just
consulted. The block proposal check had just grown the same
resolve-then-decode pair and the same per-input fence re-read.

`Query::connected_tx_outputs` now owns that algorithm for both: a
tip-only fk resolve that verifies `txid.body` only and returns the
create's fence height from the same read
(`Store::get_fk_by_txid_tip_height`), then one packed outs decode.
`Query::is_coinbase_create` answers the coinbase question from the
Class C coinbase table, the rule `coinbase_spend_is_immature` already
used, so the input read and its pruned fallback go away along with the
`block_tx_fks` meter that only counted that fallback. A height with no
Class C coinbase entry answers "not a coinbase", the lenient posture
`coinbase_spend_is_immature` already had; it is unchanged here. The
proposal's parent memo carries the height, so `coinbase_spend_is_immature`
no longer re-reads the fence per input and its no-height arm is gone: a
parent is in the memo only if it resolved on the connected chain.
`Store::resolve_txid` loses its last production caller and is deleted;
its pins move to `get_fk_by_txid` and `get_fk_by_txid_tip`.

The get_coin shape test pins one body decode per lookup on a private
pad, spends vout 1 of a two-output parent so the fee pins the vout
index, and spends the tip coinbase for `ImmatureCoinbase` so the Class C
coinbase answer is pinned.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The parent memo caches the create height. A disconnect before the input
loop leaves spentness false while the memo still holds the output, so a
cached height already past maturity priced a coin that is no longer on
the best chain. Re-read the fence and treat a missing or moved height as
bad-txns-inputs-missingorspent.

Co-authored-by: Cursor <cursoragent@cursor.com>
unwrap_or(false) treated a Class C failure as "not a coinbase", so an
immature coinbase spend could be admitted. Ok(false) stays the lenient
answer for a height with no coinbase row; Err leaves the coin unavailable,
the same way a spentness read error already does.

Co-authored-by: Cursor <cursoragent@cursor.com>
@rearden-grok
rearden-grok Bot force-pushed the mempool/prevout-decode-once branch from 6782785 to 6d164cb Compare October 9, 2026 14:07
@reardencode
reardencode merged commit 548c867 into reardencode:master Oct 9, 2026
18 checks passed
average-gary added a commit to average-gary/rbitcoin that referenced this pull request Oct 9, 2026
Plan D no longer stacks on sv2/plan-c-fee-push (merged as reardencode#949 and reardencode#951)
and its D1 step is upstream: reardencode#959 moved the proposal check onto
ChainHub, reardencode#963, reardencode#967, reardencode#968, and reardencode#969 made it complete, one decode per
parent, bounded, and right about spent coins after a reorg. Name those
as the prerequisites and why: the check is the job-validation hot path.

Drop the two risks they closed (no script execution, reardencode#967; the
per-input parent decode, reardencode#968), name both ends of the tip-change
straddle the D11 journey now accepts, fix the "no scripts" CPU-trade
lines, and remove the getblocktemplate bad-cb-amount changelog entry
that reardencode#959 already carries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
average-gary added a commit to average-gary/rbitcoin that referenced this pull request Oct 9, 2026
Plan D no longer stacks on sv2/plan-c-fee-push (merged as reardencode#949 and reardencode#951)
and its D1 step is upstream: reardencode#959 moved the proposal check onto
ChainHub, reardencode#963, reardencode#967, reardencode#968, and reardencode#969 made it complete, one decode per
parent, bounded, and right about spent coins after a reorg. Name those
as the prerequisites and why: the check is the job-validation hot path.

Drop the two risks they closed (no script execution, reardencode#967; the
per-input parent decode, reardencode#968), name both ends of the tip-change
straddle the D11 journey now accepts, fix the "no scripts" CPU-trade
lines, and remove the getblocktemplate bad-cb-amount changelog entry
that reardencode#959 already carries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants