Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion COMPAT.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Full `/tx/:txid` JSON still has `vin[]`. Electrum has no outspend-vin surface.
| Scripthash index | Optional (`--shindex`, default **off**); bulk at tip when on | External ElectrumX / Fulcrum; Core `-txindex` is different (txid→block) |
| JSON-RPC | Documented **subset** ([`docs/rpc.md`](./docs/rpc.md)); Bearer token or opt-in Core cookie (`--rpc-cookie-file`), no `--rpcuser` / `--rpcpassword`; `rbitcoin-cli`. `--rpc-work-queue` defaults to **16** (HTTP occupancy, 503 when full; **0** is that default); a JSON-RPC array is one POST | Full Core RPC; `-rpcworkqueue` is in-flight HTTP jobs (503) |
| GetData serve | Every hash is served in order. While **16** (`MAX_SERVE_BLOCKS`) getdata bodies or 4 MiB sit in the peer's queue (RAM cap), the rest waits on the session and no new message is read; none is dropped. Pings and the ping timeout still run | Core `ProcessGetData` pauses on `fPauseSend` and serves the rest before the next message |
| Inbound eviction victim | After Core-shaped protect (netgroup / recent block / recent tx / min-ping), disconnect the **longest-connected** remaining inbound | Core `SelectNodeToEvict` youngest in the oldest netgroup |
| Inbound eviction victim | After Core-shaped protect (netgroup / recent block / recent tx / min-ping / 8 longest-connected), disconnect the **newest** inbound in the largest remaining netgroup. A fully protected set rejects the new inbound | Core `SelectNodeToEvict` youngest in the oldest netgroup |
| `--sptweaks-dust` | Serve-time floor default **1000** sat (omit P2TR outs `value <=` floor). **546** matches Cake electrs. Not Cake/Electrum protocol | n/a (Electrum tweaks are not Core) |
| IBD empty-headers EOF | Empty `headers` to **our locator** + idle path latches `headers_done` even if a peer advertises a taller less-work height / junk `version.start_height`. A proven header walk at the tip that every taller peer failed to extend also ends IBD | Core header sync follows most-work; advertised `start_height` is not a remaining header count |

Expand Down
8 changes: 8 additions & 0 deletions changelog.d/docs-match-schema-26.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Fixed

- **Operator schema upgrade matches schema 26.** Opening a schema 24 or 25
datadir shrinks `header.body` from 96 B to 88 B and rewrites `meta` to 26.
The operator table no longer describes a rewrite that stops at 25 or grows
headers from 88 B to 96 B.
- **Finding 085 is closed on IBD.** A coinbase-less 64-byte body is a bad
copy: IBD drops it and asks again, and does not cache the hash as invalid.
5 changes: 3 additions & 2 deletions crates/rbitcoin-primitives/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,9 @@ pub const SCHEMA_VERSION: u16 = 26;
///
/// Schema **26** is current. Meta **22** through **26** opens. Occupied **22**
/// rewrites `create.loc.ovf` 12 B rows to 16 B. Occupied **24/25** strips
/// trailing size/weight from 96 B `header.body` rows. Occupied **25** rewrites
/// `meta` and zero-extends `txstat.body`. Meta older than **22** is not
/// trailing size/weight from 96 B `header.body` rows. Any meta below **26**
/// is rewritten to 26; a short `txstat.body` is aligned to the loc count.
/// Meta older than **22** is not
/// openable: those Class A bodies (`tx.body`, `txout.body`, `inwit.body` /
/// `seqsigwit.body`) were wiped, not rewritten.
#[inline]
Expand Down
4 changes: 2 additions & 2 deletions docs/experimental-mainnet.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,15 +131,15 @@ payloads and oversized pending-cmpct pressure; disconnects the peer.

**BIP324 v2 only** — discovery is v2-filtered (`x809` DNS + `P2P_V2` gossip);
see [`OPERATOR.md`](../OPERATOR.md) § P2P transport. Expect fewer usable
peers than a dual-stack Core node (experimental user-agent still limits inbound).
peers than a dual-stack Core node. The user-agent is `/rbitcoin:VERSION/`, which some peers still refuse.

## Ops risks

| Risk | Notes |
|------|--------|
| Disk / RAM | Multi‑100 GiB Class A; segmented 25-bit `tx.head.*` + mapped `.fuse8` (~1.5 GiB `RssFile`, heap `fuse8=0`); SH BDZ3 occ mapped (~150 MiB `file=` at ~1 B keys); sealed BDZ `g` FdOnly (not anon heap) |
| `tx.head` seal | Segment roll builds fuse8 on seal (~27 M keys); watch seal begin/done logs — not a mono-head shadow fill |
| Peer scarcity | [`OPERATOR.md`](../OPERATOR.md) § P2P transport (`x809` seeds + `P2P_V2` gossip). Experimental user-agent still limits inbound |
| Peer scarcity | [`OPERATOR.md`](../OPERATOR.md) § P2P transport (`x809` seeds + `P2P_V2` gossip). `/rbitcoin:VERSION/` is not a Core user-agent, so some peers still refuse inbound |
| Mempool | Libre policy (0.1 sat/vB, full RBF + pure RBFR 1.25×, no dust ban, Libre annex); cluster **64 / 101 kvB**; **scripts verified on accept** |
| Confirm lookup/load | **Load** recvs load-sized batches (soft **8000** inputs / hard **144** blocks) from `loadq=14`. Dense mainnet is typically **a few blocks per batch**. IBD **lookup** TipOnly-resolves at most **64000** inputs or **1080** BQ-ready heights per wave, in order from `path_lo`. Real queues loadq=14 · scriptq=4 · writeq=14 |
| Not Core/Fulcrum | No production SLA; 0.7.99 is high-scrutiny 0.x; schema unstable until 1.0; occupied 0.6.x stores refuse (wipe + IBD) |
Expand Down
19 changes: 8 additions & 11 deletions docs/external_findings/085-64-byte-tx-mutation.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
# 085 — A coinbase-less 64-byte body is mutated

**Severity:** high
**Status:** partial — tip, P2P `block`, compact block, and `submitblock`
paths fixed; the IBD confirm path needs the companion change on branch
`consensus/ibd-mutated-body-reject`
**Status:** fixed
**Found by:** consensus audit, 2026-10-02

A merkle root also matches a body made of its inner nodes read as
Expand All @@ -26,16 +24,15 @@ tip hub returns `NetError::Mutated` for any other caller.
accept, without a cache. It did cache other mutated bodies (padded
witness bytes); it now skips the cache for every `NetError::is_mutated`.

**Scope:** this change covers the tip hub, the P2P `block` handler,
compact reconstruct, and `submitblock`. The IBD body path
(`PeerEvent::BlockFramed` → confirm → `reject_bad_block_tx_layout` →
`ConfirmRejectClass::from_consensus` → `apply_consensus_invalid_reject`)
still marks the hash invalid for this body. The companion change on
branch `consensus/ibd-mutated-body-reject` fixes that path; until it
lands, this finding is not fully fixed.
Block checks compare the merkle root before the other body rules. A first
transaction that is not a coinbase, plus any 64-byte transaction, is
`merkle mutated by a 64-byte tx`. IBD classifies that as a bad copy
(`SoftWire`): the body is dropped, the hash is not cached invalid, and
the block is requested again.

**Regression:** `rbitcoin-net`
`chain::tests::hostile_peer_session`,
`peer::tests::peer_header_dos_and_self_announce`,
`compact::tests::prefilled_64_byte_body_without_coinbase_is_not_a_block`;
`compact::tests::prefilled_64_byte_body_without_coinbase_is_not_a_block`,
`ibd::events::confirm_reject_tests::ibd_mutated_body_is_refetched_not_blacklisted`;
`rbitcoin-rpc` `methods::tests::rpc_regtest_from_genesis`.
2 changes: 1 addition & 1 deletion docs/external_findings/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ rbitcoin reference, or redteam static analysis). Numbered reports live beside th
| [082](./082-invalid-hash-cap.md) | low | Invalid-hash set stops at 4096 | fixed | `invalid_hash_set_stops_at_the_cap` |
| [083](./083-mempool-expiry-cursor.md) | low | Mempool expiry runs without a new admission | fixed | `expire_stale_drops_old_tx_without_a_new_accept` |
| [084](./084-findanddelete-empty-sig.md) | critical | Empty sig in legacy CHECKMULTISIG did not delete OP_0 from scriptCode | fixed | `legacy_multisig_empty_sig_deletes_op_0_from_script_code` |
| [085](./085-64-byte-tx-mutation.md) | high | Coinbase-less 64-byte body cached as an invalid block hash | partial (IBD confirm path: `consensus/ibd-mutated-body-reject`) | `hostile_peer_session`, `peer_header_dos_and_self_announce`, `prefilled_64_byte_body_without_coinbase_is_not_a_block`, `rpc_regtest_from_genesis` |
| [085](./085-64-byte-tx-mutation.md) | high | Coinbase-less 64-byte body cached as an invalid block hash | fixed | `hostile_peer_session`, `peer_header_dos_and_self_announce`, `prefilled_64_byte_body_without_coinbase_is_not_a_block`, `ibd_mutated_body_is_refetched_not_blacklisted`, `rpc_regtest_from_genesis` |
| [086](./086-p2sh-spk-stack-size.md) | high | P2SH spend skipped the scriptPubKey `MAX_STACK_SIZE` check | fixed | `p2sh_script_pubkey_push_counts_against_max_stack_size` |
| [087](./087-forward-spend-in-batch.md) | critical | Spend of a later block in one confirm batch accepted | fixed | `same_batch_spend_of_a_later_block_is_missing`, `batched_spend_of_a_later_block_is_rejected_alone`, `spender_below_its_create_height_is_corrupt` |
| [088](./088-tip-chain-selection.md) | high | Tip-follow chain selection: total-work ranking, failed-branch prefix, sibling header check, reorg depth | partial (F3 reorg depth open) | `held_branch_with_more_total_work_beats_an_earlier_local_tie`, `failed_branch_tip_keeps_the_heavier_valid_prefix`, `sibling_claiming_more_work_with_wrong_bits_keeps_the_tip` |
Expand Down
15 changes: 9 additions & 6 deletions docs/operator/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,14 +82,17 @@ names the dirs. Corrupt files are **not** repaired in-process.

| Incoming `meta` | What this binary does |
|-----------------|------------------------|
| **25** | Open. Leftover `inwit.*` is renamed to `seqsigwit.*`. Missing `input.loc` with a matching `seqsigwit` count backfills parent edges from those prevouts. Leftover `inputs.*` is renamed to `input.*`. |
| **24** | Rewrite `meta` to 25, then create/extend zeroed `txstat.body` to `create.loc` count (no `txout.body` rewrite). Same `inwit` rename and `input` backfill as 25. Unlink leftover `txfixed.body`. |
| **23** | Rewrite `meta` to 25 first, then rewrite `header.body` 88 B rows to 96 B (size/weight 0) on open. Class A tx stems kept. A torn `header.body` rewrite is retried. Zero-extend `txstat.body`. |
| **22**, occupied Class A | Rewrite `meta` to 25 first, then `create.loc.ovf` 12 B→16 B on `TxTable::open`, then `header.body` 88→96. Crash window is 25 `meta` + old ovf/header; this binary retries those file rewrites. Zero-extend `txstat.body`. |
| **22**, empty Class A | Rewrite `meta` to 25, then open. |
| **26** | Open. |
| **25** | If `header.body` is still 96 B, rewrite each row to 88 B (`header.body.grow`, then rename), dropping trailing size and weight. An 88 B body is left alone. Then rewrite `meta` to 26. |
| **24** | Same header shrink as 25. Zero-extend `txstat.body` to the `create.loc` count (no `txout.body` rewrite). Unlink leftover `txfixed.body`. Rewrite `meta` to 26. |
| **23** | Headers stay 88 B. Rewrite `meta` to 26. `txstat.body` is created and aligned to the loc count when it is missing or short. |
| **22**, occupied Class A | Rewrite `create.loc.ovf` 12 B→16 B. Headers stay 88 B. Then the same `meta` rewrite and `txstat` align as 23. |
| **22**, empty Class A | Rewrite `meta` to 26, then open. |
| **Below 22**, empty or occupied | **Refuse.** One line: `schema before 22 refuses this datadir; wipe datadir and redo IBD`. Wipe the datadir and redo IBD. No per-version rewrite. |

A `txstat.body` cell written as four ULEBs starting with `n_in` (an unreleased 25 experiment) is not detected and is not rewritten. Resync that datadir. A **24 binary** refuses 25 `meta` (do not downgrade in place). A **23 binary** refuses 24+ `meta`. A **22 binary** refuses 23+ `meta`. A **21 binary** refuses 22+ `meta`. A **19 binary** refuses 20+ `meta`.
On every open of meta 22–25 the node also renames leftover `inwit.*` to `seqsigwit.*` and `inputs.*` to `input.*`. A missing `input.loc` with seqsigwit prevouts backfills parent edges. A crash during the 96 B→88 B rewrite leaves the old `meta`; the next open retries the shrink. Size and weight after that come from `txstat`.

A `txstat.body` cell written as four ULEBs starting with `n_in` (an unreleased 25 experiment) is not detected and is not rewritten. Resync that datadir. A **25 binary** refuses 26 `meta`. A **24 binary** refuses 25 `meta` (do not downgrade in place). A **23 binary** refuses 24+ `meta`. A **22 binary** refuses 23+ `meta`. A **21 binary** refuses 22+ `meta`. A **19 binary** refuses 20+ `meta`.

When meta is below 22, the log line is:

Expand Down
8 changes: 4 additions & 4 deletions docs/quality.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ in [`CHANGELOG.md`](../CHANGELOG.md). 1.0 product gates:
[`road-to-1.0.md`](./road-to-1.0.md). Peer-node notes:
[`peer-clients.md`](./peer-clients.md) (do not copy here).

**Last reaudit:** 2026-09-17. Live schema: [`SCHEMA.md`](../SCHEMA.md).
**Last reaudit:** 2026-10-08. Live schema: [`SCHEMA.md`](../SCHEMA.md)
(`SCHEMA_VERSION = 26`). **Q-64**, **Q-71**, and **Q-72** are off this list:
SV2 plans A–C are in tree, and the Talip and Livera indexes are fully
dispositioned.
Core functional inventory: [`core-functional.md`](./core-functional.md).
Findings **001–023** fixed. Nightly fuzz **21** jobs. Nightly mutants,
8 hour budget split between new code and the backlog, cursor on the
Expand Down Expand Up @@ -36,9 +39,6 @@ Counts and procedures stay in the owner doc. A row is the rank and the outcome.
| 5 | **Q-67** | `asked_blocks` clone on hold | `hold_body` clones `asked_blocks` before `held_bodies` insert so the read lock does not overlap the write (`HeldBodies::insert` already takes `&HashSet`). Bound is `MAX_SERVE_BLOCKS` × peers. Follow-up: pass the read guard with a documented lock order, or keep the clone as a named trade. Owner: `crates/rbitcoin-net/src/chain.rs`. |
| 6 | **Q-69** | CLN / LDK chain backend | Operator can point CLN `bcli` and ldk-node Esplora/Electrum at this node. Owner: [`lightning.md`](./lightning.md). |
| 7 | **Q-70** | Batch lookup-path hit counters if a profile names them | `head_resolve_stats::add_hit_rank` does two relaxed `fetch_add`s per resolved txid (`head_resolve_denserels`, `tx_table`). SH extract collect flushes output and hit counts once per fk batch, not per output. Still once per event: `accepted_wb` (`SeqCst`, per confirmed block), `serve_perf::note_serve` (per historical getdata), `page_ios` (per SH page read). Done: a profile shows `add_hit_rank` on confirm lookup and those counters flush per batch, or the profile shows the line is noise and this row moves to Won't-fix. `add_hit_ages` is already a batched flush; `add_hit_age` remains the one-bucket path. |
| 8 | **Q-71** | Talip review remediations | Every row in [`external_findings/024-talip-review-index.md`](./external_findings/024-talip-review-index.md) is fixed, rejected, or won't-fix, and each fixed row names a regression. |
| 9 | **Q-64** | SV2 template provider (TDP server) | Node serves Noise-encrypted Template Distribution Protocol in-process: `CoinbaseOutputConstraints` → pushed `NewTemplate` / `SetNewPrevHash` on tip change and fee delta, `RequestTransactionData`, `SubmitSolution` → `accept_block`. Roadmap (plans A → B → C, one PR each): [`sv2-template-provider.md`](./sv2-template-provider.md). |
| 10 | **Q-72** | Livera review remediations | Every row in [`external_findings/052-livera-review-index.md`](./external_findings/052-livera-review-index.md) is fixed, rejected, or won't-fix, and each fixed row names a regression. |

R-ids were the 2026-08-12 slice. Canonical id is **bold**. Do not start
**R-11+**. Next unused Q-id is **Q-73**.
Expand Down
12 changes: 6 additions & 6 deletions docs/road-to-1.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ taking on for 1.0. `--prune-seqsigwit` is in (seqsigwit-window prune,
- Wallet keys, GUI, Core block-file prune (`-prune` of `blk` files), ZMQ, IPC, plaintext v1 P2P
- Explorer-search APIs (address-prefix), Liquid, in-binary `/api/v1/`
catalogue. **0.8** is electrs HTTP drop-in (except prefix):
[`COMPAT.md`](../COMPAT.md) (**Q-68**)
[`COMPAT.md`](../COMPAT.md)
- Every Bitcoin Core functional test (no wallet / prune / v1 scripts)
- Matching Core `estimatesmartfee` numbers
- Apple notarization
Expand Down Expand Up @@ -196,9 +196,9 @@ target (cold / after a fee spike) on a real mempool — not a soak badge
When the rest is true, tag 1.0 so **every 1.x.x opens a 1.0.0 store**.
Older-than-1.0 or corrupt files can still refuse with a one-line message.

Do not freeze while Class C / sidecar / fuse8 can lose a `set` or index
OOB (**Q-57**). Mempool persist writes body before LIVE slots (**Q-58**).
Schema 21 is the current bytes; 0.x may still bump.
The durable path is fail-closed (**Q-57** / **Q-58**). Live bytes are
schema **26** ([`SCHEMA.md`](../SCHEMA.md)). 0.x may still bump; 1.0 is
the freeze.

| Done | Step |
|:----:|------|
Expand All @@ -215,11 +215,11 @@ mempool.space Node+MariaDB+frontend can retire electrs. Core JSON-RPC for
that stack is TCP plus an opt-in Core cookie (`--rpc-cookie-file`), so stock
mempool `CORE_RPC` works unpatched; a unix-socket patch stays optional. Not
their `/api/v1/` process. Not address-prefix.
**Q-68**. Surface: [`COMPAT.md`](../COMPAT.md).
Surface: [`COMPAT.md`](../COMPAT.md). (**Q-68** is the create.loc SIMD
extract in [`TESTING.md`](../TESTING.md), not this drop-in.)

| Done | Step |
|:----:|------|
| [x] | Q-68 parked (this row) |
| [x] | Esplora unix + `/internal/*` + mempool tx-JSON snapshot |
| [x] | Product docs claim the drop-in |

Expand Down
11 changes: 6 additions & 5 deletions docs/sv2-template-provider.md
Original file line number Diff line number Diff line change
Expand Up @@ -591,10 +591,11 @@ No live pool/JDC, no mainnet datadir, no plaintext mode.
`bad-diffbits` unless the sent bits were already the limit. A re-push cannot fix it: a same-hash `SetNewPrevHash`
is outside §7.4, and a `NewTemplate` has no field for the bits. Not
planned; mainnet and signet bits depend only on the prev hash.
- Pre-existing, outside this plan: consensus does not enforce the BIP94
timewarp floor (testnet4). The first block of a retarget period may carry
a timestamp more than 600 s before its parent. Core rejects it
(`time-timewarp-attack`); when consensus gains the rule, the template
timestamp floor must include it.
- Pre-existing, outside this plan: this binary has no testnet4 network.
If one is added, consensus does not yet enforce the BIP94 timewarp floor.
The first block of a retarget period may carry a timestamp more than 600 s
before its parent. Core rejects that (`time-timewarp-attack`); the template
timestamp floor must include the rule when consensus gains it. Mainnet,
testnet3, signet, and regtest do not use that rule.
- After ship: JD-server mode, weak blocks, extension negotiation, per-IP
connection limits → quality.md rows, not this roadmap.
Loading