From 66794f595a786f155b54fe346f8807a464b263a2 Mon Sep 17 00:00:00 2001 From: "rearden-grok[bot]" <317016512+rearden-grok[bot]@users.noreply.github.com> Date: Fri, 9 Oct 2026 07:32:50 -0700 Subject: [PATCH] store: keep block filters and sp tweaks on the cold volume --datadir-cold already holds seqsigwit, txstat, and input. Optional blockfilter and sp_tweaks indexes are created there too. A split open refuses those directories when they are still on the hot store. --prune-seqsigwit still only drops historical seqsigwit. --- README.md | 2 +- SCHEMA.md | 18 +++-- SCHEMA_HISTORY.md | 4 +- TESTING.md | 2 +- changelog.d/cold-index-files.md | 8 ++ crates/rbitcoin-node/src/cli.rs | 6 +- crates/rbitcoin-query/src/block_filter.rs | 2 +- crates/rbitcoin-query/src/lib.rs | 5 +- crates/rbitcoin-query/src/sp_tweaks.rs | 27 ++++++- crates/rbitcoin-rpc/src/methods/mod.rs | 2 +- crates/rbitcoin-store/src/store.rs | 94 +++++++++++++++++++++-- crates/rbitcoin-test/tests/scenarios.rs | 4 + docs/ibd-memory.md | 8 +- docs/operator/operations.md | 44 ++++++++--- docs/operator/setup.md | 2 +- docs/rpc.md | 2 +- nix/modules/rbitcoin.nix | 2 +- 17 files changed, 191 insertions(+), 41 deletions(-) create mode 100644 changelog.d/cold-index-files.md diff --git a/README.md b/README.md index cb1fc39eb..0f21d426a 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,7 @@ Most full nodes center a **UTXO set + block files** (Bitcoin Core). Most Electru backends are **external indexers** of another node. rbitcoin does neither: **no UTXO set** (relational archive), **Electrum + txindex in-process**. -- **~200 GiB** hot pin/annotate set (schema 17); **~700 GiB** with cold `seqsigwit` — +- **~200 GiB** hot pin/annotate set (schema 17); **~730 GiB** with the cold append-only files (`seqsigwit` ~486, `txstat` ~11, `input` ~29) — census in [`SCHEMA.md`](./SCHEMA.md), `--sh-index` costs in [`OPERATOR.md`](./OPERATOR.md) - **Under ~30 h** IBD on a laptop-class host with **`--milestone 0`** - **Modest RAM** during sync — no multi‑GiB `dbcache` pause diff --git a/SCHEMA.md b/SCHEMA.md index 2d778ef11..c3c792763 100644 --- a/SCHEMA.md +++ b/SCHEMA.md @@ -225,11 +225,12 @@ itself changed. seqsigwit.body / seqsigwit.loc / seqsigwit.off / seqsigwit.loc.ovf # Class A inputs+witness (cold loc) seqsigwit.prune # optional: u32 LE pruneheight sidecar (`--prune-seqsigwit`; missing = off) seqsigwit.window/ # optional prune window: one {height}.bin per kept height - seqsigwit.reloc # optional: seqsigwit lives under --datadir-cold/store + seqsigwit.reloc # optional: append-only files live under --datadir-cold/store spent.body # sole-spender 8 B × n_out; leftover spent.off unlinked tx.body / tx.idx.* # schema ≤14 packed (refused if non-empty) txid.body # dense create_fk-ordered txids (schema 13+) txstat.body / txstat.ovf / txstat.blk # 8 B/create ULEB econ + per-header tails (schema 25) + input.loc / input.off / input.body # spender → parent edges (schema 22–23) tx.head/ # meta + open OA NNNNNN; sealed NNNNNN.mphf|.fuse8 spent.ovf # multi-spender overflow (was spenders.body) confirmed.body # Class C: height → header_fk @@ -257,12 +258,19 @@ itself changed. / # only when --datadir-cold is set store/ seqsigwit.body / seqsigwit.loc / seqsigwit.off / seqsigwit.loc.ovf + txstat.body / txstat.ovf / txstat.blk + input.loc / input.off / input.body + blockfilter.idx/ blockfilter.body/ # when --block-filter-index has created them + sp_tweaks.idx/ sp_tweaks.body/ # when --sp-tweaks has created them ``` -`--datadir` holds both stems by default. `--datadir-cold PATH` places only -`seqsigwit.body` + `seqsigwit.loc` (and `seqsigwit.off` / `seqsigwit.loc.ovf`) under `PATH/store/` -(and writes `seqsigwit.reloc` in the hot store). Pin / SH / spend-annotate stay on -the hot volume. +`--datadir` holds every file by default. `--datadir-cold PATH` places the +files the IBD pin path does not read under `PATH/store/` and writes +`seqsigwit.reloc` in the hot store: `seqsigwit.*`, `txstat.*`, `input.*`, +and, once those indexes exist, `blockfilter.*` and `sp_tweaks.*`. Pin / SH / +spend-annotate stay on the hot volume. `--prune-seqsigwit` is separate: it +drops historical `seqsigwit` below tip−288 and keeps that window on the hot +store. `txstat`, `input`, block filters, and tweaks are not pruned. **Height → txs:** `confirmed[h]` → `header_fk` → contiguous Class A range `[header_txs_first[h−1], header_txs_first[h−1] + header_txs_count[h−1])`. diff --git a/SCHEMA_HISTORY.md b/SCHEMA_HISTORY.md index d6fa32e66..aae9e54ae 100644 --- a/SCHEMA_HISTORY.md +++ b/SCHEMA_HISTORY.md @@ -50,7 +50,9 @@ unstamped). `input.body` is 8 B per input: parent `create_fk` u40 LE per-header blob (`txstat.ovf` + `txstat.blk`). `seqsigwit.*` is the renamed `inwit` stem (sequence, scriptSig, witness); open renames the files in place. `txstat.*` and `input.*` sit next to `seqsigwit` (cold when `--datadir-cold` -is set). Open with no `input.loc` and a matching `seqsigwit` count backfills +is set). `blockfilter.*` and `sp_tweaks.*` use that same directory once +enabled. That placement is not a schema bump, and neither index is pruned. +Open with no `input.loc` and a matching `seqsigwit` count backfills parent edges from `seqsigwit` prevouts. A four-ULEB cell that started with `n_in` is not detected; resync that experimental datadir. Occupied 24 rewrites `meta` and extends zeros to `create.loc` count. `txout.body` is not diff --git a/TESTING.md b/TESTING.md index b7f133f8b..e634a1843 100644 --- a/TESTING.md +++ b/TESTING.md @@ -336,7 +336,7 @@ A cell that names a crate-local test describes the witness that exists today. It | ID | Layer | Description | |----|-------|-------------| | `overlay_config` | Node CLI + net | In-process onion / I2P / cjdns matrix: `onlynet`, proxy, SAM, reachable, accept-incoming, one parse or learn check per network, `--only-net` dial candidates, no cjdns dial until reachable, and one `peers` file round trip for all three. Live Tor, i2pd, and cjdns stay in overlay-functional. | -| `node_cli_and_surface_smoke` | Lifecycle/CLI | Networks, `run_node`, config errors, CLI flags (incl. `--conf`, `--peer-timeout=0` refuse / `=1` smoke, unknown conf key ignored, `min_relay_tx_fee=-1` and `network=nope` conf fail), dropped Core `--rpcuser`/`--rpcpassword`/`--rpcport`/`--rpcconnect`/`-rpcport` refuse, help/version (`rbitcoin-cli` help and version do not dial). Usage errors exit 2 before a datadir is touched: unknown flags, missing or bad values, and every concatenated, one-dash, or Core spelling of a native kebab flag. A missing conf, a bad line, a bad `log_level`, conf `rpcuser`/`rpcpassword`, and `max_outbound=0` exit 2 and open no store. Non-hex `--min-chain-work`, a signet challenge or block time off a custom signet, and `--sp-tweaks` with `--prune-seqsigwit` exit 1. One smoke takes the native flag set together; bare network conf lines parse and CLI `--datadir` wins over conf `datadir`; `--datadir-cold` puts seqsigwit on the cold store; a custom signet smokes. Signet: genesis header plus height-1 BIP325 connect. The assembled `NodeConfig` values and help text no surface reports are the node-crate `operator_conf_and_argv` (`--bytes-per-sigop`, `--block-reserved-sigops`, conf `sp_tweaks` together with `prune_seqsigwit`, conf `sp_tweaks=1` at the default dust and `sp_tweaks_dust` of 546, 0, or a non-numeric value, and Electrum or Esplora with and without `--sh-index` plus `--sh-index` alone) | +| `node_cli_and_surface_smoke` | Lifecycle/CLI | Networks, `run_node`, config errors, CLI flags (incl. `--conf`, `--peer-timeout=0` refuse / `=1` smoke, unknown conf key ignored, `min_relay_tx_fee=-1` and `network=nope` conf fail), dropped Core `--rpcuser`/`--rpcpassword`/`--rpcport`/`--rpcconnect`/`-rpcport` refuse, help/version (`rbitcoin-cli` help and version do not dial). Usage errors exit 2 before a datadir is touched: unknown flags, missing or bad values, and every concatenated, one-dash, or Core spelling of a native kebab flag. A missing conf, a bad line, a bad `log_level`, conf `rpcuser`/`rpcpassword`, and `max_outbound=0` exit 2 and open no store. Non-hex `--min-chain-work`, a signet challenge or block time off a custom signet, and `--sp-tweaks` with `--prune-seqsigwit` exit 1. One smoke takes the native flag set together; bare network conf lines parse and CLI `--datadir` wins over conf `datadir`; `--datadir-cold` puts seqsigwit, txstat, and input on the cold store; a custom signet smokes. Signet: genesis header plus height-1 BIP325 connect. The assembled `NodeConfig` values and help text no surface reports are the node-crate `operator_conf_and_argv` (`--bytes-per-sigop`, `--block-reserved-sigops`, conf `sp_tweaks` together with `prune_seqsigwit`, conf `sp_tweaks=1` at the default dust and `sp_tweaks_dust` of 546, 0, or a non-numeric value, and Electrum or Esplora with and without `--sh-index` plus `--sh-index` alone) | | `three_stage_confirm_and_parent_pin_surface` | Consensus+query | Split load→scripts→write of pad+spend from genesis (header-plan BIP68 MTP); parent pin; load ready timeout/cancel; instance-owned `last_write` / `last_pin` / `take_window` meters (a second engine's window stays empty); txstat fee / size / weight from the load assemble, restamp and its refuses; same-run create then spend; 546-shaped 2-vout merge + same-block chain + cross-batch head resolve; an already-at-height retry finishes a spend annotate | | `mempool_under_pressure` | Mempool + RPC (crate) | One entry in `orphanage`, `accept`, `tx_relay`, and `methods_tests`: orphan reserve and expiry, sigops before script, rolling fee floor, cluster cap, parked min-relay orphan, and the package RPC rejects (unsorted, missing inputs, conflict, min-relay parent with maxfeerate child). | | `sigop_adjusted_size_budget_and_reopen` | Mempool (crate) | One empty `ActiveMempool`: sigop-adjusted vsize (boundary, min relay, full-pool floor, RBF, package and 1p1c), the raw-weight cluster limit, the shared block sigop budget, and sigop cost plus bytes-per-sigop and reserve overlays across reopen and compact. Each beat restores an empty pool. | diff --git a/changelog.d/cold-index-files.md b/changelog.d/cold-index-files.md new file mode 100644 index 000000000..4f41cf013 --- /dev/null +++ b/changelog.d/cold-index-files.md @@ -0,0 +1,8 @@ +Changed + +- `--datadir-cold` is the append-only volume: `seqsigwit`, `txstat`, + `input`, and, once enabled, `blockfilter` and `sp_tweaks`. + `--prune-seqsigwit` still only drops historical seqsigwit and keeps + its 288-height window on the hot store. A split datadir that still + has `blockfilter.*` or `sp_tweaks.*` on the hot store refuses to open + until those directories are moved next to seqsigwit. diff --git a/crates/rbitcoin-node/src/cli.rs b/crates/rbitcoin-node/src/cli.rs index 4586246f1..b1b31fe78 100644 --- a/crates/rbitcoin-node/src/cli.rs +++ b/crates/rbitcoin-node/src/cli.rs @@ -375,8 +375,10 @@ SV2: --sv2-tp-listen ADDR serves the Stratum v2 Template Distribution Protocol ( --sv2-tp-fee-delta SATS pushes a same-tip template once its fees gain SATS over the last one sent (default 1000).\n\ --sv2-tp-template-interval SECS checks for that gain every SECS, 1 to 86400 (default 5).\n\ RPC: --rpc unix socket {{datadir}}/rpc.sock; --rpc-listen [ADDR] adds TCP (default 127.0.0.1 and Core-matching port). Token {{datadir}}/rpc.token (Bearer); --rpc-cookie-file opts TCP into Core cookie HTTP Basic. No --rpcuser. --rest turns on unauthenticated /rest/ on those listeners (off unless set).\n\ -Cold files: --datadir-cold PATH puts Class A seqsigwit.body/idx under PATH/store (HDD).\n\ - Default (flag omitted): hot and cold files both live under --datadir.\n\ +Cold files: --datadir-cold PATH puts append-only IBD files under PATH/store\n\ + (seqsigwit, txstat, input, and blockfilter / sp_tweaks once enabled).\n\ + Default (flag omitted): those files live under --datadir. --prune-seqsigwit\n\ + is separate: it drops old seqsigwit and keeps a 288-height window on the hot store.\n\ Conf: --conf FILE (snake_case key=value; CLI kebab overrides conf). See OPERATOR.md and docs/rpc.md.\n\ Advanced debug/IO knobs remain RBITCOIN_* env (not required for normal sync; preserved if CLI omits).\n\ IBD densify: up to 1024 concurrent getdata, max 16 in transit per peer.\n\ diff --git a/crates/rbitcoin-query/src/block_filter.rs b/crates/rbitcoin-query/src/block_filter.rs index 7bcc49a5e..460508387 100644 --- a/crates/rbitcoin-query/src/block_filter.rs +++ b/crates/rbitcoin-query/src/block_filter.rs @@ -169,7 +169,7 @@ impl Query { /// disconnect and its filter truncate, or a torn slot). pub fn set_block_filter_index(&self, enabled: bool) -> Result<(), QueryError> { if enabled && self.block_filters.get().is_none() { - let table = BlockFilterTable::open_or_create(self.store.path())?; + let table = BlockFilterTable::open_or_create(self.store.cold_files_dir())?; self.trim_block_filters_to_best_chain(&table)?; let _ = self.block_filters.set(table); } diff --git a/crates/rbitcoin-query/src/lib.rs b/crates/rbitcoin-query/src/lib.rs index e096c1c0d..76ddf3777 100644 --- a/crates/rbitcoin-query/src/lib.rs +++ b/crates/rbitcoin-query/src/lib.rs @@ -440,8 +440,9 @@ impl Query { let _ = store.strong_tx.flush(); } let store_path = store.path().to_path_buf(); - let (sp_tweaks, sptweaks_origin) = if SpTweaksTable::files_present(&store_path) { - match SpTweaksTable::open(&store_path) { + let index_dir = store.cold_files_dir().to_path_buf(); + let (sp_tweaks, sptweaks_origin) = if SpTweaksTable::files_present(&index_dir) { + match SpTweaksTable::open(&index_dir) { Ok(t) => { let origin = t.origin_height().0; (Some(t), origin) diff --git a/crates/rbitcoin-query/src/sp_tweaks.rs b/crates/rbitcoin-query/src/sp_tweaks.rs index c50a2a5dc..4fc9aba3a 100644 --- a/crates/rbitcoin-query/src/sp_tweaks.rs +++ b/crates/rbitcoin-query/src/sp_tweaks.rs @@ -184,7 +184,7 @@ impl Query { fn ensure_sp_tweaks(&self, origin: Height) -> Result<(), QueryError> { let mut g = self.sp_tweaks.lock().unwrap_or_else(|e| e.into_inner()); if g.is_none() { - let t = SpTweaksTable::open_or_create(self.store.path(), origin)?; + let t = SpTweaksTable::open_or_create(self.store.cold_files_dir(), origin)?; self.repair_sp_tweaks(&t)?; *g = Some(t); self.sptweaks_origin @@ -539,6 +539,31 @@ mod tests { crate::testutil::tiny_query_labeled("sptweaks") } + #[test] + fn split_store_puts_blockfilter_and_sp_tweaks_on_cold() { + let root = crate::testutil::TempDir::labeled("cold-indexes").unwrap(); + let hot = root.path().join("hot"); + let cold = root.path().join("cold"); + let layout = rbitcoin_store::StoreLayout::tiny(&hot).with_cold_dir(&cold); + let q = Query::open_or_create_layout(layout.clone()).unwrap(); + q.set_block_filter_index(true).unwrap(); + q.set_sptweaks_enabled(true, Height(0)).unwrap(); + assert!(cold.join("blockfilter.idx").is_dir()); + assert!(cold.join("blockfilter.body").is_dir()); + assert!(cold.join("sp_tweaks.idx").is_dir()); + assert!(cold.join("sp_tweaks.body").is_dir()); + assert!(!hot.join("blockfilter.idx").exists()); + assert!(!hot.join("blockfilter.body").exists()); + assert!(!hot.join("sp_tweaks.idx").exists()); + assert!(!hot.join("sp_tweaks.body").exists()); + drop(q); + let q = Query::open_or_create_layout(layout).unwrap(); + assert_eq!(q.sptweaks_next_height(), Some(Height(0))); + q.set_block_filter_index(true).unwrap(); + assert!(!hot.join("blockfilter.idx").exists()); + assert!(cold.join("blockfilter.idx").join("meta").is_file()); + } + #[test] fn put_and_load_noop_when_disabled() { let (dir, q) = tmp_q(); diff --git a/crates/rbitcoin-rpc/src/methods/mod.rs b/crates/rbitcoin-rpc/src/methods/mod.rs index d4a4be1f7..63a09faa7 100644 --- a/crates/rbitcoin-rpc/src/methods/mod.rs +++ b/crates/rbitcoin-rpc/src/methods/mod.rs @@ -695,7 +695,7 @@ const NAMED_HELP: &[(&str, &str)] = &[ initialblockdownload is relay-inhibited after densify (min-chain-work +\n\ max-tip-age), not still catching up. chainwork is summed header work\n\ (regtest 2/block). size_on_disk is a walk of store file lengths (plus\n\ - cold seqsigwit when split). verificationprogress is blocks/headers (1.0 when\n\ + the cold store when split). verificationprogress is blocks/headers (1.0 when\n\ headers is 0).", ), ( diff --git a/crates/rbitcoin-store/src/store.rs b/crates/rbitcoin-store/src/store.rs index e825e8a9d..5320810d0 100644 --- a/crates/rbitcoin-store/src/store.rs +++ b/crates/rbitcoin-store/src/store.rs @@ -12,14 +12,23 @@ use std::fs::OpenOptions; use std::io::Write; use std::path::{Path, PathBuf}; -/// Sidecar in the hot `{datadir}/store`: `seqsigwit.body` / `seqsigwit.loc` live under -/// `{datadir-cold}/store`. Presence-only (path always comes from the operator). +/// Sidecar in the hot `{datadir}/store`. Present when the append-only files live +/// under `{datadir-cold}/store`. Presence-only (the path comes from the operator). pub const SEQSIGWIT_RELOC_NAME: &str = "seqsigwit.reloc"; +/// Index directories that live next to seqsigwit when the store is split. +const COLD_INDEX_DIRS: &[&str] = &[ + "blockfilter.idx", + "blockfilter.body", + "sp_tweaks.idx", + "sp_tweaks.body", +]; + /// Where a store’s files live, plus open-time head geometry. /// -/// `dir` is `{datadir}/store`. When `cold_dir` is set and distinct, Class A -/// `seqsigwit.body` + `seqsigwit.loc` live there (bulk / HDD). Everything else stays +/// `dir` is `{datadir}/store`. When `cold_dir` is set and distinct, the +/// append-only IBD files live there: `seqsigwit.*`, `txstat.*`, `input.*`, +/// and (once enabled) `blockfilter.*` and `sp_tweaks.*`. The pin set stays /// in `dir`. /// /// [`Self::single`] / [`Self::with_cold`] are **Mainnet** scale (production). @@ -179,6 +188,29 @@ pub(crate) fn rename_legacy_inwit_files(dir: &Path) -> Result<(), StoreError> { Ok(()) } +fn refuse_cold_index_dirs(hot: &Path, cold: &Path) -> Result<(), StoreError> { + for name in COLD_INDEX_DIRS { + let on_hot = hot.join(name); + let on_cold = cold.join(name); + if !on_hot.is_dir() { + continue; + } + if on_cold.exists() { + return Err(StoreError::Layout(format!( + "{name} exists in both {} and {}; keep it only under the cold store", + hot.display(), + cold.display() + ))); + } + return Err(StoreError::Layout(format!( + "{name} is still in {}; move it next to seqsigwit under {}", + hot.display(), + cold.display() + ))); + } + Ok(()) +} + fn seqsigwit_files_present(dir: &Path) -> bool { dir.join("seqsigwit.body").exists() || dir.join("seqsigwit.loc").exists() @@ -262,7 +294,7 @@ fn write_seqsigwit_reloc(hot: &Path) -> Result<(), StoreError> { /// Top-level store handle for a datadir `store/` directory. pub struct Store { path: PathBuf, - /// `{datadir-cold}/store` when seqsigwit is split; `None` = seqsigwit in [`Self::path`]. + /// `{datadir-cold}/store` when append-only files are split; `None` = they live in [`Self::path`]. cold_path: Option, head_scale: HeadScale, pub headers: HeaderTable, @@ -441,6 +473,9 @@ impl Store { crate::scripthash::unlink_scripthash_run_leftovers(&path)?; open_layout_rewrite_current(&path, meta_ver)?; let seqsigwit_dir = resolve_seqsigwit_dir(&layout)?; + if layout.is_split() { + refuse_cold_index_dirs(&path, &seqsigwit_dir)?; + } let txs = TxTable::open_seqsigwit(&path, &seqsigwit_dir, layout.open_opts())?; if layout.is_split() { write_seqsigwit_reloc(&path)?; @@ -499,13 +534,20 @@ impl Store { &self.path } - /// Cold store directory when seqsigwit is split (`{datadir-cold}/store`). + /// Cold store directory when the append-only files are split + /// (`{datadir-cold}/store`). pub fn cold_path(&self) -> Option<&Path> { self.cold_path.as_deref() } + /// Where IBD append-only files live. The cold store when split, otherwise + /// [`Self::path`]. + pub fn cold_files_dir(&self) -> &Path { + self.cold_path.as_deref().unwrap_or(&self.path) + } + /// Sum of regular file lengths under the hot store and, when split, the - /// cold seqsigwit directory. Used by `getblockchaininfo.size_on_disk`. + /// cold store. Used by `getblockchaininfo.size_on_disk`. pub fn datadir_bytes(&self) -> u64 { let mut n = dir_file_bytes(&self.path); if let Some(cold) = &self.cold_path { @@ -4221,6 +4263,44 @@ mod tests { let _ = std::fs::remove_dir_all(&root); } + #[test] + fn split_refuses_blockfilter_left_in_hot() { + let root = tmp(); + let hot = root.join("hot"); + let cold = root.join("cold"); + Store::create_layout(StoreLayout::tiny(&hot).with_cold_dir(&cold)).unwrap(); + std::fs::create_dir_all(hot.join("blockfilter.idx")).unwrap(); + match Store::open_layout(StoreLayout::tiny(&hot).with_cold_dir(&cold)) { + Ok(_) => panic!("must refuse blockfilter left on the hot store"), + Err(err) => { + let msg = err.to_string(); + assert!(msg.contains("blockfilter.idx"), "{msg}"); + assert!(msg.contains("move"), "{msg}"); + assert!(msg.contains(cold.to_str().unwrap()), "{msg}"); + } + } + let _ = std::fs::remove_dir_all(&root); + } + + #[test] + fn split_refuses_sp_tweaks_in_both_dirs() { + let root = tmp(); + let hot = root.join("hot"); + let cold = root.join("cold"); + Store::create_layout(StoreLayout::tiny(&hot).with_cold_dir(&cold)).unwrap(); + std::fs::create_dir_all(hot.join("sp_tweaks.body")).unwrap(); + std::fs::create_dir_all(cold.join("sp_tweaks.body")).unwrap(); + match Store::open_layout(StoreLayout::tiny(&hot).with_cold_dir(&cold)) { + Ok(_) => panic!("must refuse sp_tweaks in both stores"), + Err(err) => { + let msg = err.to_string(); + assert!(msg.contains("sp_tweaks.body"), "{msg}"); + assert!(msg.contains("both"), "{msg}"); + } + } + let _ = std::fs::remove_dir_all(&root); + } + #[test] fn split_roundtrip_create_fk_and_seqsigwit_range() { let root = tmp(); diff --git a/crates/rbitcoin-test/tests/scenarios.rs b/crates/rbitcoin-test/tests/scenarios.rs index 51ba3cfb7..18f3f4ea0 100644 --- a/crates/rbitcoin-test/tests/scenarios.rs +++ b/crates/rbitcoin-test/tests/scenarios.rs @@ -327,6 +327,10 @@ fn pin_operator_smokes(td: &TestDatadir) { assert!(!hot.join("store/seqsigwit.body").exists()); assert!(cold.join("store/seqsigwit.body").is_file()); assert!(cold.join("store/seqsigwit.loc").is_file()); + assert!(cold.join("store/txstat.body").is_file()); + assert!(cold.join("store/input.body").is_file()); + assert!(!hot.join("store/txstat.body").exists()); + assert!(!hot.join("store/input.body").exists()); let signet = td.path().join("custom-signet"); assert!(exit_success(node(&[ diff --git a/docs/ibd-memory.md b/docs/ibd-memory.md index 8ae732b58..835097fbd 100644 --- a/docs/ibd-memory.md +++ b/docs/ibd-memory.md @@ -187,10 +187,10 @@ touches. Census: [`SCHEMA.md`](../SCHEMA.md) (tip 962298, 1.42 B creates). | Mode | Must stay hot | Approx | Cold (fault OK) | |------|---------------|--------|-----------------| -| **Tip follow / Electrum serve** | Open `tx.head` + recent `txout`/`spent`/`txid` tails + SH main idx + mempool | **8–16 GiB** page cache + **~2–3 GiB** process | `seqsigwit` (except `getrawtransaction`), sealed `tx.head` older than fuse-skip, archive `txout` | -| **Comfortable serve** (busy wallets, Electrum tweaks, RPC reconstruct) | Above + more `txout` + SH body slabs + `txid.body` | **16–32 GiB** | `seqsigwit` except rawtx | -| **IBD pin+annotate (no thrash)** | **All** `txout` + **all** `spent` + `create.loc` (~3 GiB) + `txid.body` + `tx.head` | **~221 GiB** | **`seqsigwit` (~486 GiB)** — wire still holds witness | -| **IBD + reconstruct/getdata** | Previous + `seqsigwit` | **~710 GiB** (same order as old packed `tx.body`) | — | +| **Tip follow / Electrum serve** | Open `tx.head` + recent `txout`/`spent`/`txid` tails + SH main idx + mempool | **8–16 GiB** page cache + **~2–3 GiB** process | `seqsigwit` (except `getrawtransaction`), `txstat`, `input` (except a served block), sealed `tx.head` older than fuse-skip, archive `txout` | +| **Comfortable serve** (busy wallets, Electrum tweaks, RPC reconstruct) | Above + more `txout` + SH body slabs + `txid.body` | **16–32 GiB** | `seqsigwit` except rawtx; `txstat` except fee history / `getblockstats`; block filters and `sp_tweaks` except a serve of those indexes | +| **IBD pin+annotate (no thrash)** | **All** `txout` + **all** `spent` + `create.loc` (~3 GiB) + `txid.body` + `tx.head` | **~221 GiB** | **`seqsigwit` (~486 GiB)**, **`txstat` (~11 GiB)**, **`input` (~29 GiB)** — wire still holds witness. Optional `blockfilter` / `sp_tweaks` append here and are not read back | +| **IBD + reconstruct/getdata** | Previous + `seqsigwit` + `input` | **~740 GiB** | `txstat`, block filters, `sp_tweaks` | | **SH tip materialize** | Two `txout` scans (16 MiB libc `pread` spans + 1 MiB write buffers); extract workers min(CPUs, free RAM / 1.5 GiB); ingest OA **~768 MiB** (2²⁵×24 B) | **~1.5 GiB** heap per extract worker (BDZ `g` + fuse + maps); pack ~8 MiB packed `g` + 16 MiB `body_buf` | No catalog k-way pages; no 64 × `.val` random-read set on the scan | Packed schema 13/14 needed the whole **`tx.body` (~663 GiB)** hot for the same diff --git a/docs/operator/operations.md b/docs/operator/operations.md index fe92d767d..6dfff3429 100644 --- a/docs/operator/operations.md +++ b/docs/operator/operations.md @@ -62,7 +62,7 @@ Clean smoke: | Flag | Conf | Default | |------|------|---------| | `--datadir PATH` | `datadir=` | cwd `datadir` (`./datadir` Unix, `.\datadir` Windows) | -| `--datadir-cold PATH` | `datadir_cold=` | unset — Class A `seqsigwit.body` / `seqsigwit.loc` under `{PATH}/store`; everything else stays in `--datadir` | +| `--datadir-cold PATH` | `datadir_cold=` | unset — append-only IBD files under `{PATH}/store`: `seqsigwit.*`, `txstat.*`, `input.*`, and (once created) `blockfilter.*` and `sp_tweaks.*`. The pin set stays in `--datadir` | | `--network NET` | `network=` | `mainnet` | | `--signet-challenge HEX` | `signet_challenge=` | default global Signet challenge | | `--signet-block-time SECS` | `signet_block_time=` | 600; requires a custom challenge | @@ -216,31 +216,51 @@ cjdns daemon in-process and no TUN in CI. NixOS: `cjdns.reachable`; `After`/`Wants` `cjdns.service`. Do not start a cjdns router from this module. `--datadir` holds the node root (`store/`, `mempool/`, `peers`, `rpc.token`, `rpc.sock`). After a listener binds, `{datadir}/run/{health,electrum,esplora,rpc}.addr` is that socket (the kernel port when the configured port is 0). -Omit `--datadir-cold` and cold files live there too. Set it to put the large -rarely-read Class A **seqsigwit** stem (`seqsigwit.body` + `seqsigwit.loc`, ~486 GiB + loc -on mainnet) on another volume. Pin / spend-annotate / Electrum / tweaks do not -read seqsigwit; reconstruct / `getrawtransaction` / block serve do. +Omit `--datadir-cold` and the append-only files live there too. Set it to put +those files on another volume. The confirm pin path does not read them. At the +2026-08-13 census that is `seqsigwit.body` + `seqsigwit.loc` (~486 GiB + loc), +`txstat.*` (~11 GiB), and `input.*` (~29 GiB). Optional `blockfilter.*` and +`sp_tweaks.*` are created on that same volume when the indexes are enabled. +Reconstruct / `getrawtransaction` / block serve read `seqsigwit` and `input`. +Fee history and `getblockstats` read `txstat`. Filter and tweak serve read +their own indexes. ``` --datadir /mnt/nvme/rbtc --datadir-cold /mnt/hdd/rbtc-cold -# hot: /mnt/nvme/rbtc/store/txout.body (and the rest) +# hot: /mnt/nvme/rbtc/store/txout.body (pin set, heads, Class C, scripthash) # cold: /mnt/hdd/rbtc-cold/store/seqsigwit.body # /mnt/hdd/rbtc-cold/store/seqsigwit.loc +# /mnt/hdd/rbtc-cold/store/txstat.body +# /mnt/hdd/rbtc-cold/store/input.body +# /mnt/hdd/rbtc-cold/store/blockfilter.body # if the index exists +# /mnt/hdd/rbtc-cold/store/sp_tweaks.body # if the index exists ``` A hot-store sidecar `seqsigwit.reloc` records the split. Opening without -`--datadir-cold` then refuses. Do not leave `seqsigwit.*` in both places. -`--prune-seqsigwit` writes `{store}/seqsigwit.prune` (the pruneheight) and -keeps the 288-height window at `{store}/seqsigwit.window/{height}.bin` on the -hot store, including when `seqsigwit.body` is on `--datadir-cold`. A datadir -that already has that sidecar refuses to start unless the flag is set again +`--datadir-cold` then refuses. Do not leave a cold file in both places. +`--prune-seqsigwit` is not this split. It writes `{store}/seqsigwit.prune` +(the pruneheight) and keeps the 288-height window at +`{store}/seqsigwit.window/{height}.bin` on the **hot** store, including when +`seqsigwit.body` is on `--datadir-cold`. It does not delete `txstat`, `input`, +block filters, or tweaks. A datadir that already has the prune sidecar refuses +to start unless the flag is set again (`datadir is pruned-seqsigwit; restart with --prune-seqsigwit enabled`). Disconnect at or below that pruneheight fails closed. Moving an existing datadir is operator `mv` (or copy+remove cross-device): ``` mkdir -p /mnt/hdd/rbtc-cold/store -mv /mnt/nvme/rbtc/store/seqsigwit.body /mnt/nvme/rbtc/store/seqsigwit.loc /mnt/nvme/rbtc/store/seqsigwit.off /mnt/nvme/rbtc/store/seqsigwit.loc.ovf /mnt/hdd/rbtc-cold/store/ +mv /mnt/nvme/rbtc/store/seqsigwit.body /mnt/nvme/rbtc/store/seqsigwit.loc \ + /mnt/nvme/rbtc/store/seqsigwit.off /mnt/nvme/rbtc/store/seqsigwit.loc.ovf \ + /mnt/nvme/rbtc/store/txstat.body /mnt/nvme/rbtc/store/txstat.ovf \ + /mnt/nvme/rbtc/store/txstat.blk \ + /mnt/nvme/rbtc/store/input.loc /mnt/nvme/rbtc/store/input.off \ + /mnt/nvme/rbtc/store/input.body \ + /mnt/hdd/rbtc-cold/store/ +# only when those indexes already exist: +mv /mnt/nvme/rbtc/store/blockfilter.idx /mnt/nvme/rbtc/store/blockfilter.body \ + /mnt/nvme/rbtc/store/sp_tweaks.idx /mnt/nvme/rbtc/store/sp_tweaks.body \ + /mnt/hdd/rbtc-cold/store/ ``` **Advanced** IO/perf tunables may still use `RBITCOIN_*` (see below); they are diff --git a/docs/operator/setup.md b/docs/operator/setup.md index 74fe262bd..60db0e61c 100644 --- a/docs/operator/setup.md +++ b/docs/operator/setup.md @@ -141,7 +141,7 @@ Replace the hostname and email, then apply authentication and network policy to RPC for your deployment. nginx `virtualHosts` proxy HTTP; `streamConfig` proxies the Electrum TCP protocol. -Use `coldDataDir` to place the large `seqsigwit` store on another volume. The +Use `coldDataDir` to place the append-only store (`seqsigwit`, `txstat`, `input`, and the block-filter and tweak indexes) on another volume. The service creates the directory but does not mount or size the volume. Use `environment` for documented advanced `RBITCOIN_*` settings and `extraArgs` for daemon flags not represented by module options. diff --git a/docs/rpc.md b/docs/rpc.md index 339f9e4e5..cc577ce0d 100644 --- a/docs/rpc.md +++ b/docs/rpc.md @@ -128,7 +128,7 @@ still wait for durable SH when shindex is on. |--------|-------| | `help` / `getrpcinfo` / `uptime` / `stop` | Control | | `echo` | Testing RPC. Returns arguments as a positional array. AuthServiceProxy `{args: [...], argN: ...}` is peeled only here. Mixed `submitpackage`/`sendrawtransaction`/`testmempoolaccept` `{args, maxfeerate}` is expanded in the Core-functional proxy, not on the node. | -| `getblockchaininfo` / `getblockcount` / `getbestblockhash` / `getblockhash` | Chain tip. `getblockcount` / `getbestblockhash` wait for the in-flight tip-accept job (not the rest of a catch-up burst). `headers` is the best known header height (`submitheader` / P2P headers may lead `blocks`). `chainwork` is summed header work (regtest 2 per block). `bits` and `target` are the tip compact target. `difficulty` prints with 16 significant digits. `size_on_disk` is a walk of `{datadir}/store` file lengths (plus `--datadir-cold` seqsigwit when split). `verificationprogress` is `blocks / headers` clamped to `[0, 1]` (`1.0` when `headers` is 0). `initialblockdownload` is the Core RPC name for **relay-inhibited**: `--min-chain-work` and `--max-tip-age` after densify/`enter_tip_mode`, not “still catching up”. `--prune-seqsigwit` sets `pruned: true` and `pruneheight` (omitted when off). | +| `getblockchaininfo` / `getblockcount` / `getbestblockhash` / `getblockhash` | Chain tip. `getblockcount` / `getbestblockhash` wait for the in-flight tip-accept job (not the rest of a catch-up burst). `headers` is the best known header height (`submitheader` / P2P headers may lead `blocks`). `chainwork` is summed header work (regtest 2 per block). `bits` and `target` are the tip compact target. `difficulty` prints with 16 significant digits. `size_on_disk` is a walk of `{datadir}/store` file lengths (plus `--datadir-cold` when split). `verificationprogress` is `blocks / headers` clamped to `[0, 1]` (`1.0` when `headers` is 0). `initialblockdownload` is the Core RPC name for **relay-inhibited**: `--min-chain-work` and `--max-tip-age` after densify/`enter_tip_mode`, not “still catching up”. `--prune-seqsigwit` sets `pruned: true` and `pruneheight` (omitted when off). | | `getblockheader` / `getblock` (verbosity 0/1/2) | Archive reconstruct. `getblockheader` includes `chainwork`, `target`, and `nextblockhash` when the next active block exists. Genesis omits `previousblockhash`. A header that is stored but not on the active chain (reorg or `invalidateblock`) returns `confirmations: -1` and its own `previousblockhash`. `getblock` returns a Class A body for that hash the same way, or a held never-confirmed body. Verbosity 0/2 below `pruneheight` is `-8` `Block not available (pruned data)`; verbosity 1 (txids) stays. | | `getblockstats` | All networks. Stamped `txstat` fee/size/weight/`n_in`. Size and count fields match Core for the keys we return: coinbase is left out of `ins`, fees, `total_size`, `total_weight`, and the segwit totals. `utxo_increase` is every output minus those inputs. `utxo_increase_actual` drops outputs that do not enter the UTXO set (height 0, the two mainnet BIP30-repeat coinbases, unspendable scripts). Omits coins-DB `utxo_size_inc` / `utxo_size_inc_actual` (selecting those names is invalid statistic `-8`). Unstamped leftover reconstructs then lazy-stamps. Reconstruct miss is `block body not in store`. Dummy `blk00000.dat` is shim-only so `rpc_getblockstats.py`'s rename-file needle stays Core-phrased. Pruned unstamped height is `-8` Core pruned text. | | `getdifficulty` | From tip bits, 16 significant digits | diff --git a/nix/modules/rbitcoin.nix b/nix/modules/rbitcoin.nix index d6bed45d6..b17680199 100644 --- a/nix/modules/rbitcoin.nix +++ b/nix/modules/rbitcoin.nix @@ -165,7 +165,7 @@ in coldDataDir = mkOption { type = types.nullOr types.path; default = null; - description = "Optional directory for the large, rarely read Class A seqsigwit store."; + description = "Optional directory for append-only IBD files (seqsigwit, txstat, input, block filters, silent-payment tweaks)."; }; user = mkOption {