From 98d5bcbdfc6103a38babb628d9c9c5a5a841ac9a Mon Sep 17 00:00:00 2001 From: Stavros Vlachakis <89769224+svlachakis@users.noreply.github.com> Date: Tue, 18 Aug 2026 14:10:01 +0300 Subject: [PATCH 1/3] docs: document history pruning modes and the EraE format Rolling pruning and the EraE archive format were both missing from the history pruning page, and the two EraE admin methods were missing from the JSON-RPC reference because the doc generator does not scan Nethermind.EraE. --- docs/fundamentals/history-pruning.md | 34 +++++++++- docs/interacting/json-rpc-ns/admin.md | 92 +++++++++++++++++++++++++++ 2 files changed, 125 insertions(+), 1 deletion(-) diff --git a/docs/fundamentals/history-pruning.md b/docs/fundamentals/history-pruning.md index 68ea8977c3..0dd8b40015 100644 --- a/docs/fundamentals/history-pruning.md +++ b/docs/fundamentals/history-pruning.md @@ -6,9 +6,21 @@ sidebar_position: 7 History pruning is a feature set that aims to reduce storage space requirements for a node by removing old historical data. The goal is to remove the requirement from nodes to store all the historical data but ensure the old data is preserved and accessible for anyone who needs it. For details, see [EIP-4444][eip444]. :::info -History pruning is enabled by default for the networks supporting it. To disable, set [`Sync.AncientBodiesBarrier`](./configuration.md#sync-ancientbodiesbarrier) and [`Sync.AncientReceiptsBarrier`](./configuration.md#sync-ancientreceiptsbarrier) to `0`. +On networks that support it, a fresh sync stops downloading bodies and receipts below the ancient barriers, so the node holds history only from that point onwards. To download the entire history instead, set [`Sync.AncientBodiesBarrier`](./configuration.md#sync-ancientbodiesbarrier) and [`Sync.AncientReceiptsBarrier`](./configuration.md#sync-ancientreceiptsbarrier) to `0`. + +Removing history that is already stored is a separate, opt-in step controlled by [`History.Pruning`](./configuration.md#history-pruning). ::: +## Pruning modes + +[`History.Pruning`](./configuration.md#history-pruning) selects how stored historical blocks and receipts are removed as the node runs. + +- `Disabled` (default) — nothing already stored is removed. +- `UseAncientBarriers` — prunes everything below the ancient barriers cutoff, matching what a fresh sync would have downloaded. +- `Rolling` — keeps a moving window of the most recent [`History.RetentionEpochs`](./configuration.md#history-retentionepochs) epochs and prunes below it as the head advances. The configured window must be at least the `minHistoryRetentionEpochs` chainspec parameter of the network, and a node that has just synced only starts pruning once its stored history grows past the window. + +Pruning never removes the genesis block or anything at or above the sync pivot. Use [`History.PruningInterval`](./configuration.md#history-pruninginterval) and [`History.PruningTimeoutSeconds`](./configuration.md#history-pruningtimeoutseconds) to control how often it runs and how long a single pass may take. + ## Era1 format The pre-Merge historical data is serviced in [Era1](https://github.com/status-im/nimbus-eth2/blob/stable/docs/e2store.md#era-files) format, which is an archival format initially designed for the consensus layer by Nimbus. @@ -23,6 +35,26 @@ block-index := starting-number | index | index | index ... | count Block headers, bodies, and receipts are compressed using the [Snappy framing format](https://github.com/google/snappy/blob/main/framing_format.txt). Each file contains a block index for fast lookup and an [epoch accumulator](https://github.com/ethereum/portal-network-specs/blob/master/history/history-network.md#the-historical-hashes-accumulator) for verification. The epoch accumulator can verify the entire archive with accumulators from a trusted source. It also allows a node to download a block header with a Merkle proof, proving it belongs to a particular epoch. +## EraE format + +EraE is the post-Merge archival format, specified in [ere.md](https://github.com/eth-clients/e2store-format-specs/blob/main/formats/ere.md). An EraE file holds up to 8192 blocks and is laid out as follows: + +``` +Version | CompressedHeader* | CompressedBody* | CompressedSlimReceipts* | TotalDifficulty* | AccumulatorRoot? | ComponentIndex +``` + +Receipts are stored slim, as `rlp([txType, postStateOrStatus, cumulativeGas, logs])` with no bloom filter; readers recompute it from the logs. Exported files use the `.ere` extension and carry the `noproofs` profile postfix, since Nethermind does not write `Proof` entries. The older `.erae` extension is still accepted on import. + +The `Era.*` options and `admin_importHistory`/`admin_exportHistory` described below apply to Era1. EraE has its own equivalents: + +- Import: [`EraE.ImportDirectory`](./configuration.md#erae-importdirectory), or the [`admin_importEraHistory`](../interacting/json-rpc-ns/admin.md#admin_importerahistory) JSON-RPC method. +- Export: [`EraE.ExportDirectory`](./configuration.md#erae-exportdirectory), or the [`admin_exportEraHistory`](../interacting/json-rpc-ns/admin.md#admin_exporterahistory) JSON-RPC method. Exporting post-Merge blocks requires [`EraE.BeaconNodeUrl`](./configuration.md#erae-beaconnodeurl), which supplies the beacon block and state roots. +- Range and verification: [`EraE.From`](./configuration.md#erae-from), [`EraE.To`](./configuration.md#erae-to) (`0` means head), and [`EraE.TrustedAccumulatorFile`](./configuration.md#erae-trustedaccumulatorfile). + +### Remote archives + +Set [`EraE.RemoteBaseUrl`](./configuration.md#erae-remotebaseurl) to a remote EraE server to download missing epoch files on demand instead of providing them locally. Downloads are checksummed against the server's `checksums_sha256.txt` manifest and cached in [`EraE.RemoteDownloadDirectory`](./configuration.md#erae-remotedownloaddirectory), which defaults to the import directory. + ## Import Nethermind allows importing of a historical block range from a specified location to the database. During import, the block range before the head will be inserted in parallel like old bodies, and after the head will be "suggested" like forward sync. So, it will process a new imported block. diff --git a/docs/interacting/json-rpc-ns/admin.md b/docs/interacting/json-rpc-ns/admin.md index 5ae1e3c4a7..cf405971cf 100644 --- a/docs/interacting/json-rpc-ns/admin.md +++ b/docs/interacting/json-rpc-ns/admin.md @@ -134,6 +134,51 @@ The data directory path as a string. +### admin_exportEraHistory + +Exports a range of historic blocks in erae format. + + + + +1. `destinationPath`: _string_ + +2. `from`: _string_ (hex integer) + +3. `to`: _string_ (hex integer) + + + + + +```bash +curl localhost:8545 \ + -X POST \ + -H "Content-Type: application/json" \ + --data '{ + "jsonrpc": "2.0", + "id": 0, + "method": "admin_exportEraHistory", + "params": [destinationPath, from, to] + }' +``` + + + + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": result +} +``` + +`result`: _string_ + + + + ### admin_exportHistory Exports a range of historic block in era1 format. @@ -179,6 +224,53 @@ curl localhost:8545 \ +### admin_importEraHistory + +Imports a range of historic blocks from an erae directory. + + + + +1. `sourcePath`: _string_ + +2. `from`: _string_ (hex integer) + +3. `to`: _string_ (hex integer) + +4. `accumulatorFile`: _string_ + + + + + +```bash +curl localhost:8545 \ + -X POST \ + -H "Content-Type: application/json" \ + --data '{ + "jsonrpc": "2.0", + "id": 0, + "method": "admin_importEraHistory", + "params": [sourcePath, from, to, accumulatorFile] + }' +``` + + + + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": result +} +``` + +`result`: _string_ + + + + ### admin_importHistory Import a range of historic block from era1 directory. From 61112a7520f9a8cc16110d8a598b3fc55f8583af Mon Sep 17 00:00:00 2001 From: Stavros Vlachakis <89769224+svlachakis@users.noreply.github.com> Date: Tue, 18 Aug 2026 17:31:35 +0300 Subject: [PATCH 2/3] docs: drop the generated JSON-RPC edits admin.md is produced by DocGen, so hand-inserting the two EraE methods would be overwritten on the next regeneration. NethermindEth/nethermind#12873 makes the generator emit them; until that runs, the page names the methods without linking into the reference. --- docs/fundamentals/history-pruning.md | 4 +- docs/interacting/json-rpc-ns/admin.md | 92 --------------------------- 2 files changed, 2 insertions(+), 94 deletions(-) diff --git a/docs/fundamentals/history-pruning.md b/docs/fundamentals/history-pruning.md index 0dd8b40015..c369dc2941 100644 --- a/docs/fundamentals/history-pruning.md +++ b/docs/fundamentals/history-pruning.md @@ -47,8 +47,8 @@ Receipts are stored slim, as `rlp([txType, postStateOrStatus, cumulativeGas, log The `Era.*` options and `admin_importHistory`/`admin_exportHistory` described below apply to Era1. EraE has its own equivalents: -- Import: [`EraE.ImportDirectory`](./configuration.md#erae-importdirectory), or the [`admin_importEraHistory`](../interacting/json-rpc-ns/admin.md#admin_importerahistory) JSON-RPC method. -- Export: [`EraE.ExportDirectory`](./configuration.md#erae-exportdirectory), or the [`admin_exportEraHistory`](../interacting/json-rpc-ns/admin.md#admin_exporterahistory) JSON-RPC method. Exporting post-Merge blocks requires [`EraE.BeaconNodeUrl`](./configuration.md#erae-beaconnodeurl), which supplies the beacon block and state roots. +- Import: [`EraE.ImportDirectory`](./configuration.md#erae-importdirectory), or the `admin_importEraHistory` JSON-RPC method. +- Export: [`EraE.ExportDirectory`](./configuration.md#erae-exportdirectory), or the `admin_exportEraHistory` JSON-RPC method. Exporting post-Merge blocks requires [`EraE.BeaconNodeUrl`](./configuration.md#erae-beaconnodeurl), which supplies the beacon block and state roots. - Range and verification: [`EraE.From`](./configuration.md#erae-from), [`EraE.To`](./configuration.md#erae-to) (`0` means head), and [`EraE.TrustedAccumulatorFile`](./configuration.md#erae-trustedaccumulatorfile). ### Remote archives diff --git a/docs/interacting/json-rpc-ns/admin.md b/docs/interacting/json-rpc-ns/admin.md index cf405971cf..5ae1e3c4a7 100644 --- a/docs/interacting/json-rpc-ns/admin.md +++ b/docs/interacting/json-rpc-ns/admin.md @@ -134,51 +134,6 @@ The data directory path as a string. -### admin_exportEraHistory - -Exports a range of historic blocks in erae format. - - - - -1. `destinationPath`: _string_ - -2. `from`: _string_ (hex integer) - -3. `to`: _string_ (hex integer) - - - - - -```bash -curl localhost:8545 \ - -X POST \ - -H "Content-Type: application/json" \ - --data '{ - "jsonrpc": "2.0", - "id": 0, - "method": "admin_exportEraHistory", - "params": [destinationPath, from, to] - }' -``` - - - - -```json -{ - "jsonrpc": "2.0", - "id": 0, - "result": result -} -``` - -`result`: _string_ - - - - ### admin_exportHistory Exports a range of historic block in era1 format. @@ -224,53 +179,6 @@ curl localhost:8545 \ -### admin_importEraHistory - -Imports a range of historic blocks from an erae directory. - - - - -1. `sourcePath`: _string_ - -2. `from`: _string_ (hex integer) - -3. `to`: _string_ (hex integer) - -4. `accumulatorFile`: _string_ - - - - - -```bash -curl localhost:8545 \ - -X POST \ - -H "Content-Type: application/json" \ - --data '{ - "jsonrpc": "2.0", - "id": 0, - "method": "admin_importEraHistory", - "params": [sourcePath, from, to, accumulatorFile] - }' -``` - - - - -```json -{ - "jsonrpc": "2.0", - "id": 0, - "result": result -} -``` - -`result`: _string_ - - - - ### admin_importHistory Import a range of historic block from era1 directory. From 858a05875b2456546687b2a69d1a7a6fbd16d6c3 Mon Sep 17 00:00:00 2001 From: Ruben Buniatyan <337518+rubo@users.noreply.github.com> Date: Tue, 18 Aug 2026 21:34:06 +0200 Subject: [PATCH 3/3] chore: revise updates --- docs/fundamentals/history-pruning.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/fundamentals/history-pruning.md b/docs/fundamentals/history-pruning.md index c369dc2941..ff12799d4d 100644 --- a/docs/fundamentals/history-pruning.md +++ b/docs/fundamentals/history-pruning.md @@ -16,7 +16,7 @@ Removing history that is already stored is a separate, opt-in step controlled by [`History.Pruning`](./configuration.md#history-pruning) selects how stored historical blocks and receipts are removed as the node runs. - `Disabled` (default) — nothing already stored is removed. -- `UseAncientBarriers` — prunes everything below the ancient barriers cutoff, matching what a fresh sync would have downloaded. +- `UseAncientBarriers` — removes stored block bodies and receipts below the lower of the two ancient barriers. - `Rolling` — keeps a moving window of the most recent [`History.RetentionEpochs`](./configuration.md#history-retentionepochs) epochs and prunes below it as the head advances. The configured window must be at least the `minHistoryRetentionEpochs` chainspec parameter of the network, and a node that has just synced only starts pruning once its stored history grows past the window. Pruning never removes the genesis block or anything at or above the sync pivot. Use [`History.PruningInterval`](./configuration.md#history-pruninginterval) and [`History.PruningTimeoutSeconds`](./configuration.md#history-pruningtimeoutseconds) to control how often it runs and how long a single pass may take. @@ -37,23 +37,23 @@ Block headers, bodies, and receipts are compressed using the [Snappy framing fo ## EraE format -EraE is the post-Merge archival format, specified in [ere.md](https://github.com/eth-clients/e2store-format-specs/blob/main/formats/ere.md). An EraE file holds up to 8192 blocks and is laid out as follows: +EraE is the archival format used for post-Merge history, as specified in [ere.md](https://github.com/eth-clients/e2store-format-specs/blob/main/formats/ere.md). An EraE file holds up to 8192 blocks and is laid out as follows: ``` Version | CompressedHeader* | CompressedBody* | CompressedSlimReceipts* | TotalDifficulty* | AccumulatorRoot? | ComponentIndex ``` -Receipts are stored slim, as `rlp([txType, postStateOrStatus, cumulativeGas, logs])` with no bloom filter; readers recompute it from the logs. Exported files use the `.ere` extension and carry the `noproofs` profile postfix, since Nethermind does not write `Proof` entries. The older `.erae` extension is still accepted on import. +Receipts are stored slim, as `rlp([txType, postStateOrStatus, cumulativeGas, logs])` with no bloom filter; readers recompute it from the logs. Exported files use the `.ere` extension and carry the `noproofs` profile suffix, since Nethermind does not write `Proof` entries. The older `.erae` extension is still accepted on import. The `Era.*` options and `admin_importHistory`/`admin_exportHistory` described below apply to Era1. EraE has its own equivalents: - Import: [`EraE.ImportDirectory`](./configuration.md#erae-importdirectory), or the `admin_importEraHistory` JSON-RPC method. -- Export: [`EraE.ExportDirectory`](./configuration.md#erae-exportdirectory), or the `admin_exportEraHistory` JSON-RPC method. Exporting post-Merge blocks requires [`EraE.BeaconNodeUrl`](./configuration.md#erae-beaconnodeurl), which supplies the beacon block and state roots. -- Range and verification: [`EraE.From`](./configuration.md#erae-from), [`EraE.To`](./configuration.md#erae-to) (`0` means head), and [`EraE.TrustedAccumulatorFile`](./configuration.md#erae-trustedaccumulatorfile). +- Export: [`EraE.ExportDirectory`](./configuration.md#erae-exportdirectory), or the `admin_exportEraHistory` JSON-RPC method. Set [`EraE.BeaconNodeUrl`](./configuration.md#erae-beaconnodeurl) to supply beacon block and state roots during post-Merge export. +- Range and verification: [`EraE.From`](./configuration.md#erae-from), [`EraE.To`](./configuration.md#erae-to), and [`EraE.TrustedAccumulatorFile`](./configuration.md#erae-trustedaccumulatorfile). For export, `EraE.To` set to `0` means the node's current head; for import, it means the last block available in the archive. ### Remote archives -Set [`EraE.RemoteBaseUrl`](./configuration.md#erae-remotebaseurl) to a remote EraE server to download missing epoch files on demand instead of providing them locally. Downloads are checksummed against the server's `checksums_sha256.txt` manifest and cached in [`EraE.RemoteDownloadDirectory`](./configuration.md#erae-remotedownloaddirectory), which defaults to the import directory. +When importing, set [`EraE.RemoteBaseUrl`](./configuration.md#erae-remotebaseurl) to download archive files that are missing from the import directory. You must still provide a local source directory through `EraE.ImportDirectory` or the `admin_importEraHistory` method. Downloaded files are verified against the SHA-256 hashes in the server's `checksums_sha256.txt` manifest and cached in [`EraE.RemoteDownloadDirectory`](./configuration.md#erae-remotedownloaddirectory), which defaults to the import directory. ## Import