Repository navigation
confirm: cut write-thread CPU below the milestone - #870
Merged
Merged
Conversation
The write phase looked up each spend abs in fill, again in ensure_spend_abs_layouts, and a third time inside structural. One spend_abs_jobs walk after fill is the Corrupt gate before tx.head insert, and structural reuses those jobs.
Each write batch allocated a new spentness scratch, pending-spent set, and create-height map. The write thread keeps one of each and clears slots, the pending set, and heights at the start of the batch so annotate and double-spend state cannot cross batches. The pending check is one insert.
IBD create fks are one contiguous range per block. Binary search over those spans replaces a HashMap insert per transaction on the write thread. A gap, a null fk, or an overlap keeps the map so a missing fk does not inherit a neighbor height.
The write thread re-read header.body for each connected block while holding the confirmed-hash lock. Copy the wire headers before the batch is consumed and put those on TipEvent. A count that does not match the batch is Corrupt. Mempool relay stays gated on relay_enabled. ensure_spend_abs_layouts is test-only. The non-test library was carrying that walker after the write path stopped calling it.
Clippy's argument cap is 7. The write-thread entry keeps its call-site arguments unbundled, matching the other confirm helpers.
Copying the wire headers before commit adds one arm to the write loop, which was already at clippy's complexity cap. Same allowance as the load thread.
A restart preload rewrites the snapshot and truncates the journal to its header. The next connect appends one record. When the previous run already journaled a record of that size, the two files stay the same length while the new height is stored. The cross-surface check now reads that height.
Under a full workspace run the follower was still mid-burst when the 10s wait ended (55 of 80). Forty-five seconds stays inside the 60s sync_blocks interval.
The workspace run hit the 30s wall around the getblocktxn and orphan steps while other tests were running. Neighboring multinode tests already use a 60s wall.
changelog.d fragment for the single spend-abs walk, the reused structural scratch, foreign-key span heights, and the validated wire header on the tip event.
![rearden-grok[bot]](https://avatars.githubusercontent.com/u/730881?s=60&v=4){kind=small}
Contributor
Author
There was a problem hiding this comment.
Reviewed the write-thread changes, the updated invariants, and the associated test changes against the repo's confirm pipeline and IO rules. No correctness or significant performance findings. The one-pass spend-absolute collection, span lookup fallback, per-batch scratch reset, and prevalidated wire-header forwarding are consistent with the described stage order. qc, test, windows, and macos are green; coverage and Rust CodeQL were still running when I reviewed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Below the milestone the write thread is the confirm pole. Script jobs are already off, and each batch was still walking spend absolute offsets more than once, allocating a fresh structural scratch and create-height map, and reading the wire header back from the store for the tip event.
Corruptbefore thetx.headinsert.Spentness order is unchanged: BIP30, absolute offsets, the same-batch overlay skip, the 8-byte spent-meta read, the in-batch double-spend set, coinbase maturity, BIP68, and subsidy. Class C still flushes pre-tip tables, then the confirmed set, then the tip seal. The load thread keeps its own double-spend set.
The fee-history, tip-burst, and compact-block commits only loosen deadlines that failed while the workspace suite was sharing the machine. The fee-history check now reads the new tip from the snapshot or the journal.
Test plan
cargo fmt --all -- --checkcargo clippy --workspace --all-targets -- -D warningscargo deny check./scripts/ast-grep.shcargo test --workspace --quiet./scripts/ci-os-smoke.shqc,test,windows,macos,coverage