Repository navigation
perf: stop fetching every block between two heads - #226
Open
moodysalem wants to merge 3 commits into
Open
moodysalem wants to merge 3 commits into
moodysalem wants to merge 3 commits into
Conversation
`ChainTracker` refreshes the head and the finalized block on independent intervals. On a chain that finalizes faster than the head refresh interval the finalized block returned by `updateFinalized` is regularly ahead of the head the tracker still holds, and the loop that prunes newly finalized blocks from the canonical chain then deletes the head's own entry. The next `updateHead` cannot find a parent for the block after the head and throws "Chain reorganization detected. Recovery not implemented", which takes the stream down even though no reorg happened. This is easy to hit in practice. Indexing BNB Smart Chain (0.45 s blocks, ~2 blocks of finality lag) and Monad (0.3 s blocks, 1-2 blocks) with a 2 s head refresh and the default 30 s finalized refresh crashes on almost every finalized refresh: nine and eleven restarts respectively over seven minutes, each one exactly 30, 60 or 90 seconds after start. Three changes: - `updateFinalized` adopts a finalized block that is at or ahead of the tracked head as the new head. A finalized block is canonical by definition, so this is always safe, and it leaves the head where the canonical chain can still reach it. - `updateFinalized` ignores a finalized block older than the one it already has instead of throwing "Finalized cursor moved backwards". Finalized blocks never revert, so an older answer is a stale view rather than a change to the chain. Load-balanced endpoints return one routinely: `mainnet.optimism.io` answered with a finalized block up to 39M blocks behind the previous answer on 130 of 180 consecutive polls. - The "join" path in `updateHead` recovers from a genuine reorg on a longer chain instead of throwing. It now walks back to the most recent block both chains agree on and reports it as a reorg, which is what the shorter-chain path already did. That walk is shared between the two paths and no longer requires the parent to be present in the canonical chain: the canonical chain is sparse, so a missing parent is not evidence of a reorg, and the walk continues by hash until it meets a block that is present and matches or reaches the finalized block, which is a common ancestor by definition. Adds `packages/protocol/tests/chain-tracker.test.ts`. Five of its nine cases fail against the current implementation. Claude-Session: https://claude.ai/code/session_01B95ysFTmvKneRhhjQX5bU9
The live path made one `eth_getBlockByNumber` per block produced since the last head refresh, plus one `eth_getBlockByHash` per poll for a block it had already fetched. On a chain producing blocks faster than the refresh interval that is most of the requests the stream makes, and it scales with the chain's block rate rather than with how much data the filter matches. Two changes. `ChainTracker.updateHead` used to walk every block between the old and the new head, in batches, checking that each one's parent hash matched the block before it. It now asks the chain for the single block it has at the old head's height. The node answers from the chain that ends at the new head, so if that block is still the old head then the old head is on that chain and the two are connected. If it is a different block, the chain reorganized at or below the old head, and the existing walk back to the common ancestor handles it. `EvmRpcStream` keeps the last 64 blocks it fetched, keyed by hash, and answers `fetchHeaderByHash` from them when it can. The stream asks for the header of the block it just saw at the head on every poll that matched no logs, which is the block the preceding `eth_getBlockByNumber` returned. A block is immutable once it has a hash, so this is the same answer for one fewer request. Per 2 s poll on a chain with no matching logs, using block rates measured from a production indexer: | chain | blocks/poll | requests before | after | |----------------|-------------|-----------------|-------| | Robinhood | 20 | 23 | 3 | | Monad | 6.7 | 9.7 | 3 | | BNB Smart Chain| 4.4 | 7.4 | 3 | | World Chain | 1 | 3 | 2 | The cost of a head refresh no longer depends on how far the head moved: the same three requests cover a two block advance and a two thousand block one. The canonical chain is now sparser, since the blocks between two heads are no longer added to it. Two things read it. The walk back to the common ancestor already tolerates gaps, following parent hashes until it meets a block that is present and matches. `isCanonical` returns true for a block it has not seen, so the guard in `produceLiveBlocks` now covers the heads the tracker observed rather than every block between them; a reorg it would have caught there is still caught by the next `updateHead`, which reports it and lets the stream invalidate instead of throwing. A reorg now costs more than it did, since the walk back has fewer cached blocks to stop at and follows parent hashes until it reaches one. That trade buys a constant cost on every poll for a higher cost on a rare one. `batchSize` is no longer used and is marked deprecated rather than removed, so existing callers keep compiling. Adds `packages/evm-rpc/tests/block-cache.test.ts` and three cases to `packages/protocol/tests/chain-tracker.test.ts`. Four of them fail against the branch this is stacked on. Claude-Session: https://claude.ai/code/session_01B95ysFTmvKneRhhjQX5bU9
Contributor
|
Warning Review limit reachedNext included review available in 27 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
When the linkage check finds a different block at the old head's height, the tracker already holds that block, and the two chains diverge at or below its height. Walking back from it starts at the fork; walking back from the new head first descends every block the head advanced by, one request each, only to arrive at the same place. That difference is invisible when the head moved by one or two blocks, and large exactly when it hurts: a stall that leaves the head far behind, resolved by a refresh that also happens to cross a reorg. In the tests it takes the longer-chain reorg case from five block fetches to one, and the walk to the finalized block from four to one, both returning the same cursor as before. Also renames the two change files to match the convention already in `change/`, which keeps the package name intact. Claude-Session: https://claude.ai/code/session_01B95ysFTmvKneRhhjQX5bU9
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Tailing the head costs one
eth_getBlockByNumberper block the chain produced since the last refresh, plus oneeth_getBlockByHashper poll for a block the stream already has. So the request count scales with the chain's block rate and not with how much data the filter actually matches — indexing a contract with no activity on a 0.3 s chain costs an order of magnitude more than the same contract on a 12 s chain.Two places are responsible.
ChainTracker.updateHeadwalks every block between the old and new head, in batches of 20, checking that each one's parent hash matches the block before it:The range is
head+1 .. newHeadinclusive, so it also re-fetches the block thatfetchCursor({ blockTag: "latest" })just returned.EvmRpcStream.fetchHeaderByHashissues aneth_getBlockByHashforhead.uniqueKeyon every poll where the head carried no matching logs — the block the precedingeth_getBlockByNumberalready returned in full.Changes
One lookup instead of one per block.
updateHeadnow asks the chain for the single block it has at the old head's height. The node answers from the chain that ends atnewHead, so if that block is still the old head, the old head is on that chain and the two are connected. If it is a different block, the chain reorganized at or below the old head and the walk back to the common ancestor from #225 takes over. Same question, one request.A small block cache.
EvmRpcStreamkeeps the last 64 blocks it fetched keyed by hash and answersfetchHeaderByHashfrom them when it can. A block is immutable once it has a hash, so this is the same answer for one fewer request. It falls through toeth_getBlockByHashfor anything it has not seen, and the bound is there so a long reorg walk cannot grow it without limit.Effect
Per 2 s poll on a chain with no matching logs. Block rates are measured from a production indexer; request counts follow from the code paths above.
The cost of a head refresh no longer depends on how far the head moved.
costs the same whether the head advances by 2 blocks or by 2000in the tests is exactly that.In Alchemy compute units, where
eth_getBlockByNumberandeth_getBlockByHashare 20 andeth_getLogsis 60, a Monad worker goes from ~10.1M CU/day to ~4.3M, and after the change the log query is the majority of what is left — which is the right shape, since that is the call doing the actual work.What this gives up
The canonical chain is sparser: the blocks between two heads are no longer added to it. Two things read it.
isCanonicalreturnstruefor a block it has not seen. The guard inproduceLiveBlockstherefore now covers the heads the tracker observed rather than every block between them. A reorg it would previously have caught there is still caught by the nextupdateHead, which reports it and lets the stream emitinvalidate— recovery rather than theTrying to process non-canonical blockthrow.A reorg costs a little more than before, because the walk has fewer cached blocks to stop at. It starts at the fork rather than at the new head, so it does not pay for the distance the head moved: in the tests the longer-chain reorg case takes one block fetch and the walk to the finalized block takes one, against five and four if it started from the new head.
The linkage check also assumes both requests reach the same node.
latestand the check are two calls, so under a fallback transport that failed over in between they can come from different backends, and where the old code verified every parent link in the range this verifies one. Mixed backends were already a hazard here — that is how a load-balanced endpoint produces a finalized block older than the last one, which is what #225 is about — and viem'sfallbackonly routes elsewhere on error, so this is not a new failure mode so much as one more place it shows up. Worth knowing about.If you would rather have this behind an option, say so and I will put it there — I left it unconditional because the old path's cost is paid on every poll on every chain.
batchSizeis now unused. I marked it@deprecatedrather than removing it so existing callers keep compiling.Tests
packages/evm-rpc/tests/block-cache.test.ts(new, 3 cases): the header comes from the head refresh with no second request; an unseen hash still falls through toeth_getBlockByHash; the cache evicts.packages/protocol/tests/chain-tracker.test.ts(4 new cases): one block fetched when the head advances by many; identical cost for a 2 block and a 2000 block advance; nothing fetched on a single block advance; and a reorg that walks back from the fork rather than from the new head.Four of those seven fail against the branch this is stacked on. The existing reorg cases pass unchanged, which is the part I most wanted to confirm — the sparser map does not change where a reorg resolves to.
pnpm beachball checkpasses.The repo-wide
pnpm test:ciis red here for an unrelated reason:tests/log-fetcher.test.tsandtests/merge-get-logs.integration.test.tsfall back tohttps://ethereum-rpc.publicnode.comwhenTEST_EVM_RPC_URLis unset, which it is for fork PRs since they do not get repository secrets, and that endpoint now rejects the archiveeth_getLogsranges those tests use. Those three cases are the only failures in the run, and they fail the same way on unmodifiedmainlocally. The newblock-cache.test.tsin that same package passes.