Skip to content

perf(chain): add secondary indexes to ProposalMap, eliminate O(n) linear scans#38

Open
simonzg wants to merge 2 commits into
mainnetfrom
perf/proposal-map-indexes
Open

perf(chain): add secondary indexes to ProposalMap, eliminate O(n) linear scans#38
simonzg wants to merge 2 commits into
mainnetfrom
perf/proposal-map-indexes

Conversation

@simonzg

@simonzg simonzg commented May 1, 2026

Copy link
Copy Markdown
Contributor

Why

ProposalMap stores in-flight HotStuff consensus proposals in a single map[Bytes32]*DraftBlock keyed by block ID. Three query methods perform full O(n) linear scans of the entire map on every call:

Method Called from Frequency Old complexity
GetOneByEscortQC every incoming vote per vote O(n)
GetDraftByNum block seeker, sync per query O(n)
PruneUpTo epoch commit per epoch O(n)

GetOneByEscortQC is the most critical: it runs on every vote received during consensus to match the vote's QC back to a known in-flight proposal. With N proposals in-flight (can be dozens during busy view changes), every single vote pays O(N) scan cost — a direct tax on consensus throughput.

What Changed

Two secondary indexes are maintained alongside the primary proposals map:

byNum         map[uint32][]*DraftBlock  // block number → list of drafts at that height
byHeightRound map[uint64]*DraftBlock    // height<<32|round → draft (unique per QC)

GetOneByEscortQC: O(n) full scan → O(1) lookup in byHeightRound
GetDraftByNum: O(n) full scan → O(1) lookup in byNum
PruneUpTo: O(n) scan+delete → iterates only the relevant height buckets in byNum

All three indexes are kept in sync in Add, PruneUpTo, and CleanAll. No external API changes.

Expected Impact

  • Per-vote consensus processing cost drops from O(proposals) to O(1) for the proposal lookup step.
  • Epoch cleanup (PruneUpTo) no longer scans every in-flight proposal.
  • Directly reduces consensus round latency, especially under view changes where many proposals accumulate.

simonzg and others added 2 commits July 10, 2025 09:54
…ear scans

GetOneByEscortQC, GetDraftByNum and PruneUpTo all performed full O(n) scans
over the entire proposal map on every call.

GetOneByEscortQC is on the critical consensus path: called for every incoming
vote to match the QC to a known in-flight proposal. With N proposals in
flight, each vote paid O(N) scan cost.

Add two secondary indexes maintained alongside the primary map:

  byNum        map[uint32][]*DraftBlock  — index by block number
  byHeightRound map[uint64]*DraftBlock   — index by height<<32|round

GetOneByEscortQC: O(n) scan → O(1) map lookup
GetDraftByNum:    O(n) scan → O(1) map lookup
PruneUpTo:        O(n) scan → O(height-range) using byNum

All three indexes are kept in sync in Add, PruneUpTo, and CleanAll.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant