improvements full history sync close to current epoch

[AdoAdoAdo]

Reasoning behind the pull request

Proposed changes

Testing procedure

Pre-requisites

Based on the Contributing Guidelines the PR author and the reviewers must check the following requirements are met:

• was the PR targeted to the correct branch?
• if this is a larger feature that probably needs more than one PR, is there a feat branch created?
• if this is a feat branch merging, do all satellite projects have a proper tag inside go.mod?

[Copilot]

Pull request overview

This PR refines full-history sync behavior close to the current epoch by (1) improving when “blocking unproven headers” are dropped during sync retries, and (2) routing historical data requests across main vs full-archive peers based on epoch availability, backed by a new “assumed active persisters on peers” config input.

Changes:

• Add a dedicated tracker for dropping blocking unproven headers only after a “presence window” of consecutive failures, plus tests for the tracker and updated shard bootstrap tests.
• Introduce a 3-band request routing strategy (main only / main+insurance full-archive / full-archive only) and improve full-archive peer selection (topic peers first + exploration).
• Extend epoch-provider plumbing with EpochIsAvailableOnMainPeers() and add StoragePruning.AssumedPeersNumActivePersisters to drive routing decisions.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
process/sync/baseSync.go	Adds blocking-unproven-header presence tracking and clears it on successful commit.
process/sync/unprovenHeaderTracker_test.go	New tests covering the unproven header presence-window tracker behavior.
process/sync/shardblock_test.go	Updates tests to validate the new presence-window removal semantics.
dataRetriever/topicSender/topicRequestSender.go	Implements 3-band routing + two-pass full-archive peer selection + rating-coverage plumbing.
dataRetriever/topicSender/baseTopicSender.go	Adds constant for dual-band (insurance) full-archive budget.
dataRetriever/topicSender/topicRequestSender_test.go	Adds extensive tests for band routing, selection passes, preferred-peer handling, and rating coverage.
dataRetriever/interface.go	Extends CurrentNetworkEpochProviderHandler with EpochIsAvailableOnMainPeers.
dataRetriever/resolvers/epochproviders/arithmeticEpochProvider.go	Adds assumed-persister window + EpochIsAvailableOnMainPeers implementation and validation.
dataRetriever/resolvers/epochproviders/arithmeticEpochProvider_test.go	Adds/updates tests for new assumed-persister validation and availability window logic.
dataRetriever/resolvers/epochproviders/errors.go	Adds a dedicated validation error for assumed persisters.
dataRetriever/resolvers/epochproviders/export_test.go	Extends test constructor wiring with assumed persisters.
dataRetriever/resolvers/epochproviders/disabled/disabledEpochProvider.go	Implements the new interface method for the disabled provider.
dataRetriever/factory/epochProviders/currentEpochProvidersFactory.go	Threads assumed persisters into arithmetic epoch provider creation.
dataRetriever/factory/epochProviders/currentEpochProvidersFactory_test.go	Adds coverage around assumed-persister handling (regular vs full-archive).
dataRetriever/mock/currentNetworkEpochProviderStub.go	Updates mock to satisfy the extended epoch-provider interface.
integrationTests/mock/currentNetworkEpochProviderStub.go	Updates integration stub to satisfy the extended epoch-provider interface.
epochStart/bootstrap/disabled/disabledCurrentNetworkEpochProviderHandler.go	Implements the new interface method for disabled bootstrap handler.
config/config.go	Adds AssumedPeersNumActivePersisters to StoragePruningConfig.
cmd/node/config/config.toml	Adds default config + docs for AssumedPeersNumActivePersisters.
testscommon/generalConfig.go	Updates test general config with the new storage pruning field.
factory/processing/processComponents.go	Passes the new config field into epoch provider creation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[AdoAdoAdo]

done

[Copilot]

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.

Suppressed comments (2)

dataRetriever/topicSender/topicRequestSender.go:190

• The comment says an exploration slot is always reserved while unexplored peers exist, but the implementation only reserves one when budget > 1 (and cannot when budget == 1). Please adjust the wording to reflect the actual behavior so future readers don’t assume exploration happens even with a single-slot budget.

// sendOnFullArchiveNetwork queries the topic subscribers first, then explores the rest of the
// connected peers with the leftover budget; the topic view is a cached gossip-mesh snapshot that
// can be incomplete, so one exploration slot is always reserved while unexplored peers exist

dataRetriever/resolvers/epochproviders/arithmeticEpochProvider.go:94

• The comment for EpochIsAvailableOnMainPeers describes availability of past epochs on main peers, but the implementation also returns true when epoch > currentComputedEpoch (overflow guard). Please document this explicitly to avoid confusion about how future/unknown epochs are handled.

// EpochIsAvailableOnMainPeers returns true if regular network peers are assumed
// to still hold the persister for the given epoch
func (aep *arithmeticEpochProvider) EpochIsAvailableOnMainPeers(epoch uint32) bool {

[ssd04]

*optional: budget here refers to number of peers? maybe rename to something more related

[AdoAdoAdo]

done

[ssd04]

should we still reset this map? in case of fail without rollback

[AdoAdoAdo]

the removal of the map reset was done to keep the 2 recovery mechanisms separate.

[codecov]

Codecov Report

❌ Patch coverage is 98.24561% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.70%. Comparing base (1d9bd07) to head (2390c33).
⚠️ Report is 9 commits behind head on feat/testnet-fixes.

Files with missing lines	Patch %	Lines
dataRetriever/topicSender/topicRequestSender.go	97.14%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@                  Coverage Diff                   @@
##           feat/testnet-fixes    #7965      +/-   ##
======================================================
+ Coverage               77.68%   77.70%   +0.01%     
======================================================
  Files                     893      893              
  Lines                  129612   129696      +84     
======================================================
+ Hits                   100689   100776      +87     
+ Misses                  22335    22330       -5     
- Partials                 6588     6590       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.