feat: settle CDN bandwidth per shared rail (subscription) - #17
Draft
juliangruber wants to merge 1 commit into
Draft
feat: settle CDN bandwidth per shared rail (subscription)#17juliangruber wants to merge 1 commit into
juliangruber wants to merge 1 commit into
Conversation
Bandwidth (CDN) usage accumulates per cdnRailId, the shared bandwidth rail that is the CDN subscription identity, so data sets that share a subscription settle bandwidth once with the summed amount via the new FWSS settleCDNBandwidthRail. Cache-miss stays keyed by dataSetId and settles per data set through settleFilBeamPaymentRails(dataSetId, 0, cacheMissAmount). settleCDNBandwidthRails(cdnRailIds) is the canonical bandwidth path the FilBeam worker calls. settleCDNPaymentRails(dataSetIds) is kept as a convenience that resolves data sets to their rail first. CDNSettlement is re-keyed to cdnRailId, mocks/tests/docs/tool updated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Updates
FilBeamOperatorso CDN bandwidth is metered and settled per shared bandwidth rail (the CDN subscription) instead of per data set, while cache-miss stays per data set.Full proposal, with rationale and the companion changes in the other repos: https://gist.github.com/juliangruber/a34b225f9588ec68d069054d731e20c9
The gist also links to the companion draft PRs in
FilOzone/filecoin-services,FilOzone/synapse-sdk, andfilbeam/worker.Why
Multi-copy upload stores a piece in 2 data sets on 2 providers. When CDN is enabled on both, the bandwidth service was bought twice. FWSS now lets data sets share one bandwidth rail keyed by
(payer, group), and exposessettleCDNBandwidthRail(cdnRailId, cdnAmount)to settle that shared rail once. The sharedcdnRailIdis the subscription identity, so the operator aggregates and settles bandwidth by rail id.How
cdnRailAmount[cdnRailId], resolved fromFilecoinWarmStorageServiceStateView.getDataSet(dataSetId).cdnRailIdat record time. Cache-miss stays indataSetUsage[dataSetId].settleCDNBandwidthRails(uint256[] cdnRailIds)is the canonical bandwidth settlement path, it settles each shared rail once with its aggregated amount via FWSSsettleCDNBandwidthRail. This is what the FilBeam worker calls with the distinct rail ids it has metered.settleCDNPaymentRails(uint256[] dataSetIds)is kept as a convenience that resolves each data set to its rail first, both paths share one internal settle-by-rail-id helper.settleFilBeamPaymentRails(dataSetId, 0, cacheMissAmount), withcdnAmountalways 0 so the bandwidth portion is never double-settled.CDNSettlementis keyed bycdnRailId,CacheMissSettlementstays keyed bydataSetId.MockFWSSgainssettleCDNBandwidthRail.Cross-repo interface
The wire ABI the
filbeam/workerPR targets:recordUsageRollups(uint256 toEpoch, uint256[] dataSetIds, uint256[] cdnBytesUsed, uint256[] cacheMissBytesUsed)- usage reported per data set, the contract resolves and aggregates bandwidth onto the shared rail.settleCDNBandwidthRails(uint256[] cdnRailIds)- bandwidth, once per rail.settleCacheMissPaymentRails(uint256[] dataSetIds)- cache-miss, per data set.Reviewer notes
The real FWSS must expose
settleCDNBandwidthRail(uint256,uint256)callable by the FilBeam controller, see the companionfilecoin-servicesPR. Called here through a minimal localIFilBeamBandwidthSettlementinterface againstfwssContractAddress.Follow-ups
These are intentionally out of scope for this draft.
settleCDNPaymentRails(by data set) intosettleCDNBandwidthRails(by rail) once downstream settles by rail id everywhere.maxReportedEpochguard.