Skip to content

feat: clear extended provider via ClearExtendedProvider special message #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hannahhoward merged 1 commit into from
May 19, 2026
Merged

feat: clear extended provider via ClearExtendedProvider special message #30

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions src/handlers/advertisement.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ const telemetry = require('../telemetry')
// see: https://github.com/elastic-ipfs/publisher-lambda/pull/27
const SPECIAL_MESSAGE_ANNOUNCE_HTTP_PROVIDER = 'AnnounceHTTP'

// see: https://github.com/ipni/storetheindex/pull/2814
const SPECIAL_MESSAGE_CLEAR_EXTENDED_PROVIDER = 'ClearExtendedProvider'

async function fetchHeadCid() {
try {
telemetry.increaseCount('http-head-cid-fetchs')
Expand Down Expand Up @@ -147,6 +150,7 @@ async function main(event) {

for (const record of event.Records) {
let ad
let clearExtendedProvider = false

if (record.body === SPECIAL_MESSAGE_ANNOUNCE_HTTP_PROVIDER) {
const http = new Provider({
Expand All @@ -158,6 +162,14 @@ async function main(event) {
previous: headCid,
providers: [bits, http]
})
} else if (record.body === SPECIAL_MESSAGE_CLEAR_EXTENDED_PROVIDER) {
ad = new Advertisement({
previous: headCid,
providers: [bits],
entries: null,
context: null
})
clearExtendedProvider = true
} else {
const entries = CID.parse(record.body)
const context = Buffer.from(entries.toString())
Expand All @@ -170,6 +182,14 @@ async function main(event) {
}

const value = await ad.encodeAndSign()
// The library only writes ExtendedProvider when providers.length > 1; attach

@hannahhoward hannahhoward May 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait -- won't this invalidate the signature -- value has already been signed, and now you're changing one of its struct members?

@hannahhoward hannahhoward May 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good check. No — the signature is computed over a fixed subset of fields that does NOT include the ExtendedProvider list, so post-hoc attachment is cryptographically safe.

From @web3-storage/ipni/advertisement.js signableBytes():

return concat([
  ad.previous?.bytes ?? new Uint8Array(),
  ad.entries.bytes,
  text.encode(provider.peerId.toString()),
  text.encode(provider.addresses.map(a => a.toString()).join('')),
  provider.encodeMetadata(),
  new Uint8Array([IsRm])
])

Just previous + entries + provider.peerId + provider.addresses + provider.metadata + IsRm. This matches go-libipni's canonical signaturePayload() in ingest/schema/envelope.go, which is what storetheindex reconstructs on ingest to compare against. Since the new value.ExtendedProvider field doesn't intersect any of those signed bytes, the existing value.Signature remains valid against the mutated value.

(There ARE per-provider signatures on each entry in ExtendedProvider.Providers — but we have zero entries, so none to compute.)

// the empty-Providers shape after signing so storetheindex's Empty() clearing
// path triggers (https://github.com/ipni/storetheindex/pull/2814). The AD
// signature covers previous+entries+provider+addresses+metadata+IsRm only,
// so this attachment does not invalidate it.
if (clearExtendedProvider) {
value.ExtendedProvider = { Providers: [], Override: false }
}
const block = await Block.encode({ value, codec: dagJson, hasher: sha256 })

// Upload the file to S3
Expand Down
69 changes: 68 additions & 1 deletion test/advertisement.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ const t = require('tap')
const varint = require('varint')
const dagJson = require('@ipld/dag-json')
const { MockAgent, setGlobalDispatcher } = require('undici')
const { awsRegion, s3Bucket, indexerNodeUrl } = require('../src/config')
const { awsRegion, s3Bucket, indexerNodeUrl, bitswapPeerMultiaddr } = require('../src/config')
const { handler } = require('../src/index')
const { trackAWSUsages, mockPeerIds } = require('./utils/mock')

Expand Down Expand Up @@ -291,3 +291,70 @@ t.test('advertisement - handles indexer announcing generic error', async t => {
{ message: 'ERROR' }
)
})

t.test('advertisement - clear extended provider', async t => {
t.plan(18)

const mockAgent = new MockAgent()
const mockHeadPool = mockAgent.get(`https://${s3Bucket}.s3.${awsRegion}.amazonaws.com`)
const mockIndexerPool = mockAgent.get(indexerNodeUrl)
mockPeerIds()

const head = 'baguqeeralr4pwxvbcc6voioqyc6aneg4pkoh5rhrfj35gbhrpxpeavsh6vsa'
mockAgent.disableNetConnect()
mockHeadPool
.intercept({ method: 'GET', path: '/head' })
.reply(200, `{"head": {"/": "${head}"}}`, {
headers: { 'content-type': 'application/json' }
})
mockIndexerPool
.intercept({
method: 'PUT',
path: '/ingest/announce'
})
.reply(204, '')

trackAWSUsages(t)
setGlobalDispatcher(mockAgent)

t.strictSame(
await handler({
Records: [
{ body: 'ClearExtendedProvider' }
]
}),
{}
)

t.equal(t.context.s3.puts.length, 2)
t.equal(t.context.s3.puts[0].Bucket, s3Bucket)
t.ok(t.context.s3.puts[0].Key.startsWith('bagu'))
t.equal(t.context.s3.puts[1].Bucket, s3Bucket)
t.equal(t.context.s3.puts[1].Key, 'head')

const ad = dagJson.decode(t.context.s3.puts[0].Body)

// AC6 chain continuity
t.equal(ad.PreviousID.toString(), head)

// AC3 ExtendedProvider field present with empty Providers array
t.ok(ad.ExtendedProvider)
t.ok(Array.isArray(ad.ExtendedProvider.Providers))
t.equal(ad.ExtendedProvider.Providers.length, 0)

// AC4 Override field present (value not locked)
t.ok('Override' in ad.ExtendedProvider)

// AC5 base provider unchanged - Addresses
t.ok(Array.isArray(ad.Addresses))
t.equal(ad.Addresses.length, 1)
t.equal(ad.Addresses[0], bitswapPeerMultiaddr)

// AC5 base provider unchanged - Provider peer ID
t.equal(typeof ad.Provider, 'string')
t.ok(ad.Provider.length > 0)

// AC7 signature present
t.ok(ad.Signature instanceof Uint8Array || Buffer.isBuffer(ad.Signature))
t.ok(ad.Signature.length > 0)
})
Loading