Skip to content

feat: clear extended provider via ClearExtendedProvider special message#30

Merged
hannahhoward merged 1 commit into
mainfrom
feat/ipni-queue-purge
May 19, 2026
Merged

feat: clear extended provider via ClearExtendedProvider special message#30
hannahhoward merged 1 commit into
mainfrom
feat/ipni-queue-purge

Conversation

@hannahhoward

Copy link
Copy Markdown
Contributor

Summary

Adds a new SQS special-message body ClearExtendedProvider to the publishing handler, mirroring the existing AnnounceHTTP precedent (#27). When the handler receives a message with this body, it publishes a single advertisement whose ExtendedProvider.Providers field is explicitly empty ([]). storetheindex ≥ commit e1afa6b8 (from PR #2814) honors this shape via its registry.Update Empty() branch, setting info.ExtendedProviders = nil so subsequent lookups return no ExtendedProvider entries. See ipni/storetheindex#1745 for the design rationale and @gammazero's recommendation.

The base provider (bitswap) is intentionally left intact on the advertisement — this PR only clears the ExtendedProvider half. Operators wanting to also disable the bitswap routing should handle that out-of-band (e.g. by taking the bitswap peer offline at the network layer; its still-advertised multiaddrs will fail to connect).

Deliverables

File Status Summary
src/handlers/advertisement.js Modified New constant + dispatch branch; post-encodeAndSign attachment of ExtendedProvider: { Providers: [], Override: false }
test/advertisement.test.js Modified New tap test advertisement - clear extended provider (18 assertions) mirroring the existing extended provider test pattern

How the empty-Providers attachment works

@web3-storage/ipni's Advertisement.encodeAndSign() only writes the ExtendedProvider field when providers.length > 1. The advertisement's Signature is computed over previous + entries + provider.peerId + provider.addresses + provider.metadata + IsRm — it does NOT cover the ExtendedProvider list. So we construct a single-provider Advertisement (the bitswap base), call encodeAndSign(), then attach value.ExtendedProvider = { Providers: [], Override: false } to the returned value before encoding to dag-json. The AD signature remains valid because the bytes it covers are unchanged.

Operator runbook

Trigger:

  1. Verify the indexers you care about are running storetheindex ≥ e1afa6b8 (PR #2814 merged 2026-04-08). Older indexers will ingest the ad cleanly but treat the empty-Providers attachment as a no-op; forward-compatible.
  2. Verify publisher-lambda is deployed and its input SQS queue is reachable.
  3. Send a single message:
    aws sqs send-message \
      --queue-url <publisher-lambda-input-queue-url> \
      --message-body ClearExtendedProvider
  4. Tail publisher-lambda CloudWatch logs — expect a single advertisement construction + two S3 writes (the new ad object + HEAD update).

Verify:

  1. Fetch the new HEAD: curl https://<bucket>.s3.<region>.amazonaws.com/head — confirm it points at a new CID.
  2. Fetch the new ad: curl https://<bucket>.s3.<region>.amazonaws.com/<new-cid> and confirm the JSON contains "ExtendedProvider": { "Providers": [], "Override": false }.
  3. Wait for the indexer ingest cycle (typically minutes), then query a previously-indexed multihash against the indexer's find endpoint:
    curl https://cid.contact/cid/<known-indexed-cid>
    Confirm the response no longer contains ExtendedProvider entries for the publisher peer ID.

Out-of-band (if disabling IPFS retrieval entirely): take the bitswap peer offline at the network layer.

Test plan

  • New test advertisement - clear extended provider passes (18/18 assertions)
  • Existing tap suite passes (32 prior assertions in test/advertisement.test.js + the rest of the suite — npm test shows all 5 test files green)
  • npm run lint clean
  • Manual verification against a dev/staging publisher-lambda before triggering against production

🤖 Generated with Claude Code

Adds a new SQS special-message body 'ClearExtendedProvider' that, when
received, publishes an advertisement whose ExtendedProvider.Providers
field is explicitly empty. storetheindex >= e1afa6b8 (PR #2814) honors
this shape via its registry.Update Empty() branch, setting
info.ExtendedProviders = nil so subsequent lookups return no
ExtendedProvider entries for the publisher peer ID.

See ipni/storetheindex#1745 for the design rationale.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
}

const value = await ad.encodeAndSign()
// The library only writes ExtendedProvider when providers.length > 1; attach

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait -- won't this invalidate the signature -- value has already been signed, and now you're changing one of its struct members?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good check. No — the signature is computed over a fixed subset of fields that does NOT include the ExtendedProvider list, so post-hoc attachment is cryptographically safe.

From @web3-storage/ipni/advertisement.js signableBytes():

return concat([
  ad.previous?.bytes ?? new Uint8Array(),
  ad.entries.bytes,
  text.encode(provider.peerId.toString()),
  text.encode(provider.addresses.map(a => a.toString()).join('')),
  provider.encodeMetadata(),
  new Uint8Array([IsRm])
])

Just previous + entries + provider.peerId + provider.addresses + provider.metadata + IsRm. This matches go-libipni's canonical signaturePayload() in ingest/schema/envelope.go, which is what storetheindex reconstructs on ingest to compare against. Since the new value.ExtendedProvider field doesn't intersect any of those signed bytes, the existing value.Signature remains valid against the mutated value.

(There ARE per-provider signatures on each entry in ExtendedProvider.Providers — but we have zero entries, so none to compute.)

@hannahhoward hannahhoward marked this pull request as ready for review May 19, 2026 03:11
@hannahhoward hannahhoward merged commit ead9c04 into main May 19, 2026
1 check passed
@hannahhoward hannahhoward deleted the feat/ipni-queue-purge branch May 19, 2026 03:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant