feat(s3): conclude parked blobs in chunked single-request batches (#69)

[frrist]

Note

Draft — incomplete by design. This is the client-side half of #69 and a working sketch of the better cross-service design, opened to preserve and socialize the work before a ~5-month absence. It is functional against unmodified sprue (rebased onto current main, unit suite green), but it is not the end state — see "What remains" below before investing review effort in the details.

What this does

Complete's accept phase no longer spends one HTTP round trip per parked blob. The UCAN container is already a batch primitive — the stock ucantone server executes every invocation in a request container addressed to it and returns all their receipts in one response container, and the stock client merges WithInvocations into the request it sends — so batching needs no protocol or sprue change:

• forgeclient.BlobConcludeBatch synthesizes every parked blob's put receipt and ships all the conclude invocations + receipts in one POST, recovers each conclude's receipt from the response container by task link, and fetches the accept receipts (location commitments) bounded-parallel — sprue stores them before answering, so the first fetch succeeds. Outcomes are positional and per-blob; failed entries stay parked and idempotently retryable. A round-trip test drives the real ucantone server + receipt endpoint in-process and pins the one-POST behavior.
• uploader.DeferredBodyUploader trades ConcludeBlob for ConcludeBlobBatch; s3frontend.concludeBlobs triages located/parked/never-parked digests, concludes the parked set in chunks of 16 with 4 chunks in flight, and lets every chunk run to completion on failure — each recorded location is one less conclude for the retry's resume.
• The first commit fixes Complete timing out and 404ing its own retry; its join/heal semantics (idempotent per-blob resume, join-on-retry, SlowDown past budget) are protocol-independent and survive every later design stage unchanged.

What remains — the full design this sketches toward

This PR batches at the transport level against today's protocol. The end state is protocol-level batching plus removal of the real latency floor, which lives in piri, not here. The complete cross-service plan (issue-ready drafts, verified file:line evidence, sequencing) is preserved in the piri handoff ledger, PIRI.md §9 “Batch / single-request conclude” (handoff bundle — ask @hannahhoward). Summary:

• sprue — return accept receipts + location commitments inline in the conclude response metadata (kills the per-blob receipt GET this PR still does); then, behind a libforge change, group batched accepts by provider into one multi-invocation POST per provider with a single agent-store write. (unfiled — drafts in PIRI.md §9 as sprue A / sprue B)
• libforge — additive ConcludeArguments.Receipts []cid.Cid so sprue can see the whole batch at once. (unfiled — libforge A)
• piri — the dominant remaining term: every accept publishes under a process-global mutex plus a blocking indexer HTTP call. blob/accept: make location-claim publishing (IPNI advert + indexer cache) optional piri#78 (filed) makes location-claim publishing optional for this topology; piri B (async/batched publisher) is the real fix, and piri C adds body limits / server timeouts / compensation before real batch sizes arrive. (B and C unfiled)
• ucantone — optional ergonomics: first-class ExecuteBatch, and opt-in bounded-concurrency container execution (measure first). (unfiled)

Nothing here depends on hilt; its blob.Abort/blob.Remove floors are already met.

Refs

• Closes the client half of CompleteMultipartUpload holds the connection for O(parts) conclude round trips; batch the /ucan/conclude receipt send #69 (the protocol half above stays open under it)
• blob/accept: make location-claim publishing (IPNI advert + indexer cache) optional piri#78 — location-claim publishing optional (filed; the topology escape hatch)
• Rebase note: replayed onto main post lock/tagging; one test-only fix for the new registry.CreateState parameter in mem.Create.

🤖 Generated with Claude Code