[mickvandijke]

Adds a get-or-closer request, Kademlia's FIND_VALUE for chunks (ADR-0020). A node that holds the chunk answers Found with it; one that does not answers Closer with its closest known peers.

• ChunkMessageBody::{GetOrCloserRequest, GetOrCloserResponse} carry ChunkGetOrCloserRequest and ChunkGetOrCloserResponse (Found | Closer | Error). They are appended after every existing variant, so older discriminants keep their wire values; a new test pins them.
• Closer carries an opaque payload. saorsa-core encodes it and checks it as a FIND_NODE answer (feat(dht): expose lookup steps for caller-driven lookups saorsa-core#173), so the peer format is versioned in one place.
• GET_OR_CLOSER_AGENT_TOKEN (get-or-closer/1) and advertises_get_or_closer: a node built before this request drops it without a reply, so clients send it only to peers whose user agent carries the token.

Stack, merge and release in this order:

1. feat(dht): expose lookup steps for caller-driven lookups saorsa-core#173
2. this PR
3. feat(storage): answer get-or-closer requests (ADR-0020) ant-node#249
4. feat(read): read chunks through a get-or-closer lookup (ADR-0020) ant-client#221

This PR is rebased onto main (5.0.1). ant-node#249 and ant-client#221 pin it by git rev, so it must be released before they merge.

Linear issue

Closes V2-1358

Risk tier

• T0 — docs / tooling / CI / pure UX-output. Repo CI only.
• T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
• T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
• T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Compatibility

• Wire: two new ChunkMessageBody variants, appended at indices 14 and 15. Existing messages are unchanged.
  • A node built before them cannot decode them: natively it sends no reply, and over WebRTC Direct it closes the channel.
  • So nodes announce get-or-closer/1 in their user agent, and clients only send the request to peers that do.
• Storage: none.
• API: additive. New types, a constant and a function; ChunkMessageBody is #[non_exhaustive].

Semver impact

• breaking
• feature
• fix

Test evidence

• cargo test --all-features: 108 passed. New tests cover:

  • request and response round trips
  • discriminant wire values for GetRequest (2), PointerGetRequest (12) and the new variants (14, 15)
  • token detection

• cargo clippy --all-targets --all-features -- -D warnings: clean.

• End to end through ant-node and ant-client, as listed in those PRs.

• Dev testnet comparison against main, 2026-10-27, after the review fixes. Two 60-node DigitalOcean fleets with the same layout:

  • 30% symmetric NAT across 9 regions
  • per fleet, 4 uploaders (50 MiB; 2 merkle, 2 no-merkle), 2 native downloaders and 1 headless-Chromium WASM downloader
  • 60-minute windows
  • both fleets on current main, differing only by the four PRs

  Three runs:

  1. closed loop, as staging runs its clients
  2. the same, with the code swapped between the fleets' droplets
  3. the swapped fleets, with native downloads paced to one every 40 s, so both fleets serve the same reads

  stack vs main	run 1	run 2, swapped	run 3, swapped, matched load
  native download p50	−52% (19.7 → 9.4 s)	−47% (18.5 → 9.9 s)	−43% (18.4 → 10.5 s)
  native downloads completed	733 vs 350	707 vs 372	177 vs 176
  merkle upload p50	+7.0% (p < 0.001)	+1.7% (p = 0.36)	−6.2% (p = 0.001)
  no-merkle upload p50	−0.2% (p = 0.90)	+8.6% (p = 0.001)	+0.9% (p = 0.72)
  WASM download p50	−2.0% (p = 0.43)	−4.2% (p = 0.10)	−2.8% (p = 0.13)
  node dial failures, minutes 10–60 (run 3: whole window)	21 vs 0	0 vs 12	0 vs 2
  failed transfers	0	0	0

  • Native downloads are 43–52% faster on whichever droplets run the stack.
  • At equal read load nothing regresses. In a closed loop, the stack's fleet serves about twice the reads, and uploads beside them run about 4% slower. Staging runs its clients closed loop, so it will see that.
  • Node dial and connection failures follow one fleet's droplets, not the code. Every run's steady-state tail sits on the same droplets (B), whichever code they run. saorsa-core#173 changes no node-side DHT path.
  • Duplicate copies: nodes sent 1.45–1.52 bytes per native byte downloaded, against 1.31–1.35 for main. Before the review fixes it was 1.54–1.68. The rest is the 500 ms hedge.
  • Reports: ant-testnet/state/comparisons/v2-1358-review-fixes-{vs-main,swapped,swapped-matched}-60m-1027/README.md.

New dependency

none

ADR

ADR-0020: Kademlia-style chunk GET: return the chunk or closer peers. Measurements are in the ant-client investigation.

Mitigation / rollback

Inert until nodes answer and clients send the request; revert the commit.

🤖 Generated with Claude Code

[dirvine]

Code approval at the reviewed head

No introduced blocking defect found in this PR. Appended wire variants, pinned discriminants and exact capability-token matching are consistent with the linked implementations. The peers field is deliberately opaque; its existing 5 MiB outer envelope is not a substitute for the inner decoder bound.

Linked stack is NOT approved to ship: saorsa-core#173 needs its decoder size guard, and downstream pins must follow. Preserve dependency-order publication and revalidate CI before merge.

Reviewed commit: 69a031bacd106100376f1914718bfbd44bb1e955.

Review method: six substantive seats across the linked set, including a completed z-ai/glm-5.2 API review recovered from a worker that timed out after saving its response; coordinator source checks and local execution. Initial functional/security/test seats recommended approval. The coordinator probe, independent adjudicator, API seat and replacement concurrency/release seats corroborate the missing decoder bound. This is not unanimous approval. The API seat recommended client approval with concerns; the coordinator withholds it because the exact client pin still exposes the defect.

Local verification (macOS, rustc 1.96.0): core closer-peers 3 passed and DHT-manager regression 143 passed; protocol all-features 108 passed; node get-or-closer unit tests 3 passed and real-transport E2E 1 passed; client kad_read unit tests 10 passed and real-transport E2E 1 passed. Formatting passed in all four repositories. No production/testnet deployment or merge performed. Draft git pins were verified with cargo metadata --locked; this is not published dependency closure. ADR-0020 remains Proposed.

[dirvine]

Re-review — no change at reviewed head

Head is unchanged at 69a031ba; the first-round APPROVE still stands. Nothing new in this delta.

Verdict

APPROVE (unchanged). CI all green at this head (format, clippy, tests ubuntu/macos/no-default-features, docs, wasm, security, publish dry-run).

Panel: GLM 5.3 seat billing-failed (402) — this seat is 2/3 reviewed (GLM unavailable), Qwen 3.8 + DS4 concur.

[jacderida]

Staging-shape comparison of the rebased stack (2026-11-03)

DEV-01 (registry 706: ant-node #249 @ 6a14b538, ant-client #221 @ 1061e38e, saorsa-core #173 @ c3c4c81e, ant-protocol #44 @ 0c6ae27b) against STG-01 699, the 2026-10-22 release-train staging run (ant-node 0.23.1-rc.1, ant 0.3.12-rc.1, WASM ant-cli-v0.3.12-rc.1). Both on the same staging profile: 7 bootstraps, 990 nodes (10% NAT'd) across OVH, Vultr and DigitalOcean, 10 native uploaders, 2 native downloaders, 3 WASM uploaders, 1 WASM downloader, 2 pointer clients. Both measured on a 3-hour steady-state window after a 1-hour warm-up, with the same back-to-back client workload. The candidate's WASM bundle was built from the ant-cli-v0.3.12 tag, so the browser tier ran the same code as the baseline's. Gates evaluated by saorsa-deploy deployments compare (thresholds v7). Full report: INFRA-476.

Native downloads

Per file-size bucket, native downloads started in the window, 699 → 706:

MB	n (699 / 706)	p50 699	p50 706	Δ p50	p95 699	p95 706	Δ p95
20	89 / 157	20.4 s	6.4 s	−68.6%	25.4 s	8.7 s	−65.7%
25	35 / 50	20.5	6.7	−67.3%	26.2	9.9	−62.2%
50	33 / 66	21.1	7.5	−64.5%	26.1	10.3	−60.5%
100	30 / 60	23.4	10.7	−54.3%	28.8	14.2	−50.7%
300	42 / 57	45.7	23.6	−48.4%	58.9	30.2	−48.7%
500	34 / 54	62.0	32.7	−47.3%	85.0	40.7	−52.1%
1000	63 / 99	102.5	66.3	−35.3%	127.1	89.3	−29.7%

625/625 native downloads succeeded with 0 content-verification mismatches. Because the downloaders run back-to-back, the candidate moved 198.6 GB in the window against the baseline's 122.6 GB (1.62×), so the nodes were serving more reads while they did this.

Release gates

• Hard gates: all PASS. Native upload success 99.966% (2901/2902, floor 99%), native download success 100% (floor 99.5%), data integrity 0 mismatches.
• Native uploads were not slower beside the heavier read load: p50 by bucket −9.3%, −1.5%, −12.3%, −14.0%, +2.4%, −14.6%; p95 −7.9% to −31.0%. Every within-window drift gate passes, with the candidate's drift within ±2.8 pp where the baseline's own ran +7 to +23 pp.
• Node resources: CPU +6.2%, RSS p95 +0.9%, memory-leak slope 3.85%/h against a 5%/h line, disk bytes per GB stored +0.6%. Outbound bytes per GB of traffic 22.68 vs 24.14 GiB/GB (−6.0%).
• Pointer tier: write p50 −3.6%, read p50 −4.0%, cost per write +1.0%, 0 stale reads, 0 lost writes.
• No node restarted on either fleet: 0 failed ant-node units, 0 restarts, 0 auto-upgrades across all 73 node and bootstrap VMs for the whole 5.5-hour run.
• WASM download p50s moved +5.8% to +10.6% across buckets on a bundle that is the baseline's own code (n ≤ 36 per bucket); WASM uploads +2.3% to +9.6%.

Route usage and cost

Over the window the nodes answered 94,196 get-or-closer requests on 72/72 hosts (46,440 found, 47,390 closer, 0 errors) and 44,662 plain GETs. The plain GETs are the kad read's own GETs to likely holders waiting their turn (KadGet::get → chunk_get_from_peer), as the ADR routes them; they are spread across the node fleet (45,358 on nodes, 747 on bootstraps), not a token-less-peer effect. Nodes sent 1.671 read bytes per native byte downloaded (chunk bytes 1.655, closer-peer payloads 0.015), or 1.50 once the WASM tier's 23 GB of downloads is counted in the denominator, inside the 60-node figure of 1.45–1.52.

Old client against new nodes

For the hour after the window, a cohort of released ant 0.3.12 (binary sha256 bb782247…, proven distinct from the branch build on every other client VM) ran one uploader and one downloader against the same branch nodes: 133/133 uploads, 66/66 downloads, 0 protocol handler errors on the nodes, 0 get-or-closer errors. In that same hour, same nodes, downloaders in the same region:

MB	branch client p50	released 0.3.12 p50	699 p50
20	6.9 s	19.7 s	20.4 s
50	8.4	20.4	21.1
100	12.5	29.4	23.4
300	24.8	48.4	45.7
500	36.4	65.4	62.0
1000	70.2	109.5	102.5

The released client on the branch nodes reproduces the baseline's figures; the branch client on the same nodes in the same hour is 1.6–2.9× faster. The speed-up follows the client read path, not the fleet or the day.