Skip to content

Latest commit

 

History

History
1643 lines (1557 loc) · 110 KB

TODO.md

File metadata and controls

1643 lines (1557 loc) · 110 KB

hyperdht-cpp — Master TODO / Worklist

Single source of truth for outstanding work. Everything to do lives here: the rest of the JS-parity sweep, missing parity features, hardening tasks, un-audited blind spots, and space for future sweeps. Update this file as work lands; keep the one-line status snapshot in CLAUDE.md pointing here.

Last updated: 2026-11-11 (bootstrap sample bounds).


Status snapshot

  • 2026-11-11 bootstrap sample bounds: no-ID replies contribute at most one observation per endpoint and at most max(1, configured bootstrap count) observations in total, matching JS. A verified firewall probe resets the source cache when it replaces the address sampler. Identified peers use their existing sampling rules.

  • 2026-11-11 bootstrap reply filtering: replies with a missing or invalid node ID now pass the node filter before contributing address observations, matching JS. The filter receives the sender's address-derived ID. RPC completion still occurs. Bootstrap sample deduplication and limits remain a separate item.

  • 2026-11-11 identified peer sampling: requests and replies now share the same 360-tick sampling interval. A newly admitted peer contributes its first sample on either socket. An existing peer contributes a sample only on the active socket when its sample is due. This matches JS _addNodeFromNetwork and _addNode. Bootstrap sampling remains a separate item.

  • 2026-11-10 valid-ID response node filters: rejected peers no longer contribute normal NAT/ring observations from responses, matching JS. One filter decision gates routing and sampling; the response callback and inflight completion still run. No-ID/invalid-ID bootstrap observations, response sampling age/socket policy and the separate firewall-probe sampler retain their behavior. No public API, layout, wrapper or wire changes.

  • 2026-11-10 incoming request sampling age: admitted peers now update NAT/ring observations from active-socket requests only when never sampled or after 360 ticks, matching JS's OLD_NODE gate. Newly admitted peers still sample on either socket; inactive requests retain the previous sample age. Response/bootstrap sampling policy is separate work. No API/wire change.

  • 2026-11-10 incoming request node filters: requests from a valid-ID peer rejected by filter_node no longer contribute NAT/ring observations on the active socket, matching JS. The filter is evaluated once before routing updates and request sampling; the request callback still runs. Accepted peers, public add_node_from_network and response/bootstrap sampling retain their behavior. No public API, layout, wrapper or wire changes.

  • 2026-11-09 native lookup query controls: additive C++ lookup overloads on HyperDHT and dht_ops expose copied seeds, request retries and onlyClosestNodes before dispatch, matching JS lookup options. Original calls retain generic retries and ordinary discovery. Both paths share record validation; C API, wrappers and layouts are unchanged.

  • 2026-11-08 native findPeer query controls: additive C++ overloads for find_peer and find_peer_target expose copied seed nodes, request retries and onlyClosestNodes before dispatch, using the existing worker. Original calls keep their defaults and hashing/validation semantics. C API, wrappers and layouts are unchanged; their option exposure remains separate work.

  • 2026-11-07 arbitrary-target unannounce: native C++ lookup_and_unannounce_target and unannounce_target use a supplied topic directly, sharing the existing signed LOOKUP/UNANNOUNCE path. Existing public-key methods still hash once. Local cleanup removes only the signing key's record; relay-only forwards are removed only for its own hashed topic, matching JS. Other keys, unrelated forwards and local listeners survive. Completion still awaits withdrawals; C API, wrappers and layouts are unchanged.

  • 2026-11-07 already-hashed findPeer target: named C++ HyperDHT::find_peer_target and dht_ops::find_peer_target use a supplied 32-byte target directly, matching JS's hash:false. Existing find_peer overloads still hash public keys and share the same record validation, routing replies, retries and seed handling. C API and wrappers are unchanged.

  • 2026-11-06 relayed OPEN-client socket: the server explicitly passes its active socket to the consumer for a relayed OPEN-client handshake, matching JS's dht.socket: client socket while firewalled, server socket when public. Direct handshakes retain their existing consumer fallback; propagation of their arrival socket remains separate work.

  • 2026-11-06 OPEN-client server endpoint: the server uses the client's endpoint observed by the handshake route, matching JS. The shortcut depends on OPEN alone, including an empty advertised address list. Sync and async firewall completion preserve the observed endpoint; NAT-client punching, relay bootstrap ordering, reply routing and consumer socket selection are unchanged; explicit socket propagation remains separate.

  • 2026-11-06 public-client passive wait: a client that advertised OPEN waits for the relayed server to reach it, matching JS's handshake snapshot. The NAT classifier remains the input for active punching; it cannot report OPEN and no longer decides passive waiting. Later firewall-state changes do not change the sent snapshot. Completion or raw-stream destruction stops the wait timer; the ten-second timeout and existing direct, NAT, relay and cached-route policies are preserved.

  • 2026-11-06 ordinary direct handshake (lab compatibility checked): a handshake reply without peerAddress completes on the main socket at the contacted endpoint, matching JS connectThroughNode before holepunch selection. This also covers servers advertising holepunch info or a non-OPEN firewall. Both cached relay discovery seeds and ordinary FIND_PEER use this path; cached socket-route and relayed-handshake behavior remain unchanged.

  • 2026-11-06 first inbound node observation: a valid node newly admitted to the routing table contributes its request's address observation even on the inactive socket, matching JS dht-rpc's _addNode. Repeated requests on that socket do not resample it. Invalid/missing IDs, filtered peers, self and full-bucket rejections retain the sampling exclusion. Existing expected- socket and response sampling policies are unchanged; age-based resampling remains separate work.

  • 2026-11-06 announcer pending periodic update: a periodic reannouncement waits for the current relay-ping round to settle, matching JS's awaited pings before its background update. Repeated deadlines coalesce; an explicit refresh satisfies the pending update, and stop cancels it. Reply and timeout settlement share this path. Explicit refresh/online/drift policy, intervals and watchdog are unchanged. Exact 100-round plus RTT cadence remains separate.

  • 2026-11-18 announcer ordinary refresh: refresh requests received during a relay-ping round now wait for all replies/timeouts, matching JS's awaited pings. Repeated and periodic requests coalesce, and stop cancels the request. Refresh during an update remains absorbed by that cycle; idle refresh, deliberate immediate drift recovery, online handling and watchdog are unchanged.

  • 2026-11-06 announcer periodic deadline: the existing five-minute reannounce timer restarts when a complete update settles, including retired withdrawals, instead of keeping its startup deadline after a manual refresh. Empty completed cycles restart that window too, matching JS's update-to-loop ordering. Ping cadence, watchdog and the five-minute interval are unchanged. This does not implement JS's exact 100-round plus RTT background-loop period.

  • 2026-11-06 announcer successful-cycle wake: a nonempty announcement cycle schedules the next relay-ping round immediately after all commits and retired-relay withdrawals settle, matching JS's update-to-ping ordering. Empty/failed cycles retain their existing timer schedule. If a ping round is already in flight, it retains the full pause after settlement instead of starting an extra round. Startup/resume and watchdog behavior remain; full background-loop timing parity is still separate work.

  • 2026-11-06 announcer cycle ordering: new relay keepalive rounds wait while the announcement cycle is updating, matching JS's awaited _runUpdate. The gate covers the walk, commits and retired-relay withdrawals; existing in-flight pings settle normally. Timer ticks still run the stuck-cycle watchdog. Startup/resume first pings and round spacing are unchanged; successful-cycle wake timing is covered above. Full background-loop timing parity remains separate work.

  • 2026-11-06 explicit connection discovery hints: nonempty ConnectOptions::relay_addresses seed the first FIND_PEER pass ahead of cached hints, matching JS connect.js:320-333. A persistent node's live forwarding address still overrides both. Empty options retain cache behavior; unsuccessful seeded passes retain normal discovery fallback. The existing option is copied for asynchronous route fallback; APIs, wrappers, retry budgets and cache updates are unchanged.

  • 2026-11-05 announcer first keepalive: startup and resume schedule the first relay ping round immediately after the first nonempty announcement publishes, matching JS start() awaiting the initial update before starting its background loop. Later rounds retain the three-second pause after all replies/timeouts settle. Reannounce and later refresh-cycle timing are unchanged; no public API, wrapper or wire-format changes are needed.

  • 2026-11-05 persistent forwarding hints: a persistent node's live router forwarding address replaces cached relay hints for the first FIND_PEER pass, matching JS connect.js:327-333. Ephemeral nodes, local handlers without a forwarding address and missing/expired/other-target entries retain the cache fallback. Normal discovery still follows an unsuccessful first pass. Cache, router expiry, query budgets and public APIs are unchanged.

  • 2026-11-05 cached connection discovery: cached relay addresses seed a FIND_PEER-only first pass with one retry and no table/closer-node expansion, matching JS connect.js:320-368. Handshakes start from valid peer replies. When that pass and its handshakes finish without a connection, normal discovery starts with three retries. Stale, empty and filtered hints retain fallback; shutdown cannot start another pass. Existing helper symbols and general query behavior are preserved. Persistent-router hint priority is covered above, along with explicit relay-address options.

  • 2026-11-05 connection discovery retries: the normal connection FIND_PEER walk uses three retries (four sends), matching JS connect.js:347, instead of inheriting the general query default of five retries. The original find_peer overloads retain their defaults; an additive overload sets the budget before the first send. Cached-relay seeding is covered above.

  • 2026-11-05 sampling target deduplication: each address-discovery campaign selects distinct host:port endpoints, matching JS nat.js's visited-node gate. A relay also present in the routing table no longer receives two PINGs or crowds out another target. Bootstrap fallback uses the same gate. The seven-target margin, selection order and cross-campaign policy are unchanged.

  • 2026-11-05 sampling ping retries: address-discovery PINGs now send once, matching JS nat.js retry: false. An additive PoolSocket request overload selects this behavior; the original overload and ordinary holepunch RPCs retain their four-send budget. Response timeouts, target selection, classification/early completion and campaign retry policy are unchanged.

  • 2026-11-05 pooled RPC retries: default PoolSocket requests now allow one initial send plus three retries (four total), matching JS dht-rpc's retries = 3. Previously the total was three. Retransmissions retain the same TID and encoded request; adaptive timeouts and overall connection deadlines are unchanged. The sampling override is covered above.

  • 2026-11-05 announcer keepalives: the periodic relay ping interval is now three seconds instead of five, matching JS's three-second idle delay. Ping rounds do not overlap; the three-second pause starts after every reply, timeout or queue rejection settles, matching JS's await-then-pause ordering. Timer ticks still run the cycle watchdog while a round is pending. The reannounce timer, watchdog threshold, drift handling and wire/API formats are unchanged; first-ping timing is covered above and later refresh-cycle timing remains separate work.

  • 2026-10-28 client reconnect hints: successful connections cache the server's advertised relayAddresses in preference to discovery responders, matching JS connect.js:464-468. Empty advertised lists retain the discovery fallback; failed connections leave existing hints unchanged. Cache timing, reconnect scheduling, socket reuse and the C ABI are unchanged.

  • 2026-10-28 public-server relay hints: successful public handshakes prepend the validated public endpoint to relayAddresses, matching JS announcer.js. NAT replies retain their relay-only hints; holepunch relay pairs and announced records are unchanged. This helps clients that prefer advertised hints. Client preference for the advertised list is covered above.

  • 2026-10-28 contextual error descriptions: additive C/JNI helpers describe query completions, immediate submission statuses and raw libuv errors separately. Kotlin storage failures use query messages; rejected calls, addNode and serverListen include the operation, description and unchanged numeric code. Existing connection messages/codes, callback signatures and wire behavior stay unchanged. Updated Kotlin sources require the matching JNI/core build.

  • 2026-10-28 immediate Kotlin/JNI errors: connect, immutable put/get, mutable put and ping check native submission status before awaiting callbacks. Rejected calls throw instead of hanging; accepted callbacks are unchanged. Both owned Kotlin copies match. Native code and the C ABI are unchanged; context-correct descriptions are covered by the separate item above.

  • 2026-10-27 read completion errors: immutable/mutable get results retain the existing query completion status, and C get/get_ex callbacks forward it. Explicit C cancellation keeps priority; ordinary misses remain successful. Kotlin immutableGet rejects a nonzero completion instead of returning null. No query, timeout, early-success or wire policy changes are introduced. C++ consumers must rebuild for the appended result fields; the C ABI is unchanged. Immediate JNI returns and general error descriptions remain open.

  • 2026-10-27 C/Kotlin connection errors: C++ terminal server-handshake failures map to the additive C/Kotlin SERVER_ERROR code (-9), with a native description. Existing C++ codes and C query cancellation (-8) are preserved; server errors were previously mislabeled as cancellation by C consumers. All C connect entry points use the mapping; JNI forwards the C result. Other query/storage errors and synchronous JNI status handling remain open.

  • 2026-10-27 direct socket results: OPEN, no-holepunch and LAN results report the main UDX socket already selected by consumers' fallback. This enables the existing route cache when both sides opt in. No socket ownership or C ABI layout change is introduced. A non-relayed cached main route completes directly after its handshake, matching JS instead of entering holepunching; actual firewall observations retain priority and relay setup retains its order. Pooled-socket cached retry remains separate work.

  • 2026-10-27 cache insertion: retain the first live route for each peer, matching JS socket-pool.js. Later successful sockets are marked reusable without replacing that route; removal or socket closure permits a fresh one. Existing stream-error ownership guards and cached transport selection remain.

  • 2026-10-27 stream-error routes: DHT duplex options observe failed streams before consumer close callbacks and invalidate only the matching peer/stream route. Normal close, healthy peers sharing a socket, newer route replacements, and callbacks after DHT destruction are protected. C++ consumers should use make_duplex_options(); the C wrapper already does. C ABI signatures stay the same; C++ consumers must rebuild for the appended option/cache metadata. This does not change cache replacement policy or pooled-socket selection.

  • 2026-10-27 cached-route replies: authenticated terminal handshake errors stop the connection with SERVER_ERROR before findPeer, matching JS's cached route path and the existing normal C++ handshake path. Malformed outer replies and corrupt Noise replies retain lookup fallback. Other socket-reuse and recovery items remain open.

  • 2026-10-26 forced-persistent startup: generic ephemeral=false starts ephemeral and awaits the existing address/firewall check during bootstrap, matching JS _forcePersistent. Success adopts the verified endpoint's routing ID before ready; failure stays ephemeral. The explicit bootstrapper factory retains its trusted-address initialization. The full two-pass bootstrap loop remains deferred; this change does not add it.

  • 2026-10-26 storage put results: immutable/mutable put results expose the existing query commit error, and the C callbacks forward that status instead of always reporting success. Callback signatures and the C ABI stay unchanged; C++ consumers must rebuild for the appended result field. This matches JS puts awaiting query.finished(); transport-level replies retain the existing JS success policy. Other query/storage gaps remain open.

  • 2026-10-25 lookup results: lookup and lookup-and-unannounce validate the complete peer array and optional bump before forwarding raw reply bytes or deciding to withdraw, matching JS mapLookup. Closest routing replies and the original decoder entry point remain available. Query/storage failure reporting remains open.

  • 2026-10-25 findPeer results: the shared findPeer callback path decodes peer records before forwarding raw reply bytes, matching JS mapFindPeer's rejection of codec failures. Routing/closest replies remain unfiltered, so missing records still supply announcement relays and tokens. The existing unchecked decoder remains available; the checked overload reports failure. Lookup decoding is covered above; query/storage failure reporting remains open.

  • 2026-10-25 graceful shutdown: Announcer, Server, HyperDHT and C API completion now wait for pending announcements and withdrawals to settle. Force cancels the wait; callers still retain storage through libuv's close drain. Nospoon/Holesail teardown and bounded Rust cleanup follow that contract. The filtered core suite (795 pass, two skips), twenty focused sanitizer cases, consumer suites, both static CPU builds and the ESP32 component compile pass. Sixteen selected Rust tests pass; the separate pre-existing drop_midflight test has a stale API call and was not repaired. Fifteen isolated live cases pass: withdrawal success/timeout, public/NAT/LAN consumer connectivity and 33 clean native exits. ESP32 firmware/runtime and broader platform checks remain unverified.

  • 2026-10-25 retired relays: a successful replacement cycle withdraws from relays it no longer uses, then allows the next refresh. Failed empty cycles retain the working set; force-stop cancels a pending withdrawal. Nine focused checks pass with ASan/UBSan and leak checking. Core and consumer suites, both static CPU builds and the ESP32 component compile pass. Fifteen isolated live cases pass, including replacement/failed-replacement controls, encrypted echo, fresh Nospoon/Holesail connectivity and 35 clean native app exits. Awaited whole-DHT shutdown is covered above.

  • 2026-10-25 withdrawal tokens: stop-time withdrawal obtains a fresh FIND_PEER token and replying node ID before signing UNANNOUNCE. Copied signing state survives Announcer release; restarting cancels an old pending withdrawal before it can erase the resumed record. Seven focused regressions pass with ASan/UBSan and leak checking. Core and consumer suites, both static CPU builds, the ESP32 component compile and fourteen isolated live cases pass, including 31 clean native app exits. Retired-relay cleanup is now covered above; awaited whole-DHT withdrawal is covered above too.

  • 2026-10-24 UDX shutdown: close now tears down attached and unconnected streams before their DHT storage is released. Generated UDX guards avoid duplicate closes and watcher callback lifetime errors; pending connect owners and pool retries stop referring to finalized handles. Core and consumer suites, both static CPU builds, ten Holesail TCP/UDP placements and three Nospoon public/NAT/LAN placements pass, including normal application exits. TD2 is fixed; TD1/TD3 and broader platform/soak verification remain open.

  • 2026-10-24 announcement identity: ordinary stored announcements now replace and remove by verified peer public key within each target, including refresh and local clear. Signed UDP storage and public/NAT consumer checks pass; see the announcement identity entry below.

  • 2026-10-24 relay verification: blind-relay pairing, OPEN control sockets, transport-write failure propagation and handshake dedup are fixed. Live relay-to-direct migration passes with both a public and a NAT application server, preserving encrypted payload and the original streams; see Section J.1.

  • 2026-10-13 documentation check: the protomux reentrancy UAF is fixed (3421264), and the client relay-address cache is implemented (c196205); their entries below now reflect current source. Relay-to-direct upgrade code is present on both client and server, but blind-relay pairing remains blocked by BUG A (Section J.1), and the live upgrade test remains open. This check does not revalidate every historical finding or test result.

  • 2026-07-31: announcer relay-set freeze FIXED (6ef85e6 gate split + 82d3c55 per-cycle rebuild, LOCAL, NOT PUSHED). The published relay set was write-once — the ANNOUNCE callback could only refresh an entry it already had or append while under PICK_BEST, and active_relays_ was otherwise cleared only on stop — so a dead or third-party-unreachable relay was advertised for the life of the process and no healthier node could replace it. That is the mechanism behind every "a restart fixed it" in the field handoff. JS rebuilds per _update (announcer.js:172,188). Fix = stage into cycle_relays_, swap in maybe_publish() guarded against empty; prerequisite gate split so a client holding a previous-generation record still gets a holepunch token (JS unions three _serverRelays generations, announcer.js:38-42). Also fixed a pre-existing bug found while tracing: start() never re-armed the alive_ sentinel stop_impl() clears, so a suspended-then-resumed server never re-announced again. 730/730, ASAN clean, cpp-reviewer SHIP on both. Six tests, all red-checked; the guard is IsRelayStillAcceptsPreviousGenerationAddress.

  • 2026-07-30 (later): field Finding M FIXED — Round 1 now fails over to the remaining announce relays instead of treating one dead relay as terminal. Deliberately beyond-JS (it is the fix JS's own TODO asks for), capped at PICK_BEST - 1 legs after cpp-reviewer flagged the 128-entry wire cap as a retry-amplification stall. 722/722, ASAN clean, SHIP. Round-2 failover (JS's second TODO) still open — needs field evidence.

  • 2026-07-30: field Finding M logged as the new P0 (Section A) — one unreachable relay in the announce is a permanent unrecoverable -5 because neither implementation fails over to relays[1..n]. 81/81 repro, reproduced on nospoon-JS too, and upstream JS carries the TODO asking for the fix in two places. NOT a parity bug — the fix is a deliberate beyond-JS improvement.

  • 2026-07-29 (later): field Finding J FIXED — udx_stream_destroy has no UDX_STREAM_DESTROYING guard, so a second destroy inside the deferred-close window ran close_stream_internal() twice and aborted in libuv. Now guarded by udx::destroy_stream_once(). Reproduced standalone and fixed; 719/719. Surfaced a separate pre-existing UAF in protomux.cpp:242 (logged below).

  • 2026-07-29: field Finding I FIXED — the advertised local-address list was a bind-time snapshot, permanently breaking same-LAN connects (-6) after any interface change; now re-enumerated live per advertisement (HyperDHT::local_addresses_now()), matching JS _localAddresses(). Also recorded new field Findings J (HIGH, uv_close double-close crash) and K (sampling gate), and DEMOTED B1 — Finding L exonerates it as the -5 cause, promoting B2. Suite 718/718.

  • 2026-07-28: C-API SECURITY fix — hyperdht_server_set_firewall polarity was inverted, so every C-API server with a firewall admitted exactly the peers it should reject (0203820, found via holesail-cpp; regression tests ServerFfiFirewall.* red-checked). Raises the priority of the un-audited ffi_* layer (Section F). Also landed read-side backpressure (hyperdht_stream_pause/resume) + the reusable_socket connect option (a08cade, Section D). Suite 716/716 unit.

  • Adversarial re-verification 2026-07-26 (16-subsystem Opus workflow): claimed status CONFIRMED accurate — zero FALSE_DONE, all 12 HIGH genuinely closed, all 6 claimed-open genuinely open, ~93% behavioral parity. Added 1 HIGH new hazard (server/divergence-1 UAF, NOT fixed), 3 LOW divergences + io-6 DONE→PARTIAL, re-tagged 6 stale-OPEN appendix entries as DONE, reframed B1 as beyond-JS (not a parity bug). Detail in Section A. messages + protomux were UNVERIFIED this pass (finder output-cap failures).

  • JS-parity sweep 2026-07-09: 91 confirmed findings (12 HIGH / 45 MED / 34 LOW). All 12 HIGH closed. ~86/91 addressed (fixed or accepted-divergence) (io-6 demoted to PARTIAL 2026-07-26 → ~85/91). 6 MED/LOW still open (Section A) + 4 new 2026-07-26 findings.

  • 2026-07-22 batch (on main): server-1 (firewall-reject now SILENT — presence leak closed), server-2 (holepunch reply deferred past veto+punch(), encrypted ABORTED on failure), server-3 (synchronous handshake dedup + duplicate queueing), connect-3 (reusableSocket client wiring), connect-4 (route-shortcut payload), connect-5 (bogon filter + serverAddress fallback), connect-6 (LAN shortcut exclusive, JS trigger via client_address), connect-7 (client holepunch veto), announce-4 (record signs empty relayAddresses — client independence proven both sides), announce-6 (router FROM_RELAY-only handler gate). Suite 702/702; cpp-reviewer SHIP, ASAN/UBSAN clean.

  • 2026-07-21 batch (on main): announce-5, announce-7, server-8, connect-8 (SECURITY: handshake reply validation + new Response::remote_addr source check — RPC matched responses by tid only), connect-10 (client relay-chain teardown; also broke a shared_ptr cycle that leaked ConnState on dead pairings), birthday-win SocketRef keepalive pin (holepunch.cpp:1450 residual). Suite 684/684 (excl. live); cpp-reviewer SHIP, ASAN/UBSAN clean.

  • Prior sweep commits (merged to main 2026-07-11): f266b35 deps bump, 88b8022 compact/routing, 5460cfa core sweep + relay→direct upgrade, f37ef1d wrappers, 86353db docs.

  • Detail docs: docs/PARITY-SWEEP-2026-07-09.md (FREEZE table + per-subsystem notes + REMAINING OPEN), docs/.parity-sweep-appendix.md (all 91 with JS/C++ file:line), docs/RELAY-UPGRADE-PORT.md (#266 port design).

Freeze-ready (fully swept): noise, compact, routing, tokens-nat, messages, blind-relay, dhtrpc-io, protomux, query, dhtrpc-tick, dht-top. Partial (HIGHs only): connect, server, router-announce, secret-stream.


A. Open JS-parity findings (6 prior + 4 new 2026-07-26) — the resume worklist

Remaining: connect-9, server-4 (ACCEPT candidate — likely just document), server-6/9/11, secret-stream connect-1 (deferred, needs live JS cross-test). Full text + JS/C++ file:line for each is in docs/.parity-sweep-appendix.md.

2026-07-26 adversarial re-verification (workflow) — NEW findings

Full run: 14/16 subsystems (messages + protomux finders failed on output-cap, UNVERIFIED this pass), 11 material findings, ALL skeptic-confirmed (high conf). The 12 HIGH and all 6 claimed-open items above survived independently — zero FALSE_DONE. Behavioral parity estimate ~93%. Net-new, none previously ledgered:

  • server/divergence-1 (HIGH — potential UAF) — fixed 2026-10-23 (e6adc4c). The OPEN-client direct shortcut returns before starting blind relay setup, preventing a relay continuation from reusing the emitted stream. The regression requires one direct emission and no relay pairing; all three suites and isolated public/NAT connectivity checks pass.
  • connect/divergence-1 (LOW) — fixed 2026-10-27. The cached-route callback applies the normal handshake's hs.terminal check and reports SERVER_ERROR without a findPeer walk. Previously the regression returned PEER_CONNECTION_FAILED after a lookup; retryable replies still fall back.
  • connect/divergence-2 normal direct handshake — rechecked and fixed 2026-11-06 (lab compatibility checked). JS connect.js:499-501 completes non-relayed handshakes before its holepunch function; gating the no-holepunch branch alone would miss that control flow. Ordinary C++ discovery now uses the existing main-socket direct completion too, rather than punching an already reached server or preferring advertised addresses. Cached-route and relayed paths are preserved.
  • compact/buffer64k (LOW — doc-only) — Buffer::decode 64KB cap (compact.cpp:239-249) vs JS uncapped; fails SAFE (rejects a superset), H12 anti-DoS. Add a ledger line like compact-2's array cap. No code change.
  • dhtrpc-io-6 first external request — fixed 2026-11-06: after a valid new node is admitted, its first request feeds the address samplers even on the inactive socket, matching JS _addNode (index.js:533-536). Known peers on that socket remain excluded, as do rejected nodes. Existing expected- socket sampling and separate age-based resampling work are unchanged.

Docs caught up (code was AHEAD): 6 appendix entries were labeled OPEN but are FIXED + skeptic-confirmed — now tagged [DONE — verified 2026-07-26] in docs/.parity-sweep-appendix.md: holepuncher-1 (birthday wired; symmetric-CGNAT stream completion still live-unvalidated), query-2, dhttop-1, dhttop-2, dhttop-6, dhttop-8. Headline 86/91 unaffected (already excluded them).

connect (1 open; done: connect-1..8/10/11)

  • connect-8 — DONE 2026-07-21. Handshake reply now validated (mode==REPLY, source-address match via Response::remote_addr, version/ error/udx checks); JS-terminal failures fail the connect with ConnectError::SERVER_ERROR instead of retrying relays.
  • connect-10 — DONE 2026-07-21. 15s timer now tears down the relay chain (abort_relay_chain), breaking the pair-callback↔ConnState shared_ptr cycle; deferred teardown from client-callback frames via 0ms re-arm.
  • connect-3 — DONE 2026-07-22. reusableSocket threaded: ConnState + UdxInfo advertise flag + route-cache read/write gated on both-sides opt-in.
  • connect-4 — DONE 2026-07-22. build_local_handshake_info shared by fire_handshake and the route shortcut (one payload for all attempts).
  • connect-5 — DONE 2026-07-22. is_bogon/is_reserved (npm bogon port); direct paths use first-non-bogon + serverAddress fallback.
  • connect-6 — DONE 2026-07-22. LAN shortcut EXCLUSIVE of holepunch, JS trigger (client_address.host == server_address.host, onlyNonReserved filter, ping-fail aborts). NEEDS LIVE LAN VALIDATION (Section C).
  • connect-7 — DONE 2026-07-22. opts.holepunch veto invoked after probe round, before punching; abort maps to HOLEPUNCH_TIMEOUT. Deviation: abort is local-only (no ABORTED round to the relay — see Section G).
  • connect-9 normal walk retry budget — three retries, matching JS connect.js:347 (2026-11-05). General findPeer queries retain five retries.
  • connect-9 cached relay first pass — cached-only FIND_PEER with one retry, followed by a normal walk after the query and pending handshakes settle (2026-11-05). Query filtering, cancellation and normal-walk controls covered.
  • connect-9 persistent-router hint priority — live forwarding addresses override cached hints on persistent nodes (2026-11-05).
  • connect-9 explicit relay-address seeds — honor the existing connection option ahead of cache, retaining persistent-forward priority (2026-11-06).

server (4 open; done: server-1/2/3/5/8)

  • server-1 — DONE 2026-07-22. Firewall-rejected handshake now sends NOTHING (presence leak closed); rejected session stored only for dedup-silence, reaped by the clear-wait timer.
  • server-2 — DONE 2026-07-22. Holepunch reply committed only after veto + punch()-started; failures send encrypted ABORTED (encode_abort_reply). Deviation kept: immediate clear_session vs JS ~10s defer (Section G).
  • server-3 — DONE 2026-07-22. Dedup entry written same-tick before the async firewall dispatch; duplicates queue on pending_handshakes_ and all get the same reply (or silence) on resolve.
  • server-4 — MAX_PENDING_HANDSHAKES=256 cap silently drops. ACCEPT candidate (anti-DoS, like the other caps we kept) — likely just document.
  • server-6 — neverPunch (opts.holepunch === false) not implemented.
  • server-8 — DONE 2026-07-21. 15s pairing watchdog (relay_pending_ + abort_relay): tears down the relay chain only, session + puncher keep running (JS onabort parity); pair-error no longer clears the session.
  • server-9 — server-side same-host LAN match (server.js:414-426) not implemented.
  • server-11 endpoint — OPEN-client shortcut uses the observed client endpoint regardless of advertised addresses (2026-11-06).
  • server-11 relayed socket — pass the active socket for a relayed OPEN-client handshake (2026-11-06).
  • server-11 direct socket — propagate JS's arrival socket on a direct handshake. Direct handshakes retain the existing null socket and consumer fallback.

router-announce (0 open; done: announce-1..7)

  • announce-4 — DONE 2026-07-22. Announcer signs/stores empty relayAddresses (JS parity); proven no client (JS or C++) reads relays from the record — they come from the responding DHT node + the handshake payload. Dead re-announce-once machinery removed.
  • announce-5 — DONE 2026-07-21. Server-host FROM_SECOND_RELAY reply now routed to the embedded relayAddress (first relay), dropped when absent.
  • announce-6 — DONE 2026-07-22. Holepunch handler now FROM_RELAY-only with the !peerAddress drop (router.js:221); FROM_CLIENT/FROM_SERVER route through the pure-relay path (self-hosting server reaches its own handler via one self-hop, like JS).
  • announce-7 — DONE 2026-07-21. handle_refresh ports persistent.js _onrefresh: refresh hashes stored on full announce, preimage verified, record re-added, chain rotated. (Latent feature — current JS always sends refresh:null.)

2026-07-27 field findings (handoff Findings I/J/K/L)

  • Finding I — server advertised a BIND-TIME local-address snapshot. DONE 2026-07-29. validated_local_addresses_ was assigned only in bind() (dht.cpp) and never recomputed, so an interface appearing later (DHCP completing after an early bind, interface flap) was never advertised for the process lifetime → same-LAN clients failed permanently with -6. Fatal rather than degraded because the LAN shortcut is EXCLUSIVE: a bad addresses4 removes the holepunch fallback too (that exclusivity is correct parity — connect-6 — so the fix is the address list, NOT the fallback). Genuine JS divergence: _localAddresses() (server.js:206-208) re-enumerates per handshake (server.js:272). Fix: new HyperDHT::local_addresses_now() — live enumerate → validate_local_addresses() → refresh snapshot; used by the server handshake reply and the client's build_local_handshake_info (memoized per connect() on ConnState, matching JS's one-payload-per-connect latch, connect.js:386). VERIFIED with a netns repro (interface brought up after bind: before the fix the advertised list stayed 127.0.0.1 even after the interface watcher polled; after, it tracks the live address). Tests LocalAddressesNowMatchesLiveEnumeration + LocalAddressesNowRespectsExcludeLocalAddress (the latter red-checked against a naive bypass — live re-enumeration MUST stay routed through validate_local_addresses() or it resurrects exclude_local_address()'d hosts, which nospoon relies on to hide its TUN address). 718/718, cpp-reviewer SHIP, ASAN clean.
  • Finding J (HIGH — hard crash) — DONE 2026-07-29. uv_close double-close abort in ~ConnState teardown. ROOT CAUSE: udx_stream_destroy (deps/libudx/src/udx.c:2709) guards on UDX_STREAM_CLOSED but NEVER on UDX_STREAM_DESTROYING. On the slow send path (uv_udp_try_send → UV_EAGAIN, which happens whenever ANY other send is queued on that socket — common on the busy shared DHT-RPC socket) close_stream_internal() is deferred to the send completion, leaving the stream DESTROYING-but-not-CLOSED and still CONNECTED for a loop turn. A second destroy in that window queues a second destroy packet and runs close_stream_internal() TWICE, which uv_close()s the stream's 5 timers twice → libuv assert(!uv__is_closing(handle)). libudx's own assert(CLOSED == 0) is compiled out in Release — exactly where the field crash was seen. Fix: hyperdht::udx::destroy_stream_once() (include/hyperdht/udx.hpp) skips when DESTROYING|CLOSED; applied to ~ConnState and ServerConnection's dtor + operator=. Skipping is always correct — such a stream already finalizes itself. INDEPENDENTLY REPRODUCED by cpp-reviewer with a standalone program against the vendored libudx (forced UV_EAGAIN, double destroy → assert in Debug, SIGSEGV in Release; guarded → clean, finalize_count == 1). Test DestroyStreamOnceSkipsStreamAlreadyBeingDestroyed (red-checked). 719/719, cpp-reviewer SHIP. NOTE: blind_relay.cpp's destroy sites are NOT exposed — for a relayed stream udx_stream_destroy calls close_stream_internal synchronously in one call (upstream fa576bd made it so deliberately), so there is no window.
  • Finding J follow-up — DONE 2026-10-24. The generated UDX copy now guards DESTROYING at udx_stream_destroy() itself, covering SecretStreamDuplex::destroy() and bulk teardown without changing the pending close callback that delivers on_udx_close → fire_close.
  • protomux reentrancy UAF — FIXED (3421264; source rechecked 2026-10-13). Channel::open() retains itself with shared_from_this() across synchronous transport callbacks; Mux channel ownership is shared. This closes the reported OPEN/reentrant REJECT lifetime hazard, not the separate server/divergence-1 finding above.
  • Finding K — the pool-NAT sampling gate does not hold: pool_fw is read mid-flight at holepunch.cpp:1758, while the comment at holepunch.cpp:1777-1779 asserts sampling has completed by then. Field log shows Sampling settled firing 1 ms BEFORE sampling done, with disagreeing values (read fw=3, settled fw=2); ordering varies run to run. Effect on connect outcome NOT established (n=1 each way — do not assume a direction). Minimum: enforce the invariant or correct the comment.
  • Finding M — DONE 2026-07-30 (round 1 only; round 2 deliberately left alone, see below). holepunch::pick_fallback_relays() hands the announce record minus the chosen relay to holepunch_connect(..., fallback_relays); fail_round1() pops the next leg and re-runs run_round1 on the SAME pool socket, puncher and NAT samples — only the relay changes. Terminal only once the list empties, so the single-relay path is unchanged. Scope matched to JS: the six Round-1 sites that JS reaches by throwing out of probeRound into the retry catch now fail over (request timeout, bad reply, empty payload, decrypt failure, remote error, no server address); everything JS reaches via abort() (double-random NATs, unstable NAT, probe exhaustion) stays terminal. Round 1's two addresses had to be split: the message's peer_address is the relay's forward destination (router.js:214 uses it verbatim) and moves with the relay, while the payload's remote_address stays the fresh observation the server matches for its fast-mode punch (server.js:530-538). Attempt 0 seeds both from the same value, so it is byte-identical to before. SECURITY (cpp-reviewer): the failover list is capped at announcer::PICK_BEST - 1, NOT the wire cap of 128 relays (peer_connect.cpp:175) — that bound was sized for parse safety back when extra entries were ignored, and each entry now costs a multi-second RPC, so uncapped it let a hostile connect() target stall its caller for ~6 minutes with garbage address bytes. Honest servers announce ≤3 (pickBest), so the cap costs no real connectivity. Tests: ConnectRelayFailover.{DeadRelayFallsOverToTheNextRelay, NoFallbackRelaysStaysSingleAttempt, FallbackListSkipsTheChosenRelayAndIsCapped} (first one red-checked). 722/722 unit; ASAN/UBSAN clean; cpp-reviewer SHIP. STILL OPEN: JS's second TODO (connect.js:312, round-2 failure). C++ already treats a round-2 relay timeout as non-fatal because probing is under way by then, so the remaining gap is only a round-2 error reply; failing that over means re-running round 1 for a fresh token. Needs field evidence first. Original report follows.
  • Finding M (2026-07-30, P0 — best-evidenced item in the handoff) — ONE unreachable relay in the announce = permanent -5, with no failover to the other two. 81/81 deterministic field repro (81 × round-1 TIMEOUT to the same relay, 0 successes) while six other pool nodes queried in the same millisecond all replied — so not a network block. ROOT CAUSE: the chosen relay sits behind an endpoint-dependent (symmetric) NAT, so the server's mapping for it (:58044, in the announce record) is useless to a third party; the client's own observation of the same host is a different port (:45262). The server's 5 s keepalives ride the server's own mapping, so the entry looks healthy from the server — the announcer cannot detect this; no server-side liveness check helps. NOT a C++ bug and NOT a parity divergence — cross-implementation control: nospoon-JS on the same machine/network/seed fails identically (HOLEPUNCH_ABORTED ×6). VERIFIED IN BOTH SOURCES 2026-07-30: selection is faithful parity (connect.cpp:781-790 vs pickServerRelay connect.js:812-817 — host+port match else relays[0]), and the bail-out is terminal on both (holepunch.cpp:2059-2062 state->complete({})). Upstream JS asks for this fix in its own source, twice (verified verbatim): connect.js:271 // TODO: we should retry here with some of the other relays, bail for now and connect.js:312 // TODO: retry with another relay?. FIX: on round-1 (and round-2) failure, fail over to relays[1..n] instead of completing terminally. Deliberately BEYOND-JS, but it is the fix JS itself requests. REJECTED alternative — do NOT substitute the client's own observation of the same host (:45262). relays[i] is not an address, it is a live server↔relay UDP session, and probeRound sends (peerAddress, relayAddress) as a unit (connect.js:561). JS documents why at connect.js:280-285: "If the relays were different, then the server would not have a UDP session open on this address to the client relay, which round2 uses." Port-swapping trades a round-1 timeout for a round-2 failure. Failover is safe precisely because each relays[i] carries its own peer_address and its own server-maintained session. Operational note: nothing client-side recovers; the server must re-announce (restart) to draw a fresh relay set.

Finding O (2026-07-30 handoff) — audited 2026-07-31, mechanism REFRAMED

  • The real defect was the announcer relay-set FREEZE — fixed 2026-07-31 (6ef85e6, 82d3c55; see the status snapshot). A bad relay roll persisted for the process lifetime because nothing could ever replace it, which is why restarting the server "fixed" it. The controlled A/B in the handoff (run 1: three slots on one symmetric-NAT host, total failure; run 2 after restart: three distinct hosts, instant success) is explained by the re-roll, not by diversity per se.
  • PARKED by decision 2026-08-03 (Vaios). Revisit in the mid-term, or sooner if upstream changes pickBest. Rationale: it is local selection policy, invisible on the wire, so deferring costs nothing in interoperability; and the announcer rebuild (82d3c55) already removed the permanence that made a bad roll fatal. Wake-up triggers, in order: (a) upstream implements the pickBest RTT TODO — note RTT alone would make this WORSE, since all mappings of one host share an RTT and would then win all three slots deterministically rather than by luck; (b) a single-host relay set recurs in the field post-rebuild; (c) a -5 outage that survives a relay-set re-roll. On revisit, check the statistical premise FIRST (see the note below) — if three mappings of one host really are the global closest-3 more than rarely, the selection/sort is the bug and a diversity constraint would only mask it.
  • Host diversity itself — OPEN, and NOT a parity bug. The audit (2026-07-31 workflow) refuted the parity framing: JS pickBest is replies.slice(0, 3) (announcer.js:298-301) with no host notion either, so stock JS has the identical gap. Any distinct-host constraint on the announce commit is a deliberate BEYOND-JS hardening change. Hold until the rebuild has field time: the freeze made every bad roll permanent, so re-measure how often a single-host set actually recurs before adding a constraint that could reduce availability in small/clustered DHT regions. Note the premise is also unexplained — id = BLAKE2b(host, port) makes each mapping an independent random point, so three mappings of one host being the global closest-3 is suspicious in itself; if it recurs, suspect the selection/sort, not diversity.
  • Announcement record identity uses verified public keys (2026-10-24). Within each target, full announce and refresh replace the same key after an endpoint change; different keys sharing an endpoint coexist. Signed unannounce removes only its verified key, even from a different endpoint. Self-announce cleanup and local lookup-and-unannounce also remove by key. The stored sender address remains metadata; wire records, C ABI, TTL and capacity are unchanged. Regressions cover replacement, coexistence, key isolation, refresh, local clear, expiry and capacity. A private persistent C++ node verifies six UDP storage operations with matching request/reply payloads in both endpoint captures. All three suites and Nospoon connectivity checks pass with a public server and a NAT server; x86_64/aarch64 static builds and all 35 ESP32-S3 component files compile. Firmware/board checks remain deferred. Low-level C++ store users must rebuild and supply the peer key; the C interface is unchanged.
  • Router forward cache bounded and expired (2026-10-23). Remote forwards use the existing LRU cache with DhtOptions::max_size and max_age_ms; defaults remain 65536 entries and 20 minutes. Expiry applies on lookup and periodic GC; refresh resets age. Local listening Server handlers stay separate so remote announcements cannot replace or evict them. Focused regressions cover capacity, expiry, refresh, signed announcements and listener retention. The full filtered unit suite passes in Debug and Release; consumer and isolated connectivity checks accompany this security change.
  • Firewall-probe address snapshot (2026-11-13). Before promotion or sampler replacement, require the client sampler's host/port to match the probe-start snapshot and the node to remain ephemeral (JS index.js:827). A successful probe still confirms reachability; changed-address samples survive for a later evaluation. Direct C++ RpcSocket consumers must rebuild.
  • Persistence host memo (2026-11-13). Remember the checked host even when its address is incomplete, matching JS index.js:805-815. The countdown then uses that observation after host/port discovery is restored.
  • RECHECK: the persistence gate is one-shot and cannot demote (context for Finding O and for the item above; no action decided yet). Becoming a relay requires passing the PING_NAT firewall probe once — five routing-table nodes reply to our server socket, plus the port-preservation check. That proves those five could reach that port at that moment; it does NOT establish endpoint-independence, and nothing ever re-runs it to demote (same promote-but-never-decrement property the B1 audit found in JS nat.js _updateFirewall). When a NAT mapping later moves, the node re-registers under a NEW id, because id = BLAKE2b(host, port) — so each remap enters the routing table as a fresh, legitimately-persistent node. That is exactly how 82.29.173.47 appeared under nine distinct ports in one day and how one physical host could occupy multiple announce slots (Finding O). Any host-diversity work should be designed against this, not against the naive "one node = one entry" model.
  • Fresh-token stop-time UNANNOUNCE — fixed 2026-10-25. FIND_PEER obtains the current token and node ID before signing UNANNOUNCE, matching JS's two-request _unannounce (announcer.js:205-238). Reject incomplete/error replies; preserve copied signing state after Announcer destruction; cancel old withdrawal continuations on restart. Signed-handler regressions and a live aged-token negative/control prove the record is actually removed.
  • Per-cycle UNANNOUNCE — fixed 2026-10-25 (announcer.js:191-196). Withdraw relays absent from the replacement set with fresh-token requests; await their settlement before another cycle. Retain the old set when a cycle commits nothing, and preserve the lenient relay gate. Regressions and isolated live positive/failed-replacement controls cover removal, retention and force cancellation. An unreachable relay still expires its record normally.
  • Await graceful withdrawal on whole-DHT destruction — fixed 2026-10-25 (JS index.js:123-127, server.js:88-129, announcer.js:88-104). Completion waits for pending ANNOUNCE commits, per-cycle retirement and stop-time withdrawal before closing transport. Force cancels the wait. Completion still precedes libuv close draining; callers retain storage for that drain. Regression, sanitizer and isolated live success/timeout controls pass, alongside rebuilt Nospoon/Holesail public/NAT/LAN connectivity and clean shutdown checks.
  • Keepalive/update ordering — new ping rounds wait for cycle settlement, matching JS's awaited update; watchdog ticks remain active (2026-11-06). Periodic updates also wait for in-flight ping settlement; ordinary refreshes share that wait (2026-11-18). Successful-cycle wake timing and remaining exact background-loop cadence are covered above.
  • Refresh during keepalive — coalesced until round settlement (2026-11-18). This does not add a mid-update latch: JS clears _refreshing after its update before starting the next background round. Deliberate immediate drift recovery remains separate.

nat-sampler / punch payload (from the 2026-07-22 field diagnosis; see

docs/FIELD-DIAGNOSIS-2026-07-22.md "Finding B" + "Finding E")

  • Finding E — DONE 2026-07-22 (uncommitted): announcer updating_ latch → server stopped reannouncing. Root leak: Query::visit() didn't settle a tid==0 (congestion/closing) request-drop → walk never completed → cycle wedged. Fixed the walk drop-settle + added a 60s stuck-cycle watchdog on the ping timer. Test RecoversFromWedgedCycleViaWatchdog (red-checked).
  • Finding H (pickBest) — DONE 2026-07-26 (announcer.cpp update(); see handoff Finding H, both-ends field capture + code-confirmed). The announce record was committed to EVERY walked node (~41) instead of JS pickBest(q.closestReplies) = closest 3 (announcer.js:170,298-301). The ~38 extra record-holders had EXPIRED server-forward NAT mappings (only the 3 kept-alive active_relays_ forward), so a client hunted dead relays ~28s before hitting a live one. Fix: commit only the closest PICK_BEST(3) replies in on_done (full closest set still saved for reseed). Pure JS- parity restoration. Test CommitsToPickBestThreeNotAllClosest; 709/709 unit + cpp-reviewer SHIP + ASAN clean. NEEDS LIVE VALIDATION (Section C): mobile connect ~28s → ~1s. Upstream of the punch; independent of B1/B2.
  • B1 (BEYOND-JS field-hardening — NOT a parity bug) — NatSampler classifies at sampled_ >= 3 (nat_sampler.cpp:111); MIN_SAMPLES=4 feeds an ok flag nobody reads. Three agreeing samples latch CONSISTENT and cannot be demoted. 2026-07-26 re-verification CORRECTION: this latch is JS-IDENTICAL — nat.js _updateFirewall max-hits only ever grow, never decrement — so stock JS fails the same CGNAT case. B1 is therefore a DELIBERATE divergence, not a parity gap. Fix = gate the verdict on >=4 + let disagreeing samples demote a stale CONSISTENT, to engage the wired birthday strategy on port-varying CGNAT. 2026-07-27 DEMOTED — field Finding L EXONERATES B1 as the -5 cause. The "reasonable candidate for the live -5" reading is WITHDRAWN. Mobile capture: all 6 SUCCESSES logged pool_fw=2 — B1's exact misdeclaration, with the client mapping equally unstable — so B1's precondition is present in the successes; and the one attempt that classified CORRECTLY (pool_fw=3 RANDOM, i.e. what fixing B1 produces) still failed -5. B1 discriminates in neither direction. Keep it as a remedy (a client whose mapping moves mid-handshake wants the birthday fallback), NOT as the cause. B2 is now the priority of the pair.
  • B2 — DONE 2026-08-02 (f6608b7, PUSHED). Round 2 now sends pool->addresses() (the sampler set) like round 1, with the relay's observation kept only as the last-resort fallback. Deliberately does NOT inherit round 1's ctx.local_addresses fallback: those are RFC1918 (holepunch::local_addresses only filters libuv's is_internal = loopback/link-local), JS never puts LAN addresses in this field (nat.js:139 → addresses = null when UNKNOWN, and the encoder omits it), and preferring them would advertise ONLY private addresses on an UNKNOWN classification — worse than the single public mapping this round sent before. Caught by cpp-reviewer. CORRECTION while implementing: the claim that C++ also lacks JS's nat.add(reply.to, reply.from) (connect.js:578) is WRONG. PoolSocket::handle_message (holepunch.cpp:958-963) already feeds every response's wire to field to the same sampler — a broader equivalent, since round 1's reply arrives on the pool socket. An explicit call there is dead code; caught because the red-check refused to go red. B2 was one divergence, not two. Tests PunchingRoundGossipsFullSampledSet + a failover-relay assertion (a relay that was never a pool-sampling target still gets its observation gossiped). Both red-checked. 731/731, ASAN clean, cpp-reviewer SHIP. Causality NOT claimed — the same field capture shows connects succeeding with 1 addrs, and the 2026-08-02 midnight failure had only one sampled mapping, so B2 would not have saved it. Landed on parity grounds. B1 + D-secondary remain the live pair on that path. Original entry follows.
  • B2 (NOW THE PRIORITY OF THE PAIR — see Finding L) — Round-2 holepunch payload sends ONE address (our_addr, holepunch.cpp:1880-1885) instead of nat_sampler().addresses(); JS sends the full set in both rounds (connect.js:567,654,684). Field Finding F (2026-07-23, SUCCESS) shows B2 is load-bearing, not a nicety: a port-varying SERVER (announce :45747 ≠ holepunch egress :2707) mislabeled fw=2 CONSISTENT (B1 latch) connected ONLY because it reported BOTH ports and the client probed both. B2's multi-address reporting is the mechanism that compensates for B1's mislabel — so B1 (correct classification) + B2 (full address set) are complementary; land both. Also note: Finding F shows B1 mislabels the SERVER's own self-classification too, not just clients.
  • D-secondary — server NatSampler never evicts stale external addresses after a NAT remap (:62622 lingered next to live :48008). Real but not a connect blocker; wants an eviction/aging pass.

secret-stream (1 open)

  • connect-1 (MED) — connected/on_connect gated on the REMOTE header + local write-ack, not on the LOCAL handshake as in JS (+½–1 RTT to first byte). DEFERRED — needs a live JS cross-test, not a batch edit. Design in memory parity_audit_2026_07_08.md (split is_ready → can_encrypt/can_decrypt).

B. Accepted divergences (do NOT re-flag as bugs)

Kept where C++ is safer/more correct than JS. Documented so the next sweep doesn't re-report them:

  • compact array-length cap 4096 (JS 0x100000) — anti-DoS.
  • routing exact-k-closest sort (JS bucket-order early-stop) — strictly more correct.
  • blind-relay 30s unpaired-pairing TTL + 1024-pairing cap (JS unbounded) — anti-DoS.
  • SocketPool routes cap + GC (JS no cap) — anti-DoS + leak prevention.
  • FIND_NODE/DOWN_HINT 1/sec/IP rate limits (JS unbounded).
  • ConnectionPool: local Server router entry refreshed, not clobbered (JS last-writer-wins would kill our own listener).
  • No auto-bootstrap of the public network by default (embedded/library targets must not auto-join) — opt in via default_bootstrap_nodes().
  • filterNode composes built-in + caller filter (JS discards the caller's).
  • C API can't express JS's explicit-ephemeral=true→non-adaptive state.
  • Announcer keepalive DRIFT DETECTION (08e2f47): pong to-field vs stored peer_addr triggers an early refresh (rate-limited 10s). JS discards the pong body (announcer.js:114-121) and waits out the 5-min reannounce.
  • Round-1 relay FAILOVER (Finding M, d5988ec) — FIELD-VALIDATED. On a round-1 failure we pop the next announce relay and re-run run_round1 on the SAME pool socket / puncher / NAT samples (holepunch::fail_round1, pick_fallback_relays; marked at connect.cpp:971). JS bails terminally (connect.js:270-275) — and asks for exactly this fix in its own source twice: // TODO: we should retry here with some of the other relays, bail for now (connect.js:271) and // TODO: retry with another relay? (connect.js:314). Before this, ONE unreachable relay in the announce was a permanent -5: 81/81 deterministic repro, and stock nospoon-JS failed identically on the same network, so it is NOT a C++ regression. Field- validated 2026-07-30: two dead relays skipped, rounds 1 AND 2 completed through the third. Retry scope mirrors JS exactly — the six sites JS reaches by THROWING out of probeRound fail over; those it reaches via abort() (double-random, unstable NAT, probe exhaustion) stay terminal. SECURITY: capped at announcer::PICK_BEST - 1 legs, NOT the 128-entry wire cap — 128 × multi-second RPC would hand a hostile connect() target a ~6-minute stall for free. Honest servers announce ≤3. Do NOT "simplify" this by substituting the client's own observation of the same relay host — relays[i] is a live server↔relay UDP session, not an address; JS explains why at connect.js:280-285. Failover is safe precisely because each entry carries its own peer_address and its own session.
  • Announcer STUCK-CYCLE WATCHDOG (f843b4b, announcer.cpp:223-245): if updating_ stays latched past cycle_stuck_ms_ (60s) the cycle is force-reset. JS has no equivalent; a wedged cycle there silently disables reannounce forever (field Finding E).
  • Announcer relay-set swap is guarded against an EMPTY new cycle (82d3c55): a cycle that commits nothing keeps the previous advertised set rather than publishing an empty one. JS would replace unconditionally.

C. Needs LIVE validation (user's nospoon / CGNAT phone)

Loopback can't prove these; validate against a real NAT'd JS peer:

  • Finding H fix (pickBest, 2026-07-26) — FIELD-VALIDATED 2026-07-26: phone (mobile CGNAT) now connects every time, fast. The mobile connect-latency stall (~28s hunting dead record-holders) is gone, and the one robustness risk — findPeer converging to a 3-node record set — did NOT materialize (Kademlia converges, as JS proves in prod). Committed 12c4af6.
  • Finding A fix (08e2f47, 2026-07-22): announcer publish-after-settle
    • keepalive drift detection + closestNodes reuse. Retest checklist in docs/FIELD-DIAGNOSIS-2026-07-22.md — key test: disconnect → IMMEDIATE reconnect repeatedly across >10 min, expect success at every point in the announce cycle; watch for DRIFTED log lines healing within ~5-10s.
  • Relay→direct upgrade: C++ client rides relay → punch lands → migrates with no ETIMEDOUT/-110; relay closed only after provable direct arrival.
  • Server-side migration when a JS client is on relay and the C++ server punches.
  • Holepuncher birthday win end-to-end (the pinning residual at holepunch.cpp:1450 is FIXED 2026-07-21 — winning SocketRef now rides in HolepunchResult.socket_keepalive); still needs live validation, plus server-side birthday stream completion (accepts via main-socket firewall, not birthday SocketRefs) — THE symmetric-CGNAT-server path.
  • Outgoing request id accepted by JS validateId → C++ node appears in JS routing tables; bootstrapper works as a real DHT id-holder.
  • TRY_LATER end-to-end (throttled server → client waits 10-20s → completes).
  • LAN same-NAT shortcut (connect-6, 2026-07-22): now EXCLUSIVE of holepunch — verify a real same-LAN connect still succeeds and a failed LAN ping aborts cleanly instead of hanging.
  • Empty relayAddresses announce record (announce-4, 2026-07-22): one live nospoon round-trip to confirm relay discovery is unaffected end-to-end.

D. Missing JS parity features

  • NAT sample ranking — keep following the promoted sample across multiple swaps, matching JS nat.js:addSample (2026-11-15). This fixes winner ordering and classification when a late sample overtakes several entries. Equal-hit ordering, freeze and source deduplication stay unchanged.
  • Server holepunch opt-out — native set_holepunch_enabled(false) matches JS opts.holepunch=false (2026-11-15): new sessions allocate no punch socket or sampler and ignore holepunch rounds and bare probes, unless an older enabled session still needs probe echo, while direct UDX and blind-relay pairing remain available. Default enabled; the existing holepunch veto callback is unchanged. C/wrapper exposure is separate.
  • Arbitrary-target native unannounce — target-aware lookup_and_unannounce_target and unannounce_target match JS topic semantics (2026-11-07). Legacy public-key methods retain hashing. C unannounce_target now exposes exact-topic withdrawal (2026-11-13), using the existing native completion/signing path. Python unannounce_target exposes the same exact-topic operation (2026-11-15); legacy unannounce still hashes the public key. Streaming query controls and other wrappers remain separate work.
  • Already-hashed native findPeer target — C++ find_peer_target matches JS findPeer(target, { hash: false }) (2026-11-07). Existing public-key methods keep their semantics. C find_peer_target and its cancelable _ex now expose the exact-target operation (2026-11-13), sharing legacy callback/error/cancel behavior. Higher-level wrapper methods and other target/query controls remain separate work.
  • changeRemote — DONE this session (relay→direct upgrade port: libudx udx_stream_change_remote wrapped in relay_upgrade::try_change_remote + Duplex firewall tap + FFI). Covers the path-upgrade case; mid-connection NAT remap is out of scope (JS doesn't handle it either — onsocket is one-shot).
  • _relayAddressesCache — IMPLEMENTED (c196205; source rechecked 2026-10-13). HyperDHT::cache_relay_addresses() stores successful-connect relays by server pubkey; reconnect queries only cached relays first, then falls back to ordinary discovery if needed (2026-11-05). The cache is capped at 512 entries with unordered-map eviction, rather than JS xache LRU. See src/dht.cpp and src/connect.cpp.
  • Read-side backpressure — DONE 2026-07-28 (a08cade). SecretStreamDuplex::pause_read/resume_read wrap udx_stream_read_stop/read_start, exposed as hyperdht_stream_pause/resume on the C API (idempotent, NULL/closed-safe; unordered datagrams unaffected). Test test_stream_pause.cpp. RESIDUAL: the automatic highWaterMark (16KB) trigger is NOT implemented — pause/resume is caller-driven, where JS pauses itself when the read buffer exceeds highWaterMark. protomux-2 (no true read-side pause; buffer+teardown substituted) is likewise still open.
  • Sleeping-interval wake detection — VERIFY: the tick rewrite added _onwakeup/do_wakeup + last_tick_ms_, but confirm background_tick actually compares wall-clock gap vs SLEEPING_INTERVAL (15s) and triggers the wake path (JS dht-rpc/index.js:764-799). If the gap comparison isn't wired, finish it. See memory sleeping_interval_gap.md.
  • Bootstrap DNS / ip@host form — _resolveBootstrapNodes DNS + @-host fallback absent (IP-only). Callers passing hostnames get nothing.

E. Hardening / verification tasks

  • Full ASAN/valgrind leak sweep of the hot path (confirm no hot-path leaks; re-verify after this session's large diff). The teardown leaks previously logged here as "pre-existing, in libuv/libudx internals" were pinned down 2026-07-28 — see TD1-TD3 below. Two of the three are ours, not libuv's.
  • Fuzzing (2026-08-25): 9 harnesses, 30 min each, 1.677 B runs, zero findings. No crashes, no ASAN reports, no UB. Final coverage per target: protomux 893, noise_payload 399, messages 385, noise_handshake 223, holepunch_msg 219, handshake_msg 165, compact 108, blind_relay 88, secret_stream 75.
    • Four harnesses added. fuzz_noise_handshake is the important one: NoiseIK::recv on the responder side is the deepest PRE-AUTH surface in the library — every PEER_HANDSHAKE carries attacker-chosen msg.noise bytes and nothing authenticates upstream of it, because the handshake IS the authentication. (fuzz_noise_payload only decodes the payload struct of an ALREADY-SUCCESSFUL handshake — a different thing.)
    • fuzz_protomux (Mux::on_data — varint framing, channel dispatch, OPEN/CLOSE/REJECT, handle_batch) is the deepest target overall: 400 corpus entries against ~24 for the flat decoders. fuzz_blind_relay (decode_pair/decode_unpair) closed the last uncovered header-level wire decoder. fuzz_secret_stream is thin by nature — most of its body is libsodium AEAD, so a clean run there means little.
    • decode_holepunch_payload needs no separate target: fuzz_holepunch_msg already exercises it.
    • Crypto-gated targets are worthless unseeded. Random bytes never pass Poly1305, so an unseeded fuzzer only ever exercises pre-MAC length/slicing code. fuzz/gen_seeds.cpp emits a valid msg1 / header / encrypted frame into the corpus; seeding lifted fuzz_noise_handshake from a 3-unit plateau to cov 144 in 20 s.
    • A harness can be clean because it tests nothing. Both crypto harnesses were initially broken in ways a passing 30-min run hid, and the flat coverage number was the only symptom:
      • fuzz_secret_stream fed the stream its OWN header to receive_header. local_id_ uses NS_INITIATOR and remote_id_ NS_RESPONDER (src/secret_stream.cpp:40-48), so it always failed, is_ready() stayed false, and decrypt() returned at its first line — 390 M runs tested the guard, not the AEAD. Fix: synthesize the header from a role/key-swapped peer instance.
      • fuzz_noise_handshake's is_complete() branch was dead for every input: one responder recv() advances message_index_ 0->1, completion needs

        =2 (src/noise_wrap.cpp:424,507). Fix: reply with send() (msg2) as finalize_handshake does. Verify a harness by whether coverage MOVES when you fix it, not by whether the run is green.

    • Run with UBSAN_OPTIONS=halt_on_error=1. The build enables -fsanitize=undefined, but UBSan defaults to print-and-continue, so a UB finding neither stops the fuzzer nor fails the run.
    • fuzz/CMakeLists.txt now uses fuzzer-no-link globally with -fsanitize=fuzzer per fuzz target. Plain fuzzer globally gave every executable libFuzzer's main(), so any helper with its own main (gen_seeds) failed to link.
    • Corpora persist in fuzz/corpus/<target>/ (gitignored) so runs compound instead of restarting from zero; crash reproducers land in fuzz/artifacts/.
  • Fuzzing, still uncovered: rpc_handlers per-command parse of req.value (announce record + signature paths, token-gated but the token is only address-bound), and libudx packet parsing (deps/libudx/src/udx.c — every inbound datagram; upstream has functional tests and CI but no fuzzing and no sanitizer flags anywhere in its tree). A CI job running each target ~5 min on PRs would catch parser regressions at the point they land.
  • Stress: 100 concurrent JS clients vs one C++ server; measure success rate + memory growth; confirm the probe-listener multi-slot fix holds.
  • Soak: 12h+ connection, data every 5 min; verify NAT pinhole + SecretStream keepalive, no drift.

E.1 Teardown defects (found 2026-07-28 while building holesail-cpp)

All three are shutdown-only, so a long-running server pays them once at exit and the OS reclaims. They matter for embedders that create and destroy many DHT instances in one process (test harnesses, apps that rebuild the DHT on a network change) — there they accumulate. holesail-cpp scopes them in its test/lsan.supp, which doubles as the reproduction recipe.

  • TD1 — a cancelled query leaks its state (~1.3 KB). The on_done contract is NOT violated. Corrected 2026-07-29 — the original entry claimed "hyperdht_query_cancel() never fires on_done, a contract violation". That half is REFUTED by standalone repro; the leak half stands, at a much smaller magnitude than first reported.

    • Contract HOLDS. hyperdht_query_cancel() DOES fire on_done with HYPERDHT_ERR_CANCELLED (-8), on all four cancelable paths (immutable_get_ex, mutable_get_ex, find_peer_ex, lookup_ex), in both orderings (cancel→pump→free and cancel→free→pump). Chain verified in code: ffi_core.cpp:28 st.q->destroy() → Query::destroy() → fire_done_once(QUERY_OK) → the FFI done lambda, which maps state->cancelled to -8. Matches hyperdht.h:751.
    • The one case where the callback does NOT run is hyperdht_query_free() without a preceding cancel — free deliberately detaches by nulling done_cb (ffi_core.cpp:31-40). That is documented behaviour (hyperdht.h:759), not a defect. Most likely what the original repro hit.
    • The leak is real but ~1.3 KB, not ~34 KB. A/B under ASAN, same binary: baseline create/bind/destroy = 15623 B / 18 allocs; identical run with one immutable_get_ex cancelled mid-flight = 16895 B / 35 allocs. Delta = 1272 B / 17 allocations for the cancelled query, with on_done observed firing. So QueryState / query::Query still outlive teardown even though the completion lambda ran — the ownership problem in query_free's "state may still be alive via lambda ref, which is fine" comment is genuine; the reasoning about why was not.
    • Fix: give query_free real ownership (or drop the query's self-reference once done). Do NOT "fix" it by firing on_done on cancel — that already happens, and adding a second fire would break the fires-exactly-once contract. Re-run the full 716-test lane after.
    • Repro note: measure with stdout UNBUFFERED. DHT_LOG writes to stderr unbuffered while printf is block-buffered when piped, which interleaves the transcript misleadingly and can make a callback look like it never ran.
  • TD2 — FIXED 2026-10-24: the stopped interface watcher now closes. RpcSocket::close() calls udx_teardown(), closing streams, sockets and listeners while their owner remains alive for the loop drain. The generated watcher close callback uses its saved UDX pointer after the user deleter. HyperDHT.CreateAndDestroy now requires uv_loop_close() == 0; that lifecycle and seven UDX tests pass ASan/UBSan with leak checking enabled. Eighteen relay tests also pass sanitizers with leak checking disabled; this does not establish all-project leak freedom. The original July reproduction left one inactive, unclosed timer and returned UV_EBUSY after a full drain. The consumer's defensive uv_walk fallback remains unchanged.

  • TD3 — in-flight connect/stream state dropped when hyperdht_destroy beats a graceful close round-trip. Frames: HyperDHT::do_connect, server::Server::on_handshake_result, udx__cirbuf_init / _set. ~11 KB per integration run, and it scales with connection count — consistent with per-connection state being abandoned rather than reaped. Lower confidence than TD1/TD2: established by tracing frame #1 of every unsuppressed allocation to a dependency frame, not by a standalone repro. Reproduce before fixing.


F. Round-2 blind spots (never audited line-by-line)

The 2026-07-09 sweep did NOT run a finder on these. Seed a round-2 sweep here before any full "core frozen" sign-off:

  • health.cpp vs dht-rpc/lib/health.js (only checked at the tick call-site).
    • Per-tick idle gate (2026-11-12): fewer than four responses/timeouts in the latest tick do not contribute a healthy/degraded vote, matching JS's delta-based idle getter. Window totals remain available.
    • Per-tick availability (2026-11-13): an active tick without responses reports OFFLINE immediately; a responding active tick recovers without waiting for four healthy ticks. JS's four-tick degradation latch remains, and going offline clears it. Aggregate statistics and idle/cold gates stay.
  • HyperDHT suspend()/resume() socket rebind + inbound-drop (io.js suspend/_rebind) — rpc-level stop_tick/start_tick alone don't match.
  • raw-stream-set.js, semaphore.js, refresh-chain.js, commands.js, the udx wrapper.
  • The entire ffi_* layer (the natural freeze boundary — audit last, once the core below it is frozen). RAISED IN PRIORITY 2026-07-28: the first real bug found here was a SECURITY inversion — hyperdht_server_set_firewall returned cb(...) == 0 where FirewallCb means true=REJECT, so every C-API server with a firewall admitted exactly the peers it should refuse and refused the authorised one (0203820, found via holesail-cpp, regression tests ServerFfiFirewall.*). Polarity conversions are the smell: the sibling set_holepunch (== 0) and async firewall_done (!= 0) were each re-checked and ARE correct — the two callbacks legitimately have opposite polarities (FirewallCb true=reject vs HolepunchCb false=abort). Audit the rest of ffi_* for the same class: bool/int contract mismatches at the boundary, which compile fine and fail silently.

G. Small follow-ups

  • F1 (field Finding F, 2026-07-23) — after a direct probe wins (rawStream firewall fires, connected), the client keeps the round-2 BLIND RELAY request alive and retries it 3× to timeout (Round 2: TIMEOUT (relay)). Cancel the round-2 relay once the direct probe connects — wasted relay traffic + a scary-but-harmless timeout in field logs. Not a bug.
  • F2 (field Finding F, 2026-07-23) — server logs on_peer_holepunch: unknown session id=0 for the client's round-2 that arrives AFTER the session connected+cleaned on round 1. Benign; downgrade the log level or match it against a recently-completed session so it doesn't read as an error in field captures.
  • immutable_put/mutable_put commit errors reach C++ results and C done callbacks (6bc02b1, 2026-10-26); callback signatures remain unchanged.
  • Wrapper build CI: Python/Kotlin/Rust announce-ABI change (f37ef1d) is UNVERIFIED — no wrapper build in the C++ gate. Add a wrapper-build check.
  • Tighten connect-3 wording in the sweep docs so it's not read as "reusableSocket unimplemented" — RESOLVED 2026-07-28 (a08cade): the C-API client option landed as hyperdht_connect_opts_t.reusable_socket (tail- appended; Python ctypes mirror updated in the same commit per the CLAUDE.md ABI gotcha). Server + wire + client option are all done; only the route-cache gating nuance remains, tracked as connect/divergence-1 in Section A.
  • tick-7: full 2-pass bootstrap + testNat-gated second _updateNetworkState (only the quick-firewall PING_NAT first-responder heuristic landed).
  • holepuncher-4: fresh-socket reopen (currently same-socket resample — recovers lossy-UNKNOWN, not the new-NAT-mapping case). Blocked on the upgrade-port socket-handle invariant; revisit if a real CGNAT case needs it.
  • Announcer synchronous-cycle-completion parks a stale current_query_ (cpp-reviewer, Finding E batch): if a full-congestion walk completes synchronously inside dht_ops::find_peer() before the current_query_ = find_peer(...) assignment, on_done's reset() runs on the old (null) value and the finished Query is parked until the next update(). Benign (destroy() idempotent, self-heals, no leak/UAF), pre-existing pattern now reachable via a 2nd trigger. Real fix = defer Query's first dispatch via uv_idle; not warranted yet.
  • Documented deviations from the 2026-07-22 batch (deliberate, revisit only if live behavior warrants):
    • connect-7 veto abort is local-only; JS also sends an ERROR_ABORTED round to the relay. Skipped: PunchState::complete closes the pool socket immediately (send race). Add with a deferred close if needed.
    • HOLEPUNCH_TIMEOUT (-6) now covers veto + LAN-ping-fail + passive timeout (all JS HOLEPUNCH_ABORTED). Rename would touch FFI/wrappers.
    • server veto/punch-fail clear_session immediately; JS defers ~10s via puncher teardown (pre-existing semantics, kept).
    • direct branch lacks JS's relayed && !remoteHolepunchable gate — pre-existing structural divergence; gating would break direct-to-server connects (loopback fixtures rely on it).

H2. Dependency watch — libuv 1.52 regression (recurring)

hyperdht-cpp is pinned to libuv 1.51.x (flake on nixos-25.11) because libuv 1.52.0 has a UDP POLLERR regression that wedges libudx streams on real NAT paths — full write-up in docs/LIBUV-VERSION.md; unwired fallback patch in nix/libuv-1.52-udp-pollerr.patch. Re-check whenever bumping nixpkgs/libuv:

  • Has libudx bumped its libuv pin past 1.51 or shipped a 1.52-compat fix? (deps/libudx/CMakeLists.txt:~7; watch holepunchto/libudx main.)
  • Has libuv shipped >= 1.52.2 with PR #3250? If yes → drop the 25.11 pin, delete the patch, remove the CMakeLists version guard, use stock libuv.

H. Future work

  • Extract run_round1() into named phases (src/holepunch.cpp) — the single worst readability hot spot: 457 lines, 9× the 50-line guideline. Split into probe_round → await_sampling → analyze_and_maybe_reopen → round2, each taking PunchState. Do it WITH the Finding K fix, not as separate churn — the proceed lambda, the sampling gate and the reopen path all live in one function body today, which is precisely why K (a mid-flight pool_fw read that contradicts its own comment) is subtle instead of obvious. Scope discipline — a 2026-07-29 measurement says the library is NOT spaghetti otherwise, so do NOT generalise this into a broad refactor: coupling is low (most src/*.cpp include 3-5 hyperdht headers; only dht.hpp at 12, the facade), code density is ~340 lines/file across 36 files (the ~20.4k raw src/ total is 27% comments, and those JS file:line parity refs are what made Findings H/I/J root-causable by reading). holepunch.cpp carries 36 lambdas (server 23, connect 19, rpc 17) — the cost of "every JS await becomes a callback chain". Coroutines would flatten it but are a large change and hostile to the ESP32 target: rejected. NOTE: raw indentation hits 56 columns in holepunch.cpp but that is WRAPPED ARGUMENTS, not nesting — true control-flow nesting at depth >=5 is 19 lines there, 8 in connect.
  • Round-2 adversarial JS-parity sweep — re-run jsparity-adversarial-sweep over the round-1 blind spots (Section F) + a re-diff of the subsystems changed heavily this session (rpc, connect, server, holepunch, protomux) to catch regressions the per-bucket reviews might have missed. Append new findings to Section A.

I. peeroxide cross-reference (second independent Rust oracle)

Context (2026-07-23): Rightbracket/peeroxide — independent Rust port of the FULL Hyperswarm stack (pure-Rust libudx + HyperDHT + Hyperswarm topic layer + CLI), wire-compatible with JS, v0.3.1, published (crates.io/homebrew). Local Nix build at ~/Desktop/repos/peeroxide-nix (nix build .#peeroxide). Source clone was in scratchpad; re-clone https://github.com/Rightbracket/peeroxide to act on these. It hit the SAME announce/staleness class we did — useful as a borrow source AND a second parity oracle. Two workstreams:

I.1 Announce/staleness — borrow ideas (check ours against theirs)

  • Refresh-token cheap-refresh. Their peeroxide-dht/src/persistent.rs:462 — announce carries optional refresh token; a re-announce with NO peer body but a refresh token restores the full record from a token-keyed cache → keepalives are tiny (token only), not full peer+sig+relays. Check: do WE send refresh tokens on re-announce + handle the no-peer+refresh restore node-side? (Jul-21 note: field exists in our messages, may be unused.) Parity gap + efficiency.
  • Parallel dual-announce + relay feedback. Their peeroxide/src/peer_discovery.rs do_refresh: each refresh fires topic-announce (on topic) AND self-announce (on hash(pk) → FE-holders for PEER_HANDSHAKE Phase 2) in parallel; relay addresses harvested from prior hash(pk) acker set feed the next topic record ("mirrors Node's Announcer.relayAddresses"). Check our Announcer does the same dual-target + relay feedback — this is the Finding A/D forward-state area.
  • Constants sanity-check: their REFRESH_INTERVAL=600s (+≤120s jitter), record TTL=20min ("matches Node.js"). Confirm our re-announce cadence (~10min, not 5) + TTL match.
  • Note: they PUNT unannounce (swarm.rs // leave/unannounce (future)) → their server-restart staleness window = full 20-min TTL. If we have clean unannounce-on-shutdown, we're AHEAD there — do NOT copy their omission.

I.2 JS-parity hunt — their golden fixtures as a parity oracle

  • Their committed byte-exact fixtures: tests/interop/*-fixtures.json (noise, noise-ik, dht-rpc, hyperdht, protomux, secure-payload, blind-relay), generated by tests/node/generate-*-golden.js from the JS reference. Feed their fixture inputs through our C++ encoders, assert output == their bytes. Both claim JS-parity → any mismatch localizes a bug to a specific message/field. Highest-value targets: noise-ik-fixtures.json (Ed25519 DH, our #1 gotcha) + hyperdht-fixtures.json (announce/lookup, current pain).
  • Mine their interop .rs assertions for edges we may skip (empty relays, refresh-only announce, bump handling, MAX_RELAY_ADDRESSES truncation).

J. Server per-session punch socket (branch feat/server-punch-socket)

✅ MERGED to main 2026-08-22 (fast-forward, branch HEAD cfb4101)

Merged for the JS parity, NOT as a Finding Q fix — see §J.4, the JS control proved that failure mode is not a hyperdht-cpp bug at all. Pre-merge gate re-run at merge time: 767/767, 0 ASAN errors across holepunch / socket_pool / server / server_punch_socket / relay_upgrade / pool_socket_requests / secret_stream / router / server_handshake. Server::disable_fast_mode_ping was kept (defaults false, zero runtime cost, documented diagnostic-only) — it is what made Finding Q reproducible and what let us control the comparison against the reference server.

Field validation before merge: live JS-interop against real hyperdht 6.29.1 on the public DHT; a real cross-NAT punch against a phone on mobile CGNAT; and ~100 phone sessions across two client networks and three server builds.

Still to do after this merge:

  • Push main (29 commits ahead of origin/main at merge time).
  • Bump nospoon's pin ~/Desktop/repos/nospoon/cpp/hyperdht-cpp.nix rev + hash (still 3421264) — needs the push first, it fetches from GitHub by rev.
  • Add the one-line info.socket_keepalive hold in nospoon/cpp/server.cpp; the field run confirmed the predicted socket close refused fd parking.
  • BUG A (§J.1) is still open and unrelated to this branch.

Parity actually restored (this is the justification — each is a real divergence from the JS reference that this branch closes):

  1. Per-session punch socket — JS holepuncher.js:14 (dht._socketPool.acquire()), server side server.js:436. We punched from the persistent announce socket. This is the headline item.
  2. addresses4 = [remoteAddress] + LAN (JS server.js shape) — replaced a sampler dump plus a port rewrite that produced duplicate addr[0]==addr[1] in the field.
  3. remoteAddress() reads the RING sampler (dht-rpc _nat role), not the Nat classifier, and one ourRemoteAddr keys firewall + holepunch + addresses4 together (server.js:271,357-359).
  4. Round replies deferred until NAT sampling settles — JS await p.analyze() (server.js:518-519).
  5. NAT discovery resolves on classification, not on the last straggler ping — JS nat.js:172-179.
  6. Session NAT sampler fed only by AUTHENTICATED rounds — JS server.js:491-511. (Also a security fix: 3 spoofed sources could otherwise pin an attacker address.)
  7. Fast-mode ping egresses from the punch socket — JS holepuncher.js:77.
  8. Probe echo + stream adoption on the punch socket.

NOT parity — beyond-JS additions, review them as such before merging:

  • Server::disable_fast_mode_ping (default false, no JS equivalent). Diagnostic only, but it is the ONLY way to reproduce Finding Q on demand, so it earns its place. Keep, or drop it and lose the reproduction — an explicit call to make.
  • Bug fixes JS gets for free from GC or simply does not have: BUG B (~Holepuncher releasing holders + the random-punch throttle), relay_token in both ServerConnection move ops, SocketPool::destroy checked close, ~SecretStreamDuplex writing through a freed raw_stream_, hyperdht_busy_close_count() on the C FFI.

Pre-merge checklist:

  • Re-run ctest -E 'LiveServer|test_server_live' (expect 767/767) + the ASAN per-binary sweep (expect 0 ERROR: AddressSanitizer; leaks unchanged).
  • Decide explicitly on shipping disable_fast_mode_ping.
  • Merge direct to main (this repo has no PR gate) and push.
  • Bump nospoon's pin: ~/Desktop/repos/nospoon/cpp/hyperdht-cpp.nix rev + hash (still on 3421264), and add the one-line info.socket_keepalive hold in nospoon/cpp/server.cpp — the field run confirmed the predicted socket close refused fd parking.

Do NOT claim it fixes Finding Q. A/B'd on a real CGNAT path: branch 1/14 black holes vs main 3/12, p≈0.31 — not significant, and the branch's own failure used the entire new path. Honest field line: "may reduce the -5s, unproven." Q stays open; see §J.4.

Plan: docs/superpowers/plans/2026-08-17-server-punch-socket-parity.md. Ledger: .superpowers/sdd/2026-08-17-server-punch-socket-parity/progress.md (every parked finding + deferred minor lives there — read it before resuming).

Why: the C++ server punched from the persistent announce socket (shared with announces, relay keepalives and all DHT RPC for the process lifetime); JS acquires a fresh pool socket per holepunch session (holepuncher.js:14, server.js:436). Proven by a control we already had: a nospoon-JS server on the same Pi5, same router, same clients, connected every time for months while the C++ server failed intermittently (handoff line 14 + §Finding Q). Client, network, router and probe format were each separately eliminated. Full case: memory server_punch_socket_parity, Q2-EVIDENCE-2026-08-04.md.

J.1 Blocking follow-ups found by this work (pre-existing, NOT caused by it)

  • BUG B — birthday holders leak; the server permanently loses birthday punching. FIXED e8fd86c. Server::on_socket does conn.puncher.reset() → ~Holepuncher, which only nulled on_holepunch_message + stop(); only destroy() released holders (src/holepunch.cpp:367-381 vs :740). BIRTHDAY_SOCKETS=256 (include/hyperdht/holepunch.hpp:284) vs MAX_POOL_SOCKETS=512 (src/socket_pool.cpp:178) ⇒ after ~2 successful RANDOM+CONSISTENT server punches acquire() returned nullptr for the rest of the process. Measured: 201 sockets retained after the drop, 0 with destroy(). Fixed in ~Holepuncher rather than at the one call site, so every drop path is covered; release() and decrement_randomized() are both idempotent, so it is a no-op when destroy() already ran. A second leak on the same path was found while fixing it: the dropped puncher also left PunchStats::random_punches incremented, and random_punch_limit is 1 → can_random_punch() never returned true again, so no random punch (CONSISTENT+RANDOM sprays included) could start. Test Holepuncher.DropWithoutDestroyReleasesHoldersAndThrottle, red-verified on both assertions. 763/763.
  • BUG A (HIGH) — blind-relay fallback implemented and live-verified. Readiness fixed 2026-10-23: both production relay callers prepare the channel and pairing callbacks before starting SecretStream, then send Protomux OPEN only after the encrypted header exchange completes. BlindRelayClient retains requests until the channel opens and sends each once; cancellation and synchronous client destruction have regression coverage. Production client/server regressions require an actual PAIR request; the server fixture no longer sends an unsolicited reply. A local relay carried encrypted data both ways. All three suites and isolated consumer connectivity checks pass with public and NAT server placements. Public/OPEN control socket fixed 2026-10-23: when the handshake has no selected pool socket, both relay callers connect the control stream using the node's DHT socket. All three suites and forced-relay DATA checks pass with a public application server and both NAT-board server placements, exact encrypted payload hashes, no direct board DATA, and public/NAT unused-relay-identity negatives. These negatives do not cover packet loss to a running relay. Live relay-to-direct upgrade verified 2026-10-24: one isolated run with a public application server and one with a NAT application server, both using a public persistent OPEN relay. A temporary direct-path gate requires real encrypted relay traffic before allowing the normal holepunch path. Exact 8192-byte echo and 32-byte reverse hashes match through 32 and 37 numbered round trips respectively, including at least three relayed and twenty direct rounds per run. Each endpoint opens its encrypted connection once; the original UDX streams survive migration without reconnecting. Each endpoint's packet capture proves direct traffic arrived before its relay control FIN, followed by direct DATA. Artifact hashes, effective commands and kernel executable paths match; both hosts return to byte-identical network/firewall snapshots with owned rules and lab processes removed. This verifies the existing upgrade code; no library change was needed. It does not cover relay outage recovery, other network/OS combinations or soak. Transport-write errors fixed 2026-10-24: both relay control writers destroy the encrypted stream on a rejected write; stream closure tears down the mux and fails pending pairing through the existing deferred abort path. The parallel direct attempt remains available. Client/server regressions force a rejected PAIR write and require failure before the pairing watchdog. All three suites and isolated default/forced-relay connectivity checks pass with public and NAT server placements. All 35 ESP32-S3 component files compile with ESP-IDF 5.5.2; firmware linking and board operation remain unverified. Client request reuse fixed 2026-10-23: all routes of one connect now share the same Noise request, so pending routes can deduplicate onto one server session. Reply validation uses an isolated state copy per attempt. Regressions, all three suites and public/NAT/LAN consumer checks pass. A forced-relay NAT capture also verifies identical complete Noise requests through two DHT routes and one deduplicated pending server session. Completed-handshake dedup fixed 2026-10-23: retain encrypted replies for the existing clear-wait interval after direct emission (default 10 s), with weak socket references and a bounded cache (256 desktop / 16 embedded). Active sessions and stream ownership keep their existing cleanup. Regressions cover identical replies, expiry without duplicate refresh, closed-socket fallback and suspend. Public/NAT and both LAN directions pass, including payload past the grace interval; a naturally late LAN request reused the completed reply without creating another session. Forced-relay DATA also passes. All 35 ESP32-S3 component sources compile with ESP-IDF 5.5.2 and exceptions/RTTI disabled; firmware linking and board operation remain untested.

Deferred connectivity verification

These checks remain unverified by the October lab fixes:

  • Distinct NAT networks, mobile/CGNAT paths and network handover.
  • Windows/macOS/Linux interoperability in supported client/server roles; repair the skipped IPv6 fixture and test real IPv6 paths.
  • Interruption and recovery of a running relay, and longer-session soak tests.
  • ESP32-S3 firmware linking, flashing and an actual board connection.

Broader sanitizer runs with leak detection disabled do not establish leak freedom.

J.2 Task 6

Done 2026-08-19/20 except the two items that need a network and a human, which are called out at the bottom. The branch is code-complete and clean, but it has still never carried a packet against a real peer.

  • cpp-reviewer on the whole branch — 3 HIGH + 3 MEDIUM, all verified against the code and the JS reference before acting, all fixed in 4d67452 except MEDIUM 1 (a genuinely separate parity gap, filed as J.3 below). Detail:

    • HIGH — discover_pool_addresses waited for the SLOWEST sampling ping instead of resolving on classification like JS nat.js:172-179. This gate sits in front of every parked PEER_HOLEPUNCH round, so one dead routing-table node held the reply for MAX_REQUEST_ATTEMPTS × timeout_for(dest). Measured on loopback: 3000 ms before, 150 ms after — routinely longer than the client's own round budget, so the client aborts with -5 and the server answers a tid nobody is waiting on. A regression on this branch's own target path; the branch would likely have field-failed with it in.
    • HIGH — UpgradeContext::set_socket_keepalive replaced the pin unconditionally. After the firewall tap migrates a stream onto the punch socket, the puncher's later birthday win offers a socket the stream is NOT on (its on_socket is a one-shot no-op), and the replace dropped the only ref to the live socket. Fixed in the class, since src/server.cpp and src/connect.cpp both do it.
    • HIGH — SecretStreamDuplex::on_udx_close nulled the stream's pointer to us but not ours to the stream, so ~SecretStreamDuplex wrote into memory libudx had already freed. Real heap-use-after-free, caught by ASAN.
    • MEDIUM — the per-session NAT sampler accepted a tid-matched reply from ANY source. Now source-checked (sample only; the request still completes).
    • MEDIUM — include/hyperdht/udx.hpp claimed refused sockets are "reaped at teardown"; nothing in this codebase calls udx_teardown(). Comment corrected.
  • ASAN — 0 errors across every test binary; leak totals unchanged (test_server 91-98 KB, in its noisy band; test_hyperdht 171108; test_connect_paths 21496). Two real UAFs were found and fixed getting there (the ~SecretStreamDuplex one above, plus ~FakeRelayPeer freeing itself out from under its own async uv_close callbacks).

  • hyperdht.h documents the hyperdht_stream_open contract, on both the function and hyperdht_connection_t::_internal: call it synchronously inside the connect/connection callback. _internal carries the socket keepalive and the producer frees it when the callback returns; there is no way to hold it from C without opening the stream, by design.

  • hyperdht_busy_close_count() exposed through the C FFI + the Python mirror (_ffi.py, _bindings.py, __init__.py). Rust bindgen and the Kotlin JNI regenerate from the header and need nothing.

  • relay_token added to both ServerConnection move ops (e8fd86c… 78eb2c0).

  • SocketPool::destroy() — checked close so refusals are counted, and it deliberately keeps flagging closed_ even on refusal. Do NOT "fix" that to match SocketRef::do_close: a ref that survives destroy() outlives the udx_t too (udx_t is a value member of RpcSocket, and HyperDHT declares socket_ before socket_pool_), so a late inactive() retrying the close is udx_socket_close on freed memory. A first version of this change shipped exactly that hazard and the reviewer caught it.

  • JS interop — DONE 2026-08-20, against real hyperdht 6.29.1 and the real public DHT (./build-debug/test_server_live + node test/js/connect_by_key.js <key>). JS client connected to the C++ server in 1352 ms. The new path is confirmed live, end to end, against real relays:

    • Session id=0 acquired a punch socket — the gate fires on a relayed handshake.
    • Round parked until sampling settles (id=0, 1 queued) then Punch-socket sampling done (id=0, ok=1) — the deferral works and the early-settle fix is visible in the log: it resolved after 3 of 7 sampling replies, with the other 4 still in flight. That is the HIGH-1 fix doing its job on a real network.
    • Round on punch socket (id=0) from RELAY-1 — the relay forwards the client's round straight to the per-session socket. This is the whole point of the branch and it is now proven on the wire, not just in tests.
    • Fast-mode ping to SERVER-A:19708 (peer_address, punch socket) — egresses from the punch socket, per gotcha 19b.
    • Client punching (id=0, fw=2, 2 addrs) — JS accepted our reply and declared punching.
  • CROSS-NAT PUNCH PROVEN ON THE BRANCH — 2026-08-20. Real phone on mobile CGNAT (CLIENT-MOBILE) → this laptop behind a home NAT (SERVER-A), via a scratch nospoon-cpp server on TUN 10.100.0.1, with the fast-mode ping disabled so the connection had to complete through the real probe loop. Branch connected 2/2, one probe each, both rounds arriving on the per-session punch socket: Round on punch socket ×2 → Client punching (fw=2, 1 addrs) → [hp] Sending probe ×1 → rawStream firewall → Stream open. The punch loop itself is proven to work cross-NAT. That much is solid.

  • A/B RUN PROPERLY — THE BRANCH DOES NOT FIX FINDING Q. Real phone, mobile CGNAT, fast-mode ping suppressed on both arms, same server key, alternating arms:

    | arm | sessions | landed | black holes | rate |
    |---|---|---|---|---|
    | `main` @ `ba03888` | 12 | 9 | 3 | 25% |
    | branch | 10 | 9 | 1 | 10% |
    
    25% vs 10% at these n is p≈0.6 — noise. **No improvement is demonstrable.**
    
    The decisive datum is not the rate, it is the branch's failing session, which
    used the ENTIRE new path and died anyway: `Session id=5 acquired a punch socket`
    → `Punch-socket sampling done (ok=1)` → `Round on punch socket` ×3 →
    `Client punching (fw=2, 1 addrs)` → 10 probes to a freshly-relay-observed address
    → nothing → `Session cleanup`. So punching from a fresh per-session socket whose
    mapping the relay observed *for that session* does **not** prevent the black hole.
    
    **This kills "the shared announce socket's mapping goes stale/busy" as the
    mechanism of Finding Q.** Keep the branch on its own merits (JS parity, a proven
    cross-NAT punch loop, and the real bugs the review found) — but do not merge it
    as a Finding Q fix, and do not tell the field it fixes the `-5`s.
    
    Harness, reusable: `scratchpad/arm.sh <main|branch> <runid>` swaps arms (same
    deterministic seed → same public key → the client never reconfigures);
    `test/js/connect_by_key.js` is a scriptable client. **Read the verdict from log
    counts, never from "it connected":** `[hp] Sending probe` (1 = landed, 10 = black
    hole) and `Stream open`.
    

J.4 What the A/B actually taught us about Finding Q

  • The failure is strictly bimodal. Across 22 sessions, every single one either landed on probe #1 or lost all 10. Never 2-9. That is not loss or congestion — it is binary reachability of one 4-tuple, decided before the first probe and stable for the whole 10-second schedule.

  • Base rate ~10-25%, on demand, reproducible any time by disabling the fast-mode ping. Fast mode otherwise masks the punch path completely (it connects at round 1 and the Holepuncher never starts) — which is exactly why this was never reproducible before, and why the field only ever saw "it works when fast mode fires".

  • Server-side causes are now largely excluded: fresh per-session socket, correct and freshly-observed destination address, sampling settled, Client punching with a gossiped address, probes provably sent. All present in the failing runs.

  • THE JS CONTROL SETTLES IT: this failure mode is NOT a hyperdht-cpp bug. Ran the nospoon-JS server (nix build .#nospoon-js) on the same laptop, same config, same seed ⇒ same public key, so the phone reconfigured nothing. JS fails from the hotel exactly as our C++ does, and succeeds from mobile exactly as our C++ does:

    server on the laptop from hotel from mobile
    C++ branch (fast on/off) FAIL 0/82 OK
    C++ main (fast on) FAIL 0/12 —
    JS reference impl FAIL OK

    Stock hyperdht suffers this identically. Before calling any future punch black-hole "our bug", run this control — it is cheap and decisive. Scope limit: this proves the DETERMINISTIC hotel case is implementation-independent; it does NOT prove the older INTERMITTENT mobile-CGNAT Q (10-25%) is the same phenomenon, which has never been JS-controlled.

  • A 100% reproducible instance exists (2026-08-20/21), and it exonerates the branch. One hotel wifi (CLIENT-WIFI-B) fails absolutely against this laptop while the same phone on mobile data connects on the first probe. Controlled properly after two false starts: fast mode ON and OFF both fail (57 and 25 sessions), and running main with fast mode on — config-identical to the Pi5 that the hotel CAN reach — also fails (0/12). So it is not the punch socket, not fast mode, and not any code on this branch. The broken thing is the hotel ↔ home-ISP network pair specifically: hotel ↔ Pi5-ISP (SERVER-B) works and mobile ↔ home-ISP works, and both servers are fw=2 needing a real punch, so no "one end is open" story explains it. Mapping and addressing are excluded by measurement too — the new observers: relay=… advertised=… line printed AGREE throughout, and sessions probing the relay-verified address still lost 20 probes each.

  • No server-side discriminator exists. All 25 sessions were mined across handshake source port, probed port, the delta between them, which relay forwarded the round, which arm, and position within the run. Nothing separates the 4 failures from the 21 successes — every dimension overlaps, failures are interleaved with successes rather than clustered (id=5 fail, 7 and 10 ok, 11 fail), and the only suggestive signal (probed > handshake port in 4/4 failures) is p≈0.11 against a 57% baseline, i.e. nothing. Everything the server can observe looks identical in the runs that work and the runs that die.

  • So the remaining suspect is the CLIENT's pinhole, which is where Finding Q's still-missing measurement always pointed: does the client's own round-1 fast-open (TTL-5) priming packet actually leave its carrier NAT? If that priming is dropped, there is no pinhole, every server probe dies, and the outcome is binary and intermittent — matching all 25 observations. Next test: capture at the client egress. Use a laptop tethered to mobile data as the client (the phone cannot tcpdump unrooted). Ruled out on the way: the phone is NOT running a stale client — its libhyperdht_jni.so is dated 2026-08-04 and B2 (f6608b7) landed 2026-08-02 — and the 1 addrs it gossips is correct, JS-identical behaviour for a port-consistent NAT (all samples collapse to one _samplesFull entry, so JS emits one address too).

  • Diagnostic gap to close first: the DHT_LOG output has no timestamps, so nothing in these logs can be correlated with wall-clock, inter-attempt spacing, or carrier events. Add a timestamp prefix to DHT_LOG (or run the server under systemd-cat/journal and read it there) before the next field round — several questions here were unanswerable purely for want of a clock.

  • nospoon needs one line when its pin is bumped: hold info.socket_keepalive alongside the peer's duplex (~/Desktop/repos/nospoon/cpp/server.cpp:93-97 connects synchronously today, so it currently parks one fd per punched connection).

J.3 Found while closing Task 6 (not blockers)

  • SocketRef::was_busy_ is dead — the linger path is unreachable, so reusableSocket/retryRoute does not work. git grep was_busy_ gives three hits: the = false initialiser, the read in if (reusable && was_busy_) (src/socket_pool.cpp:90), and the = false inside that branch. Never set true ⇒ LINGER_TIME_MS, lingering_, linger_timer_, on_linger_timeout, on_linger_close and the body of unlinger() are all dead code. JS lingers a reusable socket 3 s past its last ref precisely so the route survives a follow-up connect; we close immediately and SocketPool::remove → gc_routes_for_socket drops the route with it. A real JS-parity gap, separate from this branch.
  • SocketRef::close_maybe also lacks JS _closeMaybe's socket.idle gate (socket-pool.js:159) — JS never ATTEMPTS a close while a stream is attached. Worth doing WITH the item above, not before it: without the linger retry there is no hook to close on later, so gating alone just converts a counted refusal into a silent never-close.
  • PoolSocket::reply() (src/holepunch.cpp) has no production caller — only test/test_pool_socket_requests.cpp. The round reply goes out through socket_.udp_send(..., via) instead. Either wire it or delete it.
  • Sampling PINGs use { retry: false } (one send), matching JS nat.js:59 (2026-11-05); ordinary pooled RPCs retain their default retry budget.
  • Sampling campaigns deduplicate relay/routing/bootstrap endpoints before sending, matching JS nat.js's host:port visited gate (2026-11-05).
  • Review the remaining sampling campaign selection/retry policy against JS's visited-node second pass. The deliberate seven-target margin and current selection policy were retained by the per-request retry and same-campaign deduplication fixes; cross-campaign visited state is deferred.
  • QueryWalk.DownHintEmittedToReferrerOnTimeout is load-flaky: it needs ~2.02 s of wall clock and failed once in a parallel ctest run, then passed standalone 3/3 and in a full rerun. Untouched by this branch (src/query.cpp and test/test_query.cpp are identical to main). Give it slack or a deadline that is not wall-clock-tight.
  • Abandoned ServerConnection raw streams now release RawStreamCtx on destruction and move assignment (2026-10-23). All 11 test_server_punch_socket cases pass with ASan, UBSan and leak detection; this does not claim that every legacy test fixture is leak-free.
  • SocketPoolTest.DestroyClosesAll accounts for the whole 7488 B test_socket_pool ASAN leak: destroy() deliberately never deletes SocketRefs (an outstanding keepalive may still call inactive() on one). Expected, not a defect.
  • ServerConnection's hand-written move ops exist solely to null other.raw_stream. Wrapping raw_stream in a move-only holder and defaulting both would delete the bug class that just cost us relay_token.