Single source of truth for outstanding work. Everything to do lives here:
the rest of the JS-parity sweep, missing parity features, hardening tasks,
un-audited blind spots, and space for future sweeps. Update this file as work
lands; keep the one-line status snapshot in CLAUDE.md pointing here.
Last updated: 2026-11-11 (bootstrap sample bounds).
-
2026-11-11 bootstrap sample bounds: no-ID replies contribute at most one observation per endpoint and at most max(1, configured bootstrap count) observations in total, matching JS. A verified firewall probe resets the source cache when it replaces the address sampler. Identified peers use their existing sampling rules.
-
2026-11-11 bootstrap reply filtering: replies with a missing or invalid node ID now pass the node filter before contributing address observations, matching JS. The filter receives the sender's address-derived ID. RPC completion still occurs. Bootstrap sample deduplication and limits remain a separate item.
-
2026-11-11 identified peer sampling: requests and replies now share the same 360-tick sampling interval. A newly admitted peer contributes its first sample on either socket. An existing peer contributes a sample only on the active socket when its sample is due. This matches JS
_addNodeFromNetworkand_addNode. Bootstrap sampling remains a separate item. -
2026-11-10 valid-ID response node filters: rejected peers no longer contribute normal NAT/ring observations from responses, matching JS. One filter decision gates routing and sampling; the response callback and inflight completion still run. No-ID/invalid-ID bootstrap observations, response sampling age/socket policy and the separate firewall-probe sampler retain their behavior. No public API, layout, wrapper or wire changes.
-
2026-11-10 incoming request sampling age: admitted peers now update NAT/ring observations from active-socket requests only when never sampled or after 360 ticks, matching JS's OLD_NODE gate. Newly admitted peers still sample on either socket; inactive requests retain the previous sample age. Response/bootstrap sampling policy is separate work. No API/wire change.
-
2026-11-10 incoming request node filters: requests from a valid-ID peer rejected by filter_node no longer contribute NAT/ring observations on the active socket, matching JS. The filter is evaluated once before routing updates and request sampling; the request callback still runs. Accepted peers, public add_node_from_network and response/bootstrap sampling retain their behavior. No public API, layout, wrapper or wire changes.
-
2026-11-09 native lookup query controls: additive C++ lookup overloads on HyperDHT and dht_ops expose copied seeds, request retries and onlyClosestNodes before dispatch, matching JS lookup options. Original calls retain generic retries and ordinary discovery. Both paths share record validation; C API, wrappers and layouts are unchanged.
-
2026-11-08 native findPeer query controls: additive C++ overloads for find_peer and find_peer_target expose copied seed nodes, request retries and onlyClosestNodes before dispatch, using the existing worker. Original calls keep their defaults and hashing/validation semantics. C API, wrappers and layouts are unchanged; their option exposure remains separate work.
-
2026-11-07 arbitrary-target unannounce: native C++ lookup_and_unannounce_target and unannounce_target use a supplied topic directly, sharing the existing signed LOOKUP/UNANNOUNCE path. Existing public-key methods still hash once. Local cleanup removes only the signing key's record; relay-only forwards are removed only for its own hashed topic, matching JS. Other keys, unrelated forwards and local listeners survive. Completion still awaits withdrawals; C API, wrappers and layouts are unchanged.
-
2026-11-07 already-hashed findPeer target: named C++ HyperDHT::find_peer_target and dht_ops::find_peer_target use a supplied 32-byte target directly, matching JS's hash:false. Existing find_peer overloads still hash public keys and share the same record validation, routing replies, retries and seed handling. C API and wrappers are unchanged.
-
2026-11-06 relayed OPEN-client socket: the server explicitly passes its active socket to the consumer for a relayed OPEN-client handshake, matching JS's dht.socket: client socket while firewalled, server socket when public. Direct handshakes retain their existing consumer fallback; propagation of their arrival socket remains separate work.
-
2026-11-06 OPEN-client server endpoint: the server uses the client's endpoint observed by the handshake route, matching JS. The shortcut depends on OPEN alone, including an empty advertised address list. Sync and async firewall completion preserve the observed endpoint; NAT-client punching, relay bootstrap ordering, reply routing and consumer socket selection are unchanged; explicit socket propagation remains separate.
-
2026-11-06 public-client passive wait: a client that advertised OPEN waits for the relayed server to reach it, matching JS's handshake snapshot. The NAT classifier remains the input for active punching; it cannot report OPEN and no longer decides passive waiting. Later firewall-state changes do not change the sent snapshot. Completion or raw-stream destruction stops the wait timer; the ten-second timeout and existing direct, NAT, relay and cached-route policies are preserved.
-
2026-11-06 ordinary direct handshake (lab compatibility checked): a handshake reply without peerAddress completes on the main socket at the contacted endpoint, matching JS connectThroughNode before holepunch selection. This also covers servers advertising holepunch info or a non-OPEN firewall. Both cached relay discovery seeds and ordinary FIND_PEER use this path; cached socket-route and relayed-handshake behavior remain unchanged.
-
2026-11-06 first inbound node observation: a valid node newly admitted to the routing table contributes its request's address observation even on the inactive socket, matching JS dht-rpc's _addNode. Repeated requests on that socket do not resample it. Invalid/missing IDs, filtered peers, self and full-bucket rejections retain the sampling exclusion. Existing expected- socket and response sampling policies are unchanged; age-based resampling remains separate work.
-
2026-11-06 announcer pending periodic update: a periodic reannouncement waits for the current relay-ping round to settle, matching JS's awaited pings before its background update. Repeated deadlines coalesce; an explicit refresh satisfies the pending update, and stop cancels it. Reply and timeout settlement share this path. Explicit refresh/online/drift policy, intervals and watchdog are unchanged. Exact 100-round plus RTT cadence remains separate.
-
2026-11-18 announcer ordinary refresh: refresh requests received during a relay-ping round now wait for all replies/timeouts, matching JS's awaited pings. Repeated and periodic requests coalesce, and stop cancels the request. Refresh during an update remains absorbed by that cycle; idle refresh, deliberate immediate drift recovery, online handling and watchdog are unchanged.
-
2026-11-06 announcer periodic deadline: the existing five-minute reannounce timer restarts when a complete update settles, including retired withdrawals, instead of keeping its startup deadline after a manual refresh. Empty completed cycles restart that window too, matching JS's update-to-loop ordering. Ping cadence, watchdog and the five-minute interval are unchanged. This does not implement JS's exact 100-round plus RTT background-loop period.
-
2026-11-06 announcer successful-cycle wake: a nonempty announcement cycle schedules the next relay-ping round immediately after all commits and retired-relay withdrawals settle, matching JS's update-to-ping ordering. Empty/failed cycles retain their existing timer schedule. If a ping round is already in flight, it retains the full pause after settlement instead of starting an extra round. Startup/resume and watchdog behavior remain; full background-loop timing parity is still separate work.
-
2026-11-06 announcer cycle ordering: new relay keepalive rounds wait while the announcement cycle is updating, matching JS's awaited _runUpdate. The gate covers the walk, commits and retired-relay withdrawals; existing in-flight pings settle normally. Timer ticks still run the stuck-cycle watchdog. Startup/resume first pings and round spacing are unchanged; successful-cycle wake timing is covered above. Full background-loop timing parity remains separate work.
-
2026-11-06 explicit connection discovery hints: nonempty ConnectOptions::relay_addresses seed the first FIND_PEER pass ahead of cached hints, matching JS connect.js:320-333. A persistent node's live forwarding address still overrides both. Empty options retain cache behavior; unsuccessful seeded passes retain normal discovery fallback. The existing option is copied for asynchronous route fallback; APIs, wrappers, retry budgets and cache updates are unchanged.
-
2026-11-05 announcer first keepalive: startup and resume schedule the first relay ping round immediately after the first nonempty announcement publishes, matching JS start() awaiting the initial update before starting its background loop. Later rounds retain the three-second pause after all replies/timeouts settle. Reannounce and later refresh-cycle timing are unchanged; no public API, wrapper or wire-format changes are needed.
-
2026-11-05 persistent forwarding hints: a persistent node's live router forwarding address replaces cached relay hints for the first FIND_PEER pass, matching JS connect.js:327-333. Ephemeral nodes, local handlers without a forwarding address and missing/expired/other-target entries retain the cache fallback. Normal discovery still follows an unsuccessful first pass. Cache, router expiry, query budgets and public APIs are unchanged.
-
2026-11-05 cached connection discovery: cached relay addresses seed a FIND_PEER-only first pass with one retry and no table/closer-node expansion, matching JS connect.js:320-368. Handshakes start from valid peer replies. When that pass and its handshakes finish without a connection, normal discovery starts with three retries. Stale, empty and filtered hints retain fallback; shutdown cannot start another pass. Existing helper symbols and general query behavior are preserved. Persistent-router hint priority is covered above, along with explicit relay-address options.
-
2026-11-05 connection discovery retries: the normal connection FIND_PEER walk uses three retries (four sends), matching JS connect.js:347, instead of inheriting the general query default of five retries. The original find_peer overloads retain their defaults; an additive overload sets the budget before the first send. Cached-relay seeding is covered above.
-
2026-11-05 sampling target deduplication: each address-discovery campaign selects distinct host:port endpoints, matching JS nat.js's visited-node gate. A relay also present in the routing table no longer receives two PINGs or crowds out another target. Bootstrap fallback uses the same gate. The seven-target margin, selection order and cross-campaign policy are unchanged.
-
2026-11-05 sampling ping retries: address-discovery PINGs now send once, matching JS nat.js
retry: false. An additive PoolSocket request overload selects this behavior; the original overload and ordinary holepunch RPCs retain their four-send budget. Response timeouts, target selection, classification/early completion and campaign retry policy are unchanged. -
2026-11-05 pooled RPC retries: default PoolSocket requests now allow one initial send plus three retries (four total), matching JS dht-rpc's
retries = 3. Previously the total was three. Retransmissions retain the same TID and encoded request; adaptive timeouts and overall connection deadlines are unchanged. The sampling override is covered above. -
2026-11-05 announcer keepalives: the periodic relay ping interval is now three seconds instead of five, matching JS's three-second idle delay. Ping rounds do not overlap; the three-second pause starts after every reply, timeout or queue rejection settles, matching JS's await-then-pause ordering. Timer ticks still run the cycle watchdog while a round is pending. The reannounce timer, watchdog threshold, drift handling and wire/API formats are unchanged; first-ping timing is covered above and later refresh-cycle timing remains separate work.
-
2026-10-28 client reconnect hints: successful connections cache the server's advertised relayAddresses in preference to discovery responders, matching JS connect.js:464-468. Empty advertised lists retain the discovery fallback; failed connections leave existing hints unchanged. Cache timing, reconnect scheduling, socket reuse and the C ABI are unchanged.
-
2026-10-28 public-server relay hints: successful public handshakes prepend the validated public endpoint to relayAddresses, matching JS announcer.js. NAT replies retain their relay-only hints; holepunch relay pairs and announced records are unchanged. This helps clients that prefer advertised hints. Client preference for the advertised list is covered above.
-
2026-10-28 contextual error descriptions: additive C/JNI helpers describe query completions, immediate submission statuses and raw libuv errors separately. Kotlin storage failures use query messages; rejected calls, addNode and serverListen include the operation, description and unchanged numeric code. Existing connection messages/codes, callback signatures and wire behavior stay unchanged. Updated Kotlin sources require the matching JNI/core build.
-
2026-10-28 immediate Kotlin/JNI errors: connect, immutable put/get, mutable put and ping check native submission status before awaiting callbacks. Rejected calls throw instead of hanging; accepted callbacks are unchanged. Both owned Kotlin copies match. Native code and the C ABI are unchanged; context-correct descriptions are covered by the separate item above.
-
2026-10-27 read completion errors: immutable/mutable get results retain the existing query completion status, and C get/get_ex callbacks forward it. Explicit C cancellation keeps priority; ordinary misses remain successful. Kotlin immutableGet rejects a nonzero completion instead of returning null. No query, timeout, early-success or wire policy changes are introduced. C++ consumers must rebuild for the appended result fields; the C ABI is unchanged. Immediate JNI returns and general error descriptions remain open.
-
2026-10-27 C/Kotlin connection errors: C++ terminal server-handshake failures map to the additive C/Kotlin
SERVER_ERRORcode (-9), with a native description. Existing C++ codes and C query cancellation (-8) are preserved; server errors were previously mislabeled as cancellation by C consumers. All C connect entry points use the mapping; JNI forwards the C result. Other query/storage errors and synchronous JNI status handling remain open. -
2026-10-27 direct socket results: OPEN, no-holepunch and LAN results report the main UDX socket already selected by consumers' fallback. This enables the existing route cache when both sides opt in. No socket ownership or C ABI layout change is introduced. A non-relayed cached main route completes directly after its handshake, matching JS instead of entering holepunching; actual firewall observations retain priority and relay setup retains its order. Pooled-socket cached retry remains separate work.
-
2026-10-27 cache insertion: retain the first live route for each peer, matching JS socket-pool.js. Later successful sockets are marked reusable without replacing that route; removal or socket closure permits a fresh one. Existing stream-error ownership guards and cached transport selection remain.
-
2026-10-27 stream-error routes: DHT duplex options observe failed streams before consumer close callbacks and invalidate only the matching peer/stream route. Normal close, healthy peers sharing a socket, newer route replacements, and callbacks after DHT destruction are protected. C++ consumers should use make_duplex_options(); the C wrapper already does. C ABI signatures stay the same; C++ consumers must rebuild for the appended option/cache metadata. This does not change cache replacement policy or pooled-socket selection.
-
2026-10-27 cached-route replies: authenticated terminal handshake errors stop the connection with SERVER_ERROR before findPeer, matching JS's cached route path and the existing normal C++ handshake path. Malformed outer replies and corrupt Noise replies retain lookup fallback. Other socket-reuse and recovery items remain open.
-
2026-10-26 forced-persistent startup: generic
ephemeral=falsestarts ephemeral and awaits the existing address/firewall check during bootstrap, matching JS_forcePersistent. Success adopts the verified endpoint's routing ID before ready; failure stays ephemeral. The explicit bootstrapper factory retains its trusted-address initialization. The full two-pass bootstrap loop remains deferred; this change does not add it. -
2026-10-26 storage put results: immutable/mutable put results expose the existing query commit error, and the C callbacks forward that status instead of always reporting success. Callback signatures and the C ABI stay unchanged; C++ consumers must rebuild for the appended result field. This matches JS puts awaiting query.finished(); transport-level replies retain the existing JS success policy. Other query/storage gaps remain open.
-
2026-10-25 lookup results: lookup and lookup-and-unannounce validate the complete peer array and optional bump before forwarding raw reply bytes or deciding to withdraw, matching JS mapLookup. Closest routing replies and the original decoder entry point remain available. Query/storage failure reporting remains open.
-
2026-10-25 findPeer results: the shared findPeer callback path decodes peer records before forwarding raw reply bytes, matching JS mapFindPeer's rejection of codec failures. Routing/closest replies remain unfiltered, so missing records still supply announcement relays and tokens. The existing unchecked decoder remains available; the checked overload reports failure. Lookup decoding is covered above; query/storage failure reporting remains open.
-
2026-10-25 graceful shutdown: Announcer, Server, HyperDHT and C API completion now wait for pending announcements and withdrawals to settle. Force cancels the wait; callers still retain storage through libuv's close drain. Nospoon/Holesail teardown and bounded Rust cleanup follow that contract. The filtered core suite (795 pass, two skips), twenty focused sanitizer cases, consumer suites, both static CPU builds and the ESP32 component compile pass. Sixteen selected Rust tests pass; the separate pre-existing
drop_midflighttest has a stale API call and was not repaired. Fifteen isolated live cases pass: withdrawal success/timeout, public/NAT/LAN consumer connectivity and 33 clean native exits. ESP32 firmware/runtime and broader platform checks remain unverified. -
2026-10-25 retired relays: a successful replacement cycle withdraws from relays it no longer uses, then allows the next refresh. Failed empty cycles retain the working set; force-stop cancels a pending withdrawal. Nine focused checks pass with ASan/UBSan and leak checking. Core and consumer suites, both static CPU builds and the ESP32 component compile pass. Fifteen isolated live cases pass, including replacement/failed-replacement controls, encrypted echo, fresh Nospoon/Holesail connectivity and 35 clean native app exits. Awaited whole-DHT shutdown is covered above.
-
2026-10-25 withdrawal tokens: stop-time withdrawal obtains a fresh FIND_PEER token and replying node ID before signing UNANNOUNCE. Copied signing state survives Announcer release; restarting cancels an old pending withdrawal before it can erase the resumed record. Seven focused regressions pass with ASan/UBSan and leak checking. Core and consumer suites, both static CPU builds, the ESP32 component compile and fourteen isolated live cases pass, including 31 clean native app exits. Retired-relay cleanup is now covered above; awaited whole-DHT withdrawal is covered above too.
-
2026-10-24 UDX shutdown: close now tears down attached and unconnected streams before their DHT storage is released. Generated UDX guards avoid duplicate closes and watcher callback lifetime errors; pending connect owners and pool retries stop referring to finalized handles. Core and consumer suites, both static CPU builds, ten Holesail TCP/UDP placements and three Nospoon public/NAT/LAN placements pass, including normal application exits. TD2 is fixed; TD1/TD3 and broader platform/soak verification remain open.
-
2026-10-24 announcement identity: ordinary stored announcements now replace and remove by verified peer public key within each target, including refresh and local clear. Signed UDP storage and public/NAT consumer checks pass; see the announcement identity entry below.
-
2026-10-24 relay verification: blind-relay pairing, OPEN control sockets, transport-write failure propagation and handshake dedup are fixed. Live relay-to-direct migration passes with both a public and a NAT application server, preserving encrypted payload and the original streams; see Section J.1.
-
2026-10-13 documentation check: the protomux reentrancy UAF is fixed (
3421264), and the client relay-address cache is implemented (c196205); their entries below now reflect current source. Relay-to-direct upgrade code is present on both client and server, but blind-relay pairing remains blocked by BUG A (Section J.1), and the live upgrade test remains open. This check does not revalidate every historical finding or test result. -
2026-07-31: announcer relay-set freeze FIXED (
6ef85e6gate split +82d3c55per-cycle rebuild, LOCAL, NOT PUSHED). The published relay set was write-once — the ANNOUNCE callback could only refresh an entry it already had or append while under PICK_BEST, andactive_relays_was otherwise cleared only on stop — so a dead or third-party-unreachable relay was advertised for the life of the process and no healthier node could replace it. That is the mechanism behind every "a restart fixed it" in the field handoff. JS rebuilds per_update(announcer.js:172,188). Fix = stage intocycle_relays_, swap inmaybe_publish()guarded against empty; prerequisite gate split so a client holding a previous-generation record still gets a holepunch token (JS unions three_serverRelaysgenerations, announcer.js:38-42). Also fixed a pre-existing bug found while tracing:start()never re-armed thealive_sentinelstop_impl()clears, so a suspended-then-resumed server never re-announced again. 730/730, ASAN clean, cpp-reviewer SHIP on both. Six tests, all red-checked; the guard isIsRelayStillAcceptsPreviousGenerationAddress. -
2026-07-30 (later): field Finding M FIXED — Round 1 now fails over to the remaining announce relays instead of treating one dead relay as terminal. Deliberately beyond-JS (it is the fix JS's own TODO asks for), capped at
PICK_BEST - 1legs after cpp-reviewer flagged the 128-entry wire cap as a retry-amplification stall. 722/722, ASAN clean, SHIP. Round-2 failover (JS's second TODO) still open — needs field evidence. -
2026-07-30: field Finding M logged as the new P0 (Section A) — one unreachable relay in the announce is a permanent unrecoverable
-5because neither implementation fails over torelays[1..n]. 81/81 repro, reproduced on nospoon-JS too, and upstream JS carries the TODO asking for the fix in two places. NOT a parity bug — the fix is a deliberate beyond-JS improvement. -
2026-07-29 (later): field Finding J FIXED —
udx_stream_destroyhas noUDX_STREAM_DESTROYINGguard, so a second destroy inside the deferred-close window ranclose_stream_internal()twice and aborted in libuv. Now guarded byudx::destroy_stream_once(). Reproduced standalone and fixed; 719/719. Surfaced a separate pre-existing UAF inprotomux.cpp:242(logged below). -
2026-07-29: field Finding I FIXED — the advertised local-address list was a bind-time snapshot, permanently breaking same-LAN connects (
-6) after any interface change; now re-enumerated live per advertisement (HyperDHT::local_addresses_now()), matching JS_localAddresses(). Also recorded new field Findings J (HIGH,uv_closedouble-close crash) and K (sampling gate), and DEMOTED B1 — Finding L exonerates it as the-5cause, promoting B2. Suite 718/718. -
2026-07-28: C-API SECURITY fix —
hyperdht_server_set_firewallpolarity was inverted, so every C-API server with a firewall admitted exactly the peers it should reject (0203820, found via holesail-cpp; regression testsServerFfiFirewall.*red-checked). Raises the priority of the un-auditedffi_*layer (Section F). Also landed read-side backpressure (hyperdht_stream_pause/resume) + thereusable_socketconnect option (a08cade, Section D). Suite 716/716 unit. -
Adversarial re-verification 2026-07-26 (16-subsystem Opus workflow): claimed status CONFIRMED accurate — zero FALSE_DONE, all 12 HIGH genuinely closed, all 6 claimed-open genuinely open, ~93% behavioral parity. Added 1 HIGH new hazard (server/divergence-1 UAF, NOT fixed), 3 LOW divergences + io-6 DONE→PARTIAL, re-tagged 6 stale-OPEN appendix entries as DONE, reframed B1 as beyond-JS (not a parity bug). Detail in Section A. messages + protomux were UNVERIFIED this pass (finder output-cap failures).
-
JS-parity sweep 2026-07-09: 91 confirmed findings (12 HIGH / 45 MED / 34 LOW). All 12 HIGH closed. ~86/91 addressed (fixed or accepted-divergence) (io-6 demoted to PARTIAL 2026-07-26 → ~85/91). 6 MED/LOW still open (Section A) + 4 new 2026-07-26 findings.
-
2026-07-22 batch (on main): server-1 (firewall-reject now SILENT — presence leak closed), server-2 (holepunch reply deferred past veto+punch(), encrypted ABORTED on failure), server-3 (synchronous handshake dedup + duplicate queueing), connect-3 (reusableSocket client wiring), connect-4 (route-shortcut payload), connect-5 (bogon filter + serverAddress fallback), connect-6 (LAN shortcut exclusive, JS trigger via client_address), connect-7 (client holepunch veto), announce-4 (record signs empty relayAddresses — client independence proven both sides), announce-6 (router FROM_RELAY-only handler gate). Suite 702/702; cpp-reviewer SHIP, ASAN/UBSAN clean.
-
2026-07-21 batch (on main): announce-5, announce-7, server-8, connect-8 (SECURITY: handshake reply validation + new
Response::remote_addrsource check — RPC matched responses by tid only), connect-10 (client relay-chain teardown; also broke a shared_ptr cycle that leaked ConnState on dead pairings), birthday-win SocketRef keepalive pin (holepunch.cpp:1450 residual). Suite 684/684 (excl. live); cpp-reviewer SHIP, ASAN/UBSAN clean. -
Prior sweep commits (merged to main 2026-07-11):
f266b35deps bump,88b8022compact/routing,5460cfacore sweep + relay→direct upgrade,f37ef1dwrappers,86353dbdocs. -
Detail docs:
docs/PARITY-SWEEP-2026-07-09.md(FREEZE table + per-subsystem notes + REMAINING OPEN),docs/.parity-sweep-appendix.md(all 91 with JS/C++ file:line),docs/RELAY-UPGRADE-PORT.md(#266 port design).
Freeze-ready (fully swept): noise, compact, routing, tokens-nat, messages, blind-relay, dhtrpc-io, protomux, query, dhtrpc-tick, dht-top. Partial (HIGHs only): connect, server, router-announce, secret-stream.
Remaining: connect-9, server-4 (ACCEPT candidate — likely just document),
server-6/9/11, secret-stream connect-1 (deferred, needs live JS cross-test).
Full text + JS/C++ file:line for each is in docs/.parity-sweep-appendix.md.
Full run: 14/16 subsystems (messages + protomux finders failed on output-cap, UNVERIFIED this pass), 11 material findings, ALL skeptic-confirmed (high conf). The 12 HIGH and all 6 claimed-open items above survived independently — zero FALSE_DONE. Behavioral parity estimate ~93%. Net-new, none previously ledgered:
- server/divergence-1 (HIGH — potential UAF) — fixed 2026-10-23
(
e6adc4c). The OPEN-client direct shortcut returns before starting blind relay setup, preventing a relay continuation from reusing the emitted stream. The regression requires one direct emission and no relay pairing; all three suites and isolated public/NAT connectivity checks pass. - connect/divergence-1 (LOW) — fixed 2026-10-27. The cached-route
callback applies the normal handshake's
hs.terminalcheck and reports SERVER_ERROR without a findPeer walk. Previously the regression returned PEER_CONNECTION_FAILED after a lookup; retryable replies still fall back. - connect/divergence-2 normal direct handshake — rechecked and fixed 2026-11-06 (lab compatibility checked). JS connect.js:499-501 completes non-relayed handshakes before its holepunch function; gating the no-holepunch branch alone would miss that control flow. Ordinary C++ discovery now uses the existing main-socket direct completion too, rather than punching an already reached server or preferring advertised addresses. Cached-route and relayed paths are preserved.
- compact/buffer64k (LOW — doc-only) —
Buffer::decode64KB cap (compact.cpp:239-249) vs JS uncapped; fails SAFE (rejects a superset), H12 anti-DoS. Add a ledger line like compact-2's array cap. No code change. - dhtrpc-io-6 first external request — fixed 2026-11-06: after a valid
new node is admitted, its first request feeds the address samplers even on
the inactive socket, matching JS
_addNode(index.js:533-536). Known peers on that socket remain excluded, as do rejected nodes. Existing expected- socket sampling and separate age-based resampling work are unchanged.
Docs caught up (code was AHEAD): 6 appendix entries were labeled OPEN but
are FIXED + skeptic-confirmed — now tagged [DONE — verified 2026-07-26] in
docs/.parity-sweep-appendix.md: holepuncher-1 (birthday wired; symmetric-CGNAT
stream completion still live-unvalidated), query-2, dhttop-1, dhttop-2,
dhttop-6, dhttop-8. Headline 86/91 unaffected (already excluded them).
- connect-8 — DONE 2026-07-21. Handshake reply now validated
(mode==REPLY, source-address match via
Response::remote_addr, version/ error/udx checks); JS-terminal failures fail the connect withConnectError::SERVER_ERRORinstead of retrying relays. - connect-10 — DONE 2026-07-21. 15s timer now tears down the relay chain
(
abort_relay_chain), breaking the pair-callback↔ConnState shared_ptr cycle; deferred teardown from client-callback frames via 0ms re-arm. - connect-3 — DONE 2026-07-22. reusableSocket threaded: ConnState + UdxInfo advertise flag + route-cache read/write gated on both-sides opt-in.
- connect-4 — DONE 2026-07-22.
build_local_handshake_infoshared by fire_handshake and the route shortcut (one payload for all attempts). - connect-5 — DONE 2026-07-22.
is_bogon/is_reserved(npm bogon port); direct paths use first-non-bogon + serverAddress fallback. - connect-6 — DONE 2026-07-22. LAN shortcut EXCLUSIVE of holepunch, JS
trigger (
client_address.host == server_address.host, onlyNonReserved filter, ping-fail aborts). NEEDS LIVE LAN VALIDATION (Section C). - connect-7 — DONE 2026-07-22.
opts.holepunchveto invoked after probe round, before punching; abort maps to HOLEPUNCH_TIMEOUT. Deviation: abort is local-only (no ABORTED round to the relay — see Section G). - connect-9 normal walk retry budget — three retries, matching JS connect.js:347 (2026-11-05). General findPeer queries retain five retries.
- connect-9 cached relay first pass — cached-only FIND_PEER with one retry, followed by a normal walk after the query and pending handshakes settle (2026-11-05). Query filtering, cancellation and normal-walk controls covered.
- connect-9 persistent-router hint priority — live forwarding addresses override cached hints on persistent nodes (2026-11-05).
- connect-9 explicit relay-address seeds — honor the existing connection option ahead of cache, retaining persistent-forward priority (2026-11-06).
- server-1 — DONE 2026-07-22. Firewall-rejected handshake now sends NOTHING (presence leak closed); rejected session stored only for dedup-silence, reaped by the clear-wait timer.
- server-2 — DONE 2026-07-22. Holepunch reply committed only after veto +
punch()-started; failures send encrypted ABORTED (
encode_abort_reply). Deviation kept: immediate clear_session vs JS ~10s defer (Section G). - server-3 — DONE 2026-07-22. Dedup entry written same-tick before the
async firewall dispatch; duplicates queue on
pending_handshakes_and all get the same reply (or silence) on resolve. - server-4 —
MAX_PENDING_HANDSHAKES=256cap silently drops. ACCEPT candidate (anti-DoS, like the other caps we kept) — likely just document. - server-6 —
neverPunch(opts.holepunch === false) not implemented. - server-8 — DONE 2026-07-21. 15s pairing watchdog (
relay_pending_+abort_relay): tears down the relay chain only, session + puncher keep running (JS onabort parity); pair-error no longer clears the session. - server-9 — server-side same-host LAN match (server.js:414-426) not implemented.
- server-11 endpoint — OPEN-client shortcut uses the observed client endpoint regardless of advertised addresses (2026-11-06).
- server-11 relayed socket — pass the active socket for a relayed OPEN-client handshake (2026-11-06).
- server-11 direct socket — propagate JS's arrival socket on a direct handshake. Direct handshakes retain the existing null socket and consumer fallback.
- announce-4 — DONE 2026-07-22. Announcer signs/stores empty relayAddresses (JS parity); proven no client (JS or C++) reads relays from the record — they come from the responding DHT node + the handshake payload. Dead re-announce-once machinery removed.
- announce-5 — DONE 2026-07-21. Server-host FROM_SECOND_RELAY reply now routed to the embedded relayAddress (first relay), dropped when absent.
- announce-6 — DONE 2026-07-22. Holepunch handler now FROM_RELAY-only with
the
!peerAddressdrop (router.js:221); FROM_CLIENT/FROM_SERVER route through the pure-relay path (self-hosting server reaches its own handler via one self-hop, like JS). - announce-7 — DONE 2026-07-21.
handle_refreshports persistent.js_onrefresh: refresh hashes stored on full announce, preimage verified, record re-added, chain rotated. (Latent feature — current JS always sends refresh:null.)
- Finding I — server advertised a BIND-TIME local-address snapshot.
DONE 2026-07-29.
validated_local_addresses_was assigned only inbind()(dht.cpp) and never recomputed, so an interface appearing later (DHCP completing after an early bind, interface flap) was never advertised for the process lifetime → same-LAN clients failed permanently with-6. Fatal rather than degraded because the LAN shortcut is EXCLUSIVE: a badaddresses4removes the holepunch fallback too (that exclusivity is correct parity — connect-6 — so the fix is the address list, NOT the fallback). Genuine JS divergence:_localAddresses()(server.js:206-208) re-enumerates per handshake (server.js:272). Fix: newHyperDHT::local_addresses_now()— live enumerate →validate_local_addresses()→ refresh snapshot; used by the server handshake reply and the client'sbuild_local_handshake_info(memoized perconnect()onConnState, matching JS's one-payload-per-connect latch, connect.js:386). VERIFIED with a netns repro (interface brought up after bind: before the fix the advertised list stayed127.0.0.1even after the interface watcher polled; after, it tracks the live address). TestsLocalAddressesNowMatchesLiveEnumeration+LocalAddressesNowRespectsExcludeLocalAddress(the latter red-checked against a naive bypass — live re-enumeration MUST stay routed throughvalidate_local_addresses()or it resurrectsexclude_local_address()'d hosts, which nospoon relies on to hide its TUN address). 718/718, cpp-reviewer SHIP, ASAN clean. - Finding J (HIGH — hard crash) — DONE 2026-07-29.
uv_closedouble-close abort in~ConnStateteardown. ROOT CAUSE:udx_stream_destroy(deps/libudx/src/udx.c:2709) guards onUDX_STREAM_CLOSEDbut NEVER onUDX_STREAM_DESTROYING. On the slow send path (uv_udp_try_send→UV_EAGAIN, which happens whenever ANY other send is queued on that socket — common on the busy shared DHT-RPC socket)close_stream_internal()is deferred to the send completion, leaving the stream DESTROYING-but-not-CLOSED and still CONNECTED for a loop turn. A second destroy in that window queues a second destroy packet and runsclose_stream_internal()TWICE, whichuv_close()s the stream's 5 timers twice → libuvassert(!uv__is_closing(handle)). libudx's ownassert(CLOSED == 0)is compiled out in Release — exactly where the field crash was seen. Fix:hyperdht::udx::destroy_stream_once()(include/hyperdht/udx.hpp) skips whenDESTROYING|CLOSED; applied to~ConnStateandServerConnection's dtor +operator=. Skipping is always correct — such a stream already finalizes itself. INDEPENDENTLY REPRODUCED by cpp-reviewer with a standalone program against the vendored libudx (forcedUV_EAGAIN, double destroy → assert in Debug, SIGSEGV in Release; guarded → clean,finalize_count == 1). TestDestroyStreamOnceSkipsStreamAlreadyBeingDestroyed(red-checked). 719/719, cpp-reviewer SHIP. NOTE:blind_relay.cpp's destroy sites are NOT exposed — for arelayedstreamudx_stream_destroycallsclose_stream_internalsynchronously in one call (upstreamfa576bdmade it so deliberately), so there is no window. - Finding J follow-up — DONE 2026-10-24. The generated UDX copy
now guards
DESTROYINGatudx_stream_destroy()itself, coveringSecretStreamDuplex::destroy()and bulk teardown without changing the pending close callback that deliverson_udx_close→fire_close. - protomux reentrancy UAF — FIXED (
3421264; source rechecked 2026-10-13).Channel::open()retains itself withshared_from_this()across synchronous transport callbacks; Mux channel ownership is shared. This closes the reported OPEN/reentrant REJECT lifetime hazard, not the separate server/divergence-1 finding above. - Finding K — the pool-NAT sampling gate does not hold:
pool_fwis read mid-flight at holepunch.cpp:1758, while the comment at holepunch.cpp:1777-1779 asserts sampling has completed by then. Field log showsSampling settledfiring 1 ms BEFOREsampling done, with disagreeing values (read fw=3, settled fw=2); ordering varies run to run. Effect on connect outcome NOT established (n=1 each way — do not assume a direction). Minimum: enforce the invariant or correct the comment. - Finding M — DONE 2026-07-30 (round 1 only; round 2 deliberately left
alone, see below).
holepunch::pick_fallback_relays()hands the announce record minus the chosen relay toholepunch_connect(..., fallback_relays);fail_round1()pops the next leg and re-runsrun_round1on the SAME pool socket, puncher and NAT samples — only the relay changes. Terminal only once the list empties, so the single-relay path is unchanged. Scope matched to JS: the six Round-1 sites that JS reaches by throwing out ofprobeRoundinto the retry catch now fail over (request timeout, bad reply, empty payload, decrypt failure, remote error, no server address); everything JS reaches viaabort()(double-random NATs, unstable NAT, probe exhaustion) stays terminal. Round 1's two addresses had to be split: the message'speer_addressis the relay's forward destination (router.js:214 uses it verbatim) and moves with the relay, while the payload'sremote_addressstays the fresh observation the server matches for its fast-mode punch (server.js:530-538). Attempt 0 seeds both from the same value, so it is byte-identical to before. SECURITY (cpp-reviewer): the failover list is capped atannouncer::PICK_BEST - 1, NOT the wire cap of 128 relays (peer_connect.cpp:175) — that bound was sized for parse safety back when extra entries were ignored, and each entry now costs a multi-second RPC, so uncapped it let a hostile connect() target stall its caller for ~6 minutes with garbage address bytes. Honest servers announce ≤3 (pickBest), so the cap costs no real connectivity. Tests:ConnectRelayFailover.{DeadRelayFallsOverToTheNextRelay, NoFallbackRelaysStaysSingleAttempt, FallbackListSkipsTheChosenRelayAndIsCapped}(first one red-checked). 722/722 unit; ASAN/UBSAN clean; cpp-reviewer SHIP. STILL OPEN: JS's second TODO (connect.js:312, round-2 failure). C++ already treats a round-2 relay timeout as non-fatal because probing is under way by then, so the remaining gap is only a round-2 error reply; failing that over means re-running round 1 for a fresh token. Needs field evidence first. Original report follows. -
Finding M (2026-07-30, P0 — best-evidenced item in the handoff)— ONE unreachable relay in the announce = permanent-5, with no failover to the other two. 81/81 deterministic field repro (81 × round-1 TIMEOUT to the same relay, 0 successes) while six other pool nodes queried in the same millisecond all replied — so not a network block. ROOT CAUSE: the chosen relay sits behind an endpoint-dependent (symmetric) NAT, so the server's mapping for it (:58044, in the announce record) is useless to a third party; the client's own observation of the same host is a different port (:45262). The server's 5 s keepalives ride the server's own mapping, so the entry looks healthy from the server — the announcer cannot detect this; no server-side liveness check helps. NOT a C++ bug and NOT a parity divergence — cross-implementation control: nospoon-JS on the same machine/network/seed fails identically (HOLEPUNCH_ABORTED×6). VERIFIED IN BOTH SOURCES 2026-07-30: selection is faithful parity (connect.cpp:781-790vspickServerRelayconnect.js:812-817 — host+port match elserelays[0]), and the bail-out is terminal on both (holepunch.cpp:2059-2062state->complete({})). Upstream JS asks for this fix in its own source, twice (verified verbatim): connect.js:271// TODO: we should retry here with some of the other relays, bail for nowand connect.js:312// TODO: retry with another relay?. FIX: on round-1 (and round-2) failure, fail over torelays[1..n]instead of completing terminally. Deliberately BEYOND-JS, but it is the fix JS itself requests. REJECTED alternative — do NOT substitute the client's own observation of the same host (:45262).relays[i]is not an address, it is a live server↔relay UDP session, andprobeRoundsends(peerAddress, relayAddress)as a unit (connect.js:561). JS documents why at connect.js:280-285: "If the relays were different, then the server would not have a UDP session open on this address to the client relay, which round2 uses." Port-swapping trades a round-1 timeout for a round-2 failure. Failover is safe precisely because eachrelays[i]carries its ownpeer_addressand its own server-maintained session. Operational note: nothing client-side recovers; the server must re-announce (restart) to draw a fresh relay set.
- The real defect was the announcer relay-set FREEZE — fixed
2026-07-31 (
6ef85e6,82d3c55; see the status snapshot). A bad relay roll persisted for the process lifetime because nothing could ever replace it, which is why restarting the server "fixed" it. The controlled A/B in the handoff (run 1: three slots on one symmetric-NAT host, total failure; run 2 after restart: three distinct hosts, instant success) is explained by the re-roll, not by diversity per se. - PARKED by decision 2026-08-03 (Vaios). Revisit in the mid-term, or
sooner if upstream changes
pickBest. Rationale: it is local selection policy, invisible on the wire, so deferring costs nothing in interoperability; and the announcer rebuild (82d3c55) already removed the permanence that made a bad roll fatal. Wake-up triggers, in order: (a) upstream implements thepickBestRTT TODO — note RTT alone would make this WORSE, since all mappings of one host share an RTT and would then win all three slots deterministically rather than by luck; (b) a single-host relay set recurs in the field post-rebuild; (c) a-5outage that survives a relay-set re-roll. On revisit, check the statistical premise FIRST (see the note below) — if three mappings of one host really are the global closest-3 more than rarely, the selection/sort is the bug and a diversity constraint would only mask it. - Host diversity itself — OPEN, and NOT a parity bug. The audit
(2026-07-31 workflow) refuted the parity framing: JS
pickBestisreplies.slice(0, 3)(announcer.js:298-301) with no host notion either, so stock JS has the identical gap. Any distinct-host constraint on the announce commit is a deliberate BEYOND-JS hardening change. Hold until the rebuild has field time: the freeze made every bad roll permanent, so re-measure how often a single-host set actually recurs before adding a constraint that could reduce availability in small/clustered DHT regions. Note the premise is also unexplained —id = BLAKE2b(host, port)makes each mapping an independent random point, so three mappings of one host being the global closest-3 is suspicious in itself; if it recurs, suspect the selection/sort, not diversity. - Announcement record identity uses verified public keys (2026-10-24). Within each target, full announce and refresh replace the same key after an endpoint change; different keys sharing an endpoint coexist. Signed unannounce removes only its verified key, even from a different endpoint. Self-announce cleanup and local lookup-and-unannounce also remove by key. The stored sender address remains metadata; wire records, C ABI, TTL and capacity are unchanged. Regressions cover replacement, coexistence, key isolation, refresh, local clear, expiry and capacity. A private persistent C++ node verifies six UDP storage operations with matching request/reply payloads in both endpoint captures. All three suites and Nospoon connectivity checks pass with a public server and a NAT server; x86_64/aarch64 static builds and all 35 ESP32-S3 component files compile. Firmware/board checks remain deferred. Low-level C++ store users must rebuild and supply the peer key; the C interface is unchanged.
- Router forward cache bounded and expired (2026-10-23). Remote
forwards use the existing LRU cache with
DhtOptions::max_sizeandmax_age_ms; defaults remain 65536 entries and 20 minutes. Expiry applies on lookup and periodic GC; refresh resets age. Local listening Server handlers stay separate so remote announcements cannot replace or evict them. Focused regressions cover capacity, expiry, refresh, signed announcements and listener retention. The full filtered unit suite passes in Debug and Release; consumer and isolated connectivity checks accompany this security change. - Firewall-probe address snapshot (2026-11-13). Before promotion or sampler replacement, require the client sampler's host/port to match the probe-start snapshot and the node to remain ephemeral (JS index.js:827). A successful probe still confirms reachability; changed-address samples survive for a later evaluation. Direct C++ RpcSocket consumers must rebuild.
- Persistence host memo (2026-11-13). Remember the checked host even when its address is incomplete, matching JS index.js:805-815. The countdown then uses that observation after host/port discovery is restored.
- RECHECK: the persistence gate is one-shot and cannot demote (context
for Finding O and for the item above; no action decided yet). Becoming a relay
requires passing the PING_NAT firewall probe once — five routing-table nodes
reply to our server socket, plus the port-preservation check. That proves
those five could reach that port at that moment; it does NOT establish
endpoint-independence, and nothing ever re-runs it to demote (same
promote-but-never-decrement property the B1 audit found in JS
nat.js_updateFirewall). When a NAT mapping later moves, the node re-registers under a NEW id, becauseid = BLAKE2b(host, port)— so each remap enters the routing table as a fresh, legitimately-persistent node. That is exactly how82.29.173.47appeared under nine distinct ports in one day and how one physical host could occupy multiple announce slots (Finding O). Any host-diversity work should be designed against this, not against the naive "one node = one entry" model. - Fresh-token stop-time UNANNOUNCE — fixed 2026-10-25. FIND_PEER
obtains the current token and node ID before signing UNANNOUNCE, matching
JS's two-request
_unannounce(announcer.js:205-238). Reject incomplete/error replies; preserve copied signing state after Announcer destruction; cancel old withdrawal continuations on restart. Signed-handler regressions and a live aged-token negative/control prove the record is actually removed. - Per-cycle UNANNOUNCE — fixed 2026-10-25 (announcer.js:191-196). Withdraw relays absent from the replacement set with fresh-token requests; await their settlement before another cycle. Retain the old set when a cycle commits nothing, and preserve the lenient relay gate. Regressions and isolated live positive/failed-replacement controls cover removal, retention and force cancellation. An unreachable relay still expires its record normally.
- Await graceful withdrawal on whole-DHT destruction — fixed 2026-10-25 (JS index.js:123-127, server.js:88-129, announcer.js:88-104). Completion waits for pending ANNOUNCE commits, per-cycle retirement and stop-time withdrawal before closing transport. Force cancels the wait. Completion still precedes libuv close draining; callers retain storage for that drain. Regression, sanitizer and isolated live success/timeout controls pass, alongside rebuilt Nospoon/Holesail public/NAT/LAN connectivity and clean shutdown checks.
- Keepalive/update ordering — new ping rounds wait for cycle settlement, matching JS's awaited update; watchdog ticks remain active (2026-11-06). Periodic updates also wait for in-flight ping settlement; ordinary refreshes share that wait (2026-11-18). Successful-cycle wake timing and remaining exact background-loop cadence are covered above.
- Refresh during keepalive — coalesced until round settlement
(2026-11-18). This does not add a mid-update latch: JS clears
_refreshingafter its update before starting the next background round. Deliberate immediate drift recovery remains separate.
- Finding E — DONE 2026-07-22 (uncommitted): announcer
updating_latch → server stopped reannouncing. Root leak:Query::visit()didn't settle a tid==0 (congestion/closing) request-drop → walk never completed → cycle wedged. Fixed the walk drop-settle + added a 60s stuck-cycle watchdog on the ping timer. TestRecoversFromWedgedCycleViaWatchdog(red-checked). - Finding H (pickBest) — DONE 2026-07-26 (announcer.cpp
update(); see handoff Finding H, both-ends field capture + code-confirmed). The announce record was committed to EVERY walked node (~41) instead of JSpickBest(q.closestReplies)= closest 3 (announcer.js:170,298-301). The ~38 extra record-holders had EXPIRED server-forward NAT mappings (only the 3 kept-aliveactive_relays_forward), so a client hunted dead relays ~28s before hitting a live one. Fix: commit only the closestPICK_BEST(3) replies inon_done(full closest set still saved for reseed). Pure JS- parity restoration. TestCommitsToPickBestThreeNotAllClosest; 709/709 unit + cpp-reviewer SHIP + ASAN clean. NEEDS LIVE VALIDATION (Section C): mobile connect ~28s → ~1s. Upstream of the punch; independent of B1/B2. - B1 (BEYOND-JS field-hardening — NOT a parity bug) — NatSampler
classifies at
sampled_ >= 3(nat_sampler.cpp:111);MIN_SAMPLES=4feeds anokflag nobody reads. Three agreeing samples latch CONSISTENT and cannot be demoted. 2026-07-26 re-verification CORRECTION: this latch is JS-IDENTICAL —nat.js _updateFirewallmax-hits only ever grow, never decrement — so stock JS fails the same CGNAT case. B1 is therefore a DELIBERATE divergence, not a parity gap. Fix = gate the verdict on >=4 + let disagreeing samples demote a stale CONSISTENT, to engage the wired birthday strategy on port-varying CGNAT. 2026-07-27 DEMOTED — field Finding L EXONERATES B1 as the-5cause. The "reasonable candidate for the live-5" reading is WITHDRAWN. Mobile capture: all 6 SUCCESSES loggedpool_fw=2— B1's exact misdeclaration, with the client mapping equally unstable — so B1's precondition is present in the successes; and the one attempt that classified CORRECTLY (pool_fw=3RANDOM, i.e. what fixing B1 produces) still failed-5. B1 discriminates in neither direction. Keep it as a remedy (a client whose mapping moves mid-handshake wants the birthday fallback), NOT as the cause. B2 is now the priority of the pair. - B2 — DONE 2026-08-02 (
f6608b7, PUSHED). Round 2 now sendspool->addresses()(the sampler set) like round 1, with the relay's observation kept only as the last-resort fallback. Deliberately does NOT inherit round 1'sctx.local_addressesfallback: those are RFC1918 (holepunch::local_addressesonly filters libuv'sis_internal= loopback/link-local), JS never puts LAN addresses in this field (nat.js:139 →addresses = nullwhen UNKNOWN, and the encoder omits it), and preferring them would advertise ONLY private addresses on an UNKNOWN classification — worse than the single public mapping this round sent before. Caught by cpp-reviewer. CORRECTION while implementing: the claim that C++ also lacks JS'snat.add(reply.to, reply.from)(connect.js:578) is WRONG.PoolSocket::handle_message(holepunch.cpp:958-963) already feeds every response's wiretofield to the same sampler — a broader equivalent, since round 1's reply arrives on the pool socket. An explicit call there is dead code; caught because the red-check refused to go red. B2 was one divergence, not two. TestsPunchingRoundGossipsFullSampledSet+ a failover-relay assertion (a relay that was never a pool-sampling target still gets its observation gossiped). Both red-checked. 731/731, ASAN clean, cpp-reviewer SHIP. Causality NOT claimed — the same field capture shows connects succeeding with1 addrs, and the 2026-08-02 midnight failure had only one sampled mapping, so B2 would not have saved it. Landed on parity grounds. B1 + D-secondary remain the live pair on that path. Original entry follows. -
B2 (NOW THE PRIORITY OF THE PAIR — see Finding L)— Round-2 holepunch payload sends ONE address (our_addr, holepunch.cpp:1880-1885) instead ofnat_sampler().addresses(); JS sends the full set in both rounds (connect.js:567,654,684). Field Finding F (2026-07-23, SUCCESS) shows B2 is load-bearing, not a nicety: a port-varying SERVER (announce:45747≠ holepunch egress:2707) mislabeledfw=2 CONSISTENT(B1 latch) connected ONLY because it reported BOTH ports and the client probed both. B2's multi-address reporting is the mechanism that compensates for B1's mislabel — so B1 (correct classification) + B2 (full address set) are complementary; land both. Also note: Finding F shows B1 mislabels the SERVER's own self-classification too, not just clients. - D-secondary — server NatSampler never evicts stale external
addresses after a NAT remap (
:62622lingered next to live:48008). Real but not a connect blocker; wants an eviction/aging pass.
- connect-1 (MED) —
connected/on_connect gated on the REMOTE header + local write-ack, not on the LOCAL handshake as in JS (+½–1 RTT to first byte). DEFERRED — needs a live JS cross-test, not a batch edit. Design in memoryparity_audit_2026_07_08.md(split is_ready → can_encrypt/can_decrypt).
Kept where C++ is safer/more correct than JS. Documented so the next sweep doesn't re-report them:
- compact array-length cap 4096 (JS 0x100000) — anti-DoS.
- routing exact-k-closest sort (JS bucket-order early-stop) — strictly more correct.
- blind-relay 30s unpaired-pairing TTL + 1024-pairing cap (JS unbounded) — anti-DoS.
- SocketPool routes cap + GC (JS no cap) — anti-DoS + leak prevention.
- FIND_NODE/DOWN_HINT 1/sec/IP rate limits (JS unbounded).
- ConnectionPool: local Server router entry refreshed, not clobbered (JS last-writer-wins would kill our own listener).
- No auto-bootstrap of the public network by default (embedded/library targets
must not auto-join) — opt in via
default_bootstrap_nodes(). - filterNode composes built-in + caller filter (JS discards the caller's).
- C API can't express JS's explicit-ephemeral=true→non-adaptive state.
- Announcer keepalive DRIFT DETECTION (
08e2f47): pongto-field vs stored peer_addr triggers an early refresh (rate-limited 10s). JS discards the pong body (announcer.js:114-121) and waits out the 5-min reannounce. - Round-1 relay FAILOVER (Finding M,
d5988ec) — FIELD-VALIDATED. On a round-1 failure we pop the next announce relay and re-runrun_round1on the SAME pool socket / puncher / NAT samples (holepunch::fail_round1,pick_fallback_relays; marked at connect.cpp:971). JS bails terminally (connect.js:270-275) — and asks for exactly this fix in its own source twice:// TODO: we should retry here with some of the other relays, bail for now(connect.js:271) and// TODO: retry with another relay?(connect.js:314). Before this, ONE unreachable relay in the announce was a permanent-5: 81/81 deterministic repro, and stock nospoon-JS failed identically on the same network, so it is NOT a C++ regression. Field- validated 2026-07-30: two dead relays skipped, rounds 1 AND 2 completed through the third. Retry scope mirrors JS exactly — the six sites JS reaches by THROWING out ofprobeRoundfail over; those it reaches viaabort()(double-random, unstable NAT, probe exhaustion) stay terminal. SECURITY: capped atannouncer::PICK_BEST - 1legs, NOT the 128-entry wire cap — 128 × multi-second RPC would hand a hostile connect() target a ~6-minute stall for free. Honest servers announce ≤3. Do NOT "simplify" this by substituting the client's own observation of the same relay host —relays[i]is a live server↔relay UDP session, not an address; JS explains why at connect.js:280-285. Failover is safe precisely because each entry carries its ownpeer_addressand its own session. - Announcer STUCK-CYCLE WATCHDOG (
f843b4b, announcer.cpp:223-245): ifupdating_stays latched pastcycle_stuck_ms_(60s) the cycle is force-reset. JS has no equivalent; a wedged cycle there silently disables reannounce forever (field Finding E). - Announcer relay-set swap is guarded against an EMPTY new cycle
(
82d3c55): a cycle that commits nothing keeps the previous advertised set rather than publishing an empty one. JS would replace unconditionally.
Loopback can't prove these; validate against a real NAT'd JS peer:
- Finding H fix (pickBest, 2026-07-26) — FIELD-VALIDATED 2026-07-26:
phone (mobile CGNAT) now connects every time, fast. The mobile
connect-latency stall (~28s hunting dead record-holders) is gone, and the
one robustness risk — findPeer converging to a 3-node record set — did NOT
materialize (Kademlia converges, as JS proves in prod). Committed
12c4af6. - Finding A fix (
08e2f47, 2026-07-22): announcer publish-after-settle- keepalive drift detection + closestNodes reuse. Retest checklist in
docs/FIELD-DIAGNOSIS-2026-07-22.md— key test: disconnect → IMMEDIATE reconnect repeatedly across >10 min, expect success at every point in the announce cycle; watch forDRIFTEDlog lines healing within ~5-10s.
- keepalive drift detection + closestNodes reuse. Retest checklist in
- Relay→direct upgrade: C++ client rides relay → punch lands → migrates with no ETIMEDOUT/-110; relay closed only after provable direct arrival.
- Server-side migration when a JS client is on relay and the C++ server punches.
- Holepuncher birthday win end-to-end (the pinning residual at
holepunch.cpp:1450 is FIXED 2026-07-21 — winning
SocketRefnow rides inHolepunchResult.socket_keepalive); still needs live validation, plus server-side birthday stream completion (accepts via main-socket firewall, not birthday SocketRefs) — THE symmetric-CGNAT-server path. - Outgoing request id accepted by JS
validateId→ C++ node appears in JS routing tables; bootstrapper works as a real DHT id-holder. - TRY_LATER end-to-end (throttled server → client waits 10-20s → completes).
- LAN same-NAT shortcut (connect-6, 2026-07-22): now EXCLUSIVE of holepunch — verify a real same-LAN connect still succeeds and a failed LAN ping aborts cleanly instead of hanging.
- Empty relayAddresses announce record (announce-4, 2026-07-22): one live nospoon round-trip to confirm relay discovery is unaffected end-to-end.
- NAT sample ranking — keep following the promoted sample across multiple swaps, matching JS nat.js:addSample (2026-11-15). This fixes winner ordering and classification when a late sample overtakes several entries. Equal-hit ordering, freeze and source deduplication stay unchanged.
- Server holepunch opt-out — native set_holepunch_enabled(false) matches JS opts.holepunch=false (2026-11-15): new sessions allocate no punch socket or sampler and ignore holepunch rounds and bare probes, unless an older enabled session still needs probe echo, while direct UDX and blind-relay pairing remain available. Default enabled; the existing holepunch veto callback is unchanged. C/wrapper exposure is separate.
- Arbitrary-target native unannounce — target-aware lookup_and_unannounce_target and unannounce_target match JS topic semantics (2026-11-07). Legacy public-key methods retain hashing. C unannounce_target now exposes exact-topic withdrawal (2026-11-13), using the existing native completion/signing path. Python unannounce_target exposes the same exact-topic operation (2026-11-15); legacy unannounce still hashes the public key. Streaming query controls and other wrappers remain separate work.
- Already-hashed native findPeer target — C++ find_peer_target matches JS findPeer(target, { hash: false }) (2026-11-07). Existing public-key methods keep their semantics. C find_peer_target and its cancelable _ex now expose the exact-target operation (2026-11-13), sharing legacy callback/error/cancel behavior. Higher-level wrapper methods and other target/query controls remain separate work.
-
changeRemote— DONE this session (relay→direct upgrade port: libudxudx_stream_change_remotewrapped inrelay_upgrade::try_change_remote+ Duplex firewall tap + FFI). Covers the path-upgrade case; mid-connection NAT remap is out of scope (JS doesn't handle it either — onsocket is one-shot). -
_relayAddressesCache— IMPLEMENTED (c196205; source rechecked 2026-10-13).HyperDHT::cache_relay_addresses()stores successful-connect relays by server pubkey; reconnect queries only cached relays first, then falls back to ordinary discovery if needed (2026-11-05). The cache is capped at 512 entries with unordered-map eviction, rather than JS xache LRU. Seesrc/dht.cppandsrc/connect.cpp. - Read-side backpressure — DONE 2026-07-28 (
a08cade).SecretStreamDuplex::pause_read/resume_readwrapudx_stream_read_stop/read_start, exposed ashyperdht_stream_pause/resumeon the C API (idempotent, NULL/closed-safe; unordered datagrams unaffected). Testtest_stream_pause.cpp. RESIDUAL: the automatic highWaterMark (16KB) trigger is NOT implemented — pause/resume is caller-driven, where JS pauses itself when the read buffer exceeds highWaterMark. protomux-2 (no true read-side pause; buffer+teardown substituted) is likewise still open. - Sleeping-interval wake detection — VERIFY: the tick rewrite added
_onwakeup/do_wakeup+last_tick_ms_, but confirmbackground_tickactually compares wall-clock gap vsSLEEPING_INTERVAL(15s) and triggers the wake path (JSdht-rpc/index.js:764-799). If the gap comparison isn't wired, finish it. See memorysleeping_interval_gap.md. - Bootstrap DNS /
ip@hostform —_resolveBootstrapNodesDNS +@-host fallback absent (IP-only). Callers passing hostnames get nothing.
- Full ASAN/valgrind leak sweep of the hot path (confirm no hot-path leaks; re-verify after this session's large diff). The teardown leaks previously logged here as "pre-existing, in libuv/libudx internals" were pinned down 2026-07-28 — see TD1-TD3 below. Two of the three are ours, not libuv's.
- Fuzzing (2026-08-25): 9 harnesses, 30 min each, 1.677 B runs, zero
findings. No crashes, no ASAN reports, no UB. Final coverage per target:
protomux 893, noise_payload 399, messages 385, noise_handshake 223,
holepunch_msg 219, handshake_msg 165, compact 108, blind_relay 88,
secret_stream 75.
- Four harnesses added.
fuzz_noise_handshakeis the important one:NoiseIK::recvon the responder side is the deepest PRE-AUTH surface in the library — every PEER_HANDSHAKE carries attacker-chosenmsg.noisebytes and nothing authenticates upstream of it, because the handshake IS the authentication. (fuzz_noise_payloadonly decodes the payload struct of an ALREADY-SUCCESSFUL handshake — a different thing.) fuzz_protomux(Mux::on_data— varint framing, channel dispatch, OPEN/CLOSE/REJECT,handle_batch) is the deepest target overall: 400 corpus entries against ~24 for the flat decoders.fuzz_blind_relay(decode_pair/decode_unpair) closed the last uncovered header-level wire decoder.fuzz_secret_streamis thin by nature — most of its body is libsodium AEAD, so a clean run there means little.decode_holepunch_payloadneeds no separate target:fuzz_holepunch_msgalready exercises it.- Crypto-gated targets are worthless unseeded. Random bytes never pass
Poly1305, so an unseeded fuzzer only ever exercises pre-MAC length/slicing
code.
fuzz/gen_seeds.cppemits a valid msg1 / header / encrypted frame into the corpus; seeding liftedfuzz_noise_handshakefrom a 3-unit plateau to cov 144 in 20 s. - A harness can be clean because it tests nothing. Both crypto harnesses
were initially broken in ways a passing 30-min run hid, and the flat
coverage number was the only symptom:
fuzz_secret_streamfed the stream its OWN header toreceive_header.local_id_uses NS_INITIATOR andremote_id_NS_RESPONDER (src/secret_stream.cpp:40-48), so it always failed,is_ready()stayed false, anddecrypt()returned at its first line — 390 M runs tested the guard, not the AEAD. Fix: synthesize the header from a role/key-swapped peer instance.fuzz_noise_handshake'sis_complete()branch was dead for every input: one responderrecv()advancesmessage_index_0->1, completion needs=2 (
src/noise_wrap.cpp:424,507). Fix: reply withsend()(msg2) asfinalize_handshakedoes. Verify a harness by whether coverage MOVES when you fix it, not by whether the run is green.
- Run with
UBSAN_OPTIONS=halt_on_error=1. The build enables-fsanitize=undefined, but UBSan defaults to print-and-continue, so a UB finding neither stops the fuzzer nor fails the run. fuzz/CMakeLists.txtnow usesfuzzer-no-linkglobally with-fsanitize=fuzzerper fuzz target. Plainfuzzerglobally gave every executable libFuzzer'smain(), so any helper with its ownmain(gen_seeds) failed to link.- Corpora persist in
fuzz/corpus/<target>/(gitignored) so runs compound instead of restarting from zero; crash reproducers land infuzz/artifacts/.
- Four harnesses added.
- Fuzzing, still uncovered:
rpc_handlersper-command parse ofreq.value(announce record + signature paths, token-gated but the token is only address-bound), and libudx packet parsing (deps/libudx/src/udx.c— every inbound datagram; upstream has functional tests and CI but no fuzzing and no sanitizer flags anywhere in its tree). A CI job running each target ~5 min on PRs would catch parser regressions at the point they land. - Stress: 100 concurrent JS clients vs one C++ server; measure success rate + memory growth; confirm the probe-listener multi-slot fix holds.
- Soak: 12h+ connection, data every 5 min; verify NAT pinhole + SecretStream keepalive, no drift.
All three are shutdown-only, so a long-running server pays them once at exit and
the OS reclaims. They matter for embedders that create and destroy many DHT
instances in one process (test harnesses, apps that rebuild the DHT on a
network change) — there they accumulate. holesail-cpp scopes them in its
test/lsan.supp, which doubles as the reproduction recipe.
-
TD1 — a cancelled query leaks its state (~1.3 KB). The
on_donecontract is NOT violated. Corrected 2026-07-29 — the original entry claimed "hyperdht_query_cancel()never fireson_done, a contract violation". That half is REFUTED by standalone repro; the leak half stands, at a much smaller magnitude than first reported.- Contract HOLDS.
hyperdht_query_cancel()DOES fireon_donewithHYPERDHT_ERR_CANCELLED(-8), on all four cancelable paths (immutable_get_ex,mutable_get_ex,find_peer_ex,lookup_ex), in both orderings (cancel→pump→free and cancel→free→pump). Chain verified in code:ffi_core.cpp:28st.q->destroy()→Query::destroy()→fire_done_once(QUERY_OK)→ the FFI done lambda, which mapsstate->cancelledto -8. Matcheshyperdht.h:751. - The one case where the callback does NOT run is
hyperdht_query_free()without a preceding cancel — free deliberately detaches by nullingdone_cb(ffi_core.cpp:31-40). That is documented behaviour (hyperdht.h:759), not a defect. Most likely what the original repro hit. - The leak is real but ~1.3 KB, not ~34 KB. A/B under ASAN, same binary:
baseline create/bind/destroy = 15623 B / 18 allocs; identical run with one
immutable_get_excancelled mid-flight = 16895 B / 35 allocs. Delta = 1272 B / 17 allocations for the cancelled query, withon_doneobserved firing. SoQueryState/query::Querystill outlive teardown even though the completion lambda ran — the ownership problem inquery_free's "statemay still be alive via lambda ref, which is fine" comment is genuine; the reasoning about why was not. - Fix: give
query_freereal ownership (or drop the query's self-reference once done). Do NOT "fix" it by firingon_doneon cancel — that already happens, and adding a second fire would break the fires-exactly-once contract. Re-run the full 716-test lane after. - Repro note: measure with stdout UNBUFFERED.
DHT_LOGwrites to stderr unbuffered whileprintfis block-buffered when piped, which interleaves the transcript misleadingly and can make a callback look like it never ran.
- Contract HOLDS.
-
TD2 — FIXED 2026-10-24: the stopped interface watcher now closes.
RpcSocket::close()callsudx_teardown(), closing streams, sockets and listeners while their owner remains alive for the loop drain. The generated watcher close callback uses its saved UDX pointer after the user deleter.HyperDHT.CreateAndDestroynow requiresuv_loop_close() == 0; that lifecycle and seven UDX tests pass ASan/UBSan with leak checking enabled. Eighteen relay tests also pass sanitizers with leak checking disabled; this does not establish all-project leak freedom. The original July reproduction left one inactive, unclosed timer and returnedUV_EBUSYafter a full drain. The consumer's defensiveuv_walkfallback remains unchanged. -
TD3 — in-flight connect/stream state dropped when
hyperdht_destroybeats a graceful close round-trip. Frames:HyperDHT::do_connect,server::Server::on_handshake_result,udx__cirbuf_init/_set. ~11 KB per integration run, and it scales with connection count — consistent with per-connection state being abandoned rather than reaped. Lower confidence than TD1/TD2: established by tracing frame#1of every unsuppressed allocation to a dependency frame, not by a standalone repro. Reproduce before fixing.
The 2026-07-09 sweep did NOT run a finder on these. Seed a round-2 sweep here before any full "core frozen" sign-off:
-
health.cppvsdht-rpc/lib/health.js(only checked at the tick call-site).- Per-tick idle gate (2026-11-12): fewer than four responses/timeouts
in the latest tick do not contribute a healthy/degraded vote, matching
JS's delta-based
idlegetter. Window totals remain available. - Per-tick availability (2026-11-13): an active tick without responses reports OFFLINE immediately; a responding active tick recovers without waiting for four healthy ticks. JS's four-tick degradation latch remains, and going offline clears it. Aggregate statistics and idle/cold gates stay.
- Per-tick idle gate (2026-11-12): fewer than four responses/timeouts
in the latest tick do not contribute a healthy/degraded vote, matching
JS's delta-based
- HyperDHT
suspend()/resume()socket rebind + inbound-drop (io.jssuspend/_rebind) — rpc-levelstop_tick/start_tickalone don't match. -
raw-stream-set.js,semaphore.js,refresh-chain.js,commands.js, the udx wrapper. - The entire
ffi_*layer (the natural freeze boundary — audit last, once the core below it is frozen). RAISED IN PRIORITY 2026-07-28: the first real bug found here was a SECURITY inversion —hyperdht_server_set_firewallreturnedcb(...) == 0whereFirewallCbmeans true=REJECT, so every C-API server with a firewall admitted exactly the peers it should refuse and refused the authorised one (0203820, found via holesail-cpp, regression testsServerFfiFirewall.*). Polarity conversions are the smell: the siblingset_holepunch(== 0) and asyncfirewall_done(!= 0) were each re-checked and ARE correct — the two callbacks legitimately have opposite polarities (FirewallCbtrue=reject vsHolepunchCbfalse=abort). Audit the rest offfi_*for the same class: bool/int contract mismatches at the boundary, which compile fine and fail silently.
- F1 (field Finding F, 2026-07-23) — after a direct probe wins
(rawStream firewall fires, connected), the client keeps the round-2 BLIND
RELAY request alive and retries it 3× to timeout (
Round 2: TIMEOUT (relay)). Cancel the round-2 relay once the direct probe connects — wasted relay traffic + a scary-but-harmless timeout in field logs. Not a bug. - F2 (field Finding F, 2026-07-23) — server logs
on_peer_holepunch: unknown session id=0for the client's round-2 that arrives AFTER the session connected+cleaned on round 1. Benign; downgrade the log level or match it against a recently-completed session so it doesn't read as an error in field captures. -
immutable_put/mutable_putcommit errors reach C++ results and C done callbacks (6bc02b1, 2026-10-26); callback signatures remain unchanged. - Wrapper build CI: Python/Kotlin/Rust announce-ABI change (
f37ef1d) is UNVERIFIED — no wrapper build in the C++ gate. Add a wrapper-build check. - Tighten connect-3 wording in the sweep docs so it's not read as
"reusableSocket unimplemented" — RESOLVED 2026-07-28 (
a08cade): the C-API client option landed ashyperdht_connect_opts_t.reusable_socket(tail- appended; Python ctypes mirror updated in the same commit per the CLAUDE.md ABI gotcha). Server + wire + client option are all done; only the route-cache gating nuance remains, tracked as connect/divergence-1 in Section A. - tick-7: full 2-pass bootstrap +
testNat-gated second_updateNetworkState(only the quick-firewall PING_NAT first-responder heuristic landed). - holepuncher-4: fresh-socket reopen (currently same-socket resample — recovers lossy-UNKNOWN, not the new-NAT-mapping case). Blocked on the upgrade-port socket-handle invariant; revisit if a real CGNAT case needs it.
- Announcer synchronous-cycle-completion parks a stale
current_query_(cpp-reviewer, Finding E batch): if a full-congestion walk completes synchronously insidedht_ops::find_peer()before thecurrent_query_ = find_peer(...)assignment, on_done's reset() runs on the old (null) value and the finished Query is parked until the next update(). Benign (destroy() idempotent, self-heals, no leak/UAF), pre-existing pattern now reachable via a 2nd trigger. Real fix = defer Query's first dispatch via uv_idle; not warranted yet. - Documented deviations from the 2026-07-22 batch (deliberate, revisit only if
live behavior warrants):
- connect-7 veto abort is local-only; JS also sends an ERROR_ABORTED round to the relay. Skipped: PunchState::complete closes the pool socket immediately (send race). Add with a deferred close if needed.
-
HOLEPUNCH_TIMEOUT(-6) now covers veto + LAN-ping-fail + passive timeout (all JS HOLEPUNCH_ABORTED). Rename would touch FFI/wrappers. - server veto/punch-fail clear_session immediately; JS defers ~10s via puncher teardown (pre-existing semantics, kept).
- direct branch lacks JS's
relayed && !remoteHolepunchablegate — pre-existing structural divergence; gating would break direct-to-server connects (loopback fixtures rely on it).
hyperdht-cpp is pinned to libuv 1.51.x (flake on nixos-25.11) because libuv
1.52.0 has a UDP POLLERR regression that wedges libudx streams on real NAT
paths — full write-up in docs/LIBUV-VERSION.md; unwired fallback patch in
nix/libuv-1.52-udp-pollerr.patch. Re-check whenever bumping nixpkgs/libuv:
- Has libudx bumped its libuv pin past 1.51 or shipped a 1.52-compat
fix? (
deps/libudx/CMakeLists.txt:~7; watch holepunchto/libudxmain.) - Has libuv shipped >= 1.52.2 with PR #3250? If yes → drop the 25.11 pin, delete the patch, remove the CMakeLists version guard, use stock libuv.
- Extract
run_round1()into named phases (src/holepunch.cpp) — the single worst readability hot spot: 457 lines, 9× the 50-line guideline. Split intoprobe_round→await_sampling→analyze_and_maybe_reopen→round2, each takingPunchState. Do it WITH the Finding K fix, not as separate churn — theproceedlambda, the sampling gate and the reopen path all live in one function body today, which is precisely why K (a mid-flightpool_fwread that contradicts its own comment) is subtle instead of obvious. Scope discipline — a 2026-07-29 measurement says the library is NOT spaghetti otherwise, so do NOT generalise this into a broad refactor: coupling is low (mostsrc/*.cppinclude 3-5 hyperdht headers; onlydht.hppat 12, the facade), code density is ~340 lines/file across 36 files (the ~20.4k rawsrc/total is 27% comments, and those JSfile:lineparity refs are what made Findings H/I/J root-causable by reading).holepunch.cppcarries 36 lambdas (server 23, connect 19, rpc 17) — the cost of "every JSawaitbecomes a callback chain". Coroutines would flatten it but are a large change and hostile to the ESP32 target: rejected. NOTE: raw indentation hits 56 columns in holepunch.cpp but that is WRAPPED ARGUMENTS, not nesting — true control-flow nesting at depth >=5 is 19 lines there, 8 in connect. - Round-2 adversarial JS-parity sweep — re-run
jsparity-adversarial-sweepover the round-1 blind spots (Section F) + a re-diff of the subsystems changed heavily this session (rpc, connect, server, holepunch, protomux) to catch regressions the per-bucket reviews might have missed. Append new findings to Section A.
Context (2026-07-23): Rightbracket/peeroxide — independent Rust port of the
FULL Hyperswarm stack (pure-Rust libudx + HyperDHT + Hyperswarm topic layer + CLI),
wire-compatible with JS, v0.3.1, published (crates.io/homebrew). Local Nix build at
~/Desktop/repos/peeroxide-nix (nix build .#peeroxide). Source clone was in
scratchpad; re-clone https://github.com/Rightbracket/peeroxide to act on these.
It hit the SAME announce/staleness class we did — useful as a borrow source AND a
second parity oracle. Two workstreams:
- Refresh-token cheap-refresh. Their
peeroxide-dht/src/persistent.rs:462— announce carries optionalrefreshtoken; a re-announce with NO peer body but a refresh token restores the full record from a token-keyed cache → keepalives are tiny (token only), not full peer+sig+relays. Check: do WE send refresh tokens on re-announce + handle the no-peer+refresh restore node-side? (Jul-21 note: field exists in our messages, may be unused.) Parity gap + efficiency. - Parallel dual-announce + relay feedback. Their
peeroxide/src/peer_discovery.rsdo_refresh: each refresh fires topic-announce (ontopic) AND self-announce (onhash(pk)→ FE-holders for PEER_HANDSHAKE Phase 2) in parallel; relay addresses harvested from prior hash(pk) acker set feed the next topic record ("mirrors Node'sAnnouncer.relayAddresses"). Check our Announcer does the same dual-target + relay feedback — this is the Finding A/D forward-state area. - Constants sanity-check: their
REFRESH_INTERVAL=600s(+≤120s jitter), recordTTL=20min("matches Node.js"). Confirm our re-announce cadence (~10min, not 5) + TTL match. - Note: they PUNT unannounce (
swarm.rs// leave/unannounce (future)) → their server-restart staleness window = full 20-min TTL. If we have clean unannounce-on-shutdown, we're AHEAD there — do NOT copy their omission.
- Their committed byte-exact fixtures:
tests/interop/*-fixtures.json(noise, noise-ik, dht-rpc, hyperdht, protomux, secure-payload, blind-relay), generated bytests/node/generate-*-golden.jsfrom the JS reference. Feed their fixture inputs through our C++ encoders, assert output == their bytes. Both claim JS-parity → any mismatch localizes a bug to a specific message/field. Highest-value targets:noise-ik-fixtures.json(Ed25519 DH, our #1 gotcha) +hyperdht-fixtures.json(announce/lookup, current pain). - Mine their interop
.rsassertions for edges we may skip (empty relays, refresh-only announce,bumphandling,MAX_RELAY_ADDRESSEStruncation).
Merged for the JS parity, NOT as a Finding Q fix — see §J.4, the JS control proved that failure mode is not a hyperdht-cpp bug at all. Pre-merge gate re-run at merge time: 767/767, 0 ASAN errors across holepunch / socket_pool / server / server_punch_socket / relay_upgrade / pool_socket_requests / secret_stream / router / server_handshake.
Server::disable_fast_mode_pingwas kept (defaults false, zero runtime cost, documented diagnostic-only) — it is what made Finding Q reproducible and what let us control the comparison against the reference server.Field validation before merge: live JS-interop against real hyperdht 6.29.1 on the public DHT; a real cross-NAT punch against a phone on mobile CGNAT; and ~100 phone sessions across two client networks and three server builds.
Still to do after this merge:
- Push
main(29 commits ahead oforigin/mainat merge time).- Bump nospoon's pin
~/Desktop/repos/nospoon/cpp/hyperdht-cpp.nixrev+hash(still3421264) — needs the push first, it fetches from GitHub by rev.- Add the one-line
info.socket_keepalivehold innospoon/cpp/server.cpp; the field run confirmed the predictedsocket close refusedfd parking.- BUG A (§J.1) is still open and unrelated to this branch.
Parity actually restored (this is the justification — each is a real divergence from the JS reference that this branch closes):
- Per-session punch socket — JS
holepuncher.js:14(dht._socketPool.acquire()), server sideserver.js:436. We punched from the persistent announce socket. This is the headline item.addresses4=[remoteAddress] + LAN(JSserver.jsshape) — replaced a sampler dump plus a port rewrite that produced duplicateaddr[0]==addr[1]in the field.remoteAddress()reads the RING sampler (dht-rpc_natrole), not the Nat classifier, and oneourRemoteAddrkeys firewall + holepunch + addresses4 together (server.js:271,357-359).- Round replies deferred until NAT sampling settles — JS
await p.analyze()(server.js:518-519).- NAT discovery resolves on classification, not on the last straggler ping — JS
nat.js:172-179.- Session NAT sampler fed only by AUTHENTICATED rounds — JS
server.js:491-511. (Also a security fix: 3 spoofed sources could otherwise pin an attacker address.)- Fast-mode ping egresses from the punch socket — JS
holepuncher.js:77.- Probe echo + stream adoption on the punch socket.
NOT parity — beyond-JS additions, review them as such before merging:
Server::disable_fast_mode_ping(default false, no JS equivalent). Diagnostic only, but it is the ONLY way to reproduce Finding Q on demand, so it earns its place. Keep, or drop it and lose the reproduction — an explicit call to make.- Bug fixes JS gets for free from GC or simply does not have: BUG B (
~Holepuncherreleasing holders + the random-punch throttle),relay_tokenin bothServerConnectionmove ops,SocketPool::destroychecked close,~SecretStreamDuplexwriting through a freedraw_stream_,hyperdht_busy_close_count()on the C FFI.Pre-merge checklist:
- Re-run
ctest -E 'LiveServer|test_server_live'(expect 767/767) + the ASAN per-binary sweep (expect 0ERROR: AddressSanitizer; leaks unchanged).- Decide explicitly on shipping
disable_fast_mode_ping.- Merge direct to
main(this repo has no PR gate) and push.- Bump nospoon's pin:
~/Desktop/repos/nospoon/cpp/hyperdht-cpp.nixrev+hash(still on3421264), and add the one-lineinfo.socket_keepalivehold innospoon/cpp/server.cpp— the field run confirmed the predictedsocket close refusedfd parking.Do NOT claim it fixes Finding Q. A/B'd on a real CGNAT path: branch 1/14 black holes vs main 3/12, p≈0.31 — not significant, and the branch's own failure used the entire new path. Honest field line: "may reduce the -5s, unproven." Q stays open; see §J.4.
Plan: docs/superpowers/plans/2026-08-17-server-punch-socket-parity.md.
Ledger: .superpowers/sdd/2026-08-17-server-punch-socket-parity/progress.md (every
parked finding + deferred minor lives there — read it before resuming).
Why: the C++ server punched from the persistent announce socket (shared with
announces, relay keepalives and all DHT RPC for the process lifetime); JS acquires a
fresh pool socket per holepunch session (holepuncher.js:14, server.js:436). Proven
by a control we already had: a nospoon-JS server on the same Pi5, same router, same
clients, connected every time for months while the C++ server failed intermittently
(handoff line 14 + §Finding Q). Client, network, router and probe format were each
separately eliminated. Full case: memory server_punch_socket_parity, Q2-EVIDENCE-2026-08-04.md.
- BUG B — birthday holders leak; the server permanently loses birthday punching.
FIXED
e8fd86c.Server::on_socketdoesconn.puncher.reset()→~Holepuncher, which only nulledon_holepunch_message+stop(); onlydestroy()released holders (src/holepunch.cpp:367-381vs:740).BIRTHDAY_SOCKETS=256 (include/hyperdht/holepunch.hpp:284) vsMAX_POOL_SOCKETS=512 (src/socket_pool.cpp:178) ⇒ after ~2 successful RANDOM+CONSISTENT server punchesacquire()returned nullptr for the rest of the process. Measured: 201 sockets retained after the drop, 0 withdestroy(). Fixed in~Holepuncherrather than at the one call site, so every drop path is covered;release()anddecrement_randomized()are both idempotent, so it is a no-op whendestroy()already ran. A second leak on the same path was found while fixing it: the dropped puncher also leftPunchStats::random_punchesincremented, andrandom_punch_limitis 1 →can_random_punch()never returned true again, so no random punch (CONSISTENT+RANDOM sprays included) could start. TestHolepuncher.DropWithoutDestroyReleasesHoldersAndThrottle, red-verified on both assertions. 763/763. - BUG A (HIGH) — blind-relay fallback implemented and live-verified.
Readiness fixed 2026-10-23: both production relay callers prepare the
channel and pairing callbacks before starting SecretStream, then send Protomux
OPEN only after the encrypted header exchange completes.
BlindRelayClientretains requests until the channel opens and sends each once; cancellation and synchronous client destruction have regression coverage. Production client/server regressions require an actual PAIR request; the server fixture no longer sends an unsolicited reply. A local relay carried encrypted data both ways. All three suites and isolated consumer connectivity checks pass with public and NAT server placements. Public/OPEN control socket fixed 2026-10-23: when the handshake has no selected pool socket, both relay callers connect the control stream using the node's DHT socket. All three suites and forced-relay DATA checks pass with a public application server and both NAT-board server placements, exact encrypted payload hashes, no direct board DATA, and public/NAT unused-relay-identity negatives. These negatives do not cover packet loss to a running relay. Live relay-to-direct upgrade verified 2026-10-24: one isolated run with a public application server and one with a NAT application server, both using a public persistent OPEN relay. A temporary direct-path gate requires real encrypted relay traffic before allowing the normal holepunch path. Exact 8192-byte echo and 32-byte reverse hashes match through 32 and 37 numbered round trips respectively, including at least three relayed and twenty direct rounds per run. Each endpoint opens its encrypted connection once; the original UDX streams survive migration without reconnecting. Each endpoint's packet capture proves direct traffic arrived before its relay control FIN, followed by direct DATA. Artifact hashes, effective commands and kernel executable paths match; both hosts return to byte-identical network/firewall snapshots with owned rules and lab processes removed. This verifies the existing upgrade code; no library change was needed. It does not cover relay outage recovery, other network/OS combinations or soak. Transport-write errors fixed 2026-10-24: both relay control writers destroy the encrypted stream on a rejected write; stream closure tears down the mux and fails pending pairing through the existing deferred abort path. The parallel direct attempt remains available. Client/server regressions force a rejected PAIR write and require failure before the pairing watchdog. All three suites and isolated default/forced-relay connectivity checks pass with public and NAT server placements. All 35 ESP32-S3 component files compile with ESP-IDF 5.5.2; firmware linking and board operation remain unverified. Client request reuse fixed 2026-10-23: all routes of one connect now share the same Noise request, so pending routes can deduplicate onto one server session. Reply validation uses an isolated state copy per attempt. Regressions, all three suites and public/NAT/LAN consumer checks pass. A forced-relay NAT capture also verifies identical complete Noise requests through two DHT routes and one deduplicated pending server session. Completed-handshake dedup fixed 2026-10-23: retain encrypted replies for the existing clear-wait interval after direct emission (default 10 s), with weak socket references and a bounded cache (256 desktop / 16 embedded). Active sessions and stream ownership keep their existing cleanup. Regressions cover identical replies, expiry without duplicate refresh, closed-socket fallback and suspend. Public/NAT and both LAN directions pass, including payload past the grace interval; a naturally late LAN request reused the completed reply without creating another session. Forced-relay DATA also passes. All 35 ESP32-S3 component sources compile with ESP-IDF 5.5.2 and exceptions/RTTI disabled; firmware linking and board operation remain untested.
These checks remain unverified by the October lab fixes:
- Distinct NAT networks, mobile/CGNAT paths and network handover.
- Windows/macOS/Linux interoperability in supported client/server roles; repair the skipped IPv6 fixture and test real IPv6 paths.
- Interruption and recovery of a running relay, and longer-session soak tests.
- ESP32-S3 firmware linking, flashing and an actual board connection.
Broader sanitizer runs with leak detection disabled do not establish leak freedom.
Done 2026-08-19/20 except the two items that need a network and a human, which are called out at the bottom. The branch is code-complete and clean, but it has still never carried a packet against a real peer.
-
cpp-reviewer on the whole branch — 3 HIGH + 3 MEDIUM, all verified against the code and the JS reference before acting, all fixed in
4d67452except MEDIUM 1 (a genuinely separate parity gap, filed as J.3 below). Detail:- HIGH —
discover_pool_addresseswaited for the SLOWEST sampling ping instead of resolving on classification like JSnat.js:172-179. This gate sits in front of every parked PEER_HOLEPUNCH round, so one dead routing-table node held the reply forMAX_REQUEST_ATTEMPTS × timeout_for(dest). Measured on loopback: 3000 ms before, 150 ms after — routinely longer than the client's own round budget, so the client aborts with-5and the server answers a tid nobody is waiting on. A regression on this branch's own target path; the branch would likely have field-failed with it in. - HIGH —
UpgradeContext::set_socket_keepalivereplaced the pin unconditionally. After the firewall tap migrates a stream onto the punch socket, the puncher's later birthday win offers a socket the stream is NOT on (itson_socketis a one-shot no-op), and the replace dropped the only ref to the live socket. Fixed in the class, sincesrc/server.cppandsrc/connect.cppboth do it. - HIGH —
SecretStreamDuplex::on_udx_closenulled the stream's pointer to us but not ours to the stream, so~SecretStreamDuplexwrote into memory libudx had already freed. Real heap-use-after-free, caught by ASAN. - MEDIUM — the per-session NAT sampler accepted a tid-matched reply from ANY source. Now source-checked (sample only; the request still completes).
- MEDIUM —
include/hyperdht/udx.hppclaimed refused sockets are "reaped at teardown"; nothing in this codebase callsudx_teardown(). Comment corrected.
- HIGH —
-
ASAN — 0 errors across every test binary; leak totals unchanged (
test_server91-98 KB, in its noisy band;test_hyperdht171108;test_connect_paths21496). Two real UAFs were found and fixed getting there (the~SecretStreamDuplexone above, plus~FakeRelayPeerfreeing itself out from under its own asyncuv_closecallbacks). -
hyperdht.hdocuments thehyperdht_stream_opencontract, on both the function andhyperdht_connection_t::_internal: call it synchronously inside the connect/connection callback._internalcarries the socket keepalive and the producer frees it when the callback returns; there is no way to hold it from C without opening the stream, by design. -
hyperdht_busy_close_count()exposed through the C FFI + the Python mirror (_ffi.py,_bindings.py,__init__.py). Rust bindgen and the Kotlin JNI regenerate from the header and need nothing. -
relay_tokenadded to bothServerConnectionmove ops (e8fd86c…78eb2c0). -
SocketPool::destroy()— checked close so refusals are counted, and it deliberately keeps flaggingclosed_even on refusal. Do NOT "fix" that to matchSocketRef::do_close: a ref that survives destroy() outlives the udx_t too (udx_tis a value member ofRpcSocket, andHyperDHTdeclaressocket_beforesocket_pool_), so a lateinactive()retrying the close isudx_socket_closeon freed memory. A first version of this change shipped exactly that hazard and the reviewer caught it. -
JS interop — DONE 2026-08-20, against real hyperdht 6.29.1 and the real public DHT (
./build-debug/test_server_live+node test/js/connect_by_key.js <key>). JS client connected to the C++ server in 1352 ms. The new path is confirmed live, end to end, against real relays:Session id=0 acquired a punch socket— the gate fires on a relayed handshake.Round parked until sampling settles (id=0, 1 queued)thenPunch-socket sampling done (id=0, ok=1)— the deferral works and the early-settle fix is visible in the log: it resolved after 3 of 7 sampling replies, with the other 4 still in flight. That is the HIGH-1 fix doing its job on a real network.Round on punch socket (id=0) from RELAY-1— the relay forwards the client's round straight to the per-session socket. This is the whole point of the branch and it is now proven on the wire, not just in tests.Fast-mode ping to SERVER-A:19708 (peer_address, punch socket)— egresses from the punch socket, per gotcha 19b.Client punching (id=0, fw=2, 2 addrs)— JS accepted our reply and declared punching.
-
CROSS-NAT PUNCH PROVEN ON THE BRANCH — 2026-08-20. Real phone on mobile CGNAT (
CLIENT-MOBILE) → this laptop behind a home NAT (SERVER-A), via a scratch nospoon-cpp server on TUN10.100.0.1, with the fast-mode ping disabled so the connection had to complete through the real probe loop. Branch connected 2/2, one probe each, both rounds arriving on the per-session punch socket:Round on punch socket×2 →Client punching (fw=2, 1 addrs)→[hp] Sending probe×1 →rawStream firewall→Stream open. The punch loop itself is proven to work cross-NAT. That much is solid. -
A/B RUN PROPERLY — THE BRANCH DOES NOT FIX FINDING Q. Real phone, mobile CGNAT, fast-mode ping suppressed on both arms, same server key, alternating arms:
| arm | sessions | landed | black holes | rate | |---|---|---|---|---| | `main` @ `ba03888` | 12 | 9 | 3 | 25% | | branch | 10 | 9 | 1 | 10% | 25% vs 10% at these n is p≈0.6 — noise. **No improvement is demonstrable.** The decisive datum is not the rate, it is the branch's failing session, which used the ENTIRE new path and died anyway: `Session id=5 acquired a punch socket` → `Punch-socket sampling done (ok=1)` → `Round on punch socket` ×3 → `Client punching (fw=2, 1 addrs)` → 10 probes to a freshly-relay-observed address → nothing → `Session cleanup`. So punching from a fresh per-session socket whose mapping the relay observed *for that session* does **not** prevent the black hole. **This kills "the shared announce socket's mapping goes stale/busy" as the mechanism of Finding Q.** Keep the branch on its own merits (JS parity, a proven cross-NAT punch loop, and the real bugs the review found) — but do not merge it as a Finding Q fix, and do not tell the field it fixes the `-5`s. Harness, reusable: `scratchpad/arm.sh <main|branch> <runid>` swaps arms (same deterministic seed → same public key → the client never reconfigures); `test/js/connect_by_key.js` is a scriptable client. **Read the verdict from log counts, never from "it connected":** `[hp] Sending probe` (1 = landed, 10 = black hole) and `Stream open`.
-
The failure is strictly bimodal. Across 22 sessions, every single one either landed on probe #1 or lost all 10. Never 2-9. That is not loss or congestion — it is binary reachability of one 4-tuple, decided before the first probe and stable for the whole 10-second schedule.
-
Base rate ~10-25%, on demand, reproducible any time by disabling the fast-mode ping. Fast mode otherwise masks the punch path completely (it connects at round 1 and the Holepuncher never starts) — which is exactly why this was never reproducible before, and why the field only ever saw "it works when fast mode fires".
-
Server-side causes are now largely excluded: fresh per-session socket, correct and freshly-observed destination address, sampling settled,
Client punchingwith a gossiped address, probes provably sent. All present in the failing runs. -
THE JS CONTROL SETTLES IT: this failure mode is NOT a hyperdht-cpp bug. Ran the nospoon-JS server (
nix build .#nospoon-js) on the same laptop, same config, same seed ⇒ same public key, so the phone reconfigured nothing. JS fails from the hotel exactly as our C++ does, and succeeds from mobile exactly as our C++ does:server on the laptop from hotel from mobile C++ branch (fast on/off) FAIL 0/82 OK C++ main (fast on) FAIL 0/12 — JS reference impl FAIL OK Stock hyperdht suffers this identically. Before calling any future punch black-hole "our bug", run this control — it is cheap and decisive. Scope limit: this proves the DETERMINISTIC hotel case is implementation-independent; it does NOT prove the older INTERMITTENT mobile-CGNAT Q (10-25%) is the same phenomenon, which has never been JS-controlled.
-
A 100% reproducible instance exists (2026-08-20/21), and it exonerates the branch. One hotel wifi (
CLIENT-WIFI-B) fails absolutely against this laptop while the same phone on mobile data connects on the first probe. Controlled properly after two false starts: fast mode ON and OFF both fail (57 and 25 sessions), and runningmainwith fast mode on — config-identical to the Pi5 that the hotel CAN reach — also fails (0/12). So it is not the punch socket, not fast mode, and not any code on this branch. The broken thing is thehotel ↔ home-ISPnetwork pair specifically:hotel ↔ Pi5-ISP (SERVER-B)works andmobile ↔ home-ISPworks, and both servers arefw=2needing a real punch, so no "one end is open" story explains it. Mapping and addressing are excluded by measurement too — the newobservers: relay=… advertised=…line printed AGREE throughout, and sessions probing the relay-verified address still lost 20 probes each. -
No server-side discriminator exists. All 25 sessions were mined across handshake source port, probed port, the delta between them, which relay forwarded the round, which arm, and position within the run. Nothing separates the 4 failures from the 21 successes — every dimension overlaps, failures are interleaved with successes rather than clustered (
id=5fail,7and10ok,11fail), and the only suggestive signal (probed > handshake port in 4/4 failures) is p≈0.11 against a 57% baseline, i.e. nothing. Everything the server can observe looks identical in the runs that work and the runs that die. -
So the remaining suspect is the CLIENT's pinhole, which is where Finding Q's still-missing measurement always pointed: does the client's own round-1 fast-open (TTL-5) priming packet actually leave its carrier NAT? If that priming is dropped, there is no pinhole, every server probe dies, and the outcome is binary and intermittent — matching all 25 observations. Next test: capture at the client egress. Use a laptop tethered to mobile data as the client (the phone cannot tcpdump unrooted). Ruled out on the way: the phone is NOT running a stale client — its
libhyperdht_jni.sois dated 2026-08-04 and B2 (f6608b7) landed 2026-08-02 — and the1 addrsit gossips is correct, JS-identical behaviour for a port-consistent NAT (all samples collapse to one_samplesFullentry, so JS emits one address too). -
Diagnostic gap to close first: the DHT_LOG output has no timestamps, so nothing in these logs can be correlated with wall-clock, inter-attempt spacing, or carrier events. Add a timestamp prefix to
DHT_LOG(or run the server undersystemd-cat/journal and read it there) before the next field round — several questions here were unanswerable purely for want of a clock. -
nospoon needs one line when its pin is bumped: hold
info.socket_keepalivealongside the peer's duplex (~/Desktop/repos/nospoon/cpp/server.cpp:93-97connects synchronously today, so it currently parks one fd per punched connection).
-
SocketRef::was_busy_is dead — the linger path is unreachable, soreusableSocket/retryRoute does not work.git grep was_busy_gives three hits: the= falseinitialiser, the read inif (reusable && was_busy_)(src/socket_pool.cpp:90), and the= falseinside that branch. Never set true ⇒LINGER_TIME_MS,lingering_,linger_timer_,on_linger_timeout,on_linger_closeand the body ofunlinger()are all dead code. JS lingers a reusable socket 3 s past its last ref precisely so the route survives a follow-up connect; we close immediately andSocketPool::remove→gc_routes_for_socketdrops the route with it. A real JS-parity gap, separate from this branch. -
SocketRef::close_maybealso lacks JS_closeMaybe'ssocket.idlegate (socket-pool.js:159) — JS never ATTEMPTS a close while a stream is attached. Worth doing WITH the item above, not before it: without the linger retry there is no hook to close on later, so gating alone just converts a counted refusal into a silent never-close. -
PoolSocket::reply()(src/holepunch.cpp) has no production caller — onlytest/test_pool_socket_requests.cpp. The round reply goes out throughsocket_.udp_send(..., via)instead. Either wire it or delete it. - Sampling PINGs use
{ retry: false }(one send), matching JS nat.js:59 (2026-11-05); ordinary pooled RPCs retain their default retry budget. - Sampling campaigns deduplicate relay/routing/bootstrap endpoints before sending, matching JS nat.js's host:port visited gate (2026-11-05).
- Review the remaining sampling campaign selection/retry policy against JS's visited-node second pass. The deliberate seven-target margin and current selection policy were retained by the per-request retry and same-campaign deduplication fixes; cross-campaign visited state is deferred.
-
QueryWalk.DownHintEmittedToReferrerOnTimeoutis load-flaky: it needs ~2.02 s of wall clock and failed once in a parallelctestrun, then passed standalone 3/3 and in a full rerun. Untouched by this branch (src/query.cppandtest/test_query.cppare identical tomain). Give it slack or a deadline that is not wall-clock-tight. - Abandoned
ServerConnectionraw streams now releaseRawStreamCtxon destruction and move assignment (2026-10-23). All 11test_server_punch_socketcases pass with ASan, UBSan and leak detection; this does not claim that every legacy test fixture is leak-free. -
SocketPoolTest.DestroyClosesAllaccounts for the whole 7488 Btest_socket_poolASAN leak:destroy()deliberately never deletesSocketRefs (an outstanding keepalive may still callinactive()on one). Expected, not a defect. -
ServerConnection's hand-written move ops exist solely to nullother.raw_stream. Wrappingraw_streamin a move-only holder and defaulting both would delete the bug class that just cost usrelay_token.