Skip to content

feat(payments): NIP-47 client core and mock wallet for the NWC rail - #117

Open
nisargpatel7042lva wants to merge 7 commits into
ContextVM:mainfrom
nisargpatel7042lva:feat/cep8-nwc-client-b1
Open

nisargpatel7042lva wants to merge 7 commits into
ContextVM:mainfrom
nisargpatel7042lva:feat/cep8-nwc-client-b1

Conversation

@nisargpatel7042lva

Copy link
Copy Markdown

part of #116

First of the three NWC sub-PRs (B1 of B1-B3). Rail infrastructure only: no processor, no handler, and nothing touched in the Phase A traits, middlewares or registration entry points. Based on 848144c, no rebase needed.

Everything lands behind the off-by-default nwc feature, which only switches on nostr-sdk/nip47 and adds no new crate to the tree (nip47 = ["nip04"] upstream, and NIP-04 is the encryption NWC actually uses). --no-default-features, default, and --all-features all build; MSRV 1.88 compiles.

NwcClient speaks NIP-47 request/response and notifications over an injected Arc<dyn RelayPoolTrait>. The nwc crate is deliberately not used: it owns its relay pool privately, so nothing built on top of it can be driven by MockRelayPool and every test would need a live wallet. With injection, the same code path runs against a scripted mock wallet in CI and a real wallet in production, which is what makes the relay-level merge gate in #116 possible at all.

The main thing to review: one subscription, not one per request

ts-sdk opens a subscription per request and closes it on settle. RelayPoolTrait has no unsubscribe, so transliterating that shape would leak one live REQ per payment for the lifetime of the process. This client instead opens a single subscription covering kind 23195 responses and both notification kinds, runs one reader task, and correlates in process through a waiter map keyed by the request event id.

Two consequences worth checking:

  1. The waiter is registered before the publish, so a wallet that answers instantly cannot arrive before anything is listening for it. The ts shape has an explicit settled flag and a settle() closure to paper over the same race from the other direction.
  2. A lagged broadcast continues rather than killing the loop. Dropping events is survivable; killing the reader strands every waiter forever with no timeout path of its own. reader_survives_broadcast_burst floods past the channel's 1024 slots and then asserts a fresh request still correlates.

Correlation is also an authorization check, and both halves are tested: stranger_signed_response_is_ignored (a third party publishing a well-formed response to our pubkey) and uncorrelated_response_is_ignored (a genuinely wallet-signed response carrying an e tag for a request we never made). Neither settles a pending waiter.

Deliberate divergences from ts

  1. Permissive response parsing. Every field on the result types is optional and unknown fields are ignored, field for field with ts nip47/types.ts, rather than going through nostr's strict response structs. On a money path a wallet that omits an optional field, or adds one, must not turn a paid invoice into a failed verification. Settlement is read from state == "settled" or settled_at > 0, since wallets populate one or the other and requiring both would strand real payments.
  2. Notifications are decrypted by kind: 23196 is NIP-04, 23197 is NIP-44. ts decrypts everything as NIP-04 and so silently drops the NIP-44 form. Note 23197 has no named constant in nostr 0.44 (the Kind enum stops at 23196), hence Kind::from(23197u16). Both paths are tested against a mock wallet that can emit either.
  3. parse_nwc_uri accepts both URI shapes. NostrWalletConnectURI::parse reads the pubkey from url.domain() only, so it rejects the nostr+walletconnect:<pubkey>?... pathname form that ts accepts (url.pathname?.replace(/^\//, '') || url.host). A connection string that works against the ts-sdk must not fail here. parses_path_form_that_upstream_rejects pins the upstream rejection as an explicit precondition, so if upstream ever fixes this the test tells us rather than quietly going redundant.
  4. sats_to_msats rejects a non-positive amount and uses checked_mul, both before any wallet call, where ts does a bare multiply. A zero or negative amount should never reach a wallet as an invoice request, and a caller-controlled amount should not be able to wrap.
  5. Requests are concurrent, correlated by id, where ts serializes them behind a promise queue. concurrent_requests_correlate_independently runs a slow make_invoice against a fast pay_invoice and asserts neither receives the other's answer.

One API decision that B2 depends on

A wallet-reported error comes back as Ok(envelope) with error populated, not Err. B2 has to treat NOT_FOUND as "still pending, keep polling" but other codes as fatal, and that classification needs the code at the call site rather than a string parsed back out of a PaymentError. This also matches ts, which returns the parsed envelope when parsed.error is set.

So that a caller who does not classify cannot mistake a failure for a settlement, NwcResponseEnvelope::into_result() collapses the envelope into a Result, synthesizing an EMPTY_RESPONSE error for the case where a wallet sends neither result nor error. wallet_error_is_surfaced_not_read_as_success asserts both halves.

MockWallet, behind test-utils

Runs the real wire protocol over a linked MockRelayPool rather than stubbing the client: it subscribes for kind 23194, decrypts, and publishes kind 23195 back, so the client exercises its own encrypt, publish, correlate and decrypt paths in CI. Scripted per method as answer, error, silence, delay, or malformed, with settle() publishing a payment_received notification in either encryption mode and publish_info_event() for the kind 13194 path. An unscripted method stays silent, like a wallet that does not implement it.

keys() is exposed so a test can forge an event that is genuinely wallet-signed, which is what the uncorrelated-response test needs.

Verification

cargo fmt --check clean, cargo clippy --all-features --all-targets clean (0 warnings), 1084 tests pass under --all-features with 0 failures, and MSRV 1.88 compiles with --all-features. The nip47 module itself adds 38 tests covering all nine cases listed for B1 in #116.

One process note on the numbers: an earlier run of mine showed six failures in payments_bilateral_e2e, which turned out to be two concurrent cargo jobs starving each other on my machine, not a regression. On an idle machine that suite is 16/16 across three consecutive runs at the same runtime as main.

Not in this PR

The processor (B2) and the handler, e2e suite and docs (B3). docs/payments.md still describes Phase B as deferred and still lists LNbits as planned; both are B3's to correct, per the checklist in #116. No real-wallet test yet either, for the same reason: it belongs with the rail it exercises.

First of the three NWC sub-PRs in ContextVM#116. Rail infrastructure only: no
processor, no handler, and no change to the Phase A traits, middlewares
or registration entry points.

Behind the off-by-default `nwc` feature, which only switches on
`nostr-sdk/nip47` and adds no new crate to the dependency tree.

- `parse_nwc_uri` reads a connection string in either shape wallets emit.
  `NostrWalletConnectURI::parse` takes the pubkey from the URI host only,
  so it rejects the `nostr+walletconnect:<pubkey>?...` pathname form the
  ts-sdk accepts; a connection string that works there must not fail
  here. A test pins the upstream rejection as a precondition.
- `NwcClient` speaks request/response and notifications over an injected
  `RelayPoolTrait`, so the same code path runs against a scripted mock
  wallet in CI and a real wallet in production. The `nwc` crate is not
  used: its relay pool is private, so nothing built on it is testable
  without a live wallet.
- One subscription and one reader task, correlating by request event id.
  `RelayPoolTrait` has no unsubscribe, so the ts-sdk
  subscription-per-request shape would leak one live REQ per payment.
- Wallet responses are parsed permissively, field for field with ts
  `nip47/types.ts`, rather than through the strict upstream response
  structs. On a money path a missing or unknown optional field must not
  turn a paid invoice into a failed verification.
- Notifications are decrypted by kind (23196 NIP-04, 23197 NIP-44). ts
  always decrypts as NIP-04 and so silently drops the NIP-44 form. Kind
  23197 has no named constant in `nostr` 0.44, hence `Kind::from`.
- `sats_to_msats` rejects a non-positive amount and uses `checked_mul`,
  both before any wallet call, where ts does a bare multiply.
- `MockWallet` behind `test-utils` runs the real wire protocol over a
  linked `MockRelayPool`, scripting answer, error, silence, delay and
  malformed replies, and can publish a `payment_received` notification.

A wallet-reported error is returned as `Ok(envelope)` with `error` set
rather than `Err`, matching ts: the processor in B2 has to treat
`NOT_FOUND` as pending but other codes as fatal, which needs the code at
the call site. `into_result()` collapses the envelope for callers that
do not classify, so a failed payment still cannot read as a settled one.

Verified: fmt and clippy clean, 1084 tests pass under `--all-features`,
`--no-default-features` and default builds green, MSRV 1.88 compiles.

Refs ContextVM#116
@nisargpatel7042lva

Copy link
Copy Markdown
Author

The NWC rail is now up in three stacked PRs, this one first:

Each is part of #116 rather than Fixes, so none of them closes the Phase B tracker while the zap rail is still outstanding. Review order is 117, 118, 119; each is based on its parent and I will rebase onto main as they merge.

All six workflow runs (CI and FFI, one pair per PR) are sitting at action_required, so nothing has executed yet. They need the "Approve and run workflows" click on each fork PR whenever someone has a moment.

Locally, running the same gates CI does: cargo fmt --check clean, cargo clippy --all-features --all-targets clean, 1121 tests passing under --all-features with 0 failures, --no-default-features and default builds green, MSRV 1.88 compiling, and cargo doc building. One thing I could not check here is ffi.yml's macOS cross-compilation targets, since I have no macOS toolchain; the changes are confined to contextvm-sdk behind an off-by-default nwc feature and do not touch contextvm-ffi, but that is reasoning rather than a verified run.

Two items from #116 are still open and not covered by these three: the mutation testing of the money paths that B2 asks for, and the real-wallet runs. The real-wallet tests are written, #[ignore]d and environment-gated, but I do not have two funded wallets to point at them, so that exit-criteria box is not ticked. Happy to run it if someone can supply two throwaway connection strings.

@harsh04044 harsh04044 left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed against NIP-47, the ts reference and the money path this will carry, with 32 probes, a 27-mutant sweep and the full gate behind it.

What is right. The design is right, and correlation held everything I threw at it: 20 rounds of out-of-order answers, two clients and two wallets on one pool, one shared-key wallet on two connections. The gate is green on its exact commands (1162 / 0 / 4 all-features, 830 / 0 / 2 no-default, MSRV 1.88 on a fresh lockfile).

What blocks is wallet-facing robustness and money-path bounds, and no test here can see either: the mock and the client were written together, so the mock agrees with the client wherever the client is wrong. The suite kills 12 of 27 mutants. #118 and #119 will need to rebase, since some fixes touch API they consume.

Critical

  1. One panic on the reader kills the rail for the process, silently, while request() keeps publishing. Triggers: a panicking sink, and a malformed NIP-04 payload under nostr 0.44.5, which "0.44" permits. Isolate per-event handling, drain and close on reader exit, floor at 0.44.7.
  2. One off-type optional field throws out the whole response. #[serde(default)] covers absent, not wrong-typed. #118 reads that as a failed verification: the payer pays and gets nothing. Off-type becomes None.

Major

  1. The response filter needs to widen itself when it has never heard anything. Ours, not yours: keep #p, add one re-subscribe without it while nothing has ever correlated.
  2. Publish, connect and subscribe sit outside response_timeout, and the rustdoc claims otherwise. One deadline over start-up, publish and wait.
  3. Dropping a request() future orphans its waiter for good, which #118's select! does on every cancelled verification. An RAII guard here means #118 needs no change.
  4. Three of the four guards the PR body calls tested have tests that cannot fail, and the concurrency test cannot produce out-of-order answers. The guards are right; the tests need to bite.
  5. shutdown() is not fail-fast: pay_invoice still published and the wallet still paid. A closed flag checked at the top of request().
  6. Every failure is PaymentError::Processor(String), so #119 reports a wallet timeout as Processor, and nothing says "may have reached the wallet". A typed error with Timeout { published }, mapped at the trait boundary.
  7. Error text can carry a BOLT11 invoice into operator logs. Both middlewares log the whole PaymentError at warn. An invoice must never reach a log.
  8. MockWallet shares the client's assumptions, twelve of them, so #118 and #119 inherit the blind spots. Knobs for each, and spawn per request.
  9. The public API is too loose to freeze. Typed methods, request crate-internal, #[non_exhaustive] on the wire types, and the options builder landed here so #119 stops editing this file.
  10. The client must own its relay pool. connect adds the wallet's relays to whatever pool it gets, and the subscription is permanent, but #119 passes the transport's.

Minor, blocking: one-liners, or a semver break if deferred

  1. The nwc feature does add crates, aes and cbc. The "adds no new crate" claim is in six places.
  2. Decrypt notifications by ciphertext shape, not kind. A dual-encryption wallet fires every sink twice per payment, and ts's own fixture is dropped here.
  3. No since on the subscription, so every start and reconnect replays retained history. Minutes of skew, not ts's 5 s.
  4. Accept the legacy nostrwalletconnect:// scheme, add ts's own tested shape as a case, fix the doc line about the upstream type.
  5. Reject a response_timeout of zero. Today it is accepted and the request still publishes.
  6. Add an optional expiration to the request API. ts takes expirationSeconds (nwc-client.ts:174, 198-200) and the NIP allows it. #119 needs it so a late delivery cannot pay after the client gave up, and adding it later is a public API change.
  7. Five rustdoc statements do not match behavior (client.rs:20, :326, :264-268, types.rs:9-10, :18-19).

Non-blocking

  1. result_type is never checked. Permissive is the money-safe direction, since correlation by event id already proves which request an answer belongs to. Keep it, warn on a mismatch, fix the doc.
  2. Duplicate secret parameters are last-wins here, first-wins in ts. Pick deliberately.
  3. The last e tag wins where ts takes the first. Parity note only: both misbehave on multi-e answers.
  4. fetch_info_notification_types has two quiet side effects (it starts the subscription and caches), and the client tests need test-utils as well as nwc. A line of docs each.

Also CHANGELOG.md:9 says "for the Phase B Lightning rail", which is our internal planning vocabulary.
Say what it does instead.

Two other things

  • Your six payments_bilateral_e2e failures: not reproduced in 36 runs, including under heavy load and four copies at once, and not a regression from this PR, which does not touch that code. Could you send the test names and the exact command? Also, the "1084 tests" in the PR body is 1162 minus the 78 FFI tests, so the run looks like it was without --all.
  • Evidence: every "executed" or "measured" claim comes from 32 probes plus a mutation driver with the per-test kill table. Say the word and I will post both as a gist so you can reuse the probes as tests rather than take any of this on trust. Happy to send working patches for 2, 5 and 8 too.

let inner = Arc::clone(&self.inner);
let keys = self.keys.clone();
let cancel = self.cancel.clone();
tokio::spawn(async move {

@harsh04044 harsh04044 Oct 8, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(1) Critical. This task is unsupervised and nothing restarts it. *started = true below survives its death, so no resubscribe ever happens, and request() keeps publishing and waiting out the full timeout. A panicking notification sink reaches it (line 303), and so does one malformed NIP-04 payload under nostr 0.44.5, which "0.44" permits. Executed both ways: every later call times out, and pay_invoice is still paid.

};
for sink in sinks {
sink(payload.clone());
}

@harsh04044 harsh04044 Oct 8, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(1). Sinks are caller code run synchronously on the reader. One panic here takes the rail down for the life of the process. A bounded channel, or catch_unwind around the handlers, keeps it to the one sink.

/// Every field optional: wallets differ on which they populate, and a missing
/// `state` with a non-zero `settled_at` still means settled.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
pub struct NwcInvoiceResult {

@harsh04044 harsh04044 Oct 8, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(2), Critical. #[serde(default)] covers a field that is absent, not one that is present with the wrong JSON type, so one off-type optional field fails the whole envelope and request() returns Err. The doc comment above is true of absence and false of type. Executed: a string settled_at, an ISO expires_at, a float amount and an integer state each lose a response that said "state":"settled". #118 reads that as a failed verification, so the payer pays and gets nothing. ts never type-checks at runtime.

pub preimage: Option<String>,
/// Routing fees paid, msats.
#[serde(default)]
pub fees_paid: Option<i64>,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(2). Same hazard: a wallet that computes fees_paid as a float fails every pay_invoice response.

Comment thread src/payments/nip47/client.rs Outdated
Kind::from(KIND_NOTIFICATION_NIP44),
])
.author(wallet_pk)
.pubkey(client_pk)];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(3). The #p here is ours, not your mistake, and the requirement is changing: please keep it on responses, keep it on notifications, and add one self-widening re-subscribe without it while the client has never yet matched a single response. Reasoning and the wallet evidence are in (3). ts's filter has no #p but narrows per request on #e over a REQ it closes, which we cannot do, so a bare filter here is permanent and broader than anything ts opens.

Comment thread src/payments/nip47/uri.rs Outdated
//! `nostr+walletconnect:<pubkey>?...` form (no `//`, pubkey in the path) that
//! ts-sdk accepts. Wallets in the wild emit both, and a connection string the
//! ts-sdk takes must not fail here, so this module parses both shapes and then
//! hands the pieces to the upstream type.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(16). This module never constructs NostrWalletConnectURI, so the sentence is not accurate. Separately: please accept the legacy nostrwalletconnect:// scheme (ts never checks the scheme) and add ts's own tested shape, nostr+walletconnect://ignored-host/<pubkey>?..., as a case. The mutant that makes the host win over the path survives today.

Comment thread src/payments/nip47/types.rs Outdated
/// Envelope every NIP-47 response shares: `{result_type, error, result}`.
///
/// `error` and `result` are both nullable per NIP-47, and a wallet may send
/// neither. `result_type` is the only field the client relies on.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(19). result_type is never checked anywhere. Keeping it permissive is the right call, since correlation by event id already proves which request an answer belongs to, so please keep the behavior and fix this line. Warning on a present-but-different value would be worth it. See (20).

Comment thread src/payments/nip47/mod.rs Outdated
//! [`RelayPoolTrait`](crate::relay::RelayPoolTrait).
//!
//! Behind the off-by-default `nwc` feature, which only switches on
//! `nostr-sdk/nip47` and adds no new crate to the tree.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(13). Not accurate: cargo tree -e normal --features nwc adds aes v0.8.4 and cbc v0.1.2, and a fresh generate-lockfile gains aes. The same claim is in Cargo.toml, CHANGELOG.md, the PR body, the commit message and #119's docs. Suggested wording: adds two small RustCrypto crates for NIP-04.

Comment thread Cargo.toml
test-utils = []
## CEP-8 Lightning rail over NIP-47 (Nostr Wallet Connect). Off by default.
## Switches on an existing `nostr-sdk` NIP; adds no new crate to the tree.
nwc = ["nostr-sdk/nip47"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(13). Same claim as nip47/mod.rs:9: the feature does add aes and cbc.

Comment thread CHANGELOG.md Outdated

- CEP-8 capability pricing and payments (in progress; foundational pieces, not yet a
usable payment flow):
- NIP-47 (Nostr Wallet Connect) client infrastructure for the Phase B Lightning rail,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(13) and a wording note. "the Phase B Lightning rail" is our internal planning vocabulary and a changelog reader cannot resolve it; say what it does instead. The "adds no new crate" sentence below needs the same fix as nip47/mod.rs:9, and "decrypted by kind" describes the behavior in (14) that we are asking you to change.

Addresses the review on ContextVM#117. The design held; what did not was
robustness against a wallet that misbehaves, and the money-path bounds.
A mock written alongside the client agreed with the client wherever the
client was wrong, so most of this was invisible to the old suite.

Critical

- One panic on the reader killed the rail for the process, silently,
  while requests kept publishing. Per-event handling now runs under
  catch_unwind, the reader marks the client closed and drains its
  waiters on exit, and `nostr` is floored at 0.44.7 (0.44.5 panics on a
  malformed NIP-04 payload, and "0.44" permitted it).
- One off-type optional field threw out the whole response.
  `#[serde(default)]` covers absent, not wrong-typed, so a string
  `settled_at` lost a result that said `"state":"settled"` and the payer
  would have paid for nothing. Every optional field now deserializes
  off-type values to `None`.

Major

- The response filter widens itself once, dropping `#p`, while nothing
  has ever correlated, so a wallet that omits the tag is not invisible.
- One deadline now covers start-up, publish and the wait, where publish,
  connect and subscribe used to sit outside it.
- A dropped request future no longer orphans its waiter: an RAII guard
  removes it. The processor's `select!` drops one per cancelled verify.
- `close()` is fail-fast. It used to leave `pay_invoice` publishing, and
  the wallet still paid.
- Errors are typed. `NwcError::Timeout` carries `published`, so a caller
  can tell "the wallet may have acted" from "it certainly did not",
  which is what a non-idempotent `pay_invoice` needs, and the trait
  boundary maps to Processor or Handler explicitly.
- Error text can no longer carry a BOLT11 invoice into operator logs:
  wallet free text is redacted and bounded, and a parse failure reports
  its category and position instead of the offending bytes.
- `MockWallet` can now break the client's assumptions: omit the `p` tag,
  answer NOT_IMPLEMENTED, mislabel `result_type`, drop the `e` tag,
  publish both notification kinds, and answer out of order. It serves
  each request on its own task, so a fast answer really can overtake a
  slow one.
- The public API is typed: `make_invoice` / `lookup_invoice` /
  `pay_invoice`, `request` crate-internal, wire types
  `#[non_exhaustive]`, and the options builder lands here.
- `NwcClient::connect` builds the client its own pool. `with_pool` stays
  for tests, documenting why sharing a transport's pool is a trap.

Minor

- Corrected the "adds no new crate" claim, which was wrong: the feature
  pulls in NIP-04 and so adds `aes`, `cbc` and `cipher`. Measured.
- Notifications decrypt by ciphertext shape, not event kind, so NIP-04
  content on kind 23197 (ts's own fixture) decodes.
- The subscription carries `since`, so a retaining relay does not replay
  this client's whole history into the sinks.
- The legacy `nostrwalletconnect://` scheme parses.
- A zero `response_timeout` is refused at construction.
- `pay_invoice` takes an optional NIP-40 expiration.
- Rustdoc corrected where it no longer matched behavior, including the
  claim about handing pieces to the upstream URI type, which it never did.

Non-blocking: a mismatched `result_type` is tolerated and warned about,
since correlation by event id already proves which request an answer
belongs to; duplicate `secret` is documented as last-wins; the `e` tag
parity note is recorded; the two quiet side effects of
`fetch_info_notification_types` are documented.

Tests go 38 to 61, and now bite: a five-mutant sweep over the author
check, the waiter guard, the closed flag, decrypt-by-shape and the
off-type leniency kills all five. The author check needed
`MockRelayPool::publish_event_unfiltered`, which models a relay that
ignores `authors`; without it the guard is unreachable, which is why
deleting it used to survive the suite.

Verified: fmt and clippy clean, 1104 tests pass under `--all-features`,
feature matrix green, MSRV 1.88 compiles.

Refs ContextVM#116
@nisargpatel7042lva

Copy link
Copy Markdown
Author

Thank you for this. The probes found things no test of mine was going to, and two of them were real money-path bugs. All 23 items are addressed in 0ef9c44 and bf61f0b.

You were right and I was wrong on (13). I claimed "adds no new crate" in six places and never measured it. cargo tree with and without the feature: 178 crates against 181, adding aes 0.8.4, cbc 0.1.2 and cipher 0.4.4, because nip47 pulls in nip04 and NIP-04 is AES-CBC. Corrected everywhere, with the reason stated next to the feature.

Critical

(1) Per-event handling now runs inside catch_unwind, so a panicking sink is contained to its event. The reader marks the client closed and drains its waiters when it exits, so a dead rail fails fast instead of publishing into a void and waiting out the full deadline. nostr is floored with a direct >=0.44.7, <0.45 dependency, commented as a security floor rather than a usage one, since the panic is reachable from wallet-controlled NIP-04 content on the reader task.

(2) Every optional field now deserializes a wrong-typed value to None through one lenient helper. The doc comment that conflated absent with wrongly-typed is rewritten to call the distinction out explicitly, since it is exactly the trap. There is a test with a string settled_at, an ISO expires_at, a float amount and an integer state in one response that still reads "state":"settled", and a float fees_paid case for pay_invoice.

Major

(3) #p kept on both; one self-widening re-subscribe without it while nothing has ever correlated, latched off after the first success.

(4) One deadline across start-up, publish and wait. The rustdoc that claimed this before it was true is fixed.

(5) WaiterGuard removes on drop. As you say, #118 then needs no change.

(6) This was the most useful item. All three guards now have tests that bite, and I verified it rather than asserting it: a five-mutant sweep over the author check, the waiter guard, the closed flag, decrypt-by-shape and the off-type leniency kills all five.

The author check needed infrastructure. MockRelayPool filters correctly on publish, so a stranger's event never reaches the reader and the guard is unreachable; my first rewrite of that test still did not bite. There is now MockRelayPool::publish_event_unfiltered, documented as modelling a relay that ignores authors, which is the exact condition the guard defends against given verify_subscriptions is false. Deleting the author check now fails that test.

(7) close() sets a flag checked at the top of every request, and is also set by Drop and by reader exit, with waiters drained. The test asserts the wallet recorded zero pay_invoice calls afterwards, not merely that the call returned an error.

(8) A typed NwcError with Timeout { published } plus may_have_reached_wallet(), and explicit into_processor_error / into_handler_error at the trait boundary so #119 reports Handler.

(9) Two leaks, both closed. describe() redacts BOLT11-looking tokens and bounds the text; parse failures report category plus line and column and never the bytes. Tests assert an invoice in a wallet message does not survive into the error.

(10) MockWalletQuirks covers omitting the p tag, NOT_IMPLEMENTED for unscripted methods, duplicate notification kinds and NIP-44 notifications, plus behaviors for a mislabelled result_type and a missing e tag. Requests are served on a task each, so a fast answer really does overtake a slow one; the correlation test now depends on that.

(11) Typed make_invoice / lookup_invoice / pay_invoice, request is pub(crate), the wire types are #[non_exhaustive], and the options builder lands here so #119 stops editing this file.

(12) NwcClient::connect builds the client its own pool from the connection relays. with_pool stays for tests, and the module docs state the requirement and spell out the trap.

Minor

(14) by ciphertext shape, with ts's NIP-04-on-23197 fixture as a test. (15) since at subscribe. (16) legacy nostrwalletconnect:// accepted, ts's pinned shape added as a case, and the doc line about handing pieces to the upstream type removed, since it never did. (17) zero refused at construction. (18) optional NIP-40 expiration, wired through pay_invoice. (19) rewritten.

Non-blocking

(20) kept permissive, warns on mismatch, doc fixed, with the reasoning you gave recorded in the type. (21) last-wins, documented as a deliberate divergence from ts's first-wins, with a test. (22) parity note in the code. (23) both side effects documented, and the test-utils requirement noted. CHANGELOG no longer uses the internal phase vocabulary.

Your two questions

The failing suites. You were right not to reproduce them, and my report was not good enough to act on. The names were payments_bilateral_e2e and, in a later run, payments_server_e2e and e2e_happy_path; the command was cargo test --all-features. What they have in common is a hard tokio::time::timeout(Duration::from_secs(5), ...) around an rmcp handshake that normally finishes in about 0.1 s.

Isolated, both branches pass repeatedly: 8/8 on mine, 5/5 on main, around 1.1 s each. I did get one isolated failure at 5.28 s where main passed at 0.17 s, which looked damning, so I reran with the order reversed and both passed cold at about 0.10 s. The 5 s budget was being blown by build invalidation inside my own comparison script, not by either branch. So: environmental on this machine, not a regression, and nothing for you to chase. Sorry for the noise.

The test count. Your arithmetic is right: 1084 was 1162 minus the 78 FFI tests, so that run was workspace-root only rather than --all. The current numbers on 0ef9c44, same command you used, are 1107 passing with 0 failures and 4 ignored, fmt and clippy clean, --no-default-features and --all-features both green, MSRV 1.88 on a fresh lockfile.

On your offer

Yes to the gist, please, for the probes and the kill table. I would rather fold your probes in as tests than have the suite depend on my having read your findings correctly, and I would like to see which of the 27 mutants my five do not cover. No need for the patches for (2), (5) and (8); those are in, and comparing against yours will be more useful than starting from them.

#118 and #119 are parked and I am rebasing them onto this now. The ones that reach them are the typed error, request becoming pub(crate), the options builder moving here, and with_pool replacing the old constructor, so #119 will also drop the builder it was adding to this file.

The quirks struct was reachable only by its full module path, so a
downstream test could not use it, and the module doc still described the
client as taking an injected pool after `connect` made it own one.
Rust 1.99 deprecates `Atomic::fetch_update` in favour of `try_update`,
and CI lints with `-D warnings`, so `cargo clippy --all-targets` fails
on main. `try_update` is only stable from 1.95, above the declared MSRV
of 1.88, so this spells the same decrement-while-non-zero out as a
compare-exchange loop, which compiles warning-free on both.
@nisargpatel7042lva

Copy link
Copy Markdown
Author

CI's fmt + clippy was red on this PR and the cause was not in the diff. tests/transport_integration.rs has used AtomicUsize::fetch_update since August, Rust 1.99 deprecates it in favour of try_update, and the lint job runs -D warnings on dtolnay/rust-toolchain@stable. Main would fail identically today; its last green run was 4 September, before the toolchain moved. My own stable was five months behind, which is why my gate never saw it. I am now pinned to 1.99.0, the same rustc the job reported.

The compiler's suggested try_update is stable from 1.95, above the declared MSRV of 1.88, so taking it would trade the lint job for the MSRV job. d1860e0 spells the same decrement-while-non-zero out as a compare-exchange loop, which is warning-free on both ends of that range. It is a separate commit touching only that file, so if you would rather have it as its own PR against main, say so and I will lift it out.

Two rustdoc defects of mine went in with the same push, folded into 95dc0e9. Making request pub(crate) for item (11) left four intra-doc links pointing at private items (Self::request in the three typed methods, lenient in the types module header), plus one redundant explicit link target. cargo doc is back to zero warnings. The doc step does not pass -D warnings, so CI was never going to tell either of us.

Gate on 1.99.0, matching the job: fmt clean, clippy clean on --all --all-features --all-targets -D warnings, cargo doc zero warnings, 856 lib, 78 transport_integration, 5 payments_nwc_e2e, 830 / 0 on --no-default-features, MSRV 1.88 checks on a fresh lockfile.

On the flaky suites, I have the controlled comparison I should have run the first time. Two worktrees, each with its own target directory on the same filesystem, both prebuilt, ten consecutive runs of e2e_happy_path each, interleaved: main failed 1/10, this stack 0/20. Every failure sits at the hard 5 s timeout around a client startup that normally finishes in 0.09 s. Pre-existing, not branch-specific, and it does reach main, so the earlier conclusion holds but on better evidence than I had for it.

#118 and #119 are rebased onto this and pushed. All three runs will need the workflow approval again after these pushes.

@nisargpatel7042lva

Copy link
Copy Markdown
Author

Correction to my previous comment. #120 already carries this exact fix, the same compare-exchange loop in the same file, and its CI is green. I should have checked the other open pull requests before writing my own, and I did not.

So there is nothing in d1860e0 worth your review. Merge #120 whenever it suits you and I will rebase this stack onto main and drop that commit, which leaves #117 as the three payments commits it should be. The only reason it is still on the branch is so the lint job can pass if you approve these runs before #120 lands. It touches the same hunk as #120, so the rebase conflict resolves by taking main's version.

The rustdoc fix folded into 95dc0e9 is unaffected by any of this and does belong to this PR.

@harsh04044 harsh04044 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-verified against d1860e0. Each item below is backed by a probe or a mutant I ran.

Five things block merge.

(8a) A publish cut off by the deadline is reported as never sent. The Err(_) arm of timeout_at(deadline, publish_event(&event)) returns published: false. That arm fires while the publish is still in flight, and the real pool sends the EVENT before it waits on the relay's OK, so the one genuinely unknowable case is the only one reported as a definite non-delivery. In my probe the wallet had already recorded the pay_invoice. Since may_have_reached_wallet() is what a caller reads to decide whether a retry is safe, unknown has to stop collapsing into false: either a third state, or that predicate returning true for a timeout that fired during the publish phase. Ok(Err(e)) is correct as it stands, because a relay that answers with an error did not accept the event.

(24) A close or a reader exit after the publish claims the same thing. Once the publish has succeeded, the wait arm maps a dropped sender to a bare Closed, which discards the published: true sitting beside it, and may_have_reached_wallet() ends _ => false. So a close() that lands after a pay_invoice went out tells the caller a retry is safe. Closed needs to carry the same published bit. Separately, a close() during start-up does not prevent the request: the flag is checked at the top and close() returns before the publish happens, so it needs re-checking after start-up, or holding across that phase.

(25) The redactor panics on wallet-controlled text. sanitize_wallet_text ends with out.truncate(120) over raw wallet words, so it panics whenever byte 120 falls inside a multi-byte character. A "payment failed" message in Chinese is enough, and it is reachable from a wallet error on the pay path. floor_char_boundary is unstable at 1.88 (E0658, round_char_boundary) and compiles only from 1.95, which is the same MSRV trade you spotted on try_update, so
while !out.is_char_boundary(n) { n -= 1 } is the version that holds on both ends.

(26) Wallet text still reaches logs unredacted, three ways.

  • describe() copies the code raw and unbounded. format!("{code}: {msg}") sanitizes only the message, and the code is wallet-controlled.
  • A mislabelled result_type is logged raw at warn.
  • The prefix test is starts_with on whitespace-split words, so lightning:lnbc..., (lnbc...), invoice=lnbc... and quoted forms all pass through, and the leak includes the full payment hash. Matching the prefix anywhere in the token, or stripping punctuation and a lightning: scheme before the test, closes it.

(3) The widening is not in the shape it was asked for. The latch itself is right, and a wallet that omits p is recovered. Four departures:

  • the guard is only if ever_correlated { return; }, so a wallet that never answers gets a fresh broad subscription on every call (four calls gave four), and each is permanent because the pool has no unsubscribe;
  • widening runs immediately after the first publish, before anything can have correlated, so a conformant shared-key wallet also gets the broad filter and keeps it for the life of the process. The condition wants to be "the narrow filter has demonstrably failed", not "nothing has arrived yet";
  • one false goes to filters(false), so #p is dropped from 23196 and 23197 as well. Only the response filter needs widening;
  • it logs at debug, where warn is the level that gets a non-conformant wallet noticed.

Not blocking merge, but still worth doing:

Test coverage. 17 of 35 mutants live through the suite. Three of those are on money paths: the Lagged arm breaking the reader, registering the waiter after the publish instead of before, and bypassing the waiter guard on the timeout and publish-failure paths. Each of those guards is correct; nothing proves it stays correct. Nine of the twelve mutants on the new fixes also survive, so the widening latch, since at subscribe, the expiration tag, the reader-exit drain, the mismatch warning and the payload-free parse error all work but are untested. Separately, uncorrelated_response_is_ignored is worth keeping alongside its replacement: an answer carrying no e tag is dropped before the correlation fallback is reached, so the mutant that falls back on an e mismatch used to die and now survives.

Remainders on (4), (6), (8b), (9), (10), (11), (13), (14), (16), (19) and (23) are on their own lines in the diff.

…led widening

Addresses the five blocking items from the second review of the NIP-47
client.

(8a) A deadline that fires inside the publish is not a non-delivery. The
pool writes the EVENT before it waits on the relay's OK, so the wallet
may have the request. `Delivery` replaces the `published` bool with
`NotSent`, `Unknown` and `Sent`, and `may_have_reached_wallet` treats
`Unknown` as "may have", which is the safe direction for `pay_invoice`.

(24) `Closed` carries the same `Delivery`, so a `close()` landing after
the request went out no longer reports a safe retry. `request` also
re-checks the closed flag after start-up: `close()` returns as soon as
it has set the flag, so one arriving during a slow connect used to be
followed by a publish.

(25) `sanitize_wallet_text` walks back to a character boundary before
truncating. `truncate` panics mid character, and the text is
wallet-controlled, so a long non-Latin message was a remote panic on the
pay path. `floor_char_boundary` is the obvious fix and is stable only
from 1.95, above the declared MSRV.

(26) Three remaining paths for wallet text to reach a log unredacted:
the error code was copied raw and unbounded, a mislabelled `result_type`
was logged raw, and the BOLT11 prefix test was `starts_with` on
whitespace-split words, which `lightning:lnbc...`, `(lnbc...)` and
quoted forms all passed while carrying the full payment hash. The prefix
is now matched anywhere in the token.

(3) Widening now requires evidence that the narrow filter has failed: a
request that reached a relay has waited out a full deadline, and nothing
has ever correlated. It is latched, since the pool has no unsubscribe
and a silent wallet would otherwise accumulate one subscription per
call; it widens the response kind only, leaving `#p` on both
notification kinds; and it logs at warn.

Tests: a nine-mutant sweep over these fixes, each killed. Two of them
survived the first pass because the delivery states were asserted in
isolation without anything proving `request` produces them, so both now
have tests that drive the real path. Also adds a response correlated to
a different request, which the existing no-`e`-tag test cannot see,
since that one is dropped before the correlation lookup is reached.
@nisargpatel7042lva

Copy link
Copy Markdown
Author

All five blockers are fixed. The probes were right on every one of them.

(8a) Delivery replaces the bool: NotSent, Unknown, Sent. The publish-timeout arm returns Unknown and may_have_reached_wallet treats it as "may have", so the one unknowable case stops being the only one reported as a definite non-delivery. Ok(Err(e)) is unchanged, for the reason you gave.

(24) Closed carries the same Delivery, so a close() after the publish keeps the bit instead of discarding it. The start-up hole is closed too: request re-checks the flag after ensure_started, since close() returns as soon as it has set it.

(25) You are right that floor_char_boundary is the same MSRV trade as try_update, so it is the is_char_boundary walk-back. The test sweeps 26 string lengths and 8 byte offsets rather than one alignment that happens to pass.

(26) All three. The code is sanitized and bounded like the message, the mislabelled result_type is scrubbed before the warn, and the prefix is matched anywhere in the token, so lightning:lnbc..., (lnbc...), invoice=lnbc... and the quoted forms are redacted with the payment hash.

(3) Reworked to the shape you asked for. Widening now needs evidence the narrow filter has failed, not merely that nothing has arrived: a request that reached a relay has waited out a full deadline, and nothing has ever correlated. It is latched with a swap, so four silent calls produce one widened subscription rather than four permanent ones. Only the response kind loses #p; both notification kinds keep it. It logs at warn.

On the tests, you were right twice over. I wrote nine mutants across these five fixes and two survived the first pass: the publish timeout reverting to NotSent, and Closed dropping the delivery bit. Both of my new tests asserted Delivery's semantics in isolation and nothing proved request ever produces those values, which is the same gap you named last round. Both now drive the real path, one through a parked publish and one through a close() during the wait. All nine die:

mutant killed by
publish timeout to NotSent a_deadline_inside_the_publish_reports_unknown_delivery
Closed drops the delivery bit a_close_after_the_publish_keeps_the_delivery_bit
drop the post-start-up re-check a_close_during_start_up_still_prevents_the_publish
plain truncate long_multibyte_wallet_text_does_not_panic
raw error code describe_redacts_and_bounds_the_error_code_too
starts_with prefix test wrapped_and_prefixed_invoices_are_redacted_too
no widening latch widening_happens_at_most_once_however_many_requests_fail
widen right after the publish three, including the p-tag recovery
widen all three kinds the_widened_filter_covers_responses_only

The two tests that needed infrastructure use a SpyPool in the test module that records every subscribe and can park connect or publish. Recording the filters is what lets the widening tests assert what the subscription matches, through match_event, rather than its shape.

an_answer_correlated_to_a_different_request_is_ignored is added alongside the no-e-tag test, for exactly the reason you gave: the uncorrelated one is dropped before the lookup, so a fallback on an e mismatch survives it.

Two things to be straight about. Your 17 of 35 surviving mutants are a superset of my nine: I have killed the ones covering these five fixes, not your set, so the Lagged arm, the waiter registration order and the guard bypass on the timeout and publish-failure paths are still unproven. I would rather say that than imply the gap is closed.

And I cannot find the remainders. The review says (4), (6), (8b), (9), (10), (11), (13), (14), (16), (19) and (23) are on their own lines in the diff, but the API returns only the 25 inline comments from the first round, nothing dated the 15th. They may not have submitted with the review. Could you check, or paste them here, and I will take them in the same pass.

Gate on 1.99.0, the toolchain CI uses: fmt and clippy clean, 1199 passing with 0 failures and 4 ignored on --all --all-features here, 1236 / 0 / 6 across the whole stack, 830 / 0 on --no-default-features, cargo doc zero warnings, MSRV 1.88 on a fresh lockfile. #118 and #119 are rebased on this and pushed.

The review measured 17 of 35 mutants living through the suite and named
the three on money paths. This kills those and five more, so every guard
these PRs added now has a test that fails when the guard is removed.

The three named ones:

- the `Lagged` arm breaking the reader instead of continuing, which
  would close the client and fail every later request while requests
  already in flight could still have paid;
- registering the waiter after the publish rather than before, which
  loses an answer that arrives inside that window;
- bypassing the waiter guard, which leaks an `EventId` and a
  `oneshot::Sender` on every cancelled verification.

And five that were untested: `since` at subscribe, the NIP-40
expiration tag, the reader-exit drain, and the payload-free parse
error, whose existing test built its serde failure from a string with
no payload in it and so passed even when the whole serde message was
leaked.

Two needed infrastructure, so the test pool now owns the notification
stream. A test can overflow it synchronously, which on a current-thread
runtime guarantees the reader observes `Lagged` rather than racing for
it, and can close it to force a real reader exit. It can also park the
caller inside `publish_event` after the event is on the wire, which
makes the registration-order window deterministic.
@nisargpatel7042lva

Copy link
Copy Markdown
Author

Follow-up on the test coverage, now in 69212f4.

Eight more mutants, including the three money-path ones you named. Sixteen in total across everything these PRs changed, and all sixteen die:

mutant killed by
Lagged arm breaks the reader a_lagging_broadcast_does_not_kill_the_reader
waiter registered after the publish an_answer_arriving_during_the_publish_is_not_lost
WaiterGuard::drop bypassed the timeout, publish-failure and cancellation tests
no since at subscribe a_stale_notification_is_not_replayed_at_subscribe
no NIP-40 expiration tag pay_invoice_carries_the_nip40_expiration_when_asked
no reader-exit drain a_reader_exit_releases_the_waiting_caller
parse error leaks the payload two, including the rewritten one below
the nine from the blocker fixes unchanged, still dying

Three of them needed the test pool to own the notification stream, so it does now. A test can overflow it synchronously, which on a current-thread runtime makes the reader certainly observe Lagged rather than racing for it, and can close it to force a genuine reader exit. It can also hold the caller inside publish_event after the event is on the wire, which turns the registration-order window into something deterministic instead of a race the test usually wins.

Two things I got wrong on the way, both caught by a check rather than by reading the code.

My first kill_stream aborted the forwarder task but the pool still held the sender, so the channel never closed, the reader just sat there, and the test failed with Timeout where it expected Closed. The harness was not reproducing the condition it claimed to. Both senders are dropped now.

And a_parse_failure_never_reproduces_the_payload was weak: it built its serde error from "{ not json", a string with no payload in it, so a mutant that leaked the entire serde message passed it. The replacement makes serde quote a real invoice through a wrong-typed field, asserts that precondition holds, and only then checks the mapped error is clean.

On the e2e_happy_path failures, which you will see if you run the full workspace: those four are pre-existing tests neither of us has touched, and they fail at a hard 5 s timeout around a client startup that takes about 0.1 s. Isolated they pass 10 out of 10. On this machine right now the full-workspace run fails 3 times in 4 with my test commit and 2 times in 4 on the commit before it, which at that sample size is the same number, and both are much worse than the 0 in 20 I measured on the same box when it was idle. So it is contention on my side, not a regression, but I am not going to describe a run as clean when it was not.

Your 17 of 35 is your mutant set, not mine. I have killed the ones covering what these PRs changed, which is sixteen; I have not reproduced your other nineteen, so if the gist is still on offer I would still rather compare against it than keep guessing at the overlap.

The remainders on (4), (6), (8b), (9), (10), (11), (13), (14), (16), (19) and (23) still are not showing up for me. The API returns the 25 inline comments from the first round and nothing from the 15th, so I think they did not submit with the review. Paste them whenever and I will take them in one pass.

#118 and #119 are rebased on this and pushed.

Three statements no longer matched the code, all of them on points the
review raised and I said were fixed.

The client no longer speaks "over an injected `RelayPoolTrait`": item
(12) made it build its own pool from the connection relays, with
injection kept for tests. The module doc was corrected at the time and
this was not.

"Decrypted by kind (23196 NIP-04, 23197 NIP-44)" describes exactly the
behaviour item (14) asked to change, and the review said so about this
line. It is decryption by ciphertext shape now.

"A missing or unknown optional field" understates item (2), which was
about a field present with the wrong JSON type, the case that lost a
settled response.

Also records the typed error and `Delivery`, which are public API a
caller needs in order to know whether retrying `pay_invoice` is safe.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants