Vendored request-signing conformance vectors are incomplete, and the loaders cannot detect it

[KonstantinMirin]

Summary

The vendored AdCP request-signing conformance vectors under tests/conformance/vectors/request-signing/ are an incomplete copy of the spec set, and the tests that grade against them are structurally incapable of noticing.

Both graders load vectors by globbing and then assert only that the result is non-empty:

• tests/conformance/signing/test_verifier_vectors.py:77-78 — _load() does glob("*.json") then assert result, f"no vectors under ..."
• tests/conformance/signing/test_canonicalization.py:24-31 — same shape for vectors carrying expected_signature_base

Any non-empty subset satisfies that assertion. A missing vector is indistinguishable from a complete set, so the suite reports green on a partial copy — which is exactly what has been happening.

The gap

Measured against AdCP 3.1.1, the version src/adcp/ADCP_VERSION already targets (spec repo dist/compliance/3.1.1/test-vectors/request-signing/):

	spec 3.1.1	vendored	missing
positive/	12	8	009-percent-encoded-unreserved-decoded, 010-percent-encoded-slash-preserved, 011-ipv6-authority, 012-ipv6-authority-default-port-stripped
negative/	28	20	021-duplicate-signature-input-label, 022-multi-valued-content-type, 023-multi-valued-content-digest, 024-unquoted-string-param, 025-jwk-alg-crv-mismatch, 026-non-ascii-host, 027-webhook-registration-authentication-unsigned, 028-unsigned-protocol-method-required
canonicalization.json	ships	absent	the standalone signing-base fixture

canonicalization.json is the conspicuous one: the vendored README.md documents it in the file-layout tree and devotes a full section to it, describing it as "the fastest way to surface cross-implementation divergence" and recommending SDKs "run canonicalization.json on every commit". The file was never copied in.

Three further vectors had silently drifted from their upstream contents: 016-replayed-nonce, 017-key-revoked, 020-rate-abuse. 017 is the consequential one — it was vendored at a revision that revokes test-ed25519-2026, the primary signing key, rather than the dedicated test-revoked-2026 keypair the spec now uses. keys.json had never been refreshed to carry that key either.

Why it matters

Signature-base construction is the known-hard part of RFC 9421 interop, and the missing vectors sit exactly on its edges: 009/010 are percent-encoding, 011/012 are IPv6 authority, 026 is a non-ASCII host, 028 is the protocol_methods_required_for enforcement path.

A verifier can pass every currently-vendored vector and still reject validly-signed requests from a conformant counterparty. And because the SDK owns canonicalization on behalf of its consumers, these gaps are untested in every downstream implementation too, not just in this suite — this was found while adopting adcp.signing in a seller implementation that delegates canonicalization entirely to the SDK.

Fix

PR #980 vendors the complete 3.1.1 set and, more importantly, replaces the non-empty assertion with a completeness pin so the set cannot silently drift again at the next spec bump. Adding the 12 files without that guard would only reset the clock.

The pin is over content hashes, not filenames — a filename-only manifest would not have caught the 016/017/020 drift. It is tied to get_adcp_spec_version(), so bumping the targeted spec version without re-vendoring fails loudly.

Grading the new vectors turns up 12 real conformance failures

They are filed separately and marked strict-xfail in the PR rather than fixed there, since they are production behavior changes in a security-critical verifier:

• The verifier has no step-1 strict-parse stage, so malformed/ambiguous input flows through to a later check and is rejected under the wrong error code (negative 021, 022, 023, 026). Conformance requires byte-for-byte error codes, so "rejected anyway" is not a pass.
• @target-uri canonicalization applies no UTS-46 A-label conversion, drops a trailing empty query, and accepts five malformed authority shapes the spec requires be rejected.

[bokelley]

Triage

Classification: Bug
Bucket(s): signing, validation (conformance test infrastructure)
Status: ready-to-implement

What the experts said:

• ad-tech-protocol-expert: AGREE_READY — filesystem confirms all claims; the assert result defect is real in both loaders; the 017 drift is confirmed (test-ed25519-2026 is in the revocation list but no test-revoked-2026 keypair exists in keys.json); content-hash pin tied to get_adcp_spec_version() is the right circuit-breaker; strict-xfail is correct tradeoff; non-breaking.
• code-reviewer: AGREE_READY — structural defect confirmed; recommends a filename-count manifest ({version: {subdir: expected_count}}) over a full content-hash dict — same missing-file coverage with lower maintenance cost on legitimate vector refreshes; strict-xfail is honest but should be treated as strictly temporary with linked tracking issues and a milestone.

My take: The diagnosis is airtight and independently verified against the filesystem. The implementation is non-breaking (test-only changes; new parametrized IDs are additive; strict-xfail keeps CI green). The one open implementation question is content-hash vs. filename-count for the completeness guard — the code reviewer makes a reasonable point that a count manifest survives content refreshes with less ceremony, while the protocol expert notes that content hashes would have caught the silent 016/017/020 drift that the issue describes. I'd give the implementer discretion here, but flag the tradeoff explicitly.

Implementation scope:

• Vendor the missing vectors — 4 positive (009-percent-encoded-unreserved-decoded, 010-percent-encoded-slash-preserved, 011-ipv6-authority, 012-ipv6-authority-default-port-stripped), 8 negative (021–028), and canonicalization.json from dist/compliance/3.1.1/test-vectors/request-signing/ in the spec repo.
• Refresh drifted files — update 016-replayed-nonce, 017-key-revoked, 020-rate-abuse to upstream 3.1.1 contents; add test-revoked-2026 to keys.json.
• Replace the non-empty assertion in test_verifier_vectors.py:_load() and test_canonicalization.py:_vectors_with_expected_base() with a completeness guard keyed on get_adcp_spec_version(). A content-hash dict catches silent drift (what happened to 016/017/020); a filename-count manifest is lighter to maintain on refreshes. Pick one and document the tradeoff in a comment.
• Add a canonicalization.json loader — the file isn't in positive/ or negative/, so test_canonicalization.py needs an explicit path for it. Wire it into the parametrize call or a separate test function.
• Mark 12 newly-exposed failures as strict-xfail — per the linked issues (Verifier does not reject malformed structured-field input at step 1 (wrong error code; covered-component smuggling) #976, @target-uri canonicalization applies no IDNA A-label conversion, and the verifier does not reject raw U-labels #977). strict=True ensures these are revisited when production conformance improves; they must not be marked xfail without strict.
• Required checks: pytest tests/conformance/signing/ (no slow integration tier unless test_revocation_e2e.py or test_pg_replay_store_e2e.py are touched), ruff check ., black --check .. No src/ edits expected, so mypy src/ can be skipped unless the completeness guard introduces new imports.
• Non-breaking: all changes are under tests/; no public API surface is touched; existing test IDs are unchanged; strict-xfail prevents new failures from breaking CI until the production fixes land.

Note: the issue body references "PR #<PR>" as a companion PR, but no open PR was found targeting this issue. The companion PR has not yet been filed.

---

Triaged by Claude Code. Session: https://claude.ai/code/${CLAUDE_CODE_REMOTE_SESSION_ID}

---

Generated by Claude Code