Skip to content

feat: context/verify — pull-based frame revalidation#38

Merged
macanderson merged 9 commits into
mainfrom
feat/context-verify
Jul 22, 2026
Merged

feat: context/verify — pull-based frame revalidation#38
macanderson merged 9 commits into
mainfrom
feat/context-verify

Conversation

@macanderson

@macanderson macanderson commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Implements #26 from scratch. The issue was auto-closed by PR #32's keyword with no implementation landed — only design prose in docs/context-reuse.md. There was zero verify code in any contextgraph-types or contextgraph-host source file; the #4-context-verification anchor, the verify capability mentioned in the spec-version note, and the D1/D4 cross-references all pointed at a section that did not exist. This builds it.

The exchange

The host sends the frame identities it holds; the provider answers one verdict each.

pub struct VerifyRequest { pub frames: Vec<FrameId> }        // identities only
pub struct FrameVerdict  { pub frame: FrameId, pub verdict: Verdict }

pub enum Verdict {
    Valid,                                          // unchanged — keep reusing
    Stale { replacement_digest: Option<String> },   // changed — drop it
    Gone,                                           // no longer exists
    Unknown,                                        // provider cannot say
}

No frame body travels in either direction — the economic point of the whole feature, and it is asserted against the serialized envelope rather than merely documented (verify_wire.rs). Even stale carries at most the provider's current digest: enough to tell a host what it would be re-fetching, never the replacement content.

Built directly on identity.rs from #23FrameId is the verify unit and its existing is_verifiable() is the digest gate. No parallel identity type was minted.

Correctness choices worth reviewing

Verdicts echo the full identity. FrameVerdict carries the FrameId it answers rather than relying on array position, so a provider that reorders, omits, or duplicates entries cannot shift a valid onto the wrong frame. Since the digest is part of the identity, a verdict about a different digest does not answer for the one asked — there is a test pinning exactly that.

Default-deny. A frame is retained only on an explicit valid. An identity that comes back with no verdict at all is treated as unknown, not as tacit approval — silence is not validity. Host::verify_frames returns a total partition of its input into retained + dropped, and a test asserts no held identity can be silently lost.

gone is the one reason that doesn't warrant a re-query — the frame is not there to re-fetch. Everything else (stale, unknown, no digest, unsupported, failed) drops the frame and marks it worth fetching again.

The host stays stateless. verify_frames caches no frames and tracks no turn boundaries. "Re-verify past a freshness window" is written as informative host guidance in the spec, not built as machinery — a turn-boundary cache is subscribe's lifecycle, and building one here would blur the exact line §4 exists to draw.

Capability gating and fallback

Capabilities.verify defaults to false, so a pre-§4 provider is treated as unsupported and its frames are re-queried. The host never even sends a request to such a provider (asserted, not assumed). ContextProvider::verify defaults to answering unknown for everything, so a provider that implements nothing can never accidentally bless a stale frame. Verify failures are isolated exactly like query fan-out legs — one provider's timeout drops its own frames and never another's.

Conformance: honest staleness without mutating a real source

The subtle part. Because the digest is provider-declared and opaque (§1), the suite cannot check the provider's answer — only that it distinguishes. So verify-honesty asks twice about frames the provider just served:

  1. with the real digests → an honest provider says valid;
  2. with those digests mutated → indistinguishable, from the provider's side, from a source that changed underneath the host → an honest provider says stale.

That is the issue's "provider answers honestly for a mutated source (digest mismatch ⇒ stale)" requirement, testable against any provider without the suite needing write access to anyone's sources.

Two fixture modes prove the check bites in both directions:

  • --misbehave rubber-stamp-verify — always answers valid; fails ask 2. This is the dangerous lie: it would let a host go on citing evidence that changed.
  • --misbehave hollow-verify — advertises the capability but answers unknown to everything; fails ask 1.

The check is skipped, not failed, when a provider doesn't advertise verify — that is the declared fallback, not a violation.

verify_wire.rs additionally drives the real stdio fixture end-to-end over an actual pipe: a served frame verifies valid, a mutated digest comes back stale carrying the current digest and is demonstrably evicted by the host, and an unserved frame comes back gone and is correctly not queued for re-query.

Verify vs subscribe (#6) for the freeze

docs/context-reuse.md §4 includes the comparison the issue asked for: who initiates, transport needs, detection latency, cost shape, and which providers each fits. They are complementary, not alternatives — push has no answer for a host that reconnects and wants to know whether its cache is still good; pull has no answer for a host needing millisecond invalidation. Both flags are independent and default false, so the freeze can keep both, either, or one with no flag day. I did not implement any of #6's scope.

Verification

cargo fmt --all -- --check                               # clean
cargo clippy --workspace --all-targets -- -D warnings    # clean
cargo test --workspace                                   # 92 passing (65 on main before this branch)
python3 schema/validate-examples.py                      # all 14 bundled messages validate

Schema and examples are kept in lockstep with the types per the repo convention, and verify_wire.rs makes that a test rather than a habit: the published reference messages must parse as envelopes and the verify exchange must round-trip byte-for-byte. That test is how I caught the FrameVerdict schema def being wrong — allOf against a closed Verdict subschema rejects the flattened frame field.

Notes for adjacent issues

Closes #26

The request/response shapes for pull-based frame revalidation (#26,
docs/context-reuse.md §4):

- `VerifyRequest` carries `FrameId` identities only — never frame bodies.
  Verification costs bytes, not tokens.
- `Verdict`: valid | stale{replacement_digest?} | gone | unknown, internally
  tagged on `status` so it is self-describing and extensible. A stale verdict
  may offer the provider's current digest — a digest, never a body.
- `FrameVerdict` echoes the full identity, so a host correlates by matching
  rather than trusting positional alignment.
- `Verdict::permits_reuse()` is true only for `valid`: reuse requires a
  positive answer, never the absence of a negative one. `warrants_requery()`
  is false for `gone` — nothing there to re-fetch.
- `Capabilities.verify` defaults false, so a pre-§4 provider is treated as
  unsupported and the host falls back to re-query. Independent of `subscribe`
  (#6): push and pull freshness are complementary axes.

Zero new dependencies.

Claude-Session: https://claude.ai/code/session_01BvcuRRNHvpFSYLVhow4HGb
…honesty check

- Envelope::Verify / Envelope::Verified carry the request/response; stdio and
  HTTP transports dispatch them exactly as query/frames.
- ContextProvider::verify defaults to answering `unknown` for every identity,
  so no existing provider must implement anything — the host then re-queries.
- Host::verify_frames groups held identities by provider, asks each capable
  provider once, and partitions into retained vs dropped. Default-deny: a frame
  is retained only on an explicit `valid`. Every other outcome drops it with a
  DropReason (stale/gone/unknown/no-digest/verify-unsupported/unknown-provider/
  verify-failed), and DropReason::warrants_requery is false only for `gone`.
  Verdicts are correlated by full identity, never by position; a missing
  verdict is `unknown`, since silence is not validity. Per-provider failures
  are isolated like query legs — a verify timeout drops that provider's frames,
  never another's. The host holds no frame cache and no turn state.
- verify-honesty conformance check: asks twice about frames the provider just
  served — once with real digests (must be `valid`), once with mutated digests
  (must be `stale`). A mutated digest is what a changed source looks like from
  the provider's side, so honest staleness detection is testable without
  mutating a real source. Skipped, not failed, when `verify` is unadvertised —
  that is the declared fallback.
- Two new fixture modes prove the check bites from both directions:
  --misbehave rubber-stamp-verify (always `valid`) and hollow-verify
  (advertises the capability, vouches for nothing).

Claude-Session: https://claude.ai/code/session_01BvcuRRNHvpFSYLVhow4HGb
Ten tests pinning the §4 host contract: stale and gone frames are dropped
while a valid one is retained; an unknown verdict AND a missing verdict both
drop the frame (silence is not validity); a provider without the verify
capability is never asked at all and falls back to re-query; a digest-less
frame never reaches the wire; one provider's verify failure drops only its own
frames; frames from an unregistered provider are dropped rather than silently
ignored; a provider's held frames are batched into one request; the
retained/dropped partition is total, so no held identity is ever lost.

Claude-Session: https://claude.ai/code/session_01BvcuRRNHvpFSYLVhow4HGb
Keeps schema/ and examples/ in lockstep with the types, per the repo
convention:

- schema: Verify/Verified envelope variants, FrameId, VerifyRequest,
  VerdictStatus, FrameVerdict, VerifyResponse, and the optional
  Capabilities.verify flag. FrameVerdict is a flat object rather than an
  allOf composition — allOf against a closed Verdict subschema rejects the
  flattened `frame` field, which is exactly what the example caught.
- examples/reference-messages.json: the served frames now carry a
  content_digest (so they are verifiable at all), the repo-graph provider
  advertises verify, and a full verify -> verified exchange is documented in
  which one frame is valid and one is stale with a replacement digest.
- tests/verify_wire.rs: the published messages must parse as envelopes and the
  verify exchange must round-trip byte-for-byte; a verify envelope must carry
  no frame-body field; and the real stdio fixture is driven end-to-end
  (valid / stale-with-current-digest / gone), with the host demonstrably
  evicting the stale frame and re-querying only what is worth re-fetching.

Verified: all 9 reference messages validate against the JSON schema
(jsonschema Draft202012Validator).

Claude-Session: https://claude.ai/code/session_01BvcuRRNHvpFSYLVhow4HGb
- context-reuse.md §4 (the anchor and the D1/D4 cross-references were already
  reserved for it): the problem, the exchange, why no frame body travels in
  either direction, digest-as-ground-truth, and the default-deny host rule.
  Adds the verify-vs-subscribe comparison the issue asked for, so the 1.0
  freeze can keep both, either, or one — both are capability-gated and default
  false, so there is no flag day in any of those directions. Conformance rows
  V1-V4.
- protocol-surface.md: the verify types, the `verify` capability flag, the
  verify/verified envelope variants, and V1-V4 replacing the two forward-looking
  placeholder rows that named a check that did not exist yet.
- implementing-a-provider.md: the two new envelopes in the vocabulary table and
  a step explaining how to answer honestly — and that leaving `verify` unset
  costs a provider nothing.

Verified: python3 schema/validate-examples.py passes on all 14 bundled example
messages; fmt/clippy/test all green (92 tests).

Claude-Session: https://claude.ai/code/session_01BvcuRRNHvpFSYLVhow4HGb

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

- full-stdio-session.ndjson is billed as a complete session, so it now shows
  the verify/verified exchange too: the provider advertises verify, its served
  frames carry content_digests (without which nothing is verifiable at all),
  and one frame comes back valid while the other is stale with a replacement
  digest. reference-messages.json already documented the shape; this makes the
  end-to-end transcript match.
- context-reuse.md §4: explain why the freshness window is host-tolerated
  rather than provider-declared — only the host knows how much staleness a
  given task absorbs, and a wire field would push a scheduling decision into
  the protocol and give the host state to keep. Notes that a provider-supplied
  freshness *hint* is available later as an additive capability field, which
  would inform host policy without overriding it.

Verified: python3 schema/validate-examples.py passes on all 16 messages.

Claude-Session: https://claude.ai/code/session_01BvcuRRNHvpFSYLVhow4HGb
@macanderson

Copy link
Copy Markdown
Owner Author

Independently verified locally by the orchestrating session (this repo has no GitHub Actions workflows, so local gates are the only gate):

  • cargo fmt --all -- --check — clean
  • cargo test --workspace92 tests passing across 12 suites, 0 failures
  • cargo clippy --workspace --all-targets -- -D warnings — clean

Also confirmed the implementation is real code rather than spec prose: contextgraph-types/src/verify.rs (new wire types), 66 verify references in contextgraph-host/src/host.rs, capability gating in capability.rs, provider/wire/stdio/http plumbing, and the new contextgraph-conformance/tests/verify_wire.rs conformance case. This is the check that was missed when #26 was auto-closed by PR #32's keyword with docs-only changes.

Signed-off-by: Mac Anderson <mac@oxagen.sh>
@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
context-graph-protocol Ready Ready Preview, Comment Jul 22, 2026 7:36pm

@vercel vercel Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:

run_query_and_shutdown_checks is called with 3 arguments but its signature requires 4 (caps), causing an E0061 compile error.

Fix on Vercel

…ts signature requires 4 (`caps`), causing an E0061 compile error.

This commit fixes the issue reported at contextgraph-conformance/src/lib.rs:116

## Bug

In `contextgraph-conformance/src/lib.rs`, the merge commit `1197789` reintroduced the old 3-argument call site from `main` while keeping the new 4-parameter signature.

Call site (line 116):
```rust
run_query_and_shutdown_checks(host, &id, &mut checks).await;
```

Signature (line ~215):
```rust
async fn run_query_and_shutdown_checks(
    host: Host,
    id: &str,
    caps: &Capabilities,
    checks: &mut Vec<CheckResult>,
) { ... }
```

This is a hard compile error (`E0061`: this function takes 4 arguments but 3 were supplied). The crate cannot build. `caps` is already in scope from the `Ok((host, id, info, caps))` match arm, so the fix is straightforward.

## Fix

- Restored the 4-arg call: `run_query_and_shutdown_checks(host, &id, &caps, &mut checks).await;`
- Secondary merge artifact: the handshake-failure skip list contained `CHECK_FRAME_VALIDITY` twice, which would push two skip entries for the same check into the report. De-duplicated to a single `CHECK_FRAME_VALIDITY`.

Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: macanderson <mac@macanderson.com>
@macanderson
macanderson enabled auto-merge (squash) July 22, 2026 19:35
@macanderson
macanderson disabled auto-merge July 22, 2026 19:36
@macanderson
macanderson merged commit 6851d1d into main Jul 22, 2026
4 checks passed
@macanderson
macanderson deleted the feat/context-verify branch July 22, 2026 19:36
macanderson added a commit that referenced this pull request Jul 22, 2026
Reconcile the pre-freeze normative sweep (#33) with frame representations
(#41, full/compact/reference) and context/verify (#38), which landed on main
after this branch forked.

Key reconciliations:
- Capabilities: keep #33's ADR-0004 removal of upsert/subscribe/filters and its
  `correlation` field; add #41's `verify`/`representations`/`resolve`.
- ContextFrame: adopt main's optional `content` + representation model
  (representation/content_ref/canonical_content_hash/…); rename #33's
  `canonical_token_cost()` method to `expected_inline_token_cost()` to free the
  name for #41's `canonical_token_cost` field, and make it handle absent content.
- wire::Envelope: keep #33's correlation `id` on query/frames and structured
  `error` (id/code) AND #41's Verify/Verified variants.
- Conformance fixture: keep #33's strict frame-building (valid F5 digests,
  F4 timestamps) so its enhanced checks pass, grafted with main's verify
  (verify_honestly/current_digest derived from the fixture's own digests).
- schema/examples: union of #33's stricter validations (timestamp/embedding
  patterns) and #41's representation $defs; drop the removed capability fields.

Verified green: cargo build/clippy -D warnings/test, schema validate-examples,
and conformance-red.sh (all 14 misbehave modes caught, integrating both #33's
and #41's). The frame-representations witness (previously #[ignore]) now passes
against main's implementation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

context/verify: pull-based frame revalidation so hosts can reuse context cheaply without serving stale evidence

1 participant