Measure trust instead of reading it out of the manifest - #57
Merged
Merged
Conversation
Everything downstream of a trust score was rigorous about it — thresholds,
decay, route admissibility, execution authority — while the score itself
was a number someone typed into pctr.json. This closes that.
Implements protocol/aggregation-spec.md, which is marked normative and
ships nine test vectors, and which nothing in this repository implemented
until now. All nine vectors run in tests. Three do not agree with the
algorithm the document defines; they are asserted as known divergences
rather than skipped or worked around:
- agg-003 expects 0.4 where the formula yields 0.5. It is superseded by
agg-003-corrected, which has identical receipts and expects 0.5, and
its own _explanation field works the arithmetic, catches itself
mid-sentence ("wait let me recalculate") and concludes 0.5.
- agg-006 expects 0.5, which requires both issuers capped at 0.40. B's
uncapped fraction is 0.29, and step 5 specifies min(fraction, cap) — a
cap, not a floor. The formula yields 0.5799.
- agg-008 is off by 0.0010, just outside the vectors' own tolerance.
Behind agg-006 is something the spec should decide rather than the code:
capping a dominant issuer at 0.40 and then re-normalizing hands most of
the capped weight back when other issuers are light, so 50 perfect
receipts from one issuer still outvote two bad ones from two others. A
test pins that behaviour so it cannot be mistaken for a bug here.
pctr attest measures each agent from PCTR's own execution receipts,
scored on the scale in protocol/scoring-semantics.md 3.2, plus any
attestors configured per agent. Attestations are verified with TTP's own
verify_attestation, so a stale one or one about another subject
contributes nothing, and a failing attestor yields no evidence rather
than favourable evidence. Output is a TTP TrustThresholdProof naming what
it rests on.
The rule the whole thing turns on: absent evidence is not trust. An
unmeasured agent does not inherit its declared score — it is UNPROVEN,
counts as zero, and a protected consequence will not route through it.
Run against the working example, it found both agents overstated:
declared 0.98 and 0.97, evidence supports 0.8251.
192 tests (25 new).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything downstream of a trust score was rigorous about it — thresholds, decay, route admissibility, execution authority. None of it meant much while the score itself was a number someone typed into
pctr.json.pctr attestRun against the worked example, it found both agents overstated: declared 0.98 and 0.97, evidence supports 0.8251.
The rule the whole thing turns on: absent evidence is not trust. An unmeasured agent does not inherit its declared score — it is
UNPROVEN, counts as zero, and a protected consequence will not route through it (TRUST_UNPROVEN). The failure mode this exists to prevent is a typed-in0.99silently authorising a payment.Evidence comes from PCTR's own execution receipts, scored on the scale in
protocol/scoring-semantics.md§3.2 (a clean execution 0.95; reaching for authority it lacks 0.15; replaying an authority 0.20; waiting on a human approval 0.75 — an agent blocked on a human is not an agent misbehaving), plus attestors configured per agent. Attestations go through TTP's ownverify_attestation, so stale or wrong-subject ones contribute nothing, and a failing attestor yields no evidence rather than favourable evidence.The normative algorithm had no implementation
protocol/aggregation-spec.mdis marked normative and ships nine test vectors. Nothing in this repository implemented it until now. All nine run intests/aggregation.test.mjs.Three do not agree with the algorithm the document itself defines. They are asserted as known divergences rather than skipped or worked around:
agg-003agg-003-corrected(identical receipts, expects 0.5). Its own_explanationfield works through the arithmetic, catches itself mid-sentence — "wait let me recalculate" — and concludes 0.5, whileexpectedstill says 0.4.agg-006min(fraction, max_issuer_weight)— a cap, not a floor.agg-008One of these is not a typo
Behind
agg-006is a design question the spec should answer, not the code. Step 5 caps a dominant issuer at 0.40 and then re-normalizes — which hands most of the capped weight straight back when the other issuers are light. 50 perfect receipts from one issuer against two bad ones from two others still yields ~0.90, with the "capped" issuer holding 87% of the weight. The cap only bites when the rest of the field is comparable.That behaviour is pinned by a test so it can't be mistaken for an implementation bug here, but
agg-006documents an intent the algorithm doesn't deliver, and that's worth deciding deliberately.I did not change the spec or the vectors — that changes conformance for anyone implementing against them.
Verification
192 tests (25 new), Python 19, AGT parity green at 52 checks.
🤖 Generated with Claude Code