Correct the aggregation algorithm: issuer capping now binds (v1.1) - #58
Merged
Merged
Conversation
Implementing the normative spec surfaced a defect in it. Step 5 capped a dominant issuer at max_issuer_weight and then re-normalized across all issuers, which handed the capped excess straight back whenever the other issuers were light. With 50 receipts from one issuer and one each from two others, the "capped" issuer still held 87% of the weight and the aggregate came out at 0.90. The cap only bound when the field was already balanced — the case where a cap is not needed. v1.1 redistributes a capped issuer's excess to the uncapped issuers, and applies effective_cap = max(max_issuer_weight, 1 / issuer_count), since a cap below 1/n cannot be satisfied. That input now gives the dominant issuer exactly 0.40 and an aggregate of 0.52. Vector agg-006 was written to assert exactly this intent — "4 good receipts from A cannot dominate 1 bad receipt from B" — and could not pass under v1.0. It passes under v1.1 unchanged, which is the clearest evidence that the vector recorded the intent and the algorithm had drifted from it. Vectors repaired and extended: - agg-003 contradicted itself: its _explanation worked the arithmetic and concluded 0.5 while expected said 0.4. It was already superseded by agg-003-corrected, which now carries the agg-003 id. - agg-008 recomputed from unrounded weights, 0.917 -> 0.918. - Added agg-009 (redistribution), agg-010 (the 1/n floor) and agg-011 (single issuer, nobody to redistribute to). All eleven vectors now pass on their own terms; the known-divergence scaffolding in the test suite is gone. This changes conformance: a v1.0 implementation produces different scores wherever one issuer exceeds the cap while the others are light. The spec carries that warning in step 5 and the changelog repeats it. 194 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implementing the spec surfaced a defect in the spec.
What was wrong
Step 5 capped a dominant issuer at
max_issuer_weight(0.40) and then re-normalized across all issuers — which handed the capped excess straight back whenever the other issuers were light.With 50 receipts from one issuer and one each from two others:
The cap only bound when the field was already balanced — precisely the case where a cap isn't needed. An issuer that is chatty, compromised, or simply wired to emit more receipts could still decide an agent's trust alone, which is the one thing step 5 exists to prevent.
The fix
Redistribute a capped issuer's excess to the uncapped issuers, and apply:
since a cap below
1/ncannot be satisfied — with two issuers, both cannot sit under 0.40.The vectors already knew
agg-006was written to assert exactly this intent — "4 good receipts from A cannot dominate 1 bad receipt from B" — and could not pass under v1.0. It passes under v1.1 unchanged. That's the clearest evidence available that the vector recorded the intent and the algorithm had drifted from it.Two other vectors were repaired:
agg-003contradicted itself: its_explanationfield worked through the arithmetic, caught itself mid-sentence ("wait let me recalculate"), and concluded 0.5 — whileexpectedstill said 0.4. It was already superseded byagg-003-corrected, which now carries theagg-003id.agg-008recomputed from unrounded intermediate weights: 0.917 → 0.918.Three added:
agg-009(redistribution),agg-010(the1/nfloor),agg-011(single issuer, nobody to redistribute to).All eleven now pass on their own terms, and the known-divergence scaffolding I added in #57 is gone.
Migration
A v1.0 implementation produces different scores wherever one issuer exceeds the cap while the others are light. Step 5 carries the warning inline and
CHANGELOG.mdrepeats it. I did not touchmax_issuer_weight, the decay half-life, or negative-signal amplification — only how the cap is applied.Worth flagging: this sits close to the issuer-weight-normalization method
docs/patent-strategy.mdnames as patent-pending, so the published algorithm matching its published intent likely matters beyond engineering.194 tests, Python 19, AGT parity green.
🤖 Generated with Claude Code