Name the synthesizer consumption exemption and document calibration gameability#21
Merged
Merged
Conversation
Follow-up to #20. Consuming Signals gains a named exception: the consumption rules bar raw signal backlogs from re-entering a working agent's window, but the synthesizer that produces the partnership signal must read the raw backlog - that is its function. It runs out-of-band with a dedicated budget; the rules bind working agents. Trust Equation gains a known-limitation note: a hedging agent can keep calibration high by reporting middling confidence everywhere, so a flat confidence spread with a good calibration score is a tell, not a pass. Marked open with a practical check (track spread alongside the gap), not solved. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: dfa3b4bd-9ecd-4389-92b4-6dae371828c3
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the agent-signals specification to clarify two protocol-level documentation points: (1) explicitly naming the intended “synthesizer” exemption to the consumption rules, and (2) documenting a known limitation of the Trust Equation calibration metric (it can be gamed via hedged confidence reporting).
Changes:
- Documented that calibration scores can appear “healthy” under hedging behavior, and recommended tracking confidence spread alongside the calibration gap.
- Clarified that the
partnership-producing synthesizer is intentionally allowed to read the raw backlog out-of-band, while the consumption rules remain binding for working agents.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #20, two additive notes.
Consuming Signals — name the designed exception. The consumption rules ban raw signal backlogs from re-entering a working agent's window, but the synthesizer that produces the
partnershipsignal must read the raw backlog — that's its function. One paragraph makes the exemption explicit: the synthesizer runs out-of-band with a dedicated budget; the rules bind working agents.Trust Equation — document that the metric is gameable. A hedging agent can hold calibration high by reporting middling confidence everywhere. A flat confidence spread with a good calibration score is a tell, not a pass. Documented as a known limitation with a practical check (track spread alongside the gap), honestly marked open rather than solved.