fix(sourcehunt): anchor rejecting verdicts to a source location - #203
Open
tsunoda-kazuya wants to merge 2 commits into
Open
fix(sourcehunt): anchor rejecting verdicts to a source location#203tsunoda-kazuya wants to merge 2 commits into
tsunoda-kazuya wants to merge 2 commits into
Conversation
Add optional tie_breaker_file and tie_breaker_line to the validator verdict, threaded from the wire schema into the domain verdict and the persisted finding. The 4-axis prompt asks for them on rejection so a rejection can be located. The fields default to None, so verdicts that omit them are unchanged.
…line Spec 009 §3.3: a validator verdict with advance=False must anchor its rejection in the source so mechanised false-reject audits can locate it. Enforce this at the wire schema with a model_validator that raises when either tie_breaker_file or tie_breaker_line is missing on rejection, and add Field(ge=1) on tie_breaker_line to stop 0 or negatives from being persisted. The prompt is tightened to say the response will be rejected otherwise. Advancing verdicts remain unconstrained. Legacy verdicts read back via ValidatorVerdict (dataclass) bypass the wire-schema validator and still load; the internal _error_verdict path constructs ValidatorVerdict directly, so it also bypasses the check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tie_breaker_file: str | Noneandtie_breaker_line: int | None(ge=1).advanceis false, enforced by a Pydanticmodel_validator.Noneon the persisted dataclass).Problem
tie_breakeris free text. A rejection can cite evidence that no reader can locate, so false-reject audits fall back to reading prose. There is no mechanical way to point at "the single line the rejection rests on".Fix
_VerdictSchema: add the two fields plus a@model_validator(mode="after")that raisesValueErrorwhenadvance is Falseand either field is missing.ValidatorVerdict(state.py): mirror the two optional fields; carried through from schema into_verdict.apply_validator_verdict: write tofinding["verifier_tie_breaker_file"]andfinding["verifier_tie_breaker_line"]._error_verdictbypasses the wire schema by constructingValidatorVerdictdirectly, so internal error paths do not need to synthesise dummy locations.Validation
uv run --frozen --extra dev pytest -q tests/test_sourcehunt_validator.py— 45 passed; new regressions: reject without file / without line raisesValidationError;advance=trueaccepts missing fields;tie_breaker_line=0rejected; legacy verdicts without the fields still load;apply_validator_verdictwrites both keys.uv run --frozen --extra dev ruff check clearwing/sourcehunt/validator.py clearwing/sourcehunt/state.py tests/test_sourcehunt_validator.pygit diff --checkBackward compatibility
Persisted verdicts without the new fields remain readable (the dataclass defaults to
None). Only newly produced rejections at the wire boundary must carry a location. Producers that emit valid JSON without the fields onadvance=falsewill see aValidationError— this is the intended contract change, exposing silent omissions that previously slipped through as unlocatable rejections.Independent of the other 4 PRs in this batch; trivial rebase if any lands first.