Audit round three: the traceback my last fix missed, and six smaller holes - #37
Merged
Conversation
…holes **The README's own demo still ended in a stack trace on a base install.** #22 moved evaluation inside the guarded block and listed the exceptions it knew about. `SynthIdUnavailableError` was not one of them — and the demo config declares `text_marking: synthid`, so the single command the README sends a newcomer to was the case the fix did not reach. Exit 1, which in this tool means "a rule failed". I found it by building a wheel and installing it into a fresh virtualenv rather than by reading, which is the only way that class of defect shows up. The three optional-dependency errors now share a base the CLI catches by type, so the next optional dependency cannot reintroduce it. Verified end to end on a clean install: exit 2, no traceback, and the install command survives intact — which it would not have before the Rich escaping in #34. **The manifest that decides a document verdict was recorded nowhere.** A reader could re-fetch the document, find its digest matching, and still not know which manifest produced the answer. The finding now carries the sidecar digest and how it was found, header or element. **MPF-M-001 passed without saying who signed.** The rulepack allows self-signed chains, so "validly signed" without "by whom" is half an answer. The signer's common name is in the finding now, and a reader decides what it is worth. **A `<link>` in a comment or a script string was taken as the manifest pointer.** Anything that can put text on a page could choose the bytes that decide the verdict. Inert markup is stripped before the scan. **NOTICE and DISCLAIMER both claimed no component is named after a third-party mark.** False: `checks/synthid.py`, `checks/c2pa_verify.py`, the `[synthid]` extra and the `c2pa-verify` check type all are. The narrower parenthetical was true and the sentence around it was not. Both now say what is actually the case — the marks appear where they describe what the code does, which is nominative use, and nowhere as a product name or branding. Renaming them would make the code harder to read and the claim harder to check. **Nothing acknowledged that evidence can contain personal data.** Probes record responses verbatim and reports record the URLs probed. Now stated where the other scope limits are. **The determinism gate asserted nothing about which rules its goldens cover.** It checked that all four verdict values appear, which eleven cases exercising the same two rules would satisfy. Every shipped rule must now appear in some golden. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows #34, #35 and #36. This closes the remaining confirmed findings.
The one that got past a previous fix
The README's own demo still ended in a stack trace on a base install.
#22 moved evaluation inside the guarded block and listed the exceptions it knew
about.
SynthIdUnavailableErrorwas not one of them — and the demo configdeclares
text_marking: synthid, so the single command the README sends anewcomer to was precisely the case the fix did not reach. Exit 1, which in
this tool means a rule failed.
I found it by building a wheel and installing into a fresh virtualenv rather than
by reading. That is the only way this class of defect shows up, and it is why the
audit's first-run axis was worth having.
The three optional-dependency errors now share a base the CLI catches by type,
so the next optional dependency cannot reintroduce this.
Six smaller ones
The manifest that decides a document verdict was recorded nowhere. A reader
could re-fetch the document, confirm its digest, and still not know which manifest
produced the answer. The finding carries the sidecar digest and whether it came
from a header or an element.
MPF-M-001 passed without saying who signed. The rulepack allows self-signed
chains, so "validly signed" without "by whom" is half an answer.
A
<link>in a comment or a script string was taken as the manifest pointer —so anything that can put text on a page could choose the bytes that decide the
verdict.
NOTICE and DISCLAIMER both claimed no component is named after a third-party
mark. That is false —
checks/synthid.py,checks/c2pa_verify.py, the[synthid]extra and thec2pa-verifycheck type all are. The narrowerparenthetical was true and the sentence around it was not. Both now state what is
actually the case: the marks appear where they describe what the code does, which
is nominative use, and nowhere as a product name or branding. Renaming them would
make the code harder to read and the claim harder to check.
Nothing acknowledged that evidence can contain personal data. Probes record
responses verbatim and reports record the URLs probed.
The determinism gate asserted nothing about rule coverage. It checked that all
four verdict values appear — which eleven cases exercising the same two rules
would satisfy. Every shipped rule must now appear in some golden.
530 tests, ruff + mypy strict clean.
🤖 Generated with Claude Code