diff --git a/CHANGELOG.md b/CHANGELOG.md index 07dbc21..8a3bd9e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -33,6 +33,24 @@ explicitly: the **rulepack format** — what a rulepack file may contain — and ### Added +- **`MPF-M-002` verifies the C2PA manifest bound to a delivered document** (#18), + through a new `document` probe. C2PA 2.4 §A.7 binds a manifest to a document + that cannot embed one — HTML above all — by hashing the delivered bytes, and the + document points at it through an RFC 8288 `Link:` header or a + `` element. + + This is the delivery-chain regression the project exists for, in the format most + likely to suffer it: a minifier, an HTML-rewriting CDN or a template change + turns a valid provenance claim into an invalid one while the page still renders + perfectly. Nothing errors and no log line appears. + + The probe fetches the bytes the server sent rather than driving a browser — a + browser normalises markup, and only the delivered bytes are what the manifest + signs. It refuses a manifest on another origin: a provenance claim that depends + on a third party being reachable stops being checkable when they are not, and a + dangling manifest link is worse than none, because it reads as marked and + verifies as nothing. + - **`markproof init`** writes a starting `markproof.yaml`. The CLI's own docstring had promised this command since M4 and the build did not have it. The scaffold configures one chat probe and leaves media, UI and text marking commented out diff --git a/README.md b/README.md index 4b36bdc..6ea9e44 100644 --- a/README.md +++ b/README.md @@ -97,6 +97,9 @@ target: type: ui url: https://example.com/blog/latest content_selector: "article .body" # the model-written text, nothing else + - id: article-provenance + type: document # the delivered bytes, for the C2PA binding + url: https://example.com/blog/latest applicability: deepfake-labelling: false # this target generates no deep fakes text_marking: diff --git a/docs/DISCLAIMER.md b/docs/DISCLAIMER.md index b8281e4..c145603 100644 --- a/docs/DISCLAIMER.md +++ b/docs/DISCLAIMER.md @@ -83,11 +83,14 @@ than none: about a narrow test, not a defence. - **Text marking on a web page is checked in one place only.** For a rendered document markproof scores the region named by `content_selector` against the - operator's watermark configuration, and nothing else. It does **not** follow - the C2PA binding for HTML documents (C2PA Technical Specification 2.4, §A.7, - April 2026) or the `c2pa.ai-disclosure` assertion (§18.28). Those exist and are - checkable; markproof has not implemented them yet. A green run on a web page - therefore says the watermark survived, not that the document is marked. + operator's watermark configuration, and nothing else — a `ui` probe reports what + a browser rendered, which is not the bytes a C2PA manifest signs. +- **Document provenance is a separate probe.** `MPF-M-002` verifies the C2PA + binding for a delivered document (C2PA Technical Specification 2.4, §A.7) via + the `document` probe, which fetches the bytes the server sent and resolves the + manifest from a `Link:` header or a `` element. It + refuses a manifest hosted on another origin: a provenance claim that depends on + a third party being reachable stops being checkable when they are not. - **Marking is checked, detectability is not.** Article 50(2) requires that outputs be marked *and* detectable as artificially generated. markproof measures the first limb against your own configuration. Whether a third party diff --git a/docs/RULES_SOURCES.md b/docs/RULES_SOURCES.md index 411c93c..c2a620b 100644 --- a/docs/RULES_SOURCES.md +++ b/docs/RULES_SOURCES.md @@ -859,7 +859,23 @@ Konvention, deren Fehlen hier behauptet wurde — geprüft am 31.08.2026 gegen Der Satz, der Bestand hat, ist der engere: markproof erfindet keine eigene ``-Konvention. Gegen eine fremde, veröffentlichte zu prüfen, ist dagegen -genau die Aufgabe. Die Regel dafür ist offen (Issue #18). +genau die Aufgabe — und seit dem 01.09.2026 tut `MPF-M-002` das. + +**Nachtrag zur Werkzeuglage.** Hier stand zunächst, `c2pa-rs` habe keinen +HTML-Handler und die einzige A.7-Implementierung sei von dritter Seite. Das war zu +weit gefasst. Richtig ist die engere Aussage: `c2pa-rs` kann ein Manifest **nicht +in HTML einbetten** — aber es kann eines über HTML-Bytes erzeugen und prüfen, +sobald es *abgesetzt* geführt wird (`set_no_embed`, und beim Lesen +`manifest_data`). Genau das ist die A.7-Anordnung, und genau darauf steht +`MPF-M-002`. Am 01.09.2026 gegen `c2pa-python` 0.37.8 / `c2pa-rs` 0.90.15 geprüft: +Ein Dokument, das nach dem Signieren um vier Zeichen geändert wurde, meldet +`assertion.dataHash.mismatch`; das unveränderte meldet ihn nicht. + +Die Regel prüft deshalb die **ausgelieferten Bytes** und nicht die gerenderte +Seite. Ein Browser normalisiert Markup, bevor irgendetwas lesbar ist; die Bindung +deckt aber, was der Server gesendet hat. Das ist zugleich der Grund, warum die +Prüfung überhaupt lohnt: Ein Minifier oder ein HTML-umschreibendes CDN zerstört +die Bindung, während die Seite perfekt aussieht. Was allgemein *nicht* existiert, ist eine Konvention außerhalb von C2PA: Der WHATWG-Vorschlag für ein Meta-Tag (#9479, offen seit dem 02.07.2023) wartet diff --git a/src/markproof/checks/c2pa_verify.py b/src/markproof/checks/c2pa_verify.py index ed96667..376e9b1 100644 --- a/src/markproof/checks/c2pa_verify.py +++ b/src/markproof/checks/c2pa_verify.py @@ -178,22 +178,53 @@ def verify_media( check: C2paVerifyCheck, *, artifact_id: str, + sidecar_manifest: bytes | None = None, ) -> C2paResult: - """Verify one media payload against a rule's C2PA requirements. + """Verify one payload against a rule's C2PA requirements. Pure with respect to the network: the bytes are already in hand, and remote manifests are refused rather than fetched, so the same payload always yields the same verdict. + + ``sidecar_manifest`` carries a manifest that travels beside the bytes instead + of inside them, which is how C2PA binds provenance to formats that cannot + embed one — HTML being the case that matters. The binding is still a hash over + the delivered bytes, so it is verified exactly as strictly: a document altered + after signing fails with the same ``dataHash`` mismatch an altered JPEG does. """ import c2pa try: - reader = c2pa.Reader(media_type, stream=io.BytesIO(data)) + reader = c2pa.Reader(media_type, stream=io.BytesIO(data), manifest_data=sidecar_manifest) except c2pa.C2paError.ManifestNotFound: return C2paResult( outcome=C2paOutcome.MANIFEST_MISSING, artifact_id=artifact_id, - detail="no C2PA manifest embedded in the delivered bytes", + detail=( + "no C2PA manifest accompanies the delivered bytes" + if sidecar_manifest is not None + else "no C2PA manifest embedded in the delivered bytes" + ), + ) + except c2pa.C2paError.NotSupported: + if sidecar_manifest is None: + # Not an unreadable payload — a definite absence. The format cannot + # carry an embedded manifest at all (HTML is the case that matters), + # and none travelled beside it, so there is nowhere a manifest could + # be. Saying "unreadable" here would suggest the evidence was at + # fault when the finding is about the asset. + return C2paResult( + outcome=C2paOutcome.MANIFEST_MISSING, + artifact_id=artifact_id, + detail=( + f"{media_type} cannot carry an embedded manifest and none accompanied " + 'it — no Link header and no in the document' + ), + ) + return C2paResult( + outcome=C2paOutcome.UNREADABLE, + artifact_id=artifact_id, + detail=f"a manifest was supplied but {media_type} could not be read alongside it", ) except c2pa.C2paError as exc: # Includes truncated payloads and formats the SDK cannot parse. Not the diff --git a/src/markproof/cli.py b/src/markproof/cli.py index 3bbeab1..520b5e3 100644 --- a/src/markproof/cli.py +++ b/src/markproof/cli.py @@ -39,6 +39,7 @@ from markproof.checks.synthid import WatermarkConfig, load_watermark_config from markproof.config import ( ConfigError, + DocumentProbeConfig, HttpChatProbeConfig, MarkproofConfig, MediaProbeConfig, @@ -47,6 +48,7 @@ load_config, ) from markproof.probes.base import Evidence, ProbeError +from markproof.probes.document import DocumentProbe from markproof.probes.http_chat import HttpChatProbe from markproof.probes.media import MediaProbe from markproof.probes.ui import UiProbe @@ -250,6 +252,8 @@ def _collect(config: MarkproofConfig) -> tuple[list[Evidence], list[Finding]]: # kind this build does not know as a chat probe. if isinstance(probe_config, MediaProbeConfig): evidences.append(MediaProbe(probe_config).collect()) + elif isinstance(probe_config, DocumentProbeConfig): + evidences.append(DocumentProbe(probe_config).collect()) elif isinstance(probe_config, UiProbeConfig): evidences.append(UiProbe(probe_config).collect()) elif isinstance(probe_config, HttpChatProbeConfig): diff --git a/src/markproof/config.py b/src/markproof/config.py index dc4d924..42427df 100644 --- a/src/markproof/config.py +++ b/src/markproof/config.py @@ -26,6 +26,7 @@ "Applicability", "AuthConfig", "ConfigError", + "DocumentProbeConfig", "HttpChatProbeConfig", "MarkproofConfig", "MediaProbeConfig", @@ -283,10 +284,59 @@ def _supported_lang(cls, v: str) -> str: return v +class DocumentProbeConfig(BaseModel): + """A document to fetch as bytes and check for provenance. + + Deliberately not the UI probe. A C2PA binding is a hash over what the server + sent; a browser normalises markup before anything is readable, so the rendered + document and the delivered one are different bytes and only one of them is + what the manifest signs. Fetching plainly is both more faithful and cheaper — + no browser, no extra. + """ + + model_config = ConfigDict(extra="forbid", frozen=True) + + id: str = Field(min_length=1) + type: Literal["document"] + url: str = Field(min_length=1) + lang: str = "de" + auth: AuthConfig | None = None + timeout_seconds: float = Field(default=30.0, gt=0, le=300) + prompt_id: str = Field(default="document-fetch", min_length=1) + + max_bytes: int = Field(default=8 * 1024 * 1024, ge=1024) + """Refuse to hash a document larger than this. + + A provenance check reads the whole body into memory, so an endpoint that + answers with a stream would otherwise decide how much memory this process + uses. Eight megabytes is far past any HTML document and far short of a + problem. + """ + + @property + def probe_kind(self) -> ProbeKind: + return ProbeKind.DOCUMENT + + @field_validator("url") + @classmethod + def _http_url(cls, v: str) -> str: + if not v.startswith(("http://", "https://")): + raise ValueError("url must start with http:// or https://") + return v + + @field_validator("lang") + @classmethod + def _supported_lang(cls, v: str) -> str: + if v not in SUPPORTED_LANGS: + raise ValueError(f"lang {v!r} is not supported (have: {', '.join(SUPPORTED_LANGS)})") + return v + + #: A probe entry in the config. Discriminated on ``type`` so an unknown probe #: kind is a loud config error rather than a silently ignored block. ProbeConfig = Annotated[ - HttpChatProbeConfig | UiProbeConfig | MediaProbeConfig, Field(discriminator="type") + HttpChatProbeConfig | UiProbeConfig | MediaProbeConfig | DocumentProbeConfig, + Field(discriminator="type"), ] diff --git a/src/markproof/probes/base.py b/src/markproof/probes/base.py index 8d23054..c293065 100644 --- a/src/markproof/probes/base.py +++ b/src/markproof/probes/base.py @@ -90,9 +90,39 @@ class Artifact(BaseModel): into the artefacts directory. """ + sidecar_manifest: bytes | None = Field(default=None, exclude=True, repr=False) + """A C2PA manifest that travels *beside* these bytes rather than inside them. + + Some formats cannot carry an embedded manifest — HTML is the one that matters + here — so C2PA binds an external manifest to the document by hashing it, and + points at the manifest from a ```` element or an + RFC 8288 ``Link:`` response header. The bytes below are then exactly what the + server sent, which is the point: the binding covers them, so anything that + rewrites them on the way — a minifier, an HTML-transforming CDN — invalidates + the provenance claim while the page still renders perfectly. + + Excluded from serialisation for the same reason as ``data``. + """ + + sidecar_source: str | None = None + """How the sidecar manifest was found — ``link-header`` or ``link-element``. + + Kept because the two are not equivalent in practice: a header survives an HTML + rewrite that would strip or move the element, so a report saying which one the + delivery chain actually used tells an operator something they cannot see from + a pass alone. + """ + @classmethod def of( - cls, data: bytes, *, artifact_id: str, media_type: str, source_url: str | None = None + cls, + data: bytes, + *, + artifact_id: str, + media_type: str, + source_url: str | None = None, + sidecar_manifest: bytes | None = None, + sidecar_source: str | None = None, ) -> Artifact: """Build an artefact from payload bytes, computing size and digest.""" return cls( @@ -102,6 +132,8 @@ def of( sha256=sha256_hex(data), source_url=source_url, data=data, + sidecar_manifest=sidecar_manifest, + sidecar_source=sidecar_source, ) diff --git a/src/markproof/probes/document.py b/src/markproof/probes/document.py new file mode 100644 index 0000000..33ef5d8 --- /dev/null +++ b/src/markproof/probes/document.py @@ -0,0 +1,205 @@ +# SPDX-FileCopyrightText: 2026 Lukas Friedrich / Tippel +# SPDX-License-Identifier: Apache-2.0 +"""Document probe — fetch what the server sent, and find the manifest bound to it. + +Why the bytes and not the page +------------------------------ +A C2PA manifest binds to a document by hashing it. For formats that cannot carry +an embedded manifest — HTML is the one that matters — the manifest travels +alongside, and the document is pointed at it by an RFC 8288 ``Link:`` response +header or a ```` element (C2PA 2.4 §A.7, §15.5.3.2). + +So the thing under test is the response body exactly as delivered. The UI probe +cannot supply it: a browser normalises markup before anything is readable, so +what it reports is a projection of the document rather than the document. That +difference is the whole reason this probe exists, and it is also the point of the +check — a minifier, a CDN's HTML rewriting, or a template change invalidates the +binding while the page still renders perfectly, and nothing else notices. + +What it deliberately does not do +-------------------------------- +It does not follow a manifest URL to another host. A provenance claim that +depends on a third party being up is a claim that stops being checkable when they +are not, and the C2PA specification's own preference for external manifests does +not extend to letting them wander. +""" + +from __future__ import annotations + +import re +from urllib.parse import urljoin, urlparse + +import httpx + +from markproof.config import DocumentProbeConfig +from markproof.probes.base import ( + Artifact, + Evidence, + Message, + ProbeError, + Role, + Turn, + sha256_hex, +) +from markproof.rules.schema import ProbeKind + +__all__ = ["DocumentProbe", "manifest_link_from_header", "manifest_link_from_html"] + +#: The relation type C2PA registers for an external manifest. +_REL = "c2pa-manifest" + +#: `Link: ; rel="c2pa-manifest"` — one entry of a comma-separated list. +#: Bounded pieces only: no nested quantifier, so matching stays linear. +_LINK_HEADER = re.compile( + r"<(?P[^>]{1,2048})>\s*;(?P[^,]{0,512})", +) + +#: `` in either attribute order. +_LINK_ELEMENT = re.compile( + r"[^>]{0,1024})>", + re.IGNORECASE, +) +_HREF = re.compile(r'href\s*=\s*["\']([^"\']{1,2048})["\']', re.IGNORECASE) +_REL_ATTR = re.compile(r'rel\s*=\s*["\']?([^"\'>\s]{1,64})', re.IGNORECASE) + + +def manifest_link_from_header(value: str) -> str | None: + """The manifest URL advertised in a ``Link:`` header, if there is one. + + Preferred over the element where both are present: a header survives an HTML + rewrite that would move or strip the element, and it needs no modification of + the document body — which is what makes it the natural fit for a server that + generates pages. + """ + for match in _LINK_HEADER.finditer(value): + params = match.group("params").lower() + if re.search(rf'rel\s*=\s*"?{re.escape(_REL)}"?(\s|;|$)', params): + return match.group("url").strip() + return None + + +def manifest_link_from_html(body: str) -> str | None: + """The manifest URL advertised by a ```` element, if there is one.""" + for element in _LINK_ELEMENT.finditer(body): + attrs = element.group("attrs") + rel = _REL_ATTR.search(attrs) + if rel is None or rel.group(1).lower() != _REL: + continue + href = _HREF.search(attrs) + if href is not None: + return href.group(1).strip() + return None + + +class DocumentProbe: + """Fetches a document and whatever manifest is bound to it.""" + + def __init__(self, config: DocumentProbeConfig) -> None: + self.config = config + self.probe_id = config.id + self.probe_kind = ProbeKind.DOCUMENT + + def collect(self) -> Evidence: + """Fetch the document, resolve its manifest, and record both. + + Raises: + ProbeError: for any transport failure, a non-2xx status, or a body + larger than ``max_bytes``. A document that could not be fetched is + an operational finding, never a silent pass. + """ + headers = {} + if self.config.auth is not None: + name, value = self.config.auth.resolve() + headers[name] = value + + try: + with httpx.Client(timeout=self.config.timeout_seconds, follow_redirects=True) as client: + response = client.get(self.config.url, headers=headers) + body = self._body_of(response) + manifest, source = self._manifest_for(client, response, body) + except ProbeError: + raise + except httpx.HTTPError as exc: + raise ProbeError(f"{self.config.url}: could not be fetched — {exc}") from exc + + media_type = ( + (response.headers.get("content-type") or "application/octet-stream") + .split(";")[0] + .strip() + ) + artifact = Artifact.of( + body, + artifact_id=f"{self.probe_id}-document", + media_type=media_type or "application/octet-stream", + source_url=str(response.url), + sidecar_manifest=manifest, + sidecar_source=source, + ) + + # The response body is the evidence, but it is not perceivable text: a + # label check must not read markup and call it what a person sees. + summary = f"{len(body)} byte(s) of {media_type}" + turn = Turn( + prompt_id=self.config.prompt_id, + request=[], + response=Message(role=Role.ASSISTANT, content=summary), + response_sha256=sha256_hex(summary), + status_code=response.status_code, + artifacts=(artifact,), + ) + return Evidence( + probe_id=self.probe_id, + probe_kind=self.probe_kind, + target_name=self.config.id, + lang=self.config.lang, + turns=(turn,), + ) + + def _body_of(self, response: httpx.Response) -> bytes: + if response.status_code >= 400: + raise ProbeError( + f"{self.config.url} returned HTTP {response.status_code} — " + "an error page is not the document under test" + ) + body = response.content + if len(body) > self.config.max_bytes: + raise ProbeError( + f"{self.config.url}: {len(body)} bytes exceeds max_bytes " + f"({self.config.max_bytes}); raise it deliberately if the document is really " + "this large" + ) + return body + + def _manifest_for( + self, client: httpx.Client, response: httpx.Response, body: bytes + ) -> tuple[bytes | None, str | None]: + """Resolve the external manifest, header first, then the document.""" + target = manifest_link_from_header(response.headers.get("link", "")) + source = "link-header" if target else None + if target is None: + # Decoding for the element scan only. The bytes handed to the checker + # stay exactly as delivered — the binding covers those, not a + # re-encoding of them. + target = manifest_link_from_html(body.decode("utf-8", errors="replace")) + source = "link-element" if target else None + if target is None: + return None, None + + url = urljoin(str(response.url), target) + if urlparse(url).netloc != urlparse(str(response.url)).netloc: + raise ProbeError( + f"{self.config.url}: the manifest is hosted on another origin ({url}). " + "Refusing to follow it — a provenance claim that depends on a third party " + "being reachable stops being checkable when they are not." + ) + try: + manifest = client.get(url) + except httpx.HTTPError as exc: + raise ProbeError(f"{url}: the linked manifest could not be fetched — {exc}") from exc + if manifest.status_code >= 400: + raise ProbeError( + f"{url}: the document advertises a manifest that answers HTTP " + f"{manifest.status_code}. A dangling provenance link is worse than none: " + "it reads as marked and verifies as nothing." + ) + return manifest.content, source diff --git a/src/markproof/rulepacks/art50-eu-2026.07.yaml b/src/markproof/rulepacks/art50-eu-2026.07.yaml index af606ac..14b22d2 100644 --- a/src/markproof/rulepacks/art50-eu-2026.07.yaml +++ b/src/markproof/rulepacks/art50-eu-2026.07.yaml @@ -181,6 +181,44 @@ rules: on_uncertain: fail severity: fail + # --------------------------------------------------------------------------- + # MPF-M-002 — the same marking question as MPF-M-001, asked of a document that + # cannot carry a manifest inside it. Added once C2PA 2.4 made the binding a + # published convention rather than something markproof would have had to invent. + # --------------------------------------------------------------------------- + - id: MPF-M-002 + title: "A delivered document carries a manifest binding it as AI-generated" + article: "Art. 50(2)" + obligation: synthetic-text-marking + guideline_ref: "Guidelines C(2026) 5054, §4; Code of Practice, Section 1, Commitment 1" + rationale: >- + The marking duty covers text, and text is usually published as a document + rather than returned from a chat endpoint. A document that cannot carry an + embedded manifest — HTML above all — is bound to an external one instead: + C2PA 2.4 Appendix A.7 defines the binding, and the document points at the + manifest through an RFC 8288 `Link:` response header or a + `` element. + What makes this worth checking rather than assuming is that the binding is a + hash over the bytes the server sent. Anything that rewrites them on the way — + a minifier, an HTML-transforming CDN, a template change — turns a valid + provenance claim into an invalid one while the page still renders perfectly. + Nothing errors, no log line appears, and the operator finds out when somebody + from outside asks. That is the failure this project exists to catch, in the + format most likely to suffer it. + Severity is fail rather than warn because every question here is decidable: + either a manifest is bound to these bytes or it is not, either it validates + or it does not, and either it declares an AI source type or it declares + something else. + applies_to: [document] + check: + type: c2pa-verify + accept_source_types: + - trainedAlgorithmicMedia + - compositeWithTrainedAlgorithmicMedia + trust: + allow_self_signed: true + severity: fail + # --------------------------------------------------------------------------- # MPF-L-001 — the deployer-side counterpart to MPF-M-001. Article 50(2) asks # the provider for a machine-readable mark; Article 50(4) asks the deployer diff --git a/src/markproof/rules/engine.py b/src/markproof/rules/engine.py index b6a4a8d..f9eb825 100644 --- a/src/markproof/rules/engine.py +++ b/src/markproof/rules/engine.py @@ -407,6 +407,7 @@ def _finding_from_c2pa(rule: Rule, evidence: Evidence, check: C2paVerifyCheck) - artifact.media_type, check, artifact_id=artifact.id, + sidecar_manifest=artifact.sidecar_manifest, ) ) diff --git a/src/markproof/rules/schema.py b/src/markproof/rules/schema.py index 23b05e0..168a001 100644 --- a/src/markproof/rules/schema.py +++ b/src/markproof/rules/schema.py @@ -70,6 +70,16 @@ class ProbeKind(StrEnum): UI = "ui" MEDIA = "media" + DOCUMENT = "document" + """A document fetched as bytes, for provenance rather than for rendering. + + Distinct from ``ui`` on purpose. A UI probe drives a browser and reports what a + person would read; a C2PA binding is a hash over what the *server sent*, and + those are not the same bytes — a browser normalises markup, and a rendered + document is a projection of the delivered one. Verifying provenance needs the + delivered bytes and nothing else, which needs no browser at all. + """ + class Obligation(StrEnum): """Which Article 50 duty a rule serves. diff --git a/tests/fixtures/documents/MANIFEST.json b/tests/fixtures/documents/MANIFEST.json new file mode 100644 index 0000000..bf9ed1c --- /dev/null +++ b/tests/fixtures/documents/MANIFEST.json @@ -0,0 +1,11 @@ +{ + "_comment": "Generated by generate.py. Signatures are not reproducible.", + "c2pa_python": "0.37.8", + "c2pa_rs_sdk": "0.90.15", + "cases": { + "signed-valid.html": "manifest bound to these bytes, AI source type -> PASS", + "tampered.html": "signed-valid's manifest, document edited after -> FAIL", + "signed-wrong-type.html": "validly bound, declares a camera capture -> FAIL", + "unsigned.html": "no manifest and no link to one -> FAIL" + } +} diff --git a/tests/fixtures/documents/generate.py b/tests/fixtures/documents/generate.py new file mode 100644 index 0000000..197e8c4 --- /dev/null +++ b/tests/fixtures/documents/generate.py @@ -0,0 +1,122 @@ +# SPDX-FileCopyrightText: 2026 Lukas Friedrich / Tippel +# SPDX-License-Identifier: Apache-2.0 +"""Build the HTML/manifest fixtures for the document check. + +Run with ``python tests/fixtures/documents/generate.py``. + +Why these are generated rather than committed by hand: the manifests are signed +with the test CA in ``tests/fixtures/media/generate.py``, and an ECDSA signature +differs on every run, so the bytes are not reproducible and a committed pair would +drift the moment anything about signing changed. + +The pairing is the point. A document that cannot embed a manifest is bound to an +external one by hashing the delivered bytes (C2PA 2.4 §A.7), so each case is a +document *and* the manifest that claims it — and ``tampered.html`` is the same +manifest with a document that changed by four characters afterwards. +""" + +# ruff: noqa: T201 - a generator script reports what it wrote. + +from __future__ import annotations + +import io +import json +import sys +from pathlib import Path + +_HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(_HERE.parent / "media")) + +import c2pa # noqa: E402 +import generate as media # type: ignore[import-not-found] # noqa: E402 - path set above + +_AI = "http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia" +_CAMERA = "http://cv.iptc.org/newscodes/digitalsourcetype/digitalCapture" + +#: A small, realistic generated page. The year is what `tampered.html` changes, +#: because a fact edited after signing is the failure an operator would actually +#: make: nothing about the page looks wrong afterwards. +PAGE = ( + '\n\n\n' + "Bäckerei Mustermann\n" + '\n\n' + "\n

Frisch gebacken seit 1932.

\n" + "

Unsere Backstube öffnet um sechs.

\n\n\n" +) + + +def _manifest(source_type: str) -> dict[str, object]: + return { + "claim_generator_info": [{"name": "markproof-fixtures", "version": "1"}], + "title": "index.html", + "format": "text/html", + "assertions": [ + { + "label": "c2pa.actions", + "data": {"actions": [{"action": "c2pa.created", "digitalSourceType": source_type}]}, + }, + { + "label": "c2pa.ai-disclosure", + "data": {"modelType": "text", "humanOversightLevel": "prompt_guided"}, + }, + ], + } + + +def _sign(document: bytes, source_type: str) -> bytes: + """A detached manifest bound to these exact bytes. + + ``set_no_embed`` is what makes this possible at all: c2pa-rs refuses to embed + into HTML, but it will hash the bytes and hand back a manifest that stands + beside them — which is exactly the A.7 arrangement. + """ + with media._builder_context(): + builder = c2pa.Builder(_manifest(source_type)) + builder.set_no_embed() + sink = io.BytesIO() + signed: bytes = builder.sign(media.build_signer(), "text/html", io.BytesIO(document), sink) + return signed + + +def write() -> None: + document = PAGE.encode("utf-8") + + (_HERE / "signed-valid.html").write_bytes(document) + (_HERE / "signed-valid.html.c2pa").write_bytes(_sign(document, _AI)) + + # Same manifest, document edited afterwards. Four characters, no visible + # damage, and the provenance claim no longer holds. + (_HERE / "tampered.html").write_bytes(document.replace(b"1932", b"1888")) + + wrong = document.replace(b"Mustermann", b"Musterfrau") + (_HERE / "signed-wrong-type.html").write_bytes(wrong) + (_HERE / "signed-wrong-type.html.c2pa").write_bytes(_sign(wrong, _CAMERA)) + + (_HERE / "unsigned.html").write_bytes( + document.replace(b'\n', b"") + ) + + (_HERE / "MANIFEST.json").write_text( + json.dumps( + { + "_comment": "Generated by generate.py. Signatures are not reproducible.", + "c2pa_python": c2pa.version("c2pa-python"), + "c2pa_rs_sdk": c2pa.sdk_version(), + "cases": { + "signed-valid.html": "manifest bound to these bytes, AI source type -> PASS", + "tampered.html": "signed-valid's manifest, document edited after -> FAIL", + "signed-wrong-type.html": "validly bound, declares a camera capture -> FAIL", + "unsigned.html": "no manifest and no link to one -> FAIL", + }, + }, + indent=2, + ) + + "\n" + ) + + +if __name__ == "__main__": + write() + for path in sorted(_HERE.glob("*")): + if path.name != "generate.py": + print(f" {path.name} {path.stat().st_size} B") diff --git a/tests/fixtures/documents/signed-valid.html b/tests/fixtures/documents/signed-valid.html new file mode 100644 index 0000000..6c62baa --- /dev/null +++ b/tests/fixtures/documents/signed-valid.html @@ -0,0 +1,11 @@ + + + +Bäckerei Mustermann + + + +

Frisch gebacken seit 1932.

+

Unsere Backstube öffnet um sechs.

+ + diff --git a/tests/fixtures/documents/signed-valid.html.c2pa b/tests/fixtures/documents/signed-valid.html.c2pa new file mode 100644 index 0000000..8cc8381 Binary files /dev/null and b/tests/fixtures/documents/signed-valid.html.c2pa differ diff --git a/tests/fixtures/documents/signed-wrong-type.html b/tests/fixtures/documents/signed-wrong-type.html new file mode 100644 index 0000000..8f56f60 --- /dev/null +++ b/tests/fixtures/documents/signed-wrong-type.html @@ -0,0 +1,11 @@ + + + +Bäckerei Musterfrau + + + +

Frisch gebacken seit 1932.

+

Unsere Backstube öffnet um sechs.

+ + diff --git a/tests/fixtures/documents/signed-wrong-type.html.c2pa b/tests/fixtures/documents/signed-wrong-type.html.c2pa new file mode 100644 index 0000000..983c5de Binary files /dev/null and b/tests/fixtures/documents/signed-wrong-type.html.c2pa differ diff --git a/tests/fixtures/documents/tampered.html b/tests/fixtures/documents/tampered.html new file mode 100644 index 0000000..e779baa --- /dev/null +++ b/tests/fixtures/documents/tampered.html @@ -0,0 +1,11 @@ + + + +Bäckerei Mustermann + + + +

Frisch gebacken seit 1888.

+

Unsere Backstube öffnet um sechs.

+ + diff --git a/tests/fixtures/documents/unsigned.html b/tests/fixtures/documents/unsigned.html new file mode 100644 index 0000000..9462049 --- /dev/null +++ b/tests/fixtures/documents/unsigned.html @@ -0,0 +1,10 @@ + + + +Bäckerei Mustermann + + +

Frisch gebacken seit 1932.

+

Unsere Backstube öffnet um sechs.

+ + diff --git a/tests/golden/chat-conformant/expected_report.json b/tests/golden/chat-conformant/expected_report.json index 64e13c5..94e0b3d 100644 --- a/tests/golden/chat-conformant/expected_report.json +++ b/tests/golden/chat-conformant/expected_report.json @@ -70,7 +70,7 @@ "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", "id": "art50-eu-2026.07", "license": "CC-BY-4.0", - "sha256": "d8c9be8eab221fbd654d987c71d12b24d61ed4587e0537b1f799b0701d8c9ccb", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", "version": "1.0.0" }, "run": { diff --git a/tests/golden/chat-near-miss/expected_report.json b/tests/golden/chat-near-miss/expected_report.json index 858a315..b501ca8 100644 --- a/tests/golden/chat-near-miss/expected_report.json +++ b/tests/golden/chat-near-miss/expected_report.json @@ -72,7 +72,7 @@ "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", "id": "art50-eu-2026.07", "license": "CC-BY-4.0", - "sha256": "d8c9be8eab221fbd654d987c71d12b24d61ed4587e0537b1f799b0701d8c9ccb", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", "version": "1.0.0" }, "run": { diff --git a/tests/golden/chat-silent/expected_report.json b/tests/golden/chat-silent/expected_report.json index 3cab8e1..7b2c356 100644 --- a/tests/golden/chat-silent/expected_report.json +++ b/tests/golden/chat-silent/expected_report.json @@ -64,7 +64,7 @@ "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", "id": "art50-eu-2026.07", "license": "CC-BY-4.0", - "sha256": "d8c9be8eab221fbd654d987c71d12b24d61ed4587e0537b1f799b0701d8c9ccb", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", "version": "1.0.0" }, "run": { diff --git a/tests/golden/document-marked/evidence.json b/tests/golden/document-marked/evidence.json new file mode 100644 index 0000000..37ec9e5 --- /dev/null +++ b/tests/golden/document-marked/evidence.json @@ -0,0 +1,36 @@ +{ + "description": "An HTML page bound to an external C2PA manifest. The format cannot embed one, so the binding is a hash over the delivered bytes.", + "evidences": [ + { + "lang": "de", + "probe_id": "page", + "probe_kind": "document", + "target_name": "golden", + "turns": [ + { + "artifacts": [ + { + "_fixture": "documents/signed-valid.html", + "_sidecar": "documents/signed-valid.html.c2pa", + "id": "document-fetch-document", + "media_type": "text/html", + "sha256": "8cc23ef8d1e33877c433382a4793455df5609c8615951526079a4f8e5fbea960", + "sidecar_source": "link-header", + "size_bytes": 257, + "source_url": "https://pages.example.invalid/index.html" + } + ], + "prompt_id": "document-fetch", + "request": [], + "response": { + "content": "257 byte(s) of text/html", + "role": "assistant" + }, + "response_sha256": "d672e483243ab1440245ba95fd03f26b36a1ad7ec51371407043e0f2407d4371", + "status_code": 200 + } + ] + } + ], + "rulepack": "art50-eu-2026.07" +} diff --git a/tests/golden/document-marked/expected_report.json b/tests/golden/document-marked/expected_report.json new file mode 100644 index 0000000..dde4359 --- /dev/null +++ b/tests/golden/document-marked/expected_report.json @@ -0,0 +1,45 @@ +{ + "findings": [ + { + "article": "Art. 50(2)", + "detail": { + "assets": [ + "document-fetch-document" + ], + "checked": 1, + "outcome": "verified" + }, + "evidence_sha256": [ + "8cc23ef8d1e33877c433382a4793455df5609c8615951526079a4f8e5fbea960" + ], + "guideline_ref": "Guidelines C(2026) 5054, \u00a74; Code of Practice, Section 1, Commitment 1", + "message": "1 asset carries a valid, correctly marked manifest", + "obligation": "synthetic-text-marking", + "probe_id": "page", + "result": "PASS", + "rule_id": "MPF-M-002", + "title": "A delivered document carries a manifest binding it as AI-generated" + } + ], + "rulepack": { + "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", + "id": "art50-eu-2026.07", + "license": "CC-BY-4.0", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", + "version": "1.0.0" + }, + "run": { + "markproof_version": "0.0.0-golden", + "platform": "golden", + "python_version": "0.0.0", + "timestamp": "2026-08-31T12:00:00+00:00" + }, + "schema_version": 1, + "summary": { + "failed": 0, + "passed": 1, + "skipped": 0, + "warned": 0 + }, + "target": "golden" +} diff --git a/tests/golden/document-rewritten/evidence.json b/tests/golden/document-rewritten/evidence.json new file mode 100644 index 0000000..f97d777 --- /dev/null +++ b/tests/golden/document-rewritten/evidence.json @@ -0,0 +1,36 @@ +{ + "description": "The same manifest, four characters edited afterwards. The page still renders perfectly and the provenance claim no longer holds — the silent delivery-chain regression this project exists to catch.", + "evidences": [ + { + "lang": "de", + "probe_id": "page", + "probe_kind": "document", + "target_name": "golden", + "turns": [ + { + "artifacts": [ + { + "_fixture": "documents/tampered.html", + "_sidecar": "documents/signed-valid.html.c2pa", + "id": "document-fetch-document", + "media_type": "text/html", + "sha256": "166c7e32179b19ee0a4f41168fa297a6f3673c72e9b1d12173bba98d6e1a11c3", + "sidecar_source": "link-header", + "size_bytes": 257, + "source_url": "https://pages.example.invalid/index.html" + } + ], + "prompt_id": "document-fetch", + "request": [], + "response": { + "content": "257 byte(s) of text/html", + "role": "assistant" + }, + "response_sha256": "d672e483243ab1440245ba95fd03f26b36a1ad7ec51371407043e0f2407d4371", + "status_code": 200 + } + ] + } + ], + "rulepack": "art50-eu-2026.07" +} diff --git a/tests/golden/document-rewritten/expected_report.json b/tests/golden/document-rewritten/expected_report.json new file mode 100644 index 0000000..56fd878 --- /dev/null +++ b/tests/golden/document-rewritten/expected_report.json @@ -0,0 +1,48 @@ +{ + "findings": [ + { + "article": "Art. 50(2)", + "detail": { + "assets": [ + "document-fetch-document" + ], + "checked": 1, + "failed_assets": [ + "document-fetch-document" + ], + "outcome": "invalid" + }, + "evidence_sha256": [ + "166c7e32179b19ee0a4f41168fa297a6f3673c72e9b1d12173bba98d6e1a11c3" + ], + "guideline_ref": "Guidelines C(2026) 5054, \u00a74; Code of Practice, Section 1, Commitment 1", + "message": "1 of 1 asset(s) failed: manifest present but validation failed \u2014 the asset was altered after signing (assertion.dataHash.mismatch)", + "obligation": "synthetic-text-marking", + "probe_id": "page", + "result": "FAIL", + "rule_id": "MPF-M-002", + "title": "A delivered document carries a manifest binding it as AI-generated" + } + ], + "rulepack": { + "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", + "id": "art50-eu-2026.07", + "license": "CC-BY-4.0", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", + "version": "1.0.0" + }, + "run": { + "markproof_version": "0.0.0-golden", + "platform": "golden", + "python_version": "0.0.0", + "timestamp": "2026-08-31T12:00:00+00:00" + }, + "schema_version": 1, + "summary": { + "failed": 1, + "passed": 0, + "skipped": 0, + "warned": 0 + }, + "target": "golden" +} diff --git a/tests/golden/generate.py b/tests/golden/generate.py index 4d36340..9274488 100644 --- a/tests/golden/generate.py +++ b/tests/golden/generate.py @@ -60,6 +60,33 @@ def _turn(prompt_id: str, response: str, *, user_said: str | None = None) -> dic } +def _document_turn(prompt_id: str, document: str, manifest: str | None) -> dict[str, Any]: + """A fetched document plus the manifest bound to it, if there is one.""" + import hashlib + + data = (_FIXTURES / document).read_bytes() + summary = f"{len(data)} byte(s) of text/html" + artifact: dict[str, Any] = { + "id": f"{prompt_id}-document", + "media_type": "text/html", + "size_bytes": len(data), + "sha256": hashlib.sha256(data).hexdigest(), + "source_url": "https://pages.example.invalid/index.html", + "sidecar_source": "link-header" if manifest else None, + "_fixture": document, + } + if manifest: + artifact["_sidecar"] = manifest + return { + "prompt_id": prompt_id, + "request": [], + "response": {"role": "assistant", "content": summary}, + "response_sha256": hashlib.sha256(summary.encode()).hexdigest(), + "status_code": 200, + "artifacts": [artifact], + } + + def _media_turn(prompt_id: str, fixture: str, media_type: str) -> dict[str, Any]: import hashlib @@ -190,6 +217,51 @@ def _media_turn(prompt_id: str, fixture: str, media_type: str) -> dict[str, Any] } ], }, + "document-marked": { + "description": ( + "An HTML page bound to an external C2PA manifest. The format cannot " + "embed one, so the binding is a hash over the delivered bytes." + ), + "rulepack": "art50-eu-2026.07", + "evidences": [ + { + "probe_id": "page", + "probe_kind": "document", + "target_name": "golden", + "lang": "de", + "turns": [ + _document_turn( + "document-fetch", + "documents/signed-valid.html", + "documents/signed-valid.html.c2pa", + ) + ], + } + ], + }, + "document-rewritten": { + "description": ( + "The same manifest, four characters edited afterwards. The page still " + "renders perfectly and the provenance claim no longer holds — the " + "silent delivery-chain regression this project exists to catch." + ), + "rulepack": "art50-eu-2026.07", + "evidences": [ + { + "probe_id": "page", + "probe_kind": "document", + "target_name": "golden", + "lang": "de", + "turns": [ + _document_turn( + "document-fetch", + "documents/tampered.html", + "documents/signed-valid.html.c2pa", + ) + ], + } + ], + }, "probe-unreachable": { "description": ( "The endpoint could not be reached. FAIL, never a silent pass — and the " diff --git a/tests/golden/media-marked/expected_report.json b/tests/golden/media-marked/expected_report.json index c5e6813..77e89e5 100644 --- a/tests/golden/media-marked/expected_report.json +++ b/tests/golden/media-marked/expected_report.json @@ -41,7 +41,7 @@ "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", "id": "art50-eu-2026.07", "license": "CC-BY-4.0", - "sha256": "d8c9be8eab221fbd654d987c71d12b24d61ed4587e0537b1f799b0701d8c9ccb", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", "version": "1.0.0" }, "run": { diff --git a/tests/golden/media-tampered/expected_report.json b/tests/golden/media-tampered/expected_report.json index 06c15a6..d924495 100644 --- a/tests/golden/media-tampered/expected_report.json +++ b/tests/golden/media-tampered/expected_report.json @@ -44,7 +44,7 @@ "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", "id": "art50-eu-2026.07", "license": "CC-BY-4.0", - "sha256": "d8c9be8eab221fbd654d987c71d12b24d61ed4587e0537b1f799b0701d8c9ccb", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", "version": "1.0.0" }, "run": { diff --git a/tests/golden/media-wrong-source-type/expected_report.json b/tests/golden/media-wrong-source-type/expected_report.json index c4353bd..9483645 100644 --- a/tests/golden/media-wrong-source-type/expected_report.json +++ b/tests/golden/media-wrong-source-type/expected_report.json @@ -47,7 +47,7 @@ "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", "id": "art50-eu-2026.07", "license": "CC-BY-4.0", - "sha256": "d8c9be8eab221fbd654d987c71d12b24d61ed4587e0537b1f799b0701d8c9ccb", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", "version": "1.0.0" }, "run": { diff --git a/tests/golden/multi-probe/expected_report.json b/tests/golden/multi-probe/expected_report.json index d858c0f..98437d1 100644 --- a/tests/golden/multi-probe/expected_report.json +++ b/tests/golden/multi-probe/expected_report.json @@ -162,7 +162,7 @@ "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", "id": "art50-eu-2026.07", "license": "CC-BY-4.0", - "sha256": "d8c9be8eab221fbd654d987c71d12b24d61ed4587e0537b1f799b0701d8c9ccb", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", "version": "1.0.0" }, "run": { diff --git a/tests/golden/probe-unreachable/expected_report.json b/tests/golden/probe-unreachable/expected_report.json index 223da39..493fd9d 100644 --- a/tests/golden/probe-unreachable/expected_report.json +++ b/tests/golden/probe-unreachable/expected_report.json @@ -17,7 +17,7 @@ "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", "id": "art50-eu-2026.07", "license": "CC-BY-4.0", - "sha256": "d8c9be8eab221fbd654d987c71d12b24d61ed4587e0537b1f799b0701d8c9ccb", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", "version": "1.0.0" }, "run": { diff --git a/tests/golden/scope-declared-out/expected_report.json b/tests/golden/scope-declared-out/expected_report.json index 0306eb4..c52250e 100644 --- a/tests/golden/scope-declared-out/expected_report.json +++ b/tests/golden/scope-declared-out/expected_report.json @@ -54,7 +54,7 @@ "attribution": "Derived from: European Commission, \"Guidelines on the implementation of the transparency obligations for certain AI systems under Article 50 of Regulation (EU) 2024/1689 (the AI Act)\", C(2026) 5054 final, ANNEX, 20.07.2026, and the \"Code of Practice on Transparency of AI-generated Content\", 10.06.2026 \u2014 both published by the European Commission under CC BY 4.0 (Commission Decision 2011/833/EU). Rules are paraphrased and cite the paragraph numbers of the source; refer to the cited paragraphs for the authoritative text. Neither the European Commission nor the AI Board endorses this rulepack or the reading of their material expressed in it.", "id": "art50-eu-2026.07", "license": "CC-BY-4.0", - "sha256": "d8c9be8eab221fbd654d987c71d12b24d61ed4587e0537b1f799b0701d8c9ccb", + "sha256": "e644cbac5b75202f2308ea6fafd3c6dbc72cedee01d67c6338b654a5fd734994", "version": "1.0.0" }, "run": { diff --git a/tests/test_determinism.py b/tests/test_determinism.py index c8442e8..878192e 100644 --- a/tests/test_determinism.py +++ b/tests/test_determinism.py @@ -68,8 +68,13 @@ def _evidence_with_bytes(raw: dict[str, Any]) -> Evidence: for art in turn.get("artifacts", []): art = dict(art) fixture = art.pop("_fixture", None) + sidecar = art.pop("_sidecar", None) artifacts.append( - Artifact(**art, data=(_FIXTURES / fixture).read_bytes() if fixture else None) + Artifact( + **art, + data=(_FIXTURES / fixture).read_bytes() if fixture else None, + sidecar_manifest=(_FIXTURES / sidecar).read_bytes() if sidecar else None, + ) ) turns.append({**turn, "artifacts": tuple(artifacts)}) return Evidence.model_validate({**raw, "turns": tuple(turns)}) diff --git a/tests/test_document_probe.py b/tests/test_document_probe.py new file mode 100644 index 0000000..8eb8014 --- /dev/null +++ b/tests/test_document_probe.py @@ -0,0 +1,236 @@ +# SPDX-FileCopyrightText: 2026 Lukas Friedrich / Tippel +# SPDX-License-Identifier: Apache-2.0 +"""The document probe and MPF-M-002 — provenance for what cannot embed it. + +C2PA binds a manifest to a document by hashing it. A format that cannot carry the +manifest inside — HTML above all — points at an external one through an RFC 8288 +``Link:`` response header or a ```` element +(C2PA 2.4 §A.7, §15.5.3.2). + +The binding is a hash over the bytes the server sent, which is exactly why this is +worth checking rather than assuming: a minifier, an HTML-rewriting CDN or a +template change turns a valid provenance claim into an invalid one while the page +still renders perfectly. Nothing errors and no log line appears. +""" + +from __future__ import annotations + +from pathlib import Path + +import httpx +import pytest +import respx + +from markproof.checks.c2pa_verify import C2paOutcome, C2paResult, verify_media +from markproof.config import DocumentProbeConfig +from markproof.probes.base import ProbeError +from markproof.probes.document import ( + DocumentProbe, + manifest_link_from_header, + manifest_link_from_html, +) +from markproof.rules.schema import C2paVerifyCheck + +_FIXTURES = Path(__file__).resolve().parent / "fixtures" / "documents" +_URL = "https://pages.example.invalid/index.html" +_MANIFEST_URL = "https://pages.example.invalid/index.html.c2pa" + + +def _fixture(name: str) -> bytes: + return (_FIXTURES / name).read_bytes() + + +def _probe(**overrides: object) -> DocumentProbe: + return DocumentProbe( + DocumentProbeConfig.model_validate( + {"id": "page", "type": "document", "url": _URL, **overrides} + ) + ) + + +class TestFindingTheManifest: + """Two ways a document can point at its manifest, and they are not equivalent.""" + + def test_a_link_header(self) -> None: + assert ( + manifest_link_from_header('; rel="c2pa-manifest"') == "index.html.c2pa" + ) + + def test_a_link_header_among_others(self) -> None: + value = '; rel=preload, ; rel="c2pa-manifest", ; rel=next' + assert manifest_link_from_header(value) == "m.c2pa" + + def test_an_unrelated_link_header_is_not_a_manifest(self) -> None: + assert manifest_link_from_header("; rel=preload") is None + + def test_a_link_element(self) -> None: + html = '' + assert manifest_link_from_html(html) == "m.c2pa" + + def test_attribute_order_does_not_matter(self) -> None: + html = '' + assert manifest_link_from_html(html) == "m.c2pa" + + def test_a_stylesheet_link_is_not_a_manifest(self) -> None: + assert manifest_link_from_html('') is None + + def test_no_link_at_all(self) -> None: + assert manifest_link_from_html("nothing") is None + + +class TestCollecting: + @respx.mock + def test_the_header_form(self) -> None: + respx.get(_URL).mock( + return_value=httpx.Response( + 200, + content=_fixture("signed-valid.html"), + headers={ + "content-type": "text/html; charset=utf-8", + "link": '; rel="c2pa-manifest"', + }, + ) + ) + respx.get(_MANIFEST_URL).mock( + return_value=httpx.Response(200, content=_fixture("signed-valid.html.c2pa")) + ) + artifact = _probe().collect().turns[0].artifacts[0] + assert artifact.sidecar_source == "link-header" + assert artifact.media_type == "text/html" + assert artifact.data == _fixture("signed-valid.html"), "the delivered bytes must be intact" + + @respx.mock + def test_the_element_form_when_there_is_no_header(self) -> None: + respx.get(_URL).mock( + return_value=httpx.Response( + 200, content=_fixture("signed-valid.html"), headers={"content-type": "text/html"} + ) + ) + respx.get(_MANIFEST_URL).mock( + return_value=httpx.Response(200, content=_fixture("signed-valid.html.c2pa")) + ) + artifact = _probe().collect().turns[0].artifacts[0] + assert artifact.sidecar_source == "link-element" + + @respx.mock + def test_a_document_with_no_manifest_still_produces_evidence(self) -> None: + """ "Nothing found" is a finding, not a reason to abort.""" + respx.get(_URL).mock( + return_value=httpx.Response( + 200, content=_fixture("unsigned.html"), headers={"content-type": "text/html"} + ) + ) + artifact = _probe().collect().turns[0].artifacts[0] + assert artifact.sidecar_manifest is None + assert artifact.sidecar_source is None + + @respx.mock + def test_the_manifest_bytes_never_reach_the_report(self) -> None: + """Evidence stays diffable; the digest is what ties a finding to the bytes.""" + respx.get(_URL).mock( + return_value=httpx.Response( + 200, + content=_fixture("signed-valid.html"), + headers={ + "content-type": "text/html", + "link": '; rel="c2pa-manifest"', + }, + ) + ) + respx.get(_MANIFEST_URL).mock( + return_value=httpx.Response(200, content=_fixture("signed-valid.html.c2pa")) + ) + dumped = _probe().collect().model_dump(mode="json") + assert "sidecar_manifest" not in dumped["turns"][0]["artifacts"][0] + assert dumped["turns"][0]["artifacts"][0]["sidecar_source"] == "link-header" + + +class TestRefusals: + @respx.mock + def test_a_manifest_on_another_origin_is_refused(self) -> None: + """A provenance claim that depends on a third party is not self-contained.""" + respx.get(_URL).mock( + return_value=httpx.Response( + 200, + content=_fixture("unsigned.html"), + headers={ + "content-type": "text/html", + "link": '; rel="c2pa-manifest"', + }, + ) + ) + with pytest.raises(ProbeError, match="another origin"): + _probe().collect() + + @respx.mock + def test_a_dangling_manifest_link_is_worse_than_none(self) -> None: + respx.get(_URL).mock( + return_value=httpx.Response( + 200, + content=_fixture("signed-valid.html"), + headers={ + "content-type": "text/html", + "link": '; rel="c2pa-manifest"', + }, + ) + ) + respx.get(_MANIFEST_URL).mock(return_value=httpx.Response(404)) + with pytest.raises(ProbeError, match="reads as marked and verifies as nothing"): + _probe().collect() + + @respx.mock + def test_an_error_page_is_not_the_document_under_test(self) -> None: + respx.get(_URL).mock(return_value=httpx.Response(503, content=b"

down

")) + with pytest.raises(ProbeError, match="HTTP 503"): + _probe().collect() + + @respx.mock + def test_an_oversized_body_is_refused_rather_than_hashed(self) -> None: + respx.get(_URL).mock( + return_value=httpx.Response( + 200, content=b"x" * 5000, headers={"content-type": "text/html"} + ) + ) + with pytest.raises(ProbeError, match="exceeds max_bytes"): + _probe(max_bytes=1024).collect() + + +class TestTheVerdicts: + """The four outcomes, through the real check against generated fixtures.""" + + @staticmethod + def _verify(document: str, manifest: str | None) -> C2paResult: + return verify_media( + _fixture(document), + "text/html", + C2paVerifyCheck(type="c2pa-verify"), + artifact_id=document, + sidecar_manifest=_fixture(manifest) if manifest else None, + ) + + def test_a_bound_manifest_with_an_ai_source_type_passes(self) -> None: + result = self._verify("signed-valid.html", "signed-valid.html.c2pa") + assert result.outcome is C2paOutcome.VERIFIED + assert result.passed + + def test_four_edited_characters_break_the_binding(self) -> None: + """`1932` became `1888` after signing. The page still renders perfectly. + + This is the whole reason the rule exists: a minifier or an HTML-rewriting + CDN does the same thing, silently, to every page it touches. + """ + result = self._verify("tampered.html", "signed-valid.html.c2pa") + assert result.outcome is C2paOutcome.INVALID + assert not result.passed + + def test_validly_signed_is_not_the_same_as_marked_as_ai(self) -> None: + """The distinction Article 50(2) turns on, and the one presence checks miss.""" + result = self._verify("signed-wrong-type.html", "signed-wrong-type.html.c2pa") + assert result.outcome is C2paOutcome.WRONG_SOURCE_TYPE + assert "digitalCapture" in (result.source_type or "") + + def test_no_manifest_is_an_absence_not_an_unreadable_payload(self) -> None: + """HTML cannot embed one, so "none found" is definite rather than inconclusive.""" + result = self._verify("unsigned.html", None) + assert result.outcome is C2paOutcome.MANIFEST_MISSING + assert "cannot carry an embedded manifest" in (result.detail or "")