diff --git a/CHANGELOG.md b/CHANGELOG.md index 8a3bd9e..ed83b3d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,18 @@ explicitly: the **rulepack format** — what a rulepack file may contain — and ## [Unreleased] +_Nothing yet._ + +## [0.1.0] - 2026-09-01 + +The first public release. Until now you could describe your Article 50 setup; from +here you can measure it against the system your users actually reach, and hand +someone else the measurement. + +Everything below is in this release. There was no earlier one: the entries that +lived under "Unreleased" during development are folded in here rather than left +implying a version nobody could install. + ### Security - **The signed report now binds the rules and the endpoint it judged.** A report @@ -33,93 +45,6 @@ explicitly: the **rulepack format** — what a rulepack file may contain — and ### Added -- **`MPF-M-002` verifies the C2PA manifest bound to a delivered document** (#18), - through a new `document` probe. C2PA 2.4 §A.7 binds a manifest to a document - that cannot embed one — HTML above all — by hashing the delivered bytes, and the - document points at it through an RFC 8288 `Link:` header or a - `` element. - - This is the delivery-chain regression the project exists for, in the format most - likely to suffer it: a minifier, an HTML-rewriting CDN or a template change - turns a valid provenance claim into an invalid one while the page still renders - perfectly. Nothing errors and no log line appears. - - The probe fetches the bytes the server sent rather than driving a browser — a - browser normalises markup, and only the delivered bytes are what the manifest - signs. It refuses a manifest on another origin: a provenance claim that depends - on a third party being reachable stops being checkable when they are not, and a - dangling manifest link is worse than none, because it reads as marked and - verifies as nothing. - -- **`markproof init`** writes a starting `markproof.yaml`. The CLI's own docstring - had promised this command since M4 and the build did not have it. The scaffold - configures one chat probe and leaves media, UI and text marking commented out - with the reason each is optional — a config full of settings nobody chose is how - people end up running checks they cannot interpret. - -- **PDF output is reachable** (#21). `report.formats` accepts `pdf` and - `pdf-html`; both renderers existed, were tested, and could not be produced by any - invocation of the shipped tool. `report.sign_key` and `report.output_dir` are - honoured too — they were validated and then ignored. - -- **Targets can declare which Article 50 obligations bind them** (#14). A new - `applicability` block in `markproof.yaml` maps obligations — - `ai-interaction`, `synthetic-media-marking`, `synthetic-text-marking`, - `emotion-recognition`, `deepfake-labelling`, `public-interest-text` — to - whether they apply. Rules serving an obligation declared inapplicable are - reported as `SKIP` with the reason and the source of the claim, instead of - warning about a duty the target does not have. Omitting the block, or any - single obligation, leaves the rule running: silence never removes a check. - - The declaration is written into the **signed** report, which is what makes it - a claim rather than a mute switch. A green run that skipped the deep fake rule - now states, over the operator's own key, that they declared no deep fakes; - rewriting that scope afterwards breaks the signature. - - This came out of a field test against three pages generated by a real product. - Every one produced the same two warnings — no AI disclosure, no deep fake label - — and neither obligation bound any of them. A tool that says the same thing - about every target teaches its users to ignore it. - -- **Text marking is now checkable on rendered pages** (#16). `MPF-T-001` applies - to `ui` probes, scoring the region named by the new `content_selector` on the - probe. The chain from model to reader is longer for published text than for a - chat response — database, template, CDN, an editor's pass — and each step can - drop the mark. - - Only the named region is scored, never the whole document: a page is mostly - navigation, headings and footer, and the fixture sweep puts partly-marked text - at 0.586–0.657 against a `watermarked_at` of 0.70, so scoring a mixture would - land a correctly marked page in the uncertain band and fail it. Without the - selector the rule declines and says why. - -### Changed - -- **A declared-applicable obligation with no configuration warns instead of - skipping quietly** (#15). Previously, a missing `text_marking.watermark_config` - always produced `SKIP`, which sets no exit code. That is right for an operator - who never claimed to watermark and wrong for one who did: "we mark our text", - "nothing was checked" and a green build is a silent pass. It stays `WARN` - rather than becoming `FAIL` — markproof was not given the means to look, which - is not proof that no marking exists. - -- Rules now carry a required `obligation` field naming the duty they serve, and - findings carry it too. `markproof rules list` shows it. **Rulepack format:** a - rulepack without `obligation` on every rule no longer loads. - -- **Report schema:** reports may carry an `applicability` object. It is omitted - entirely when nothing was declared, so a report from a plain config is - byte-identical to what this build produced before. The field is covered by the - signature. - -## [0.1.0] - unreleased - -The first public release. Until now you could describe your Article 50 setup; from -here you can measure it against the system your users actually reach, and hand -someone else the measurement. - -### Added - - **Probe a running endpoint, not a configuration file.** `markproof run --config markproof.yaml` calls your deployed system the way a user would and judges what comes back. Three probe types cover the surfaces where marks get lost: an HTTP @@ -215,6 +140,85 @@ someone else the measurement. dependencies; every heavy component (torch, Playwright, WeasyPrint) sits behind an extra you have to ask for. +- **`MPF-M-002` verifies the C2PA manifest bound to a delivered document** (#18), + through a new `document` probe. C2PA 2.4 §A.7 binds a manifest to a document + that cannot embed one — HTML above all — by hashing the delivered bytes, and the + document points at it through an RFC 8288 `Link:` header or a + `` element. + + This is the delivery-chain regression the project exists for, in the format most + likely to suffer it: a minifier, an HTML-rewriting CDN or a template change + turns a valid provenance claim into an invalid one while the page still renders + perfectly. Nothing errors and no log line appears. + + The probe fetches the bytes the server sent rather than driving a browser — a + browser normalises markup, and only the delivered bytes are what the manifest + signs. It refuses a manifest on another origin: a provenance claim that depends + on a third party being reachable stops being checkable when they are not, and a + dangling manifest link is worse than none, because it reads as marked and + verifies as nothing. + +- **`markproof init`** writes a starting `markproof.yaml`. The CLI's own docstring + had promised this command since M4 and the build did not have it. The scaffold + configures one chat probe and leaves media, UI and text marking commented out + with the reason each is optional — a config full of settings nobody chose is how + people end up running checks they cannot interpret. + +- **PDF output is reachable** (#21). `report.formats` accepts `pdf` and + `pdf-html`; both renderers existed, were tested, and could not be produced by any + invocation of the shipped tool. `report.sign_key` and `report.output_dir` are + honoured too — they were validated and then ignored. + +- **Targets can declare which Article 50 obligations bind them** (#14). A new + `applicability` block in `markproof.yaml` maps obligations — + `ai-interaction`, `synthetic-media-marking`, `synthetic-text-marking`, + `emotion-recognition`, `deepfake-labelling`, `public-interest-text` — to + whether they apply. Rules serving an obligation declared inapplicable are + reported as `SKIP` with the reason and the source of the claim, instead of + warning about a duty the target does not have. Omitting the block, or any + single obligation, leaves the rule running: silence never removes a check. + + The declaration is written into the **signed** report, which is what makes it + a claim rather than a mute switch. A green run that skipped the deep fake rule + now states, over the operator's own key, that they declared no deep fakes; + rewriting that scope afterwards breaks the signature. + + This came out of a field test against three pages generated by a real product. + Every one produced the same two warnings — no AI disclosure, no deep fake label + — and neither obligation bound any of them. A tool that says the same thing + about every target teaches its users to ignore it. + +- **Text marking is now checkable on rendered pages** (#16). `MPF-T-001` applies + to `ui` probes, scoring the region named by the new `content_selector` on the + probe. The chain from model to reader is longer for published text than for a + chat response — database, template, CDN, an editor's pass — and each step can + drop the mark. + + Only the named region is scored, never the whole document: a page is mostly + navigation, headings and footer, and the fixture sweep puts partly-marked text + at 0.586–0.657 against a `watermarked_at` of 0.70, so scoring a mixture would + land a correctly marked page in the uncertain band and fail it. Without the + selector the rule declines and says why. + +### Changed + +- **A declared-applicable obligation with no configuration warns instead of + skipping quietly** (#15). Previously, a missing `text_marking.watermark_config` + always produced `SKIP`, which sets no exit code. That is right for an operator + who never claimed to watermark and wrong for one who did: "we mark our text", + "nothing was checked" and a green build is a silent pass. It stays `WARN` + rather than becoming `FAIL` — markproof was not given the means to look, which + is not proof that no marking exists. + +- Rules now carry a required `obligation` field naming the duty they serve, and + findings carry it too. `markproof rules list` shows it. **Rulepack format:** a + rulepack without `obligation` on every rule no longer loads. + +- **Report schema:** reports may carry an `applicability` object. It is omitted + entirely when nothing was declared, so a report from a plain config is + byte-identical to what this build produced before. The field is covered by the + signature. + ### Known limits These are boundaries of the release, not bugs, and they are stated here so nobody diff --git a/README.md b/README.md index 6ea9e44..0f2ed44 100644 --- a/README.md +++ b/README.md @@ -10,17 +10,16 @@ SPDX-License-Identifier: Apache-2.0 markproof calls your *running* AI endpoint the way a user would, and checks what actually arrives: is the image still carrying its C2PA manifest, is the text still watermarked, does the bot say it's a bot? Deterministic pass/fail, a signed evidence report, and an exit code your pipeline can gate on. ```bash -pipx install git+https://github.com/Tippel-AI/markproof +pipx install markproof markproof init # writes a starting markproof.yaml markproof run --config markproof.yaml ``` -> **Status: unreleased.** Not on PyPI yet, so install from git as above. Every check -> in the table below runs against a live endpoint and is covered by tests, but the -> rulepack format and the report schema will still change before 1.0 — treat a -> report produced today as evidence about today, not as a stable artefact. The -> first tagged release, `pipx install markproof`, and a versioned action reference -> land together ([#1](https://github.com/Tippel-AI/markproof/issues/1)). +> **Status: 0.1.0, the first release.** Every check in the table below runs against +> a live endpoint and is covered by tests. The rulepack format and the report +> schema will still change before 1.0, so pin the version if you need a report to +> stay reproducible — and treat one produced today as evidence about today rather +> than as a stable artefact. --- @@ -151,7 +150,7 @@ markproof verify-report report.json --key public.pem ## Use in CI ```yaml -- uses: Tippel-AI/markproof/action@main # a versioned tag follows the first release +- uses: Tippel-AI/markproof/action@v0.1.0 with: config: markproof.yaml extras: synthid # only if you verify text marking diff --git a/pyproject.toml b/pyproject.toml index f80e236..ba84323 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -20,7 +20,7 @@ build-backend = "hatchling.build" # --------------------------------------------------------------------------- [project] name = "markproof" -version = "0.1.0.dev0" +version = "0.1.0" description = "Deterministic CI checks for EU AI Act Article 50: verify that deployed GenAI endpoints disclose, mark (C2PA / SynthID) and prove it with a signed report." readme = "README.md" requires-python = ">=3.11" diff --git a/tests/test_readme.py b/tests/test_readme.py index baa6c57..bc06afb 100644 --- a/tests/test_readme.py +++ b/tests/test_readme.py @@ -227,30 +227,64 @@ class TestTheFrontPageMatchesReality: @staticmethod def _is_prerelease() -> bool: - from markproof import __version__ + """Read pyproject.toml, not the installed metadata. - return any(marker in __version__ for marker in ("dev", "a", "b", "rc")) + These tests compare two files in the working tree, and the release state + is decided by pyproject.toml — ``markproof.__version__`` reports whatever + was last installed, which lags a version bump until someone reinstalls. + Asking the artefact would make the guard pass or fail depending on the + state of a virtualenv rather than on the state of the repository. + """ + import tomllib + + pyproject = tomllib.loads((_README.parent / "pyproject.toml").read_text(encoding="utf-8")) + version: str = pyproject["project"]["version"] + return any(marker in version for marker in ("dev", "a", "b", "rc")) - def test_no_pypi_install_before_there_is_a_pypi_release(self) -> None: - """`pipx install markproof` must not appear while the version is a pre-release.""" - if not self._is_prerelease(): - pytest.skip("released version — the PyPI install line is correct") - for command in _commands(): - assert not re.match(r"^markproof\s*$", command), command + def test_the_install_line_matches_the_release_state(self) -> None: + """Both directions, because both have been wrong. + + Before the first release the README offered `pipx install markproof` + against a PyPI 404. The obvious guard — skip once released — would then go + quiet exactly when the opposite mistake becomes possible: shipping a + release whose front page still tells people to install from git. + """ text = _readme() - assert "pipx install markproof\n" not in text, ( - "the README offers a PyPI install, but this version is unreleased — " - "use `pipx install git+https://github.com/Tippel-AI/markproof`" - ) + offers_pypi = "pipx install markproof\n" in text + if self._is_prerelease(): + assert not offers_pypi, ( + "the README offers a PyPI install, but this version is a pre-release — " + "use `pipx install git+https://github.com/Tippel-AI/markproof`" + ) + else: + assert offers_pypi, ( + "this version is released, but the README still installs from git — " + "flip it back to `pipx install markproof`" + ) - def test_the_action_is_not_pinned_to_a_tag_that_does_not_exist(self) -> None: - if not self._is_prerelease(): - pytest.skip("released version — a version tag is expected") + def test_the_action_reference_matches_the_release_state(self) -> None: pinned = re.findall(r"Tippel-AI/markproof/action@(\S+)", _readme()) assert pinned, "the README no longer shows how to use the action" for ref in pinned: - assert not re.match(r"^v\d", ref), ( - f"the action example pins {ref}, but no release tag exists yet — use @main" + if self._is_prerelease(): + assert not re.match(r"^v\d", ref), ( + f"the action example pins {ref}, but no release tag exists yet — use @main" + ) + else: + assert re.match(r"^v\d", ref), ( + f"this version is released, but the action example points at {ref} — " + "pin the tag so a copied snippet is reproducible" + ) + + def test_the_status_callout_matches_the_release_state(self) -> None: + """The paragraph a careful reader treats as the honesty declaration.""" + text = _readme() + says_unreleased = "Status: unreleased" in text + if self._is_prerelease(): + assert says_unreleased, "a pre-release must say so where the reader looks" + else: + assert not says_unreleased, ( + "this version is released, but the status callout still says unreleased" ) def test_the_readme_links_no_repository_that_does_not_exist(self) -> None: