Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
178 changes: 91 additions & 87 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,18 @@ explicitly: the **rulepack format** — what a rulepack file may contain — and

## [Unreleased]

_Nothing yet._

## [0.1.0] - 2026-09-01

The first public release. Until now you could describe your Article 50 setup; from
here you can measure it against the system your users actually reach, and hand
someone else the measurement.

Everything below is in this release. There was no earlier one: the entries that
lived under "Unreleased" during development are folded in here rather than left
implying a version nobody could install.

### Security

- **The signed report now binds the rules and the endpoint it judged.** A report
Expand All @@ -33,93 +45,6 @@ explicitly: the **rulepack format** — what a rulepack file may contain — and

### Added

- **`MPF-M-002` verifies the C2PA manifest bound to a delivered document** (#18),
through a new `document` probe. C2PA 2.4 §A.7 binds a manifest to a document
that cannot embed one — HTML above all — by hashing the delivered bytes, and the
document points at it through an RFC 8288 `Link:` header or a
`<link rel="c2pa-manifest">` element.

This is the delivery-chain regression the project exists for, in the format most
likely to suffer it: a minifier, an HTML-rewriting CDN or a template change
turns a valid provenance claim into an invalid one while the page still renders
perfectly. Nothing errors and no log line appears.

The probe fetches the bytes the server sent rather than driving a browser — a
browser normalises markup, and only the delivered bytes are what the manifest
signs. It refuses a manifest on another origin: a provenance claim that depends
on a third party being reachable stops being checkable when they are not, and a
dangling manifest link is worse than none, because it reads as marked and
verifies as nothing.

- **`markproof init`** writes a starting `markproof.yaml`. The CLI's own docstring
had promised this command since M4 and the build did not have it. The scaffold
configures one chat probe and leaves media, UI and text marking commented out
with the reason each is optional — a config full of settings nobody chose is how
people end up running checks they cannot interpret.

- **PDF output is reachable** (#21). `report.formats` accepts `pdf` and
`pdf-html`; both renderers existed, were tested, and could not be produced by any
invocation of the shipped tool. `report.sign_key` and `report.output_dir` are
honoured too — they were validated and then ignored.

- **Targets can declare which Article 50 obligations bind them** (#14). A new
`applicability` block in `markproof.yaml` maps obligations —
`ai-interaction`, `synthetic-media-marking`, `synthetic-text-marking`,
`emotion-recognition`, `deepfake-labelling`, `public-interest-text` — to
whether they apply. Rules serving an obligation declared inapplicable are
reported as `SKIP` with the reason and the source of the claim, instead of
warning about a duty the target does not have. Omitting the block, or any
single obligation, leaves the rule running: silence never removes a check.

The declaration is written into the **signed** report, which is what makes it
a claim rather than a mute switch. A green run that skipped the deep fake rule
now states, over the operator's own key, that they declared no deep fakes;
rewriting that scope afterwards breaks the signature.

This came out of a field test against three pages generated by a real product.
Every one produced the same two warnings — no AI disclosure, no deep fake label
— and neither obligation bound any of them. A tool that says the same thing
about every target teaches its users to ignore it.

- **Text marking is now checkable on rendered pages** (#16). `MPF-T-001` applies
to `ui` probes, scoring the region named by the new `content_selector` on the
probe. The chain from model to reader is longer for published text than for a
chat response — database, template, CDN, an editor's pass — and each step can
drop the mark.

Only the named region is scored, never the whole document: a page is mostly
navigation, headings and footer, and the fixture sweep puts partly-marked text
at 0.586–0.657 against a `watermarked_at` of 0.70, so scoring a mixture would
land a correctly marked page in the uncertain band and fail it. Without the
selector the rule declines and says why.

### Changed

- **A declared-applicable obligation with no configuration warns instead of
skipping quietly** (#15). Previously, a missing `text_marking.watermark_config`
always produced `SKIP`, which sets no exit code. That is right for an operator
who never claimed to watermark and wrong for one who did: "we mark our text",
"nothing was checked" and a green build is a silent pass. It stays `WARN`
rather than becoming `FAIL` — markproof was not given the means to look, which
is not proof that no marking exists.

- Rules now carry a required `obligation` field naming the duty they serve, and
findings carry it too. `markproof rules list` shows it. **Rulepack format:** a
rulepack without `obligation` on every rule no longer loads.

- **Report schema:** reports may carry an `applicability` object. It is omitted
entirely when nothing was declared, so a report from a plain config is
byte-identical to what this build produced before. The field is covered by the
signature.

## [0.1.0] - unreleased

The first public release. Until now you could describe your Article 50 setup; from
here you can measure it against the system your users actually reach, and hand
someone else the measurement.

### Added

- **Probe a running endpoint, not a configuration file.** `markproof run --config
markproof.yaml` calls your deployed system the way a user would and judges what
comes back. Three probe types cover the surfaces where marks get lost: an HTTP
Expand Down Expand Up @@ -215,6 +140,85 @@ someone else the measurement.
dependencies; every heavy component (torch, Playwright, WeasyPrint) sits behind an
extra you have to ask for.

- **`MPF-M-002` verifies the C2PA manifest bound to a delivered document** (#18),
through a new `document` probe. C2PA 2.4 §A.7 binds a manifest to a document
that cannot embed one — HTML above all — by hashing the delivered bytes, and the
document points at it through an RFC 8288 `Link:` header or a
`<link rel="c2pa-manifest">` element.

This is the delivery-chain regression the project exists for, in the format most
likely to suffer it: a minifier, an HTML-rewriting CDN or a template change
turns a valid provenance claim into an invalid one while the page still renders
perfectly. Nothing errors and no log line appears.

The probe fetches the bytes the server sent rather than driving a browser — a
browser normalises markup, and only the delivered bytes are what the manifest
signs. It refuses a manifest on another origin: a provenance claim that depends
on a third party being reachable stops being checkable when they are not, and a
dangling manifest link is worse than none, because it reads as marked and
verifies as nothing.

- **`markproof init`** writes a starting `markproof.yaml`. The CLI's own docstring
had promised this command since M4 and the build did not have it. The scaffold
configures one chat probe and leaves media, UI and text marking commented out
with the reason each is optional — a config full of settings nobody chose is how
people end up running checks they cannot interpret.

- **PDF output is reachable** (#21). `report.formats` accepts `pdf` and
`pdf-html`; both renderers existed, were tested, and could not be produced by any
invocation of the shipped tool. `report.sign_key` and `report.output_dir` are
honoured too — they were validated and then ignored.

- **Targets can declare which Article 50 obligations bind them** (#14). A new
`applicability` block in `markproof.yaml` maps obligations —
`ai-interaction`, `synthetic-media-marking`, `synthetic-text-marking`,
`emotion-recognition`, `deepfake-labelling`, `public-interest-text` — to
whether they apply. Rules serving an obligation declared inapplicable are
reported as `SKIP` with the reason and the source of the claim, instead of
warning about a duty the target does not have. Omitting the block, or any
single obligation, leaves the rule running: silence never removes a check.

The declaration is written into the **signed** report, which is what makes it
a claim rather than a mute switch. A green run that skipped the deep fake rule
now states, over the operator's own key, that they declared no deep fakes;
rewriting that scope afterwards breaks the signature.

This came out of a field test against three pages generated by a real product.
Every one produced the same two warnings — no AI disclosure, no deep fake label
— and neither obligation bound any of them. A tool that says the same thing
about every target teaches its users to ignore it.

- **Text marking is now checkable on rendered pages** (#16). `MPF-T-001` applies
to `ui` probes, scoring the region named by the new `content_selector` on the
probe. The chain from model to reader is longer for published text than for a
chat response — database, template, CDN, an editor's pass — and each step can
drop the mark.

Only the named region is scored, never the whole document: a page is mostly
navigation, headings and footer, and the fixture sweep puts partly-marked text
at 0.586–0.657 against a `watermarked_at` of 0.70, so scoring a mixture would
land a correctly marked page in the uncertain band and fail it. Without the
selector the rule declines and says why.

### Changed

- **A declared-applicable obligation with no configuration warns instead of
skipping quietly** (#15). Previously, a missing `text_marking.watermark_config`
always produced `SKIP`, which sets no exit code. That is right for an operator
who never claimed to watermark and wrong for one who did: "we mark our text",
"nothing was checked" and a green build is a silent pass. It stays `WARN`
rather than becoming `FAIL` — markproof was not given the means to look, which
is not proof that no marking exists.

- Rules now carry a required `obligation` field naming the duty they serve, and
findings carry it too. `markproof rules list` shows it. **Rulepack format:** a
rulepack without `obligation` on every rule no longer loads.

- **Report schema:** reports may carry an `applicability` object. It is omitted
entirely when nothing was declared, so a report from a plain config is
byte-identical to what this build produced before. The field is covered by the
signature.

### Known limits

These are boundaries of the release, not bugs, and they are stated here so nobody
Expand Down
15 changes: 7 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,16 @@ SPDX-License-Identifier: Apache-2.0
markproof calls your *running* AI endpoint the way a user would, and checks what actually arrives: is the image still carrying its C2PA manifest, is the text still watermarked, does the bot say it's a bot? Deterministic pass/fail, a signed evidence report, and an exit code your pipeline can gate on.

```bash
pipx install git+https://github.com/Tippel-AI/markproof
pipx install markproof
markproof init # writes a starting markproof.yaml
markproof run --config markproof.yaml
```

> **Status: unreleased.** Not on PyPI yet, so install from git as above. Every check
> in the table below runs against a live endpoint and is covered by tests, but the
> rulepack format and the report schema will still change before 1.0 — treat a
> report produced today as evidence about today, not as a stable artefact. The
> first tagged release, `pipx install markproof`, and a versioned action reference
> land together ([#1](https://github.com/Tippel-AI/markproof/issues/1)).
> **Status: 0.1.0, the first release.** Every check in the table below runs against
> a live endpoint and is covered by tests. The rulepack format and the report
> schema will still change before 1.0, so pin the version if you need a report to
> stay reproducible — and treat one produced today as evidence about today rather
> than as a stable artefact.

---

Expand Down Expand Up @@ -151,7 +150,7 @@ markproof verify-report report.json --key public.pem
## Use in CI

```yaml
- uses: Tippel-AI/markproof/action@main # a versioned tag follows the first release
- uses: Tippel-AI/markproof/action@v0.1.0
with:
config: markproof.yaml
extras: synthid # only if you verify text marking
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ build-backend = "hatchling.build"
# ---------------------------------------------------------------------------
[project]
name = "markproof"
version = "0.1.0.dev0"
version = "0.1.0"
description = "Deterministic CI checks for EU AI Act Article 50: verify that deployed GenAI endpoints disclose, mark (C2PA / SynthID) and prove it with a signed report."
readme = "README.md"
requires-python = ">=3.11"
Expand Down
68 changes: 51 additions & 17 deletions tests/test_readme.py
Original file line number Diff line number Diff line change
Expand Up @@ -227,30 +227,64 @@ class TestTheFrontPageMatchesReality:

@staticmethod
def _is_prerelease() -> bool:
from markproof import __version__
"""Read pyproject.toml, not the installed metadata.

return any(marker in __version__ for marker in ("dev", "a", "b", "rc"))
These tests compare two files in the working tree, and the release state
is decided by pyproject.toml — ``markproof.__version__`` reports whatever
was last installed, which lags a version bump until someone reinstalls.
Asking the artefact would make the guard pass or fail depending on the
state of a virtualenv rather than on the state of the repository.
"""
import tomllib

pyproject = tomllib.loads((_README.parent / "pyproject.toml").read_text(encoding="utf-8"))
version: str = pyproject["project"]["version"]
return any(marker in version for marker in ("dev", "a", "b", "rc"))

def test_no_pypi_install_before_there_is_a_pypi_release(self) -> None:
"""`pipx install markproof` must not appear while the version is a pre-release."""
if not self._is_prerelease():
pytest.skip("released version — the PyPI install line is correct")
for command in _commands():
assert not re.match(r"^markproof\s*$", command), command
def test_the_install_line_matches_the_release_state(self) -> None:
"""Both directions, because both have been wrong.

Before the first release the README offered `pipx install markproof`
against a PyPI 404. The obvious guard — skip once released — would then go
quiet exactly when the opposite mistake becomes possible: shipping a
release whose front page still tells people to install from git.
"""
text = _readme()
assert "pipx install markproof\n" not in text, (
"the README offers a PyPI install, but this version is unreleased — "
"use `pipx install git+https://github.com/Tippel-AI/markproof`"
)
offers_pypi = "pipx install markproof\n" in text
if self._is_prerelease():
assert not offers_pypi, (
"the README offers a PyPI install, but this version is a pre-release — "
"use `pipx install git+https://github.com/Tippel-AI/markproof`"
)
else:
assert offers_pypi, (
"this version is released, but the README still installs from git — "
"flip it back to `pipx install markproof`"
)

def test_the_action_is_not_pinned_to_a_tag_that_does_not_exist(self) -> None:
if not self._is_prerelease():
pytest.skip("released version — a version tag is expected")
def test_the_action_reference_matches_the_release_state(self) -> None:
pinned = re.findall(r"Tippel-AI/markproof/action@(\S+)", _readme())
assert pinned, "the README no longer shows how to use the action"
for ref in pinned:
assert not re.match(r"^v\d", ref), (
f"the action example pins {ref}, but no release tag exists yet — use @main"
if self._is_prerelease():
assert not re.match(r"^v\d", ref), (
f"the action example pins {ref}, but no release tag exists yet — use @main"
)
else:
assert re.match(r"^v\d", ref), (
f"this version is released, but the action example points at {ref} — "
"pin the tag so a copied snippet is reproducible"
)

def test_the_status_callout_matches_the_release_state(self) -> None:
"""The paragraph a careful reader treats as the honesty declaration."""
text = _readme()
says_unreleased = "Status: unreleased" in text
if self._is_prerelease():
assert says_unreleased, "a pre-release must say so where the reader looks"
else:
assert not says_unreleased, (
"this version is released, but the status callout still says unreleased"
)

def test_the_readme_links_no_repository_that_does_not_exist(self) -> None:
Expand Down
Loading