Native OO-LD validator driven by the specification rule catalogue

.gitattributes

@@ -0,0 +1,7 @@
+# Files copied verbatim from the oold-schema repository at a release tag must keep their exact
+# bytes: `src/oold/validation/meta/index.json` records a sha256 for each vendored meta-schema, and
+# the fixtures are refreshed by copying bytes straight out of a tag. With `core.autocrlf=true` -
+# the Windows default - git would rewrite their line endings on checkout, silently breaking those
+# checksums and making every refresh from upstream show a whole-file diff.
+src/oold/validation/meta/*/**  -text
+tests/data/oold/**             -text

.github/workflows/main.yml

@@ -57,6 +57,30 @@ jobs:
         env:
           CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
 
+  validate-schemas:
+    # Runs the OO-LD validator over the committed fixtures, offline, so a regression in the
+    # validation pipeline fails CI even when the unit tests still pass.
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out
+        uses: actions/checkout@v4
+
+      - name: Set up the environment
+        uses: ./.github/actions/setup-python-env
+
+      - name: Validate the committed OO-LD fixtures
+        run: make validate
+
+      - name: Check parity against the reference harness
+        run: |
+          # oold-schema owns the reference implementation. Comparing against its current main
+          # catches drift that the committed fixture snapshot cannot see.
+          git clone --depth 1 https://github.com/OO-LD/oold-schema.git /tmp/oold-schema
+          npm --prefix /tmp/oold-schema install --no-audit --no-fund
+          OOLD_SCHEMA_DIR=/tmp/oold-schema uv run python -m pytest \
+            tests/test_validation/test_parity_live.py -v --no-cov
+        continue-on-error: true
+
   check-docs:
     runs-on: ubuntu-latest
     steps:
@@ -66,6 +90,12 @@ jobs:
       - name: Set up the environment
         uses: ./.github/actions/setup-python-env
 
+      # A project that declares any markdown_extensions replaces Zensical's defaults
+      # rather than extending them, so a missing one silently switches an extension
+      # off without failing the build below. Catch that before it does.
+      - name: Check the restated Markdown extensions match Zensical's defaults
+        run: uv run python scripts/check_markdown_extensions.py
+
       - name: Check if documentation can be built
         run: uv run zensical build -s
 

.gitignore

@@ -224,6 +224,5 @@ benchmark_comparison.txt
 */osw_files/*
 
 # Local
-CLAUDE.md
 AGENTS.md
 .ign

.mcp.json

@@ -0,0 +1,15 @@
+{
+  "mcpServers": {
+    "oold-validation": {
+      "command": "uv",
+      "args": [
+        "run",
+        "--directory",
+        ".",
+        "python",
+        "-m",
+        "oold.validation.mcp_server"
+      ]
+    }
+  }
+}

.pre-commit-config.yaml

@@ -2,6 +2,14 @@
 # `pre-commit install` (no --hook-type flags needed).
 default_install_hook_types: [pre-commit, commit-msg]
 
+# Files copied verbatim from the oold-schema repository at a release tag: the vendored
+# meta-schemas (whose sha256 is recorded in meta/index.json) and the OO-LD test fixtures.
+# Reformatting them would break those checksums and make every future refresh from upstream
+# produce a spurious diff, so the whitespace/JSON-formatting hooks must leave them alone.
+# `check-json` still runs on them - it validates without rewriting.
+x-vendored: &vendored >-
+  ^(src/oold/validation/meta/[0-9]|tests/data/oold/)
+
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: "v6.0.0"
@@ -13,10 +21,12 @@ repos:
       - id: check-json
         exclude: ^.devcontainer/devcontainer.json
       - id: pretty-format-json
-        exclude: ^.devcontainer/devcontainer.json
+        exclude: ^(\.devcontainer/devcontainer\.json|src/oold/validation/meta/[0-9]|tests/data/oold/)
         args: [--autofix, --no-sort-keys]
       - id: end-of-file-fixer
+        exclude: *vendored
       - id: trailing-whitespace
+        exclude: *vendored
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
     rev: "v0.15.7"
@@ -33,3 +43,17 @@ repos:
     hooks:
       - id: conventional-pre-commit
         stages: [commit-msg]
+
+  # Declaring any markdown_extensions in zensical.toml replaces Zensical's defaults
+  # instead of extending them, so dropping one switches an extension off with no
+  # warning and no build failure. See scripts/check_markdown_extensions.py.
+  - repo: local
+    hooks:
+      - id: markdown-extensions
+        name: check zensical.toml restates Zensical's default Markdown extensions
+        # Goes through make so the invocation matches `make check` and CI exactly.
+        entry: make check-extensions
+        language: system
+        pass_filenames: false
+        files: ^(zensical\.toml|scripts/check_markdown_extensions\.py|Makefile)$
+        require_serial: true

CLAUDE.md

@@ -0,0 +1,86 @@
+# Working in this repository
+
+Guidance for AI agents. Human contributors want `CONTRIBUTING.md`, which this file does not repeat.
+
+## Commands
+
+```bash
+make check      # lint, type-check, dependency audit
+make test       # pytest with coverage
+make validate   # run the validator over the committed fixtures
+make docs-test  # strict docs build, fails on any warning
+
+OOLD_SCHEMA_DIR=../oold-schema uv run pytest -m parity   # compare against the reference harness
+```
+
+The parity tests skip silently without `OOLD_SCHEMA_DIR`, so a green `make test` does not mean
+parity holds. Run them explicitly when touching `src/oold/validation/`.
+
+## The validation subsystem
+
+`src/oold/validation/` is a native Python port of `oold-schema/scripts/validate.mjs`, deliberately
+not a subprocess wrapper. The reference harness is still the oracle: the parity tests assert this
+port reaches the same verdicts on the same fixtures, including check labels and triple counts.
+
+**Changes here must not change verdicts unless that is the point of the change.** Reporting,
+wording and detail payloads are free to move; a schema that passed must still pass. If parity
+drops, treat it as a defect in this port until proven otherwise. Upstream has been wrong before,
+but that is the rarer case.
+
+### Rules come from the specification, not from this code
+
+The OO-LD spec numbers its normative statements (`OOLD-RT-08f2`) and publishes them as
+`oold-rules.json`, vendored per version under `src/oold/validation/meta/<version>/`. Three
+consequences that are easy to get wrong:
+
+- **Severity is read, never written.** A check reports a problem; whether that is a failure or a
+  warning comes from the rule's `level` in the catalogue. Do not reintroduce a hardcoded
+  FAIL/WARN column. This is what lets one code base validate against several spec versions.
+- **Skip rather than guess.** A rule absent from the selected version's catalogue, or marked
+  deprecated there, is skipped with a message saying so. Older versions ship no catalogue at all
+  and skip the whole `rule.*` family. Never fall back to "check it anyway".
+- **Judge the resolved context.** Checks receive a `ContextView`, which is what terms mean after
+  remote contexts and prefixes are applied. Reading `schema["@context"]` directly reports
+  violations against correct schemas.
+
+A false positive costs far more than a missed finding, because it teaches people to ignore the
+output. When a rule is only partly decidable, check the part you are sure of.
+
+### Vendored meta-schemas are byte-exact
+
+`src/oold/validation/meta/<version>/` holds verbatim copies from oold-schema release tags, and
+`index.json` records a sha256 of each. They are therefore not ordinary source files:
+
+- never reformat them, and never let a formatting hook touch them (`.pre-commit-config.yaml`
+  excludes these paths, `.gitattributes` marks them `-text`);
+- they must be LF. A CRLF copy hashes differently, which passes on Windows and fails on Linux.
+  This has happened; `test_the_vendored_files_are_stored_with_unix_line_endings` now guards it;
+- to add a version, follow `src/oold/validation/meta/README.md` and recompute the digests.
+
+## This repo and oold-schema are decoupled on purpose
+
+They release on separate schedules, so neither pipeline waits on the other:
+
+- `coverage.rules` **warns** when a rule has no check, rather than failing. A spec that has moved
+  ahead must not break this build.
+- Adding a check for a rule is described in `CONTRIBUTING.md#translating-a-specification-rule`.
+  oold-schema's `make check` prints that link when the catalogue changes.
+
+Do not add a check for a rule that is not in any vendored catalogue. Vendor the version first.
+
+## Check ids are a public interface
+
+Check ids (`lint.container`, `roundtrip.instance`, `rule.id-fragment`) appear in reports, CI logs
+and, before long, in suppression comments. Renaming one silently breaks whatever depended on it,
+and unlike rule ids there is no guard. Treat a rename as a breaking change: say so in the commit,
+and prefer adding a new id over repurposing an existing one.
+
+## Conventions
+
+- Conventional Commits; releases are automated by python-semantic-release, so the type prefix
+  decides the version bump.
+- No AI attribution or co-author trailers in commits or PR descriptions.
+- In prose and comments, use regular dashes rather than em or en dashes.
+- Do not create scratch files inside this repository or in `../oold-schema`. To see what a file
+  looks like on a clean checkout, read git state (`git show :path`, `git check-attr`) instead of
+  deleting and restoring it.

CONTRIBUTING.md

@@ -70,6 +70,98 @@ uv run zensical serve
 uv run zensical build -s
 ```
 
+## Translating a specification rule
+
+The OO-LD specification numbers each of its normative statements (`OOLD-RT-08f2`, ...) and
+publishes them as `oold-rules.json`, which this repository vendors per meta-schema version. When
+oold-schema adds a rule, its `make check` prints a pointer back to this section, because a new
+rule is the moment the validator falls behind the specification.
+
+Not every rule becomes a check, so start by reading it:
+
+```bash
+uv run oold rules explain OOLD-RT-08f2
+uv run oold rules list --unchecked        # everything still waiting for a check
+```
+
+`applies_to` decides whether there is anything to do here:
+
+| `applies_to` | Meaning | Action |
+| --- | --- | --- |
+| `document` + `machine_checkable: true` | Decidable by looking at a schema or instance | Add a check, as below |
+| `document`, not `machine_checkable` | Binds documents but needs human judgement | Nothing; it stays listed as unchecked |
+| `implementation` | Constrains what the library *does*, which no validator can see | A test against the library, not a `CheckInfo` |
+| `advisory` | Guidance only | Nothing |
+
+To add a check, write the predicate and append a `CheckInfo` to `CHECKS` in
+`src/oold/validation/check_registry.py`, alongside the existing entries:
+
+```python
+def _missing_id(schema: dict[str, Any], context: ContextView) -> list[str]:
+    if not schema.get("$id"):
+        return ["schema declares no $id, so it has no global identifier"]
+    return []
+
+
+CHECKS = (
+    CheckInfo("rule.id", "a schema has a $id", rule="OOLD-VER-3b96", per_version=True, run=_missing_id),
+    ...
+)
+```
+
+The check id, a short description, the rule it enforces, and the predicate are what matter here.
+Use a `rule.*` check id: `lint.*`, `schema.*` and `roundtrip.*` are the checks carried over from
+the reference harness, and several of them already cite a rule.
+
+The predicate returns a list of problem strings, empty when the schema conforms. Four things
+about it are easy to get wrong:
+
+- **Judge the resolved context, not the literal one.** `ContextView` is what the term definitions
+  mean after remote contexts and prefixes are applied. Reading `schema["@context"]` directly will
+  report violations for schemas that are perfectly correct.
+- **Do not set a severity.** It comes from the rule's own `level` in the catalogue, so a `MUST`
+  fails and a `SHOULD` warns without the check deciding anything. That is what lets one code base
+  validate against several specification versions.
+- **Prefer skipping to guessing.** A rule absent from the selected version's catalogue is skipped
+  automatically. If a rule is only partially decidable, check the part you are sure of; a false
+  positive costs far more than a missed finding, because it teaches people to ignore the output.
+- **Leave `predates_catalog` alone.** It defaults to `False`, which is right for a new rule.
+  Setting it `True` claims the requirement is older than the catalogue itself, and makes the check
+  run against 0.7.0 and 0.8.0, which ship no catalogue and never stated your rule. It is reserved
+  for the four checks carried over from the reference harness. This one fails silently in the
+  wrong direction: nothing breaks, the old versions are simply judged by a rule that postdates
+  them. `test_predates_catalog_is_exactly_what_runs_under_a_pre_catalogue_version` is the guard.
+
+### What the check owes in tests
+
+Unit tests in `tests/test_validation/test_check_registry.py` are the obligation: one schema that
+conforms and one that violates. A check that only ever sees valid input is not known to fire at
+all. A `rule.*` predicate is a pure function of `(schema, ContextView)`, so a test constructs the
+`ContextView` directly and there is nothing else to arrange.
+
+A fixture under `tests/data/oold/broken/` is **not** expected of a `rule.*` check, and none of the
+existing ones has one. Those fixtures exist for checks whose verdict depends on machinery a unit
+test cannot stub - `schema.meta` compiling a meta-schema, `roundtrip.generated` making a real RDF
+round trip, `context.predicates` running a real JSON-LD expansion. Add one only if your check is of
+that kind.
+
+There is a third obligation neither of those covers, and it is the one that has actually gone
+missing: **at least one schema in `tests/data/oold/` must exercise the predicate through the
+pipeline.** Isolated unit tests prove the predicate is correct, never that it is reached with a
+correctly resolved `ContextView`. If the corpus gives your check nothing to judge, it passes
+everywhere and proves nothing; extend a fixture until it does. `remote_context/Leaf.schema.json`
+carries a `required` for exactly this reason.
+
+Finally, confirm the gap actually closed:
+
+```bash
+uv run oold rules list --unchecked        # the rule should be gone from this list
+make validate                             # coverage.rules reports one fewer unchecked rule
+```
+
+`coverage.rules` warns rather than fails, deliberately: the specification and this validator
+release on separate schedules, and a spec that has moved ahead should not break this build.
+
 ## Commit messages (Conventional Commits)
 
 This project uses [Conventional Commits](https://www.conventionalcommits.org/).

Makefile

@@ -14,12 +14,24 @@ check: ## Run code quality tools.
 	@uv run ty check
 	@echo "🚀 Checking for obsolete dependencies: Running deptry"
 	@uv run deptry src
+	@echo "🚀 Checking docs build config: Markdown extensions still match Zensical's defaults"
+	@uv run python scripts/check_markdown_extensions.py
 
 .PHONY: test
 test: ## Test the code with pytest
 	@echo "🚀 Testing code: Running pytest"
 	@uv run python -m pytest --cov --cov-config=pyproject.toml --cov-report=xml
 
+.PHONY: validate
+validate: ## Validate the committed OO-LD fixtures with the built-in validator
+	@echo "🚀 Validating OO-LD schemas: oold validate"
+	@uv run oold validate tests/data/oold --offline
+	@uv run oold compliance tests/data/oold/compliance --offline
+
+.PHONY: check-extensions
+check-extensions: ## Check zensical.toml still restates Zensical's default Markdown extensions
+	@uv run python scripts/check_markdown_extensions.py
+
 .PHONY: benchmark
 benchmark: ## Run performance benchmarks with pytest-benchmark
 	@echo "🚀 Running benchmarks: pytest-benchmark"

README.md

@@ -238,6 +238,29 @@ loaded = MyModel["ex:foo"]  # resolves via registered backend
 
 Custom backends implement the `Backend` interface (`resolve_iris`, `store_json_dicts`).
 
+### Validation
+
+Check that an OO-LD schema is well formed, and that its `@context` actually carries every
+declared property into RDF. A property declared in `properties` but missing from `@context` is
+neither a JSON Schema error nor a JSON-LD error - it just quietly disappears, and the data loses
+meaning.
+
+```bash
+pip install "oold[validation]"
+
+oold validate Person.schema.json          # one schema
+oold validate ./schemas/                  # a whole directory
+oold validate-instance doc.instance.json  # a document against the schema it names
+```
+
+Exit code is 0 only when every check passes, so it drops straight into CI. The same pipeline is
+available as a Python API and as an MCP server (`oold[mcp]`).
+
+It is a native port of the reference harness in
+[oold-schema](https://github.com/OO-LD/oold-schema), verified to agree with it on verdicts, and
+it validates against versioned meta-schemas - a released version, the unreleased upstream state,
+or several at once. See [docs/how-to/validation.md](docs/how-to/validation.md).
+
 ## Development
 
 This project uses [uv](https://docs.astral.sh/uv/) and `make`. Clone and set up:

docs/architecture.md

@@ -106,6 +106,18 @@ Backends are registered per IRI prefix via `set_resolver` / `set_backend`, so mu
 
 `oold.ui` contains optional integrations for [Panel](https://panel.holoviz.org/), [NiceGUI](https://nicegui.io/), and [Jupyter anywidget](https://anywidget.dev/). These are not installed by default.
 
+### Validation Layer (optional)
+
+`oold.validation` checks that a schema is well formed and that its `@context` actually carries
+every declared property into RDF. It is a native port of the reference harness in
+[oold-schema](https://github.com/OO-LD/oold-schema), and reuses `pyld` from the serialization
+layer, so the JSON-LD half of it adds no dependencies.
+
+One pipeline backs three surfaces - the library API, the `oold validate` CLI, and an MCP server -
+so there is a single implementation to keep correct. The meta-schemas it validates against are
+versioned: a hand-curated history ships in the package, and a schema can be checked against
+several versions in one run. See [Validation](how-to/validation.md).
+
 ---
 
 ## Data flow

docs/how-to/index.md

@@ -34,6 +34,12 @@ New here? Start with [Get Started](../get-started.md) to run your first end-to-e
 
     Serialize models to JSON-LD, load them into RDFLib, and query with SPARQL - context injection, cross-object links, and round-trip fidelity.
 
+- :material-check-decagram:{ .lg .middle } **[Validation](validation.md)**
+
+    ---
+
+    Check that a schema is well formed and that its `@context` carries every property into RDF - the CLI, the MCP server, meta-schema version selection, and what each check means.
+
 - :material-tune:{ .lg .middle } **[BaseController](controller.md)**
 
     ---