Native OO-LD validator driven by the specification rule catalogue - #114
Open
LukasOro wants to merge 29 commits into
Open
Native OO-LD validator driven by the specification rule catalogue#114LukasOro wants to merge 29 commits into
LukasOro wants to merge 29 commits into
Conversation
- Ports scripts/validate.mjs, pattern_lint.mjs and schema_to_frame.mjs into src/oold/validation/ - Backs the library API, an `oold validate` CLI (aliased `oold-validate`), and an MCP server - Covers meta-schema, $ref composition, @context lint, RDF round-trip, remote-context and oneOf/anyOf checks, plus the compliance-suite and vocabulary-coverage cross-checks - Meta-schemas are versioned under src/oold/validation/meta/, seeded from tag v0.7.0 - Parity is checked in tests/test_validation/test_parity_live.py against 98 pinned ajv-format outcomes
- Vendors v0.8.0 into the meta-schema version history; `latest` now resolves to it, 0.7.0 stays - Refreshes the fixture slice from the same tag - Excludes vendored meta-schemas and fixtures from pre-commit's pretty-format-json and whitespace fixers, which broke their recorded sha256 checksums - Adds .gitattributes marking vendored files -text to stop core.autocrlf rewriting line endings - Splits test_selection_expands_and_deduplicates into three tests pinning selector order, `all` version order, and explicit order
- MetaBundle loads the optional oold-rules.json, exposing rule(), has_rules and checkable_rules() - Check gains a `rule` field, surfaced in the CLI (--verbose), the JSON report and MCP payloads - CHECK_RULES maps checks to rules centrally in _Run.add, covering the four checks that enforce a single identifiable requirement - New `oold rules list|explain` commands (--area, --unchecked) and a list_oold_rules MCP tool - New coverage.rules check reports checkable rules with no check; it only ever warns
- MUST-level: rule.id (OOLD-VER-001), rule.id-fragment (OOLD-CMP-005), rule.range-ref (OOLD-EXT-005), rule.instance-type (OOLD-INS-002), rule.free-text-iri (OOLD-INS-009), rule.closed-object (OOLD-INS-005) - SHOULD-level: rule.version (OOLD-VER-002), rule.id-alias (OOLD-INS-007), rule.dialect (OOLD-EXT-002), rule.processing-mode (OOLD-EXT-001) - Checks are declared in a registry and export their own check-id-to-rule mapping - Checks judge the resolved context rather than the schema's literal @context - Takes coverage.rules from 21 of 25 checkable rules unchecked to 11
- Vendors meta-schema 1.0.0-rc.1, the first version to ship oold-rules.json - Severity now comes from the catalogue; the FAIL/WARN column is removed from the check registry - A rule absent from the selected version, or marked deprecated, is skipped with the reason given - Fixes _version_key so pre-releases sort before their release (1.0.0-rc.1 before 1.0.0) - Fixtures refreshed from the v1.0.0-rc.1 tag
- Seven hand-authored fixtures were CRLF while every fixture copied from an oold-schema tag is LF - `.gitattributes` marks the fixture directory -text, so mixed endings looked deliberate - Whitespace only; each file re-parsed and compared to its previous value after conversion - tests/data/oold/README.md refreshed by copying from upstream to avoid a line-ending diff
- Linked from oold-schema's `make check` whenever the catalogue gains a rule - Covers `applies_to`, severity coming from the catalogue rather than the check, and judging the resolved context instead of the literal @context
- Documents that verdicts are pinned by parity with the reference harness - Documents that severity is read from the specification catalogue, not decided by a check - Documents that vendored meta-schemas are checksummed bytes no formatter or line-ending conversion may touch - AGENTS.md stays ignored
- Rule ids answer which requirement was violated and are permanent; check ids answer which check found it and follow the implementation - Replaces an earlier "frozen inventory" proposal that hand-synced a second file of ids - Verified against what the validator actually emits, and holds function references rather than path strings - Kept out of docs/, since an unlisted page there is still built and published
- Deletes RULE_CHECKS, RULE_CHECK_MAP and CHECK_RULES in favor of one registry - CheckInfo absorbs RuleCheck by gaining an optional `run` predicate - rule_checks.py folds into check_registry.py - Drops the emitting site from `oold checks explain`
- Gating is by presence in the vendored catalogue; a new rule's check needs no backward-compatibility code - Four phase checks carry a rule id but were not gated by anything - New `requires_rule` flag extends catalogue gating to any check, defaulting to off - `since` cannot serve this purpose: all 34 rules carry since=1.0.0-rc.1, when the catalogue was minted - A fourth drift test pins the promise
- `requires_rule` is insufficient for a legacy check whose rule is later superseded - Replaced with `predates_catalog`, which asks what a pre-catalogue version cannot answer - A rewording changes nothing, a data-driven change (0.8.0's no-coercion rule) is free, only bespoke detection needs a new predicate - Marginal cost of a changed rule is one registry line
- Lists the full set of check ids; the previous registry missed six ids the validator actually emits - Fixes the verification command: grepping `rule_checks` also matched `run_rule_checks()` - Records the `compliance.<kind>` decision as a single normalised `compliance.*` family entry - Notes `lint.iri-format` has never been observed to fire; drift test 2 should fail on it
- RuleCheck, RULE_CHECKS, RULE_CHECK_MAP and CHECK_RULES collapse into `CheckInfo` - rule_checks.py folds into check_registry.py - Purely structural: verdicts, messages and report shape unchanged, confirmed byte-for-byte and by the parity suite - cli.py, mcp_server.py and two test modules move to importing from the registry
- Every emittable id now has a registry entry, including rule.checks, meta.self-check, compliance.suite and a single compliance.* family entry - `predates_catalog` decides check behaviour against versions that ship no rule catalogue - Adds the missing lint.iri-format fixture so the check is actually exercised - Fixes drift test 4, which previously counted a gated SKIP as a run - Verdicts unchanged on all three tracked versions, confirmed by diff and the parity suite
- Leaf.schema.json now requires `name`, defined only by the remote Thing context via ../Thing.schema.json - Leaf's own inline context defines just `nickname`, so a literal-@context check would misreport `name` as undefined - No new test needed; test_a_context_chain_leaving_the_directory_resolves already asserts the whole report passes
- Pre-existing drift, not introduced here: the line was exactly 120 characters, which failed `make check` on a clean checkout
- MetaBundle.self_check now validates the catalogue against the new oold-rules.schema.json, vendored alongside it - A truncated or malformed catalogue previously looked like a version stating fewer rules, with checks skipping silently - Loading stays lenient; the loss is now reported via meta.self-check instead of failing validation - Fixture slice provenance moves from README prose into index.json; `fixtures.tag` is compared against the newest tracked version by a test - Re-vendors rc.1's catalogue, which gained only the $schema line upstream
- meta/README.md now lists all five files to copy, including the catalogue and its schema added in 1.0.0-rc.1 - CONTRIBUTING.md now documents `predates_catalog` and its default - Clarifies that a new rule.* check needs no broken-fixture test but does need a corpus schema that exercises its predicate - Fixture README no longer states the slice's tag, now recorded in index.json
- Rule ids move to a minted hex suffix (e.g. OOLD-RT-002 -> OOLD-RT-08f2); all 14 registry-cited ids remapped - `checkable` renamed to `machine_checkable` throughout, including checkable_rules() and the CLI label - Four new checks: rule.uuid (OOLD-VER-edb9), rule.multilang-default (OOLD-EXT-dd76), rule.base-alignment (OOLD-CMP-53bf), rule.scoped-context (OOLD-CMP-5266) - rule.scoped-context only flags an embed arriving by $ref to another document, not a self-reference - Verdicts unchanged; parity holds at 6/6
- rule.instance-type and rule.closed-object previously passed without ever running, found via line coverage over a full corpus run - New fixtures make each fail exactly the check it targets; closed_object also trips roundtrip.generated - base_uri_misaligned is registered in the fixture table alongside them, having been added earlier without being listed - rule.id-alias is left with a known gap: no fixture yet exercises its non-violating path
- Declaring any markdown_extensions replaces Zensical's default set rather than extending it, silently dropping fifteen defaults - Admonitions in docs/how-to/backends.md, codegen.md, object-graph-mapping.md and rdf-export.md rendered as literal text - zensical.toml now restates all 22 defaults of the installed 0.0.45 - scripts/check_markdown_extensions.py compares that restatement against the installed Zensical and fails on drift; wired into `make check`, a pre-commit hook, and docs CI - pymdownx.smartsymbols stays enabled here, unlike oold-schema, since its trigger sequences only appear inside Mermaid fences
- rule.multilang-shape (OOLD-EXT-ef09), rule.dialect-version (OOLD-EXT-af50), rule.context-array-order (OOLD-CMP-e4a3), rule.versioned-id (OOLD-VER-534a) - rule.context-array-order and rule.versioned-id read the literal @context and the rule's summary respectively, both documented exceptions - Three fixtures added so rule.versioned-id is actually reached, not just guarded - Not implemented: OOLD-CMP-a05a, OOLD-INS-9416, OOLD-RT-d376, OOLD-CMP-f3c7, each with a reason recorded in the code - Verdicts unchanged; parity holds at 6/6
- Refreshes meta/1.0.0-rc.1/oold-rules.json from oold-schema: 40 rules to 43 - Adds OOLD-VER-befc, OOLD-VER-4261 and OOLD-EXT-1f92, split out of lead-in lists upstream - `oold rules list --unchecked` goes from 7 to 10 - rules_source now records feat/rule-list-scope (oold-schema PR #124, unmerged) - rule.versioned-id's docstring corrected: it is now explicitly the umbrella over OOLD-VER-befc and OOLD-VER-4261
- rules_source now records the pull request number alongside the repository, branch and pre-merge commit - A rebased branch orphans a recorded commit; the entry it replaces pointed at exactly such an orphan on feat/rule-catalog-rc1 - Adds a `merged` slot to fill in once #124 lands
- oold-schema splits the dialect meta-schema into a wrapper plus oold-meta-schema-base.json - `files` in meta/index.json is now per-source via `meta_files(source)`, so only the remote bundle carries the new base file - Restores parity, which had been failing 4 of 6 because the wrapper's $ref could not resolve - declared_keywords() now collects x-oold-* across every document in the bundle, restoring all 26 keywords and picking up x-oold-sssom - Two meta_store tests updated to serve the four-file remote list
- rule.context-reflects-refs (OOLD-CMP-b926) and rule.branch-context-conflict (OOLD-CMP-1d7e); unenforced count 10 to 8 - Both read the authored @context rather than the resolved view, the same exception rule.context-array-order already takes - rule.branch-context-conflict is narrowed via `entries`, distinguishing an authored override (dict) from a reflected conflict (string) - OOLD-INS-1df7 deliberately not implemented; it duplicates what rule.free-text-iri already checks - Both checks produce zero findings on the corpus and needed a broken fixture each to be reached
- rule.narrow-only enforces OOLD-CMP-f3c7; unenforced count 8 to 7 - CheckInfo gains `run_resolved`, letting a check receive the dereferenced schema instead of the authored one - Ten comparable keywords are checked per member: numeric bounds, length/item/property bounds, multipleOf, enum, const, type, uniqueItems, additionalProperties - `pattern` and `required` are deliberately excluded, with reasons recorded in the code - Zero findings across the corpus; parity holds at 6/6
- Vendor 1.0.0-rc.2 as its own entry; rc.1 keeps its pre-release copy - Declare its own four-file set: the dialect split into wrapper plus base - Close rc.1's rules_source.merged; refresh fixture slice and fixtures.tag - Unchecked rules 7 -> 15 of 40: 4 carried over, 11 newly released - Fix a UI cross-reference test that asserted on an undefined keyword
Contributor
Release previewMerging this PR would release v0.17.0 (current: Changelog preview (truncated)## v0.17.0 (2026-08-15)
### Bug Fixes
- **docs**: Restate and guard Zensical's default Markdown extensions
([`ebc3907`](https://github.com/OO-LD/oold-python/commit/ebc3907cae24a685b78edcd856ca514331148a95))
### Chores
- License fix
([`daa8d8c`](https://github.com/OO-LD/oold-python/commit/daa8d8cfa325a3745c851c845b9fae7b5e51a0cc))
- **validation**: Vendor the 43-rule catalogue
([`cbcf95a`](https://github.com/OO-LD/oold-python/commit/cbcf95a3042c41078c7b670de4ce77b21db36a31))
### Code Style
- Apply ruff-format to the check-registry drift test
([`ca11109`](https://github.com/OO-LD/oold-python/commit/ca1110907aa194b32458cfb90599c4b8a51609b5))
- **tests**: Store the hand-written fixtures with LF line endings
([`8c965d6`](https://github.com/OO-LD/oold-python/commit/8c965d6cd3aea4963e709fc05e304e789aa73995))
### Continuous Integration
- **release**: Add whats-changed notes with changelog link for zenodo
([`147d70f`](https://github.com/OO-LD/oold-python/commit/147d70fbf1c5da070f3cb9055fbe96e831fe8eaa))
- **release**: Update on title and authors
([`996c4de`](https://github.com/OO-LD/oold-python/commit/996c4dee6b4f07b6ec994e87cb341155a12e955c))
### Documentation
- Add CLAUDE.md with the conventions agents keep getting wrong
([`4fdda5e`](https://github.com/OO-LD/oold-python/commit/4fdda5e796089c4950437918d212c802b4e2e8b4))
- Explain how to turn a specification rule into a check
([`988bc38`](https://github.com/OO-LD/oold-python/commit/988bc38de68724a027268721fad8818786bd5184))
- Fix the vendoring procedure and say what a new check owes
([`92524af`](https://github.com/OO-LD/oold-python/commit/92524af7fd721819d8d0874757d23dcfbdc6ba92))
- **spec**: Collapse the check mappings into one registry structure
([`264d71e`](https://github.com/OO-LD/oold-python/commit/264d71e300095719cdfe52c86726fb5bb64888e1))
- **spec**: Correct the version gate, and cost out a changed rule
([`08b9945`](https://github.com/OO-LD/oold-python/commit/08b9945ea338c3a261eb08d28e82b2d2b7a062ea))
- **spec**: Design a check registry and an `oold checks` command
([`26ceba9`](https://github.com/OO-LD/oold-python/commit/26ceba9c63a9170baaa376c63b039ebfa666a100))
- **spec**: Pin where compatibility for a new rule's check lives
([`3ba00a6`](https://github.com/OO-LD/oold-python/commit/3ba00a6ddff0aa3c739f6739ae7b76b1b11ffda2))
- **spec**: State which ids the registry covers, and fix the grep guard
([`3b497b5`](https://github.com/OO-LD/oold-python/commit/3b497b586ed1e8b13699263b492a27e1acdb1141))
- **validation**: Record catalogue's source so a rebase cannot orphan it
([`5bcd043`](https://github.com/OO-LD/oold-python/commit/5bcd043721226b48280e7980890264df9cdc623b))
### Features
- **validation**: Add native OO-LD schema and instance validator
([`80ee29a`](https://github.com/OO-LD/oold-python/commit/80ee29a1c61cb429f8f39d8db70375508f7a8375))
- **validation**: Check catalogue and fixture slice against facts
([`a7a7f3c`](https://github.com/OO-LD/oold-python/commit/a7a7f3cd167e24e38731a117ad9a87494acbd666))
- **validation**: Cite the specification's rule ids in findings
([`f39b280`](https://github.com/OO-LD/oold-python/commit/f39b28057f24609789b5718099579fcffb49d560))
- **validation**: Drive rule checks from the specification catalogue
([`71c734c`](https://github.com/OO-LD/oold-python/commit/71c734c6fb1864a6d9274efaf017be6032b2b4c6))
- **validation**: Enforce four more catalogued rules
([`fb99f5c`](https://github.com/OO-LD/oold-python/commit/fb99f5c3aebdec7227bd2cd68840b4cc2bd8c061))
- **validation**: Enforce narrow-only composition
([`13ade19`](https://github.com/OO-LD/oold-python/commit/13ade196c9ba8d6a9a9e8501e20ac213a688aaa6))
Preview via python-semantic-release and conventional commits. |
Contributor
📊 Benchmark ResultsClick to see benchmark comparisonThreshold: 1.3x (30% slower triggers a regression warning) Note: Benchmarks are informational only and won't fail the build. 💡 Tip: Download the |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
LukasGold
force-pushed
the
feat/validator-migration
branch
from
August 15, 2026 15:17
36aaf1c to
5a6b6d7
Compare
Contributor
📊 Benchmark ResultsClick to see benchmark comparisonThreshold: 1.3x (30% slower triggers a regression warning) Note: Benchmarks are informational only and won't fail the build. 💡 Tip: Download the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A native Python validation subsystem under
src/oold/validation/, replacing use of the reference harnessoold-schema/scripts/validate.mjsas a subprocess.schema.*,lint.*,context.*,roundtrip.*,generate.*,instance.*,variants,compliance.*,coverage.*,meta.*, and 22rule.*checksoold validate,oold validate-instance,oold compliance,oold meta,oold rules,oold checksRationale
@contextchains that leave the schema's own directory, which the reference harness cannot do: its loader only maps its own directorySeverity comes from the specification, never from this code
Each
rule.*check reports a problem; whether that is a failure or a warning is read from the rule'slevelinoold-rules.json, the catalogue of numbered normative statements.rule.*family@context, so a schema whose terms arrive through a remote context is not reported as violating themMeta-schema vendoring
src/oold/validation/meta/<version>/holds verbatim copies from oold-schema release tags, with a sha256 per file recorded inindex.json. Nothing is fetched at runtime for a tracked version, so a released version cannot change meaning between runs. Unreleased upstream state is reachable with--meta remote.-textin.gitattributes; a CRLF copy hashes differently, passing on Windows and failing on Linux$refs, so it declares its own file settests/data/oold/is refreshed from the same tag it records infixtures.tag, so fixtures and meta-schemas never come from different releasesCoupling to oold-schema
The two repositories release on separate schedules, so neither pipeline waits on the other.
coverage.ruleswarns when a catalogued rule has no check, rather than failing, so a specification that has moved ahead cannot break this buildOOLD-CMP-a05aas not machine-checkable; it reaches this repository at the next release, since tracked catalogues come from tagsVerification
make check,make validateandmake docs-testexit 0OOLD_SCHEMA_DIRsettests/data/oold/broken/is mapped to the check it must trip, so the checks are proven to fire rather than only to pass on valid input