M5 Phase 3: the rule studio onto the fixture corpus

[kjpatel]

Phase 3, part 4b(ii)c-1. test_rules_api moves off the teaching packs onto a content root assembled from fixtures/.

The measurement

	before	after
test_rules_api failures under deletion	29	0
demo/tests total under deletion	57	28

Full suite 969 passed / 7 skipped. verify replays all 351 golden receipts byte-for-byte; impact reports 0 of 351 decisions flip. The fixture builder reproduces byte-identically across runs.

Two of the 29 were not failures

test_declared_cases_run_green_against_every_committed_pack looped over a glob of packs — with the packs deleted, the loop body never ran. test_every_refusal_example_is_reported_not_raised was @pytest.mark.parametrized over a directory, and parametrize is evaluated at collection, so an empty directory produced zero test cases. pytest reports the count that remains and says nothing about what left.

Both passed. Both are named after claims nothing was checking. CLAUDE.md already said "asserts over an empty glob" was a failure mode; these are the two shapes it actually takes, and both fixes are one line.

What the fixtures lacked

Each added because a test needed it, not for completeness:

• fixtures/expected.yaml — declared outcomes. The studio runs them beside impact analysis, so a fixture corpus without them left half that surface untestable.
• fixtures/dmn/ — one table that compiles (2 decisions, 4 rules), one that is refused. The import panel and compiler happy path were reachable only through dmn/examples/, which an adopter deletes.
• nested abstentionPolicy.attributes — two scalars cannot catch a re-emitter that flattens.
• fx-0006, a restricted widget scoring 60.

Pack 2026.2.0 → 2026.3.0; receipts, digest vectors and the corpus aggregate re-pinned. Only rulePack.version and the resulting hashes moved — no decision changed.

Why fx-0006 exists

Every existing case scored 12 or 80 against a threshold of 50, so any threshold edit flipped all three restricted cases at once or none. Impact analysis over these fixtures could only ever answer "everything moved" — which cannot demonstrate the thing it exists for: a pack whose meaning moved while every declared outcome stayed green. It clears a new bar for this corpus: not a behaviour that was unreachable, but a distinction the corpus could not draw.

Found on the way

Editing the derived threshold from 50 to 70 moves all three of the pack's decisions, while two of the three rules stay byte-identical on the page — they read the moved value as a derived binding. The equivalence panel returns a witness naming a widget scoring exactly 50, the boundary where the packs part.

The diff is complete and honest and it is still not where the effect lives. The blast radius of an edit is not a syntactic property of the edit — that is the argument for the panel, and it is stronger than the one the docs had been making. Sharpened into docs/neuro-symbolic-architecture.md.

Plan corrections

Three claims about this task were wrong:

• the suite is 29 failures, not 31 (31 is coincidentally the number of tests needing conversion — unrelated)
• test_api has 6 residual failures, not 4 — verified against M5 Phase 3: test_api onto the fixture pack, and a docs pass that is not optional #62, so not a c-0 regression
• part 5's inherited "needs a date attribute in the fixture pack" is false; placeholder validation is syntactic and that test builds its own facts

Part 5's fixture DMN prerequisite is also delivered here. The progress log had lost the PR links for #61–63 and never recorded #64; both fixed.

Docs pass

Per CLAUDE.md's definition of done: fixtures/README.md (three stale counts), the CLAUDE.md layout row and two gotchas, the architecture doc, CHANGELOG, and the plan. A stale docstring in test_receipts_api.py claiming it runs against the committed corpus — false since #59 — is corrected.

🤖 Generated with Claude Code