test: assert the schema-payload invariant once and share one candidate wheel across pristine-root tests - #231
Merged
Merged
Conversation
… once Eighteen per-version contract modules each ran the identical whole-repository check `assert_schema_payload_references(build_package_repository(_ROOT)) == []`. Every copy walked the full package repository (~2.4 s) to prove the same catalog-wide property, so a single stale schema reference failed eighteen tests and seventeen of the walks bought no coverage. The invariant now lives in tests/package_contract/test_schema_payload_references.py. Each per-version module keeps its own payload-specific sweep untouched; none of the eighteen passed a version-scoped argument, so none had to be left in place. test_project_toolbox_1_0.py's copy was a whole test function with no other assertion and is removed; the other seventeen lose one line inside a larger test. Refs #14
…ests Five sites each ran `uv build --wheel` over the pristine repository root (2.1 s plus a 71 MB extraction apiece) to inspect what the released artifact contains. They now take a session-scoped `built_wheel` fixture in tests/conftest.py, which honors PROJECT_STANDARDS_COMPATIBILITY_WHEEL — the override tests/package_compatibility/conftest.py already used and scripts/verify.sh already exports — so a verify.sh run reuses one artifact for every worker and an ad-hoc run builds once per worker rather than once per test. Deliberately left building their own tree, because the fixture would substitute the real repository and void the property under test: every site that synthesizes a minimal project with its own `[tool.uv.build-backend]` (tests/package_contract/test_end_to_end.py, test_adr_reconstruction.py, test_self_hosting.py, the *_reconstruction.py modules, tests/control_plane/*), test_projection.py's sdist-to-wheel equivalence build, tests/test_installed_wrappers.py's two version-bumped source trees, and tests/mcp_server/e2e/test_installed_wheel.py, whose contract is to run the exact commands the verification leg records a digest for. Refs #14
The comment claimed the compatibility matrix installs wheels per case. That stopped being true when both distributions became session fixtures (tests/package_compatibility/conftest.py) and the wheel arm started honoring PROJECT_STANDARDS_COMPATIBILITY_WHEEL, which verify.sh itself exports. The claim carried documentation authority and had already misdirected the wall-clock attribution in docs/research/2026-09-01-release-train-wall-clock.md:57. Comment-only change; the reason smoke mode collects only is unchanged. Refs #14
chrisdpurcell
marked this pull request as ready for review
September 1, 2026 12:39
This was referenced Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test-suite efficiency fixes from this session's survey (#227), none of which changes coverage:
package_contractmodules each ran the identicalassert_schema_payload_references(build_package_repository(_ROOT)) == [](≈2.4 s each). The assertion now lives intests/package_contract/test_schema_payload_references.py; each module keeps its own payload-specific sweep. Collected test count is unchanged (17 copies were one line inside a larger test; the one whole-function copy is replaced by the new module).tests/conftest.pygains abuilt_wheelfixture honoringPROJECT_STANDARDS_COMPATIBILITY_WHEEL(the variablescripts/verify.shalready exports), and the five sites that built the pristine root now share it. The thirteen sites that deliberately build a modified tree, the sdist/wheel equivalence test, and the installed-wheel e2e suite are untouched, by design.scripts/verify.shcomment — the compatibility-lane note claimed the matrix installs wheels per case; the distributions have been session fixtures sincetests/package_compatibility/conftest.py:33,45. This stale claim was the origin of Reduce release-train wall-clock time: cost out options against the 106-minute full battery #207's own wrong premise.Measured on the worker:
tests/package_contractat-n auto→ 1784 passed in 96.75 s, so the "~24-minute" attribution recorded in #227 was an invocation artifact, not suite cost; the long tail (21 of the top 25 durations) is the python-tooling provider path.Governing work
Standalone
Change risk: R1 Low
Acceptance coverage
tests/package_contractunchanged (1870); the invariant still fails on a synthetic violation (test_graph.pycases retained).scripts/verify.sh.Verification
rexec -- … pytest tests/package_contract -q -n auto --durations=25→ 4 failed, 1784 passed — the four are the documented reconcile-lag reds (lint-markdown.ymlpin,PC-RELEASE-PROJECTION,.standards/catalog.toml,test_runner_label_advisorymarkdown-tooling 1.15→1.16), none in a file this branch touches.rexec -- … pytest tests/test_spec_wheel_contents.py tests/test_adopt_packaging.py tests/agent_handoff/test_packaging.py tests/test_installed_wrappers.py -n 4→ 34 passed.