Repository navigation
Conversation
doublewhy
force-pushed
the
1344-fixture-provenance-audit
branch
2 times, most recently
from
October 9, 2026 10:57
011ef3f to
a44c84a
Compare
doublewhy
marked this pull request as ready for review
October 9, 2026 13:36
This was referenced Oct 9, 2026
…fixture convention Inventory the test boundaries the audit examined that stand in for an external system, with the fixture's provenance, the real producer's observed response, a verdict and a blast-radius rating; keep the observed shapes; register each divergence with its fix status; and write down how a new boundary fixture establishes where its shape came from.
The convention and the captures no longer point at files that exist only in #1453: the LV-2 and LV-3 readback and missing-object codes are recorded in captures.md, and the convention names the existing tests/data directory. Align provenance labels and verdicts with the record's own definitions, give capture dates, versions and the expired artifact behind NS-3, add fixture locations to the register, and use present tense for current behavior.
GH-6 misread a deliberate check. The fixture's distinct GITHUB_SHA and GITHUB_WORKFLOW_SHA values let the release-identity test prove that the two fields map separately, as the issue-1226 preflight record requires. Rate that row a match and drop GH-6 from the register. The record covers the boundaries the audit examined. Say so, and name examples it did not examine: the pinned z3 binding and its clock fakes, the TechVault host probe stub, and podman. Also rate NS-6 n/a, since nothing reads its fields; define the qualified labels and "match in effect"; label the two LilRAE rows by how their fixtures were made; show the nested REST shape for GH-1 and mark the field GH-2 derives; name the NS-5 manifest; call the store, time, MCP and snapshot entries summaries; mark libvirt code 62 as taken from source; and correct the lease close() range.
doublewhy
force-pushed
the
1344-fixture-provenance-audit
branch
from
October 10, 2026 17:24
62899a8 to
0a528fc
Compare
This was referenced Oct 10, 2026
Closed
Closed
Open
This was referenced Oct 12, 2026
Requirement governance lets Ground Control transport and decoding errors escape classification
#1557
Closed
Open
…ter entry Each open entry in the divergence register now names its issue: #1525 to #1557 for the production changes, and #1521 for the smaller corrections. The entries that open pull requests now address move to "Addressed by pull requests" with their issues: GH-7 (#1491, #1490), NS-1 (#1476, #1475), NS-2 (#1480, #1479), MF-2 (#1483, #1482), CL-2 (#1474, #1473), ST-2 (#1472, #1559) and NS-7 and NS-8 (#1488, #1560 and #1561). ST-1 names #1558, GH-3 records that #1415 merged, and "What remains" names #1522 for the unverified and unexamined boundaries.
This was referenced Oct 12, 2026
Merged
Collaborator
|
Recommendation: do not merge as is. Reviewed head
Required action:
Coverage and outstanding work: The inventory, captures, sibling findings and fixture convention are useful and fit the repository research/reference surfaces. High-risk observation coverage is incomplete, and #1344 correctly remains the broader audit umbrella. The existing linked follow-ups are tracked rather than silently omitted. Verification limits: Complete file patches but incomplete linked-issue inventory. #1461 is OPEN, so conditional audit updates in #1459 are currently not applicable. |
11 tasks done
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of #1344: the audit record and the convention it asks for. Most divergences registered need production changes, several involving release-admission or semantics choices. The register names the issue that tracks each open divergence: #1525, #1527, #1529, #1531, #1533, #1535, #1537, #1539, #1541, #1543, #1545, #1547, #1549, #1551, #1553, #1555 and #1557.
docs/research/fixture-provenance/inventory.mdlists each boundary class the audit examined where a test stands in for something this repository does not control, with consumer, fixture, captured or inferred, the real producer's response, a verdict and a blast-radius rating. The record's limits name stand-ins the audit did not examine, such as the pinned z3 solver binding.docs/research/fixture-provenance/captures.mdkeeps the observed responses, trimmed, with signed URL query strings removed. Observed 2026-10-09 (NS-5: 2026-10-10); store, time, MCP and runtime-snapshot entries are marked as summaries.docs/research/fixture-provenance/index.mdhas the method and its limits, results per class, a divergence register (smaller corrections tracked in Apply the smaller fixture-provenance corrections from the audit register #1521), what remains (unverified and unexamined boundaries tracked in Validate the remaining unverified fixture boundaries #1522), and the sibling-repository findings (Seat image pull requests its token from /token instead of the registry's advertised realm lilrae#1331, the seat image token realm; Release publish job runs gh release upload without a repository adapters#108,gh release uploadwithout a repository; a first pass over hub and env-packs found nothing to fix).docs/explain/reference/fixture-provenance.md(indexed in the reference README) is the convention for new boundary fixtures: capture before encoding, keep the capture where the tests use it, prove the fake reaches the real branch, keep captures current.The PR stands alone: no file in it points at a file that exists only in an open pull request. Clear-cut divergences are fixed in separate pull requests, which the register cites: #1453, #1455 (#1454), #1459 (#1558), #1464 (#1462), #1472 (#1559), #1474 (#1473), #1476 (#1475), #1480 (#1479), #1483 (#1482), #1488 (#1560, #1561) and #1491 (#1490); #1415 (#1414) merged on 2026-10-11.
Requirement UIDs
Related Issues
Closes #1460
ADR Impact
Changes
Documentation only: the four new files and the index entry described in the Summary; the reference README also has sentence-case heading and plainer entry wording.
inventory.mdcaptures.mddocs/explain/reference/fixture-provenance.mdindex.md(sibling-repository findings)devat35122105.path:lineresolver scriptnox -s docs-local.Test Plan
CI on head
f7a757fd(branch base35122105), merged with dev348c4903: run 38516317607 succeeded; Docs run 38516317552 built.Local checks on
f7a757fd:nox -s verify-fast-feedback -- --base-rev origin/devpassed (28 stages PASS, 1 SKIP; it includes gitleaks andpolicy / identity cutover),make policypassed its 15 stages, andnox -s docs-localpassed its 5 stages, including Sphinx. Vale 3.15.2 with the repository config reports no alerts on the five files.A script resolved each of the 285
path:linereferences in the four record files against35122105: each names a file that exists there and stays within its length. GH-1, GH-2, GH-5, LV-1, LV-4, NS-1, NS-2 and CL-1 were captured again while the record was written, and the store, time and runtime-snapshot probes rerun; libvirt source citations were read at itsv12.7.0tag.Ground Control Checks
make policyonf7a757fd.path:linereference resolves at35122105.Traceability
Checklist
CHANGELOG.mdfrom it)Documentation
Updated:
docs/explain/reference/README.md. Added:docs/research/fixture-provenance/anddocs/explain/reference/fixture-provenance.md. Both sit outsidedocs/public/.