Repository navigation
Conversation
The convention and the captures no longer point at files that exist only in #1453: the LV-2 and LV-3 readback and missing-object codes are recorded in captures.md, and the convention names the existing tests/data directory. Align provenance labels and verdicts with the record's own definitions, give capture dates, versions and the expired artifact behind GH-6 and NS-3, add fixture locations to the register, and use present tense for current behavior.
3c4b7bf to
72534a1
Compare
The convention and the captures no longer point at files that exist only in #1453: the LV-2 and LV-3 readback and missing-object codes are recorded in captures.md, and the convention names the existing tests/data directory. Align provenance labels and verdicts with the record's own definitions, give capture dates, versions and the expired artifact behind NS-3, add fixture locations to the register, and use present tense for current behavior.
…dback The TechVault-family fakes raised KeyError for a missing object and echoed the defined XML from XMLDesc. libvirt raises libvirtError with code 42 or 43 and reads memory back in KiB, so the production branches that handle the real shapes ran in no hermetic test. Commit a libvirt 12.7.0 test:///default readback of production-rendered TechVault XML. The fakes raise its lookup errors and apply libvirt's memory and vcpu normalization to their readback, which a test checks against the capture. Drive the production readers and concern gate with the captured XML.
The destroy test for a known domain that libvirt no longer finds deleted the domain outright, so listAllDomains returned nothing and the verified-absence check had nothing to refuse. Forcing _resolve_verified_absence to report absence left every libvirt test passing. Parametrize the test with a second case: the domain is stopped and renamed out of band, which libvirt allows only for an inactive domain. It is still listed under its RAES UUID, so destroy must report residual state instead of absence. Read the domain's isActive() value from the capture instead of a constant.
libvirt reports a missing domain or network with code 42 or 43, never KeyError (tests/data/boundary_captures/libvirt-test-driver.json). The KeyError branches in _ensure_name_available and _resolve_by_name existed only for the old fakes; a KeyError is now an unexpected lookup failure, so realize refuses to create and destroy reports residual state. The shared fake lookup() no longer finds an undefined object and quotes the requested name in the missing-network message, as libvirt does.
72534a1 to
22daea6
Compare
|
Recommendation: do not merge as is. Reviewed head
Required action:
Coverage and outstanding work: #1602 has captured readback/error provenance, production error-handling fixes and regression coverage for absence and KiB normalization. Reusing libvirt error codes and refusing unverified absence is sound. LV-1 is separately tracked by #1533 and concerns the generic driver rather than the corrected TechVault paths. Verification limits: Head checks are green; current base synchronization and conditional audit reconciliation remain unverified. |
|
Closing, since there's nothing left to merge into. #1621 (ADR-115) removed |
Summary
Fixes findings LV-2 and LV-3 of the #1344 fixture-provenance audit (#1602). The TechVault libvirt fakes in
test_libvirt_backend_techvault_native.py,test_libvirt_backend_guest_certified.pyandtest_libvirt_evidence_run.pyused shapes libvirt never produces:KeyError. libvirt raiseslibvirt.libvirtErrorwith code 42 (VIR_ERR_NO_DOMAIN) or 43 (VIR_ERR_NO_NETWORK).XMLDesc(0)echoed the defined XML (<memory unit="MiB">128</memory>). libvirt reads it back normalized:<memory unit='KiB'>131072</memory>, acurrentMemoryelement and<vcpu placement='static'>.This PR takes the fakes' shapes from a committed capture of libvirt's built-in test driver and adds tests that run the production readers and the TechVault concern gate on the captured XML. The production code that agreed with the old fakes is fixed too:
_ensure_name_availableand_resolve_by_nameno longer read a lookupKeyErroras verified absence, so realize refuses to create and destroy reports residual state.Requirement UIDs
Related Issues
Closes #1602
Context: #1344 (the audit), #1461 (open audit record; lists LV-2 and LV-3), #1533 (LV-1).
ADR Impact
Changes
tests/data/boundary_captures/libvirt-test-driver.json(new): the XML the production renderers produce, what libvirt 12.7.0 (test:///default, libvirt-python 12.8.0) read back after define andcreate(), and its lookup errors for never-defined names, with procedure and limits.tests/libvirt_native_shapes.py(new): a registeredlibvirtErrorstand-in,lookup()raising the captured code for a missing or undefined object with the requested name in the message, andlibvirt_readback()applying the normalization.VIR_ERR_INTERNAL_ERROR); a parametrized destroy test coversdeleted,renamed-out-of-bandandlookup-key-error.raes_backend_libvirt/_techvault_native_ops.py,techvault_lifecycle.py: the twoexcept KeyErrorabsence branches are removed. Both research captures are recorded for the changed source.Done-when matrix for #1602 (CI lane
canonical / test-shard; each mutation disables one production line; counts are failures among the four modules' 137 tests):libvirt_native_shapes.py:lookup)_ensure_name_available's code check disableddeletedcase with_resolve_by_name's code check disabledlibvirt_native_shapes.py:libvirt_readbackmemory_mib's KiB factor set to 1test_issue_1344_libvirt_native_shapes.pytest_libvirt_readback_satisfies_the_techvault_concern_gate,test_captured_input_is_what_the_renderer_produces_todayKeyErrormakes realize refuse and destroy report residual statetest_native_driver_does_not_create_when_a_name_lookup_raises_key_error,lookup-key-errorcase (both fail with the package files stashed)Test Plan
uv run --project implementations/python --frozen --all-extras python -m pyteston the four touched modules: 137 passed. The 40 test modules that importraes_backend_libvirtor the helper: 986 passed; with-m integration, 2 passed and the 2 real-libvirt tests skipped (noRAES_REAL_LIBVIRT_URI). With the package files stashed, the two newKeyErrortests fail; with the oldlookup()restored, bothtest_fake_lookup_reports_libvirt_absence_for_the_requested_namecases fail.How the capture was collected: the recorded procedure ran on XML from the production renderers (libvirt 12.7.0, 2026-10-09T09:48Z); a
virsh -c test:///defaultreplay at 13:46Z reproduced all three readbacks byte for byte. For the rename case, libvirt-python 12.7.0 refuses to rename a running domain (code 55); afterdestroyit succeeds, the old name raises code 42 and the UUID is unchanged.Research evidence:
blitz-tools/collect_research_evidence.pyre-ran every pinned specification artifact through the production pipeline, replayed the 10 retained formal cases and ran the 14 named participant test functions on this source;python -m tools.research_capture recordwrotespecification-coverage/captures/capture-4252bed4…andformal-semantic-validation/captures/capture-5c2ed225….tools/check_specification_coverage.pyandtools/check_formal_semantic_validation.pypassed.nox -s verify-fast-feedback -- --base-rev origin/dev,nox -s lint,make policyand ruff (fromimplementations/python) passed.CI: head 22daea6 on
dev348c490; run 38513063785 passed every job (canonical / coverage-reduceincluded),deployskipped; Docs 38513063644 passed.Ground Control Checks
Traceability
test_issue_1344_libvirt_native_shapes.py,test_libvirt_backend_techvault_native.py,test_libvirt_backend_guest_certified.py,test_libvirt_evidence_run.pyASR-519,ASR-532andRUN-314citing the modified modules still hold.Checklist