Skip to content

fix(libvirt): match the TechVault fakes and lookup absence to captured libvirt shapes - #1453

Closed
doublewhy wants to merge 4 commits into
devfrom
1344-libvirt-native-shapes
Closed

doublewhy wants to merge 4 commits into
devfrom
1344-libvirt-native-shapes

Conversation

@doublewhy

@doublewhy doublewhy commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes findings LV-2 and LV-3 of the #1344 fixture-provenance audit (#1602). The TechVault libvirt fakes in test_libvirt_backend_techvault_native.py, test_libvirt_backend_guest_certified.py and test_libvirt_evidence_run.py used shapes libvirt never produces:

  • A lookup of a missing domain or network raised KeyError. libvirt raises libvirt.libvirtError with code 42 (VIR_ERR_NO_DOMAIN) or 43 (VIR_ERR_NO_NETWORK).
  • XMLDesc(0) echoed the defined XML (<memory unit="MiB">128</memory>). libvirt reads it back normalized: <memory unit='KiB'>131072</memory>, a currentMemory element and <vcpu placement='static'>.

This PR takes the fakes' shapes from a committed capture of libvirt's built-in test driver and adds tests that run the production readers and the TechVault concern gate on the captured XML. The production code that agreed with the old fakes is fixed too: _ensure_name_available and _resolve_by_name no longer read a lookup KeyError as verified absence, so realize refuses to create and destroy reports residual state.

Requirement UIDs

  • None. Backend fixture and error-handling correction with no requirement scope.

Related Issues

Closes #1602

Context: #1344 (the audit), #1461 (open audit record; lists LV-2 and LV-3), #1533 (LV-1).

ADR Impact

  • None.

Changes

  • tests/data/boundary_captures/libvirt-test-driver.json (new): the XML the production renderers produce, what libvirt 12.7.0 (test:///default, libvirt-python 12.8.0) read back after define and create(), and its lookup errors for never-defined names, with procedure and limits.
  • tests/libvirt_native_shapes.py (new): a registered libvirtError stand-in, lookup() raising the captured code for a missing or undefined object with the requested name in the message, and libvirt_readback() applying the normalization.
  • The three modules use the helper; the failing-lookup fake raises code 1 (VIR_ERR_INTERNAL_ERROR); a parametrized destroy test covers deleted, renamed-out-of-band and lookup-key-error.
  • raes_backend_libvirt/_techvault_native_ops.py, techvault_lifecycle.py: the two except KeyError absence branches are removed. Both research captures are recorded for the changed source.

Done-when matrix for #1602 (CI lane canonical / test-shard; each mutation disables one production line; counts are failures among the four modules' 137 tests):

Done-when Change Test that fails without it Evidence
fails if code 42/43 handling stops at realize fakes raise captured codes (libvirt_native_shapes.py:lookup) 46 tests with _ensure_name_available's code check disabled 137 passed
same, at destroy same deleted case with _resolve_by_name's code check disabled same
same, KiB readback libvirt_native_shapes.py:libvirt_readback 42 tests with memory_mib's KiB factor set to 1 same
captured XML passes the concern gate; renderer drift fails test_issue_1344_libvirt_native_shapes.py test_libvirt_readback_satisfies_the_techvault_concern_gate, test_captured_input_is_what_the_renderer_produces_today same
KeyError makes realize refuse and destroy report residual state KeyError branches removed test_native_driver_does_not_create_when_a_name_lookup_raises_key_error, lookup-key-error case (both fail with the package files stashed) same
#1461 records the fix if merged not applicable: #1461 is open - -

Test Plan

  • Unit tests pass
  • Integration tests pass if applicable
  • Full completion suite required in CI before merge
  • No coverage regression

uv run --project implementations/python --frozen --all-extras python -m pytest on the four touched modules: 137 passed. The 40 test modules that import raes_backend_libvirt or the helper: 986 passed; with -m integration, 2 passed and the 2 real-libvirt tests skipped (no RAES_REAL_LIBVIRT_URI). With the package files stashed, the two new KeyError tests fail; with the old lookup() restored, both test_fake_lookup_reports_libvirt_absence_for_the_requested_name cases fail.

How the capture was collected: the recorded procedure ran on XML from the production renderers (libvirt 12.7.0, 2026-10-09T09:48Z); a virsh -c test:///default replay at 13:46Z reproduced all three readbacks byte for byte. For the rename case, libvirt-python 12.7.0 refuses to rename a running domain (code 55); after destroy it succeeds, the old name raises code 42 and the UUID is unchanged.

Research evidence: blitz-tools/collect_research_evidence.py re-ran every pinned specification artifact through the production pipeline, replayed the 10 retained formal cases and ran the 14 named participant test functions on this source; python -m tools.research_capture record wrote specification-coverage/captures/capture-4252bed4… and formal-semantic-validation/captures/capture-5c2ed225…. tools/check_specification_coverage.py and tools/check_formal_semantic_validation.py passed.

nox -s verify-fast-feedback -- --base-rev origin/dev, nox -s lint, make policy and ruff (from implementations/python) passed.

CI: head 22daea6 on dev 348c490; run 38513063785 passed every job (canonical / coverage-reduce included), deploy skipped; Docs 38513063644 passed.

Ground Control Checks

  • Repository policy command passes
  • Tests are proven to observe the production branches they cover (matrix above)

Traceability

  • IMPLEMENTS: (none)
  • TESTS: test_issue_1344_libvirt_native_shapes.py, test_libvirt_backend_techvault_native.py, test_libvirt_backend_guest_certified.py, test_libvirt_evidence_run.py
  • Existing TESTS lines in ASR-519, ASR-532 and RUN-314 citing the modified modules still hold.

Checklist

  • Code follows the project coding standards
  • FM0: local error classification in the libvirt backend driver, covered by unit tests; no executable or runtime adoption is claimed.
  • PR title is a Conventional Commit

doublewhy added a commit that referenced this pull request Oct 9, 2026
The convention and the captures no longer point at files that exist only
in #1453: the LV-2 and LV-3 readback and missing-object codes are recorded
in captures.md, and the convention names the existing tests/data directory.
Align provenance labels and verdicts with the record's own definitions,
give capture dates, versions and the expired artifact behind GH-6 and NS-3,
add fixture locations to the register, and use present tense for current
behavior.
@doublewhy
doublewhy marked this pull request as ready for review October 9, 2026 13:49
@doublewhy
doublewhy force-pushed the 1344-libvirt-native-shapes branch from 3c4b7bf to 72534a1 Compare October 10, 2026 17:13
doublewhy added a commit that referenced this pull request Oct 10, 2026
The convention and the captures no longer point at files that exist only
in #1453: the LV-2 and LV-3 readback and missing-object codes are recorded
in captures.md, and the convention names the existing tests/data directory.
Align provenance labels and verdicts with the record's own definitions,
give capture dates, versions and the expired artifact behind NS-3, add
fixture locations to the register, and use present tense for current
behavior.
…dback

The TechVault-family fakes raised KeyError for a missing object and echoed
the defined XML from XMLDesc. libvirt raises libvirtError with code 42 or
43 and reads memory back in KiB, so the production branches that handle
the real shapes ran in no hermetic test.

Commit a libvirt 12.7.0 test:///default readback of production-rendered
TechVault XML. The fakes raise its lookup errors and apply libvirt's
memory and vcpu normalization to their readback, which a test checks
against the capture. Drive the production readers and concern gate with
the captured XML.
The destroy test for a known domain that libvirt no longer finds deleted
the domain outright, so listAllDomains returned nothing and the
verified-absence check had nothing to refuse. Forcing
_resolve_verified_absence to report absence left every libvirt test
passing.

Parametrize the test with a second case: the domain is stopped and
renamed out of band, which libvirt allows only for an inactive domain.
It is still listed under its RAES UUID, so destroy must report residual
state instead of absence.

Read the domain's isActive() value from the capture instead of a
constant.
libvirt reports a missing domain or network with code 42 or 43, never
KeyError (tests/data/boundary_captures/libvirt-test-driver.json). The
KeyError branches in _ensure_name_available and _resolve_by_name existed
only for the old fakes; a KeyError is now an unexpected lookup failure,
so realize refuses to create and destroy reports residual state.

The shared fake lookup() no longer finds an undefined object and quotes
the requested name in the missing-network message, as libvirt does.
@doublewhy
doublewhy force-pushed the 1344-libvirt-native-shapes branch from 72534a1 to 22daea6 Compare October 14, 2026 04:45
@doublewhy doublewhy changed the title test(libvirt): take TechVault fake shapes from a captured libvirt readback fix(libvirt): match the TechVault fakes and lookup absence to captured libvirt shapes Oct 14, 2026
@Brad-Edwards

Copy link
Copy Markdown
Collaborator

Recommendation: do not merge as is.

Reviewed head 22daea63f17af24b1f4a8eb300372863ff668d81, targeting dev.

  1. Merge conflicts against dev (DIRTY).

Required action:

Coverage and outstanding work: #1602 has captured readback/error provenance, production error-handling fixes and regression coverage for absence and KiB normalization. Reusing libvirt error codes and refusing unverified absence is sound. LV-1 is separately tracked by #1533 and concerns the generic driver rather than the corrected TechVault paths.

Verification limits: Head checks are green; current base synchronization and conditional audit reconciliation remain unverified.

@doublewhy

Copy link
Copy Markdown
Contributor Author

Closing, since there's nothing left to merge into. #1621 (ADR-115) removed raes_backend_libvirt from dev, including every file this PR changes (_techvault_native_ops.py, techvault_lifecycle.py and the three libvirt test modules), and docs/migration/concrete-backends.md records that libvirt has no successor. Merging current dev would delete everything this PR touches. The bug in #1602 survives only in the released 7.0.0 line on main. Reopen this PR if you want that fixed there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants