Skip to content

nvx-rs slimming 2.3.2: snapshots have one strict format, and a capture finishes into it #1102

Closed

Description

Parent: #1081, in #1076. Needs: #1097, #1099, #1100. Unblocks: #1103, #1114, #1118, #1120, #1130.

Goal. Snapshots have one strict format: the template format's types take the snapshot's names, Snapshot::open reads a snapshot directory strictly, and Snapshot::accepts says whether a snapshot can restore a given VM. A blockless capture VM launches from its VmConfig and CaptureConfig, and snapshot::finish_capture turns its exit into the snapshot's three entries, key.bin and snapshot.json beside openvmm/ in the VM directory; the caller moves those three into its own private snapshot directory, which it keeps immutable.

Notes.

  • No version bump. The key's encoding changes in place: no snapshot outlives a deploy.
  • host::template keeps aliases until the template store goes (nvx-rs slimming 4.2.2: the old clone path, template store and sandbox store are gone #1121).
  • The caller keeps an opened snapshot still. After Snapshot::open, the caller keeps the directory and its entries private and immutable while it uses the Snapshot: open checks them once, and a restore reads them again.

PRs, in order. Each PR has one goal, leaves dev green, and names in its description each old test of its area as kept, rewritten or retired, with each change that check_test_changes.py reports. Its tests follow the repository's test layout: a module's unit tests, and the code that only they use, sit in its tests.rs or tests/ and move with it. The estimates count core lines (code, not comments, docs or tests) and counted files; each PR records its measured values. Each PR does what its Scope says and no more: a case, input or layout that its Scope doesn't name, and whatever "Not in this PR" lists, is out of scope. Address every review comment within the scope, and ask before acting on one that asks for more. A PR whose measured core lines pass the stop that its Estimate names (twice the estimate, at most 600) stops and asks before continuing.

  1. SL2.12a: The snapshot format has its new names.

    • Objective: Snapshot metadata still lives under template names even though the new interface has one snapshot type and no template store. This PR moves and renames that metadata surface while preserving its validation behavior and public time information.
    • Contents: git mv of nvx-rs/src/host/template/{manifest, wire, key, canonical, eligibility}.rs into host::snapshot. TemplateKey becomes SnapshotKey, and TemplateManifest becomes SnapshotManifest; the hash domains take the new names, and the key's encoding changes in place, without a VERSION bump; host::template keeps aliases until SL6.2a. SnapshotKey::identity and KeyIdentity, SnapshotManifest::{read, parse, time, hypervisor}, TimeContract with its eligibility, HostClock, HostTime with now, Eligibility, Downtime, DowntimeSource, MAX_DOWNTIME_NS, MANIFEST_NAME and MAX_MANIFEST_LEN stay public, and guest_build goes. Time remains public through SnapshotManifest::time and TimeContract; SnapshotKey::time is introduced in SL2.14 with the per-snapshot capture time contract, not during this rename.
    • Scope:
      • nvx-rs/src/host/template/{manifest,wire,key,canonical,eligibility}.rs and their tests.rs companions move under host/snapshot/; shared metadata fixtures move from host/template/tests.rs to host/snapshot/tests.rs, and remaining template tests import them there.
      • TemplateKey becomes SnapshotKey, TemplateManifest becomes SnapshotManifest, and their hash domains use the snapshot names.
      • The key encoding changes in place without a version increment, and the obsolete guest_build value and accessor are removed.
      • The key, manifest, time, eligibility, downtime, host-clock and manifest-limit APIs listed in the group issue remain public under host::snapshot.
      • host::template keeps aliases for old users until SL6.2a.
    • Not in this PR:
      • Behavior changes to manifest parsing, time eligibility or geometry validation beyond removing guest_build.
      • A reader for the prior key encoding or a key version increment; no snapshot survives a deployment.
      • Snapshot::open, directory validation or snapshot.json; SL2.12b adds the strict directory reader.
      • Removing the host::template aliases or the template store; later removal PRs handle them.
    • Approach: Use git mv for the five existing modules, rename the two public types and their hash-domain constants in place, and keep aliases at the old module boundary. Update direct imports and tests without introducing wrapper metadata types, a conversion layer or parallel old-format decoding.
    • Done when:
      • The moved key and manifest tests pass under host::snapshot, including strict parsing and time eligibility.
      • SnapshotKey and SnapshotManifest expose the retained public methods and types, while guest_build is absent.
      • Hash and encoding fixtures use the new names without a version increment.
      • Existing host::template users continue to compile through the temporary aliases.
    • Needs: SL2.3c, in nvx-rs slimming 2.1.1: one value for a VM's configuration #1097.
    • Estimate: 170 core lines, 3 counted files. Past 340 core lines, stop and ask before continuing.
  2. SL2.12b: A snapshot directory opens strictly.

    • Objective: A snapshot must be checked once before restores trust its files, rather than accepted as an arbitrary directory. Snapshot::open admits only the one private, complete format and exposes its verified key and manifest.
    • Contents: Snapshot::{open, key, manifest} fill SL2.8's shell, with KEY_FILE, DESCRIPTION_FILE and OPENVMM_DIR and strict checks: exactly the expected entries, regular files private to their owner, the lengths that the manifest records, the manifest read by the strict reader, and a key consistent with the manifest; it never reads snapshot.json. Snapshot's and open's docs state the caller's duty: to keep the directory and its entries private and immutable while it uses the Snapshot.
    • Scope:
      • host::snapshot::Snapshot::{open,key,manifest} fills the opaque shell introduced by SL2.8.
      • A snapshot directory contains exactly key.bin, snapshot.json and openvmm/; the OpenVMM directory contains exactly its manifest, memory and state files.
      • Expected files must be regular and private to their owner, and OpenVMM artifact lengths must equal the lengths recorded by the manifest.
      • The strict SnapshotManifest reader parses the OpenVMM manifest, and key.bin must describe the same snapshot.
      • KEY_FILE, DESCRIPTION_FILE and OPENVMM_DIR are public, and the docs require callers to keep the opened directory and entries private and immutable.
    • Not in this PR:
      • Reading, parsing or trusting snapshot.json; it is informational and SL2.14 writes it.
      • Watching files for later mutation or retaining open handles to every entry; the caller owns immutability after the one check.
      • Accepting extra files, symbolic links, alternate names or optional OpenVMM artifacts; this repository has one strict format.
      • Version negotiation, or readers and migrations for alpha.7 keys and snapshots: this release opens only its one strict Fresh format.
      • VM compatibility checks such as devices, capacity, runtime or time; SL2.12c adds Snapshot::accepts.
    • Approach: Build open from the existing private-directory, regular-file, bounded manifest and key checks, with fixed entry-name lists. Read the manifest once through SnapshotManifest::read, compare its recorded lengths and key, and store the parsed values; do not add format discovery, recursive traversal, a watcher or a schema for snapshot.json.
    • Done when:
      • open accepts a directory containing only the three snapshot entries with matching private files, lengths, manifest and key.
      • open refuses each missing or extra entry, an irregular or non-private file, and every recorded-length mismatch.
      • open refuses a malformed manifest and a key inconsistent with it.
      • Changing snapshot.json does not affect opening because open never reads it.
    • Needs: SL2.12a, above; SL2.8, in nvx-rs slimming 2.2.1: a VM is created in a private directory, with its restore-ready listener, then spawned #1099.
    • Estimate: 250 core lines, 2 counted files. Past 500 core lines, stop and ask before continuing.
  3. SL2.12c: A snapshot says whether it can restore a given VM.

    • Objective: Opening proves that snapshot files are well formed, but not that a requested VM can use them. A snapshot reports compatibility with the concrete Fresh VM shapes in this repository and exposes its captured NIC without carrying a network policy.
    • Contents: Snapshot::accepts (a layered root; processors and memory between the captured boot size and the capacity, processor counts of 1, 2, 4 or 8, and memory aligned to 128 MiB; an image no longer than the placeholder, with the same block sizes, and a scratch, if any, with the dormant scratch's geometry; the NIC's presence, and a named address equal to the captured one; a console listener only if the snapshot was captured with one; the same backend, CPU profile and runtime identity; and an eligible time contract) and Snapshot::nic, which gives the captured NIC's presence and address, not a policy: each restore installs its own. An ineligible time contract is an Unavailable error, and any other refusal an Unsupported one, as a restore under another runtime is today. The snapshot.json writer comes with its caller, in SL2.14.
    • Scope:
      • Snapshot::accepts requires a layered root and processors and memory between captured boot size and capacity, with processor counts 1, 2, 4 or 8 and memory aligned to 128 MiB.
      • The image must fit the placeholder and keep its logical and physical block sizes; a scratch, when used, must match the dormant scratch's complete geometry.
      • NIC presence must match, and a named restore address must equal the captured address; Snapshot::nic returns only that presence and address.
      • A console listener is accepted only when the capture had one, and backend, CPU profile and runtime identity must match.
      • The snapshot's time contract must be eligible at the supplied host time; an ineligible one is an Unavailable error, and any other refusal an Unsupported one.
    • Not in this PR:
      • Fresh-to-blockless restores, paired disk captures or claim-style snapshots; this release restores only the layered Fresh format.
      • NIC re-addressing or comparing network policies; each restore supplies its own policy at the captured address.
      • Resizing, padding or converting image and scratch files; compatibility is a direct geometry check.
      • Writing snapshot.json; SL2.14 writes it after a capture exits.
      • Opening or revalidating snapshot paths; SL2.12b performs the one strict open.
    • Approach: Compare the checked VmConfig, Runtime and host time directly with fields already parsed into the snapshot key and manifest. Return the captured NIC as a small value, not a restored policy. Keep one ordered compatibility check and existing validation helpers; do not introduce negotiable capabilities, converters or a pluggable compatibility framework.
    • Done when:
      • accepts accepts one fully matching layered VM.
      • Tests refuse a blockless root and each processor, memory, image, scratch, NIC, console, backend, CPU-profile, runtime and time mismatch independently.
      • Processor counts outside 1, 2, 4 and 8 and memory not aligned to 128 MiB are refused.
      • Snapshot::nic reports the captured NIC's presence and address without a policy.
    • Needs: SL2.12b, above.
    • Estimate: 250 core lines, 3 counted files. Past 500 core lines, stop and ask before continuing.
  4. SL2.13: A blockless capture VM launches from VmConfig and its CaptureConfig.

    • Objective: A Fresh snapshot needs a narrowly defined capture VM whose device shape can later bound restores. host::vm::create prepares that blockless launch, records its checked context and gives OpenVMM only the capture inputs it supports.
    • Contents: create with boot.capture: the blockless check and the refusal of an export; the placeholder image and the dormant scratch as sparse files, whose real geometry is verified and recorded; one nvx_snapshot_tier=platform token and no root, view, mapping or workload tokens; only tokens of OpenVMM's platform allowlist, others refused by name; the destination <vm_dir>/openvmm; the NIC under the deny-all policy; the cloneable, serializable CaptureContext (the checked capture configuration, the VM's devices, the runtime identity, the hypervisor and the CPU profile), which the caller persists, and Prepared::capture_context.
    • Scope:
      • host::vm::create handles boot.capture only with a blockless root and refuses any export.
      • The launch creates sparse placeholder-image and dormant-scratch files, verifies their actual geometry, and records it in the capture context.
      • The command line contains exactly one nvx_snapshot_tier=platform token, no root, view, mapping or workload tokens, and only OpenVMM's named platform allowlist.
      • OpenVMM writes to <vm_dir>/openvmm, and a configured NIC is attached under a deny-all policy.
      • CaptureContext is cloneable and serializable and holds the checked capture configuration, VM devices, runtime identity, hypervisor and CPU profile; Prepared::capture_context exposes it.
    • Not in this PR:
      • Layered capture VMs, paired disk captures or captures of restored VMs; this release implements Fresh only.
      • Exports or caller-defined tokens outside OpenVMM's platform allowlist; the capture refuses them by name.
      • Finishing a capture, writing key.bin or snapshot.json, or publishing a named snapshot: SL2.14 finishes a capture after its proved exit, and SL3.7b publishes the named snapshot.
      • A capture store, journal or prescribed persistence format for CaptureContext; callers choose whether and how to persist it.
    • Approach: Add a capture branch to the existing two-phase create path and reuse its checked runtime, device and command construction. Create the two sparse files at fixed VM-directory paths, verify them with the existing geometry helpers, and snapshot the checked values into one serde type; do not revive the template-store capture transaction or build a general token policy engine.
    • Done when:
      • Capture creation requires a blockless root, refuses an export and rejects every non-allowlisted token by name.
      • The placeholder and dormant scratch are sparse files whose verified geometries appear in CaptureContext.
      • Arguments contain the one capture-tier token, omit root, view, mapping and workload tokens, and target <vm_dir>/openvmm.
      • A NIC uses deny-all, and CaptureContext round-trips through serde with every checked value.
    • Needs: SL2.8, in nvx-rs slimming 2.2.1: a VM is created in a private directory, with its restore-ready listener, then spawned #1099; SL2.12a, above.
    • Estimate: 400 core lines, 4 counted files. Past 600 core lines, stop and ask before continuing.
  5. SL2.14: A finished capture leaves a snapshot's three entries.

    • Objective: OpenVMM's capture output is not yet a complete NVX snapshot. After a proved VM exit, finish_capture validates the outcome and OpenVMM files, writes the key and informational description, and leaves exactly the three entries a caller publishes.
    • Contents: snapshot::finish_capture(&VmDir, &CaptureContext, &Exited) → CapturedSnapshot { pub key: SnapshotKey }: the post-exit checks, then key.bin and the snapshot.json writer, in the VM directory beside openvmm/. Introduce SnapshotKey::time here with the per-snapshot capture time contract; do not add capture-time metadata to the shared key during SL2.12a. The caller moves those three entries into its own private snapshot directory and keeps them immutable; the VM directory's other files aren't part of the snapshot. A failure that left the VM directory intact can be retried.
    • Scope:
      • host::snapshot::finish_capture(&VmDir, &CaptureContext, &Exited) checks the capture's proved exit, outcome and OpenVMM snapshot.
      • It writes key.bin and snapshot.json beside the existing <vm_dir>/openvmm/ and returns CapturedSnapshot { key }.
      • The key is derived from the checked capture context and the validated OpenVMM manifest.
      • Only key.bin, snapshot.json and openvmm/ form the snapshot; logs, outcome files and other VM-directory entries remain outside it.
      • A failed finish may be retried when it left the VM directory intact.
    • Not in this PR:
      • Moving or copying the three entries into a snapshot store; the caller publishes them into its own private directory.
      • Including the whole VM directory or cleaning unrelated VM files; they are not snapshot contents.
      • Finishing while the VMM may still run or manufacturing an exit proof; callers pass Exited.
      • Versioned snapshot output, alpha.7 migration tooling or capture tiers other than Fresh: this release writes one Fresh format and reads no alpha.7 snapshot.
      • A recovery journal or durable capture transaction; retry uses the unchanged VM directory and capture context.
    • Approach: Extract the post-exit outcome, manifest and artifact checks from the current template capture path, then write the two fixed metadata files atomically enough for retry in the caller-owned VM directory. Return only the snapshot key; do not create a store abstraction, publish by rename or serialize unrelated launch state.
    • Done when:
      • finish_capture refuses an unproved or unsuccessful exit and every invalid OpenVMM snapshot condition covered by the existing capture checks.
      • A successful finish leaves key.bin and snapshot.json beside openvmm/ and returns their matching key.
      • Other VM-directory files are not treated as snapshot entries.
      • A failure that leaves the directory intact can be corrected and retried, and the moved three-entry directory opens through Snapshot::open.
    • Needs: SL2.10, in nvx-rs slimming 2.2.2: a VM stops only with a proved exit, and can be reattached #1100; SL2.12c, above; SL2.13, above.
    • Estimate: 310 core lines, 2 counted files. Past 600 core lines, stop and ask before continuing.

Tests. open refuses missing or extra entries, files that aren't regular or private, lengths that differ from the manifest's, a malformed manifest, and a key that doesn't match the manifest, and never reads snapshot.json. accepts takes a matching VM, and refuses, one at a time: a blockless root; processors or memory outside the boot size and the capacity; a processor count other than 1, 2, 4 or 8; unaligned memory; a longer image, or other block sizes; a scratch of another geometry; another NIC presence or address; a listener for a snapshot captured Disconnected; another backend, CPU profile or runtime identity; and an ineligible time contract. A capture launch requires a blockless root and refuses an export; creates the sparse placeholder and dormant scratch files and verifies their geometry; renders exactly one nvx_snapshot_tier=platform token and no root, view, mapping or workload token; refuses by name every token outside OpenVMM's platform allowlist; captures into <vm_dir>/openvmm; attaches the NIC under deny-all; and returns a CaptureContext that round-trips through serde with those checked values. finish_capture requires the proved exit, checks the outcome and OpenVMM's snapshot, writes key.bin and snapshot.json beside openvmm/ in the VM directory, leaves the VM directory's other files outside the snapshot, and can be retried after a failure that left the directory intact. Once the caller has moved only those three entries into a separate private directory, Snapshot::open accepts it.

Done when: every PR above has merged, with dev green.

Activity

  1. esaurez commented on Nov 10, 2026

    @esaurez
    ContributorAuthor

    Workstream source/progress checkpoint - 2026-11-10 UTC

    Checked boxes below mean merged scope, not merely an implemented first iteration. Implementation branches are source checkpoints, not automatic qualification, Ready or merge approval. Some starting inputs are frozen/anonymous commits rather than standalone branches; their exact SHAs and known merged producer refs are recorded explicitly.

    The original technical scope remains authoritative, supplemented only by any portable technical clarification in this issue. No machine-specific execution policy is part of this checkpoint.

  2. esaurez commented on Nov 11, 2026

    @esaurez
    ContributorAuthor

    Retiring the historical multi-input aggregate #1449 under the sequential-PR workflow. This records its snapshot-compatibility/owned-command input scope before closure, not completion of SL2.13 or this issue.

    Keep branch esaurez/issue1102-input-join-20261110-6789d491 and source dfaca512c5cbcf1646938b195ad61eded25d06ea. Related supporting input composition is published in #1465 at 0eaadb9c51d74e4de200710dc64df1f822bb9aa0; that separate active work is not retired here.

    This is workflow-directed retirement, not a claim that every unique change is merged, production SL2.16 is complete, tests passed, or remaining issue scope is waived.

  3. ppenna commented on Nov 11, 2026

    @ppenna
    Contributor

    All five PRs of this group have merged into feature/nvx-refactor: SL2.12a (#1345), SL2.12b (#1440), SL2.12c (#1443), SL2.13 (#1454) and SL2.14 (#1455).

    The feature branch's post-merge CI run at 394c89025e, which contains all five, passed: https://github.com/microsoft/nvx/actions/runs/42952780739. Closing this group as complete; dev receives it when feature/nvx-refactor lands there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions