nvx-rs slimming workstream: make nvx a library of mechanisms and value types #1076

Objective. Make nvx a library of mechanisms and value types for microVMs, which NVX's own binaries and downstream users build on, and release it as a breaking prerelease. Today nvx also keeps durable state (sandbox and template stores, leases, clone records and quarantine), NVX's CLI, an exec engine with the guest's process modules, the guest's root assembly, and NVX's own control protocol. After this workstream, nvx keeps the mechanisms that every user needs: a VM's configuration and launch, its snapshots and restores, its transport, and the guest's capture and activation. Each user keeps its own state and policy. NVX's CLI is the first such user, and NVX's Rust PID 1, nvx-init, the reference for a guest init. NVX's own programs stay in nvx, private to it: the public API holds only what users need. The release is not backward compatible: users that need alpha.7 stay on it.

Today (dev e42b1795, published as nvx 0.1.0-alpha.7) and after:

Design.

• Mechanisms and value types. nvx keeps no durable state, runs no execution and builds no guest space. Its only lock is internal: the Windows spawn lock, private to host::vm::spawn. It spawns no helper or workload; the one exception is the time daemon, which TimeAbi::start_daemon forks in the guest.
• What a user gets from nvx:
  1. a VM through one entry point: a cold boot, a cold boot that may be captured, or a restore (host::vm::create);
  2. one snapshot type with a strict directory format (host::snapshot, host::config::CaptureConfig);
  3. clear outputs for the files that a VM uses: the log, the outcome report, the control socket or pipe, and the console (host::vm::{VmDir, VmOutputs});
  4. the transport, and reattaching to a running VM (protocol::nvxs, host::transport, host::vm::{verify, reattach});
  5. activation after a restore (activation, host::restore, guest::activation::Activation);
  6. the contracts of the guest's space: the tokens of the root, the view, the export and the shares, the mapping table, and the activation record (protocol::{root, view, export, share, maps}, activation);
  7. the guest's side of a snapshot, in one call that checks the devices, captures and repairs a restore (guest::snapshot::capture);
  8. one public error, whose kinds callers act on, policy description types, and one VM configuration for cold boots and restores (Error, host::{network, filesystem, config}).
• Described, not provided. The guest's space (the root, an image and its upper combined by overlayfs; its runtime filesystems; the switch into it; the view files and the host shares) and the executions are each init's own code. doc/guest-agent-guidelines.md describes both, with the space's interface and rules, and nvx-init implements them. NVX enforces none of it.
• One crate, with NVX's programs private in it. nvx stays the only package, published as today. Its public API is the library: the features host (the default), guest and serde. NVX's own programs live in it, private: the nvx-rs CLI and its stores, locks, path rules, image staging and guest tokens; nvx-init, NVX's Rust PID 1 for layered roots, and the managed agent for blockless VMs, with their plain-process exec core; NVXC v2, NVX's own protocol, with the host's control session and a blocking restore driver; the guest helper binaries (nvx-time, nvx-time-probe, nvx-ingress, nvx-console-pending, nvx-mmio-write, nvx-exit, nvx-device-io and nvx-reseed); the fake OpenVMM, for tests; and the OS layers. A binary can only call a public item, so each one calls a main that #[doc(hidden)] hides: these entry points are the only public items that NVX's programs add. The cli and testing features build the CLI and the fake OpenVMM. CI compares the public API, hidden items marked, with snapshots in the repository, so each PR shows its API change. Downstream users install the guest tools from the package: cargo install nvx --no-default-features --features guest --profile guest --bin nvx-ingress. Nightly publishing continues throughout: each nightly, then the release, takes the next prerelease number that crates.io hasn't used.
• No backward compatibility. Nothing reads alpha.7's records, keys or snapshots: a deploy starts from fresh state, and snapshots are captured again.

The API-ready commit. Phases 1 and 2 check the public API, hide NVX's programs from it, and add the new interface. The first commit at which all their PRs have merged is the API-ready commit; no PR of phases 4 and 5 merges before it. That commit still has the old surfaces that a migration needs, but it is not source- or behavior-compatible with alpha.7: activation record v2, the RAM upper's cap (now in the root's tokens, in activation::Upper::Tmpfs and in the hidden guest::root::Upper::Tmpfs), Process::wait, Process::open, report::read, and activation without its fence and announcer need downstream changes. The guest's space is still there, #[doc(hidden)] and documented as unstable. A downstream user pins it to migrate, does its own cleanup there, then crosses the removals once, at a commit that contains SL6.8c. NVX's PRs never wait for a downstream PR.

PR rules. Each PR:

• has one goal, which its group issue states as what is true once it merges, and leaves dev green;
• does what its Scope in the group issue says and no more: a case, input or layout that the Scope doesn't name, and whatever its "Not in this PR" lists, is out of scope;
• addresses every review comment within its scope, and asks before acting on one that asks for more;
• stops and asks before continuing once its measured core lines pass twice its estimate, or the 600-line limit if that comes first, unless it changes only code that leaves;
• changes at most 600 core lines, the non-blank lines of code that it adds or changes (comments, docs and tests don't count), and at most 5 counted files (a few may take 6 or 7, where a split would be harder to follow): the Rust files outside tests, build scripts, Cargo manifests, CI workflows and actions, and shell and Python scripts that it adds or edits (a file that it moves with only its import, visibility and link edits doesn't count, though those edited lines do; a file whose items it only makes crate-private or hidden, or gates by the feature that uses them, with the import edits that this forces, doesn't count, nor do those lines; nor do files that it only deletes, tests, docs, fixtures, lockfiles and generated files);
• keeps code where it is: code that only NVX's programs use becomes crate-private there, gated by the features of its users, and new code of a program goes under that program's module (host::cli, guest::init). A file that does move, moves with git mv, counting only the lines that it then edits;
• adds no code without a production caller in the same PR, except a temporary public entry point whose docs name the PR that ends it, or crate-private code whose caller comes in a later PR, which carries #[cfg_attr(not(test), expect(dead_code, reason = "…"))] naming that caller; nothing becomes public, even hidden, only to avoid that. Each PR passes cargo clippy -p nvx --lib --no-default-features -- -D warnings beside the crate's other checks;
• keeps the public-API snapshots in nvx-rs/public-api/ current: a PR that changes the public API updates them, and their diff shows the change. Nothing that only NVX's programs use becomes public, and each binary calls a hidden main;
• deletes consumers before their providers, and with them the crate-private code that the deletion leaves unused, which the compiler's dead-code check finds in CI's warning-denied build of each feature set;
• keeps, rewrites or retires each old test of the area that it changes, by name, and explains in its description each change to an existing test that python3 scripts/check_test_changes.py --base <its base> reports, the list behind the Test changes workflow's tests-modified label: a test that moves shows as removed and added, and the description pairs the two. An integration test that uses an item that the PR makes crate-private becomes a unit test of the item's owner, or tests through a binary, asserting the same things;
• follows the repository's test layout (doc/contribute.md, Test layout umbrella: move tests out of production code, and flag changes to existing tests #1207): a module's unit tests, and the code that only they use, live in its tests.rs or tests/ directory and move with it; a production file declares them only as #[cfg(test)] mod tests; and holds no other test code than the hooks that the seams of scripts/test-layout-allowlist.json allow; and a PR that moves, splits or deletes a file with seams updates the file's entry. check_test_layout.py passes;
• leaves BOOT_CONTRACT_VERSION and CONTROL_CONTRACT_REVISION as they are, with the revision's copies: nothing that this workstream changes is deployed, so no PR increments them, although contract's docs ask for an increment with each change. A test that pins a contract's tokens or features takes the new content, at the same version;
• updates the rustdoc, examples and docs that it affects;
• records its measured core lines and counted files in its description. The estimates in the group issues come from line counts at e42b1795, rechecked at 4f9a3214.

Code that leaves. A PR that changes only code that this workstream deletes from NVX, with the edits that remove references to it, has no line or file limit. It keeps one goal and one area, records its core lines and counted files, and changes nothing that stays and nothing that a later PR uses. Its issue and its description open with this note: "Leaving code only. This PR changes only code that this effort removes from NVX: area. Nothing that stays changes, and no later PR uses what it changes, so the size limits for kept code don't apply." Code that becomes crate-private stays in NVX, and follows the limits. No group issue mixes the two.

The test layout. #1207 moved NVX's tests out of the code that they test (#1188 and #1195–#1198, merged on 2026-11-02), so every PR of this workstream starts from a dev with the layout, and follows it.

Phases. Each phase ends at a milestone. Its subphases run in parallel lanes, as far as their groups' Needs allow; each group issue lists its PRs, in order, with their Needs.

Children and order. Each subphase issue lists its group issues, and each group issue its PRs. "Needs" names the subphases of earlier phases that a subphase's groups need; inside a phase, the subphase and group issues name their own.

Inside phase 2, #1080 needs #1079; #1081 needs #1079 and #1080; #1082 needs #1079 and #1081. Inside phase 3, the lanes meet at group level: the CLI's store, run and sandbox-lifecycle groups come before its snapshot commands, which come before its last switch to the new interface; the CLI's sandbox-lifecycle and managed-sandbox run groups, and its snapshot-command group, wait for nvx-init's capture, restore and layered boot; nvx-init's last capture-and-restore group waits for NVXC's group that adds the capture request, the activate message and the blocking restore driver; and the managed agent's group waits for the CLI's run group, whose shell /init reads the CLI's own tokens. Inside phase 4, #1087 needs #1086, and #1088 needs #1087. Inside phase 5, #1090 needs #1089.

Not in scope (later work):

• OpenVMM restoring any snapshot from any location, which would replace the hard links that a restore uses today;
• snapshot-and-continue, and captures of restored VMs;
• the paired snapshot tiers: a workload-start snapshot with a copied or reflinked scratch, and an instance checkpoint that claims its scratch;
• re-addressing a restored VM's NIC;
• snapshots of blockless VMs;
• kernel tuning tokens in a capture's command line (nvx-rs: boot templates with an adapter's kernel tuning parameters, such as mitigations=off #995);
• performance work.

Acceptance. nvx is released, and:

• its public API holds exactly the planned surface, and one hidden entry point per binary, as the checked snapshots show;
• CI builds each feature set and compiles the documented examples, and a checklist test covers each security requirement that the docs of filesystem, vm and snapshot state;
• before the test adapter is deleted, a case matrix in the repository lists each test of the adapter, of CI and of the docs, by name, as kept, rewritten or retired;
• smoke tests on real VMs create, spawn, reattach and stop a VM; capture a snapshot, open it and restore from it, with nvx-init as the guest; run concurrent restores of one snapshot under the CLI's per-snapshot lock; and check that a failed restore reports its attribution;
• the CLI runs its 12 cold cells (EROFS and GPT, with a scratch or a RAM upper, on KVM, MSHV and WHP) and its restores (nvx-rs: final acceptance for the image-backed runtime #624);
• the release CI checks the package's contents, installs the guest tools from it outside the repository with the guest profile, and runs nvx-ingress;
• the docs describe only the final library, and doc/migrate-from-alpha.7.md says what the release breaks.

Related:

• NVX host/guest library wiring workstream: move NVX's binaries and tools onto the shared host and guest libraries #816 (the NVX library wiring workstream) stays open, rescoped to what this workstream keeps of it, which this workstream's groups deliver: nvx-init (nvx-rs slimming, phase 3.3: nvx-init boots, captures and restores layered roots #1085; the old layered init goes in nvx-rs slimming, phase 4.2: the old code is deleted, consumers first #1087, and the space becomes private in nvx-rs slimming, phase 4.3: the code that only NVX's programs use, and the error codes, become private #1088), the CLI's image formats, uppers and protected paths (nvx-rs slimming, phase 3.1: the CLI switches to the new interface #1083), run's shares through the export planner (nvx-rs slimming, phase 3.1: the CLI switches to the new interface #1083), the exec lock (nvx-rs slimming, phase 3.1: the CLI switches to the new interface #1083), NVXC v2 (nvx-rs slimming, phase 5.1: NVXC v2 replaces v1 #1089), snapshots in NVX's binaries (nvx-rs slimming, phase 3.2: snapshots through NVXC and the CLI #1084, nvx-rs slimming, phase 3.3: nvx-init boots, captures and restores layered roots #1085), DNS through the view (nvx-rs slimming, phase 2.4: the guest's space as reference code, and the guest's activation and capture #1082), and a minimal acceptance gate (nvx-rs slimming, phase 4.1: acceptance on real VMs, then the test adapter goes #1086, nvx-rs: final acceptance for the image-backed runtime #624). Its obsolete child issues close, with these dispositions:
  • nvx-rs: give each guest build an identity from its build inputs, and package it #820: the runtime's labels already come from the build's provenance files or SHA256SUMS, and NVXC v2's Ready checks only the protocol version;
  • runtime adapter: the consumer-contract scenarios C2, C3, C6 and C7 on real VMs #859: the test adapter goes (nvx-rs slimming, phase 4.1: acceptance on real VMs, then the test adapter goes #1086);
  • Test the managed agent's direct-mode launch without privileges #601: the new exec core's error paths are tested on VMs as root, with no fork-safe stubs and no setpriv shim;
  • nvx-rs: decide whether adapters' layered launches need OpenVMM's workload lifecycle #861: layered roots are always managed, blockless run keeps both modes through the CLI's own tokens, and NVX stops passing OpenVMM's lifecycle and identity flags.
• nvx-rs: keep processes spawned during a launch from holding its VMM's log on Windows #966: the Windows spawn lock stays, private to host::vm::spawn; a caller keeps its own spawns out of a VM's start.
• nvx-rs: boot templates with an adapter's kernel tuning parameters, such as mitigations=off #995 is deferred: a capture accepts only OpenVMM's platform tokens.
• nvx-rs: let an adapter read the boot console of a template's clones #996: a restored VM has a console listener when its snapshot was captured with one.