Skip to content

nvx-rs slimming 2.2.1: a VM is created in a private directory, with its restore-ready listener, then spawned #1099

Closed

Description

Parent: #1080, in #1076. Needs: #1091, #1098. Unblocks: #1100, #1101, #1102, #1103, #1118.

Goal. A caller creates a VM in a private directory, with fixed, documented names; inspects the prepared launch, or hands it to its own launcher; and spawns it with host::vm::spawn, which leaves no child behind if it fails. The restore-ready listener, a Linux socket in the VM directory or a Windows named pipe, accepts only the VM's own process.

Notes.

  • The Windows spawn lock is private to host::vm::spawn. A Windows caller keeps its own spawns out of a VM's start, for example with its own lock around host::vm::spawn and its other spawns.
  • Persist what recovery needs before the spawn. nvx never writes the Capability. A caller that recovers or reattaches journals Prepared::capability() and Prepared::outputs() before the spawn, then records the process reference once the spawn succeeds. Capability's Debug is redacted, and as_bytes() gives the credential, to attach a session or to persist it.
  • Restores bind the listener later. nvx-rs slimming 2.3.3: a VM restores from a snapshot, through a sans-I/O restore machine #1103 binds it before a restore's spawn.

PRs, in order. Each PR has one goal, leaves dev green, and names in its description each old test of its area as kept, rewritten or retired, with each change that check_test_changes.py reports. Its tests follow the repository's test layout: a module's unit tests, and the code that only they use, sit in its tests.rs or tests/ and move with it. The estimates count core lines (code, not comments, docs or tests) and counted files; each PR records its measured values. Each PR does what its Scope says and no more: a case, input or layout that its Scope doesn't name, and whatever "Not in this PR" lists, is out of scope. Address every review comment within the scope, and ask before acting on one that asks for more. A PR whose measured core lines pass the stop that its Estimate names (twice the estimate, at most 600) stops and asks before continuing.

  1. SL2.7: A VM's files have fixed, documented names, in a private directory.

    • Objective: Callers need one documented, private directory contract for every VM instead of store-owned paths. Linux and Windows callers can open or create that directory, derive every launch output from fixed names, and retain the generated capability without the library writing the credential to disk.
    • Contents: host::vm::{VmDir, VmOutputs, Capability} and host::vm::create_private_dir: VmDir::{open, path}; the private-directory check and creation (0700, or an owner-only protected DACL); the reserved names; VmOutputs { log, outcome, endpoint, console }, with the Windows pipe names; and Capability::as_bytes with a redacted Debug. A non-private directory is an Io error, as a non-private file is in today's stores.
    • Scope:
      • The new host::vm module and the existing Linux and Windows private-directory helpers.
      • VmDir::{open, path} and create_private_dir: current-user ownership with mode 0700 on Linux, and an owner-only protected DACL on Windows.
      • The fixed and reserved VM-directory names, including log, outcome, control, console, restore-ready, snapshot and capture entries; Windows control, console and restore-ready outputs use fresh named-pipe names.
      • VmOutputs { log, outcome, endpoint, console } and Capability::as_bytes(), with capability bytes omitted from Debug and never written to a capability file.
    • Not in this PR:
      • A general secure-directory library, recursive permission repair or support for shared VM directories: callers use one private directory per VM.
      • Configurable output names, name discovery or arbitrary files beneath the reserved .openvmm prefix: no caller needs them.
      • Creating launch outputs, rendering arguments or starting a process: SL2.8 and SL2.9 add those lifecycle steps.
      • Persisting or encrypting capabilities: the caller decides how to save the returned bytes.
    • Approach: Reuse the platform ownership, mode and ACL checks already used by the stores, tightening Windows creation to the documented protected owner-only DACL. Keep the names as constants and direct path joins in host::vm; use the existing random named-pipe pattern on Windows. Its tests reuse the ownership, store-directory and Windows set_access helpers from the platform's private helpers. Do not add a directory schema, registry or credential file.
    • Done when:
      • VmDir::open refuses a non-private directory with an Io error, as the stores refuse a non-private file, and create_private_dir creates mode 0700 on Linux and an owner-only protected DACL on Windows.
      • Tests cover every fixed and reserved name and the three Windows pipe-name forms.
      • Capability::as_bytes() returns the credential, Debug redacts it, and creating or opening a VM directory writes no capability file.
    • Needs: SL0.4, in nvx-rs slimming 1.1.1: the public API is checked, and NVX's programs are hidden from it #1091.
    • Estimate: 260 core lines, 5 counted files. Past 520 core lines, stop and ask before continuing.
  2. SL2.8: A cold launch can be created, and inspected before it runs.

    • Objective: A cold VM launch must be fully checked and inspectable before any process starts. create produces an owned prepared launch, removes its partial outputs on failure, and rejects unsupported restores or an oversized final command line before returning control to the caller.
    • Contents: host::vm::create with from: None, which requires VmConfig::boot, and Prepared (capability, outputs, inherit, log, arguments), which owns the opened log. It creates the outputs and the capability, and a failed create removes what it made. SL2.4c's renderer becomes internal. host::snapshot::Snapshot starts as an opaque shell, and create refuses from: Some until SL2.16. create checks the complete command line, with no reserve, before it returns, and refuses one that doesn't fit, so no spawn starts.
    • Scope:
      • The new host::vm::create and Prepared, using the renderer from nvx-rs/src/host/launch.rs, which becomes internal.
      • Cold launches only: from: None requires VmConfig::boot; the new opaque host::snapshot::Snapshot shell makes from: Some an explicit refusal until SL2.16.
      • Prepared exposes the capability, outputs, inherited handles, opened log and rendered arguments, and owns the opened log without reopening it.
      • Creation of the capability and output paths, with rollback of everything this call made when any creation or validation step fails.
      • A final command-line fit check with the actual runtime and VM-directory paths and no reserve, before create returns.
    • Not in this PR:
      • Restore planning or launch from Snapshot: SL2.16 fills that path.
      • Starting, supervising or stopping OpenVMM: SL2.9 and SL2.10 own those process operations.
      • An external-launcher command object or owned platform handles: SL2.8b adds that transfer.
      • A general filesystem transaction or rollback framework: cleanup covers only entries this create invocation made.
      • Caller journals, launch recovery or a durable VM record: callers own persistence.
    • Approach: Call the existing whole-configuration argument renderer, then add the fixed paths and perform the existing platform command-line calculation on the complete command. Open the log once and keep it in Prepared; track only entries created by this call for direct cleanup on error. Keep Snapshot opaque and reject Some rather than designing restore behavior early.
    • Done when:
      • A cold create(from: None) refuses a configuration without Boot and returns an inspectable Prepared for a valid one.
      • A failure after entries are created removes every entry that this call made.
      • create(from: Some) is refused until SL2.16.
      • A complete command line over the Windows limit is refused before any spawn, on every host.
    • Needs: SL2.4c, in nvx-rs slimming 2.1.2: the kernel command line and OpenVMM's arguments come from the configuration alone #1098; SL2.7, above.
    • Estimate: 340 core lines, 4 counted files. Past 600 core lines, stop and ask before continuing.
  3. SL2.8b: A caller can spawn a prepared launch itself.

    • Objective: A caller with its own launcher needs the exact prepared launch without reconstructing paths or reopening resources. Prepared::into_command() transfers one complete cold-launch command and its owned resources while documenting the same launch contract as host::vm::spawn.
    • Contents: Prepared::into_command() → VmmCommand: the program, the arguments, the working directory, the capability, the outputs, the handles to allow, the opened log and the Windows breakaway setting, handed over without reopening anything, with spawn's contract in its docs; InheritedHandle (new) owns its OS handle and implements From<OwnedFd> on Unix and From<OwnedHandle> on Windows. A compiled external-launcher example covers a cold launch.
    • Scope:
      • Prepared::into_command() and VmmCommand: program, arguments, working directory, capability, outputs, inherited handles, opened log and the Windows breakaway setting.
      • Ownership transfer without reopening the log or inherited resources by path.
      • New InheritedHandle, with ownership-preserving conversions from OwnedFd on Unix and OwnedHandle on Windows.
      • A compiled external-launcher example that consumes VmmCommand and launches a cold VM on the repository's Linux and Windows host paths.
    • Not in this PR:
      • A reusable launcher, daemon, process registry or supervision framework: the caller supplies its own spawn implementation.
      • Shell command construction, quoting, environment-variable support or arbitrary standard streams: OpenVMM receives the prepared program, arguments and resources directly.
      • Borrowed raw descriptor or handle conversions: InheritedHandle owns what it transfers.
      • A restore-ready listener in the external-launcher example: SL2.16 extends the command and example for restores.
      • Changing host::vm::spawn: SL2.9 implements the library launcher.
    • Approach: Consume Prepared and move its fields into one VmmCommand; do not clone, rediscover or reopen resources. Wrap only the platform owned descriptor or handle types that callers already use, and keep launch-policy decisions outside the crate. Adapt the existing spawn contract as documentation rather than introducing a launcher trait.
    • Done when:
      • into_command() hands over the same opened log and each owned inherited handle without reopening either.
      • Unix and Windows compile-time tests cover conversion from their standard owned descriptor or handle types.
      • The compiled external-launcher example consumes a prepared command and launches a cold VM.
    • Needs: SL2.8, above.
    • Estimate: 160 core lines, 2 counted files. Past 320 core lines, stop and ask before continuing.
  4. SL2.17: The restore-ready event arrives, on both platforms, from the VM's process only.

    • Objective: Restore coordination needs one exact readiness event and a listener whose peer is the launched VM, not merely another process of the same user. Linux and Windows callers can hand the listener to blocking or external I/O while retaining process-identity verification.
    • Contents: protocol::restore_ready::parse_ready_event, the parser of OpenVMM's OPENVMM_RESTORE_READY_V1\n event, and restore::ReadyListener with into_std: the Linux socket in the VM directory, and the Windows pipe (as in nvx-rs/src/host/platform/windows/endpoint.rs, lines 48–150 at 4f9a3214). The listener checks that the client that connects is the VM's process, against the held process reference.
    • Scope:
      • New protocol::restore_ready::parse_ready_event for the exact OPENVMM_RESTORE_READY_V1\n payload.
      • New host::restore::ReadyListener, adapted from the clone restore-ready code in the Linux and Windows endpoint modules.
      • A mode-0600 Unix socket in the VM directory on Linux, and a fresh first-instance, owner-only named pipe on Windows.
      • Acceptance only when the connecting client matches the held VM process reference, using Linux peer credentials or the Windows named-pipe client process ID.
      • ReadyListener::into_std() transfers the platform listener for a caller's own blocking or asynchronous driver.
    • Not in this PR:
      • A version-negotiated or extensible readiness protocol: OpenVMM emits one fixed event.
      • The restore state machine, activation messages or retry policy: SL2.18a owns restore sequencing.
      • Binding a listener during create(from: Some): SL2.16 connects this primitive to restore launch.
      • A general local peer-authentication framework or support for TCP endpoints: the repository uses Unix sockets and Windows named pipes.
      • An asynchronous runtime adapter: callers adopt the standard listener returned by into_std().
    • Approach: Extract the existing clone listener mechanics instead of designing a new transport. Parse by exact byte equality, retain the platform listener and held process identity, and perform the existing peer-PID check at acceptance. Expose the standard listener as the only runtime transfer; do not add callbacks, executors or a generic event codec.
    • Done when:
      • parse_ready_event's cases in protocol/restore_ready/tests.rs accept the exact OpenVMM event and reject truncated, extended or otherwise malformed input.
      • ReadyListener::into_std() transfers a usable listener, and host/restore/tests.rs keeps the Linux socket and Windows pipe cases adapted from the platform tests.
      • On Linux and Windows, the listener accepts the launched VM process and refuses a different process.
    • Needs: SL2.7, above.
    • Estimate: 350 core lines, 5 counted files. Past 600 core lines, stop and ask before continuing.
  5. SL2.9: A created launch runs, and leaves no child if it fails.

    • Objective: A prepared launch needs one library spawn that either returns a detached VM owned by the caller or leaves no child behind. It preserves the repository's Linux umask behavior and Windows log-job claim while keeping the Windows inheritance lock internal.
    • Contents: host::vm::spawn → Launched { process, outputs, capability }, on nvx-rs/src/host/spawn.rs and the platforms' spawn modules, with the Windows spawn lock private to it. If the spawn fails, no child is left running; if it succeeds, the caller owns a VM that outlives it. It keeps what dev added: on Linux, the child takes the caller's umask without the owner's bits, and the caller's own umask doesn't change; on Windows, OpenVMM starts in its log's job, a taken or untrusted job name is refused, and a process that something else spawns during the spawn doesn't get the VM's claim.
    • Scope:
      • host::vm::spawn and Launched { process, outputs, capability }, built from nvx-rs/src/host/spawn.rs and the Linux and Windows spawn modules.
      • Failure cleanup after a child exists: terminate it, wait for its exit and return only when no spawned child remains.
      • Successful launches detach from the caller while returning the verified Process, outputs and capability.
      • Linux keeps the caller's umask with owner bits cleared for the child, without changing the caller's own umask.
      • Windows keeps the log-named job, rejects a taken or untrusted job name, isolates the claim from unrelated spawns, and holds its inheritance lock only inside spawn.
    • Not in this PR:
      • A caller-wide spawn lock or interception of every process start: Windows callers coordinate their own other spawns.
      • A generic process supervisor, restart policy, daemon manager or child registry: this function starts one OpenVMM process.
      • Launch journals, interrupted-launch recovery, log liveness claims or durable process records: callers build those policies.
      • External-launcher execution: callers use the VmmCommand from SL2.8b.
      • Stopping or reattaching a running VM: SL2.10 and SL2.11b add those operations.
    • Approach: Refactor the existing detached OpenVMM spawn rather than introducing a process-management layer. Keep its inherited-handle allowlist, platform detachment, Linux umask adjustment and Windows log job, and hold the Windows lock only across handle inheritance and process creation. If a later setup step fails, kill the returned child and confirm its exit. Its cases move from host/spawn/tests.rs to host/vm/tests.rs, using the platforms' private spawn-descriptor and umask hooks.
    • Done when:
      • An injected failure after child creation ends that child and returns with no child left running.
      • A successful spawn returns a verified process and a VM that outlives its caller.
      • Linux tests preserve the child umask rule and show that the caller's umask is unchanged.
      • Windows tests preserve the log's job, reject taken or untrusted job names, and keep an unrelated concurrent process out of the VM's claim.
    • Needs: SL2.8, above; SL2.17, above.
    • Estimate: 300 core lines, 4 counted files. Past 600 core lines, stop and ask before continuing.

Tests. VmDir::open refuses a directory that isn't private, and create_private_dir creates one (0700 on Linux, an owner-only protected DACL on Windows). The tests cover every fixed and reserved output name, the Windows pipe names, Capability::as_bytes() and its redacted Debug, and check that no capability file is written. A cold create(from: None) refuses a missing Boot. A failed create removes everything that it made, and create refuses from: Some until #1103. Prepared::into_command() hands over the opened log and the owned inherited handles without reopening them, and the compiled external-launcher example launches a cold VM. protocol::restore_ready accepts the exact event and rejects malformed input; ReadyListener::into_std() hands the listener over; and, on Linux and Windows, the listener accepts the VM's process and refuses any other. A failure injected after the child exists shows that host::vm::spawn ends it, and returns with no child left running. create refuses a complete command line over Windows' limit before any spawn. spawn keeps the caller's umask rule on Linux, and the log's job, the refusal of a taken or untrusted job name and the claim's isolation on Windows.

Done when: every PR above has merged, with dev green.

Privacy-contract clarifications

  • Check every component of the VM-directory path, including a symlink followed by an otherwise valid-looking trailing path. Reject an unsafe existing directory; do not repair its permissions or broaden unrelated store/template policies.
  • On Windows, owner-only means a current-user owner and a present, non-null, protected DACL with no grants to other principals. Full-access checks use bitmask containment, not numeric ordering: a high-valued partial mask is not full access. Keep the existing creation/inheritance behavior rather than adding a general ACL framework.

These clarify the existing SL2.7 privacy boundary; they do not add a new filesystem service.

Activity

  1. esaurez commented on Nov 10, 2026

    @esaurez
    ContributorAuthor

    Workstream source/progress checkpoint - 2026-11-10 UTC

    Checked boxes below mean merged scope, not merely an implemented first iteration. Implementation branches are source checkpoints, not automatic qualification, Ready or merge approval. Some starting inputs are frozen/anonymous commits rather than standalone branches; their exact SHAs and known merged producer refs are recorded explicitly.

    The original technical scope remains authoritative, supplemented only by any portable technical clarification in this issue. No machine-specific execution policy is part of this checkpoint.

  2. ppenna commented on Nov 11, 2026

    @ppenna
    Contributor

    All five PRs of this group have merged into feature/nvx-refactor: SL2.7 (#1280), SL2.8 (#1342), SL2.8b (#1434), SL2.17 (#1297) and SL2.9 (#1403, which merged into SL7.2's branch and reached feature/nvx-refactor with #1339; both of its commits are on the branch).

    The feature branch's post-merge CI run at 394c89025e, which contains all five, passed: https://github.com/microsoft/nvx/actions/runs/42952780739. Closing this group as complete; dev receives it when feature/nvx-refactor lands there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions