Skip to content

nvx-rs slimming 3.1.3: the CLI keeps its own sandbox store #1108

Closed

Description

Parent: #1083, in #1076. Needs: #1100. Unblocks: #1110, #1114.

Goal. the CLI keeps its own sandbox store, on the host's private-file helpers (host::platform, host::sys): the layout and the lock; a launch journal, written before the spawn; the runtime record, which holds SavedVm; the capability file; the launch's liveness claim on its log; the endpoint-owner lookup; and the recovery of interrupted launches by the next command.

PRs, in order. Each PR has one goal, leaves dev green, and names in its description each old test of its area as kept, rewritten or retired, with each change that check_test_changes.py reports. Its tests follow the repository's test layout: a module's unit tests, and the code that only they use, sit in its tests.rs or tests/ and move with it. The estimates count core lines (code, not comments, docs or tests) and counted files; each PR records its measured values. Each PR does what its Scope says and no more: a case, input or layout that its Scope doesn't name, and whatever "Not in this PR" lists, is out of scope. Address every review comment within the scope, and ask before acting on one that asks for more. A PR whose measured core lines pass the stop that its Estimate names (twice the estimate, at most 600) stops and asks before continuing.

  1. SL3.2a: The CLI owns its sandbox layout and lock.

    • Objective: The CLI must own the state directory and lifecycle lock used by its sandbox commands before it can stop depending on nvx's general host store. This PR adds only that layout and lock, with private directories and files on Linux and Windows.
    • Contents: host::cli::store: the directory layout and the lock, adapted from nvx-rs/src/host/store/, on the host's private-file helpers (host::platform, host::sys). The store's source is split along its sections (names, locked, record, vmm and others), and the adaptation reads all of them. Parts whose first command caller lands later carry expect(dead_code) with that reason.
    • Scope:
      • Add nvx-rs/src/host/cli/store/, split into focused modules such as names, locked and the root module, adapted from the corresponding sections of nvx-rs/src/host/store/.
      • Define the fixed sandbox paths needed by the CLI's provision, start, exec, stop and deprovision commands, with one lock per sandbox state directory.
      • Use the host's private-directory, private-file, regular-file, link-refusal and file-lock helpers (host::platform, host::sys) rather than copying platform system calls.
      • Create and open only directories private to the current user, and create store files privately on Linux and Windows.
      • Make the lock exclude concurrent lifecycle holders and support the command deadlines already used by the CLI.
      • nvx-rs/src/host/cli/store/tests.rs defines local released and Windows junction helpers adapted from host::tests; the CLI tests do not import nvx's cfg(test) module.
    • Not in this PR:
      • Launch journals, runtime records or capability files; SL3.2b adds them after the layout and lock exist.
      • Log claims, endpoint-owner lookup or interrupted-launch recovery; SL3.3a and SL3.3b add those mechanisms.
      • Template generations, clone leases, clone bindings, capture transactions, failure or quarantine records, or migration of old record formats; the CLI's commands do not need the old stores' machinery.
      • A caller-file registry, pluggable file-name ownership or a reusable store framework; this store has one owner and a fixed layout.
      • Network-filesystem recovery, lingering-directory coordination or arbitrary files added by external callers; the CLI uses its own local state directory.
    • Approach: Extract the old store's directory validation, fixed names and exclusive lock, then use the host's private-file helpers. Keep only the operations the CLI's five sandbox lifecycle commands call; mark code awaiting those callers with expect(dead_code), and do not port whole files, generic ownership models or old compatibility paths.
    • Done when:
      • Store tests create and reopen a sandbox directory and its lock privately on Linux and Windows.
      • Opening refuses a sandbox directory that is not private to the current user, and refuses a lock that is not regular, is a link or is accessible by another user.
      • One lock holder excludes another until release or the caller's existing deadline.
      • The five sandbox lifecycle commands can address their fixed state paths through host::cli::store.
    • Needs: nothing.
    • Estimate: 250 core lines, 3 counted files. Past 500 core lines, stop and ask before continuing.
  2. SL3.2b: The CLI journals its launches, and saves what reattach needs.

    • Objective: A CLI process can die between preparing and recording a VM, so the CLI needs durable facts that the next command can inspect. This PR writes a pre-spawn launch journal and capability, then records the exact SavedVm needed for verified reattach.
    • Contents: The launch journal, written from Prepared::capability() and outputs() before the spawn; the runtime record that holds SavedVm; the capability file. Store transitions whose first command caller is SL3.5b carry expect(dead_code) with that reason.
    • Scope:
      • Add the launch journal to nvx-rs/src/host/cli/store/, populated from Prepared::capability() and outputs() and made durable before spawn.
      • Add the private capability file containing the prepared launch's capability bytes.
      • Add the runtime record containing every field of the SavedVm returned for the launched VM.
      • Publish each JSON record whole within the locked sandbox directory, using the existing host private-file helpers (host::platform, host::sys).
      • Provide store launch transitions that write the journal and capability before spawn, durably add the process reference to the journal after spawn, and publish the runtime record only when the launch commits; SL3.5b wires the sandbox commands to these transitions.
    • Not in this PR:
      • Deciding whether a journaled VM still runs; SL3.3a adds the log claim and endpoint-owner lookup.
      • Recovering, terminating or cleaning an interrupted launch; SL3.3b consumes these records.
      • Resuming a partially completed launch or retrying a spawn; no command requires either behavior.
      • Old artifact identities, clone records, leases, template generations, configuration records or record-format migration; they are not inputs to SavedVm.
      • A generic transaction log, schema framework or database; the store has one journal, one runtime record and one capability file.
    • Approach: Serialize the fixed journal and runtime records directly from Prepared and SavedVm under the store lock. Adapt the old store's small private atomic-replacement code on the host's file-privacy primitives; mark transitions awaiting sandbox start with expect(dead_code), and do not add a schema framework or old-store machinery.
    • Done when:
      • A launch test observes a durable journal and capability before the spawn hook runs.
      • The journal contains the prepared capability and every output needed to identify the pending launch.
      • The committed runtime record round-trips every SavedVm value used by verify and reattach.
      • After spawn, the process reference is durable in the journal before the runtime record replaces it.
      • Record and capability tests create private regular files on Linux and Windows and refuse links, non-regular files and files accessible by another user.
    • Needs: SL3.2a, above; SL2.11b, in nvx-rs slimming 2.2.2: a VM stops only with a proved exit, and can be reattached #1100.
    • Estimate: 450 core lines, 4 counted files. Past 600 core lines, stop and ask before continuing.
  3. SL3.3a: The CLI can tell whether a recorded VM still runs.

    • Objective: A journal alone cannot distinguish a launch that never spawned from one whose process still runs. This PR gives the CLI two bounded proofs: the inherited claim on the launch log, and a verified owner of the recorded control endpoint.
    • Contents: The launch's liveness claim on its log: while it is held, a launch process may still run; once it is released, the launch has exited. The endpoint-owner lookup holds the process that serves the endpoint, then checks that the endpoint still names it; on Linux, that process must also hold the claimed log.
    • Scope:
      • Add a per-launch liveness claim on the opened OpenVMM log before spawn; holding it means a launch process may still run, and release proves exit.
      • Pass the claimed log through the existing prepared launch so OpenVMM inherits it on Linux and Windows.
      • Add an endpoint-owner lookup that holds the serving process, then checks that the endpoint still names that same process.
      • On Linux, accept an endpoint owner for an unrecorded launch only when that process also holds the claimed log.
      • On Windows, keep the per-log job-object claim and its identity checks, with the inherited-log-handle fallback when no trustworthy job is available.
      • Keep these mechanisms inside the CLI's store and use the process and private-file primitives already provided by nvx.
    • Not in this PR:
      • Choosing recovery actions or deleting records; SL3.3b applies these proofs to interrupted launches.
      • Scanning all processes, guessing from a process name or signaling an endpoint owner before verification; no recovery path may trust those shortcuts.
      • Monitoring VMs in a background service or keeping a registry outside the sandbox directory; commands inspect state under the existing lock.
      • Claims for scratch images, snapshots, templates or clone leases; this PR covers only a launch's log.
      • Support for remote state directories or alternate log transports; the CLI uses the local private sandbox directory.
    • Approach: Adapt only the old store's log-claim and endpoint-owner portions to the new fixed records. Build on existing OS file locks, process handles and endpoint inspection; do not port the old supervisor, process enumeration, lease logic or a general liveness abstraction.
    • Done when:
      • A held log claim reports that the launch may run, and its release proves the launch has exited.
      • The endpoint-owner lookup holds the server and rejects it if the endpoint changes before verification completes.
      • Linux tests reject an endpoint server that does not also hold the claimed log.
      • Linux tests cover the file-lock claim and endpoint/log match; Windows tests cover the per-log job object, its identity checks and the inherited-log-handle fallback.
    • Needs: SL3.2b, above.
    • Estimate: 250 core lines, 3 counted files. Past 500 core lines, stop and ask before continuing.
  4. SL3.3b: The next command recovers or cleans up an interrupted launch.

    • Objective: After a CLI process is interrupted, the next sandbox command must resolve its launch journal without risking an unrelated process. This PR cleans launches proved absent or ended, stops only a fully verified VM, and preserves indeterminate evidence for a later command.
    • Contents: The recovery of interrupted launches, which never signals an unverified process: a launch that stopped before its spawn is cleaned up once its log claim is released; an unrecorded VMM is ended only when both the endpoint and the log claim verify it, and otherwise its journal stays, as indeterminate; a recorded VMM is opened by its boot, process ID and start time, and only that process is ended. The journal and the capability file go only once the launch is proved exited or ended.
    • Scope:
      • Run interrupted-launch recovery under the sandbox lock before the next command changes that sandbox's state.
      • For a journal left before spawn, clean it only after the log claim is released.
      • For an unrecorded VMM, end it only when the endpoint owner and log claim both verify the same launch; otherwise keep the journal as indeterminate.
      • For a recorded VMM, open it by the saved host boot, process ID and start time, and end only the process that matches all three.
      • Remove the journal and capability only after the launch is proved exited or the verified VMM has ended.
    • Not in this PR:
      • Signaling a process found only by ID, endpoint, executable name or log access; none alone verifies the launched VMM.
      • Deleting an indeterminate journal, waiting forever or guessing that a slow endpoint means no VM; a later command may retry recovery.
      • Resuming the interrupted launch, replaying its spawn or reconstructing command options; recovery ends or cleans it.
      • Recovering committed runtime records through this journal path; normal verify and reattach use their saved SavedVm.
      • Old clone, lease, template, quarantine, failure-record or report-staging cleanup; the CLI's store does not keep those features.
    • Approach: Implement a small recovery decision over the journal state and the proofs from SL3.3a. Keep evidence rather than act without proof, and retain the existing one-second Linux claim recheck so a transient descriptor inherited across a fork is not mistaken for the launch; do not scan processes, run a daemon or port old-store cleanup machinery.
    • Done when:
      • A launch interrupted before spawn is cleaned after its released log claim proves no process remains.
      • A launch interrupted after spawn ends only the VMM verified by both its endpoint and log claim.
      • A launch interrupted after process recording opens and ends only the process matching its boot, ID and start time.
      • An unverified or still-ambiguous launch keeps its journal and capability as indeterminate, and no process is signaled.
      • Records are removed only after tests prove the launch exited or the verified VMM ended.
    • Needs: SL3.3a, above; SL2.10b, in nvx-rs slimming 2.2.2: a VM stops only with a proved exit, and can be reattached #1100.
    • Estimate: 350 core lines, 2 counted files. Past 600 core lines, stop and ask before continuing.

Tests. A launch's journal, with Prepared::capability() and outputs(), is durable before the spawn, and its runtime record keeps every SavedVm value. The next command recovers a launch interrupted before its spawn, after it, and after its process was recorded: it cleans up a launch that ended, ends only a verified VMM, and keeps the journal of an indeterminate launch; no other process is signaled. The log claim proves that an unrecorded VMM exited; the endpoint-owner lookup counts a server only while the endpoint still names it and, on Linux, only if it holds the claimed log. The store creates and opens only a directory private to the current user, and creates its files privately; it refuses a lock, record or capability that isn't a regular file, is a link, or another user may access; and its lock excludes other holders.

Done when: every PR above has merged, with dev green.

Activity

  1. changed the title [-]nvx-rs slimming 3.1.3: nvx-cli keeps its own sandbox store[/-] [+]nvx-rs slimming 3.1.3: the CLI keeps its own sandbox store[/+] on Nov 3, 2026
  2. esaurez commented on Nov 10, 2026

    @esaurez
    ContributorAuthor

    Workstream source/progress checkpoint - 2026-11-10 UTC

    Checked boxes below mean merged scope, not merely an implemented first iteration. Implementation branches are source checkpoints, not automatic qualification, Ready or merge approval. Some starting inputs are frozen/anonymous commits rather than standalone branches; their exact SHAs and known merged producer refs are recorded explicitly.

    The original technical scope remains authoritative, supplemented only by any portable technical clarification in this issue. No machine-specific execution policy is part of this checkpoint.

  3. ppenna commented on Nov 11, 2026

    @ppenna
    Contributor

    All four PRs of this group have merged into feature/nvx-refactor: SL3.2a (#1259), SL3.2b (#1433), SL3.3a (#1436) and SL3.3b (#1446).

    The feature branch's post-merge CI run at 394c89025e, which contains all of them, passed: https://github.com/microsoft/nvx/actions/runs/42952780739. Closing this group as complete; dev receives it when feature/nvx-refactor lands there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions