Skip to content

nvx-rs slimming 3.3.1: nvx-init boots a layered root, and runs each exec as a plain process #1115

Closed

Description

Parent: #1085, in #1076. Needs: #1091, #1104, #1105. Unblocks: #1116, #1117.

Goal. nvx-init runs as PID 1 through hidden guest::init, ahead of the workloads. It builds a layered root's space from guest::{root, view, gpt} and the mapper (the root, the export and the shares, the runtime mounts with the time status, the view, then the switch), with no cgroups, no namespaces and no limits; it runs each exec as a plain process in its own process group; and it provides the capture's sweep and the restore's announcer step that #1117 calls.

Notes.

  • Two helpers. nvx-init's helpers are the time daemon, which the shell /init's nvx-time boot forks, and, when listeners are published, the announcer /sbin/nvx-ingress. nvx-init runs no other guest tool: the capture's repair reseeds in process, and nvx-init powers off itself.
  • Two steps wait for their callers. The capture's sweep (SL7.5c) and the restore's announcer step (SL7.5d) carry expect(dead_code) whose reasons name SL7.7a and SL7.7b, their callers in nvx-rs slimming 3.3.3: nvx-init captures and restores, and boots the new launches' layered roots #1117.

PRs, in order. Each PR has one goal, leaves dev green, and names in its description each old test of its area as kept, rewritten or retired, with each change that check_test_changes.py reports. Its tests follow the repository's test layout: a module's unit tests, and the code that only they use, sit in its tests.rs or tests/ and move with it. The estimates count core lines (code, not comments, docs or tests) and counted files; each PR records its measured values. Each PR does what its Scope says and no more: a case, input or layout that its Scope doesn't name, and whatever "Not in this PR" lists, is out of scope. Address every review comment within the scope, and ask before acting on one that asks for more. A PR whose measured core lines pass the stop that its Estimate names (twice the estimate, at most 600) stops and asks before continuing.

  1. SL7.5a: nvx-init runs as PID 1, ahead of the workloads.

    • Objective: nvx-init becomes the VM's PID 1 and owns signal handling, child reaping, console handling, exit statuses, and shutdown. It gives itself, the time daemon, and the optional announcer higher priority than workloads, and ends the VM if a registered helper fails.
    • Contents: The PID 1 core in guest::init, a hidden module whose only public item is #[doc(hidden)] pub fn main: signals, reaping, the console, exit statuses, and shutdown by sync and power-off, which it does itself; oom_score_adj -1000 and nice -10 for PID 1; the registry of helpers, the two processes whose IDs the shell /init hands over (the time daemon, which nvx-time boot forks, and, when listeners are published, the announcer /sbin/nvx-ingress), and the VM's end when one fails. A new src/bin/nvx-init.rs, with required-features = ["guest"], calls it; the guest build includes nvx-init, and the API snapshot gains the hidden entry point. nvx-init runs no other guest tool.
    • Scope:
      • The PID 1 core in the hidden guest::init module, whose private submodules have focused unit tests and whose only public item is #[doc(hidden)] pub fn main.
      • A new src/bin/nvx-init.rs, selected by the guest feature, calls guest::init::main; the guest build's binary inventory builds and stages it.
      • Signal handling, orphan reaping, console handling, exit-status collection, and shutdown by sync followed by direct power-off.
      • oom_score_adj -1000 and nice -10 for PID 1 and the registered helper process IDs.
      • Registration of exactly the time daemon and, when listeners are published, /sbin/nvx-ingress, using the process IDs handed over by the shell /init; a helper's exit is fatal to the VM.
      • The API snapshot gains the hidden guest::init::main entry point.
    • Not in this PR:
      • Layered-root assembly and switching, which SL7.5b adds.
      • Workload execution, process groups, cancellation, timeouts, and capture sweeping, which SL7.5c adds.
      • Starting helpers at cold boot: the shell /init starts them and hands over their process IDs.
      • Running nvx-reseed, nvx-exit, or any other guest tool: reseeding is a library call and PID 1 powers off directly.
      • Restart policies, helper pools, dependency ordering, health probes, or graceful degradation: the VM ends when either registered helper fails.
    • Approach: Build the small PID 1 loop directly in guest::init, reusing the existing guest system wrappers for signals, waits, console operations, priority, sync, and power-off. Add the thin binary and select it in the guest build like the other guest binaries. Keep a fixed registry for the two possible helpers rather than a general service manager.
    • Done when:
      • Tests cover PID 1 signal handling, orphan reaping, console handling, and exit statuses.
      • Tests observe oom_score_adj -1000 and nice -10 on PID 1, the time daemon, and the optional announcer.
      • A registered helper's exit ends the VM.
      • Shutdown syncs filesystems and powers off without invoking another guest tool.
      • The guest build stages nvx-init, and the API snapshot marks only its entry point hidden.
    • Needs: SL0.4, in nvx-rs slimming 1.1.1: the public API is checked, and NVX's programs are hidden from it #1091.
    • Estimate: 300 core lines, 3 counted files. Past 600 core lines, stop and ask before continuing.
  2. SL7.5b: nvx-init builds the layered root's space, and switches into it.

    • Objective: nvx-init can cold-boot the layered root described by the command line, prepare every mount and mapping handle before workloads start, and switch into that root exactly once. Executions then inherit one shared VM root without per-exec containment.
    • Contents: Parse the root's and the view's tokens, nvx_maps and nvx_share, the RAM upper's cap included; then use the space where it is (guest::{root, view} and the mapper) to assemble, mount_export and mount_share; it opens and keeps the export's SourceRoot, the root's TargetRoot, their Mapper and the host's MappingTable; then mount_runtime with the time status, view::apply and switch, with no cgroups, no namespaces and no limits. The kept mapper applies MAPS after the switch, through the table, which is complete before any workload starts. This is a layered cold boot; a Fresh capture VM, which has no root's tokens, gets its own entry in SL7.7a.
    • Scope:
      • The cold-boot path in guest::init, using guest::{root, view} and the mapper directly where they are.
      • Parsing the root and view tokens, nvx_maps, nvx_share, and the RAM upper's percentage cap.
      • Root assembly, export and share mounts, runtime mounts with the time-status bind, view application, and one root switch.
      • Opening and retaining the export SourceRoot, root TargetRoot, Mapper, and public MappingTable before the switch.
      • Applying MAPS after the switch through the retained mapper, only after the table is complete and before any workload starts.
    • Not in this PR:
      • Fresh capture VMs without root tokens, which enter their separate boot path in SL7.7a.
      • Restore activation and application of a restored root, which SL7.7b adds.
      • Per-exec roots, chroots, mount namespaces, PID namespaces, cgroups, or resource limits: all executions inherit the single switched root.
      • A second switch, root teardown after switching, or unmount-on-shutdown machinery: the VM syncs and powers off.
      • Dynamic mount discovery or a generic mount planner: the root, export, shares, runtime filesystems, view, and mapping handles are the fixed sequence.
    • Approach: Parse only the command-line contracts that the CLI emits, then call guest::{root, view} and the mapper directly in their required order. Retain the mapping descriptors across the one switch and feed completed MAPS parts to the existing MappingTable and mapper. Do not build an alias module, new root abstraction, mount graph, namespace layer, or per-exec setup path.
    • Done when:
      • A cold-boot test parses every named token, including shares and the RAM upper's cap.
      • Tests assemble the root, mount the export, shares, runtime filesystems and time status, apply the view, and switch exactly once.
      • The mapping source and target are opened before the switch, and a mapping applies through the retained mapper afterward.
      • No workload starts before the mapping table is complete.
    • Needs: SL7.5a, above; SL7.1b, in nvx-rs slimming 2.4.1: the guest's space becomes reference code, with runtime mounts, one switch and the view #1104; SL7.2, in nvx-rs slimming 2.4.1: the guest's space becomes reference code, with runtime mounts, one switch and the view #1104.
    • Estimate: 300 core lines, 2 counted files. Past 600 core lines, stop and ask before continuing.
  3. SL7.5c: nvx-init runs each exec as a plain process.

    • Objective: nvx-init runs each requested execution as an ordinary child process with a dedicated process group, hardened credentials, working pipes, and a reported status. Every ending path kills that group, while capture can sweep processes that deliberately leave it.
    • Contents: Fork and exec in its own process group in guest::init, with no cgroups, namespaces or limits: the child resets oom_score_adj and nice to 0, sets its user and group IDs and account mode, clears its supplementary groups and capability sets, sets no_new_privs, changes to its working directory, so that one that the workload can't search fails with EACCES, and execs. Its pipes, signals and exit status; end, cancellation, timeout and reset, each killing the process group; and the sweep that a capture's kill uses, which spares PID 1 and the registered helpers. Until SL7.7a calls the crate-private sweep, it carries expect(dead_code), whose reason names that caller. End, cancellation, timeout and reset are its only signals: NVXC v2 has no signal message, so exec's signal, which dev added, isn't carried.
    • Scope:
      • The plain-process exec core in guest::init, including launch, pipes, status collection, process-group termination, and unit tests.
      • The child resets oom_score_adj and nice to 0 before dropping privileges, then sets user and group IDs and account mode.
      • The child clears supplementary groups and capability sets, sets no_new_privs, changes to the requested working directory after dropping credentials, and execs.
      • End, cancellation, timeout, and session reset each kill the complete process group.
      • A crate-private sweep, carrying expect(dead_code) whose reason names SL7.7a, repeatedly kills every process except PID 1 and the registered helpers.
    • Not in this PR:
      • Cgroups, per-exec memory or process limits, mount or PID namespaces, seccomp policies, or per-exec root entry: the VM and process group are the only containment.
      • Arbitrary signal forwarding: NVXC v2 carries no signal request, so only end, cancellation, timeout, and reset terminate an exec.
      • Killing a descendant that called setsid during ordinary exec termination: the capture sweep handles it.
      • Capture request handling and the decision to invoke the sweep, which SL7.7a adds.
      • Job control, restart policies, detached execution, concurrent execs, or process-tree accounting beyond the process group: no caller needs them.
    • Approach: Fork once in guest::init, create the child's process group, configure descriptors and credentials with the existing guest system calls, then execve. Track the leader, pipes, and status directly; use group-directed termination for every ending path. Implement the crate-private capture sweep as a narrow process-table pass, not as a cgroup replacement or process supervisor.
    • Done when:
      • Exec tests cover pipes, exit statuses, credential setup order, priority reset, no_new_privs, cleared groups and capabilities, and account mode.
      • A working directory searchable by the workload succeeds, while one it cannot search fails with EACCES.
      • End, cancellation, timeout, and reset kill the leader and descendants that remain in its process group.
      • A setsid descendant survives group termination, then the repeated sweep kills it and every other unregistered process while sparing PID 1 and both helpers.
    • Needs: SL7.5a, above.
    • Estimate: 350 core lines, 4 counted files. Past 600 core lines, stop and ask before continuing.
  4. SL7.5d: nvx-init starts the announcer after a restore.

    • Objective: A restored nvx-init can start the announcer required by the restore's published listeners, retain it as a supervised helper, and do so before entering the restored root. This supplies one ordered restore step for SL7.7b without adding another lifecycle.
    • Contents: A crate-private guest::init step that starts /sbin/nvx-ingress with the gateway after apply and before switch, when listeners are published, and registers it as a helper. Until SL7.7b's restore calls it, it carries expect(dead_code), whose reason names that caller. There is no one-shot lifecycle. nvx-init holds its announcer, so it doesn't use nvx-ingress start, which detaches the announcer for a caller that can't hold one.
    • Scope:
      • A crate-private guest::init restore helper that starts /sbin/nvx-ingress with the restored gateway.
      • The step runs only when the applied network policy reports published listeners.
      • The announcer starts after space application and before the root switch, then joins PID 1's helper registry.
      • Until SL7.7b first calls it, the helper carries expect(dead_code) whose reason names that caller.
    • Not in this PR:
      • The restore state machine and its call to this step, which SL7.7b adds.
      • Cold-boot helper startup: the shell /init starts the announcer and hands its process ID to nvx-init.
      • nvx-ingress start, which detaches an announcer for a caller that cannot retain it; nvx-init retains its child.
      • A one-shot lifecycle, announcer restart policy, multiple announcers, or a generic helper launcher: layered roots have one managed lifecycle.
      • Starting the time daemon after restore: it was forked before nvx-init existed and survives as a registered helper.
    • Approach: Fork directly from PID 1 in guest::init so the announcer inherits oom_score_adj -1000 and nice -10; exec the fixed /sbin/nvx-ingress program with one gateway argument and register its process ID. Do not send it through the workload child path, add a service abstraction, use detached launch, or build a network-policy engine.
    • Done when:
      • With published listeners, the step starts /sbin/nvx-ingress with the gateway after apply and before switch.
      • Without published listeners, the step starts no process.
      • The started announcer is registered as a helper and therefore retains PID 1's priority and fatal-exit behavior.
    • Needs: SL7.5b, above; SL7.5c, above; SL7.3, in nvx-rs slimming 2.4.2: activation holds no fence, and a capture is one guest call #1105.
    • Estimate: 100 core lines, 2 counted files. Past 200 core lines, stop and ask before continuing.

Tests. PID 1's signals, reaping, console and exit statuses, its oom_score_adj of -1000 and nice of -10, and its shutdown by sync and power-off. The time daemon's and the announcer's process IDs are registered, both keep that priority, and a failed helper ends the VM. A cold boot parses the root's and the view's tokens, nvx_maps and nvx_share, the RAM upper's cap included; assembles the root; mounts the export, the shares and the runtime filesystems with the time status; applies the view; opens the mapping source and target before the switch; switches once; and applies a mapping through the kept mapper after the switch. Each exec has working pipes, signals and exit status; its child resets oom_score_adj and nice to 0 before it drops its credentials, sets its user and group IDs and account mode, clears its supplementary groups and capabilities, sets no_new_privs, and changes directory after dropping them; a working directory that the workload can't search fails with EACCES. End, cancellation, timeout and reset each kill every member of the exec's process group, including the descendants that remain in it. A descendant that calls setsid survives that kill; the capture's sweep kills it and every other unregistered process, repeats until none remains, and spares PID 1 and the registered helpers. The announcer step starts /sbin/nvx-ingress with the gateway before the switch only when listeners are published, and registers it as a helper.

Done when: every PR above has merged, with dev green.

Activity

  1. esaurez commented on Nov 10, 2026

    @esaurez
    ContributorAuthor

    Workstream source/progress checkpoint - 2026-11-10 UTC

    Checked boxes below mean merged scope, not merely an implemented first iteration. Implementation branches are source checkpoints, not automatic qualification, Ready or merge approval. Some starting inputs are frozen/anonymous commits rather than standalone branches; their exact SHAs and known merged producer refs are recorded explicitly.

    The original technical scope remains authoritative, supplemented only by any portable technical clarification in this issue. No machine-specific execution policy is part of this checkpoint.

  2. esaurez commented on Nov 10, 2026

    @esaurez
    ContributorAuthor

    SL7.5d is source-locally qualified in new Draft #1439, without changing the older #1429 checkpoint or its publication chain.

    Exact source: branch esaurez/private-simple-1115-sl75d-02e4-20261110, HEAD 87528167b284851b5e4b7b4df057868b5e2c9904, tree 8cef6ba1940615b5f6d9e9d7dd27e3f7c666e556, sole qualified parent 1cb779e8daf8716f1d650feeb176a10a8b1bf83f (SL7.7a / #1431), unchanged OpenVMM 14dc4116ec7e57567633c4b7b07892aca3a58522. Actual SL7.5b/SL7.5c/SL7.3 prerequisite ancestry and APIs were verified; no unqualified reader history was imported.

    Both complete fresh Linux/Windows software matrices passed, including six normal announcer process/helper regressions, all ten fresh guest executables and guest integration33/33. Independent native Opus 5.5/max review 6fb57b3a-2f7b-404d-916c-e02cdb313c03 is CLEAR for the exact tuple, including the test-hook allowance6to8 and all new tests.

    Actual MSHV and WHP each passed the privileged pre-switch-root/nice-10/OOM-1000 fixture using its explicitly labelled observing-program override. Each backend also ran the separate real current fixed /sbin/nvx-ingress 10.88.0.1, checking image hash, one-gateway argv, PID-1 child registration, priority and liveness, then proving fatal helper supervision with guest status125 and complete native teardown. That separate real case uses the existing cold helper handoff, not a future SL7.7b restore caller. Complete current effective inputs, archive/copy hashes and Windows' independent reassembled images were verified. A Linux fixture-only endpoint-directory preflight failure is preserved; its isolated0700-directory repair passed without production or host/global changes.

    Both named host jobs are complete/released after clean exact source/native and zero owned compiler/VMM/controller reconciliation. Compact source-bound evidence archives were downloaded and verified through SHA256, ZIP CRC, all member hashes and receipt hashes. Draft1439's Validation section has the detailed results and explicit coverage limits.

    This is local producer qualification, not full public CI or merge readiness. The PR stays Draft; there is no KVM, future restore-caller/state-machine or production SL7.8a route claim. Remaining public CI/review/merge coordination is separate; this issue remains open and unmerged.

  3. esaurez commented on Nov 10, 2026

    @esaurez
    ContributorAuthor

    Completed under the confirmed campaign target, feature/nvx-refactor. All declared slices SL7.5a-d are merged in #1274, #1397, #1296 and #1439; linked support #1276, #1295 and #1328 is also merged. The whole CI workflow passed on current feature commit eba4c8b: https://github.com/microsoft/nvx/actions/runs/42909697374 . The subsequent capture/restore callers remain tracked in #1117; they are outside this issue's declared producer scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions