Repository navigation
Description
Parent: #1083, in #1076. Needs: #1108, #1109, #1117. Unblocks: #1111, #1112, #1114.
Goal. The sandbox lifecycle runs on the new interface. --memory-max and --pids-max go first: the VM is the limit. Then sandbox start launches through host::vm::{create, spawn}, journaled and recoverable, with the CLI's tokens for the managed mode and the workload identity, and the shell /init hands its layered root to nvx-init. In the same PR, exec, stop and deprovision take their lock, records, recovery and reattach from the CLI's store, with the runtime identity check: stop stops gracefully over NVXC when it can attach and forces the VM down once its grace period ends, and deprovision removes everything that a sandbox owned. start first builds the VmConfig from the record that provision writes today; then provision records a GPT or EROFS image, and a scratch or a RAM upper with its cap (50% by default), in the CLI's store, and start reads them back. A sandbox can't share its own files: a protected path is refused at provision and again at start (#818).
Notes.
sandbox startwaits for nvx-init, whose layered boot comes from nvx-rs slimming 3.3.3: nvx-init captures and restores, and boots the new launches' layered roots #1117. Until then the oldstartstays, and its VMs keep the old layered init.- A launch's records and their readers switch together. The old launch writes the old store's records, which
exec,stopanddeprovisionread, so the PR that switchesstartswitches their lock, record lookup, recovery and reattach too: no build has astartwhose VMs the other commands can't manage. startswitches beforeprovision. The newstartreads the record thatprovisionwrites today, so each PR leaves both commands working;provisionthen extends the record andstart's reader together.
PRs, in order. Each PR has one goal, leaves dev green, and names in its description each old test of its area as kept, rewritten or retired, with each change that check_test_changes.py reports. Its tests follow the repository's test layout: a module's unit tests, and the code that only they use, sit in its tests.rs or tests/ and move with it. The estimates count core lines (code, not comments, docs or tests) and counted files; each PR records its measured values. Each PR does what its Scope says and no more: a case, input or layout that its Scope doesn't name, and whatever "Not in this PR" lists, is out of scope. Address every review comment within the scope, and ask before acting on one that asks for more. A PR whose measured core lines pass the stop that its Estimate names (twice the estimate, at most 600) stops and asks before continuing.
-
SL3.5d:
--memory-maxand--pids-maxgo: the VM is the limit.- Objective: Sandbox workloads no longer advertise per-workload memory or process limits that the new guest lifecycle does not enforce. The VM's resources are the boundary.
- Contents:
provisionandsandbox runno longer take--memory-maxor--pids-max, andprovisionno longer records them, so the old launch sets no per-workload limit.resource_limits_are_bounded_ascii_decimalretires with them. - Scope:
nvx-rs/src/host/cli/sandbox/options.rsremoves--memory-maxand--pids-maxfromprovisionandsandbox runparsing and help.nvx-rs/src/host/cli/sandbox/mod.rsstops carrying those values into a launch.nvx-rs/src/host/cli/sandbox/state.rsstops writing the two limits, so the old launch sets no per-workload limit.
- Not in this PR:
- Adding replacement limit flags or another process-control mechanism: the VM is the limit.
- Changing
--memory-mibor the RAM upper's cap: those size the VM and its upper, not an individual workload. - Migrating old configuration records or preserving the removed CLI options: no compatibility reader or old-output compatibility is required.
- Changing any other sandbox option, command or output.
- Approach: Delete the two option specifications, fields and record writes along their existing straight-line path. Do not retain ignored aliases, record placeholders or a generalized limits object; retire
resource_limits_are_bounded_ascii_decimalfrom the sandbox module'stests.rswith the options. - Done when:
provisionandsandbox runreject--memory-maxand--pids-maxas unknown options.- New provision records contain neither per-workload limit.
resource_limits_are_bounded_ascii_decimalis retired with the removed options.
- Needs: nothing.
- Estimate: 40 core lines, 3 counted files. Past 80 core lines, stop and ask before continuing.
-
SL3.5b:
sandbox startlaunches through the new APIs, and the lifecycle commands manage its VM.- Objective:
sandbox startlaunches a VM that every lifecycle command can recover, reattach, stop and remove through the CLI's store and the new VM interface. The layered root reaches nvx-init without an OpenVMM lifecycle option. - Contents:
startonhost::vm::{create, spawn}, journaled and recoverable, with the CLI's tokens for the managed mode and the workload identity. It builds theVmConfigfrom the record thatprovisionwrites today: the EROFS image, the scratch or RAM upper, the network and the export. Its command line holdsnvx_sandbox=1and no OpenVMM lifecycle token, so the shell/inithands its root to nvx-init (SL7.8a). The readers of its records switch with it:exec,stopanddeprovisiontake the lock, the record lookup, the recovery and the reattach from the CLI's store, throughhost::vm::reattachwith the runtime identity check;stopsends NVXC STOP when it can attach, and forces the VM down throughhost::vm::stoponce its grace period ends;deprovisionreleases the log and scratch claims, and removes the VM directory and every record of the sandbox. A managed VM's outcome report stays readable under a restrictive umask, and the store and the supervisor are read from their split files. - Scope:
nvx-rs/src/host/cli/sandbox/lifecycle/switchesstart,exec,stopanddeprovisiontogether to the CLI's lock, journal, runtime record, capability and claim files.startbuildsVmConfigfrom today's provision record (its EROFS image, scratch or RAM upper, network and export), callshost::vm::create, journals the prepared capability and outputs, then callshost::vm::spawn.startuses the CLI's managed-mode and workload-identity tokens, writesnvx_sandbox=1, and passes no OpenVMM lifecycle option.execandstopreattach withhost::vm::reattachand the runtime identity check;stoprequests NVXC STOP, then useshost::vm::stopafter the grace period.deprovisionreleases log and scratch claims and removes the VM directory plus every sandbox record; restrictive umasks do not hide the outcome report.
- Not in this PR:
- Writing the new GPT, image-format and RAM-cap provision record: SL3.5a follows after this PR.
- Reading both old and new runtime stores or migrating running sandboxes: a launch and all of its readers switch together.
- Changing exec admission: SL3.9 adds the separate per-sandbox exec lock.
- New lifecycle commands, flags or output redesigns beyond the required launch switch.
- Making the CLI helpers private: phase 4 owns those visibility changes after their library users leave.
- Approach: Adapt the existing lifecycle flow in place to the CLI's store and
host::vm; preserve today's provision-record reader until SL3.5a. JournalPreparedvalues before spawning, reuse the existing recovery and command tests, and switch all record readers in this PR rather than adding dual-store adapters. - Done when:
- With today's provision record and each existing upper, provision, start, exec, stop, a second start and stop, and deprovision complete.
- The launch journal exists before spawn, and a later command recovers an interrupted start while retaining an unverified live launch.
execandstopcheck runtime identity; stop is graceful when attachment works and forced after its grace period.deprovisionremoves every sandbox record, claim and VM file, and the outcome report remains readable under a restrictive umask.startsendsnvx_sandbox=1, sends no OpenVMM lifecycle option, and reads the split store and supervisor files.
- Needs: SL3.5d, above; SL3.3b, in nvx-rs slimming 3.1.3: the CLI keeps its own sandbox store #1108; SL3.4a2, in nvx-rs slimming 3.1.4:
runuses the new interface and the CLI's own guest tokens #1109; SL7.8a, in nvx-rs slimming 3.3.3: nvx-init captures and restores, and boots the new launches' layered roots #1117. - Estimate: 540 core lines, 5 counted files. Past 600 core lines, stop and ask before continuing.
- Objective:
-
SL3.5a:
sandbox provisionrecords a GPT or EROFS image, and the RAM upper's cap, in the CLI's store.- Objective:
sandbox provisionrecords the root image and upper configuration that the new launcher needs. A laterstartcan reproduce EROFS or GPT with scratch or a bounded RAM upper from the CLI's own record. - Contents:
provisiononhost::cli::store, with aVmConfigbuilt from its options: a GPT or EROFS image by path, a scratch or RAM upper, and the RAM upper's cap (50% by default). Its records name the format and the upper, andstartreads them back (nvx-rs: GPT images, the RAM upper and protected paths innvx-rs sandbox#818). - Scope:
nvx-rs/src/host/cli/sandbox/builds aVmConfigfrom provision's image and upper options.- Provision accepts an EROFS or GPT image by path, and a scratch upper or RAM upper.
- A RAM upper records an explicit cap from 1 through 90 percent, or 50 percent when omitted.
- the CLI's configuration record names the image format, image path, upper kind and RAM cap, and
startreads those fields back.
- Not in this PR:
- File-descriptor or final-path transfer for images: the new interface takes image paths.
- Additional image formats, multiple lower images or upper kinds: the plan has EROFS, GPT, scratch and RAM only.
- Migrating older records or writing two record formats: no compatibility reader is required.
- New sandbox commands, unrelated flags or output redesigns.
- Protected-path checks for shares: SL3.5c adds those after the record shape lands.
- Approach: Extend the CLI's existing provision record and its
startreader together, usingRootConfig::Layered, the existing image option parsing andUpperConfig. Store the enum choices and cap directly; do not add format discovery, an image registry or a record-conversion layer. - Done when:
- The lifecycle passes with EROFS and GPT, each using scratch and RAM uppers.
- An explicit RAM cap and the 50 percent default are persisted and read back by
start. - The record names the image format and upper kind that
startplaces inVmConfig. configurations_of_every_format_start_and_exec,lifecycle_of_a_sandbox_whose_upper_is_in_ramandthe_upper_is_scratch_or_ramare kept or rewritten.
- Needs: SL3.5b, above.
- Estimate: 300 core lines, 4 counted files. Past 600 core lines, stop and ask before continuing.
- Objective:
-
SL3.5c: A sandbox can't share its own files.
- Objective: A sandbox cannot expose its own control or storage files back into its guest. Provision and start both reject such shares after resolving the relevant host paths.
- Contents: A share whose host path equals, contains or lies inside the sandbox's state directory, scratch, images or artifacts is refused, by canonical path, at
provisionand again atstart(nvx-rs: GPT images, the RAM upper and protected paths innvx-rs sandbox#818). No alias matrix. - Scope:
nvx-rs/src/host/cli/sandbox/checks each share's canonical host path against the sandbox state directory, scratch, images and artifacts.- A share is refused when its path equals a protected path, contains one, or lies inside one.
provisionchecks before recording, andstartrepeats the check before launching so changed path resolution cannot bypass it.
- Not in this PR:
- An alias matrix, inode graph, bind-mount discovery or continuous monitoring: the required rule is canonical-path containment.
- Protecting arbitrary host paths beyond the sandbox's state, scratch, images and artifacts.
- Changing share syntax, adding share flags or redesigning refusal output.
- Checking
sandbox runhere: SL3.4b checks the protected paths that apply to its stateless launch.
- Approach: Reuse the CLI's existing path resolution and containment helpers to compare canonical paths in both call sites. Keep one bounded list of protected roots; do not build a filesystem topology model or attempt to detect every possible alias.
- Done when:
provisionrefuses a share equal to, containing or inside the state directory, scratch, an image or the artifacts.startrepeats the same refusals from the recorded configuration.- Ordinary shares outside the protected paths still provision and start.
- Needs: SL3.5a, above.
- Estimate: 120 core lines, 2 counted files. Past 240 core lines, stop and ask before continuing.
Tests. provision and sandbox run refuse --memory-max and --pids-max, and resource_limits_are_bounded_ascii_decimal is retired with them. At the launch's PR, with the record that provision still writes and each upper: provision, start, exec, stop, a second start and stop, and deprovision. start's command line holds nvx_sandbox=1 and no OpenVMM lifecycle token, so its root boots through nvx-init; its journal is written before the spawn, and a command after an interrupted start recovers it through the new journal and keeps an unverified live launch. exec and stop check the reattached VM's runtime identity; stop sends NVXC STOP when it can attach, and forces the VM down once its grace period ends; deprovision releases the log and scratch claims, and removes the VM directory and every record of the sandbox. After provision's new record, the same lifecycle with GPT and EROFS, each with a scratch and with a RAM upper: an explicit cap and the 50% default, the format and the upper persisted and read back by start. A share whose host path equals, contains or lies inside the sandbox's state directory, scratch, images or artifacts is refused, by canonical path, at provision and again at start. The CLI tests of these commands, such as lifecycles_of_every_configuration, lifecycle_of_a_sandbox_whose_upper_is_in_ram, configurations_of_every_format_start_and_exec and the_upper_is_scratch_or_ram, are kept or rewritten.
Done when: every PR above has merged, with dev green.
Activity
esaurez commentedon Nov 10, 2026
Workstream source/progress checkpoint - 2026-11-10 UTC
Checked boxes below mean merged scope, not merely an implemented first iteration. Implementation branches are source checkpoints, not automatic qualification, Ready or merge approval. Some starting inputs are frozen/anonymous commits rather than standalone branches; their exact SHAs and known merged producer refs are recorded explicitly.
SL3.5d - merged via pull request 1264.
esaurez/slim-3.1.5-sl3.5d-20261103ata52557d7dd51739ae8f640f878fc4c2ed2994227.devat26e5db7cabd1a126202b5b7fb866835e46ffc864; merge commit607a3938ca642080ce6e3751c86c37b99e6dbfab.SL3.5b - First implementation committed; not merged.
esaurez/slim-3.1.5-sl3.5b-private-20261108; exact head4334a234fd4d7ef550987aac8f58358c54de2f70.fa73e4415bd5297eb0c7de6432c386cc7f45f7da.SL3.5a - First implementation committed; not merged.
esaurez/slim-3.1.5-sl3.5a-private-20261108; exact head122909f9b4f7b3a9a392026a1c7f6b86b0bba724.2e936243e51b3de6b00f9d7da9024e771fc23096.ddc46d71864a4401651a4fda96f7e431d26fa140.SL3.5c - First implementation committed; not merged.
esaurez/slim-3.1.5-sl3.5c-private-20261108; exact head9ccd5f7b0eb1afab6ca8fa0c9953878c585030d7.067bbfd0e060f195a20d79136528ca9bd49dab1d.esaurez/slim-3.1.5-sl3.5a-private-20261108at122909f9b4f7b3a9a392026a1c7f6b86b0bba724.The original technical scope remains authoritative, supplemented only by any portable technical clarification in this issue. No machine-specific execution policy is part of this checkpoint.
ppenna commentedon Nov 11, 2026
All four PRs of this group are on
feature/nvx-refactor: SL3.5d (#1264, merged intodev, which the branch contains), SL3.5b (#1460), SL3.5a (#1466) and SL3.5c (#1485).The feature branch's post-merge CI run at
f834bcb967, which contains all of them, passed: https://github.com/microsoft/nvx/actions/runs/43114171980. Closing this group as complete;devreceives the rest whenfeature/nvx-refactorlands there.