Warning
Experimental Demo, Not Audited! Works on x86_64.
SuperVM is an aggressively memory-optimized runner for heterogeneous VMs. SuperVM VMs can run different packages, services, etc, but must use NixOS. Any software packages from public Nix caches are stored exactly once on disk and cached once in shared memory across all guests; private state remains private.
nix run .#supervm -- prepare ./my-vm
nix run .#supervm -- launch ./my-vmYet, each guest retains its own kernel, distinct writable Nix store, and state that is invisible to every other guest.
SuperVM is a proof of concept for seeing how far VM memory efficiency can be pushed under production constraints using selective kernel-page merging and policy-controlled sharing of Nix package files. It demonstrates that we can exercise granular control over what enters shared memory, avoiding broad cache-side-channel exposure. The same mechanism has near-zero discovery cost at runtime: known pages are mapped directly rather than found by scanning memory for duplicates, so the safer approach is also faster than scan-based deduplication.
Firstly, SuperVM makes the core kernel text and rodata in each guest's memory KSM-mergeable, so VMs running the same kernel share one physical copy of those pages. Notably, no other guest memory is eligible. This cuts the overhead of the read-only kernel image without scanning changeable or secret data, where copy-on-write churn would hurt performance and merging could create side channels.
Secondly, Nix packages can annotate which files, including binaries, are safe to share; unmarked files never enter the shared cache. SuperVM applies that policy file by file, serving annotated files from one common cache instead of copying them into every VM. This captures most of the memory savings without exposing secret-handling code to cross-guest cache side-channel attacks.
SuperVM is built around a patched crosvm, which provides the memory mappings these mechanisms need. Its stripped build has Firecracker-class overhead: 6.7 MiB of measured host memory per VM outside guest RAM, versus 2.8 MiB for Firecracker.
Measured in idle fixed-count runs at 1, 4, and 32 VMs on 512 MiB minimal NixOS guests, including runtime infrastructure and resident host caches.
Aggregate physical memory, with shared pages counted once:
| VM count | LameVM | SuperVM |
|---|---|---|
| 1 | 416.7 MiB | 480.0 MiB |
| 4 | 1183.6 MiB | 776.0 MiB |
| 32 | 8329.9 MiB | 3512.8 MiB |
At 32 VMs the deployments measure 8.1 GiB against 3.4 GiB.
| Shared, once per host | SuperVM |
|---|---|
| DAX pool (mapped) | ~50 MiB |
| Metadata index | ~9.8 MiB |
Earlier guest-side changes cut an idle guest's host-resident RAM from 270 MiB to 118 MiB without KSM, mainly by reducing DAX window page metadata from 129 MiB to 5 MiB; these measurements predate dynamic DAX metadata allocation.
Use a Linux host with Nix flakes and KVM.
Preparation builds the guest and adds its packages to the shared store. Launching boots that prepared guest with the state folder as its private layer.
Prepare first:
nix run .#supervm -- prepare ./vm-a
nix run .#supervm -- prepare ./vm-bThen, start the VMs in two terminals:
nix run .#supervm -- launch ./vm-a
nix run .#supervm -- launch ./vm-bThe optional --profile flag selects a NixOS configuration:
nix run .#supervm -- prepare --profile ./machines#web ./vm-web
nix run .#supervm -- launch ./vm-weblaunch boots the last successfully prepared runner without evaluating the
guest again.
launch --dax-window=BYTES sets the DAX window size (default 1 GiB). It
costs the guest 16 MiB of RAM per GiB, so size it to the files a guest maps at
once, not to the store:
nix run .#supervm -- launch --dax-window=536870912 ./vm-aSelect --dax=never, --dax=inode (the default), or --dax=always while
preparing:
nix run .#supervm -- prepare --dax=always ./vm-a
nix run .#supervm -- launch ./vm-alamevm runs the same NixOS guest with mainline microvm.nix and nixpkgs crosvm:
nix run .#lamevm -- prepare ./baseline-vm
nix run .#lamevm -- launch ./baseline-vmAs with SuperVM, select a profile while preparing; launch reuses the prepared
runner without evaluating it again:
nix run .#lamevm -- prepare --profile ./machines#web ./baseline-web
nix run .#lamevm -- launch ./baseline-webFor a fair comparison, run multiple instances of each, since SuperVM's benefit
comes from sharing common pages between guests. If you just want to see how much
performance SuperVM can achieve, pass --dax=always so every common store page
is shared. Alternatively, to test the production-style fine-grained sharing
policy used to avoid cross-guest cache side channels, use the default
--dax=inode mode and add an $out/nix-support/fsmeta file to packages whose
files should be shared. Each line in fsmeta has the form
snix.dax <path-relative-to-the-package-root>, for example
snix.dax bin/example; listed files enter the shared cache, whereas unlisted
files stay out.
