[CMGS]

Adds cocoon vm migrate capture|receive|abort, which moves a running Cloud Hypervisor VM to another node with its processes and memory intact. Cocoon captures and restores. The caller moves the files, picks the round count and decides which node owns the VM. Firecracker returns ErrMigrateUnsupported.

# source
cocoon vm migrate capture my-vm --output json                 # full baseline, guest resumes
cocoon vm migrate capture my-vm --series 123 --output json    # optional delta, repeatable
cocoon vm migrate capture my-vm --series 123 --final --output json   # guest stays paused
# target, after copying every listed file into one directory
cocoon vm migrate receive --from-dir /xfer/123 --id <source-vm-id> --name my-vm --pull
cocoon vm start my-vm
# source
cocoon vm rm --force my-vm

cocoon vm migrate abort my-vm --series 123 gives up at any point before that. The contract is in docs/snapshots.md, section Live Migration.

How it works

• Memory: the CH diff snapshot series. Needs the fork at 35b098c65260 or newer, which adds PUT /vm.end-snapshot-series. An older VMM answers 404 on the first capture and nothing changes.
• Disks: each round compares FIEMAP extents against a reflinked reference, so a round costs what the guest changed. A run dir without reflink is refused before the pause.
• Failure: a failed round ends the series, resumes the guest and deletes the directory. The caller starts a new baseline.
• Target: reuses the clone path, skips the resume and reseed, and emits metering reason migrate. The guest gets fresh NICs like a clone, so it needs the post-clone network setup.

Hot-path cost: no new I/O on claim, clone, restore, snapshot or create. Clone gains a flag read, a branch and one string check per snapshot file, about 33 ns per clone of 5 files in a benchmark. snapshotVM marshals a small struct instead of a map.

Behaviour changes outside the feature

• prereserveVM no longer calls RollbackCreate when the reservation fails. A failed reservation writes nothing, and with --id that rollback would delete the live record holding the ID.
• FinalizeClone takes the metering reason. Clones still emit clone.

Test plan

Real cross-host runs between two bare-metal nodes (AMD EPYC 9Y24, XFS), kit torn down afterwards:

• Migrate and migrate back: disk sha256 matches, same guest pid, memory stamps show 0 errors, sequence logs on the COW and a data disk have no gap
• Heavy writes with and without --no-direct-io: sha256 matches with a 3.7 GiB final delta
• Default balloon, and restore modes copy and mmap; ondemand is rejected by CH
• qcow2 overlay: offline replay of the deltas matches the paused source, qemu-img check is clean, cold boot reaches login. The hash differs once the VMM opens the file, because the dirty bit makes it rebuild refcounts
• NIC swap on an isolated bridge: new MAC, guest pings the target bridge once its new NIC has an address. A NIC added mid-series arrives on the target
• SIGKILL during baseline, delta and receive (exit 137 confirmed): vm start or abort recovers the guest, a killed delta fails its retry and discards the series, a killed receive retries with one VMM
• Refusals with the guest left running and no directory left: v54 VMM (404), tmpfs run dir, hot-attached disk. ext4 was not run; it has no reflink and takes the same check
• Old series ID: delta fails and abort does nothing, including while a newer series holds the guest paused
• Truncated disk delta fails the receive with no residue, and a retry after a full copy succeeds
• Lifecycle on a migrated VM with a raw COW and a data disk: file checksums from before and after the move, vm net up and down, vm disk attach and detach, snapshot save, clone, vm restore, vm hibernate and wake, vm stop, e2fsck -fn returning 0 on reflinked copies of both disks, cold boot, guest reboot, then another migration with matching sha256

Timings with a 2 GiB guest under light load: baseline capture about 0.3 s (8 GiB about 0.7 s), final capture about 0.13 s, receive about 0.26 s, vm start about 0.12 s. Guest downtime was not measured directly: the runs hashed both disks during the pause. The sum with a 16 MB final round over a 57 MB/s link is about 1.4 s.

On 7395cde6 these were rerun: migrate and migrate back, default balloon, restore modes, old series ID, SIGKILL during a delta, NIC swap and mid-series NIC add, truncated delta, and the lifecycle set. Heavy writes, the qcow2 overlay, SIGKILL during baseline and receive, and the three refusals last ran on earlier commits of this branch.

95921b85 came after that run and has no hardware run of its own. It refuses a delta when a capture was killed between committing a disk delta and moving its reference. A unit test plants that residue; no real kill was injected into the window.

Gates on 7395cde6: make lint 0 issues on linux and darwin, make fmt-check and asl clean, go test -race ./... passes in a Linux container.

Not tested: TSC across hosts of different frequency, nested virtualization, Windows guests, a real orchestrator as the caller, network connectivity after a cold boot, reboot or restore of a migrated VM, and the same lifecycle set on a qcow2 overlay beyond stop and cold boot. Firecracker's refusal has a unit test only. All runs used the same root_dir and run_dir paths on both nodes.

Known limits

• vm clone --from-dir on a series directory pairs the last memory with baseline disks. The docs forbid it; there is no guard.
• A heavy writer can produce deltas that take longer to copy than the guest takes to dirty them. Round policy is the caller's.
• A receive that fails after the VMM starts leaves an empty cgroup scope for gc, like any clone failure. Reading the code, the existing cgroup sweep collects it; gc was not run in these tests.
• TestStartSequenceRelaunchesWhenTheScannedVMMDiedBeforeTheLock is flaky on the base commit too (1 of 60 under load); this branch does not touch that path.