fix(colima): Upgrade Colima/Lima and repair M4+ SVE without data loss

AGENTS.md

@@ -18,6 +18,17 @@ Co-Authored-By: <agent and model> <email>
 - Run your checkout against another repo: `uv run --project ~/code/devenv devenv sync` from that repo's root
 - Run the `bootstrap` workflow locally with `gh act`: `make test`
 
+## Cross-repo upgrade checks
+
+When you change devenv's internal Python pin or its Colima or Lima versions, smoke-test the installed devenv against `~/dev/sentry`. Do this before calling the upgrade complete.
+
+- From `~/dev/sentry`, run `devenv sync` and check `devenv --version`, `.venv/bin/python --version`, `colima version`, and `limactl --version`.
+- Read Sentry's `.python-version`. Devenv's internal Python pin does not set Sentry's virtualenv version.
+- Run `devservices up` and confirm `devservices status` reports the needed services as healthy.
+- Run a few targeted pytest tests that use those services. Confirm they use a test database, then report the commands and results.
+- Do not stop, delete, or prune the developer's default Colima VM.
+- Use a disposable profile only after verifying that every VM and data path is isolated.
+
 ## Invariants
 
 - Python is pinned to 3.13.16 (`requires-python` in `pyproject.toml`, `.python-version`).

README.md

@@ -75,7 +75,9 @@ If your devenv is old enough that it has no `update` command, run:
 
 `devenv colima start`
 
-Starts colima with resources sized for your machine. The default `colima start` may underprovision resources. Run `devenv colima start` on a new machine, and again after `colima delete`. The other subcommands are `restart`, `check`, and `stop`.
+Starts or creates devenv's Colima VM with resources sized for your machine. Existing VM settings are kept. The other subcommands are `restart`, `check`, and `stop`.
+
+On affected Apple M4/M5 systems, this and `devenv colima restart` check whether the VM kernel reports SVE support the CPU does not have. Devenv upgrades the VM kernel when needed, restarts once, and verifies the result. `devenv doctor` checks this too.
 
 `devenv pin_gha [file ...]`
 
@@ -268,7 +270,40 @@ def main(context: dict[str, str]) -> int:
 
 ### colima
 
-On macOS, `devenv bootstrap` installs colima and the docker CLI globally. If you installed devenv before 1.14.0, run `devenv update` to get them.
+`devenv bootstrap` and `devenv update` install Colima `v0.10.3` and Lima `v2.2.1` on macOS.
+
+#### New macOS installations
+
+Colima 0.10.3 creates new VMs from its Ubuntu 24.04.4 image, whose kernel is expected to report SVE correctly. M4 Pro validation confirmed this on that host. New VMs use a separate persistent 512 GiB container data disk and Docker's containerd image store. Inotify is off by default.
+
+#### Existing macOS installations
+
+Updating devenv keeps the existing VM, Docker images, containers, and named volumes. It does not recreate the VM or convert its disk layout. Colima 0.8.1 VMs keep Docker data on their root disk; newer VMs use a separate persistent data disk. Do not delete a legacy VM as routine cleanup: its containers and named volumes can be lost with its root disk.
+
+`devenv update` does not stop a running VM. Before the next upgraded start, devenv checks a running daemon through the `colima` Docker context and records its store choice in `~/.colima/default/colima.yaml`. For a stopped VM, it uses Colima's persisted store and Lima disk metadata to identify the layout. If the evidence is ambiguous, startup stops with instructions to set `docker.features.containerd-snapshotter` explicitly. Existing explicit values are preserved, and devenv does not remove Docker data.
+
+Affected M4/M5 Macs may need an in-VM kernel update because existing VMs keep their original kernel. Devenv installs and boots the update when needed. If a previous repair installed the kernel but did not boot it, run `devenv colima restart`. This does not convert the VM to the new data disk layout.
+
+Lima 2.1 changed the instance disk layout. After starting a VM with Lima 2.x, Lima 1.x cannot start that instance. Downgrading devenv or its binaries does not restore the old layout. See [Lima's compatibility discussion](https://github.com/lima-vm/lima/discussions/4747).
+
+#### Check or change the Docker image store
+
+Run `docker --context=colima info` to see the active storage driver. The classic store reports `overlay2`. The containerd store reports `io.containerd.snapshotter.v1` in the driver status. Use the explicit `colima` context so Docker Desktop's current context does not affect the result.
+
+If images or containers appear missing after an upgrade, inspect `docker --context=colima info`, `docker --context=colima images`, `docker --context=colima ps -a`, and `docker --context=colima volume ls`. Check `docker.features.containerd-snapshotter` in `~/.colima/default/colima.yaml` against the store containing your data. For a legacy `overlay2` VM, use:
+
+```yaml
+docker:
+  features:
+    buildkit: true
+    containerd-snapshotter: false
+```
+
+Set it to `true` only after confirming the VM's data is in the containerd store. Start Colima after correcting the value. Switching stores hides images and containers in the other store. Do not delete the VM or its disks to diagnose the problem.
+
+#### Enable inotify propagation
+
+To propagate host file events into the VM, set `mountInotify: true` in `~/.colima/default/colima.yaml` or start Colima with `--mount-inotify`. Devenv preserves an existing `true` setting. Inotify is off for new VMs because Colima watches configured mounts, including devenv's writable home mount. Watching a large home directory can increase host CPU and memory use; see [Colima issue 1569](https://github.com/abiosoft/colima/issues/1569).
 
 ### gcloud
 

devenv/checks/colimaSve.py

@@ -0,0 +1,44 @@
+from __future__ import annotations
+
+from devenv.lib import colima
+from devenv.lib_check.types import checker
+from devenv.lib_check.types import fixer
+
+tags: set[str] = {"builtin", "colima"}
+name = "colima's VM shouldn't report SVE support on CPUs without SVE"
+
+
+@checker
+def check() -> tuple[bool, str]:
+    try:
+        misreported = colima.vm_misreports_sve()
+    except RuntimeError:
+        # colima isn't running, which another check covers
+        return True, ""
+
+    if misreported:
+        return (
+            False,
+            "colima's VM kernel reports SVE2, but this CPU has no SVE. "
+            "Programs that use it crash with 'Illegal instruction'.",
+        )
+
+    return True, ""
+
+
+@fixer
+def fix() -> tuple[bool, str]:
+    # upgrades the VM's kernel and restarts colima to boot it
+    status = colima.start(restart=True)
+    if status != colima.ColimaStatus.UP:
+        return False, "colima is unhealthy"
+
+    # start only warns if it couldn't upgrade the kernel
+    try:
+        misreported = colima.vm_misreports_sve()
+    except RuntimeError as e:
+        return False, f"{e}"
+    if misreported:
+        return False, "colima's VM still misreports SVE, see above"
+
+    return True, ""

devenv/checks/limaDns.py

@@ -10,35 +10,49 @@
 tags: set[str] = {"builtin", "colima"}
 name = "colima's DNS isn't working"
 
+_DNS_CHECK = """
+import socket
+
+try:
+    socket.getaddrinfo("ghcr.io", None)
+except socket.gaierror as e:
+    raise SystemExit(f"failed to resolve ghcr.io: {e}")
+"""
+
+_DNS_CONFIG = """
+from pathlib import Path
+
+resolv_conf = Path("/etc/resolv.conf")
+print(resolv_conf.resolve())
+print(resolv_conf.read_text())
+"""
+
+
+def _resolver_service(target: str, resolv_conf: str) -> str | None:
+    if target.startswith("/run/systemd/resolve/"):
+        return "systemd-resolved.service"
+
+    nameservers = {
+        line.split()[1]
+        for line in resolv_conf.splitlines()
+        if line.strip().startswith("nameserver ") and len(line.split()) == 2
+    }
+    if "127.0.0.53" in nameservers:
+        return "systemd-resolved.service"
+    if "# Generated by Colima" in resolv_conf and nameservers:
+        return "dnsmasq.service"
+    return None
+
 
 @checker
 def check() -> tuple[bool, str]:
-    # dns resolution can... stop working if colima's running
-    # and wifi changes to some other network that gives macos some
-    # weird nameservers
-
     status = colima.check()
     if status != colima.ColimaStatus.UP:
         return False, "Colima isn't running."
 
     try:
         proc.run(
-            (
-                "colima",
-                "exec",
-                "--",
-                "python3",
-                "-Su",
-                "-c",
-                """
-import socket
-
-try:
-    socket.getaddrinfo("ghcr.io", None)
-except socket.gaierror as e:
-    raise SystemExit(f"failed to resolve ghcr.io: {e}")
-""",
-            ),
+            ("colima", "exec", "--", "python3", "-Su", "-c", _DNS_CHECK),
             stdout=True,
         )
     except RuntimeError as e:
@@ -50,25 +64,35 @@ def check() -> tuple[bool, str]:
 @fixer
 def fix() -> tuple[bool, str]:
     status = colima.start()
-    if status == colima.ColimaStatus.UNHEALTHY:
-        return False, "colima started, but it's unhealthy"
+    if status != colima.ColimaStatus.UP:
+        return False, f"Colima is not healthy: {status.name.lower()}"
+
+    try:
+        config = proc.run(
+            ("colima", "exec", "--", "python3", "-Su", "-c", _DNS_CONFIG),
+            stdout=True,
+        )
+    except RuntimeError as e:
+        return False, f"Unable to inspect Colima's /etc/resolv.conf: {e}"
+
+    target, _, resolv_conf = config.partition("\n")
+    service = _resolver_service(target, resolv_conf)
+    if service is None:
+        return (
+            False,
+            "Unable to identify Colima's DNS resolver from /etc/resolv.conf. "
+            "Inspect the VM resolver configuration and restart the selected "
+            "resolver manually.",
+        )
 
     try:
         proc.run(
-            (
-                "colima",
-                "exec",
-                "--",
-                "sudo",
-                "systemctl",
-                "restart",
-                "systemd-resolved.service",
-            ),
+            ("colima", "exec", "--", "sudo", "systemctl", "restart", service),
             stdout=True,
         )
     except RuntimeError as e:
         print(f"""
-failed to restart the vm's resolved:
+failed to restart the vm's {service}:
 {e}
 
 we're going to try restarting colima
@@ -80,4 +104,7 @@ def fix() -> tuple[bool, str]:
         except RuntimeError as e:
             return False, f"{e}"
 
+    healthy, detail = check()
+    if not healthy:
+        return False, f"DNS resolution is still failing after repair: {detail}"
     return True, ""

devenv/colima.py

@@ -18,17 +18,8 @@ def main(context: Context, argv: Sequence[str] | None = None) -> int:
 
     args = parser.parse_args(argv)
 
-    # TODO: in addition to returning 1 we should print colima logs
-    #       (and/or send to sentry)
-
     if args.command == "start":
         status = colima.start()
-        if status == colima.ColimaStatus.UNHEALTHY:
-            # https://github.com/abiosoft/colima/issues/949
-            print("colima seems unhealthy, we'll try restarting once")
-            status = colima.restart()
-            if status != colima.ColimaStatus.UP:
-                return 1
         if status != colima.ColimaStatus.UP:
             return 1
     elif args.command == "restart":