Repository navigation
fix(colima): Upgrade Colima/Lima and repair M4+ SVE without data loss - #236
Merged
Merged
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
joshuarli
approved these changes
Oct 14, 2026
joshuarli
self-requested a review
October 14, 2026 16:57
joshuarli
requested changes
Oct 14, 2026
joshuarli
left a comment
joshuarli
left a comment
Member
There was a problem hiding this comment.
i gotta get some m5 hardware to test this on! one sec
joshuarli
changed the title
colima 0.8.1 creates VMs from Ubuntu 24.04.1 with Linux 6.8.0-50, which derives the SVE2 hwcaps from an ID register that SME also populates without checking that SVE exists (fixed upstream in 064737920bdb, Ubuntu 6.8.0-64.67). Apple M4 and newer have SME but no SVE, and macOS 15.2+ exposes SME to VMs, so arm64 containers are told they can use SVE2 and anything that believes it dies with SIGILL: OpenSSL 4 in cryptography >= 47 at import time (poetry, `import jwt`), and numba's JIT (seer works around it with NUMBA_DISABLE_JIT=1). Upgrading colima doesn't help existing VMs since its image is only used at creation, and recreating a 0.8.1 VM loses its containers and volumes. Installing linux-image-virtual in the VM and restarting does fix it without losing data, so `devenv colima start` and `restart` now do that when the VM's /proc/cpuinfo lists SVE hwcaps without sve. The VM is only probed on hosts with FEAT_SME, so M1-M3 and Intel Macs are unaffected. It restarts at most once per run and never just to boot a kernel that was already installed, so a VM that won't boot the new kernel can't end up restarted on every start. Failures only warn, since everything that doesn't use SVE still works. `devenv doctor` gets a matching check whose fix runs the same upgrade. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
proc.run now flushes stdout before running a command. Otherwise, when stdout isn't a tty (CI, devservices, `| tee`), everything devenv prints lands after the output of the commands it runs, so the explanation of the kernel upgrade showed up after apt's output instead of before it. Also say so when colima doesn't come back after restarting into the upgraded kernel, and don't suggest `devenv colima restart` without saying what to do if it doesn't help. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
colima.start() returns UP even when it only warned that it couldn't upgrade the VM's kernel, so the fix claimed success while the VM still misreported SVE, and only the final re-check said otherwise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Grok 4.7 <noreply@x.ai>
devenv colima restart booted the current kernel and then, when that kernel was still the newest, installed a new one and booted again. Install first so the requested restart is the boot. The unhealthy retry after start relaunches the VM without running the upgrade again. Co-Authored-By: Grok 4.7 <noreply@x.ai>
A restart that found the fixed kernel already installed warned the user to restart, then rebooted without checking which kernel came up. Boot that kernel, stop colima gracefully first, and report what is running. An unhealthy first start relaunches, then runs start again so the upgrade still happens when the VM never came up. Co-Authored-By: Grok 4.7 <noreply@x.ai>
Upgrade Colima and Lima while retaining existing Docker storage choices, guarding ambiguous migration state, and preserving developer-owned VM data. Keep kernel repair bounded across health retries and support both Colima resolver generations. Co-Authored-By: Codex (GPT-6) <codex@openai.com>
Document new and existing VM behavior, storage inspection and recovery, inotify opt-in, and Lima 2 disk compatibility so developers can assess the upgrade before starting it. Co-Authored-By: Codex (GPT-6) <codex@openai.com>
Do not require Lima’s optional Intel MCP plugin for the Colima runtime. Validate staged release contents and versions before replacing files, then restore the previous managed installation if replacement fails. Co-Authored-By: Codex (GPT-6) <codex@openai.com>
joshuarli
force-pushed
the
ivan/colima-sve-kernel-upgrade
branch
from
October 20, 2026 17:47
4bc67ae to
9a5474e
Compare
Lima 2.2.1 delegates the lima wrapper version command through LIMACTL. Pin that lookup to the matching staged or installed binary so host PATH state cannot break bootstrap. Co-Authored-By: Codex (GPT-6) <codex@openai.com>
Member
|
Both findings from this review are fixed in 9a5474e, and the inline threads are resolved. The Lima payload check no longer treats the optional Intel MCP plugin as a required driver. Installation validates the staged release and restores the prior files on replacement failure. The current head also pins the Lima 2.2.1 wrapper version lookup to its bundled limactl. |
Lima defaults to ~/.lima, while Colima stores its default instance under ~/.colima/_lima. Use the effective Colima Lima home for instance discovery and disk metadata so unrelated Lima disks cannot change storage migration decisions. Co-Authored-By: Codex (GPT-6) <codex@openai.com>
Member
|
The latest finding is fixed in 0bee0bf, and its inline thread is resolved. The stopped-VM probe now sets Lima home to Colima’s effective instance directory and instance discovery uses the same path. Tests cover standalone ~/.lima data, Colima persistent disks, and an explicit LIMA_HOME. |
joshuarli
self-requested a review
October 20, 2026 18:33
Colima generates a fresh profile during startup. Persist and verify the selected image store so later starts do not mistake a new containerd VM for an ambiguous legacy VM. Co-Authored-By: Codex (GPT-6) <codex@openai.com>
Keep the required format check green for the storage lifecycle regression. Co-Authored-By: Codex (GPT-6) <codex@openai.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 9e61371. Configure here.
A fresh VM becomes an existing instance after its first Colima start, so kernel and health retries must not replay first-provision settings. Co-Authored-By: OpenAI Codex (GPT-6) <noreply@openai.com>
The lint job rejects the multiline recovery call, so keep it in the configured Ruff format. Co-Authored-By: OpenAI Codex (GPT-6) <noreply@openai.com>
joshuarli
approved these changes
Oct 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why these changes ship together
This PR consolidates #240 into #236. The runtime upgrade fixes new VMs, while existing VMs keep their original kernel and need an in-VM repair. Shipping both together gives new and existing macOS installs a supported path without hiding existing Docker data.
M4/M5 SVE kernel repair
M4/M5 hosts can expose SME without SVE. Older Ubuntu 6.8 kernels can incorrectly derive SVE2 hardware capabilities from an SME-populated register without checking SVE. This can make OpenSSL 4, Python cryptography, and numba select unsupported instructions. The upstream fix is 064737920bdb.
On affected Apple Silicon hosts, devenv probes the VM, installs the Ubuntu kernel update when needed, boots it, then checks the reported capabilities again. A bounded health recovery does not reinstall the kernel or loop. Healthy VMs stop gracefully for a kernel reboot. Force stop is limited to unhealthy or stuck VMs.
devenv doctorreports the verified post-repair result. Apt failures warn without unnecessarily blocking other workloads.New Colima 0.10.3 VMs use Ubuntu 24.04.4, which is expected to include the corrected kernel based on the M4 Pro validation in #240. Existing VMs retain their old kernel until repair runs.
Runtime and data behavior
v0.10.3uses the verified native artifacts: Darwin x86_64 SHA-2563082737fe8a98afda11cba7d9a20b6e56fe80c6153464beda04bec630758770b; Darwin arm64980ad8bf61a4ca370243f4cb41401a61276dcd2c2502bee7b9b86f9250169f34.v2.2.1uses the verified native release tarballs: Darwin x86_64 SHA-2566b776e4bc41f358e6ee01fefcc78e2eb26f11635a0fcff632234af77a566c6c4; Darwin arm649e9eacce88f37e185c346bad73aa6136f738d8cdf8c3bb23cd42b071824bc66e. Lima stays in its upstreambin/,share/lima/, andlibexec/lima/layout. Installation validates both executables, their versions, the native guest agent, templates, and required drivers. Incomplete installs are repaired without removing the previous working install first.containerd-snapshotter: trueto an existing profile before first startup, or to the Colima-generated profile immediately after a new VM starts. The first managed start explicitly saves Colima's generated profile. Explicit choices in a profile or default template are preserved. On later starts, the saved expected backend remains in the policy and Docker is checked after VM startup or restart.overlay2VMs receivedocker.features.containerd-snapshotter: falsebefore their next upgraded start, unless the user explicitly chose a backend. Running VMs are not stopped during update. The code checks the reachable Docker daemon throughdocker --context=colima infoand inspects persisted Colima/Lima disk metadata for stopped VMs. It sets Lima's home to Colima's effective instance directory. That defaults to~/.colima/_limaunlessLIMA_HOMEis set (Colima v0.10.3 source, Lima environment variables). Standalone~/.limadisks cannot affect that decision. Ambiguous or malformed state blocks startup with recovery guidance instead of changing the backend silently.--mount-inotify. Existing savedmountInotifyvalues remain intact. Developers who need host-to-container file events can opt in through Colima configuration. Watching a writable home mount can consume substantial host CPU and memory (upstream report).systemd-resolvedor dnsmasq. It retries a selected but crashed dnsmasq, verifies name resolution, and uses the existing Colima restart recovery when resolution still fails.~/.limaVMs,~/.colimaprofiles and disks, credentials, and unrelated binaries, including during repair or interrupted installation.--cpus; existing VM resource settings remain fixed. VZ, Rosetta, virtiofs mounts, DNS settings, resource sizing, and Docker context handling are retained.Verification
Local checks on
9e61371with CPython 3.13.16:uv run pytest tests/lib/test_colima_storage.py tests/lib/test_colima.py -q: 62 passed. Includes fresh VM → start → stop → start, persistence and backend verification on each start, explicit profile/template choices, legacy overlay2 verification, and bounded SVE health recovery.devenv/lib/colima.pyandtests/lib/test_colima_storage.py.git diff --checkpassed.9e61371, GitHub test, lint, both bootstrap jobs, Warden and its security review, CodeQL, dependency review, secret scan, Semgrep, and Socket checks passed. Cursor Bugbot is still in progress.A local first-boot smoke test was performed by running
devenv syncin~/dev/sentrywith devenv 1.29.0, CPython 3.13.16, Colima 0.10.3, and Lima 2.2.1. It created a fresh VZ VM with a persistent container data disk. Docker 29.5.2 reportedio.containerd.snapshotter.v1; Postgres and Redis became healthy, migrations completed, and the local admin user was created. The VM now contains Sentry devservices data, so it was not stopped or restarted for a lifecycle test. No legacy overlay2 VM upgrade was run. M5 hardware was unavailable for host-level kernel validation.Rollout
After release,
devenv updateinstalls the compatible Colima/Lima pair. New macOS VMs get the updated image, containerd, a persistent data disk, and inotify off. Existing VMs retain their Docker backend and disk layout; affected M4/M5 hosts repair the old kernel when needed. Check the store withdocker --context=colima info. Opt into inotify only when file-event propagation is needed.