Repository navigation
feat(runtime): Default to hull on Linux when hull is installed - #683
Conversation
dd3de2f to
754d3fd
Compare
754d3fd to
37bf6ee
Compare
37bf6ee to
7db2f8e
Compare
7db2f8e to
fa29315
Compare
fa29315 to
c999cb2
Compare
cmainas
left a comment
There was a problem hiding this comment.
Thank you @ananos , just a few comments.
Also, there are still references around the repo and docs naming nerdctl as the defaul (e.g. docs/security.md, docs/troubleshooting.md).
Also, script/check-guest-image.sh:117-123 copies brig's runtime choice and still picks nerdctl on Linux when BRIG_RUNTIME is unset, so on a host with hull on PATH it boots through hull and then runs every check with nerdctl exec. It's outside this diff, but it changes with this default.
| // gateway, so a host that carries it gets it. brig-standalone-linux ships | ||
| // hull beside nerdctl since v0.1.0-rc22; a host without it keeps nerdctl, and | ||
| // BRIG_RUNTIME still names either one. | ||
| func defaultKind() string { |
There was a problem hiding this comment.
After an upgrade, a host with hull on PATH moves sessions booted on nerdctl under hull. The index records a session's runtime only since 154f53e (2026-11-17), and an entry with no runtime belongs to whichever runtime reads it: recordedRuntime (internal/wrap/index.go:432) returns "" for it, so the run is not refused, and OtherRuntimeSessions (index.go:527) skips it, so brig ls does not keep it. Every such entry was booted on nerdctl, the old default.
A session runs on nerdctl from a release before 154f53e, and brig is upgraded on a host with hull on PATH. brig ls lists through hull, which cannot see the nerdctl sandbox, and the prune drops its entry. The next brig run of it boots a second VM on the same guest home, or brig rm deletes the home while the nerdctl VM still runs. This is what 154f53e was written to prevent. The commit message and docs/cli.md say "one booted on nerdctl is refused under hull", which holds only for entries written since 154f53e.
Read an entry with no runtime as nerdctl on Linux, or keep nerdctl the default while such entries exist.
There was a problem hiding this comment.
good catch, that was exactly the hole 154f53e closed. On Linux an index entry with no recorded runtime is now read as nerdctl's, the default every such entry was booted under, in one place every reader goes through (entryRuntime): runtimeRefusal/RuntimeRefusal for run, stop, rm, rm --target and --forget, and OtherRuntimeSessions for the brig ls prune. Under hull on Linux such a session is refused like a nerdctl-recorded one, and brig ls keeps it; under nerdctl (and docker through it) it is still the run's own; on macOS nothing changes. Tests: TestOnLinuxHullRefusesASessionWithNoRecordedRuntime and TestOnLinuxListingUnderHullKeepsASessionWithNoRecordedRuntime (both negative: no boot, stop or remove, no tokens revoked, home and entry untouched), plus the nerdctl and macOS sides. A docker-only host also gets the right hint now: BRIG_RUNTIME=nerdctl reaches such a session through docker, so the refusal no longer says --forget. One case this cannot tell apart: a session booted with BRIG_RUNTIME=hull on Linux before 154f53e also has no runtime, and hull now refuses it as nerdctl's; brig rm --forget is the way past, and the comment says so.
| | Variable | Default | Meaning | | ||
| | --- | --- | --- | | ||
| | `BRIG_RUNTIME` (global only) | `hull` on macOS, `nerdctl` on Linux | which runtime to drive | | ||
| | `BRIG_RUNTIME` (global only) | `hull` on macOS. On Linux, `hull` when a `hull` is on `PATH`, else `nerdctl` | which runtime to drive. A session stays on the runtime that booted it: one booted on nerdctl is refused under hull until `BRIG_RUNTIME=nerdctl` names it again or `brig rm` removes it | |
There was a problem hiding this comment.
"until … brig rm removes it": brig rm under hull is refused by RuntimeRefusal too. Only BRIG_RUNTIME=nerdctl brig rm (or brig rm --forget once nerdctl is gone) works. Suggest naming that instead.
There was a problem hiding this comment.
right, plain brig rm is refused under hull too. The row now says that under hull a run, brig stop and brig rm refuse a session booted on nerdctl, that on Linux one an older release recorded with no runtime counts as booted on nerdctl, and that BRIG_RUNTIME=nerdctl brig rm <ref> removes it and brig rm --forget <ref> forgets it once nerdctl is gone. I corrected the same claim in the commit message.
brig drove nerdctl on Linux unless BRIG_RUNTIME named hull. hull on Linux now serves what nerdctl does through the same urunc shim, and also what hull on hvi serves on the same network gateway: an egress policy, routes at brig.internal, diverts and published ports, including publishing on a running sandbox, which nerdctl cannot. So a host that carries hull gets it by default. The default is hull when a hull is on PATH, and nerdctl otherwise, so a host installed from a bundle older than brig-standalone-linux v0.1.0-rc22, which carries no hull, keeps working on nerdctl. BRIG_RUNTIME still names either one, and a session stays on the runtime that booted it: under hull, a run, brig stop and brig rm of one booted on nerdctl are refused with the setting that reaches it. BRIG_RUNTIME=nerdctl brig rm removes it, and brig rm --forget forgets it once nerdctl is gone. On Linux a session an older release recorded with no runtime counts as booted on nerdctl, the default it ran under. hull on Linux still refuses an image whose USER is a name, and a numeric non-root user boots without a network. The shipped profiles pin root tags; the docs say BRIG_RUNTIME=nerdctl runs the others. Signed-off-by: Anastassios Nanos <ananos@nofire.ai>
The session index records the runtime only since 154f53e. An entry without it belonged to whichever runtime read it. On Linux every such entry was booted on nerdctl, the default before hull. With hull the default where it is on PATH, an upgrade moves such a host under hull with those entries in the index. hull cannot see their sandboxes. Its brig ls pruned the entry, its next run booted a second VM on the same guest home, and its brig rm deleted the home under the VM still running on nerdctl. That is what 154f53e was written to prevent. On Linux hull now reads such an entry as nerdctl's. A run, stop and rm under hull refuse it as they refuse a session recorded on nerdctl, and brig ls keeps it and lists it as on nerdctl. nerdctl, under either of its binaries, still reads it as its own, and its next run records nerdctl with it. On macOS nothing changes. Every reader goes through entryRuntime, and the host is a variable, so the tests read the index as either host on any host. The refusal asks the runtime its kind only when an entry names the sandbox, and on macOS not for an entry with no runtime, as before. The cmd/brig fixtures that record a session for a fake hull now name hull, since on Linux an entry with no runtime is no longer hull's. The internal/wrap cases that record a session before they set the fake hull read the index as macOS, unless a case names the host. Refs: brig-sh/hull#105 Signed-off-by: Anastassios Nanos <ananos@nofire.ai>
brig drives hull on Linux when a hull is on PATH, and nerdctl otherwise. Several pages, the install.sh header and the runtime package comment still said that nerdctl is the Linux runtime, or that only nerdctl enforces a policy, serves routes or boots a generic profile there. A reader on a host with hull, which is the default host now, read those as the runtime they run and as features it lacks. Each now says hull when a hull is on PATH, and nerdctl otherwise, or names hull beside nerdctl where both do the thing. Sentences about how nerdctl itself behaves are unchanged. Refs: brig-sh/hull#105 Signed-off-by: Anastassios Nanos <ananos@nofire.ai>
check-guest-image.sh boots the image through brig and then runs every
check through the runtime's own exec, so it has to find the runtime brig
finds. It copied the old default and took nerdctl on Linux whenever
BRIG_RUNTIME was unset. On a host with hull on PATH brig boots through
hull, and nerdctl exec then found no sandbox of that name, so every
check failed against an image that may be fine.
It now takes hull on Linux when a hull is on PATH and nerdctl otherwise,
and BRIG_RUNTIME still wins. The checks already exec through hull with
the command after --, which hull takes on Linux as on macOS.
shellcheck now passes on the script. The runtimeBin it reads from a
profile is expanded from ~/ correctly: bash tilde-expands the pattern of
${RT#~/}, so the strip removed nothing and left $HOME/~/... in RT. The
other findings were false positives, quieted with a quote or a directive.
Refs: brig-sh/hull#105
Signed-off-by: Anastassios Nanos <ananos@nofire.ai>
The conformance suite's nerdctl backend ran brig with BRIG_RUNTIME_BIN=nerdctl and the shell's BRIG_RUNTIME, which is usually unset. brig takes the runtime from BRIG_RUNTIME before it reads BRIG_RUNTIME_BIN, and unset it is now hull on a Linux host with hull on PATH. brig then refused the nerdctl binary as the other runtime, and every case of script/egress-conformance.sh failed on a host that carries both, which a bundle from v0.1.0-rc22 on is. The nerdctl backend now runs brig with BRIG_RUNTIME=nerdctl, once, in place of any value the shell exported. The hvi backend names none and keeps the shell's, as before. Refs: brig-sh/hull#105 Signed-off-by: Anastassios Nanos <ananos@nofire.ai>
On Linux hull refuses an entry with no runtime as nerdctl's. On a host with docker and no nerdctl, the refusal added that brig finds no nerdctl on PATH and pointed at brig rm --forget. Such an entry was booted by the nerdctl runtime through docker, which it drives when no nerdctl is on PATH, and the BRIG_RUNTIME=nerdctl the refusal names reaches it there. Forgetting it instead would leave its sandbox running with no session. For an entry with no runtime, a docker on PATH now counts as the runtime being there, and the refusal names only the setting. An entry that records nerdctl is unchanged: nerdctl booted it, and docker does not see it. Refs: brig-sh/hull#105 Signed-off-by: Anastassios Nanos <ananos@nofire.ai>
c999cb2 to
c1843c7
Compare
|
thanks @cmainas! On the two general points: the docs and comments that still named nerdctl as the Linux default now say hull when a hull is on |
Summary
brig drove nerdctl on Linux unless
BRIG_RUNTIMEnamed hull. With #675 and #682, hull on Linux serves what nerdctl does through the same urunc shim, plus what hull on hvi serves on the same network gateway, including publishing on a running sandbox. So a host that carries hull gets it by default: hull when ahullis onPATH, nerdctl otherwise.Stacked on #682. The bundle side, which sets
BRIG_RUNTIMEin its launcher, is NOFireAI/brig-standalone-linux#44.Related issues
Refs: brig-sh/hull#105
Changes
internal/runtime/default_other.go:defaultKindis hull whenhullis onPATH, else nerdctl. A host installed from a bundle older than v0.1.0-rc22, which carries no hull, stays on nerdctl.default_other_test.go, Linux only).BRIG_RUNTIMErow in cli.md, and the limitations section of hull on Linux point atBRIG_RUNTIME=nerdctlfor what hull refuses.A session stays on the runtime that booted it (154f53e): under hull, a run,
brig stopandbrig rmof one booted on nerdctl are refused, naming the setting that reaches it.BRIG_RUNTIME=nerdctl brig rm <ref>removes it, andbrig rm --forget <ref>forgets it once nerdctl is gone. On Linux a session an older release recorded with no runtime counts as booted on nerdctl, so it is kept and refused the same way rather than booted a second time. That is a visible change for anyone with nerdctl sandboxes; the release notes should say it.Known limits, unchanged: hull on Linux refuses an image whose
USERis a name, and a numeric non-root user boots without a network. The shipped profiles pin root tags.Checklist
make allpasses (vet, test, build)script/smoke.shpassesI have rungo test ./... -race, if the change touches concurrency, subprocesses or the daemonbrig run <agent>), if it touches the run, exec or credential pathThe new test builds only on Linux, so CI is its first run. The real-runtime evidence for the paths the default now takes is in #682; the default itself is not yet run on a host.
Review round 1 (cmainas): b3d7709 reads a session with no recorded runtime as nerdctl on Linux; 6a49377 names hull as the Linux default across the docs; 7c4f02a makes
script/check-guest-image.shpick the runtime as brig does; 86709bc names nerdctl to brig on the conformance suite's nerdctl backend; c1843c7 gives a docker-only host the right hint. The smoke and the wrap, cmd, runtime and conformance tests pass natively on a Linux host.