fix(daemon): SIGABRT on shutdown, SIGCHLD=SIG_IGN breaking every exec, in-process runCrate on HTTP workers, fd CLOEXEC, IPv6-route gateway, stack DNS never stopped, physmem 500 (1.1.26) - #229
Merged
Conversation
…, in-process runCrate on HTTP workers, fd CLOEXEC, IPv6-route gateway detect, stack DNS never stopped, physmem 500 (1.1.26)
Seven correctness fixes from a correctness-lens audit — "the daemon
works":
- server.cpp: the Unix-socket httplib::Server lived only in its thread's
lambda; stop() never stopped it and ~Impl destroyed a joinable thread
-> std::terminate on every clean `service crated stop`. Now an Impl
member, stopped and joined.
- ws_console.cpp: signal(SIGCHLD, SIG_IGN) was process-wide; on FreeBSD
that makes waitpid() return ECHILD for ALL children, so every
Util::execCommand*/execPipeline threw "waitpid failed" the moment
console.port was enabled. Removed; session shells are reaped
explicitly (SIGTERM, grace, SIGKILL + blocking wait).
- routes.cpp: POST .../start and .../restart called runCrate() in-process
on the httplib worker: blocked the worker for the jail's lifetime
(~8 starts -> whole API incl. /healthz dead), installed SIGINT/SIGTERM
handlers process-wide (`service crated stop` hung), and keyed
jailXname/FwSlots/FwUsers on getpid() so all daemon-started jails
shared one key (slot overwrite, NAT rule removed while jails ran).
Now fork+setsid+exec `crate run` like the control-socket path; the
response is async ({"started":true,"async":true,"pid":N}).
- util.cpp/jail_query.cpp/control_socket.cpp/privops_listener.cpp/
ws_console.cpp: pipe2(O_CLOEXEC) for capture pipes and SOCK_CLOEXEC
for the hand-rolled listeners, so long-lived forked children no longer
inherit a pipe write end (request thread hung on EOF) or a LISTEN fd
(EADDRINUSE on restart).
- run_net.cpp detectGateway: `netstat -rn -f inet` and `>= 4` tokens —
an IPv6 default route doubled the `default` lines and every NAT-mode
`crate run` failed.
- stack.cpp: unbound now foreground (do-daemonize: no) under setsid, and
stopStackDns signals the pid from the per-network unbound.pid — DNS
was never actually stopped before (orphan bound to :53).
- routes.cpp/util.{h,cpp}: hw.physmem (ULONG) read via new
getSysctlUInt64; the 4-byte read made GET /api/v1/host a permanent 500.
All runtime-only, compile-gated by FreeBSD lite. Remaining audit items
(httplib/accept fd hygiene, run.cpp teardown-after-throw leaks, FwUsers
GC, listener timeouts, `stack up` in-process runCrate, LOW batch)
recorded in TODO. Bumps to 1.1.26; CHANGELOG + trust-model.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X6t6tzVypHye5bDGLxzmZK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Seven correctness fixes from a correctness-lens audit (exception safety, process/fd hygiene, threading) — the "daemon works" batch. Three of them break
cratedin ordinary configurations today.1.
cratedaborted with SIGABRT on every clean shutdown —daemon/server.cpp(HIGH)The Unix-socket
httplib::Serverlived only inside its thread's lambda:Server::stop()could never stop it,listen()never returned, and~Impldestroyed a still-joinablestd::thread→std::terminate. With the defaultunixSocketconfigured this hit everyservice crated stop. The UDS server is now anImplmember;stop()stops it and joins its thread.2. Enabling
console.portbroke every exec in the daemon —daemon/ws_console.cpp(HIGH)WsConsole::startsetsignal(SIGCHLD, SIG_IGN)process-wide. On FreeBSD that flagsPS_NOCLDWAIT: children are auto-reaped andwaitpid()returnsECHILDfor all of crated's children, so everyUtil::execCommand*/execPipeline*(privops verbs, stats, export/import, control-socket ops) threw "waitpid failed" even on success. Removed; session shells are reaped explicitly (SIGTERM → ~2s grace → SIGKILL + blocking wait).3.
POST …/start/…/restartran the container in-process —daemon/routes.cpp(HIGH)Both called
runCrate()on the httplib worker thread: (a) the worker blocked for the jail's lifetime — after ~8 starts the whole API incl./healthzstopped responding; (b)runCrateinstalled SIGINT/SIGTERM handlers process-wide →service crated stophung; (c)jailXname/FwSlots/FwUsersare keyed ongetpid()→ all daemon-started jails shared one key (slot overwrite; shared NAT rule removed while other jails ran). Both routes now fork+setsid+execcrate runas its own process, exactly like the control-socket start path.API note: the response is now async —
{"started":true,"async":true,"pid":N,…}returns on spawn; pollGET /api/v1/containers/:name.4. fds leaked into long-lived children —
lib/util.cpp,lib/jail_query.cpp, three listeners (HIGH)Capture pipes had no
O_CLOEXEC, so a pipe's write end leaked into any child forked concurrently on another thread (control-socketcrate run, ws-console shells), and the reading request thread hung on an EOF that only came when that child exited. Listening sockets leaked the same way (EADDRINUSEon restart). Nowpipe2(O_CLOEXEC)+SOCK_CLOEXECon the hand-rolled listeners. httplib's TCP listener andaccept()ed fds are recorded inTODOas follow-up.5. NAT-mode
crate runfailed on any host with an IPv6 default route —lib/run_net.cpp(MED)detectGatewayrannetstat -rnwithout-f inet; the inet6defaultline doubled the token count and the strict!= 4check threw. Now-f inetand>= 4, mirroring the IPv6 query.6. Stack DNS was never stopped —
lib/stack.cpp(MED)unbound ran with
do-daemonize: yes, so the returned pid was the short-lived launcher (and the 200ms liveness check reported "exited immediately" on success);stack downpassed-1→ orphan unbound stayed bound to:53. unbound now runs foreground undersetsid;stopStackDnssignals the pid from the per-networkunbound.pid.7.
GET /api/v1/hostwas a permanent 500 on 64-bit —daemon/routes.cpp,lib/util.cpp(MED)hw.physmemisCTLTYPE_ULONG(8 bytes) read through the 4-bytegetSysctlInt→ENOMEM. NewUtil::getSysctlUInt64.Testing
All seven are runtime-only (daemon / FreeBSD runtime — no pure unit surface). Compile-gated by the FreeBSD lite build. Exercising on a live host is recommended, especially the async start/restart semantics and clean shutdown.
Remaining (in
TODO)stack upstill callsrunCratein-process; httplib/accept fd hygiene;run.cppteardown-after-throw leaks;FwUsersdead-pid GC; listener read timeouts; a LOW batch.Version
Bumps to 1.1.26;
CHANGELOG.md+docs/trust-model.{md,uk.md}updated.🤖 Generated with Claude Code
https://claude.ai/code/session_01X6t6tzVypHye5bDGLxzmZK
Generated by Claude Code