Skip to content

fix(daemon): SIGABRT on shutdown, SIGCHLD=SIG_IGN breaking every exec, in-process runCrate on HTTP workers, fd CLOEXEC, IPv6-route gateway, stack DNS never stopped, physmem 500 (1.1.26) - #229

Merged
click0 merged 1 commit into
mainfrom
claude/analyze-test-coverage-nCOJW
Sep 14, 2026

Conversation

@click0

@click0 click0 commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Summary

Seven correctness fixes from a correctness-lens audit (exception safety, process/fd hygiene, threading) — the "daemon works" batch. Three of them break crated in ordinary configurations today.

1. crated aborted with SIGABRT on every clean shutdown — daemon/server.cpp (HIGH)

The Unix-socket httplib::Server lived only inside its thread's lambda: Server::stop() could never stop it, listen() never returned, and ~Impl destroyed a still-joinable std::threadstd::terminate. With the default unixSocket configured this hit every service crated stop. The UDS server is now an Impl member; stop() stops it and joins its thread.

2. Enabling console.port broke every exec in the daemon — daemon/ws_console.cpp (HIGH)

WsConsole::start set signal(SIGCHLD, SIG_IGN) process-wide. On FreeBSD that flags PS_NOCLDWAIT: children are auto-reaped and waitpid() returns ECHILD for all of crated's children, so every Util::execCommand*/execPipeline* (privops verbs, stats, export/import, control-socket ops) threw "waitpid failed" even on success. Removed; session shells are reaped explicitly (SIGTERM → ~2s grace → SIGKILL + blocking wait).

3. POST …/start / …/restart ran the container in-process — daemon/routes.cpp (HIGH)

Both called runCrate() on the httplib worker thread: (a) the worker blocked for the jail's lifetime — after ~8 starts the whole API incl. /healthz stopped responding; (b) runCrate installed SIGINT/SIGTERM handlers process-wide → service crated stop hung; (c) jailXname/FwSlots/FwUsers are keyed on getpid() → all daemon-started jails shared one key (slot overwrite; shared NAT rule removed while other jails ran). Both routes now fork+setsid+exec crate run as its own process, exactly like the control-socket start path.
API note: the response is now async — {"started":true,"async":true,"pid":N,…} returns on spawn; poll GET /api/v1/containers/:name.

4. fds leaked into long-lived children — lib/util.cpp, lib/jail_query.cpp, three listeners (HIGH)

Capture pipes had no O_CLOEXEC, so a pipe's write end leaked into any child forked concurrently on another thread (control-socket crate run, ws-console shells), and the reading request thread hung on an EOF that only came when that child exited. Listening sockets leaked the same way (EADDRINUSE on restart). Now pipe2(O_CLOEXEC) + SOCK_CLOEXEC on the hand-rolled listeners. httplib's TCP listener and accept()ed fds are recorded in TODO as follow-up.

5. NAT-mode crate run failed on any host with an IPv6 default route — lib/run_net.cpp (MED)

detectGateway ran netstat -rn without -f inet; the inet6 default line doubled the token count and the strict != 4 check threw. Now -f inet and >= 4, mirroring the IPv6 query.

6. Stack DNS was never stopped — lib/stack.cpp (MED)

unbound ran with do-daemonize: yes, so the returned pid was the short-lived launcher (and the 200ms liveness check reported "exited immediately" on success); stack down passed -1 → orphan unbound stayed bound to :53. unbound now runs foreground under setsid; stopStackDns signals the pid from the per-network unbound.pid.

7. GET /api/v1/host was a permanent 500 on 64-bit — daemon/routes.cpp, lib/util.cpp (MED)

hw.physmem is CTLTYPE_ULONG (8 bytes) read through the 4-byte getSysctlIntENOMEM. New Util::getSysctlUInt64.

Testing

All seven are runtime-only (daemon / FreeBSD runtime — no pure unit surface). Compile-gated by the FreeBSD lite build. Exercising on a live host is recommended, especially the async start/restart semantics and clean shutdown.

Remaining (in TODO)

stack up still calls runCrate in-process; httplib/accept fd hygiene; run.cpp teardown-after-throw leaks; FwUsers dead-pid GC; listener read timeouts; a LOW batch.

Version

Bumps to 1.1.26; CHANGELOG.md + docs/trust-model.{md,uk.md} updated.

🤖 Generated with Claude Code

https://claude.ai/code/session_01X6t6tzVypHye5bDGLxzmZK


Generated by Claude Code

…, in-process runCrate on HTTP workers, fd CLOEXEC, IPv6-route gateway detect, stack DNS never stopped, physmem 500 (1.1.26)

Seven correctness fixes from a correctness-lens audit — "the daemon
works":

- server.cpp: the Unix-socket httplib::Server lived only in its thread's
  lambda; stop() never stopped it and ~Impl destroyed a joinable thread
  -> std::terminate on every clean `service crated stop`. Now an Impl
  member, stopped and joined.

- ws_console.cpp: signal(SIGCHLD, SIG_IGN) was process-wide; on FreeBSD
  that makes waitpid() return ECHILD for ALL children, so every
  Util::execCommand*/execPipeline threw "waitpid failed" the moment
  console.port was enabled. Removed; session shells are reaped
  explicitly (SIGTERM, grace, SIGKILL + blocking wait).

- routes.cpp: POST .../start and .../restart called runCrate() in-process
  on the httplib worker: blocked the worker for the jail's lifetime
  (~8 starts -> whole API incl. /healthz dead), installed SIGINT/SIGTERM
  handlers process-wide (`service crated stop` hung), and keyed
  jailXname/FwSlots/FwUsers on getpid() so all daemon-started jails
  shared one key (slot overwrite, NAT rule removed while jails ran).
  Now fork+setsid+exec `crate run` like the control-socket path; the
  response is async ({"started":true,"async":true,"pid":N}).

- util.cpp/jail_query.cpp/control_socket.cpp/privops_listener.cpp/
  ws_console.cpp: pipe2(O_CLOEXEC) for capture pipes and SOCK_CLOEXEC
  for the hand-rolled listeners, so long-lived forked children no longer
  inherit a pipe write end (request thread hung on EOF) or a LISTEN fd
  (EADDRINUSE on restart).

- run_net.cpp detectGateway: `netstat -rn -f inet` and `>= 4` tokens —
  an IPv6 default route doubled the `default` lines and every NAT-mode
  `crate run` failed.

- stack.cpp: unbound now foreground (do-daemonize: no) under setsid, and
  stopStackDns signals the pid from the per-network unbound.pid — DNS
  was never actually stopped before (orphan bound to :53).

- routes.cpp/util.{h,cpp}: hw.physmem (ULONG) read via new
  getSysctlUInt64; the 4-byte read made GET /api/v1/host a permanent 500.

All runtime-only, compile-gated by FreeBSD lite. Remaining audit items
(httplib/accept fd hygiene, run.cpp teardown-after-throw leaks, FwUsers
GC, listener timeouts, `stack up` in-process runCrate, LOW batch)
recorded in TODO. Bumps to 1.1.26; CHANGELOG + trust-model.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X6t6tzVypHye5bDGLxzmZK
@click0
click0 merged commit 9f29066 into main Sep 14, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants