Skip to content

silkd, sdk/python, ci: find on /proc, glibc headroom, keepalive, images base, guest speedups - #428

Merged
CMGS merged 16 commits into
mainfrom
fix/guest-ci-docs
Nov 3, 2026
Merged

CMGS merged 16 commits into
mainfrom
fix/guest-ci-docs

Conversation

@CMGS

@CMGS CMGS commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Fixes and speedups from a whole-repo read of silkd, the Python SDK, CI, packaging and docs. Stacked on #420.

Fixes

  • silkd fs_find hung forever on a zero-length file that never yields data (/proc/kmsg) or a FIFO, and fs_replace on a writerless FIFO did the same; the blocking thread leaked per retry. The file is opened with O_NONBLOCK, as fs_read already does: find skips it, replace answers an error frame.
  • silkd glibc headroom. The glibc build ran on trixie (glibc 2.41) and already needed GLIBC_2.39, Ubuntu 24.04's own version. It builds on bookworm now: highest needed symbol GLIBC_2.34 (checked on a local linux/arm64 build before and after).
  • Python relay had no TCP keepalive, so a partitioned node blocked a call without a timeout forever. The socket sets SO_KEEPALIVE and Go's default timings where the platform has them.
  • systemd unit gave up after a short PG outage at boot: sandboxd exits fast when the meta store refuses, and the default restart limit trips in about 2 s. RestartSec=5s in the unit and the docs copy.
  • images workflow could go green without building a change: a cancelled pending run's change was never diffed again (happened twice), and a force push diffed against a missing object and built nothing. The base is now the last successful run's commit; no base or a missing object builds everything.
  • release workflow refuses a tag that does not match silkd/Cargo.toml, as the PyPI workflow already does.
  • egress-e2e prints the missing-bridge message instead of exiting silently.
  • Docs corrected where they promised more than the code does: pipelining behind a read on a sleeping relay, request frames during fs_write/fs_push, the dirty marker and other mount namespaces, e2e scripts removing every sbx-* VM on the root, layer reproducibility with unpinned packages, goldens after a cocoon or Cloud Hypervisor upgrade.

Performance (A/B in one process, arms alternated, order swapped; silkd in a linux/arm64 container)

  • fs_find skips a file whose whole body has no match before the per-line scan (not for anchored patterns): 5 MB / 592 files, sparse pattern 4.55 -> 1.89 ms, dense err 25.6 -> 25.0 ms.
  • stdin frames use the hand-written decode path data frames use: 128 MiB of stdin 69.4 -> 47.5 ms.
  • fs_read reads the first chunk in the open hop: 4 KiB file 53.8 -> 23.8 us p50, 64 KiB 84.1 -> 43.6 us.
  • Python reads relay frames through a 64 KiB buffer instead of 8 KiB: 512 MB of 256 KiB frames 1351 -> 1251 ms (Linux).

Not changed

  • attach noticing a client hang-up: the protocol keeps streaming after a client half-close, which a hang-up cannot be told apart from. The residue is one guest task until the process writes or exits.

Tests

Each fix and speedup has a test that fails against the old code or a mutant. cargo fmt/clippy/test on macOS and in a linux/arm64 container; ruff, mypy, pytest; shellcheck.

Update

CMGS added 16 commits November 3, 2026 16:25
… a FIFO cannot hang the RPC

read_bounded opened with a blocking open: a zero-length /proc/kmsg blocks in read and a writerless FIFO blocks in open, so fs_find over /proc never answered and fs_replace on a FIFO hung. fs_read already opens with O_NONBLOCK for the same reason.
…does

A partitioned node left sb.exec() and every other data-plane call blocked forever; probes at 15 s idle, 15 s interval and 9 counts, where the platform has them, end the call once the peer is gone.
… boot does not hit the start limit

openBackends exits in 0.37 s when PostgreSQL refuses; systemd's 100 ms default restart reaches 5 starts in 10 s in about 2 s and leaves the unit failed.
…h headroom

The slim tag moved to trixie (glibc 2.41), and the shipped /silkd required GLIBC_2.39 (pidfd_spawnp, pidfd_getpid), Ubuntu 24.04's exact version. Built on bookworm the highest version it needs is GLIBC_2.34.
The concurrency queue cancels a pending push, and the next run diffed from that push, so its change was never built (237a5b8 and 55b0c1a, both saved only because the next push touched protocol/). A force push's before is not an object, and without pipefail the diff reported nothing and the run went green. An unknown or missing base now builds all, and build-os-images takes the same base.
publish-pypi already checks its tag against pyproject; a stale v* tag could otherwise ship old binaries under a new number.
…ing silently

Under pipefail the failing ip pipeline ended the script inside the command substitution, before the message.
…amespaces, e2e VM cleanup, layer reproducibility, goldens after an upgrade
…r-line scan

One regex pass over the body rejects a file without walking its lines; anchored patterns (^, $, \A, \z) match differently on a body than on a line, so they keep the per-line scan. A/B on a 5 MB, 592-file tree, linux/arm64, 41 interleaved rounds: sparse pattern p50 4.55 -> 1.89 ms, absent 4.79 -> 2.00 ms, dense 'err' 25.6 -> 25.0 ms.
Both SDKs render stdin as {"v":1,"op":"stdin","data":"<base64>"}, the fixture shape, so the data-frame decoder takes its head as a parameter. A/B, 128 MiB in 512 x 256 KiB frames, linux/arm64, 31 interleaved rounds: p50 69.4 -> 47.5 ms.
…costs one blocking-pool trip

The open, the read and the EOF read were three spawn_blocking hops; the open hop now reads up to BULK_CHUNK and the async stream runs only when that chunk came back full. A/B, linux/arm64, 4000 calls per arm interleaved: 4 KiB p50 53.8 -> 23.8 us, 64 KiB 84.1 -> 43.6 us.
…8 KiB default

One module constant for both makefile calls; 64 KiB, not more, bounds the memory each parked connection holds. A/B over TCP loopback, linux/arm64, 256K stdout frames (512 MiB payload), 8 interleaved runs: p50 1349 -> 1257 ms.
macOS names the idle option TCP_KEEPALIVE (Python 3.10+), not TCP_KEEPIDLE, so a Mac client started probing only after the 7200 s default.
…HUNK

Every read reserved 256 KiB, an mmap and munmap per call under musl. A /proc file reports 0 and grows inside read_to_end, still capped by take. A/B, linux/arm64, 8000 calls per arm interleaved: musl 100 B p50 21.1 -> 15.0 us (rerun 25.7 -> 20.4), 4 KiB 24.5 -> 17.5 us (rerun 28.2 -> 22.0); glibc unchanged within 0.3 us over three runs.
@CMGS
CMGS force-pushed the fix/guest-ci-docs branch from ef77fc3 to 70489ab Compare November 3, 2026 08:25
@CMGS
CMGS changed the base branch from review/post-417 to main November 3, 2026 08:25
@CMGS
CMGS merged commit 2f336dc into main Nov 3, 2026
4 checks passed
@CMGS
CMGS deleted the fix/guest-ci-docs branch November 3, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant