Repository navigation
silkd, sdk/python, ci: find on /proc, glibc headroom, keepalive, images base, guest speedups - #428
Merged
Merged
Conversation
This was referenced Nov 3, 2026
… a FIFO cannot hang the RPC read_bounded opened with a blocking open: a zero-length /proc/kmsg blocks in read and a writerless FIFO blocks in open, so fs_find over /proc never answered and fs_replace on a FIFO hung. fs_read already opens with O_NONBLOCK for the same reason.
…does A partitioned node left sb.exec() and every other data-plane call blocked forever; probes at 15 s idle, 15 s interval and 9 counts, where the platform has them, end the call once the peer is gone.
… boot does not hit the start limit openBackends exits in 0.37 s when PostgreSQL refuses; systemd's 100 ms default restart reaches 5 starts in 10 s in about 2 s and leaves the unit failed.
…h headroom The slim tag moved to trixie (glibc 2.41), and the shipped /silkd required GLIBC_2.39 (pidfd_spawnp, pidfd_getpid), Ubuntu 24.04's exact version. Built on bookworm the highest version it needs is GLIBC_2.34.
The concurrency queue cancels a pending push, and the next run diffed from that push, so its change was never built (237a5b8 and 55b0c1a, both saved only because the next push touched protocol/). A force push's before is not an object, and without pipefail the diff reported nothing and the run went green. An unknown or missing base now builds all, and build-os-images takes the same base.
publish-pypi already checks its tag against pyproject; a stale v* tag could otherwise ship old binaries under a new number.
…ing silently Under pipefail the failing ip pipeline ended the script inside the command substitution, before the message.
…amespaces, e2e VM cleanup, layer reproducibility, goldens after an upgrade
…r-line scan One regex pass over the body rejects a file without walking its lines; anchored patterns (^, $, \A, \z) match differently on a body than on a line, so they keep the per-line scan. A/B on a 5 MB, 592-file tree, linux/arm64, 41 interleaved rounds: sparse pattern p50 4.55 -> 1.89 ms, absent 4.79 -> 2.00 ms, dense 'err' 25.6 -> 25.0 ms.
Both SDKs render stdin as {"v":1,"op":"stdin","data":"<base64>"}, the fixture shape, so the data-frame decoder takes its head as a parameter. A/B, 128 MiB in 512 x 256 KiB frames, linux/arm64, 31 interleaved rounds: p50 69.4 -> 47.5 ms.
…costs one blocking-pool trip The open, the read and the EOF read were three spawn_blocking hops; the open hop now reads up to BULK_CHUNK and the async stream runs only when that chunk came back full. A/B, linux/arm64, 4000 calls per arm interleaved: 4 KiB p50 53.8 -> 23.8 us, 64 KiB 84.1 -> 43.6 us.
…8 KiB default One module constant for both makefile calls; 64 KiB, not more, bounds the memory each parked connection holds. A/B over TCP loopback, linux/arm64, 256K stdout frames (512 MiB payload), 8 interleaved runs: p50 1349 -> 1257 ms.
macOS names the idle option TCP_KEEPALIVE (Python 3.10+), not TCP_KEEPIDLE, so a Mac client started probing only after the 7200 s default.
…HUNK Every read reserved 256 KiB, an mmap and munmap per call under musl. A /proc file reports 0 and grows inside read_to_end, still capped by take. A/B, linux/arm64, 8000 calls per arm interleaved: musl 100 B p50 21.1 -> 15.0 us (rerun 25.7 -> 20.4), 4 KiB 24.5 -> 17.5 us (rerun 28.2 -> 22.0); glibc unchanged within 0.3 us over three runs.
CMGS
force-pushed
the
fix/guest-ci-docs
branch
from
November 3, 2026 08:25
ef77fc3 to
70489ab
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes and speedups from a whole-repo read of silkd, the Python SDK, CI, packaging and docs. Stacked on #420.
Fixes
fs_findhung forever on a zero-length file that never yields data (/proc/kmsg) or a FIFO, andfs_replaceon a writerless FIFO did the same; the blocking thread leaked per retry. The file is opened withO_NONBLOCK, asfs_readalready does: find skips it, replace answers an error frame.GLIBC_2.39, Ubuntu 24.04's own version. It builds on bookworm now: highest needed symbolGLIBC_2.34(checked on a local linux/arm64 build before and after).SO_KEEPALIVEand Go's default timings where the platform has them.sandboxdexits fast when the meta store refuses, and the default restart limit trips in about 2 s.RestartSec=5sin the unit and the docs copy.silkd/Cargo.toml, as the PyPI workflow already does.fs_write/fs_push, the dirty marker and other mount namespaces, e2e scripts removing everysbx-*VM on the root, layer reproducibility with unpinned packages, goldens after a cocoon or Cloud Hypervisor upgrade.Performance (A/B in one process, arms alternated, order swapped; silkd in a linux/arm64 container)
fs_findskips a file whose whole body has no match before the per-line scan (not for anchored patterns): 5 MB / 592 files, sparse pattern 4.55 -> 1.89 ms, denseerr25.6 -> 25.0 ms.fs_readreads the first chunk in the open hop: 4 KiB file 53.8 -> 23.8 us p50, 64 KiB 84.1 -> 43.6 us.Not changed
attachnoticing a client hang-up: the protocol keeps streaming after a client half-close, which a hang-up cannot be told apart from. The residue is one guest task until the process writes or exits.Tests
Each fix and speedup has a test that fails against the old code or a mutant. cargo fmt/clippy/test on macOS and in a linux/arm64 container; ruff, mypy, pytest; shellcheck.
Update
TCP_KEEPALIVE.fs_readsizes its first chunk by the file's length: musl 100 B 21.1 -> 15.0 us, glibc unchanged.docs/deploy.mddescribes pool, store: dead warm VMs, shared staging, cache reclaim, volume unmount budget #427, so merge that first.