Skip to content

fix: complete gate to green -- fork ledger flush (#243), coverage, Linux Rust base, release gh login - #258

Merged
ebursztein merged 12 commits into
mainfrom
fix/243-fork-flushes-ledger
Sep 26, 2026
Merged

ebursztein merged 12 commits into
mainfrom
fix/243-fork-flushes-ledger

Conversation

@ebursztein

@ebursztein ebursztein commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Everything the complete local gate (just test on main 48400d5) turned up, fixed and re-proven on this branch.

Commit What just test hit Fix
fix(process) Fork of a running sandbox lost the last ≤5 s of ledger rows (#243) CloneState flushes the writer under the guest freeze before copying; a failed flush fails the fork. Test fails without the barrier.
fix(release) release-binaries died at channel-source after 8 min: GITHUB_TOKEN is required Source-manifest fetch falls back to the operator's gh auth token, the login the release already dispatches with
test(telemetry) capsem-telemetry 63% vs its 97% floor: OTLP export landed with only pure helpers tested End-to-end: a local collector receives a real OTLP POST to /v1/metrics with service, session and metric; install refuses a second exporter; exact UCUM codes. 98.6%
refactor(mcp-builtin) 45% vs 52.7% floor after #228 removed the tested snapshot tools Startup rules lifted out of main and tested; grep_http/http_headers get the refusal test fetch_http had. 59%
test(network) 95% vs 96.6% floor A connection reset is an error, not a clean EOF; Table::is_empty. 100%
chore(gate) Ratchet: 11 crates >3 pts above their floors Darwin floors raised to measured − 2.5
fix(build) ×2 warm-base: Linux Rust base image couldn't build (sdk/rust workspace member never copied), then the sealed lane couldn't run warcio tests (no Python env, --network none) Copy every workspace member (citadel guard added); bake build_system/.venv + uv cache; add build_system/{pyproject.toml,uv.lock} to the image identity
test(fs-monitor) Linux-only: restored-mtime forgery test raced Linux's coarse ctime clock Rewrite until the kernel records a new ctime (the test's actual precondition)
fix(service) assets.assemble: service never listened It exited at once: main.db was derived from the run dir's parent, so every /tmp run dir shared /tmp/sessions/main.db, and a v3-format copy stopped each new service. Now CAPSEM_HOME/sessions/main.db (foundation's rule); the path is the file the handle opened
fix(gate) …and the gate called that crash "did not accept a connection within 15s" WaitForSocket checks the launched pid each attempt and reports an exit at once, with its log dir
test Integration helper read main.db by the old rule Aligned with every other reader; home-layout guard text updated

Verification on this branch

Every complete-gate module, run on this branch: fast ok · static ok · artifacts ok · functional ok (29m) · glow-up ok (18m).

  • capsem-gate test-static: ok. Coverage ratchet plus the full Linux Rust lane (5,345 tests) in the sealed --network none container.
  • capsem-gate warm-linux-rust-base: ok.
  • just fast-test: ok; citadel 1253.
  • Crate suites: process 201, telemetry (export) 18, builtin 18, network 22, fs_monitor 21, admin 152.

After merge, just test <main sha> normal '' slow re-proves the whole plan on main in one run.

🤖 Generated with Claude Code

The fork of a running sandbox copies session.db while the writer still holds the last accepted rows in memory, up to its 5 s disk flush, so the fork lost the source's last seconds of events. The CloneState job now runs flush_checked under the guest freeze, then clones; a failed flush fails the fork. The new test writes a row, forks without waiting, and finds it in the clone; it failed before the barrier. Fixes #243.
fetch-channel-source-manifest required GITHUB_TOKEN in the environment. release-binaries already dispatches its hosted lane through gh, so the gh login is a release prerequisite anyway, and the missing export surfaced only at channel-source, after eight minutes of citadel, contract and build-system suites had passed. An exported GITHUB_TOKEN still wins; without one the operator's gh login supplies it, and only neither is an error.
@codecov-commenter

codecov-commenter commented Sep 26, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 41.79104% with 39 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.3%. Comparing base (48400d5) to head (e40f442).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...der/release/tools/fetch_channel_source_manifest.py 8.3% 11 Missing ⚠️
crates/capsem-process/src/vsock/clone_state.rs 35.2% 9 Missing and 2 partials ⚠️
crates/capsem-mcp-builtin/src/main.rs 62.5% 8 Missing and 1 partial ⚠️
build_system/builder/gate/service.py 0.0% 4 Missing ⚠️
crates/capsem-service/src/service_runtime.rs 0.0% 2 Missing ⚠️
build_system/builder/gate/pidfiles.py 50.0% 1 Missing ⚠️
crates/capsem-process/src/vsock.rs 0.0% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #258   +/-   ##
=======================================
  Coverage    65.2%    65.3%           
=======================================
  Files        1452     1452           
  Lines      127831   127864   +33     
  Branches    91782    91801   +19     
=======================================
+ Hits        83457    83544   +87     
+ Misses      39391    39328   -63     
- Partials     4983     4992    +9     
Flag Coverage Δ
integration 17.2% <ø> (-0.1%) ⬇️
linux-unit 70.5% <53.0%> (+<0.1%) ⬆️
mcp-server 93.8% <ø> (ø)
python-sdk 98.7% <ø> (ø)
typescript-sdk 97.8% <ø> (ø)
unit 63.4% <41.7%> (+<0.1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
TypeScript SDK 97.8% <ø> (ø)
Python SDK 98.7% <ø> (ø)
Network 84.4% <ø> (-0.1%) ⬇️
Security 81.6% <ø> (ø)
Tooling 88.6% <ø> (ø)
Monitoring 87.5% <ø> (+0.6%) ⬆️
Virtualization 64.4% <ø> (ø)
Confined Port Router 77.1% <ø> (ø)
Private Network 79.3% <ø> (+1.3%) ⬆️
Assets 80.8% <ø> (ø)
Gateway API 96.7% <ø> (ø)
Rust SDK 96.3% <ø> (ø)
Configuration 86.8% <ø> (ø)
Credentials 80.7% <ø> (ø)
Host Foundation 76.1% <ø> (-0.1%) ⬇️
Core Platform 55.8% <ø> (ø)
Runtime 60.6% <ø> (-0.3%) ⬇️
Daemon 41.6% <ø> (ø)
Service 71.5% <71.4%> (-0.1%) ⬇️
Process 48.5% <33.3%> (+<0.1%) ⬆️
Admin 63.7% <ø> (ø)
CLI 47.9% <ø> (ø)
MCP Server 93.8% <ø> (ø)
MCP Aggregator 61.8% <ø> (ø)
MCP Builtin 57.4% <62.5%> (+12.8%) ⬆️
Gateway 78.9% <ø> (ø)
TUI 68.5% <ø> (ø)
System Tray 53.2% <ø> (ø)
Guard 92.2% <ø> (ø)
UI 86.6% <ø> (ø)
Release Site 15.0% <ø> (ø)
Builder 43.5% <11.1%> (-0.1%) ⬇️
Mock Server 59.0% <ø> (ø)
Bench 47.8% <ø> (-0.1%) ⬇️
Files with missing lines Coverage Δ
crates/capsem-service/src/main.rs 79.5% <100.0%> (ø)
crates/capsem-service/src/vm_files.rs 76.5% <100.0%> (ø)
build_system/builder/gate/pidfiles.py 29.8% <50.0%> (ø)
crates/capsem-process/src/vsock.rs 18.8% <0.0%> (ø)
crates/capsem-service/src/service_runtime.rs 4.1% <0.0%> (-0.1%) ⬇️
build_system/builder/gate/service.py 57.7% <0.0%> (-2.3%) ⬇️
crates/capsem-mcp-builtin/src/main.rs 57.4% <62.5%> (+12.8%) ⬆️
...der/release/tools/fetch_channel_source_manifest.py 41.8% <8.3%> (-2.4%) ⬇️
crates/capsem-process/src/vsock/clone_state.rs 17.6% <35.2%> (+17.6%) ⬆️

... and 9 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The OTLP/HTTP export path (provider, HTTP client, install) landed with only its pure helpers tested, which dropped capsem-telemetry from its 97% coverage floor to 63%: nothing proved a metric ever left the process. A local collector now receives a real OTLP POST to /v1/metrics carrying the service, a session attribute and the recorded metric; install is refused a second time with RecorderAlreadySet; every facade unit maps to its exact UCUM code; uncataloged names, unit-bearing histograms and gauge increments reach the exporter. export.rs: 104/185 -> 182/185 lines.
Removing the snapshot tools in #228 took the builtin server's tested code with them and left main() -- peer lock naming, peer index parsing, active-profile loading -- as untestable process wiring, below the crate's coverage floor. Those rules are now functions with tests, and grep_http and http_headers get the refusal test fetch_http already had: a blocked URL comes back as an error result carrying its ledger record.
read_frame's non-EOF error branch and Table::is_empty had no test, which put capsem-network below its coverage floor. A connection reset must surface as an error rather than read as a clean end of stream.
…measure

The complete gate's per-crate ratchet requires a floor within 3 points of its measurement. Raised to measured minus 2.5, per the ratchet's own guidance, for capsem, agent, api, config, foundation, logger, mcp-builtin, network, process, router and service, in the Darwin table since they were measured on macOS.
The image runs cargo fetch --locked over a manifest-only copy that held crates/ alone; sdk/rust joined the workspace outside crates/, and the cached image hid it until the 0.6.4 lockfile change invalidated it and the complete gate failed at warm-base (failed to read /src/sdk/rust/Cargo.toml). A citadel guard now checks every workspace member is copied.
@ebursztein ebursztein changed the title fix: fork flushes the ledger first (#243); release reads the gh login fix: complete gate to green -- fork ledger flush (#243), coverage, Linux Rust base, release gh login Sep 26, 2026
The restored-mtime forgery test rewrote the file straight after its first write. Linux stamps inodes from a coarse clock that advances every few milliseconds, so both writes shared one ctime and nothing in the metadata could show the change; macOS stamps finer, so it only failed in the Linux coverage lane (reached for the first time once the Linux Rust base image built again). The test now rewrites until the kernel records a new ctime, the precondition its claim is about, and documents the granularity limit.
… environment

Two Rust tests hand a WARC export to warcio through uv run --project build_system. The Linux lane runs with --network none to prove the base image is complete, and the image never built that environment, so uv tried to download it and the tests failed on DNS (hidden until the base image rebuilt). The image now builds build_system/.venv and keeps uv's cache under /src, which the lane's copy leaves alone (.venv is dockerignored) and the ownership step hands to the lane user; build_system/pyproject.toml and uv.lock join the image identity so a Python dependency change rebuilds it.
The service derived main.db from its run directory's parent, while foundation's capsem_sessions_dir() says <home>/sessions. With the run directory outside the home -- the gate puts it under /tmp for a short socket path -- every such service shared /tmp/sessions/main.db, and a copy left in ledger format v3 made each new service exit at startup (archive_state format version 3 is unsupported), which the artifacts module reported as a socket that never accepted. The runtime now opens capsem_sessions_dir(); main_db_path() reports the file the handle opened, one source of truth; test states keep their own temporary home.
WaitForSocket spent its whole readiness budget on a daemon that had already exited with a clear error, then reported that it did not accept a connection -- a hang, where there was a crash. It now checks the pid the launch recorded on every attempt and fails immediately, naming the pid and where its log is.
The integration helper located main.db by the old run-root rule, which the service no longer follows; check_session.py, doctor_session_test.py and list_sessions.py already used CAPSEM_HOME/sessions/main.db. The service home-layout guard keeps its rule (harnesses use the installed layout) with its text updated: the shared-ledger hazard it described is fixed in the service.
@ebursztein
ebursztein merged commit 1aee630 into main Sep 26, 2026
13 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants