From 4f7bb8e90aed4e0ec2c384f75d0479558f1f2e36 Mon Sep 17 00:00:00 2001 From: Nick Foster Date: Tue, 3 Nov 2026 13:06:24 -0600 Subject: [PATCH 1/3] docs: explain proxy upload memory recovery gates --- AGENTS.md | 8 ++++++++ README.md | 6 ++++++ docs/UPDATING.md | 52 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 66 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index af4591c..e69431b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -176,3 +176,11 @@ through a reviewed ownership map. Never replace the private suite with upstream local_validate.py or policy tests. Repository suites precede installation rehearsal because installed fleet-role markers block validation. A modest rehearsal cannot substitute for actual target capacity and resource admission. + +For socket-proxy upload memory recovery, follow +[the ordered guidance](docs/UPDATING.md#socket-proxy-upload-memory-and-recovery). +Keep spools disk-backed and preserve the 128 MiB memory limit, 1 GiB body cap +and two upload slots. Accept exact source and complete immutable image evidence +separately from authorized host adoption and independent live availability. +Guidance imports must preserve runtime/image pins and uncertainty guards; +source acceptance cannot release a held application retry. diff --git a/README.md b/README.md index 42432fc..33d711a 100644 --- a/README.md +++ b/README.md @@ -359,6 +359,12 @@ follow [manager adoption and recovery outcomes](docs/UPDATING.md#engine-manager- An unchanged applied SHA or a successful systemd unit does not prove health or successful reconciliation. +For Docker build or image-load upload memory incidents, follow the +[socket-proxy recovery sequence](docs/UPDATING.md#socket-proxy-upload-memory-and-recovery). +It requires disk-backed spools with the unchanged 128 MiB memory limit, +1 GiB body cap and two upload slots. Accepted source, complete immutable image +artifacts and independent live availability have separate acceptance gates. + ### Optional host-enforced configuration approval A compatible installed engine advertising `configuration_approval_gate` can diff --git a/docs/UPDATING.md b/docs/UPDATING.md index d694651..a1c7bb7 100644 --- a/docs/UPDATING.md +++ b/docs/UPDATING.md @@ -572,6 +572,58 @@ and retry behavior remain unchanged. These observations authorize no cleanup or gate bypass. Engine merge, template publication, private engine promotion and separately authorized host acceptance are distinct milestones. +### Socket-proxy upload memory and recovery + +Merged engine `1c685dacc109597641fd13dfae48606e76f73ee7` adds bounded +file-cache handling for Docker build and image-load uploads. Its reviewed +[host-maintenance contract](https://github.com/RandomDevelopment/ci-fleet/blob/1c685dacc109597641fd13dfae48606e76f73ee7/docs/HOST-MAINTENANCE.md#docker-storage-retention) +requires disk-backed temporary spools. The proxy synchronizes writes and +discards cached pages in 4 MiB windows while receiving uploads, then discards +consumed pages while forwarding them. Unlinking a spool alone does not release +its memory-cgroup file-cache charges. Memory-backed `tmpfs` and `ramfs` spools +are refused before forwarding. Keep the 128 MiB proxy memory limit, 1 GiB +per-body cap and two shared upload slots unchanged. + +This guidance does not select that engine or change the compatibility baseline. +Apply the following order before treating an upload-memory incident as recovered: + +1. Accept the exact engine source and its isolated regression evidence, including + actual cgroup memory and file-cache measurements during receiving and + forwarding. Keep earlier failed results under their original identities. +2. Accept the actual template guidance, then import its exact reviewed immutable + revision through the [unpublished-revision procedure](#import-an-unpublished-reviewed-revision). + Preserve adopter-owned validation, configuration, image pins and + `TEMPLATE_RELEASE`. Complete each repository's required suites and independent + review. These guidance changes do not stage a new runtime. +3. Separately prepare and review complete controller and runner artifacts for + the accepted merged engine and target architecture. Follow the + [managed-image procedure](MANAGED-IMAGES.md), compare original config digests + across clean repetitions, and retain complete archives, hashes and original + descriptor evidence outside Git. Temporary CI tags, an older source's proof + or one completed role cannot establish a complete target image pair. Keep + config digest C distinct from store runtime ID R and manifest bindings. +4. Only through a separately authorized adoption, stage target evidence and + image policy against the actual predecessor, then use the supported ordered + manager/runtime installation and recovery procedure. Verify current disk + admission and workload protections. Retain compatible incumbent source, + exact images and checkpoints. Preserve installer/proxy locks and uncertainty + guards. Never copy a live binary into place, bypass the proxy or clear + current-boot markers to force an upgrade. +5. Verify the actual installed source and image identities against that packet. + Independently establish sustained Docker daemon, proxy and controller + availability before releasing held application retries. A source proof, + successful build, running container, automatic proxy restart or one healthy + observation does not establish live recovery. + +A failed or incomplete artifact build leaves the image gate open even when +source validation passed. Preserve its evidence and diagnose the failed +assertion before proposing a changed-source correction or another build. +An upload EOF or unavailable Docker endpoint alone does not distinguish proxy +memory-cgroup OOM, daemon failure or another transport fault. Retain scoped +incident evidence rather than inferring a cause from a timeout exit code. +Storage reclamation, capacity changes, workload retirement and application +retry authority remain separate decisions. + ## Optional Docker network and storage policy The reviewed ordinary-CI compatibility baseline is From b1ca13d07979330ca609741e53a699921fcff6f5 Mon Sep 17 00:00:00 2001 From: Nick Foster Date: Thu, 5 Nov 2026 03:21:54 -0600 Subject: [PATCH 2/3] fix: omit optional Git templates from validation seeds --- docs/LOCAL-VALIDATION.md | 5 ++++- scripts/local_validate.py | 2 +- scripts/test_local_validation.py | 26 ++++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 2 deletions(-) diff --git a/docs/LOCAL-VALIDATION.md b/docs/LOCAL-VALIDATION.md index 0c7d446..06f911f 100644 --- a/docs/LOCAL-VALIDATION.md +++ b/docs/LOCAL-VALIDATION.md @@ -278,7 +278,10 @@ of these seeds in memory, including digests of file bytes, entry types and modes symlink targets and `.git` metadata. Git status or index checks alone cannot detect all changes, such as assume-unchanged files or edits to `.git/config`. -Initial seeds start as empty Git repositories. Source fetches use +Initial seeds start as empty Git repositories with `git init --template=`. +Optional initialization templates, such as sample hooks, are not copied into +the seeds or their stage copies. Committed files and all generated Git metadata +remain covered by the same manifests and tamper checks. Source fetches use `--no-tags` and explicit source, original reviewed head, main and nonzero base SHAs with their complete ancestry. Engine-policy and trusted-suite fetches include only their selected commit and its ancestry. Checkouts inherit no diff --git a/scripts/local_validate.py b/scripts/local_validate.py index 6ee4a87..9764ddf 100644 --- a/scripts/local_validate.py +++ b/scripts/local_validate.py @@ -72,7 +72,7 @@ def clone(repository: Path, revision: str, target: Path, identities: list[str]) if any(not SHA.fullmatch(value) or value == NO_PRIOR_STATE for value in identities): raise ValueError("checkout identities must be full nonzero commit SHAs") target.mkdir() - subprocess.run(["git", "init", "--quiet", str(target)], env=git_environment(), check=True) + subprocess.run(["git", "init", "--quiet", "--template=", str(target)], env=git_environment(), check=True) subprocess.run(["git", "-C", str(target), "config", "remote.origin.url", str(repository)], env=git_environment(), check=True) subprocess.run(["git", "-C", str(target), "fetch", "--quiet", "--no-tags", "--keep", "origin", *identities], env=git_environment(), check=True) subprocess.run(["git", "-C", str(target), "checkout", "--quiet", "--detach", revision], env=git_environment(), check=True) diff --git a/scripts/test_local_validation.py b/scripts/test_local_validation.py index b5bcbab..887040e 100644 --- a/scripts/test_local_validation.py +++ b/scripts/test_local_validation.py @@ -508,6 +508,32 @@ def test_git_wrapper_and_clone_ignore_inherited_repository_redirection(self) -> self.assertEqual(local.git(decoy, "rev-parse", "HEAD"), self.base) self.assertEqual((decoy / ".git/config").read_bytes(), config) + def test_clone_omits_optional_git_init_templates(self) -> None: + template = self.root / "git-template" + (template / "hooks").mkdir(parents=True) + (template / "hooks/pre-commit.sample").write_text("unused sample hook\n") + (template / "optional-metadata").write_text("not source content\n") + target = self.root / "without-templates" + original_run = subprocess.run + + def configured_init(command, **kwargs): + if command[:2] == ["git", "init"]: + command = ["git", "-c", "init.templateDir=" + str(template), *command[1:]] + return original_run(command, **kwargs) + + with patch.object(local.subprocess, "run", side_effect=configured_init): + local.clone(self.repository, self.source, target, [self.source, self.base]) + self.assertFalse((target / ".git/hooks").exists()) + self.assertFalse((target / ".git/optional-metadata").exists()) + self.assertEqual(local.git(target, "rev-parse", "HEAD"), self.source) + self.assertEqual(local.git(target, "rev-parse", self.base + "^{commit}"), self.base) + self.assertEqual((target / "marker").read_text(), "candidate\n") + manifest = local.checkout_manifest(target) + self.assertIn(".git/config", manifest) + (target / ".git/config").write_text("changed metadata\n") + with self.assertRaisesRegex(RuntimeError, "checkout changed"): + local.verify_checkout(target, manifest) + def test_stage_commands_receive_only_the_protected_git_environment(self) -> None: def inspect(command, env, log): self.assertEqual({key: value for key, value in env.items() if key.startswith("GIT_")}, GIT_PROTECTION) From 9ce1e35d6bac8d7a7b5f4af9a49f39c91125219b Mon Sep 17 00:00:00 2001 From: Nick Foster Date: Fri, 6 Nov 2026 21:44:38 -0600 Subject: [PATCH 3/3] fix: pin merged receipt executor --- .github/workflows/validate.yml | 2 +- docs/LOCAL-VALIDATION.md | 14 +++++++++----- scripts/local_validate.py | 2 +- 3 files changed, 11 insertions(+), 7 deletions(-) diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index 9a2342d..90fada2 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -58,7 +58,7 @@ jobs: uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 with: repository: RandomDevelopment/ci-fleet - ref: 689c7c00deb0eebcb25f2284334a9f7168a25575 + ref: c51844cf05f7a9f647322f4b7cbbb6379f40ad5d path: .ci-fleet-executor persist-credentials: false fetch-depth: 1 diff --git a/docs/LOCAL-VALIDATION.md b/docs/LOCAL-VALIDATION.md index 06f911f..8097770 100644 --- a/docs/LOCAL-VALIDATION.md +++ b/docs/LOCAL-VALIDATION.md @@ -30,9 +30,13 @@ protected workspace parent. It adds no daemon, timer or unique fleet state. Use a Linux VM or remote prepared worker from macOS or Windows. Shared executor support merged in core [PR #144](https://github.com/RandomDevelopment/ci-fleet/pull/144) at `689c7c00deb0eebcb25f2284334a9f7168a25575`. The ordinary-CI configuration-policy baseline is `840fb9be8a1c69404a5eebcf17855eff13c6c928`. -The executor remains independently pinned to the reviewed -`689c7c00deb0eebcb25f2284334a9f7168a25575` implementation. Engine merge, -template acceptance and host activation remain separate. +The executor is independently pinned to merged core +[PR #165](https://github.com/RandomDevelopment/ci-fleet/pull/165) at +`c51844cf05f7a9f647322f4b7cbbb6379f40ad5d`. Its completion hook avoids site +initialization and unnecessary imports while preserving strict receipt checks, +deadlines and cgroup cleanup. The optional workflow checks out this same executor; +adopters must select a reviewed suite pin requiring that revision. +Engine merge, template acceptance and host activation remain separate. Executor and policy pins are independent inputs. Pinning an executor alone does not adopt the managed-image contract or activate a runtime. @@ -446,8 +450,8 @@ The core local-evidence contract merged through passed all eight stages outside Actions on reviewed source and policy `a8552da5c902c54c0d78b19e6f093e2e96482f62`, with `expected_skips: []`. That engine result does not replace this complete standalone suite. -The development ordinary-CI compatibility baseline is -`689c7c00deb0eebcb25f2284334a9f7168a25575`. The historical Git/output helpers retain their reviewed implementation provenance at +The standalone executor pin is +`c51844cf05f7a9f647322f4b7cbbb6379f40ad5d`. The historical Git/output helpers retain their reviewed implementation provenance at `96f277b3a75f3d2bef3c04ceb7b3c8778b8998c2`. Core availability and functional suite results establish no template release, private engine promotion or signed operational acceptance. diff --git a/scripts/local_validate.py b/scripts/local_validate.py index 9764ddf..9a5b44e 100644 --- a/scripts/local_validate.py +++ b/scripts/local_validate.py @@ -20,7 +20,7 @@ ROOT = Path(__file__).resolve().parents[1] EXECUTOR_PATH = Path("/usr/local/lib/ci-fleet-validation/executor.py") # Shared execution tooling is independent of the configuration-policy baseline. -EXECUTOR_REVISION = "689c7c00deb0eebcb25f2284334a9f7168a25575" +EXECUTOR_REVISION = "c51844cf05f7a9f647322f4b7cbbb6379f40ad5d" SHA = re.compile(r"[0-9a-f]{40}") NO_PRIOR_STATE = "0" * 40 SUITE = "ci-fleet-config-standalone-v2"