From e435c389544ef99750cdd68d629e293a53da8fb2 Mon Sep 17 00:00:00 2001 From: Nick Foster Date: Sat, 31 Oct 2026 14:00:59 -0500 Subject: [PATCH 1/4] docs: prepare installed recovery acceptance guidance --- docs/MANAGED-IMAGES.md | 7 +++++++ docs/UPDATING.md | 31 +++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/docs/MANAGED-IMAGES.md b/docs/MANAGED-IMAGES.md index 0a0c8f0..3f50cd0 100644 --- a/docs/MANAGED-IMAGES.md +++ b/docs/MANAGED-IMAGES.md @@ -136,6 +136,13 @@ to make a stale pending target disappear. ## Failure, removal and rollback +An installed portable image binding and an approved runtime do not update an +older last-known-good configuration automatically. The separately gated +[installed-baseline acceptance guidance](UPDATING.md#accepting-a-verified-installed-recovery-baseline) +preserves the old recovery evidence while verifying the installed policy. Follow +its final supporting source and delivery order before use; it authorizes no +image deletion, workload retirement or storage admission. + The selected runtime source and the executing manager can differ. Preserve trusted incumbent-compatible source before checkpoint capture, preflight, drain and restoration. A portable candidate cannot supply an older runtime's diff --git a/docs/UPDATING.md b/docs/UPDATING.md index 5430c31..19588fd 100644 --- a/docs/UPDATING.md +++ b/docs/UPDATING.md @@ -449,6 +449,37 @@ isolated machine for candidate code and verify actual-host admission separately through the authorized reviewed rollout. Live workers are never candidate-engine test machines. +## Accepting a verified installed recovery baseline + +Preparation only: [engine PR159](https://github.com/RandomDevelopment/ci-fleet/pull/159) +is not yet accepted. Replace this preparation notice with the final reviewed, +merged supporting source and immutable guide link before publication or live use. +The current ordinary-CI baseline and every desired runtime pin remain unchanged. + +A supported installation can already have the approved engine, capacity and +resources while its older last-known-good record still refers to an incompatible +configuration. Normal same-commit checks do not replace that recovery record. +Preserve its protected metadata and recovery evidence outside Git; do not edit it +to manufacture a successful transition or replay an oversized prior controller. + +After the supporting engine, actual template and private adopter changes pass +their required checks and reviews, an independently pinned supporting checkout +can run `scripts/remote-reconcile.sh --accept-installed`. This explicit operation +holds the installer lock, fetches only the locked installed commit, enforces +existing approval and installed checks, and requires fresh full health matching +the controller state before checked LKG persistence and readback. It stops on +drift without repairing configuration and never applies the older LKG on failure. +Authored managed-image policy and verified activation are required. Image-free +or staged baselines retain their separate protected-checkpoint recovery route. +Normal, check-only, installed-ref and no-op behavior remain separate. + +Keep the approved capacities, resources, workload protections and automation +holds through verification. Accept the actual persisted result before releasing +writers or testing a later timer cycle. This operation does not supply storage +admission, legacy retirement authority or proof of applying/scheduled reclamation. +Older pinned managers do not support the command. Follow the final supporting +engine guide rather than patching an installed checkout. + ## Engine manager upgrades and recovery outcomes An engine revision selected in `fleet.json`, an installed manager, and the From 3f8c02b644b184709f5bda4c8ce0f278e4fa938f Mon Sep 17 00:00:00 2001 From: Nick Foster Date: Sat, 31 Oct 2026 14:07:23 -0500 Subject: [PATCH 2/4] docs: clarify installed recovery acceptance checks --- docs/MANAGED-IMAGES.md | 6 +++--- docs/UPDATING.md | 17 ++++++++++++++--- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/docs/MANAGED-IMAGES.md b/docs/MANAGED-IMAGES.md index 3f50cd0..2cfaced 100644 --- a/docs/MANAGED-IMAGES.md +++ b/docs/MANAGED-IMAGES.md @@ -139,9 +139,9 @@ to make a stale pending target disappear. An installed portable image binding and an approved runtime do not update an older last-known-good configuration automatically. The separately gated [installed-baseline acceptance guidance](UPDATING.md#accepting-a-verified-installed-recovery-baseline) -preserves the old recovery evidence while verifying the installed policy. Follow -its final supporting source and delivery order before use; it authorizes no -image deletion, workload retirement or storage admission. +requires saving the prior recovery evidence outside Git before accepting the +installed policy. Follow its final supporting source and delivery order before +use; it authorizes no image deletion, workload retirement or storage admission. The selected runtime source and the executing manager can differ. Preserve trusted incumbent-compatible source before checkpoint capture, preflight, diff --git a/docs/UPDATING.md b/docs/UPDATING.md index 19588fd..28804a5 100644 --- a/docs/UPDATING.md +++ b/docs/UPDATING.md @@ -473,10 +473,21 @@ Authored managed-image policy and verified activation are required. Image-free or staged baselines retain their separate protected-checkpoint recovery route. Normal, check-only, installed-ref and no-op behavior remain separate. +The fresh health report and the health command's exit must agree. Active controllers require +matching `active` state and `healthy` or `warning` health. Drained and disabled +controllers require their matching state and `maintenance` or `warning` health. +An inconsistent report, process exit or state cannot establish acceptance. +The supporting command checkout is independent of the runtime and installed +manager; this verification does not require replacing either one. + Keep the approved capacities, resources, workload protections and automation -holds through verification. Accept the actual persisted result before releasing -writers or testing a later timer cycle. This operation does not supply storage -admission, legacy retirement authority or proof of applying/scheduled reclamation. +holds through verification. LKG replacement can precede a later readback or +terminal-state failure. After any failure, inspect the actual persisted LKG and +terminal reconciliation state; do not assume the old record is still installed. +Retain the saved prior recovery evidence and all holds until the actual result +is established, before releasing writers or testing a later timer cycle. +This operation does not supply storage admission, legacy retirement authority or +proof of applying/scheduled reclamation. Older pinned managers do not support the command. Follow the final supporting engine guide rather than patching an installed checkout. From 46d4db89b98389911ef5076f7f546c1348d8f90d Mon Sep 17 00:00:00 2001 From: Nick Foster Date: Sat, 31 Oct 2026 16:45:22 -0500 Subject: [PATCH 3/4] docs: pin merged installed recovery helper --- docs/UPDATING.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/UPDATING.md b/docs/UPDATING.md index 28804a5..cc08959 100644 --- a/docs/UPDATING.md +++ b/docs/UPDATING.md @@ -451,10 +451,11 @@ test machines. ## Accepting a verified installed recovery baseline -Preparation only: [engine PR159](https://github.com/RandomDevelopment/ci-fleet/pull/159) -is not yet accepted. Replace this preparation notice with the final reviewed, -merged supporting source and immutable guide link before publication or live use. -The current ordinary-CI baseline and every desired runtime pin remain unchanged. +The supporting reconciliation source is merged ci-fleet +`e61831c72787df57e76db3030ab7bb0b0986d369`. Follow its immutable +[installed-baseline acceptance guide](https://github.com/RandomDevelopment/ci-fleet/blob/e61831c72787df57e76db3030ab7bb0b0986d369/docs/EXPLICIT-DRAINED-RECOVERY.md#accept-an-already-installed-recovery-baseline). +This is a separate supporting executable pin. The ordinary-CI compatibility +baseline, desired runtime pins and independent executor/deployer pins stay unchanged. A supported installation can already have the approved engine, capacity and resources while its older last-known-good record still refers to an incompatible @@ -473,9 +474,10 @@ Authored managed-image policy and verified activation are required. Image-free or staged baselines retain their separate protected-checkpoint recovery route. Normal, check-only, installed-ref and no-op behavior remain separate. -The fresh health report and the health command's exit must agree. Active controllers require -matching `active` state and `healthy` or `warning` health. Drained and disabled -controllers require their matching state and `maintenance` or `warning` health. +The fresh health report and the health command's exit must agree. Active +controllers require matching `active` state and `healthy` or `warning` health. +Drained and disabled controllers require their matching state and `maintenance` +or `warning` health. An inconsistent report, process exit or state cannot establish acceptance. The supporting command checkout is independent of the runtime and installed manager; this verification does not require replacing either one. @@ -488,7 +490,7 @@ Retain the saved prior recovery evidence and all holds until the actual result is established, before releasing writers or testing a later timer cycle. This operation does not supply storage admission, legacy retirement authority or proof of applying/scheduled reclamation. -Older pinned managers do not support the command. Follow the final supporting +Older pinned managers do not support the command. Follow the merged supporting engine guide rather than patching an installed checkout. ## Engine manager upgrades and recovery outcomes From 5f53e2755afad161af0721ea6aff3c206b97e221 Mon Sep 17 00:00:00 2001 From: Nick Foster Date: Sat, 31 Oct 2026 16:50:18 -0500 Subject: [PATCH 4/4] docs: identify the external recovery helper path --- docs/UPDATING.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/UPDATING.md b/docs/UPDATING.md index cc08959..d694651 100644 --- a/docs/UPDATING.md +++ b/docs/UPDATING.md @@ -464,8 +464,14 @@ Preserve its protected metadata and recovery evidence outside Git; do not edit i to manufacture a successful transition or replay an oversized prior controller. After the supporting engine, actual template and private adopter changes pass -their required checks and reviews, an independently pinned supporting checkout -can run `scripts/remote-reconcile.sh --accept-installed`. This explicit operation +their required checks and reviews, run the helper from the independently pinned +supporting checkout: + +```bash +sudo /path/to/reviewed-ci-fleet/scripts/remote-reconcile.sh --accept-installed +``` + +This explicit operation holds the installer lock, fetches only the locked installed commit, enforces existing approval and installed checks, and requires fresh full health matching the controller state before checked LKG persistence and readback. It stops on