Skip to content

docs: explain verified installed recovery acceptance #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Nickfost merged 4 commits into from
Oct 31, 2026
Merged

docs: explain verified installed recovery acceptance #46

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e435c38
docs: prepare installed recovery acceptance guidance
Nickfost Oct 31, 2026
3f8c02b
docs: clarify installed recovery acceptance checks
Nickfost Oct 31, 2026
46d4db8
docs: pin merged installed recovery helper
Nickfost Oct 31, 2026
5f53e27
docs: identify the external recovery helper path
Nickfost Oct 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/MANAGED-IMAGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,13 @@ to make a stale pending target disappear.

## Failure, removal and rollback

An installed portable image binding and an approved runtime do not update an
older last-known-good configuration automatically. The separately gated
[installed-baseline acceptance guidance](UPDATING.md#accepting-a-verified-installed-recovery-baseline)
requires saving the prior recovery evidence outside Git before accepting the
installed policy. Follow its final supporting source and delivery order before
use; it authorizes no image deletion, workload retirement or storage admission.

The selected runtime source and the executing manager can differ. Preserve
trusted incumbent-compatible source before checkpoint capture, preflight,
drain and restoration. A portable candidate cannot supply an older runtime's
Expand Down
50 changes: 50 additions & 0 deletions docs/UPDATING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,56 @@ isolated machine for candidate code and verify actual-host admission separately
through the authorized reviewed rollout. Live workers are never candidate-engine
test machines.

## Accepting a verified installed recovery baseline

The supporting reconciliation source is merged ci-fleet
`e61831c72787df57e76db3030ab7bb0b0986d369`. Follow its immutable
[installed-baseline acceptance guide](https://github.com/RandomDevelopment/ci-fleet/blob/e61831c72787df57e76db3030ab7bb0b0986d369/docs/EXPLICIT-DRAINED-RECOVERY.md#accept-an-already-installed-recovery-baseline).
This is a separate supporting executable pin. The ordinary-CI compatibility
baseline, desired runtime pins and independent executor/deployer pins stay unchanged.

A supported installation can already have the approved engine, capacity and
resources while its older last-known-good record still refers to an incompatible
configuration. Normal same-commit checks do not replace that recovery record.
Preserve its protected metadata and recovery evidence outside Git; do not edit it
to manufacture a successful transition or replay an oversized prior controller.

After the supporting engine, actual template and private adopter changes pass
their required checks and reviews, run the helper from the independently pinned
supporting checkout:

```bash
sudo /path/to/reviewed-ci-fleet/scripts/remote-reconcile.sh --accept-installed
```

This explicit operation
holds the installer lock, fetches only the locked installed commit, enforces
existing approval and installed checks, and requires fresh full health matching
the controller state before checked LKG persistence and readback. It stops on
drift without repairing configuration and never applies the older LKG on failure.
Authored managed-image policy and verified activation are required. Image-free
or staged baselines retain their separate protected-checkpoint recovery route.
Normal, check-only, installed-ref and no-op behavior remain separate.

The fresh health report and the health command's exit must agree. Active
controllers require matching `active` state and `healthy` or `warning` health.
Drained and disabled controllers require their matching state and `maintenance`
or `warning` health.
An inconsistent report, process exit or state cannot establish acceptance.
The supporting command checkout is independent of the runtime and installed
manager; this verification does not require replacing either one.

Keep the approved capacities, resources, workload protections and automation
holds through verification. LKG replacement can precede a later readback or
terminal-state failure. After any failure, inspect the actual persisted LKG and
terminal reconciliation state; do not assume the old record is still installed.
Retain the saved prior recovery evidence and all holds until the actual result
is established, before releasing writers or testing a later timer cycle.
This operation does not supply storage admission, legacy retirement authority or
proof of applying/scheduled reclamation.
Older pinned managers do not support the command. Follow the merged supporting
engine guide rather than patching an installed checkout.

## Engine manager upgrades and recovery outcomes

An engine revision selected in `fleet.json`, an installed manager, and the
Expand Down