diff --git a/AGENTS.md b/AGENTS.md index 3f0bae2..af4591c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -111,17 +111,24 @@ execution machine's role separately, and keep the signing authority elsewhere. ## Current configuration contract -The development compatibility baseline is ci-fleet -`689c7c00deb0eebcb25f2284334a9f7168a25575`. Fresh strict configurations require +The ordinary-CI compatibility baseline is merged ci-fleet +`840fb9be8a1c69404a5eebcf17855eff13c6c928`. It includes PR157 maintenance +and unused-pending support. Template acceptance, adopter import and host +activation remain separate gates. Fresh strict configurations require `managed_images` with exact nonzero controller and runner image IDs for each selected engine and architecture. Initializer callers must supply architecture -and both image IDs. Public fixtures use placeholders and non-strict validation. +and both image scalars. The selected runtime source capability chooses original +config digests for portable engines and exact opaque Docker IDs for older +engines. Derived runtime IDs and host binding proof remain outside Git. Public +fixtures use placeholders and non-strict validation. Keep manager-only staging limited to engine-ref changes against a complete previous configuration. Preserve incumbent runtime evidence through staging and image-policy introduction. Do not weaken next-engine, network or storage gates. -Both bootstrap revisions require prior integrated target-engine sidecar evidence -for configured reporting/network capabilities. Stage it with the incumbent -compatible contract before importing the new manager-staging contract. +Both bootstrap revisions and ordinary image-enforced upgrades require prior +integrated target-engine evidence, including controllers without optional blocks. +Prepare explicit incumbent evidence first, retain active/pending records through +bootstrap and promote only after runtime verification. Any unused-pending +cancellation route needs its reviewed supporting engine; never fabricate prior state. Managed-image policy changes need explicit updater/manual migration handling even when fleet schema remains v3. Keep cache endpoints and bypass hosts in protected host-local configuration. See [managed images](docs/MANAGED-IMAGES.md) @@ -156,3 +163,16 @@ and [APT cache](docs/APT-CACHE.md). Development support is not publication of - When reviewing, report missing or inaccurate documentation as a finding. - Before declaring the work complete, check updated examples, commands, links, and configuration names. Run applicable documentation checks. - When creating or updating a PR, state the documentation impact in its description. In all cases, include the assessment in the completion summary. List updated documents, or explain why no documentation changes are needed. Report any remaining gaps. + +## Required delivery order + +Complete engine acceptance first, then implement, validate and review the actual +template specification, import that exact reviewed revision into the adopter, +then perform authorized deployment. Follow-up issues alone do not complete the +template or adopter steps. Candidate engines run only on isolated test machines. +Preserve customized adopter validation, private test inventories, required workflows, +policy, evidence, delivery configuration, historical compatibility and TEMPLATE_RELEASE +through a reviewed ownership map. Never replace the private suite with upstream +local_validate.py or policy tests. Repository suites precede installation rehearsal +because installed fleet-role markers block validation. A modest rehearsal cannot +substitute for actual target capacity and resource admission. diff --git a/README.md b/README.md index 177b8e8..42432fc 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ flowchart LR ## Start a private organization configuration 1. Create a **private** repository from this public template. -2. Review the engine revision and obtain exact controller and runner image IDs +2. Review the engine revision and obtain controller and runner image scalars for each target architecture through the [managed-image procedure](docs/MANAGED-IMAGES.md). Build verification needs a separately authorized isolated build host. Clone the configuration repository and initialize its first project and controller: @@ -38,8 +38,8 @@ flowchart LR --max-runners 1 \ --engine-ref REVIEWED_ENGINE_COMMIT_SHA \ --architecture amd64 \ - --controller-image-digest sha256:REVIEWED_CONTROLLER_IMAGE_ID \ - --runner-image-digest sha256:REVIEWED_RUNNER_IMAGE_ID + --controller-image-digest sha256:REVIEWED_CONTROLLER_SCALAR \ + --runner-image-digest sha256:REVIEWED_RUNNER_SCALAR ``` 3. Edit `fleet.json` to add the organization's real logical mappings. @@ -147,7 +147,7 @@ Core support for trusted local evidence merged through `67beabbddaad73c19f15bc3baa227779bc88dc96`. Its [complete engine validation](https://github.com/RandomDevelopment/ci-fleet/issues/135#issuecomment-6200457374) passed outside Actions. The development compatibility baseline is now -`689c7c00deb0eebcb25f2284334a9f7168a25575`. Trusted host acceptance still needs +`840fb9be8a1c69404a5eebcf17855eff13c6c928`. Trusted host acceptance still needs an independent signing authority, compatible managers and tested recovery. Earlier standalone tooling is published in [v1.1.0](docs/RELEASE.md#published-v110-release). The current systemd suite and managed-image support are unreleased. @@ -155,7 +155,7 @@ Publication does not activate host policy. ## Reviewed compatibility and release state -`template-compatibility.json` pins the reviewed embedded template to ci-fleet commit `689c7c00deb0eebcb25f2284334a9f7168a25575`. It records schema and validator/initializer support, optional capabilities, standalone safety checks, and fictional examples pinned to that same engine with placeholder image IDs. `engine-rollout-evidence.json` records active per-controller capability evidence in the shape consumed by that pinned core. New status-reporting, Docker network-policy, and default-bridge fields require prior-state evidence. +`template-compatibility.json` pins the reviewed embedded template to ci-fleet commit `840fb9be8a1c69404a5eebcf17855eff13c6c928`. It records schema and validator/initializer support, optional capabilities, standalone safety checks, and fictional examples pinned to that same engine with placeholder image IDs. `engine-rollout-evidence.json` records active per-controller capability evidence in the shape consumed by that pinned core. New status-reporting, Docker network-policy, and default-bridge fields require prior-state evidence. Managed-image scalars follow the selected runtime source's capability, with portable config digests kept separate from derived host runtime IDs. Remove storage-retention admission in an earlier integrated change and verify convergence before changing `engine_ref`. The reviewed evidence format has no @@ -164,12 +164,21 @@ the new engine and daemon inventory before a later re-admission change. Adding or retaining admission also requires current and previous controller states to be `active` or `drained`. Remove admission when disabling a controller, and enable it without admission before a later admission change. -For any reporting or Docker network capabilities that remain active, use -exactly two integrated commits: +For ordinary image-enforced upgrades, including controllers without optional +reporting/network blocks, use two integrated commits: 1. Keep `fleet.json` and `engine-rollout-evidence.json` on engine A. Add reviewed engine B capability evidence to optional adopter-owned `next-engine-rollout-evidence.json`, using the same schema-version-1 record shape. 2. Change `engine_ref` to engine B, promote the matching sidecar record to `engine-rollout-evidence.json`, and remove the sidecar when it has no remaining staged records. +Image-free adopters instead follow the separate [manager bootstrap and evidence +preparation sequence](docs/MANAGED-IMAGES.md#upgrade-an-image-free-adopter). Import +the actual reviewed template into the adopter before any rollout. The +[ownership map](docs/UPDATING.md#import-ownership-for-engine-specification-updates) +preserves private tests, policy, workflows and release records. The selected +merged engine includes PR157 maintenance and unused-pending preparation support. +Template validation/review and the adopter import remain separate gates before +authorized host activation. + The default examples and initializer omit optional fields because their engine capability has not been staged. The [network-policy review fragment](examples/docker-network-policy.json) shows syntax only. Before private activation, replace its RFC 5737 CIDRs with reviewed operational pools and bridge values. Storage retention stays absent by default and has a separate engine-first admission rule. Follow [Docker policy adoption and rollback](docs/UPDATING.md#optional-docker-network-and-storage-policy). Capacity telemetry is another engine capability. A private adopter may @@ -179,13 +188,14 @@ update does not select an engine or activate sampling on an adopter host. Fresh strict configurations require `managed_images`, keyed by full engine commit and `amd64` or `arm64`, with different nonzero controller and runner -`sha256:` image IDs. The initializer requires those inputs and validates them -before replacing its output. Public example IDs are placeholders and fail +`sha256:` image scalars. The selected runtime source chooses config-digest or +legacy opaque-ID semantics. The initializer validates those inputs before +replacing its output. Public example values are placeholders and fail strict validation. Existing image-free adopters must use the [manager bootstrap migration](docs/MANAGED-IMAGES.md#upgrade-an-image-free-adopter). -For configured reporting/network capabilities, integrate reviewed target-engine -sidecar evidence under the incumbent contract before importing the new staging -contract. Both bootstrap revisions require it and retain incumbent runtime evidence. +Import the reviewed specification, then prepare incumbent and target evidence +through a supporting trusted validator. Both bootstrap revisions require prior +target evidence for every controller and retain incumbent runtime evidence. Importing new files or changing only `engine_ref` cannot complete that migration. The optional [host-local APT cache](docs/APT-CACHE.md) needs no fleet field. @@ -308,8 +318,8 @@ size. The public ci-fleet engine owns the host installer. Run it from a reviewed, immutable ci-fleet checkout on a separately authorized isolated target host. This template checkout does not contain the installer. Before bootstrap, follow -the reviewed [host prerequisites and identity setup](https://github.com/RandomDevelopment/ci-fleet/blob/689c7c00deb0eebcb25f2284334a9f7168a25575/docs/DESIRED-STATE.md#prepare-host-local-identity) -and [GitHub App setup](https://github.com/RandomDevelopment/ci-fleet/blob/689c7c00deb0eebcb25f2284334a9f7168a25575/docs/GITHUB-APP-SETUP.md). +the reviewed [host prerequisites and identity setup](https://github.com/RandomDevelopment/ci-fleet/blob/840fb9be8a1c69404a5eebcf17855eff13c6c928/docs/DESIRED-STATE.md#prepare-host-local-identity) +and [GitHub App setup](https://github.com/RandomDevelopment/ci-fleet/blob/840fb9be8a1c69404a5eebcf17855eff13c6c928/docs/GITHUB-APP-SETUP.md). They require Debian 12 or newer, Docker Engine and Compose v2, the documented host tools and direct local socket access, Docker disk use below 80%, protected host identity and private key, GitHub App/runner-group provisioning, and read-only diff --git a/docs/LOCAL-VALIDATION.md b/docs/LOCAL-VALIDATION.md index e86559b..0c7d446 100644 --- a/docs/LOCAL-VALIDATION.md +++ b/docs/LOCAL-VALIDATION.md @@ -28,8 +28,12 @@ Preparation installs the shared executor and a revision/hash manifest under `/usr/local/lib/ci-fleet-validation` and creates a locked execution account and protected workspace parent. It adds no daemon, timer or unique fleet state. Use a Linux VM or remote prepared worker from macOS or Windows. Shared executor support merged in core [PR #144](https://github.com/RandomDevelopment/ci-fleet/pull/144) -at `689c7c00deb0eebcb25f2284334a9f7168a25575`. The development configuration-policy baseline now uses that same commit, -but executor and policy pins remain independent inputs. Pinning an executor +at `689c7c00deb0eebcb25f2284334a9f7168a25575`. The ordinary-CI +configuration-policy baseline is `840fb9be8a1c69404a5eebcf17855eff13c6c928`. +The executor remains independently pinned to the reviewed +`689c7c00deb0eebcb25f2284334a9f7168a25575` implementation. Engine merge, +template acceptance and host activation remain separate. +Executor and policy pins are independent inputs. Pinning an executor alone does not adopt the managed-image contract or activate a runtime. Never execute candidate scripts on an active fleet host or a machine holding @@ -114,9 +118,10 @@ not a value inferred from candidate files or its remote. Only fixtures. Never supply that identity for an adopter. The suite preserves the public fixture's origin identity in isolated checkouts so upstream-only tests do not silently disappear after cloning from a local path. Fictional upstream -reference configurations run non-strict checks. Managed-image migration ordering -applies to strict adopters; fictional upstream pin/image changes are not host -migrations. Both modes retain optional-capability predecessor and evidence checks. Initialized adopters must pass +reference configurations use non-strict validation for pin/image updates. +Strict adopters require managed-image target-evidence promotion. Both modes +retain reporting/network predecessor and evidence checks when configured. +Initialized adopters must pass strict validation, including transition checks against committed prior fleet, active evidence and next-engine evidence when a prior integrated state exists. @@ -161,7 +166,7 @@ The pinned engine checks current state. The independently committed template validator enforces the reviewed standalone transition rules and active/next evidence, including the standalone next-engine, network and storage safety checks. Both checks finish before any candidate initializer or test program executes. -Fresh initialized adopters need reviewed managed image IDs. Image-free adopter +Fresh initialized adopters need reviewed managed image scalars. Image-free adopter staging needs its complete committed predecessor and retained active/next-engine evidence; the all-zero no-prior-state sentinel cannot authorize that migration. Use a suite that supports [managed-image migration](MANAGED-IMAGES.md), not the @@ -279,6 +284,9 @@ SHAs with their complete ancestry. Engine-policy and trusted-suite fetches include only their selected commit and its ancestry. Checkouts inherit no unrelated branches or tags, and the standalone suite needs no inherited release refs. Updater and release-update tests still use their isolated release fixtures. +Fetches retain received packs with `--keep` instead of unpacking small histories +into many loose files. This reduces repeated seed-copy and manifest work while +preserving the selected histories and every verification step. Before freezing a seed manifest, the harness records the intended source main ref and normalizes origins. Suite and public-template source origins name the diff --git a/docs/MANAGED-IMAGES.md b/docs/MANAGED-IMAGES.md index 70de4b5..0a0c8f0 100644 --- a/docs/MANAGED-IMAGES.md +++ b/docs/MANAGED-IMAGES.md @@ -1,109 +1,159 @@ # Managed controller and runner images -The development template implements the schema-v3 managed-image contract at -ci-fleet `689c7c00deb0eebcb25f2284334a9f7168a25575`. Published template `v1.1.0` -does not support this field. Review and import the complete compatible template -and trusted suite before using these commands. This guide changes no host. +This development specification follows merged ci-fleet +`840fb9be8a1c69404a5eebcf17855eff13c6c928`, including portable-image, +PR157 maintenance and unused-pending preparation support. Validate and review +the actual template revision, import and accept it in the adopter, then perform +the authorized rollout. Published template +`v1.1.0` does not provide this contract. This guide changes no host. + +## Image identity + +`managed_images` keeps the schema-v3 engine, architecture and role scalar map. +Each full nonzero engine commit maps `amd64` and/or `arm64` to distinct nonzero +`controller` and `runner` values in `sha256:` form. + +The selected runtime source determines their meaning. If that exact source +advertises `portable_managed_image_ids`, the scalars are original image config +digests. Older sources retain exact opaque local Docker IDs. Do not infer the +meaning from digest syntax, storage backend or the manager executing the update. +Do not rewrite historical catalogs. The root installer binds portable config +digests to immutable runtime IDs and protects the binding proof outside Git. +Derived runtime IDs, rendered environments and host proof do not belong in +`managed_images` or rollout evidence. + +Follow the reviewed engine's +[build and verification procedure](https://github.com/RandomDevelopment/ci-fleet/blob/840fb9be8a1c69404a5eebcf17855eff13c6c928/docs/MANAGED-IMAGES.md#record-image-ids-for-an-engine-commit) +on an authorized isolated build host. Build the same actual reviewed merged +engine twice from clean state for each target architecture. Portable builds +must match their original config digests. Native import, export, launch, +resource limits and deadline acceptance remain separate evidence. A revision +label or a build of an unmerged head cannot substitute for that proof. Do not +build candidate engines on live workers. + +The reviewed transport lets Docker prepare the first export until +the remaining shared 180-second pair deadline. Connections, metadata responses +and gaps after streaming begins retain 30-second inactivity limits. Starting the +second role does not reset the pair budget. This adds no capability or evidence +field. Follow the pinned engine procedure above when recording build evidence. ## Fresh configuration -`managed_images` maps a full nonzero engine commit to `amd64` and/or `arm64`. -Each architecture declares different, nonzero `controller` and `runner` Docker -image IDs in `sha256:` format. Every controller's selected engine needs a map -entry. These are exact local image IDs, not mutable tags or registry names. -Public examples contain fictional IDs that strict validation rejects. - -Obtain reviewed IDs through the pinned -[build and verification procedure](https://github.com/RandomDevelopment/ci-fleet/blob/689c7c00deb0eebcb25f2284334a9f7168a25575/docs/MANAGED-IMAGES.md#record-image-ids-for-an-engine-commit) -on a separately authorized isolated build host. It requires repeated clean -builds for each target architecture using the reviewed build inputs. Do not -invent IDs, use a revision label as identity proof, or build on a live daemon. - -Initialize a private configuration with those reviewed values: +Initialize with the reviewed scalar appropriate to the selected source: ```bash ./scripts/init.sh \ --organization your-org --project your-app \ --engine-ref REVIEWED_ENGINE_COMMIT_SHA \ --architecture amd64 \ - --controller-image-digest sha256:REVIEWED_CONTROLLER_IMAGE_ID \ - --runner-image-digest sha256:REVIEWED_RUNNER_IMAGE_ID + --controller-image-digest sha256:REVIEWED_CONTROLLER_SCALAR \ + --runner-image-digest sha256:REVIEWED_RUNNER_SCALAR ./scripts/validate.sh --strict ``` -Replace every placeholder. Add other architectures to the same engine entry -only after separate verification. The initializer validates strict configuration -before atomically replacing its output. Missing inputs, unsupported architecture, -malformed, equal, zero or fixture IDs fail without replacing an existing file. -It still omits optional reporting, Docker network and storage policy. +Replace every placeholder. The initializer requires architecture and both +scalars, validates before atomically replacing its output, and rejects equal, +zero, malformed or fixture values. It omits optional reporting, network and +storage policy. An explicit bridge input prints a review-only fragment rather +than enabling it in `fleet.json`. + +Review and integrate the initial configuration with an empty active evidence +map and no optional policies. After installing that exact configuration, +require `--check` to report `CHECK_OK`. Review capabilities at the selected +source and observed runtime activation, then integrate active evidence in an +evidence-only commit. Add optional policy only in a later reviewed change. +`CHECK_STAGED` proves an incumbent during manager staging, not a fresh install. -Commit and independently review the source, then run the complete -[exact-source suite](LOCAL-VALIDATION.md) with a compatible reviewed suite and -independent current engine validator. A focused strict check alone is incomplete. +Run the complete committed [exact-source suite](LOCAL-VALIDATION.md) using a +compatible independently reviewed suite and engine validator. A focused strict +check alone is incomplete acceptance. ## Upgrade an image-free adopter -This is a semantic migration within schema v3. The updater leaves managed-image -policy changes for manual handling. Preserve adopter-owned fleet configuration, -active/next-engine evidence, private delivery configuration and release history. -Before importing the new contract, use the incumbent compatible validator and -reviewed suite to integrate target-engine evidence for every controller with -reporting or network policy. Keep engine A selected and record reviewed engine B -capabilities in `next-engine-rollout-evidence.json`. Include required-reporting -and default-bridge evidence when configured. The new image-free staging contract -permits only engine-ref changes and retains pending evidence, so it cannot add -that sidecar in a separate same-engine revision. Then import the complete -compatible schema, validator, initializer, tests and suite with the staging -revision; do not merge a mixed partial contract. - -Follow the pinned -[manager bootstrap procedure](https://github.com/RandomDevelopment/ci-fleet/blob/689c7c00deb0eebcb25f2284334a9f7168a25575/docs/MANAGED-IMAGES.md#two-step-desired-state-rollout) -under separate host-specific authorization: - -1. Remove storage-retention admission in an earlier integrated configuration and - verify convergence before any engine change. Prepare and review target image - IDs separately. Keep signing authority away from execution and build hosts. -2. Pause reconciliation on every activation target. Integrate a staging revision - that changes only controller `engine_ref` values and omits `managed_images`. - Keep every controller, other fleet value and incumbent capability record. - Reporting and network policy require matching engine-B capability evidence - from the previous integrated sidecar. Retain that sidecar unchanged. - Validate against the complete actual previous configuration and evidence. - Fresh installation cannot use image-free staging or an empty predecessor. -3. Apply staging with the reviewed candidate engine installer's `--upgrade` and - the approved configuration SHA. The incumbent installer cannot bootstrap the - newer manager. Require `MANAGER_STAGED` on every target. That result leaves - the incumbent controller and image running and proves no runtime activation. -4. In a later integrated revision, add reviewed `managed_images` while retaining - all staged controller settings and incumbent runtime capability evidence. - Validation checks the target engine against the previous integrated sidecar - again, even though the desired engine ref already changed during staging. - Apply it through the staged manager. Only this revision may build and activate - the new runtime. Do not change capacity, routing or controller state as part - of manager staging or alter controller settings during image-policy introduction. -5. After actual runtime verification, update active capability evidence to that - runtime in a follow-up revision. Resume reconciliation through the staged - manager and verify health and a later timer cycle. Retire a promoted next-engine - record when its new engine becomes the active runtime. Retain unchanged - pending evidence through bootstrap rather than manufacturing new attestations. - -Image-free staging cannot remove enforced image policy, add/remove controllers, -change unrelated settings or activate a runtime. Ordinary later engine upgrades -need the template's prior integrated next-engine sidecar for configured -reporting/network capabilities, as do both bootstrap revisions. Storage admission -must be absent across engine changes and separately re-admitted after engine -and exclusive-daemon verification. +This is a semantic migration within schema v3. The updater requires manual +handling when managed-image requirements or identity semantics change. Import +the complete reviewed specification while preserving private policy, tests, +workflow and release history through the [ownership procedure](UPDATING.md#import-ownership-for-engine-specification-updates). +Do not replace a customized adopter harness with the public upstream suite. +For an established image-free predecessor with unchanged fleet and active +evidence, the import may include only the explicitly reviewed pending preparation +transition described there. Unchanged fleet and evidence do not qualify as +manager staging. + +Use actual immediate predecessors throughout the migration: + +1. Remove retention admission in an earlier integrated same-engine change and + verify convergence. Keep every other fleet setting. Obtain and review the + actual merged target engine's image scalars separately. +2. For an established image-free fleet, integrate missing incumbent active + records in an evidence-only commit. They describe already observed runtime + and exact source capabilities, including explicit all-false records where + appropriate. Preserve all existing active and pending records. +3. In a later evidence-only commit, add the target's reviewed pending records. + Keep the entire fleet and active evidence unchanged. Every controller needs + a matching previously integrated pending record, including controllers with + no optional status/network blocks. `portable_managed_image_ids`, when present + in active or pending evidence, must be a boolean. Source review does not + attest runtime activation. +4. After the repository and isolated rehearsal gates pass, pause reconciliation + on the authorized activation targets. Integrate an image-free manager-staging + revision changing only controller `engine_ref` values. Keep every incumbent + active and target pending record. Apply it through the reviewed candidate + installer and require `MANAGER_STAGED`. The incumbent runtime remains active. +5. Introduce reviewed `managed_images` in a later integrated revision while + retaining all other fleet settings and both evidence files. Apply through + the staged manager. This is the runtime activation step. Neither capacity + nor routing may change as part of image-policy introduction. +6. After actual runtime verification, promote the matching pending records + unchanged into active evidence in a fleet-unchanged revision. Remove promoted + pending entries and the sidecar when empty. Resume approved reconciliation, + verify health and retain a later automatic cycle. + +Ordinary image-enforced upgrades also require previously integrated matching +pending evidence, even with no optional policy. Selection promotes that exact +record into active evidence and removes it from the pending map. Staging and +selection cannot share one commit. A true no-op cannot serve as image-free +manager staging. Image policy cannot be removed after enforcement. + +### Cancel or replace an unused pending target + +The selected merged engine supports this preparation. Use a compatible reviewed +template and independently pinned supporting trusted policy. Before manager +selection, a separate evidence-only transition may cancel or replace +unused pending records only when the complete image-free fleet and every active +record remain unchanged. Every controller must have explicit active evidence +matching its selected engine. Validate the actual immediate predecessor and +review the replacement target's capabilities normally. Remove the sidecar when +canceling its last entry; an empty sidecar is invalid. +The independently pinned trusted validator and executing manager must support +this route; candidate-local acceptance cannot override an older trusted policy. + +Integrate a replacement before selecting it. Missing incumbent evidence, +manager/runtime mismatch, image introduction, simultaneous selection and +unrelated changes do not qualify. Do not change the base, reinitialize the +adopter, infer activation from a source declaration or rewrite active evidence +to make a stale pending target disappear. ## Failure, removal and rollback -An image map cannot be removed after enforcement. Missing or mismatched images -fail closed. The engine prepares and verifies candidate images before drain; -revision labels alone do not satisfy exact image identity. Keep the incumbent -runtime and protected checkpoints until separately authorized acceptance. - -Rollback is the engine's protected checkpoint procedure, not deletion of -`managed_images`, release records or evidence. Retain legacy image objects; -pre-enforcement builds may be unreproducible. Lost legacy objects need verified -saved-image recovery or a separately authorized reinstall. Follow the pinned -[recovery contract](https://github.com/RandomDevelopment/ci-fleet/blob/689c7c00deb0eebcb25f2284334a9f7168a25575/docs/MANAGED-IMAGES.md). -Repository validation proves neither available host images nor live recovery. +The selected runtime source and the executing manager can differ. Preserve +trusted incumbent-compatible source before checkpoint capture, preflight, +drain and restoration. A portable candidate cannot supply an older runtime's +identity semantics or Compose context. Missing compatible source fails before +drain. For opaque sources, validate candidate Compose before checkpoint capture; +for portable sources, validate only after both role bindings are verified. +Failed Compose must not drain or publish canonical image tags. + +Keep exact old image catalogs, raw checkpoint IDs, original absence and retained +objects. A rebuild cannot recover a lost legacy opaque image. Recovery needs +verified retained-object import or separately authorized reinstall. Maintenance +must authenticate historical `release-target` context, including retained +manager source when the canonical release never existed. Source normalization +does not reconstruct an old image object. Portable cleanup still requires valid +binding proof, and unresolved opaque rollback identities defer removal. + +Use the engine's protected checkpoint procedure rather than deleting image +policy, checkpoints or evidence. See the reviewed +[recovery requirements](https://github.com/RandomDevelopment/ci-fleet/blob/840fb9be8a1c69404a5eebcf17855eff13c6c928/docs/MANAGED-IMAGES.md#recovery-and-retained-images). +Repository checks, a published template and source availability do not prove +native image availability, resource admission, runtime activation or rollback. diff --git a/docs/RELEASE.md b/docs/RELEASE.md index a49f0cf..a5243fd 100644 --- a/docs/RELEASE.md +++ b/docs/RELEASE.md @@ -14,20 +14,30 @@ They publish no release and change no required checks or host acceptance policy. ## Unreleased v1.2.0 preparation The development template targets ci-fleet commit -`689c7c00deb0eebcb25f2284334a9f7168a25575` for ordinary-CI configuration and the -independent external staging-deployer contract. Fleet schema remains v3. -Fresh strict configurations require reviewed controller/runner image IDs by +`840fb9be8a1c69404a5eebcf17855eff13c6c928` for portable ordinary-CI +configuration. The independent external staging-deployer and shared executor +pins remain `689c7c00deb0eebcb25f2284334a9f7168a25575`. The merged engine includes +PR157 maintenance and unused-pending preparation support. This template remains +unreleased, and an import does not deploy it. Fleet schema remains v3. +Fresh strict configurations require reviewed controller/runner image scalars by engine revision and architecture. The initializer requires those inputs. Image-free adopters need the separate manager bootstrap migration, retained runtime evidence and later image-policy activation. Both bootstrap revisions -require previously integrated target-engine sidecar evidence for configured -reporting/network capabilities. Integrate that evidence under the incumbent -compatible contract before importing the new staging contract. Standalone -network and storage safety checks remain in force. +require previously integrated target-engine sidecar evidence even when optional +reporting/network blocks are absent. Prepare explicit incumbent and pending +evidence against actual predecessors, preserve it through bootstrap, and promote +only after runtime verification. Do not replace stale pending evidence without +the reviewed cancellation contract. Standalone network/type and disabled-storage +safety checks remain in force. +Maintenance sequencing, preserved legacy idle blockers, strict admitted accounting +and actual scheduled reclamation acceptance are part of the prepared handoff. +The updater treats changed image-identity semantics as manual migration. The +engine, actual template revision and adopter import must pass their own gates +before deployment; an issue or preparation commit is not adoption. The updater treats a managed-image policy change as manual migration even without a schema-version bump. Current examples use the new engine pin with -fictional image IDs and omit unstaged optional policies. APT-cache adoption is +fictional image scalars and omit unstaged optional policies. APT-cache adoption is host-local and optional. See [managed images](MANAGED-IMAGES.md), [APT cache](APT-CACHE.md) and [updating](UPDATING.md). diff --git a/docs/UPDATING.md b/docs/UPDATING.md index 64333c6..5430c31 100644 --- a/docs/UPDATING.md +++ b/docs/UPDATING.md @@ -295,8 +295,8 @@ durable storage before discarding the worker. Keep signing separate and do not claim a sandbox against hostile root fixtures or same-stage transient tampering. The development ordinary-CI compatibility pin is -`689c7c00deb0eebcb25f2284334a9f7168a25575`. Public examples now use that engine -with fictional image IDs. Immutable release records remain unchanged. The local-evidence contract merged +`840fb9be8a1c69404a5eebcf17855eff13c6c928`. Public examples use that engine +with fictional image scalars. Immutable release records remain unchanged. The local-evidence contract merged through [ci-fleet PR #137](https://github.com/RandomDevelopment/ci-fleet/pull/137) at `67beabbddaad73c19f15bc3baa227779bc88dc96`, with [complete engine validation outside Actions](https://github.com/RandomDevelopment/ci-fleet/issues/135#issuecomment-6200457374). @@ -397,6 +397,58 @@ activation or make an unmerged engine a released compatibility baseline. Migration programs use `scripts/migrate-v-to-v.py`. Release notes name the exact path. If a release changes `schema_version` but does not ship that named program, stop the update. +## Import ownership for engine specification updates + +The delivery order is engine acceptance, actual reviewed template update, adopter +import from that exact template revision, then authorized deployment. An issue, +prepared branch or host-local edit does not complete either repository step. +Until engine acceptance establishes the final merged identity, preparation must +not select an unmerged engine or claim a final compatibility pin. + +Use the [unpublished reviewed revision procedure](#import-an-unpublished-reviewed-revision) +when no supporting template release exists. Record the full reviewed template +SHA and its engine contract in the adopter PR. Preserve `TEMPLATE_RELEASE` and +historical compatibility records exactly; a source import is not a release. +Apply this ownership map before merging, and add actual customized paths to +`ADOPTER_PATHS` in that procedure: + +| Paths | Import action | +| --- | --- | +| `fleet.schema.json`, `scripts/validate.py`, `scripts/init.py` | Merge the complete reviewed contract by hunk. Keep private rejection checks, initializer defaults, real-predecessor validation and all active/pending inputs. Verify parity with the selected engine. | +| `README.md`, `AGENTS.md`, `docs/MANAGED-IMAGES.md`, `docs/UPDATING.md` | Merge specification and operator guidance while retaining adopter-specific procedures and historical evidence. Record the exact source revision and documentation impact. | +| `scripts/local_validate.py`, `scripts/run_candidate_tests.py`, private validation wrappers | Preserve the adopter harness and its independently pinned inventories. Import only reviewed necessary interface changes. Upstream paths are not a replacement private suite. | +| `scripts/test_policy.py`, private delivery, approval and initializer tests | Preserve complete private corpora and add focused contract cases. Do not copy public upstream test counts or replace private tests with the upstream corpus. | +| `.github/workflows/validate.yml`, updater policy and customized delivery workflows | Preserve required checks, permissions, immutable suite/engine pins and established workflow ownership. No optional upstream job may replace a required producer. | +| `fleet.json`, active evidence, delivery policy, `TEMPLATE_RELEASE`, historical compatibility | Preserve byte-for-byte, including absence, during specification import. Review engine selection and policy changes separately against their actual immediate predecessors. | +| `next-engine-rollout-evidence.json` | Preserve unless the import includes the explicitly reviewed evidence-preparation transition below. Retain its actual predecessor bytes as validation input. | + +An established image-free fleet with unchanged active and pending evidence fails +the strict manager-staging no-change guard. For an adopter with explicit active +records matching every selected engine, the specification import may include a +separately reviewed unused-pending cancellation or replacement through the +[supported preparation route](MANAGED-IMAGES.md#cancel-or-replace-an-unused-pending-target). +Only the necessary reviewed specification files and that pending transition may +change together. Keep the entire fleet and active evidence byte-for-byte, supply +the actual complete predecessor, and use an independently pinned supporting +trusted validator. Do not combine this preparation with manager selection, image +introduction, runtime-evidence promotion or unrelated policy changes. A local +candidate validator cannot override older trusted policy. + +Review the full resulting diff, run private strict configuration/delivery checks, +then run complete committed source/adopter suites with the reviewed suite and +engine policy. Preserve mandatory test inventories and reject unexpected skips. +The engine suite, standalone suite and private suite prove different contracts. +Do not substitute one report for another. + +Only after repository suites and evidence export, remove temporary Actions +runner grants before an isolated installation/transaction/rollback rehearsal. +Fleet-role markers would prevent later repository validation on that machine. +A modest synthetic controller can prove its own lifecycle but cannot establish +resource admission or maximum capacity for a larger live controller. Use an +isolated machine for candidate code and verify actual-host admission separately +through the authorized reviewed rollout. Live workers are never candidate-engine +test machines. + ## Engine manager upgrades and recovery outcomes An engine revision selected in `fleet.json`, an installed manager, and the @@ -407,37 +459,37 @@ validator and evidence support before changing a pin. Remove and converge any storage-retention admission before the pin change, then preserve the staging rules for remaining reporting/network capabilities. -The reviewed [upgrade and reconciliation contract](https://github.com/RandomDevelopment/ci-fleet/blob/689c7c00deb0eebcb25f2284334a9f7168a25575/docs/DESIRED-STATE.md) +The reviewed [upgrade and reconciliation contract](https://github.com/RandomDevelopment/ci-fleet/blob/840fb9be8a1c69404a5eebcf17855eff13c6c928/docs/DESIRED-STATE.md) documents the installer and result handling. Ordinary upgrades use the reviewed installer's `--upgrade` with the exact configuration SHA. Upgrades introducing managed image enforcement require the separate -[manager bootstrap procedure](https://github.com/RandomDevelopment/ci-fleet/blob/689c7c00deb0eebcb25f2284334a9f7168a25575/docs/MANAGED-IMAGES.md#two-step-desired-state-rollout): - -1. Obtain host-specific authorization and pause reconciliation before publishing - staging. Before importing the new contract, integrate reviewed target-engine - reporting/network capability evidence with the incumbent compatible validator - and suite, keeping the incumbent engine selected. Both manager staging and - image activation require that prior sidecar and retain it unchanged. - The incumbent installer cannot bootstrap the newer manager. -2. Apply the reviewed staging configuration with the candidate checkout's - `scripts/install-worker-controller.sh --upgrade`, the approved full config - SHA and controller ID. The upstream procedure preserves controller settings - and incumbent runtime capability evidence, omits `managed_images`, and must - report `MANAGER_STAGED` without activating the new runtime. -3. Only after every target has staged its manager, follow that contract's - reviewed image-ID activation and subsequent runtime-evidence steps. Resume - reconciliation with the compatible manager and verify runtime health and a - later timer cycle. Manager staging alone does not prove runtime activation. - -The development template supports `managed_images` and manager-only staging. -Use [the migration guide](MANAGED-IMAGES.md) when importing it into an image-free -adopter. `v1.1.0` has no such support. Import and review the complete compatible -schema, validator, initializer, tests and suite with the staging revision, after -integrating the required target-engine sidecar under the incumbent contract. -Capability staging and manager bootstrap are separate requirements. -Host bootstrap, image builds and runtime activation require separate authorization. - -The reviewed [rejection and recovery runbook](https://github.com/RandomDevelopment/ci-fleet/blob/689c7c00deb0eebcb25f2284334a9f7168a25575/docs/DESIRED-STATE.md#rejected-upgrades-and-uncertain-recovery) +[manager bootstrap procedure](https://github.com/RandomDevelopment/ci-fleet/blob/840fb9be8a1c69404a5eebcf17855eff13c6c928/docs/MANAGED-IMAGES.md#staged-desired-state-rollout): + +Follow the complete [managed-image migration](MANAGED-IMAGES.md). Prepare explicit +incumbent records from observed runtime activation and exact source review, then +integrate reviewed target pending records without changing the fleet. Every +controller needs predecessor target evidence, even without optional blocks. +Retain active and pending records through image-free manager staging and later +image-policy introduction. Promote matching evidence only after runtime verification +in a fleet-unchanged revision. Ordinary image-enforced selection also promotes +matching previously integrated target evidence and removes the pending record. + +The selected runtime source chooses original config-digest semantics when it +advertises `portable_managed_image_ids`; older scalars remain opaque Docker IDs. +The executing manager and digest syntax cannot choose those semantics. Keep +runtime IDs and protected proof outside configuration Git. Retain compatible +incumbent source and exact legacy image objects for preparation and rollback. + +The unused-pending cancellation route is described in +[managed images](MANAGED-IMAGES.md#cancel-or-replace-an-unused-pending-target). +Use supporting independently pinned trusted policy. Do not replace a stale target by +changing the predecessor, rewriting active evidence or combining retargeting with +selection. Current capability booleans remain typed and target validation stays +mandatory. Manager staging is an operational step and must wait for the actual +template and adopter repository gates below. It cannot use a live worker as a +candidate test machine. + +The reviewed [rejection and recovery runbook](https://github.com/RandomDevelopment/ci-fleet/blob/840fb9be8a1c69404a5eebcf17855eff13c6c928/docs/DESIRED-STATE.md#rejected-upgrades-and-uncertain-recovery) distinguishes these outcomes: - An explicit rejection during initial read-only admission emits `REJECTED`. @@ -473,7 +525,7 @@ and separately authorized host acceptance are distinct milestones. ## Optional Docker network and storage policy The reviewed ordinary-CI compatibility baseline is -`689c7c00deb0eebcb25f2284334a9f7168a25575`. These optional schema-v3 fields do +`840fb9be8a1c69404a5eebcf17855eff13c6c928`. These optional schema-v3 fields do not change an existing configuration that omits them. The default initializer and public fleet examples leave them absent. Full fleet examples use the current development baseline with fictional image IDs. Review fragments illustrate syntax without attesting host support. @@ -481,7 +533,7 @@ See the [review-only network-policy fragment](../examples/docker-network-policy. ### Address pools and the default bridge -The [reviewed network contract](https://github.com/RandomDevelopment/ci-fleet/blob/689c7c00deb0eebcb25f2284334a9f7168a25575/docs/DESIRED-STATE.md#schema-v3) +The [reviewed network contract](https://github.com/RandomDevelopment/ci-fleet/blob/840fb9be8a1c69404a5eebcf17855eff13c6c928/docs/DESIRED-STATE.md#schema-v3) permits `docker_network_policy.default_bridge_cidr`, an IPv4 interface CIDR with a usable bridge gateway. Its subnet must not overlap an address pool. After network, broadcast and gateway addresses, it must provide at least @@ -529,7 +581,7 @@ state and controller health. Retain recovery evidence if rollback fails. ### Disposable image and build-cache retention -The [reviewed maintenance contract](https://github.com/RandomDevelopment/ci-fleet/blob/689c7c00deb0eebcb25f2284334a9f7168a25575/docs/HOST-MAINTENANCE.md#docker-storage-retention) +The [reviewed maintenance contract](https://github.com/RandomDevelopment/ci-fleet/blob/840fb9be8a1c69404a5eebcf17855eff13c6c928/docs/HOST-MAINTENANCE.md#docker-storage-retention) requires activation of an engine advertising `docker_storage_retention` before adding this admission in a separate reviewed configuration change: @@ -560,6 +612,16 @@ supplies that predecessor context. A supplied predecessor must contain a active or that daemon inventory is complete; operators must verify both before admission. +The cleanup changes merged through engine PR157 use complete container inventory +independently of image/cache accounting, +run expired-resource and eligible network cleanup before admitted storage under +the existing idle and maintenance gates, and retain classic-builder parents with +surviving children. Later passes can select those parents after the last child +is gone. An admitted storage-accounting or image-protection failure still fails +the service after its preceding network pass. Network failures stop storage work. +Without retention admission, cleanup skips image/cache accounting. Missing lists +on an admitted daemon remain an error; an unknown response is not empty storage. + Before any engine change, remove retention admission in an earlier integrated same-engine change and verify convergence. Upgrade with admission absent, preserving the two-commit evidence procedure for any reporting or network @@ -588,6 +650,25 @@ Repository checks do not satisfy this live acceptance gate in An engine already blocked by 80% disk use cannot bootstrap this feature through that blocked upgrade. Initial scoped recovery requires operator authorization. +Runnable legacy or unknown workloads can defer applying maintenance indefinitely. +An old empty network and a full reference check under the proxy lock do not prove +that a running workload will not need it after that lock is released. Keep the +idle, Swarm, creation-grace, full-reference and uncertainty guards. Do not stop +legacy work or delete markers to make a timer succeed. Classify and retire +resources only through a separately authorized supported procedure. Record +actual scheduled eligible removals, protected-resource survival and later +reconciliation; timer enablement and deletion counts do not prove recovered disk +or subnet headroom. + +For repeated empty-name completion errors, compare installed and intended source. +PR148 already handles only explicit unassigned cancellations with its required +identity fields. Old logs without result, runner ID, assignment time and job/request +IDs leave the event unclassified. After supported rollout, retain those diagnostics +and actual job starts/completions, runner cleanup and later reconciliation. A +passing fixture or recovered job processing on an old engine is not listener-fix +or retention acceptance. See the reviewed +[controller diagnostics](https://github.com/RandomDevelopment/ci-fleet/blob/840fb9be8a1c69404a5eebcf17855eff13c6c928/docs/CONTROLLER-DESIGN.md). + To stop retention, remove the admission and converge that reviewed change before an unsupported engine downgrade. Do not edit rendered environment variables or delete uncertainty markers. Removing policy cannot restore deleted images or @@ -935,7 +1016,7 @@ The development baseline supports the current schema-v3 managed-image contract. Read [managed images](MANAGED-IMAGES.md) for fresh initialization, image-free manager staging, retained runtime evidence, image activation and rollback. This is a semantic migration within schema v3. The updater detects a changed -`template_contract.managed_images` declaration and leaves a draft/manual report +`template_contract.managed_images` or `managed_image_identity` declaration and leaves a draft/manual report without importing a partial tree. A new engine pin alone is insufficient. [APT-cache adoption](APT-CACHE.md) is optional host-local work. No cache field, diff --git a/examples/multi-host/fleet.json b/examples/multi-host/fleet.json index cdc61fc..2514633 100644 --- a/examples/multi-host/fleet.json +++ b/examples/multi-host/fleet.json @@ -18,7 +18,7 @@ } }, "managed_images": { - "689c7c00deb0eebcb25f2284334a9f7168a25575": { + "840fb9be8a1c69404a5eebcf17855eff13c6c928": { "amd64": { "controller": "sha256:1111111111111111111111111111111111111111111111111111111111111111", "runner": "sha256:2222222222222222222222222222222222222222222222222222222222222222" @@ -32,7 +32,7 @@ "state": "active", "scale_set_name": "sample-ci-primary", "lifecycle": "stable", - "engine_ref": "689c7c00deb0eebcb25f2284334a9f7168a25575", + "engine_ref": "840fb9be8a1c69404a5eebcf17855eff13c6c928", "min_runners": 0, "max_runners": 4, "runner_resources": { @@ -46,7 +46,7 @@ "state": "active", "scale_set_name": "sample-ci-remote", "lifecycle": "stable", - "engine_ref": "689c7c00deb0eebcb25f2284334a9f7168a25575", + "engine_ref": "840fb9be8a1c69404a5eebcf17855eff13c6c928", "min_runners": 0, "max_runners": 2, "runner_resources": { diff --git a/fleet.json b/fleet.json index b8036d1..d4eb441 100644 --- a/fleet.json +++ b/fleet.json @@ -18,7 +18,7 @@ } }, "managed_images": { - "689c7c00deb0eebcb25f2284334a9f7168a25575": { + "840fb9be8a1c69404a5eebcf17855eff13c6c928": { "amd64": { "controller": "sha256:1111111111111111111111111111111111111111111111111111111111111111", "runner": "sha256:2222222222222222222222222222222222222222222222222222222222222222" @@ -32,7 +32,7 @@ "state": "active", "scale_set_name": "example-ci-01", "lifecycle": "experimental", - "engine_ref": "689c7c00deb0eebcb25f2284334a9f7168a25575", + "engine_ref": "840fb9be8a1c69404a5eebcf17855eff13c6c928", "min_runners": 0, "max_runners": 1, "runner_resources": { diff --git a/fleet.schema.json b/fleet.schema.json index 82749f5..df8e1fa 100644 --- a/fleet.schema.json +++ b/fleet.schema.json @@ -71,6 +71,7 @@ } }, "managed_image_ids": { + "description": "Role scalars are original config digests when the selected runtime source declares portable_managed_image_ids; older source values remain exact opaque local Docker IDs. Derived runtime IDs and host binding proof never belong in this catalog.", "type": "object", "additionalProperties": false, "required": ["controller", "runner"], diff --git a/scripts/init.py b/scripts/init.py index 59d2505..e9c02f7 100755 --- a/scripts/init.py +++ b/scripts/init.py @@ -45,8 +45,8 @@ def parse_args() -> argparse.Namespace: parser.add_argument("--runner-memory-mib", type=positive_integer, default=4096, help="memory available to each runner") parser.add_argument("--engine-ref", required=True, help="reviewed full ci-fleet commit SHA") parser.add_argument("--architecture", required=True, choices=("amd64", "arm64"), help="architecture of the reviewed image IDs") - parser.add_argument("--controller-image-digest", required=True, help="reviewed controller image ID (sha256:...)") - parser.add_argument("--runner-image-digest", required=True, help="reviewed runner image ID (sha256:...)") + parser.add_argument("--controller-image-digest", required=True, help="reviewed controller scalar, config digest for portable-capable runtime sources, opaque Docker ID otherwise (sha256:...)") + parser.add_argument("--runner-image-digest", required=True, help="reviewed runner scalar, config digest for portable-capable runtime sources, opaque Docker ID otherwise (sha256:...)") parser.add_argument("--output", type=Path, default=ROOT / "fleet.json", help="output configuration path") parser.add_argument("--force", action="store_true", help="replace an existing non-example output file") return parser.parse_args() @@ -202,6 +202,8 @@ def main() -> int: temporary.unlink(missing_ok=True) print(f"Initialized {output}") print("Next: review controller capacity, run ./scripts/validate.sh --strict, and keep every secret value outside Git.") + print("Fresh controllers include managed_images and omit active-evidence records and optional policies. After install, require --check at the exact config to report CHECK_OK, then record observed runtime capabilities in an evidence-only commit before adding policy.") + print("Existing installations retain actual history and incumbent evidence through the documented upgrade path; do not reinitialize them.") if network_policy is not None: print("Review-only docker_network_policy fragment, not added to fleet.json.") print("Replace documentation ranges with reviewed operational pools and bridge policy, and stage per-controller engine capability evidence before manual insertion.") diff --git a/scripts/local_validate.py b/scripts/local_validate.py index 3828c74..6ee4a87 100644 --- a/scripts/local_validate.py +++ b/scripts/local_validate.py @@ -74,7 +74,7 @@ def clone(repository: Path, revision: str, target: Path, identities: list[str]) target.mkdir() subprocess.run(["git", "init", "--quiet", str(target)], env=git_environment(), check=True) subprocess.run(["git", "-C", str(target), "config", "remote.origin.url", str(repository)], env=git_environment(), check=True) - subprocess.run(["git", "-C", str(target), "fetch", "--quiet", "--no-tags", "origin", *identities], env=git_environment(), check=True) + subprocess.run(["git", "-C", str(target), "fetch", "--quiet", "--no-tags", "--keep", "origin", *identities], env=git_environment(), check=True) subprocess.run(["git", "-C", str(target), "checkout", "--quiet", "--detach", revision], env=git_environment(), check=True) diff --git a/scripts/test_core_compatibility.py b/scripts/test_core_compatibility.py index a88ceb5..7cf2c7e 100644 --- a/scripts/test_core_compatibility.py +++ b/scripts/test_core_compatibility.py @@ -17,7 +17,7 @@ from pathlib import Path from unittest.mock import patch -CORE_COMMIT = "689c7c00deb0eebcb25f2284334a9f7168a25575" +CORE_COMMIT = "840fb9be8a1c69404a5eebcf17855eff13c6c928" DEPLOYER_CORE_COMMIT = "689c7c00deb0eebcb25f2284334a9f7168a25575" DEPLOYER_CORE_HASHES = { "docs/DEPLOYER-HOST.md": "c54954aca9a5c11f08e96f7baa6a44664ba7427676fa3b3671f04f0143304c70", @@ -78,24 +78,24 @@ CONFIG_FILES = ("fleet.json", "examples/multi-host/fleet.json") ALLOWED_STANDALONE_HASHES = { "examples/multi-host/fleet.json": ( - "9109c11cdc91c117cd5da5e9b66bd49ca8161e47d894f833353742a249458867", + "516852a9ad4af78b1cd0004c87b12a3e19381b65dcf4396349237dcdc7057655", "d9b4fc8d6f412cdf54c1bf274df75ea8e531baef6e0d31d884c62092d23822db", ), "fleet.json": ( - "2c1a94e12f36921a966c295a1cb1cb40f800ade75bef47945605b6b14c0b78cf", + "f2aa8dd318e745784e084199cb1c04835b850da88877aa0f86a07de0e2a2f1bc", "3fffaa1b988077f7995b0c2ff9691af971ac042c870ebfbf6656b24c20c05593", ), "fleet.schema.json": ( - "d0d7a9de39d2f296c9f9ec192a19168d247cfcc1c8d425ff1f9a1bbdd6c9ca9b", - "95304063211e62cb48e6688462cee7b445f01a71987cdab1b8e60bbfeee8dcdf", + "7eb412024675f976004b4b85df7eb545cabf66c0479813fbc097db85af6f2451", + "dbcddbb04abf6e209fa86baf5475cdae81cac0a4a5d95ad300d978c3a86e99d3", ), "scripts/init.py": ( - "dad6b8e7c913c21f1d9b479f6e0cd7fdd5216af4d631b38f5e4fa259d71155c6", - "4e7f58162a83d65673304aebbd939dd4b419bd77795e7882240f48d30ee5ff5e", + "d2b583fb84434dcf2faae0b429d13c1be66bfd010e922ef7fe1f2d1dd610c9b8", + "cae2c34239fe1d40218107072ce91ca1f3c0f23c841e8e9845150edb51fc859a", ), "scripts/validate.py": ( - "b74f7deeb96561d7a25090b3c303659255a739c2c514a64e61643b28cf9f8024", - "dff9416365a8f3ce8cee63d6398563ceb8c6bfbe271355f35d0256e485a870bd", + "4664dbaecda8d68ba809fcc9cf6695f1b04812d6af8ef34ae005bbf1bfe724c8", + "bc2398b470625939b6228146b3ad2bae768244d58b1b7e2c47e4be2764a54ca9", ), } diff --git a/scripts/test_policy.py b/scripts/test_policy.py index 472b987..9cb0859 100755 --- a/scripts/test_policy.py +++ b/scripts/test_policy.py @@ -87,6 +87,324 @@ def documented_artifact_errors(root: Path) -> list[str]: class PolicyTests(unittest.TestCase): + def test_portable_image_capability_is_boolean_in_active_and_pending_evidence(self): + for source in ("engine-rollout-evidence.json", "next-engine-rollout-evidence.json", + "previous engine-rollout-evidence.json", "previous next-engine-rollout-evidence.json"): + for value in (False, True, 0, 1, None, "true", []): + with self.subTest(source=source, value=value): + validation = Validation() + records = validate_rollout_evidence({"schema_version": 1, + "status_reporting_engine_capabilities": {"example-ci-01": { + "engine_ref": "a" * 40, "status_reporting_config": False, + "required_status_reporting": False, "portable_managed_image_ids": value, + }}}, validation, source) + self.assertEqual(bool(validation.errors), type(value) is not bool) + if type(value) is bool: + self.assertIs(records["example-ci-01"]["portable_managed_image_ids"], value) + + def pending_rollout(self): + previous = reference_config() + first_controller(previous).pop("status_reporting", None) + previous["managed_images"] = {ref: {"amd64": {"controller": "sha256:" + "a" * 64, + "runner": "sha256:" + "b" * 64}} + for ref in previous["managed_images"]} + previous["organization"].update(slug="sample-org", registry="ghcr.io/sample-org") + project = first_project(previous) + previous["runner_pools"][project["ci_pool"]]["allowed_repositories"] = ["sample-org/sample-app"] + project.update(repository="sample-org/sample-app", image="ghcr.io/sample-org/sample-app") + name = next(iter(previous["controllers"])) + first_controller(previous)["docker_network_policy"] = { + "networks_per_runner": 1, "reserve_subnets": 1, + "default_address_pools": [{"base": "10.64.0.0/24", "size": 28}], + } + incumbent = first_controller(previous)["engine_ref"] + record = dict(engine_ref=incumbent, status_reporting_config=True, required_status_reporting=True, + docker_network_policy_config=True, docker_default_bridge_cidr_config=True) + active = {name: record} + pending = {name: {**record, "engine_ref": "a" * 40, "portable_managed_image_ids": True}} + return previous, name, active, pending + + def rollout_cli(self, previous, current, active, current_active, pending=None, current_pending=None, *, strict=True): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + command = [sys.executable, str(ROOT / "scripts/validate.py"), "--skip-path-scan"] + if strict: + command.append("--strict") + for option, value in (("previous-config", previous), ("config", current), + ("previous-rollout-evidence", {"schema_version": 1, "status_reporting_engine_capabilities": active}), + ("rollout-evidence", {"schema_version": 1, "status_reporting_engine_capabilities": current_active})): + path = root / option + path.write_text(json.dumps(value)) + command.extend(("--" + option, str(path))) + for option, value in (("previous-next-engine-rollout-evidence", pending), ("next-engine-rollout-evidence", current_pending)): + if value is not None: + path = root / option + path.write_text(json.dumps({"schema_version": 1, "status_reporting_engine_capabilities": value})) + command.extend(("--" + option, str(path))) + return subprocess.run(command, capture_output=True, text=True) + + def test_cli_fictional_pin_updates_keep_strict_and_optional_evidence_guards(self): + previous, name, active, pending = self.pending_rollout() + network = previous["controllers"][name].pop("docker_network_policy") + current = copy.deepcopy(previous) + current["controllers"][name]["engine_ref"] = "a" * 40 + current["managed_images"]["a" * 40] = copy.deepcopy(next(iter(previous["managed_images"].values()))) + result = self.rollout_cli(previous, current, {}, {}, strict=False) + self.assertEqual(result.returncode, 0, result.stderr) + result = self.rollout_cli(previous, current, {}, {}) + self.assertNotEqual(result.returncode, 0) + self.assertIn("requires next-engine capability evidence", result.stderr) + result = self.rollout_cli(previous, current, {}, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + for field, value in (("docker_network_policy", network), + ("status_reporting", {"enabled": False, "config_file": "/etc/ci-fleet/monitoring.env"})): + before, after = copy.deepcopy(previous), copy.deepcopy(current) + before["controllers"][name][field] = value + after["controllers"][name][field] = value + result = self.rollout_cli(before, after, active, pending, strict=False) + self.assertNotEqual(result.returncode, 0) + self.assertIn("requires next-engine capability evidence", result.stderr) + + def test_pending_record_must_be_integrated_before_ordinary_selection(self): + previous, name, active, pending = self.pending_rollout() + current = copy.deepcopy(previous) + current["controllers"][name]["engine_ref"] = "a" * 40 + current["managed_images"]["a" * 40] = copy.deepcopy(next(iter(current["managed_images"].values()))) + result = self.rollout_cli(previous, current, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + for prior in (None, {name: {**pending[name], "engine_ref": "b" * 40}}, + {name: {**pending[name], "required_status_reporting": False}}): + result = self.rollout_cli(previous, current, active, pending, prior, pending) + self.assertNotEqual(result.returncode, 0) + + def test_manager_bootstrap_retains_active_and_pending_through_image_introduction(self): + previous, name, active, pending = self.pending_rollout() + images = previous.pop("managed_images") + staged = copy.deepcopy(previous) + staged["controllers"][name]["engine_ref"] = "a" * 40 + result = self.rollout_cli(previous, staged, active, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + for prior, next_record, evidence in ((None, pending, active), (pending, None, active), + (pending, {name: {**pending[name], "required_status_reporting": False}}, active), + (pending, None, pending)): + result = self.rollout_cli(previous, staged, active, evidence, prior, next_record) + self.assertNotEqual(result.returncode, 0) + introduced = copy.deepcopy(staged) + introduced["managed_images"] = {"a" * 40: next(iter(images.values()))} + result = self.rollout_cli(staged, introduced, active, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + promoted = self.rollout_cli(introduced, introduced, active, pending, pending) + self.assertEqual(promoted.returncode, 0, promoted.stderr) + changed = copy.deepcopy(introduced) + changed["controllers"][name]["state"] = "drained" + result = self.rollout_cli(introduced, changed, active, pending, pending) + self.assertNotEqual(result.returncode, 0) + result = self.rollout_cli(staged, introduced, active, pending, pending) + self.assertNotEqual(result.returncode, 0) + + def test_pending_contract_preserves_no_change_image_free_rejection(self): + previous, _, active, pending = self.pending_rollout() + previous.pop("managed_images") + result = self.rollout_cli(previous, previous, active, active, pending, pending) + self.assertNotEqual(result.returncode, 0) + self.assertIn("at least one engine_ref change", result.stderr) + + def test_cli_pending_contract_with_and_without_optional_features(self): + for optional in (False, True): + with self.subTest(optional=optional): + previous, name, active, pending = self.pending_rollout() + if optional: + previous["controllers"][name]["status_reporting"] = { + "enabled": True, "config_file": "/etc/ci-fleet/monitoring.env"} + else: + previous["controllers"][name].pop("docker_network_policy") + previous["controllers"][name].pop("status_reporting", None) + for records in (active, pending): + for key in records[name]: + if key != "engine_ref": + records[name][key] = False + ordinary = copy.deepcopy(previous) + ordinary["controllers"][name]["engine_ref"] = "a" * 40 + ordinary["managed_images"]["a" * 40] = copy.deepcopy(next(iter(previous["managed_images"].values()))) + result = self.rollout_cli(previous, ordinary, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + wrong = {name: {**pending[name], "engine_ref": "b" * 40}} + changed = {name: {**pending[name], "required_status_reporting": not pending[name]["required_status_reporting"]}} + for prior, selected, current_pending in ((None, pending, pending), (wrong, pending, None), + (pending, changed, None)): + self.assertNotEqual(self.rollout_cli(previous, ordinary, active, selected, prior, current_pending).returncode, 0) + images = previous.pop("managed_images") + staged = copy.deepcopy(previous) + staged["controllers"][name]["engine_ref"] = "a" * 40 + introduced = copy.deepcopy(staged) + introduced["managed_images"] = {"a" * 40: next(iter(images.values()))} + for before, after in ((previous, staged), (staged, introduced)): + result = self.rollout_cli(before, after, active, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + for prior, current in ((None, None), (None, pending), (wrong, wrong), (pending, changed)): + self.assertNotEqual(self.rollout_cli(before, after, active, active, prior, current).returncode, 0) + self.assertNotEqual(self.rollout_cli(before, after, active, pending, pending).returncode, 0) + result = self.rollout_cli(introduced, introduced, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + + def test_cli_image_free_evidence_preparation_uses_each_immediate_predecessor(self): + for missing_incumbent in (False, True): + for portable in (False, True): + with self.subTest(missing_incumbent=missing_incumbent, portable=portable): + fleet, name, active, pending = self.pending_rollout() + images = fleet.pop("managed_images") + fleet["controllers"][name].pop("docker_network_policy") + for records in (active, pending): + for key in records[name]: + if key != "engine_ref": + records[name][key] = False + pending[name]["portable_managed_image_ids"] = portable + before_active = {} if missing_incumbent else active + if missing_incumbent: + result = self.rollout_cli(fleet, fleet, {}, active) + self.assertEqual(result.returncode, 0, result.stderr) + result = self.rollout_cli(fleet, fleet, active, active, None, pending) + self.assertEqual(result.returncode, 0, result.stderr) + staged = copy.deepcopy(fleet) + staged["controllers"][name]["engine_ref"] = pending[name]["engine_ref"] + result = self.rollout_cli(fleet, staged, active, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + introduced = copy.deepcopy(staged) + introduced["managed_images"] = {pending[name]["engine_ref"]: next(iter(images.values()))} + result = self.rollout_cli(staged, introduced, active, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + # Configuration-only proof of the final transition. Real activation is external. + result = self.rollout_cli(introduced, introduced, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + changed = copy.deepcopy(fleet) + changed["controllers"][name]["state"] = "drained" + wrong = {name: {**pending[name], "engine_ref": "b" * 40}} + cases = ( + (fleet, fleet, active, active, None, None), # true no-op + (fleet, changed, active, active, None, pending), + (fleet, staged, before_active, active, None, pending), + (fleet, staged, active, pending, pending, None), + (fleet, fleet, before_active, pending, None, pending), + (fleet, staged, active, active, wrong, pending), + (staged, staged, active, pending, pending, None), + (staged, introduced, active, pending, pending, None), + (introduced, staged, pending, pending, None, None), + ) + for case in cases: + result = self.rollout_cli(*case) + self.assertNotEqual(result.returncode, 0, result.stdout) + empty = copy.deepcopy(fleet) + empty["controllers"] = {} + self.assertNotEqual(self.rollout_cli(empty, fleet, {}, active, None, pending).returncode, 0) + + def test_cli_unused_image_free_pending_evidence_can_be_canceled_or_retargeted(self): + fleet, name, active, pending = self.pending_rollout() + images = fleet.pop("managed_images") + replacement = {name: {**pending[name], "engine_ref": "b" * 40}} + for selected in (None, replacement): + result = self.rollout_cli(fleet, fleet, active, active, pending, selected) + self.assertEqual(result.returncode, 0, result.stderr) + result = self.rollout_cli(fleet, fleet, active, active, None, replacement) + self.assertEqual(result.returncode, 0, result.stderr) + selected = copy.deepcopy(fleet) + selected["controllers"][name]["engine_ref"] = replacement[name]["engine_ref"] + result = self.rollout_cli(fleet, selected, active, active, replacement, replacement) + self.assertEqual(result.returncode, 0, result.stderr) + + changed = copy.deepcopy(fleet) + changed["controllers"][name]["state"] = "drained" + staged = copy.deepcopy(fleet) + staged["controllers"][name]["engine_ref"] = pending[name]["engine_ref"] + introduced = copy.deepcopy(staged) + introduced["managed_images"] = {pending[name]["engine_ref"]: next(iter(images.values()))} + changed_active = {name: {**active[name], "required_status_reporting": False}} + for before, after, old_active, new_active, target in ( + (fleet, changed, active, active, replacement), + (fleet, fleet, active, changed_active, replacement), + (fleet, fleet, {}, {}, replacement), + (staged, staged, active, active, replacement), + (staged, introduced, active, active, replacement), + (fleet, staged, active, active, replacement), + (fleet, fleet, active, active, {name: {**replacement[name], "engine_ref": active[name]["engine_ref"]}}), + (fleet, fleet, active, active, {name: {**replacement[name], "portable_managed_image_ids": "true"}}), + ({}, fleet, active, active, replacement), + ): + result = self.rollout_cli(before, after, old_active, new_active, pending, target) + self.assertNotEqual(result.returncode, 0, result.stdout) + unknown = copy.deepcopy(fleet) + unknown["controllers"]["sample-ci-02"] = {**unknown["controllers"][name], "scale_set_name": "sample-ci-02"} + first_pool = next(iter(unknown["runner_pools"].values())) + first_pool["capacity_budget"] += unknown["controllers"][name]["max_runners"] + result = self.rollout_cli(unknown, unknown, active, active, pending, replacement) + self.assertNotEqual(result.returncode, 0, result.stdout) + all_active = {**active, "sample-ci-02": {**active[name]}} + result = self.rollout_cli(unknown, unknown, all_active, all_active, pending, replacement) + self.assertEqual(result.returncode, 0, result.stderr) + fleet["controllers"][name].pop("docker_network_policy") + inactive = {name: {key: value if key == "engine_ref" else False for key, value in active[name].items()}} + result = self.rollout_cli(fleet, fleet, inactive, inactive, pending, replacement) + self.assertEqual(result.returncode, 0, result.stderr) + + def test_empty_pending_sidecar_is_rejected(self): + validation = Validation() + validate_rollout_evidence({"schema_version": 1, "status_reporting_engine_capabilities": {}}, + validation, "next-engine-rollout-evidence.json") + self.assertTrue(any("remove the sidecar" in error for error in validation.errors)) + + def test_cli_image_free_bootstrap_requires_explicit_incumbent_records(self): + for portable in (False, True): + previous, name, _, pending = self.pending_rollout() + previous["controllers"][name].pop("docker_network_policy") + previous["controllers"][name].pop("status_reporting", None) + for key in pending[name]: + if key != "engine_ref": + pending[name][key] = False + pending[name]["portable_managed_image_ids"] = portable + images = previous.pop("managed_images") + staged = copy.deepcopy(previous) + staged["controllers"][name]["engine_ref"] = "a" * 40 + introduced = copy.deepcopy(staged) + introduced["managed_images"] = {"a" * 40: next(iter(images.values()))} + wrong = {name: {**pending[name], "engine_ref": "b" * 40}} + changed = {name: {**pending[name], "status_reporting_config": True}} + for before, after in ((previous, staged), (staged, introduced)): + for prior, current in ((pending, pending), (None, None), (wrong, wrong), (pending, changed)): + with self.subTest(portable=portable, introducing=after is introduced, pending=prior): + result = self.rollout_cli(before, after, {}, {}, prior, current) + self.assertNotEqual(result.returncode, 0, result.stdout + result.stderr) + self.assertIn("requires explicit incumbent runtime capability evidence", result.stderr) + + def test_cli_all_false_incumbent_bootstrap_retains_pending_until_promotion(self): + for portable in (False, True): + previous, name, active, pending = self.pending_rollout() + previous["controllers"][name].pop("docker_network_policy") + previous["controllers"][name].pop("status_reporting", None) + for records in (active, pending): + for key in records[name]: + if key != "engine_ref": + records[name][key] = False + pending[name]["portable_managed_image_ids"] = portable + images = previous.pop("managed_images") + staged = copy.deepcopy(previous) + staged["controllers"][name]["engine_ref"] = "a" * 40 + introduced = copy.deepcopy(staged) + introduced["managed_images"] = {"a" * 40: next(iter(images.values()))} + wrong = {name: {**pending[name], "engine_ref": "b" * 40}} + changed = {name: {**pending[name], "status_reporting_config": True}} + for before, after in ((previous, staged), (staged, introduced)): + with self.subTest(portable=portable, introducing=after is introduced): + result = self.rollout_cli(before, after, active, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + for prior, current in ((None, None), (None, pending), (wrong, wrong), + (pending, changed), (pending, None)): + self.assertNotEqual(self.rollout_cli(before, after, active, active, prior, current).returncode, 0) + self.assertNotEqual(self.rollout_cli(before, after, active, pending, pending).returncode, 0) + result = self.rollout_cli(introduced, introduced, active, pending, pending) + self.assertEqual(result.returncode, 0, result.stderr) + for selected, prior, current in ((pending, None, None), (pending, wrong, None), + (changed, pending, None), (pending, pending, pending)): + self.assertNotEqual(self.rollout_cli(introduced, introduced, active, selected, prior, current).returncode, 0) + @classmethod def compatibility_roots(cls) -> tuple[Path, Path]: if not hasattr(cls, "_compatibility_roots"): @@ -191,7 +509,7 @@ def validate(before, after, current_evidence=active, *, pending=None, prior_pend self.assertIn("requires next-engine capability evidence from the previous integrated sidecar", result.stderr) for before, after, error in ( (previous, previous, "requires at least one engine_ref change"), - (previous, enforced, "image-policy introduction must retain controller settings"), + (previous, enforced, "image-policy introduction must retain all manager-staging settings"), (enforced, staged, "cannot be removed after image enforcement"), ({"controllers": {}}, staged, "initial strict configuration requires reviewed image IDs"), ): @@ -230,7 +548,7 @@ def validate(before, after, current_evidence=active, *, pending=None, prior_pend result = validate(admitted, admitted_staging) self.assertNotEqual(result.returncode, 0) self.assertIn("must be removed in an earlier commit before changing engine_ref", result.stderr) - result = validate(enforced, enforced, promoted) + result = validate(enforced, enforced, promoted, prior_pending=promoted) self.assertEqual(result.returncode, 0, result.stderr) def test_bootstrap_requires_target_capabilities_before_staging_and_activation(self) -> None: @@ -248,7 +566,7 @@ def test_bootstrap_requires_target_capabilities_before_staging_and_activation(se active = {"worker": {"engine_ref": "1" * 40, **capabilities}} target = {"engine_ref": "2" * 40, **capabilities} for before, after in ((previous, staged), (staged, enforced)): - records = [(None, bool(settings)), ({**target, "engine_ref": "3" * 40}, bool(settings)), (target, False)] + records = [(None, True), ({**target, "engine_ref": "3" * 40}, True), (target, False)] records.extend(({**target, key: False}, True) for key in capabilities) for record, rejected in records: with self.subTest(settings=settings, activation="managed_images" in after, record=record): @@ -755,10 +1073,19 @@ def test_storage_retention_engine_first_admission_through_cli(self) -> None: current_path = root / "current.json" previous_path.write_text(json.dumps(previous)) current_path.write_text(json.dumps(current)) + extra = [] + if name == "pin change after removal": + target = {controller_name: {"engine_ref": "2" * 40, + "status_reporting_config": False, "required_status_reporting": False}} + staged = root / "staged.json" + staged.write_text(json.dumps({"schema_version": 1, + "status_reporting_engine_capabilities": target})) + extra = ["--previous-next-engine-rollout-evidence", str(staged), + "--rollout-evidence", str(staged)] result = subprocess.run( [sys.executable, str(ROOT / "scripts" / "validate.py"), "--strict", "--skip-path-scan", "--config", str(current_path), - "--previous-config", str(previous_path)], + "--previous-config", str(previous_path), *extra], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, ) if rejection is None: @@ -1691,7 +2018,7 @@ def test_old_engine_examples_omit_optional_capabilities(self) -> None: self.assertNotIn("docker_storage_retention", controller) self.assertEqual( controller["engine_ref"], - "689c7c00deb0eebcb25f2284334a9f7168a25575", + "840fb9be8a1c69404a5eebcf17855eff13c6c928", ) self.assertEqual(errors_for(config), []) @@ -1711,13 +2038,14 @@ def test_compatibility_record_names_exact_core_and_staged_template_contract(self compatibility["reviewed_core"], { "repository": "RandomDevelopment/ci-fleet", - "commit": "689c7c00deb0eebcb25f2284334a9f7168a25575", + "commit": "840fb9be8a1c69404a5eebcf17855eff13c6c928", "embedded_template_path": "templates/config-repository", }, ) self.assertEqual(compatibility["template_contract"]["fleet_schema_version"], 3) self.assertEqual(compatibility["template_contract"]["validator"], "scripts/validate.py") self.assertEqual(compatibility["template_contract"]["initializer"], "scripts/init.py") + self.assertEqual(compatibility["template_contract"]["managed_image_identity"], "runtime-source-capability") self.assertEqual( set(compatibility["optional_capabilities"]), {"status_reporting", "docker_network_policy", "docker_default_bridge_cidr", "docker_storage_retention"}, @@ -1731,37 +2059,35 @@ def test_compatibility_record_names_exact_core_and_staged_template_contract(self self.assertEqual(compatibility["optional_capabilities"]["docker_storage_retention"], { "support": "optional-engine-gated", "engine_capability": "docker_storage_retention", }) - self.assertEqual(compatibility["standalone_safety_deltas"], [{ - "path": "scripts/validate.py", - "reviewed_core_gap": "new controllers may introduce docker_network_policy without prior staged capability evidence", - "standalone_enforcement": "new controllers must omit docker_network_policy until engine rollout is proven", - }, { - "path": "scripts/validate.py", - "reviewed_core_gap": "malformed referenced host groups and non-string enum values can crash validation", - "standalone_enforcement": "type-check referenced host groups and enum values and return accumulated validation errors", - }, { - "path": "scripts/init.py", - "reviewed_core_gap": "initializer emits Docker network policy before per-controller engine evidence", - "standalone_enforcement": "initializer omits optional fields; explicit bridge input emits a review-only policy fragment", - }, { - "path": "scripts/validate.py", - "reviewed_core_gap": "bridge capability checks depend on transition context and engine upgrades cannot promote staged next-engine records", - "standalone_enforcement": "always require matching bridge evidence and verify promotion from the previous integrated next-engine sidecar", - }, { - "path": "scripts/validate.py", - "reviewed_core_gap": "storage retention admission is not ordered after an existing controller engine", - "standalone_enforcement": "require same-engine admission on active or drained predecessors and remove admission in an earlier commit before any engine_ref change", - }, { - "path": "fleet.schema.json", - "reviewed_core_gap": "storage retention is permitted on disabled controllers", - "standalone_enforcement": "restrict storage retention declarations to active or drained controllers", - }, { - "path": "scripts/validate.py", - "reviewed_core_gap": "manager bootstrap does not integrate standalone pending next-engine evidence or storage transition ordering", - "standalone_enforcement": "require prior target-engine capability evidence and retain pending sidecars through both bootstrap revisions; require storage removal before changing the selected engine even during manager staging", - }]) - self.assertEqual(compatibility["example_engine"]["commit"], "689c7c00deb0eebcb25f2284334a9f7168a25575") - self.assertEqual(compatibility["example_engine"]["pin_status"], "updated-for-managed-images") + self.assertEqual(compatibility["standalone_safety_deltas"], [ + { + "path": "scripts/validate.py", + "reviewed_core_gap": "malformed referenced host groups and non-string enum values can crash validation", + "standalone_enforcement": "type-check referenced host groups and enum values and return accumulated validation errors" + }, + { + "path": "scripts/init.py", + "reviewed_core_gap": "initializer emits Docker network policy before per-controller engine evidence", + "standalone_enforcement": "initializer omits optional fields; explicit bridge input emits a review-only policy fragment" + }, + { + "path": "fleet.schema.json", + "reviewed_core_gap": "storage retention is permitted on disabled controllers", + "standalone_enforcement": "restrict storage retention declarations to active or drained controllers" + }, + { + "path": "scripts/validate.py", + "reviewed_core_gap": "non-strict network-only CLI validation can omit current capability checks", + "standalone_enforcement": "require matching current network and bridge evidence in every CLI validation mode" + }, + { + "path": "scripts/validate.py", + "reviewed_core_gap": "non-strict fictional managed-image pin updates require host target-evidence promotion", + "standalone_enforcement": "enforce managed-image target promotion for strict adopters; retain reporting/network predecessor checks in both modes" + } + ]) + self.assertEqual(compatibility["example_engine"]["commit"], "840fb9be8a1c69404a5eebcf17855eff13c6c928") + self.assertEqual(compatibility["example_engine"]["pin_status"], "updated-for-portable-managed-images") self.assertEqual(compatibility["template_release"]["state"], "prepared-not-published") def test_exact_core_drift_check_is_pinned_and_runnable(self) -> None: @@ -1769,7 +2095,7 @@ def test_exact_core_drift_check_is_pinned_and_runnable(self) -> None: self.skipTest("ordinary-CI drift probe is upstream-only; CI separately verifies the deployer") checker = ROOT / "scripts" / "test_core_compatibility.py" text = checker.read_text(encoding="utf-8") - self.assertIn("689c7c00deb0eebcb25f2284334a9f7168a25575", text) + self.assertIn("840fb9be8a1c69404a5eebcf17855eff13c6c928", text) core, deployer = self.compatibility_roots() result = subprocess.run( [sys.executable, str(checker), "--core-root", str(core), "--deployer-core-root", str(deployer)], diff --git a/scripts/test_template_updater.py b/scripts/test_template_updater.py index 493ffd4..b8bbaf2 100644 --- a/scripts/test_template_updater.py +++ b/scripts/test_template_updater.py @@ -167,6 +167,19 @@ def test_managed_image_contract_change_requires_manual_migration_with_schema_v3( self.assertEqual(updater.read_file(proposed, "fleet.json"), updater.read_file(self.head, "fleet.json")) self.assertEqual(updater.read_file(proposed, "TEMPLATE_RELEASE"), updater.read_file(self.head, "TEMPLATE_RELEASE")) + def test_managed_image_identity_semantics_require_manual_import(self) -> None: + self.compatibility["template_contract"].pop("managed_image_identity", None) + self.write(self.template, "template-compatibility.json", self.compatibility) + baseline = self.release("v1.3.0") + self.compatibility["template_contract"]["managed_image_identity"] = "runtime-source-capability" + self.write(self.template, "template-compatibility.json", self.compatibility) + self.write(self.template, "validator.txt", "portable identity migration\n") + target = self.release("v1.4.0") + proposed, reasons = updater.prepare_tree(self.head, baseline, target) + self.assertTrue(any("Managed-image policy changed" in reason for reason in reasons), reasons) + for path in ("validator.txt", "fleet.json", "TEMPLATE_RELEASE"): + self.assertEqual(updater.read_file(proposed, path), updater.read_file(self.head, path)) + def test_three_way_merge_keeps_nonconflicting_local_edits_and_template_deletions(self) -> None: self.write(self.template, "shared.txt", "first\nmiddle\nlast\n") self.write(self.template, "removed.txt", "remove in next release\n") diff --git a/scripts/update_template.py b/scripts/update_template.py index 3920282..81edebc 100644 --- a/scripts/update_template.py +++ b/scripts/update_template.py @@ -187,7 +187,8 @@ def prepare_tree(head: str, baseline: dict, target: dict) -> tuple[str, list[str contract = target["compatibility"]["template_contract"] if json.loads(read_file(head, "fleet.json"))["schema_version"] != contract["fleet_schema_version"]: reasons.append("Schema migration required; follow docs/UPDATING.md and the release migration notes.") - if contract.get("managed_images") != baseline["compatibility"]["template_contract"].get("managed_images"): + if any(contract.get(key) != baseline["compatibility"]["template_contract"].get(key) + for key in ("managed_images", "managed_image_identity")): reasons.append("Managed-image policy changed; follow the manager bootstrap migration in docs/UPDATING.md.") policy = target["compatibility"]["deployment_policy_boundary"] if policy != baseline["compatibility"]["deployment_policy_boundary"]: diff --git a/scripts/validate.py b/scripts/validate.py index 5569182..238885f 100755 --- a/scripts/validate.py +++ b/scripts/validate.py @@ -591,7 +591,7 @@ def validate_engine_evidence( evidence, path, {"engine_ref", "status_reporting_config", "required_status_reporting"}, - optional, + optional | {"docker_default_bridge_cidr_config", "portable_managed_image_ids"}, ): return None ref = evidence.get("engine_ref") @@ -620,6 +620,12 @@ def validate_engine_evidence( parsed["docker_network_policy_config"] = network_policy if "docker_default_bridge_cidr_config" in evidence: parsed["docker_default_bridge_cidr_config"] = default_bridge + if "portable_managed_image_ids" in evidence: + portable = evidence["portable_managed_image_ids"] + validation.require(type(portable) is bool, f"{path}.portable_managed_image_ids", "must be a boolean") + if type(portable) is not bool: + return None + parsed["portable_managed_image_ids"] = portable return parsed if not validation.exact_keys( @@ -700,55 +706,57 @@ def validate_transition( validation: Validation, previous_compatible_engine_refs: dict[str, dict[str, Any]] | None = None, previous_next_engine_refs: dict[str, dict[str, Any]] | None = None, - *, enforce_managed_images: bool = True, + *, manager_bootstrap: bool = False, enforce_managed_images: bool = True, ) -> None: if not isinstance(previous, dict) or not isinstance(current, dict): return + validation.require( + "managed_images" not in previous or "managed_images" in current, + "$.managed_images", + "cannot be removed after image enforcement", + ) old_controllers = previous.get("controllers") new_controllers = current.get("controllers") if not isinstance(old_controllers, dict) or not isinstance(new_controllers, dict): return - # Fictional fixtures may update pins and image maps together; strict adopters must stage them. - if old_controllers and enforce_managed_images: + manager_bootstrap = manager_bootstrap or ( + enforce_managed_images and bool(old_controllers) + and ("managed_images" not in previous or "managed_images" not in current) + ) + if manager_bootstrap or (old_controllers and enforce_managed_images): if "managed_images" not in current: - validation.require("managed_images" not in previous, "$.managed_images", "cannot be removed after image enforcement") validation.require( {key: value for key, value in previous.items() if key != "controllers"} == {key: value for key, value in current.items() if key != "controllers"}, - "$", - "manager-only staging may change only controller engine_ref values", + "$", "manager-only staging may change only controller engine_ref values", ) validation.require(set(old_controllers) == set(new_controllers), "$.controllers", "manager-only staging cannot add or remove controllers") validation.require( - any( - isinstance(old_controllers[name], dict) and isinstance(new_controllers[name], dict) + any(isinstance(old_controllers[name], dict) and isinstance(new_controllers[name], dict) and old_controllers[name].get("engine_ref") != new_controllers[name].get("engine_ref") - for name in set(old_controllers) & set(new_controllers) - ), + for name in set(old_controllers) & set(new_controllers)), "$.controllers", "manager-only staging requires at least one engine_ref change", ) - for name in set(old_controllers) & set(new_controllers): - old = old_controllers[name] - new = new_controllers[name] - if not isinstance(old, dict) or not isinstance(new, dict): - continue - validation.require( - {key: value for key, value in old.items() if key != "engine_ref"} - == {key: value for key, value in new.items() if key != "engine_ref"}, - f"$.controllers.{name}", - "manager-only staging may change only engine_ref", - ) - if "managed_images" not in previous and "managed_images" in current: - validation.require(old_controllers == new_controllers, "$.controllers", "image-policy introduction must retain controller settings from manager staging") - validation.require(set(old_controllers) == set(new_controllers), "$.controllers", "image-policy introduction cannot add or remove controllers") for name in set(old_controllers) & set(new_controllers): old, new = old_controllers[name], new_controllers[name] if isinstance(old, dict) and isinstance(new, dict): validation.require( - old.get("engine_ref") == new.get("engine_ref"), - f"$.controllers.{name}.engine_ref", - "image policy must be introduced after the same compatible engine_ref is active", + {key: value for key, value in old.items() if key != "engine_ref"} + == {key: value for key, value in new.items() if key != "engine_ref"}, + f"$.controllers.{name}", "manager-only staging may change only engine_ref", ) + elif "managed_images" not in previous: + validation.require(set(old_controllers) == set(new_controllers), "$.controllers", "image-policy introduction cannot add or remove controllers") + for name in set(old_controllers) & set(new_controllers): + old, new = old_controllers[name], new_controllers[name] + if isinstance(old, dict) and isinstance(new, dict): + validation.require(old.get("engine_ref") == new.get("engine_ref"), + f"$.controllers.{name}.engine_ref", "image policy requires the same compatible engine_ref from manager staging") + validation.require( + {key: value for key, value in previous.items() if key != "managed_images"} + == {key: value for key, value in current.items() if key != "managed_images"}, + "$", "image-policy introduction must retain all manager-staging settings", + ) for name, new in new_controllers.items(): old = old_controllers.get(name) if not isinstance(new, dict): @@ -763,43 +771,31 @@ def validate_transition( f"$.controllers.{name}.docker_network_policy: must be omitted from a new controller until its engine rollout is proven" ) if "docker_storage_retention" in new: - validation.errors.append( - f"$.controllers.{name}.docker_storage_retention: must be omitted from a new controller until its engine rollout is proven" - ) + validation.errors.append(f"$.controllers.{name}.docker_storage_retention: must be omitted from a new controller until its engine rollout is proven") continue if "docker_storage_retention" in new: - validation.require( - old.get("state") in ("active", "drained") - and new.get("state") in ("active", "drained"), - f"$.controllers.{name}.docker_storage_retention", - "requires active or drained controller states in both the previous and current integrated configurations; enable without retention first", - ) + validation.require(old.get("state") in ("active", "drained") and new.get("state") in ("active", "drained"), + f"$.controllers.{name}.docker_storage_retention", "requires active or drained controller states in both the previous and current integrated configurations; enable without retention first") if "docker_storage_retention" not in old and "docker_storage_retention" in new: - validation.require( - old.get("engine_ref") == new.get("engine_ref"), - f"$.controllers.{name}.docker_storage_retention", - "must be introduced in a later commit after the compatible engine_ref is active", - ) + validation.require(old.get("engine_ref") == new.get("engine_ref"), f"$.controllers.{name}.docker_storage_retention", "must be introduced in a later commit after the compatible engine_ref is active") if "docker_storage_retention" in old: - validation.require( - old.get("engine_ref") == new.get("engine_ref"), - f"$.controllers.{name}.docker_storage_retention", - "must be removed in an earlier commit before changing engine_ref", - ) + validation.require(old.get("engine_ref") == new.get("engine_ref"), f"$.controllers.{name}.docker_storage_retention", "must be removed in an earlier commit before changing engine_ref") current_evidence = compatible_engine_refs.get(name, {}) - bootstrap = "managed_images" not in current or "managed_images" not in previous - runtime_ref = current_evidence.get("engine_ref", old.get("engine_ref")) if bootstrap else old.get("engine_ref") - if runtime_ref != new.get("engine_ref") and ( - "docker_network_policy" in new or "status_reporting" in new - ): - next_engine = (previous_next_engine_refs or {}).get(name, {}) + if manager_bootstrap: + runtime_ref = current_evidence.get("engine_ref") validation.require( - next_engine.get("engine_ref") == new.get("engine_ref"), + runtime_ref is not None, f"$.controllers.{name}.engine_ref", - "requires next-engine capability evidence from the previous integrated sidecar", + "requires explicit incumbent runtime capability evidence during manager bootstrap", ) - validate_reporting_evidence(name, new, next_engine, validation) - if bootstrap: + if runtime_ref != new.get("engine_ref"): + next_engine = (previous_next_engine_refs or {}).get(name, {}) + validation.require( + next_engine.get("engine_ref") == new.get("engine_ref"), + f"$.controllers.{name}.engine_ref", + "requires next-engine capability evidence from the previous integrated sidecar", + ) + validate_reporting_evidence(name, new, next_engine, validation) old = {**old, "engine_ref": runtime_ref} new = {**new, "engine_ref": runtime_ref} @@ -811,10 +807,23 @@ def validate_transition( previous_evidence = previous_evidence_source.get(name, {}) old_reporting = old.get("status_reporting") new_reporting = new.get("status_reporting") - old_network_policy = old.get("docker_network_policy") - new_network_policy = new.get("docker_network_policy") - old_default_bridge = isinstance(old_network_policy, dict) and "default_bridge_cidr" in old_network_policy - new_default_bridge = isinstance(new_network_policy, dict) and "default_bridge_cidr" in new_network_policy + if old.get("engine_ref") != new.get("engine_ref") and ( + "docker_network_policy" in new or "status_reporting" in new + or (enforce_managed_images and "managed_images" in current) + ): + next_engine = (previous_next_engine_refs or {}).get(name, {}) + validation.require( + next_engine.get("engine_ref") == new.get("engine_ref"), + f"$.controllers.{name}.engine_ref", + "requires next-engine capability evidence from the previous integrated sidecar", + ) + validate_reporting_evidence(name, new, next_engine, validation) + if enforce_managed_images and "managed_images" in current: + validation.require( + current_evidence == next_engine and next_engine.get("engine_ref") == new.get("engine_ref"), + f"$.controllers.{name}.engine_ref", + "requires promotion of the matching managed-image engine evidence from the previous integrated sidecar", + ) if "docker_network_policy" not in old and "docker_network_policy" in new: validation.require( old.get("engine_ref") == new.get("engine_ref"), @@ -827,7 +836,11 @@ def validate_transition( f"$.controllers.{name}.docker_network_policy", "requires reviewed evidence from the previous integrated state that this controller activated the same engine_ref with Docker network policy configuration capability", ) - if not old_default_bridge and new_default_bridge: + old_policy = old.get("docker_network_policy") if isinstance(old, dict) else None + new_policy = new.get("docker_network_policy") if isinstance(new, dict) else None + old_has_bridge = isinstance(old_policy, dict) and "default_bridge_cidr" in old_policy + new_has_bridge = isinstance(new_policy, dict) and "default_bridge_cidr" in new_policy + if not old_has_bridge and new_has_bridge: validation.require( old.get("engine_ref") == new.get("engine_ref"), f"$.controllers.{name}.docker_network_policy.default_bridge_cidr", @@ -837,14 +850,21 @@ def validate_transition( previous_evidence.get("engine_ref") == new.get("engine_ref") and previous_evidence.get("docker_default_bridge_cidr_config") is True, f"$.controllers.{name}.docker_network_policy.default_bridge_cidr", - "requires reviewed evidence from the previous integrated state that this controller activated the same engine_ref with default bridge CIDR capability", + "requires reviewed evidence from the previous integrated state that this controller activated the same engine_ref with Docker default bridge CIDR configuration capability", ) - if old_default_bridge and not new_default_bridge: + if old_has_bridge and not new_has_bridge: validation.require( old.get("engine_ref") == new.get("engine_ref"), f"$.controllers.{name}.docker_network_policy.default_bridge_cidr", "must be removed before changing engine_ref", ) + if new_has_bridge: + validation.require( + current_evidence.get("engine_ref") == new.get("engine_ref") + and current_evidence.get("docker_default_bridge_cidr_config") is True, + f"$.controllers.{name}.docker_network_policy.default_bridge_cidr", + "requires reviewed evidence that this controller activated the current engine_ref with Docker default bridge CIDR configuration capability", + ) staged_capability_required = ( "status_reporting" not in old or ( @@ -891,7 +911,7 @@ def parse_args() -> argparse.Namespace: return parser.parse_args() -def main() -> int: +def main(*, retained_configuration: bool = False) -> int: args = parse_args() validation = Validation() config_path = args.config.absolute() @@ -922,31 +942,81 @@ def main() -> int: schema = load_json(ROOT / "fleet.schema.json", validation) if schema is not None: validation.require(schema.get("$schema") == "https://json-schema.org/draft/2020-12/schema", "fleet.schema.json.$schema", "must use JSON Schema draft 2020-12") + if config is None: + validate_config(config, validation, args.strict) if config is not None: scan_secret_material(config, validation) previous = load_json(args.previous_config.absolute(), validation) if args.previous_config else None - bootstrap = ( - isinstance(previous, dict) and isinstance(config, dict) + current_controllers = config.get("controllers", {}) if isinstance(config, dict) else {} + previous_evidence = ( + load_json(args.previous_rollout_evidence.absolute(), validation) + if args.previous_rollout_evidence + else None + ) + previous_compatible_engine_refs = ( + validate_rollout_evidence(previous_evidence, validation) + if previous_evidence is not None + else {} + ) + previous_next_evidence = ( + load_json(args.previous_next_engine_rollout_evidence.absolute(), validation) + if args.previous_next_engine_rollout_evidence + else None + ) + previous_next_engine_refs = ( + validate_rollout_evidence( + previous_next_evidence, + validation, + "previous next-engine-rollout-evidence.json", + ) + if previous_next_evidence is not None + else {} + ) + # Preparation changes evidence without selecting a manager or runtime. + evidence_preparation = ( + isinstance(previous, dict) and isinstance(current_controllers, dict) and config == previous + and bool(previous.get("controllers")) and "managed_images" not in previous + and (current_compatible_engine_refs != previous_compatible_engine_refs + or current_next_engine_refs != previous_next_engine_refs) + and all(current_compatible_engine_refs.get(name) == record + for name, record in previous_compatible_engine_refs.items()) + and ( + all(current_next_engine_refs.get(name) == record + for name, record in previous_next_engine_refs.items()) + or ( + current_compatible_engine_refs == previous_compatible_engine_refs + and all(isinstance(controller, dict) + and current_compatible_engine_refs.get(name, {}).get("engine_ref") == controller.get("engine_ref") + for name, controller in current_controllers.items()) + ) + ) + and all(isinstance(current_controllers.get(name), dict) + and current_controllers[name].get("engine_ref") == record["engine_ref"] + for name, record in current_compatible_engine_refs.items()) + ) + manager_bootstrap = ( + not evidence_preparation and isinstance(config, dict) and isinstance(previous, dict) and isinstance(previous.get("controllers"), dict) and bool(previous["controllers"]) + and isinstance(current_controllers, dict) and ("managed_images" not in config or "managed_images" not in previous) + and ( + args.strict + or any((isinstance(current_controllers.get(name), dict) + and current_controllers[name].get("engine_ref") != record["engine_ref"]) + or (isinstance(previous["controllers"].get(name), dict) + and previous["controllers"][name].get("engine_ref") != record["engine_ref"]) + for name, record in current_compatible_engine_refs.items()) + ) ) - if bootstrap: + validate_config(config, validation, args.strict, allow_manager_staging=manager_bootstrap or evidence_preparation or retained_configuration) + if manager_bootstrap: validate_config(previous, validation, False) - validate_config(config, validation, args.strict, allow_manager_staging=bootstrap) - - current_controllers = config.get("controllers", {}) if isinstance(config, dict) else {} - if bootstrap: - previous_bootstrap_evidence = load_json(args.previous_rollout_evidence.absolute(), validation) if args.previous_rollout_evidence else None - retained_evidence = validate_rollout_evidence(previous_bootstrap_evidence, validation) if previous_bootstrap_evidence is not None else {} - validation.require(current_compatible_engine_refs == retained_evidence, "engine-rollout-evidence.json", "manager bootstrap must retain incumbent runtime capability evidence") - if isinstance(current_controllers, dict): - current_controllers = dict(current_controllers) - for name, value in current_controllers.items(): - if isinstance(value, dict): - old = previous["controllers"].get(name) - retained_ref = old.get("engine_ref") if isinstance(old, dict) else value.get("engine_ref") - runtime_ref = current_compatible_engine_refs.get(name, {}).get("engine_ref", retained_ref) - current_controllers[name] = {**value, "engine_ref": runtime_ref} + current_controllers = dict(current_controllers) + for name, controller in current_controllers.items(): + old = previous["controllers"].get(name) + if isinstance(controller, dict) and isinstance(old, dict): + runtime_ref = current_compatible_engine_refs.get(name, {}).get("engine_ref") + current_controllers[name] = {**controller, "engine_ref": runtime_ref} for controller, evidence in current_compatible_engine_refs.items(): current_controller = current_controllers.get(controller) if isinstance(current_controllers, dict) else None validation.require( @@ -979,48 +1049,35 @@ def main() -> int: if args.previous_config is not None: validation.require( isinstance(previous, dict) and isinstance(previous.get("controllers"), dict), - "--previous-config", - "must contain a controllers object for rollout validation", - ) - previous_evidence = ( - load_json(args.previous_rollout_evidence.absolute(), validation) - if args.previous_rollout_evidence - else None - ) - previous_compatible_engine_refs = ( - validate_rollout_evidence(previous_evidence, validation) - if previous_evidence is not None - else {} - ) - previous_next_evidence = ( - load_json(args.previous_next_engine_rollout_evidence.absolute(), validation) - if args.previous_next_engine_rollout_evidence - else None - ) - previous_next_engine_refs = ( - validate_rollout_evidence( - previous_next_evidence, - validation, - "previous next-engine-rollout-evidence.json", - ) - if previous_next_evidence is not None - else {} + "--previous-config", "must contain a controllers object for rollout validation", ) - if bootstrap: - validation.require(current_next_engine_refs == previous_next_engine_refs, "next-engine-rollout-evidence.json", "manager bootstrap must retain pending next-engine evidence") + if manager_bootstrap: + validation.require(current_compatible_engine_refs == previous_compatible_engine_refs, + "engine-rollout-evidence.json", "manager bootstrap must retain incumbent runtime capability evidence") + validation.require(current_next_engine_refs == previous_next_engine_refs, + "next-engine-rollout-evidence.json", "manager bootstrap must retain pending next-engine evidence") if previous is not None: previous_controllers = previous.get("controllers", {}) if isinstance(previous, dict) else {} + if ( + isinstance(previous, dict) and "managed_images" in previous and "managed_images" in config + and isinstance(previous_controllers, dict) + and any(isinstance(previous_controllers.get(name), dict) + and previous_controllers[name].get("engine_ref") != record["engine_ref"] + for name, record in previous_compatible_engine_refs.items()) + ): + validation.require(config == previous, "$", + "post-bootstrap evidence promotion must retain all runtime-verified fleet settings") for controller, evidence in previous_compatible_engine_refs.items(): previous_controller = previous_controllers.get(controller) if isinstance(previous_controllers, dict) else None validation.require( isinstance(previous_controller, dict) and ( previous_controller.get("engine_ref") == evidence["engine_ref"] - or (bootstrap and current_compatible_engine_refs.get(controller) == evidence) - # The first post-bootstrap attestation replaces retained runtime evidence. + or (manager_bootstrap and current_compatible_engine_refs.get(controller) == evidence) or ( - "managed_images" in previous + "managed_images" in previous and "managed_images" in config and current_compatible_engine_refs.get(controller, {}).get("engine_ref") == previous_controller.get("engine_ref") + and current_compatible_engine_refs.get(controller) == previous_next_engine_refs.get(controller) ) ), f"previous engine-rollout-evidence.json.status_reporting_engine_capabilities.{controller}.engine_ref", @@ -1054,7 +1111,8 @@ def main() -> int: validation, previous_compatible_engine_refs, previous_next_engine_refs, - enforce_managed_images=args.strict, + manager_bootstrap=manager_bootstrap, + enforce_managed_images=args.strict and not evidence_preparation, ) if args.tree_paths is not None: scan_tree_path_list(args.tree_paths, validation) diff --git a/template-compatibility.json b/template-compatibility.json index 1837681..700558e 100644 --- a/template-compatibility.json +++ b/template-compatibility.json @@ -2,7 +2,7 @@ "schema_version": 1, "reviewed_core": { "repository": "RandomDevelopment/ci-fleet", - "commit": "689c7c00deb0eebcb25f2284334a9f7168a25575", + "commit": "840fb9be8a1c69404a5eebcf17855eff13c6c928", "embedded_template_path": "templates/config-repository" }, "template_contract": { @@ -11,7 +11,8 @@ "validator": "scripts/validate.py", "initializer": "scripts/init.py", "rollout_evidence": "engine-rollout-evidence.json", - "managed_images": "required-for-fresh-strict-configurations" + "managed_images": "required-for-fresh-strict-configurations", + "managed_image_identity": "runtime-source-capability" }, "optional_capabilities": { "status_reporting": { @@ -32,11 +33,6 @@ } }, "standalone_safety_deltas": [ - { - "path": "scripts/validate.py", - "reviewed_core_gap": "new controllers may introduce docker_network_policy without prior staged capability evidence", - "standalone_enforcement": "new controllers must omit docker_network_policy until engine rollout is proven" - }, { "path": "scripts/validate.py", "reviewed_core_gap": "malformed referenced host groups and non-string enum values can crash validation", @@ -47,16 +43,6 @@ "reviewed_core_gap": "initializer emits Docker network policy before per-controller engine evidence", "standalone_enforcement": "initializer omits optional fields; explicit bridge input emits a review-only policy fragment" }, - { - "path": "scripts/validate.py", - "reviewed_core_gap": "bridge capability checks depend on transition context and engine upgrades cannot promote staged next-engine records", - "standalone_enforcement": "always require matching bridge evidence and verify promotion from the previous integrated next-engine sidecar" - }, - { - "path": "scripts/validate.py", - "reviewed_core_gap": "storage retention admission is not ordered after an existing controller engine", - "standalone_enforcement": "require same-engine admission on active or drained predecessors and remove admission in an earlier commit before any engine_ref change" - }, { "path": "fleet.schema.json", "reviewed_core_gap": "storage retention is permitted on disabled controllers", @@ -64,13 +50,18 @@ }, { "path": "scripts/validate.py", - "reviewed_core_gap": "manager bootstrap does not integrate standalone pending next-engine evidence or storage transition ordering", - "standalone_enforcement": "require prior target-engine capability evidence and retain pending sidecars through both bootstrap revisions; require storage removal before changing the selected engine even during manager staging" + "reviewed_core_gap": "non-strict network-only CLI validation can omit current capability checks", + "standalone_enforcement": "require matching current network and bridge evidence in every CLI validation mode" + }, + { + "path": "scripts/validate.py", + "reviewed_core_gap": "non-strict fictional managed-image pin updates require host target-evidence promotion", + "standalone_enforcement": "enforce managed-image target promotion for strict adopters; retain reporting/network predecessor checks in both modes" } ], "example_engine": { - "commit": "689c7c00deb0eebcb25f2284334a9f7168a25575", - "pin_status": "updated-for-managed-images", + "commit": "840fb9be8a1c69404a5eebcf17855eff13c6c928", + "pin_status": "updated-for-portable-managed-images", "compatibility": "fictional-image-ids-optional-fields-omitted" }, "template_release": {