Repository navigation
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Context
Epic #98 changes many command and module paths. The owner requires that updating the managed fleet from v0.18.1 to the next release causes no disruption. This issue is the release and rollout procedure. The release itself and every fleet mutation still need the owner's explicit go-ahead (release guard in AGENTS.md).
Facts from the 2026-11-10 review (verified against main 79f1dc5 and the open PRs #101, #105, #114):
- verified: no config struct, fingerprint, loader-default or
minimal/base/rockyprofile change since v0.18.1 (only fix(profiles)!: keep site-specific and no-op values out of the embedded profiles #101 adds--vlan-addresshandling in the loader). Applied-config fingerprints stay stable, so the v0.18.0 "fingerprint is missing" failure (fix(config): keep config fingerprints stable when exporter_firewall is unset #85) cannot recur. - verified: the remote command lines the controller sends (
apply --yes [--dry-run] --config,update --version,schedule enable|disable,status|check|doctor -o json) and the schedule unit commands (check -o json|prometheus,doctor -o json|prometheus) use only flags the new CLI still accepts on those commands (fix(cli): reject ignored root flags and validate capabilities on every mutating command #91 reader declarations). - verified: release binaries are
CGO_ENABLED=0, so the Go 1.26.10 toolchain (ci: improvement plan from the 2026-11-07 CI survey (toolchain pinning and required checks) #87) adds no glibc requirement for Ubuntu 20.04 or Rocky 8 hosts. - verified: managed sshd drop-ins already carry
Port <n>whenssh.portis set, so the fix(firewall): block SSH and UFW changes when firewall state is unknown #89/fix(ssh): run the Rocky SSH firewall check only when the port changes #104 firewall verification runs only on a real port change; an unchanged port stays untouched. - verified 2026-11-10:
rootfiles fleet check --allon the managed fleet (v0.18.1) is clean on every host, so any drift reported after the update comes from the new release. - inferred, to prove on the canary: fix(module): route writes through Runner, align Check with Apply, keep tests off the host #93 changes some Check output (time-sync service, Docker service and compose link, locale tools). On hosts already converged these should not report drift.
- chore: remove dead code and update outdated defaults #95 was amended on 2026-11-10 so the header rename and compose link removal cannot rewrite files or restart services on existing hosts.
Decisions
- Merge all epic PRs first and cut one release. No partial fleet rollout of an intermediate version.
- Canary first, then one host at a time. Rocky hosts and the tenant host get their own steps.
- Stop condition: any host where
fleet checkafterfleet updateis not clean, or wheredoctorgets worse than before. Investigate before continuing; neverfleet applyto make an unexpected drift go away. - The operator Mac's controller binary is updated to the same release before the fleet steps.
Design
- Release candidate: all epic PRs merged, CI green, release notes compiled from the PR bodies (
### Action neededmust be empty for fleet hosts, or each item must be handled below). - Pre-flight (read-only):
fleet status --all,fleet check --all,fleet doctor --allon the current release; save the JSON as the baseline. - Controller: update the Mac binary; re-run
fleet status --allto confirm the new controller reads the old hosts. - Canary (one GPU host with monitoring and no tenants):
fleet update --host <canary> --version vX --timeout 25m, thenfleet checkandfleet doctoron it. Expected: clean, no apply-pending. Only if a planned change shows, review it, thenfleet apply --host <canary> --dry-runbefore a real apply. - Remaining Ubuntu GPU and studio hosts, one at a time with the same check.
- Rocky hosts, one at a time (different package, SSH Include and SELinux paths).
- Tenant host last: also confirm running GPU containers still see their GPUs (
nvidia-smi -Linside each) after the update. - Post:
fleet check --allandfleet doctor --allclean; record versions and backup IDs in the private fleet record.
Changes
- No code. Release notes
docs/releases/vX.Y.Z.mdin the release PR.
Acceptance criteria
- AC1: every epic chore: 2026-11-09 audit plan (commands, modules, docs, specs) #98 PR is merged and CI is green on the release commit.
- AC2: canary shows clean
checkand unchanged-or-betterdoctorafterfleet update, with nofleet applyneeded. - AC3: all hosts updated;
fleet check --allandfleet doctor --allclean afterwards. - AC4: no service restart caused by the update other than those listed in the release notes (sshd, fail2ban, journald, cloudflared, docker untouched on converged hosts).
Tasks
- T1: release PR with notes (owner authorizes the tag)
- T2: baseline JSON (read-only)
- T3: controller update
- T4: canary update and verification
- T5: serial rollout (Ubuntu, then Rocky, then tenant host)
- T6: final verification and record
Out of scope
- Changing host site configs. Hosts keep their current configs.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Activity
entelecheia commentedon Nov 10, 2026
v0.19.0 is released (
82d8f2d, release run green; checksum and provenance attestation verified) and rolled out to all 11 managed hosts with nofleet apply.checksatisfied, the applied-config digest unchanged,doctorwith 0 failures and the same findings as the v0.18.1 baseline.fleet check --allandfleet doctor --allmatch the baseline on every host.Release authorization: the agent-hub guard cannot read a version from a repo without
package.json(tracked in STAIxBWLB/agent-hub#1381), so the owner pushed the tag directly.