Skip to content

feat: manual rollback that deploys, arbitrary-depth and divergence-safe - #158

Merged
joshua-temple merged 6 commits into
mainfrom
feat/manual-rollback-deploy
Jun 14, 2026
Merged

joshua-temple merged 6 commits into
mainfrom
feat/manual-rollback-deploy

Conversation

@joshua-temple

Copy link
Copy Markdown
Collaborator

Problem

cascade rollback resolved a prior target and rewrote manifest state but never deployed, never consulted the populated history ring, and left the env on the linear promotion line. An operator could not return an environment to a prior version with a real deploy, and a rewritten-but-undeployed state could silently propagate upward.

Fix

Manual, arbitrary-depth rollback that actually deploys and is divergence-safe.

  • Target resolution now reads live state, then the previous history ring (newest-first, any retained depth), then git history. --to <version|sha> is optional; with no target it defaults to N-1 from the ring (git fallback when the ring is empty).
  • A new generated cascade-rollback workflow (workflow_dispatch: environment, optional target, optional deployable, dry_run) wires preflight resolves the target -> deploy-<name> redeploys the resolved SHA through the same deploy callbacks promote uses -> finalize writes state and marks the env diverged. It emits only when the manifest declares environments.
  • A rolled-back env is marked diverged with a synthetic ref: rollback/<env> (plus base_sha for context) so the existing diverged-source guard blocks promoting the downgrade upward, and a later roll-forward rejoins and clears it through the existing finalize path.
  • Rejoin cleanup is made safe: a rollback-origin divergence has no real integration branch, so rejoin skips branch and hotfix-release deletion (which would otherwise error on a nonexistent branch) while still clearing the divergence fields. The hotfix-divergence path is unchanged.
  • Finalize honors the deployable scope, so a deployable-scoped rollback moves only that deployable and does not mark the whole env diverged.
  • Composes with the existing downgrade gate: the divergence guard governs promoting from the env, the downgrade gate governs promoting a lower version onto an env; orthogonal, no double-block, no bypass.

Additive only: promote and hotfix generation are unchanged.

Verification

  • go build ./..., go test ./... (1285 passing), go vet, golangci-lint run ./... clean on the main module.
  • e2e/ module builds and vets clean; golangci-lint clean.
  • Dogfood: generate-workflow --force then git diff --exit-code -- .github/workflows .github/actions is clean (the dogfood manifest declares no environments, so no rollback workflow emits).
  • New e2e gitea scenarios under e2e/scenarios/rollback/: deploys a prior version, marks-diverged-and-blocks-upward-promote, roll-forward-rejoins-without-cleaner-error, failed-deploy-leaves-state-unchanged, and deployable-scoped-leaves-env.
  • Unit coverage for ring resolution at depth 1 and >1, N-1 default and git fallback, diverged-after-rollback, rejoin-skips-branch-deletion (spy cleaner asserts zero deletions) with a hotfix regression guard, and the generated workflow shape.

…ce-safe

Resolve rollback targets ring-first (live state, then the deploy-history
ring newest-first, then git history) and default --to to the previous
version (N-1) when omitted. Mark a rolled-back environment diverged with a
rollback/<env> ref so upward promotion from it is guarded, and skip the
integration-branch and tag cleanup on rejoin for rollback refs since no such
branch exists. Generate a cascade-rollback workflow (preflight resolves the
target, deploy jobs redeploy the resolved SHA, finalize writes state) when
the manifest declares at least one environment. Finalize gates the state
write on deploy results so a failed deploy leaves the environment unchanged.

Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
…et env

The rollback scenarios with environments [dev, test, prod] used a single default-mode promote per version. With the release-tier topology (top env published as a release), reaching prod from dev takes a full cascade, so prod was never populated and the setup assertion that prod equals the prior commit failed before the rollback step ran.

Use a single cascade promote targeting prod (mode dev-to-prod) for each version, which advances through every hop and publishes prod in one atomic step, mirroring the cascade-promotion scenario. Rollback assertions are unchanged.

Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
@joshua-temple
joshua-temple force-pushed the feat/manual-rollback-deploy branch from 94b53e2 to 8569639 Compare June 14, 2026 17:08
The two-env deployable-scoped scenario also needs a cascade promote to reach prod (top env publishes a release), and the deployable-scoped rollback must target the per-deployable published version, not a raw commit SHA: the preflight resolves a scoped target by matching the per-deployable Version recorded in manifest history. Promote to prod via cascade and target v0.1.0.

Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
…tory reader

Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
The failed-deploy scenario stubbed the deploy as exit 1, which aborted the
setup promotions before the rollback re-deploy ran, so the deploy-result
gate was never exercised. Key the conditional failure on the invoking
workflow name so the Promote setup succeeds and only the Rollback re-deploy
fails.

The diverged-blocks-promote scenario sourced its onward cascade from
Environments[0] (on-trunk dev), so the test-to-prod leg from the diverged
env was never the source and the guard was not hit. Add an optional source
field to the harness promote step, threaded into the source-to-target mode
string (defaulting to the existing behavior), and set the blocked step to
source from the rolled-back test env.

Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
The harness suffixes every workflow name with [scenario-<tag>], so the
rollback re-deploy sees GITHUB_WORKFLOW as 'Rollback [scenario-...]' rather
than the bare 'Rollback'. Match the prefix so only the rollback re-deploy
fails and the deploy-result gate is exercised.

Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
@joshua-temple
joshua-temple merged commit 7940213 into main Jun 14, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant