Skip to content

feat: fix forward after a failed rollback instead of a pause that needs root #34

Closed

Description

@groupsky

Why

When an apply fails and its rollback fails too, pullgate pauses the instance, and only root on the host can lift that pause. That is safe, but it means a shell on the host for every such case, often from a phone, at the moment something is already broken. Most such failures are fixed by a new commit (fix forward), which pullgate could ask about as usual.

Proposal

A new state after a failed rollback: broken (not a plain pause).

  • The instance keeps fetching. For each newer commit it asks the normal question with a strong warning first: "⚠ The last apply and its rollback failed (…reason…). The host may be in a half state. This commit is a fix forward." The owner taps Apply as usual.
  • A successful apply plus health check ends the broken state.
  • A failed fix forward stays broken (no rollback is tried against a broken state) and the next newer commit is asked again with the warning.
  • The broken state never lifts itself, never re-applies the failed or the old commit, and /resume (the limited one) cannot clear it. Root can still pause/resume as today.
  • pullgate status and /status show "broken since …, last error …".
  • A notice goes out once when the state starts, with the reason and the two ways out (a fix-forward commit, or root on the host).

Done when

  • The broken state with the behaviour above; tests: failed apply + failed rollback → broken; a newer commit is asked with the warning; a good fix forward ends it; a bad one stays broken; the failed and the old commit are never asked again.
  • The question's warning and the notice show the reason (cut so the reason stays, as fix: a failed check notice cuts the output before the reason #25 asks).
  • DESIGN.md (sections 10–12) and THREAT-MODEL.md describe it.

Activity

  1. groupsky commented on Nov 19, 2026

    @groupsky
    ContributorAuthor

    Done in #47 (squash commit 91404b8).

    • The broken state with the behaviour above. Tests in internal/gate/broken_test.go: TestFailedRollbackIsBroken (failed apply + failed rollback → broken, no pause file, one message, the failed and the old commit are not asked again, pullgate status shows broken: since …, last error: …), TestBrokenFixForward (a newer commit is asked with the warning; a good fix forward ends it), TestBrokenBadFixForwardStaysBroken (no rollback, stays broken, the next newer commit is asked), TestBrokenRefusesNonDescendant, TestBrokenTapRefusesNonFixForward, TestBrokenAsksInAutoMode, TestBrokenDropsOlderQuestions, TestRootResumeEndsBroken.
    • The question's warning and the notice show the reason, cut in the middle so its end stays (message.Reason, message.BrokenReason; the checks in TestBrokenFixForward and TestBrokenBadFixForwardStaysBroken).
    • DESIGN.md sections 10, 12 and 13 and THREAT-MODEL.md T6 describe it, and the README too.

    Note: the limited /resume (#33) must not end the broken state. Only a good fix forward or root's pullgate status NAME --resume ends it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions