Skip to content

feat(operator): the chart runs two operator pods, on different nodes where it can - #673

Merged
bmscomp merged 1 commit into
mainfrom
feat/operator-two-replicas
Nov 15, 2026
Merged

bmscomp merged 1 commit into
mainfrom
feat/operator-two-replicas

Conversation

@bmscomp

@bmscomp bmscomp commented Nov 15, 2026

Copy link
Copy Markdown
Owner

The first step toward a mature operator: with `replicas: 1`, a lost node holds every `Kates` resource until a new pod is scheduled and started.

  • `replicas: 2` by default. One pod holds the Lease, and the other takes over when the first or its node is lost.
  • With `affinity` empty and more than one replica, the pods prefer one node each (preferred anti-affinity on `kubernetes.io/hostname`), so single-node CI still installs.
  • The CI failure dump reads every operator pod's log (`-l … --prefix`), not one picked at random.
  • README updated.

The controllers and the startup cleanup run only after a pod takes the Lease (`run.rs`), so the standby does nothing.

Proof

On a three-node Kind cluster, both pods ran on different nodes. With the Lease holder force-deleted, a new pod held the Lease 13 s later. In that run the freshly created replacement won the race, not the standby; on a real node loss the replacement can't start that fast. `helm lint` passes; `helm template --set replicas=1` renders no affinity.

…where it can

With one pod, a lost node holds every Kates resource until a new pod is
scheduled and started. With two, the standby takes the Lease within 5 to 15
seconds. The pods prefer one node each unless affinity is set. CI's failure
dump reads every operator pod's log, not one of them.
@bmscomp
bmscomp merged commit 24a2837 into main Nov 15, 2026
62 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant