K8SPG-1057 refactor dcs

[yoav-katz]

As part of K8SPG-1057, Patroni's DCS logic is hard-coded to Kubernetes Endpoints throughout internal/patroni and internal/controller/postgrescluster - config generation, RBAC, Service reconciliation, and status observation all directly assume "Kubernetes DCS," with no seam between generic Patroni logic and DCS-specific behavior.

Cause:
K8SPG-1057 - we want to add support for an additional DCS backend (etcd) in a follow-up stacked PR, but the current code has no ownership boundary to build that on without scattering if-branches for the new backend across the reconciler.

Solution:
Introduce a new package:

internal/patroni/dcs

This package becomes the single place that knows about specific DCS implementations.
It exposes a Backend interface. This PR ships one implementation, migrating the existing Kubernetes-Endpoints behavior into it with no functional change:

internal/patroni/dcs/kubernetes_endpoints.go

A second implementation (etcd) is out of scope here and lands as a follow-up stacked PR on top of this one.
The backend is selected once per reconcile: backend := dcs.For(cluster)
After that, callers interact only with backend capabilities and do not know which DCS is active.

Backend-owned behavior migrated in this PR (Kubernetes Endpoints only; etcd column shows the target shape for the follow-up):

• Patroni configuration additions
• Kubernetes: kubernetes DCS section
• etcd (follow-up): etcd3 DCS section and related callbacks
• Pod additions
• Kubernetes: PATRONI_KUBERNETES_* environment variables
• etcd (follow-up): TLS mounts and PATRONI_ETCD3_* configuration
• RBAC requirements
• Kubernetes: Endpoint-related permissions
• etcd (follow-up): no Kubernetes DCS permissions
• Service behavior
• Kubernetes: Patroni-managed leader/config Services, and primary-Service routing
• etcd (follow-up): no DCS Services
• Runtime observations
• Kubernetes: Endpoints annotation read (Observe)
• etcd (follow-up): Patroni REST API and etcd-backed state

Design rules

1. No DCS branching outside internal/patroni/dcs.
   Callers request capabilities (dcs.For(cluster).X(...)) and handle results generically - including the primary-Service routing seam (PrimaryService), which previously would have forced DCS-specific branching back into the controller.
2. Backend implementations are stateless. Dependencies (client, executor, event recorder) are passed as parameters, not stored.
3. The interface evolved during implementation as real gaps surfaced - e.g. Observe returns a small Observation{SystemIdentifier, RequeueAfter} struct rather than a bare string, and PrimaryService was added after review caught that the primary-Service routing logic hadn't actually been moved out of the controller.

Import direction
controller → patroni
controller → dcs → patroni

patroni does not import dcs. The controller wires the two together by passing backend-provided data (dcsYAML, dcsEnvVars, etc.) into existing patroni helpers.

No behavior change. Verified via: full build/vet clean, complete unit + envtest suite passing (internal/patroni, internal/patroni/dcs, internal/controller/postgrescluster), and make generate-rbac producing a zero diff against the generated ClusterRole.

CHECKLIST

Jira

• Is the Jira ticket created and referenced properly?
• Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
• Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

• Is an E2E test/test case added for the new feature/change? - Not added. This is a behavior-preserving refactor (no new feature, no CRD/API change), so existing e2e/kuttl coverage should catch any regression; no new e2e scenario is needed. Recommend running the existing suite once before merge if it isn't already gated in CI.
• Are unit tests added where appropriate? - Yes: internal/patroni/dcs/kubernetes_endpoints_test.go added, covering every Backend method (ClusterYAML, InstanceYAML, InstanceEnvVars, Permissions, DistributedConfigurationService, LeaderLeaseService, PrimaryService, Observe, Delete); existing tests in internal/patroni and internal/controller/postgrescluster updated for the new signatures and re-verified passing.

Config/Logging/Testability

• Are all needed new/changed options added to default YAML files? - N/A, no new user-facing options; no CRD field added in this PR.
• Are all needed new/changed options added to the Helm Chart? - N/A, same reason.
• Did we add proper logging messages for operator actions? - No new logging needed; the one existing log line ("detected ready instance but no initialize value") was preserved as-is, just relocated into kubernetesEndpointsBackend.Observe.
• Did we ensure compatibility with the previous version or cluster upgrade process? - Yes, by design: no CRD/API changes, identical generated RBAC (make generate-rbac diff-clean), and identical runtime behavior confirmed by the full test suite.
• Does the change support oldest and newest supported PG version? - N/A, this touches Patroni/DCS wiring only; no PG-version-conditional logic was added or changed (existing CompareVersion checks were preserved untouched).
• Does the change support oldest and newest supported Kubernetes version? - N/A, no Kubernetes API version-specific behavior touched; same corev1.Service/Endpoints shapes as before.

[yoav-katz]

the lint failure - this is the known Endpoints deprecation warning (Kubernetes recommends EndpointSlice going forward), not something introduced by this PR. @egegunes confirmed it's acceptable to leave as-is for now; tracked separately in K8SPG-866.

I did name the dcs backend as kubernetes_endpoints so in the feature it will be easy to add kubernetes_configmap if wanted.

[JNKPercona]

Test Name	Result	Time
backup-enable-disable	passed	00:13:05
builtin-extensions	passed	00:06:28
custom-extensions	failure	00:05:51
cert-manager-tls	passed	00:13:05
cert-management-policy	passed	00:02:34
custom-envs	passed	00:18:34
custom-tls	failure	00:05:24
database-init-sql	passed	00:02:35
demand-backup	passed	00:27:54
demand-backup-offline-snapshot	passed	00:13:40
dynamic-configuration	passed	00:04:19
extra-volumes	passed	00:04:05
finalizers	failure	00:02:43
init-deploy	passed	00:03:47
huge-pages	passed	00:02:54
major-upgrade-14-to-15	passed	00:11:39
major-upgrade-15-to-16	passed	00:11:11
major-upgrade-16-to-17	passed	00:09:36
major-upgrade-17-to-18	passed	00:12:03
ldap	passed	00:06:53
ldap-tls	passed	00:07:27
logcollection	passed	00:06:50
monitoring	passed	00:09:06
one-pod	passed	00:06:04
repo-host-autogrow	passed	00:02:01
operator-self-healing	passed	00:10:36
pgbouncer-mtls	passed	00:03:32
pg-tde	passed	00:12:17
pitr	passed	00:12:28
scaling	passed	00:05:27
scheduled-backup	passed	00:27:08
self-healing	passed	00:09:51
sidecars	passed	00:02:58
standby-pgbackrest	passed	00:19:35
standby-streaming	passed	00:14:19
start-from-backup	passed	00:11:17
tablespaces	passed	00:07:10
telemetry-transfer	passed	00:04:39
upgrade-consistency	passed	00:06:28
upgrade-minor	passed	00:06:36
users	passed	00:04:49
migration-from-crunchy-standby	passed	00:19:19
migration-from-crunchy-pv	passed	00:07:19
migration-from-crunchy-backup-restore	failure	00:08:01

Summary	Value
Tests Run	44/44
Job Duration	02:38:01
Total Test Time	06:43:57

commit: 3f15a9d
image: perconalab/percona-postgresql-operator:PR-1727-3f15a9d57

[egegunes]

honestly i don't think i like calling dcs.For(cluster).<interfaceMethod> everywhere. i think if functions accept interface as a parameter and call its method it'd be much easier to reason about. but i don't have very strong opinion on this.

@mayankshah1607 please take a look when you have the chance