Gatewayapi Namespaced Mode by radixo · Pull Request #4690 · tigera/operator

radixo · 2026-04-14T18:03:23Z

Description

Replace the previous Gateway-API install — which ran an Envoy Gateway controller in tigera-gateway and deployed all proxy workloads in that same namespace — with a single envoy-gateway controller in calico-system running with deploy.type=GatewayNamespace, so proxy workloads land in each Gateway's own namespace.

This is a breaking change for clusters running the legacy install. Existing Gateway CRs do not need edits — tigera-gateway-class and its controllerName are preserved — but proxy Pods, their Services and LoadBalancer addresses are recreated in each Gateway's own namespace on first reconcile after upgrade. Anything pinned to tigera-gateway (NetworkPolicies, monitoring, RBAC, external DNS) must follow.

Single controller in calico-system with controllerName=gateway.envoyproxy.io/gatewayclass-controller (chart default) and deploy.type=GatewayNamespace. ControllerName + GatewayClass name are deliberately reused from the legacy install so existing Gateway CRs continue to be claimed.
Auto-provisioned tigera-gateway-class GatewayClass + EnvoyProxy. Users can declare additional classes via GatewayAPI.Spec.GatewayClasses; all classes target the single controller.
Embed gateway-helm.tgz and render at runtime via the Helm SDK; result is cached per process via sync.Once. Replaces the previous pre-rendered YAML.
Per-namespace Enterprise resources for each Gateway-hosting namespace: waf-http-filter SA + per-namespace waf-http-filter-gateway-resources RoleBinding (least-privilege Gateway-API reads), tigera-operator-secrets RoleBinding, tigera-pull-secret copy. Cluster-scoped perms (licensekeys, tokenreviews) go through a single shared waf-http-filter-gateway-namespaces ClusterRoleBinding whose Subjects list is recomputed each reconcile.
Reserved-namespace guard (calico-system, tigera-operator): the operator does not create or delete the shared tigera-operator-secrets RoleBinding or tigera-pull-secret copy in those namespaces — the core Installation controller owns them.
Per-namespace cleanup when a namespace no longer hosts a Gateway: pull Secret → SA + RoleBinding → tigera-operator-secrets RoleBinding. That RoleBinding is what grants the operator Secret-delete perms, so reversing the order yields a 403 and aborts the reconcile.
Explicit upgrade-cleanup queue for the legacy tigera-gateway install: controller Deployment/Service/SAs/ConfigMap, certgen Job + RBAC, namespaced Role/RoleBinding, copied pull Secrets, tigera-operator-secrets RoleBinding, envoy-gateway-topology-injector.tigera-gateway MWC, the orphaned waf-http-filter-cluster-scoped and waf-http-filter-gateway-resources ClusterRoleBindings, and the deprecated combined waf-http-filter ClusterRole/ClusterRoleBinding. Pull Secrets are queued before tigera-operator-secrets. The tigera-gateway Namespace itself is intentionally not queued — users may have placed their own resources in it.
v3 NetworkPolicy (calico-system.envoy-gateway) under the calico-system tier to keep the controller + certgen Job working under default-deny. Selector covers both app.kubernetes.io/name=gateway-helm and app=certgen (the chart applies different labels to the certgen Job vs its pod template). Egress: DNS + kube-apiserver, then Pass. Ingress: 9443 (topology-injector webhook), 18000-18002 (xDS), 19001 (metrics).
Tests: UT coverage for OSS + Enterprise renders, GatewayClass + EnvoyProxy customisation, per-namespace lifecycle (create + cleanup), reserved-namespace guards, Secret-before-RoleBinding ordering, and the explicit legacy-cleanup queue (with the tigera-gateway Namespace asserted not in the delete list). FV coverage: deploys the controller in calico-system and asserts nothing lands in tigera-gateway, provisions and cleans up per-namespace resources, GatewayClass + EnvoyProxy cleanup, custom EnvoyProxy watch, l7-log-collector owning-gateway env wiring, custom EnvoyGateway ConfigMap.

Security

The Enterprise per-namespace render copies tigera-pull-secret into every namespace that hosts a Gateway, so permissive RBAC on those namespaces can expose the pull secret. Reserved namespaces are excluded from create and delete of the shared resources, so the operator does not clobber core-Installation-owned secrets.

Upgrade / compatibility

In-place upgrade. The single controller in calico-system claims all tigera-gateway-class Gateways unchanged. Proxy Pods, their Services, and LoadBalancer addresses are recreated in each Gateway's own namespace on first reconcile, so any cluster setting pinned to tigera-gateway (NetworkPolicies, monitoring, RBAC, external DNS) must be repointed to the Gateway's own namespace. The legacy controller install is removed automatically; the tigera-gateway Namespace itself is preserved in case it holds user resources.

Calico-private operator RBAC update required: envoy-gateway-topology-injector.calico-system added to the mutatingwebhookconfigurations resourceNames list with update and delete verbs (the legacy .tigera-gateway entry is retained with delete so the upgrade-cleanup can reap it).

Release Note

**Breaking change.** The Calico Ingress Gateway controller now runs in
`calico-system` and provisions proxy workloads in the Gateway's own namespace
(`deploy.type=GatewayNamespace`). Existing `tigera-gateway-class` Gateways
keep working without CR edits, but their proxy Pods are relaunched in the
Gateway's own namespace; any NetworkPolicy, monitoring, RBAC or external DNS
previously pinned to `tigera-gateway` must be repointed to the Gateway's own
namespace. The legacy controller install is removed automatically on upgrade;
the `tigera-gateway` Namespace itself is left in place in case it holds user
resources.

For PR author

Tests for change.
If changing pkg/apis/, run make gen-files
If changing versions, run make gen-versions

For PR reviewers

A note for code reviewers - all pull requests must have the following:

Milestone set according to targeted release.
Appropriate labels:
- kind/bug if this is a bugfix.
- kind/enhancement if this is a a new feature.
- enterprise if this PR applies to Calico Enterprise only.

electricjesus

Good work on the runtime Helm rendering migration — dropping 53K lines of pre-rendered YAML is a huge win. The GatewayNamespace mode looks solid with good test coverage. A few observations below.

electricjesus · 2026-04-15T12:55:47Z

 		CurrentGatewayClasses: set.New[string](),
 	}

+	if gatewayAPI.Spec.GatewayDeploymentMode == nil {


Nit: The CRD already has +kubebuilder:default=ControllerNamespace, so any persisted GatewayAPI resource will have this field populated by the API server. This runtime defaulting only matters for in-memory objects that were never persisted (tests?). Not a problem, just noting the redundancy — if the CRD default is the source of truth, a comment here explaining why you also default in code would help future readers.

It is used by the tests

radixo · 2026-04-15T15:51:17Z

Good work on the runtime Helm rendering migration — dropping 53K lines of pre-rendered YAML is a huge win. The GatewayNamespace mode looks solid with good test coverage. A few observations below.

All good catches man, all sorted

electricjesus

LGTM!

caseydavenport · 2026-04-20T22:33:37Z

+	// Gateway resources using operator-managed GatewayClasses. These namespaces need
+	// per-namespace Enterprise resources (SA, CRB, pull secrets).
+	if *gatewayAPI.Spec.GatewayDeploymentMode == operatorv1.GatewayDeploymentModeGatewayNamespace &&
+		variant.IsEnterprise() {


Why does the Variant matter here at all?

For resources that are rendered only on EE license, like WAF.

- Swap the checked-in gateway_api_resources.yaml for the embedded gateway-helm.tgz rendered via the helm SDK at startup; K8SGatewayAPICRDs/GatewayAPICRDs now take a runtime.Scheme and return an error (istio_controller updated for the new signature) - Deploy two envoy-gateway controllers: legacy in tigera-gateway (user-declared classes via Spec.GatewayClasses) and a new one in calico-system with deploy.type=GatewayNamespace; auto-provision the tigera-gateway-class-ns GatewayClass bound to the new controller - Group the tigera-gateway install behind legacyObjects/legacyTeardownObjects so the eventual deprecation is a single delete - HasLegacyGateways classifier in the controller: build a className -> controllerName map seeded from Spec.GatewayClasses + existing GatewayClass resources, classify every live Gateway; when no Gateway targets the tigera-gateway controller, the install is torn down; during the teardown-then-redeploy race the legacy render is deferred to avoid a "Namespace is terminating, skipping creation" log flood - Legacy teardown queues only the Namespace + cluster-scoped objects + the Deployment (for status.RemoveDeployments); in-namespace RBAC/Secrets ride the cascade to avoid the tigera-operator-secrets RoleBinding race - Move the shared waf-http-filter ClusterRoles out of the legacy bundle so the calico-system-side proxies keep their cluster-scoped perms after tigera-gateway is retired - Per-namespace Enterprise resources (SA, RoleBindings, pull secret, shared CRB subject) for namespaces hosting a namespaced-class Gateway; reserved namespaces skip shared resource create/delete; Secret goes before RoleBinding on cleanup to avoid 403 - Gate v3 NetworkPolicies on the calico-system Tier; render calico-system.envoy-gateway allow for the controller and certgen - Update unit tests and Makefile/docs accordingly Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

- Cover the calico-system envoy-gateway controller lifecycle, per-namespace resource provisioning and cleanup, custom EnvoyProxy and EnvoyGateway ConfigMap watches, owning-gateway env vars in l7-log-collector, and the legacy-class teardown path - Teardown sequencing for tigera-gateway cascading Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…lico-system - Render one envoy-gateway controller in calico-system with deploy.type=GatewayNamespace - Auto-provision tigera-gateway-class; honour user overrides if redeclared in Spec.GatewayClasses - Enumerate every operator-owned object from the legacy tigera-gateway install for cleanup (pull Secrets before tigera-operator-secrets); keep the Namespace itself in case users placed their own resources there - Point GatewayAPI finalizer at the calico-system envoy-gateway Deployment - Drop dual-controller fixtures and the legacy-undeploy test; consolidate FV tests to the calico-system layout

Upstream envoy-gateway rejects the combination of mergeGateways: true and GatewayNamespaceMode, so any user-supplied EnvoyProxy with merging enabled would cause its referenced Gateways to silently stop being programmed after the switch to GatewayNamespace (https://gateway.envoyproxy.io/docs/tasks/operations/gateway-namespace-mode/). In the GatewayAPI reconciler, when a Spec.GatewayClasses[].EnvoyProxyRef points at an EnvoyProxy with Spec.MergeGateways == true, force the field to false in our managed copy and log a warning naming the EnvoyProxy and GatewayClass. The user's source CR is not mutated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

- remove controllerName param (never set by callers) - inline ReleaseName and GatewayNamespace deploy type - add DeploymentNamespace constant for the install namespace - drop now-unused helmGateway type

…nt-enterprise

- parseManifest now errors on kinds it doesn't recognize so a chart bump that emits a new kind trips the existing render tests

Bug introduced in this branch; reverts the render and UT to master's behavior. - drop the render-side auto-provision of tigera-gateway-class - flip the UT that asserted the buggy output

Under deploy.type=GatewayNamespace (tigera#4690), envoy-proxy pods land in the Gateway's own namespace and mount the operator trust bundle at /etc/pki/tls/certs (added by tigera#4796). The mount references a ConfigMap in the proxy pod's own namespace, but tigera#4796 only writes the ConfigMap into calico-system (the controller's namespace), so the proxy Pod stops at Init:0/2 with: Warning FailedMount MountVolume.SetUp failed for volume "tigera-ca-bundle": configmap not found Mirror the trust bundle into each Gateway namespace alongside the existing per-namespace propagation of tigera-pull-secret and the waf-http-filter SA / RoleBindings. Reuses the existing reserved-NS guard and follows the same delete-before-RoleBinding ordering as the pull-secret cleanup. Reproduced live on seth-ez-a3b5 2026-05-19 with operator walter-merge-2026-05-18 (has both tigera#4690 and tigera#4796): fresh Gateway namespace -> everything else propagates but tigera-ca-bundle does not, proxy Pod stuck Init:0/2. Brief: tigera/gateway-extensions-controller/docs/planning/briefs/2026-05-19-ca-bundle-propagation-brief.md

Walter-supplied positive test: configure two Gateway namespaces ("default" and "app-ns") with a TrustedBundle, render, assert the trust bundle ConfigMap (TrustedCertConfigMapName) lands in each Gateway namespace. Companion to the per-NS ConfigMap copy added in the previous commit.

rene-dekker · 2026-05-21T16:56:55Z

+				{
+					Action:   v3.Allow,
+					Protocol: &networkpolicy.TCPProtocol,
+					Source:   v3.EntityRule{Nets: []string{"0.0.0.0/0"}},


This wouldn't work on ipv6, should we add "::/0"?

rene-dekker · 2026-05-21T17:43:34Z

+}
+
+// gatewayAPIControllerPolicy allows the controller + certgen to reach kube-apiserver and DNS.
+func gatewayAPIControllerPolicy(namespace string, openShift bool) *v3.NetworkPolicy {


The labels on the deployment are taken from helm. And so if they decide to change something upstream, our network policy may start to block traffic unnoticed by us. There also seems to be a mismatch between the Deploy label and the pod label. Is it possible that we add our own labels to the deploy and pod, and use these for the network policy as well? Maybe calico-gateway-controller?

rene-dekker · 2026-05-21T17:48:53Z

 		}
 	}

+	// Start watching Gateway resources now that the CRDs are in place, so future


Should we add a check here to disallow gateways with namespace tigera-gateway? If they specify this namespace, will there be issues with the clean up and creation of resources?

- add ::/0 to controller NetworkPolicy ingress for dual-stack / IPv6-only - switch policy selector to our own k8s-app labels (controller + certgen) so a chart-side label rename can't silently break it - legacyTeardownObjects now takes the current create-set and skips any candidate it'd otherwise delete (e.g. user Gateway in tigera-gateway)

radixo requested a review from a team as a code owner April 14, 2026 18:03

marvin-tigera added this to the v1.43.0 milestone Apr 14, 2026

marvin-tigera added docs-pr-required release-note-required labels Apr 14, 2026

electricjesus reviewed Apr 15, 2026

View reviewed changes

electricjesus reviewed Apr 16, 2026

View reviewed changes

Comment thread pkg/render/gatewayapi/gateway_api.go Outdated

electricjesus approved these changes Apr 20, 2026

View reviewed changes

caseydavenport reviewed Apr 20, 2026

View reviewed changes

radixo force-pushed the gatewayapi-deployment-enterprise branch from 04d49c6 to 8c6f64c Compare April 23, 2026 21:22

radixo and others added 3 commits May 7, 2026 17:31

radixo force-pushed the gatewayapi-deployment-enterprise branch from 0d63b8f to d3ef961 Compare May 8, 2026 17:05

rene-dekker reviewed May 15, 2026

View reviewed changes

Comment thread pkg/render/gatewayapi/gateway_api.go Outdated

rene-dekker reviewed May 15, 2026

View reviewed changes

Comment thread pkg/render/gatewayapi/gateway_api.go Outdated

radixo and others added 5 commits May 18, 2026 11:32

Simplify renderChart: drop unused params

acb989f

- remove controllerName param (never set by callers) - inline ReleaseName and GatewayNamespace deploy type - add DeploymentNamespace constant for the install namespace - drop now-unused helmGateway type

Merge remote-tracking branch 'origin/master' into gatewayapi-deployme…

e5c429f

…nt-enterprise

gatewayapi: error on unknown kinds from helm chart

c57e513

- parseManifest now errors on kinds it doesn't recognize so a chart bump that emits a new kind trips the existing render tests

gatewayapi: fix double-rendering of the default GatewayClass

e581acb

Bug introduced in this branch; reverts the render and UT to master's behavior. - drop the render-side auto-provision of tigera-gateway-class - flip the UT that asserted the buggy output

This was referenced May 19, 2026

feat(applicationlayer): WAF v3 render + kube-controllers config plumbing #4821

Draft

feat(applicationlayer): WAF v3 render + kube-controllers config plumbing #4779

Draft

feat(gatewayapi): mirror tigera-ca-bundle into each Gateway namespace #4822

Closed

electricjesus added 2 commits May 19, 2026 13:49

electricjesus mentioned this pull request May 19, 2026

PMREQ-384: deprecate waf-http-filter ext_proc sidecar electricjesus/operator#3

Draft

5 tasks

rene-dekker reviewed May 21, 2026

View reviewed changes

Conversation

radixo commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Security

Upgrade / compatibility

Release Note

For PR author

For PR reviewers

Uh oh!

electricjesus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

electricjesus Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

radixo Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

radixo commented Apr 15, 2026

Uh oh!

Uh oh!

electricjesus left a comment

Choose a reason for hiding this comment

Uh oh!

caseydavenport Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

radixo May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rene-dekker May 21, 2026

Choose a reason for hiding this comment

Uh oh!

rene-dekker May 21, 2026

Choose a reason for hiding this comment

Uh oh!

rene-dekker May 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

radixo commented Apr 14, 2026 •

edited

Loading