Skip to content

Commit f0899f7

Browse files
feat(networking): add tenant agent isolation NetworkPolicy
Implements Phase 1 of the DevOps roadmap: zero-trust network isolation for agent pods across tenant namespaces. Changes: - internal/controller/agentdeployment_controller.go: Add 'agentrax.io/agent: true' label to agentLabels(). This label acts as the NetworkPolicy pod selector key. Safe for existing Deployments because reconcileDeployment() only writes spec.selector on creation (ResourceVersion == ''). - config/network-policy/tenant-agent-isolation.yaml: New NetworkPolicy targeting pods with agentrax.io/agent=true. Default-denies all ingress/egress, then allows: - Ingress: Prometheus scrape on port 8080 from namespaces labelled monitoring=enabled - Egress: kube-apiserver port 6443, CoreDNS port 53 UDP+TCP - config/network-policy/kustomization.yaml: Add new manifest to the network-policy Kustomize component. - config/default/kustomization.yaml: Uncomment the network-policy component so it is included in the default overlay. - docs/networking/README.md: Two-tier policy model documentation, traffic diagrams, and per-tenant application instructions. Verified: make test -> all packages pass (controller: 71.8%) make lint -> 0 errors make manifests -> 0 errors helm lint -> 0 failures YAML validate -> NetworkPolicy schema correct
1 parent 168e93d commit f0899f7

5 files changed

Lines changed: 208 additions & 1 deletion

File tree

config/default/kustomization.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ resources:
3131
# Only Pod(s) running a namespace labeled with 'metrics: enabled' will be able to gather the metrics.
3232
# Only CR(s) which requires webhooks and are applied on namespaces labeled with 'webhooks: enabled' will
3333
# be able to communicate with the Webhook Server.
34-
#- ../network-policy
34+
- ../network-policy
3535

3636
# Uncomment the patches line if you enable Metrics, and/or are using webhooks and cert-manager
3737
patches:
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
resources:
22
- allow-metrics-traffic.yaml
3+
- tenant-agent-isolation.yaml
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
---
2+
# Tenant Agent Isolation NetworkPolicy
3+
#
4+
# Purpose: Restricts agent pods across tenant namespaces, enforcing zero-trust
5+
# isolation between tenants and preventing unauthorized outbound traffic.
6+
#
7+
# Selector: Matches all pods labelled `agentrax.io/agent: "true"`.
8+
# The AgentDeployment reconciler sets this label on every pod template
9+
# it manages, so this policy applies to all managed agent pods.
10+
#
11+
# Ingress rules:
12+
# - Allow Prometheus to scrape metrics on port 8080 from namespaces
13+
# labelled `monitoring: enabled` (kube-prometheus-stack namespace).
14+
#
15+
# Egress rules:
16+
# - Allow egress to kube-apiserver on port 6443 (required for agent-to-API
17+
# communication and tool-calling via the Kubernetes API).
18+
# - Allow CoreDNS lookups on port 53 (UDP and TCP) for service discovery
19+
# within the cluster.
20+
# - All other egress (internet, cross-tenant) is denied by default.
21+
#
22+
# Usage: Apply this manifest to every tenant namespace:
23+
# kubectl apply -n tenant-<name> -f tenant-agent-isolation.yaml
24+
#
25+
# Note: This policy does NOT apply to the agentrax-system namespace (operator pods).
26+
# The operator namespace is protected by allow-metrics-traffic.yaml.
27+
apiVersion: networking.k8s.io/v1
28+
kind: NetworkPolicy
29+
metadata:
30+
name: tenant-agent-isolation
31+
labels:
32+
app.kubernetes.io/name: agentrax
33+
app.kubernetes.io/managed-by: kustomize
34+
spec:
35+
# Select all pods carrying the agentrax.io/agent=true label.
36+
# This label is set by agentLabels() in the AgentDeployment reconciler.
37+
podSelector:
38+
matchLabels:
39+
agentrax.io/agent: "true"
40+
policyTypes:
41+
- Ingress
42+
- Egress
43+
ingress:
44+
# Allow Prometheus to scrape /metrics on port 8080.
45+
# Prometheus Operator runs in a namespace labelled `monitoring: enabled`.
46+
- from:
47+
- namespaceSelector:
48+
matchLabels:
49+
monitoring: enabled
50+
ports:
51+
- port: 8080
52+
protocol: TCP
53+
egress:
54+
# Allow outbound to kube-apiserver on port 6443.
55+
# Agents may call the Kubernetes API to discover services or use cluster tools.
56+
- ports:
57+
- port: 6443
58+
protocol: TCP
59+
# Allow CoreDNS resolution on UDP and TCP port 53.
60+
# Without this, service name lookups fail and MCP tool endpoints are unreachable.
61+
- ports:
62+
- port: 53
63+
protocol: UDP
64+
- port: 53
65+
protocol: TCP

docs/networking/README.md

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
# Agentrax — Tenant Network Isolation
2+
3+
This document explains the two-tier network policy model shipped with Agentrax
4+
and how platform operators apply it to tenant namespaces.
5+
6+
## Overview
7+
8+
Agentrax uses Kubernetes `NetworkPolicy` to enforce a **zero-trust perimeter**
9+
around all agent pods. This prevents a compromised or misbehaving agent in one
10+
tenant from reaching another tenant's services, the operator control plane, or
11+
arbitrary internet destinations.
12+
13+
Two policies are maintained:
14+
15+
| Policy File | Namespace | Purpose |
16+
| ----------------------------- | -------------------------- | -------------------------------------------------------------------------- |
17+
| `allow-metrics-traffic.yaml` | `agentrax-system` | Allows Prometheus to scrape the operator `/metrics` endpoint |
18+
| `tenant-agent-isolation.yaml` | Every `tenant-*` namespace | Isolates agent pods — restricts all ingress/egress to the minimum required |
19+
20+
## How the Label Selector Works
21+
22+
The `tenant-agent-isolation` policy uses `podSelector.matchLabels`:
23+
24+
```yaml
25+
podSelector:
26+
matchLabels:
27+
agentrax.io/agent: "true"
28+
```
29+
30+
The `AgentDeployment` reconciler (`internal/controller/agentdeployment_controller.go`)
31+
stamps this label onto every agent `Deployment`'s pod template via `agentLabels()`.
32+
No manual labelling is needed — all agent pods are automatically covered.
33+
34+
## Traffic Model
35+
36+
```
37+
┌─────────────────────────────────────────────────────┐
38+
│ tenant-finance namespace │
39+
│ │
40+
[Agent Pod] agentrax.io/agent=true │
41+
│ │ │
42+
│ ├─ Ingress ← port 8080 ← [Prometheus]
43+
│ │ (monitoring namespace only) │
44+
│ │ │
45+
│ ├─ Egress → port 6443 → [kube-apiserver]
46+
│ ├─ Egress → port 53 → [CoreDNS]
47+
│ │ │
48+
│ └─ ALL OTHER TRAFFIC: BLOCKED │
49+
└─────────────────────────────────────────────────────┘
50+
```
51+
52+
## Applying the Policy to Tenant Namespaces
53+
54+
The `tenant-agent-isolation.yaml` NetworkPolicy must be applied to each tenant
55+
namespace. The policy is **not** automatically applied by the operator — it is
56+
applied once by a platform admin when provisioning a tenant namespace.
57+
58+
### Apply Manually
59+
60+
```bash
61+
# Apply to a specific tenant namespace:
62+
kubectl apply -n tenant-finance \
63+
-f config/network-policy/tenant-agent-isolation.yaml
64+
65+
kubectl apply -n tenant-marketing \
66+
-f config/network-policy/tenant-agent-isolation.yaml
67+
```
68+
69+
### Apply via Kustomize (Development)
70+
71+
The default Kustomize overlay applies both network policies to the `agentrax-system`
72+
namespace for development/testing. The `tenant-agent-isolation` policy in this
73+
context validates the manifest schema; in production it must be applied per tenant
74+
namespace as above.
75+
76+
```bash
77+
kubectl apply -k config/default/
78+
```
79+
80+
### Apply via Helm (Recommended for Production)
81+
82+
When installing via Helm, set `networkPolicy.enabled: true` (Phase 1 Helm
83+
integration — coming in a future release):
84+
85+
```bash
86+
helm upgrade --install agentrax charts/agentrax/ \
87+
--set networkPolicy.enabled=true
88+
```
89+
90+
## Labelling the Prometheus Namespace
91+
92+
The ingress rule allows traffic from namespaces labelled `monitoring: enabled`.
93+
Apply this label to the namespace where Prometheus Operator / kube-prometheus-stack
94+
is installed:
95+
96+
```bash
97+
kubectl label namespace monitoring monitoring=enabled
98+
# Or, if using the default kube-prometheus-stack namespace name:
99+
kubectl label namespace monitoring monitoring=enabled
100+
```
101+
102+
## Required CNI Support
103+
104+
This NetworkPolicy relies on a Container Network Interface (CNI) plugin that
105+
**enforces** `NetworkPolicy` objects. Verify your CNI supports this:
106+
107+
| Environment | Supported CNI |
108+
| ---------------- | ----------------------------- |
109+
| Kind (local dev) | Kindnet (default) ✅ |
110+
| Azure AKS | Azure CNI or Calico ✅ |
111+
| AWS EKS | VPC CNI + Calico or Cilium ✅ |
112+
| GKE | Dataplane V2 (Cilium) ✅ |
113+
114+
> **Note**: Flannel does **not** enforce NetworkPolicy by default. Use Calico or
115+
> Cilium as a replacement CNI if Flannel is your cluster default.
116+
117+
## Verifying the Policy
118+
119+
After applying, verify that the policy is active and that an agent pod has
120+
the correct label:
121+
122+
```bash
123+
# Confirm agent pod has the isolation label:
124+
kubectl get pods -n tenant-finance -L agentrax.io/agent
125+
126+
# Confirm the NetworkPolicy is present:
127+
kubectl get networkpolicy -n tenant-finance
128+
129+
# Test that cross-tenant traffic is blocked (from within an agent pod):
130+
kubectl exec -n tenant-finance <agent-pod> -- \
131+
curl --connect-timeout 2 http://<service-in-tenant-marketing>
132+
# Expected: connection timed out (blocked)
133+
134+
# Test that Kubernetes API access is allowed:
135+
kubectl exec -n tenant-finance <agent-pod> -- \
136+
curl -k https://kubernetes.default.svc:443/healthz
137+
# Expected: "ok"
138+
```

internal/controller/agentdeployment_controller.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -659,12 +659,15 @@ func (r *AgentDeploymentReconciler) reconcileMCPRegistration(ctx context.Context
659659

660660
// agentLabels returns the canonical label set applied to all resources owned by ad.
661661
// For stable resources (Deployment, Service), this includes variant=stable.
662+
// The agentrax.io/agent label is the NetworkPolicy selector key — all agent pod
663+
// templates carry it so the tenant-agent-isolation policy applies automatically.
662664
func agentLabels(ad *agentraxv1alpha1.AgentDeployment) map[string]string {
663665
return map[string]string{
664666
"app.kubernetes.io/name": ad.Name,
665667
"app.kubernetes.io/managed-by": "agentrax",
666668
"agentrax.io/tenant": ad.Spec.TenantRef,
667669
"agentrax.io/variant": "stable",
670+
"agentrax.io/agent": "true",
668671
}
669672
}
670673

0 commit comments

Comments
 (0)