Skip to content

Add multi-NIC support to EgressGateway#4852

Draft
fasaxc wants to merge 1 commit into
tigera:masterfrom
fasaxc:egw-multi-nic
Draft

Add multi-NIC support to EgressGateway#4852
fasaxc wants to merge 1 commit into
tigera:masterfrom
fasaxc:egw-multi-nic

Conversation

@fasaxc
Copy link
Copy Markdown
Member

@fasaxc fasaxc commented May 22, 2026

Summary

  • Adds spec.network to EgressGateway so the primary pod interface can attach to a named Calico v3.Network (VRF today, L2 incoming) via the cni.projectcalico.org/networks annotation. Mutually exclusive with the (deprecating) spec.externalNetworks, enforced with a CEL rule.
  • Adds spec.additionalInterfaces[] (max 9) to attach secondary NICs through Multus NetworkAttachmentDefinition references. Each entry has a deterministic pod-side interface name plus an attachment union (only multus for now; comment block reserves space for a native Calico-Network arm later).
  • Controller discovers the NAD CRD at startup, conditionally watches NADs, and refuses to render additionalInterfaces unless Installation.spec.calicoNetwork.multiInterfaceMode=Multus and the NAD CRD is installed. Missing referenced NADs are a soft warning so NAD/EGW creation order doesn't fail reconcile.
  • Render emits k8s.v1.cni.cncf.io/networks JSON for the secondary NICs and cni.projectcalico.org/networks for the primary's Network.

Design discussion / decisions captured in: `~/.claude/plans/write-up-the-plan-recursive-octopus.md`

Status

Draft. Open items I'd appreciate review on before un-drafting:

  • Per-secondary ipPools wire path. The API surface accepts `ipPools` per `additionalInterfaces[]` entry, but the Multus → Calico IPAM plumbing (passing pool selection through `cni-args` in the Multus JSON) is not yet implemented and not yet verified end-to-end. The field is reserved; first render pass leaves it as a no-op. We should either wire it through `cni-args` or temporarily reject the field in validation, both additive follow-ups.
  • AWS `nativeIP` + `spec.network` interaction is intentionally not validated in v1 (deferred).
  • `externalNetworks` deprecation milestone — the CEL rule blocks setting both, but no removal timeline is in this PR.

Test plan

  • `make gen-files` (regenerates deepcopy + CRD manifest cleanly)
  • `go test ./pkg/render/egressgateway/...` (existing + 3 new cases: `network` annotation, multus annotation JSON, no-annotation default)
  • `go test ./pkg/controller/egressgateway/...` (existing + 3 new cases: reject when `MultiInterfaceMode != Multus`, reject when NAD CRD absent, soft-warning when NAD missing)
  • `go vet ./pkg/controller/egressgateway/... ./pkg/render/egressgateway/... ./pkg/controller/utils/... ./internal/controller/...`
  • Manual smoke test on kind: install Multus, set `multiInterfaceMode=Multus`, create a NAD, apply an EgressGateway with `additionalInterfaces`, verify `ip link` inside the pod shows the secondary interface and that the pod annotations match. Pending.
  • Negative manual: `multiInterfaceMode` unset → EGW shows Degraded with the documented message; delete the NAD → warning surfaces but Deployment remains.

🤖 Generated with Claude Code

Extend EgressGateway to attach the primary interface to a named Calico
Network (via the cni.projectcalico.org/networks annotation) and to attach
additional secondary NICs through Multus NetworkAttachmentDefinitions.

API:
- spec.network references a v3.Network and emits cni.projectcalico.org/networks
  on the pod. Mutually exclusive with the (deprecated) spec.externalNetworks,
  enforced by a CEL rule on the spec.
- spec.additionalInterfaces[] adds secondary NICs (max 9). Each entry names
  the pod-side interface and selects an attachment mechanism via a
  union-shaped attachment block. Only the multus arm is wired up now; the
  comment in InterfaceAttachment reserves space for a native Calico Network
  arm in a later release.

Controller:
- Discovers the NetworkAttachmentDefinition CRD at startup; conditionally
  watches NADs so create/delete events re-trigger reconciliation.
- Rejects additionalInterfaces unless both
  Installation.spec.calicoNetwork.multiInterfaceMode=Multus and the Multus
  CRD are present.
- Treats a missing referenced NAD as a soft warning so NAD-after-EGW
  creation order doesn't fail reconcile; Multus will surface the per-pod
  error.

Render:
- Emits cni.projectcalico.org/networks for spec.network and a
  k8s.v1.cni.cncf.io/networks JSON array for additionalInterfaces, with
  deterministic interface names taken from each entry's Name field.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants