feat: replace ingress-nginx with Traefik v3 as the ingress controller#565
feat: replace ingress-nginx with Traefik v3 as the ingress controller#565squidboylan wants to merge 2 commits into
Conversation
squidboylan
force-pushed
the
traefik
branch
3 times, most recently
from
f93765a to
de60503
Compare
| # Goal: Rewrite rules for in-cluster access to a service: gitea, argocd, etc using the same FQDN as for external access | ||
|
|
||
| # subdomain names e.g. gitea.cnoe.localtest.me resolves to the IP address of the kubernetes ingress service and then will become ingress-nginx-controller.ingress-nginx.svc.cluster.local | ||
| # subdomain names e.g. gitea.cnoe.localtest.me resolves to the IP address of the kubernetes ingress service and then will become my-traefik.traefik.svc.cluster.local |
There was a problem hiding this comment.
Would it make sense to make it similar to what we have so like maybe omething in line of ingress-controller.traefik.svc.cluster.local ?
RafPe
left a comment
RafPe
left a comment
There was a problem hiding this comment.
Looks good from here - only thing I would change is most likely the service name for ingress
|
Regarding going straight to gateway-api - if thats supported by all the frameworks we plan to use then that also might be idea - as otherwise we will confuse users first with switch to Traefik and then I would guess shortly after we will migrate them to gateway api |
I like the idea of having an option that is secure and doesnt introduce large breaking changes like gateway api. Additionally I don't have a PR for implementing gateway api at the moment. |
- Add TraefikPackageName constant, TraefikStatus type, and TraefikNamespace - Hand-craft Traefik v3 install manifests (CRDs, RBAC, Deployment with hostPorts, IngressClass, TLSStore) for KinD - Add ReconcileTraefik reconciler mirroring ReconcileNginx pattern - Wire Traefik into controller (installers map, bootStrapApps, GetEmbeddedRawInstallResources); comment out nginx codepaths (not deleted) - Update tls.go to create cert secret in traefik namespace instead of nginx - Update create/root.go corePkgs map to include traefik instead of nginx - Add TraefikPackageName to util.go SetPackageLabels switch case - Rewrite ArgoCD ingress to use Traefik (remove nginx annotations, add StripPrefix Middleware for path-routing, backend port http) - Make ArgoCD --insecure flag unconditional in install.yaml - Rewrite Gitea ingresses to use Traefik (remove nginx annotations, add Middleware CRDs for path-routing rewrites) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Caleb Boylan <calebboylan@gmail.com>
Signed-off-by: Caleb Boylan <calebboylan@gmail.com>
2 participants
I decided to tackle this as a middle ground before tackling migrating to gateway api. Happy to toss this if we want to go straight to gateway API. Interested in hearing what others think
There shouldn't be anything surprising here but there are some implementation details due to traefik's quirks vs nginx including renaming traefik pod port names from web and websecure to http and https, and hardcoding the ingress loadbalancer IP to localhost which prevents the ArgoCD applications from getting stuck in the "Progressing" state.
Additionally in
pkg/k8s/deserialize.gowe add functionality to applying resources that arent part of our schema by representing them as unstructured manifests. This prevents the need to import all of the traefik CRDs into our code which isn't very convenient and these CRs are only applied by reading them off disk, we dont build any in go code.