You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Envoy DNS resolver pinned V4_ONLY (atenet/.../xds.go:241), 0.0.0.0 binds, and the actor interior net is hard-coded 169.254.17.x/30 forced through .To4() which rejects non-IPv4.
I agree that ipv6 only support is super important, but I also think we have so many open networking questions we're looking to answer that maybe this should be a fast follow from those.
Some examples:
I think that would be good to have as a first step is a way to start the Kind setup with IPv6 single-stack and IPv6 dual stack. Otherwise, if we get patches to fix IPv6 support, we would not be able to test the fixes.
I think that would be good to have as a first step is a way to start the Kind setup with IPv6 single-stack and IPv6 dual stack. Otherwise, if we get patches to fix IPv6 support, we would not be able to test the fixes.
We have to be careful about how much actions time we use up on every PR as long as we're operating with the free default resources. It can add up every quickly covering every dimension.
Also FYI single-stack ipv6 kind has some gaps currently, in general we mostly see ipv4 or dual-stack demand and contributions ...
We're not even running ipv6 kind on every k8s PR currently. We run it periodically post-merge.
Following up after PR #559 merged: the pod-IP:80 → actor-veth:80 DNAT is gone with no fallback, so atunnel's mTLS listener is now the only actor ingress path. The listen address itself is flag-configurable, but the egress redirect in internal/atunnel/original_dst_linux.go is SO_ORIGINAL_DST-only (IPv6 variant still a TODO), and internal/ateomnet's interior actor network is still hard IPv4 after the #548 refactor moved it into a shared package.
I was previously able to fork-patch around this to run Substrate on an IPv6-only cluster; with atunnel now mandatory and no fallback, that's no longer straightforward. Could we raise the priority on IPv6/dual-stack support, and what's actually blocking it at this point? Please let me know I could help here.
Adding a downstream data point, in case it helps with the "where does IPv6 demand come from" question earlier in this thread.
Setup: EKS with an IPv6 service plane. Services are ipFamilyPolicy: SingleStack and ipFamilies: [IPv6], so cluster DNS returns AAAA records only. Substrate is v0.6.0-alpha3 from the kagent-dev fork, with the gVisor sandbox class. kagent is v1.0.0-alpha16 on top.
What happens: Since kagent-dev/kagent#2939, the kagent runtime inside the actor saves A2A task state over gRPC to the kagent controller Service (runtimeDefaults.controlPlaneUrl). On this cluster that Service only has an IPv6 ClusterIP. The actor resolves it fine, because the DNS relay forwards the query and the AAAA record comes back. The dial then fails inside the guest before the model is ever called:
failed to save task state: rpc error: code = Unavailable
desc = connection error: desc = "transport: Error while dialing:
dial tcp [<ipv6-clusterip>]:8083: connect: network is unreachable"
This matches the current actornet shape. The actor only gets 169.254.17.2/30 with an IPv4 default route, and the atunnel redirect is only installed in the IPv4 nftables table. So there's no route for an IPv6 destination, and #945 would drop the traffic further along anyway. Agent readiness and snapshot checks all stay green; the failure only shows up on the first session input. Every kagent-on-Substrate chat on an IPv6-service EKS cluster fails the same way.
Offer: we'd be glad to test the #1057 / #1116 stack on a real IPv6 EKS cluster (gVisor) once it's in a testable state, and report back on the per-family paths it touches, mainly actor → in-cluster IPv6 Service. If there's a branch or image tag you'd like exercised, or specific scenarios that would help get the fixes past the IPv6 e2e gate, let us know.
Activity
EItanya commentedon Jun 15, 2026
I agree that ipv6 only support is super important, but I also think we have so many open networking questions we're looking to answer that maybe this should be a fast follow from those.
Some examples:
bowei commentedon Jun 15, 2026
I think that would be good to have as a first step is a way to start the Kind setup with IPv6 single-stack and IPv6 dual stack. Otherwise, if we get patches to fix IPv6 support, we would not be able to test the fixes.
BenTheElder commentedon Jun 15, 2026
We have to be careful about how much actions time we use up on every PR as long as we're operating with the free default resources. It can add up every quickly covering every dimension.
Also FYI single-stack ipv6 kind has some gaps currently, in general we mostly see ipv4 or dual-stack demand and contributions ...
We're not even running ipv6 kind on every k8s PR currently. We run it periodically post-merge.
bowei commentedon Jun 15, 2026
I'd be ok with starting with dual-stack for local testing but not in the CI.
bowei commentedon Jun 22, 2026
Ok, some concrete steps towards chipping away at this:
ixcans commentedon Jul 31, 2026
Hello folks, Bowei Du (@bowei), Benjamin Elder (@BenTheElder), Eitan Yarmush (@EItanya),
Following up after PR #559 merged: the pod-IP:80 → actor-veth:80 DNAT is gone with no fallback, so atunnel's mTLS listener is now the only actor ingress path. The listen address itself is flag-configurable, but the egress redirect in internal/atunnel/original_dst_linux.go is SO_ORIGINAL_DST-only (IPv6 variant still a TODO), and internal/ateomnet's interior actor network is still hard IPv4 after the #548 refactor moved it into a shared package.
I was previously able to fork-patch around this to run Substrate on an IPv6-only cluster; with atunnel now mandatory and no fallback, that's no longer straightforward. Could we raise the priority on IPv6/dual-stack support, and what's actually blocking it at this point? Please let me know I could help here.
Thanks!
bowei commentedon Jul 31, 2026
See my comment: first step is to enable creating an ipv6 enabled kind cluster so so ipv6 support can be tested.
e2e: assert the actor DNS zone answers benign rcodes
e2e: assert the router ingress works on every IP family
ci: add an opt-in IPv6-only kind e2e job
41 remaining items
[WIP] e2e: give TestActorEgress an in-cluster origin (#1440)
ateomnet: move the actor nftables table to the inet family
ateomnet: move the actor nftables table to the inet family
ateomnet: move the actor nftables table to the inet family
ateomnet: move the actor nftables table to the inet family
ateomnet: give the actor an IPv6 address when the pod has one
ateomnet: give the actor an IPv6 address when the pod has one
ci: run the IPv6-only e2e job on both sandbox classes
e2e: check an actor's address families against its worker pod's
ateom-microvm: give the guest an IPv6 address when the pod has one
ateom-microvm: reconcile the guest's address families on restore
e2e: check an actor's address families against its worker pod's
ateom-microvm: give the guest an IPv6 address when the pod has one
ateom-microvm: reconcile the guest's address families on restore
go-faustino commentedon Oct 21, 2026
Adding a downstream data point, in case it helps with the "where does IPv6 demand come from" question earlier in this thread.
Setup: EKS with an IPv6 service plane. Services are
ipFamilyPolicy: SingleStackandipFamilies: [IPv6], so cluster DNS returns AAAA records only. Substrate isv0.6.0-alpha3from the kagent-dev fork, with the gVisor sandbox class. kagent isv1.0.0-alpha16on top.What happens: Since kagent-dev/kagent#2939, the kagent runtime inside the actor saves A2A task state over gRPC to the kagent controller Service (
runtimeDefaults.controlPlaneUrl). On this cluster that Service only has an IPv6 ClusterIP. The actor resolves it fine, because the DNS relay forwards the query and the AAAA record comes back. The dial then fails inside the guest before the model is ever called:This matches the current
actornetshape. The actor only gets169.254.17.2/30with an IPv4 default route, and the atunnel redirect is only installed in the IPv4 nftables table. So there's no route for an IPv6 destination, and #945 would drop the traffic further along anyway. Agent readiness and snapshot checks all stay green; the failure only shows up on the first session input. Every kagent-on-Substrate chat on an IPv6-service EKS cluster fails the same way.Offer: we'd be glad to test the #1057 / #1116 stack on a real IPv6 EKS cluster (gVisor) once it's in a testable state, and report back on the per-family paths it touches, mainly actor → in-cluster IPv6 Service. If there's a branch or image tag you'd like exercised, or specific scenarios that would help get the fixes past the IPv6 e2e gate, let us know.