Skip to content

docs(architecture): document egress control vs sandbox MMDS boundary (#3625) - #3627

Open
AdaAibaby wants to merge 1 commit into
e2b-dev:mainfrom
AdaAibaby:docs/egress-mmds-boundary
Open

docs(architecture): document egress control vs sandbox MMDS boundary (#3625)#3627
AdaAibaby wants to merge 1 commit into
e2b-dev:mainfrom
AdaAibaby:docs/egress-mmds-boundary

Conversation

@AdaAibaby

Copy link
Copy Markdown
Contributor

Summary

Documents the supported contract requested in #3625: how sandbox egress control (denyOut / allowPublicTraffic) relates to sandbox-local MMDS (169.254.169.254), and why controlling one does not control the other.

Adds a ### Network egress control and sandbox metadata (MMDS) subsection to docs/ARCHITECTURE.md (under Core flows, after Sandbox traffic). No code change.

What it clarifies

  • Egress filtering acts only on tap→host traffic. TCP is REDIRECTed to the host-netns TCP proxy, which enforces denyOut plus the built-in DeniedSandboxCIDRs baseline (already includes 169.254.0.0/16) and re-checks the resolved IP before connect() (DNS-rebinding guard); non-TCP is matched by the nftables chain. allowPublicTraffic:falsedenyOut 0.0.0.0/0. BYOP only changes nftables Rule 3 (drop non-TCP to denied ranges); BYOP endpoints are themselves validated against DeniedSandboxCIDRs.
  • MMDS is not on the filtered path. 169.254.169.254 is Firecracker's per-microVM MMDS (v2), configured per-sandbox by the orchestrator; those packets are handled by the VMM's virtio-net device and never egress the tap, so denyOut / allowPublicTraffic / full deny-egress have no effect on it. envd pins a self-healing RETURN rule for 169.254.169.254:80 to keep /init working.
  • MMDS contents are sandbox-scoped (instanceID, envID, address, accessTokenHash) and do not proxy host/cloud IMDS or IAM credentials; accessTokenHash is a hash, not the token. The "never routes to a real IMDS" property is partly a deployment contract (guest tap must not be bridged onto a network where a real IMDS is routable).
  • Consequence for users: to block outbound access to a real cloud IMDS, rely on denyOut / the DeniedSandboxCIDRs baseline; there is currently no supported knob to deny user-workload access to the in-guest MMDS endpoint without breaking envd init.

Why this is not a duplicate

Verification

  • Every file path and symbol cited in the new section was checked to exist on main (firewall.go, tcpfirewall/, sandbox-network/firewall.go, fc/client.go, fc/mmds.go, mmds_route_linux.go, init.go, ValidateEgressProxy).
  • Docs-only change; no build/test impact.

Closes #3625

AI assistance

AI assistance was used to trace the firewall/MMDS call chains and draft the documentation. A human submitter has reviewed every line.

Clarifies the two independent reachability mechanisms and why they are often
conflated (issue e2b-dev#3625): egress filtering (denyOut / allowPublicTraffic) acts
only on tap->host traffic, while MMDS (169.254.169.254) is a Firecracker
per-microVM service that never egresses the tap and is therefore not affected
by any egress control.

Documents the supported contract: MMDS is sandbox-scoped (instanceID, envID,
address, accessTokenHash) and does not proxy host/cloud IMDS or IAM
credentials; accessTokenHash is a hash, not the token; the 'no real IMDS'
property is partly a deployment contract; and DeniedSandboxCIDRs (incl.
169.254.0.0/16) plus the pre-connect resolved-IP check are the testable egress
boundary. Also notes there is currently no supported way to deny user-workload
access to MMDS without breaking envd /init.

Signed-off-by: AdaAibaby <shaolila@buaa.edu.cn>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clarify denyOut semantics for sandbox-local MMDS (169.254.169.254)

2 participants