Repository navigation
perf(connlib): batch routing and crypto per flow - #15943
Draft
thomaseizinger wants to merge 66 commits into
Draft
thomaseizinger wants to merge 66 commits into
thomaseizinger wants to merge 66 commits into
Conversation
`Node::encapsulate` returns an `EncapsulateBatch` that borrows the node until `finish(len)` reserves `len` consecutive counters for the connection's fixed source, destination and channel. `begin_decapsulate` returns a `DecapsulateBatch` that takes further segments of the same datagram with `increment` and releases the node with `finish`. The pending values cover a whole batch: `encrypt_into` takes the batch's packets and buffers, `decrypt` its segments, and `commit_decrypted` stays per message. Adding a segment only parses WireGuard data messages, because parsing a handshake has side effects in the interface. A segment that is not a data message of the same session and path is left for a new batch. Callers keep their per-packet shape for now by finishing every batch with one message.
`CurrentFlowGuard::into_batch` moves the facts gathered for the first packet of a flow out of the thread-local into a `FlowBatch`, which only counts the payload bytes of further packets. `Tracker::insert_batch` then records the flow once with a packet count and a byte sum. The batch owns its facts instead of borrowing the tracker so that a caller can keep one open per flow of a received batch. A packet that opens, closes or resets a TCP connection changes the flow's state, so callers begin a new batch for it and the first packet's flags keep deciding whether a flow is created or split.
`GatewayState::begin_tun_batch` routes the first packet of a batch as `handle_tun_input` does but keeps the borrows it took, so that further packets of the same 5-tuple join the batch through `TunBatch::try_push` without a peer lookup, a NAT lookup or a resource classification. `TunBatch::finish` reserves one run of consecutive counters for the whole batch and records its flow once. Per packet, the slow path records the flow's peer, resource and outer tuple, refreshes the NAT entry's `last_incoming`, accumulates incoming FIN and RST flags, marks the connection as active and ticks the tunnel's send timers. All of these take the same `now` for every packet of a batch and depend only on the 5-tuple, so doing them once per batch is equivalent. The flow's packet and byte counts come from the batch. A TCP SYN, FIN or RST changes the NAT entry's flags, an ICMP error has no flow of its own and a packet the NAT turns into an ICMP error is no longer part of its flow, so none of them join a batch.
`GatewayState::handle_peer_batch` handles the data messages of one received datagram in order. The first packet of each flow takes the slow path and, if it was forwarded as is or through an existing NAT session, its decision is kept in a small per-batch table: later packets of the same client and 5-tuple only apply the cached destination rewrite and count towards the flow's record. The table is cleared and the flows recorded when the batch ends. `handle_peer_input` handles one message as a batch of its own. Per packet, the slow path commits the message to the replay window, records the flow's peer, resource, token, domain and translated destination, refreshes the NAT entry's `last_outgoing` and accumulates outgoing FIN and RST flags. Committing stays per message. The flow facts, `last_outgoing` and the flags are refreshed at the miss, which is equivalent because every packet of a batch shares `now`, later packets carry no SYN, FIN or RST and the state a decision depends on can only change through the batch itself, where a later slow-path message creates NAT sessions for other flows or removes the peer, after which committing fails. A packet that creates a NAT session, an ICMP error, a p2p control message and a TCP SYN, FIN or RST are never cached.
The Gateway's TUN loop begins a batch with the first packet, pushes the following packets of the same flow while they fit the GSO train the batch is encrypted into, and hands the finished batch with its packets to the crypto workers as one unit. The network loop collects the segments of one received datagram that belong to the same session into one decrypt job, and the decrypted loop handles each datagram's packets as one batch. The Client keeps its per-packet state entry points for now, as batches of one. A batch shares one counter reservation and one key, so the crypto workers take whole batches and a train never splits one. Packets of a batch go out in the order they came in with consecutive counters, and batches in the order they were opened.
`ClientState::begin_tun_batch` routes the first packet of a batch as `handle_tun_input` does and lets further packets of the same flow join through `TunBatch::try_push`, which is now shared with the Gateway. Flows to other clients stay on the slow path for every packet because connection tracking inspects each of them, and a flow to a DNS resource only batches once the Gateway confirmed its NAT, since until then every packet may have to be buffered. `handle_peer_batch` mirrors the Gateway's: packets of one flow from a Gateway reuse the first packet's decision, packets from other clients take the slow path. The per-flow batching of the TUN loop moves into one helper for both roles, and `CurrentFlowGuard::into_batch` hands the tracker back so a batch can be opened after the guard served an early return.
thomaseizinger
force-pushed
the
connlib-batched-crypto
branch
from
October 23, 2026 21:17
6e41cce to
2b7f789
Compare
|
coverage: 80.087% (-0.01%) from 80.101% — connlib-batched-crypto into main |
Prefetching the next packet is the caller's business; snownet only needs to walk the packets of a batch in order.
thomaseizinger
added a commit
that referenced
this pull request
Oct 26, 2026
The tunnel-proto fuzzer drives the real event loop, but so far only ever with one packet per TUN read and one segment per received datagram. Production hosts batch both, so simulated Clients and Gateways now hand the tunnel all packets that are ready in one TUN read and coalesce same-size datagrams between the same two sockets into GRO datagrams on receive. A single packet still travels alone whenever nothing else is ready with it. Related: #15943 --------- Co-authored-by: Claude <noreply@anthropic.com>
# Conflicts: # rust/libs/connlib/snownet/src/node/connections.rs # rust/tests/tunnel-bench/src/throughput.rs
When a segment of a received GRO datagram was handled internally (a handshake, a TURN message) or failed, the receive loop moved on to the next datagram and dropped the segments after it. GRO coalesces a short data message behind a handshake initiation of the same size class, so the first packet sent after a re-handshake could be lost. Take the next segment from the datagram whenever no data batch is left pending.
# Conflicts: # rust/Cargo.lock
Conflicts came from #16009, #16054 and #16056 meeting the batching changes in `gateway.rs`, `client_on_gateway.rs` and `lib.rs`: the imports and re-exports now carry both sides, and main's `only_allows_dns_port_on_gateway_tun_ip` test matches on `TranslateOutboundResult::Send(_, _)`. #16025's early return for a connection without peer state drops the flow guard that the slow path now names `guard`. None of main's changes needs handling on the fast paths. The new check in `ensure_allowed_outbound` that only admits TCP and UDP to the Gateway TUN IP on `TUN_DNS_PORT` depends only on the destination address and protocol, which are part of the cached `Flow`, so a decision-table hit gives the same answer as the slow path and ICMP to that IP is rejected and never cached. #16025 closes the connection only when no peer exists, in which case no decision is cached for that Client. Clearing a Client's state on a new key (#16009), the `ProxyIpAlreadyAssigned` cleanup, keying DNS lookups by public key (#16028) and the site DNS query authorization (#16056) run on authorization, DNS resolution and DNS socket input, none of which can happen while a `TunBatch` or `PeerBatch` borrows the state.
# Conflicts: # rust/Cargo.lock # rust/Cargo.toml
Follows the rename of `DecapsulateBatch::increment` to `push`, and renames connlib's wrapper to match.
# Conflicts: # rust/Cargo.lock
|
Bencher Report| Project | Firezone |
| Branch | connlib-batched-crypto |
| Testbed | bare-metal-intel-v1 |
🚨 6 Alerts
| Benchmark | Measure Units | View | Benchmark Result (Result Δ%) | Upper Boundary (Limit %) |
|---|---|---|---|---|
| event_loop/client_inbound/cidr | UDP thread CPU ns/packet | 📈 plot 🚷 threshold 🚨 alert (🔔) | 230.33(+6.24%)Baseline: 216.79 | 229.43 (100.39%) |
| event_loop/client_inbound/dns | UDP thread CPU ns/packet | 📈 plot 🚷 threshold 🚨 alert (🔔) | 237.23(+9.09%)Baseline: 217.46 | 227.25 (104.39%) |
| event_loop/client_outbound/cidr | TUN threads CPU ns/packet | 📈 plot 🚷 threshold 🚨 alert (🔔) | 326.56(+6.02%)Baseline: 308.02 | 317.55 (102.84%) |
| event_loop/client_outbound/dns | TUN threads CPU ns/packet | 📈 plot 🚷 threshold 🚨 alert (🔔) | 341.80(+8.12%)Baseline: 316.12 | 324.01 (105.49%) |
| event_loop/gateway_outbound/cidr | TUN threads CPU ns/packet | 📈 plot 🚷 threshold 🚨 alert (🔔) | 347.28(+11.00%)Baseline: 312.87 | 322.66 (107.63%) |
| event_loop/gateway_outbound/dns | TUN threads CPU ns/packet | 📈 plot 🚷 threshold 🚨 alert (🔔) | 346.00(+9.94%)Baseline: 314.71 | 321.29 (107.69%) |
Click to view all benchmark results
| Benchmark | Crypto workers CPU | Benchmark Result ns/packet x 1e3 (Result Δ%) | Upper Boundary ns/packet x 1e3 (Limit %) | Event loop CPU | Benchmark Result ns/packet (Result Δ%) | Upper Boundary ns/packet (Limit %) | TUN threads CPU | Benchmark Result ns/packet (Result Δ%) | Upper Boundary ns/packet (Limit %) | Throughput | Benchmark Result bits/s x 1e9 (Result Δ%) | Lower Boundary bits/s x 1e9 (Limit %) | UDP thread CPU | Benchmark Result ns/packet (Result Δ%) | Upper Boundary ns/packet (Limit %) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| event_loop/client_inbound/cidr | 📈 view plot 🚷 view threshold | 1.23 x 1e3(-2.86%)Baseline: 1.27 x 1e3 | 1.29 x 1e3 (95.81%) | 📈 view plot 🚷 view threshold | 263.79(-45.60%)Baseline: 484.88 | 506.66 (52.07%) | 📈 view plot 🚷 view threshold | 237.63(-0.74%)Baseline: 239.39 | 250.14 (95.00%) | 📈 view plot 🚷 view threshold | 12.24 x 1e9(+8.14%)Baseline: 11.32 x 1e9 | 11.06 x 1e9 (90.35%) | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 230.33(+6.24%)Baseline: 216.79 | 229.43 (100.39%) |
| event_loop/client_inbound/dns | 📈 view plot 🚷 view threshold | 1.23 x 1e3(-3.06%)Baseline: 1.27 x 1e3 | 1.29 x 1e3 (95.73%) | 📈 view plot 🚷 view threshold | 268.22(-45.06%)Baseline: 488.17 | 511.39 (52.45%) | 📈 view plot 🚷 view threshold | 235.87(-2.53%)Baseline: 241.98 | 251.39 (93.83%) | 📈 view plot 🚷 view threshold | 12.38 x 1e9(+9.74%)Baseline: 11.28 x 1e9 | 11.05 x 1e9 (89.30%) | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 237.23(+9.09%)Baseline: 217.46 | 227.25 (104.39%) |
| event_loop/client_outbound/cidr | 📈 view plot 🚷 view threshold | 1.07 x 1e3(-6.76%)Baseline: 1.14 x 1e3 | 1.16 x 1e3 (92.05%) | 📈 view plot 🚷 view threshold | 106.57(-61.72%)Baseline: 278.38 | 284.61 (37.44%) | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 326.56(+6.02%)Baseline: 308.02 | 317.55 (102.84%) | 📈 view plot 🚷 view threshold | 17.27 x 1e9(+25.25%)Baseline: 13.79 x 1e9 | 13.41 x 1e9 (77.64%) | 📈 view plot 🚷 view threshold | 266.44(-12.03%)Baseline: 302.87 | 308.88 (86.26%) |
| event_loop/client_outbound/dns | 📈 view plot 🚷 view threshold | 1.06 x 1e3(-7.06%)Baseline: 1.14 x 1e3 | 1.16 x 1e3 (91.55%) | 📈 view plot 🚷 view threshold | 107.61(-64.91%)Baseline: 306.65 | 312.36 (34.45%) | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 341.80(+8.12%)Baseline: 316.12 | 324.01 (105.49%) | 📈 view plot 🚷 view threshold | 17.41 x 1e9(+30.28%)Baseline: 13.37 x 1e9 | 12.99 x 1e9 (74.58%) | 📈 view plot 🚷 view threshold | 250.35(-14.96%)Baseline: 294.38 | 302.73 (82.70%) |
| event_loop/gateway_inbound/cidr | 📈 view plot 🚷 view threshold | 1.23 x 1e3(-2.48%)Baseline: 1.26 x 1e3 | 1.28 x 1e3 (96.01%) | 📈 view plot 🚷 view threshold | 253.25(-53.17%)Baseline: 540.81 | 553.73 (45.73%) | 📈 view plot 🚷 view threshold | 230.20(-6.14%)Baseline: 245.27 | 267.96 (85.91%) | 📈 view plot 🚷 view threshold | 12.19 x 1e9(+8.94%)Baseline: 11.19 x 1e9 | 10.96 x 1e9 (89.91%) | 📈 view plot 🚷 view threshold | 242.05(+3.24%)Baseline: 234.46 | 251.47 (96.26%) |
| event_loop/gateway_inbound/dns | 📈 view plot 🚷 view threshold | 1.23 x 1e3(-1.84%)Baseline: 1.25 x 1e3 | 1.27 x 1e3 (96.82%) | 📈 view plot 🚷 view threshold | 276.89(-55.54%)Baseline: 622.78 | 635.42 (43.58%) | 📈 view plot 🚷 view threshold | 242.51(-4.91%)Baseline: 255.02 | 283.94 (85.41%) | 📈 view plot 🚷 view threshold | 12.41 x 1e9(+11.63%)Baseline: 11.12 x 1e9 | 10.86 x 1e9 (87.49%) | 📈 view plot 🚷 view threshold | 226.85(+2.92%)Baseline: 220.41 | 230.01 (98.63%) |
| event_loop/gateway_outbound/cidr | 📈 view plot 🚷 view threshold | 1.06 x 1e3(-6.89%)Baseline: 1.14 x 1e3 | 1.17 x 1e3 (91.21%) | 📈 view plot 🚷 view threshold | 97.77(-65.62%)Baseline: 284.36 | 291.65 (33.52%) | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 347.28(+11.00%)Baseline: 312.87 | 322.66 (107.63%) | 📈 view plot 🚷 view threshold | 16.99 x 1e9(+22.40%)Baseline: 13.88 x 1e9 | 13.29 x 1e9 (78.23%) | 📈 view plot 🚷 view threshold | 251.73(-13.27%)Baseline: 290.25 | 298.43 (84.35%) |
| event_loop/gateway_outbound/dns | 📈 view plot 🚷 view threshold | 1.09 x 1e3(-4.64%)Baseline: 1.15 x 1e3 | 1.15 x 1e3 (94.80%) | 📈 view plot 🚷 view threshold | 120.31(-64.96%)Baseline: 343.39 | 352.34 (34.15%) | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 346.00(+9.94%)Baseline: 314.71 | 321.29 (107.69%) | 📈 view plot 🚷 view threshold | 14.19 x 1e9(+10.18%)Baseline: 12.88 x 1e9 | 12.67 x 1e9 (89.33%) | 📈 view plot 🚷 view threshold | 294.72(-3.22%)Baseline: 304.53 | 308.30 (95.60%) |
`Tracker::begin_tun_batch` / `begin_network_batch` now return the batch directly instead of a per-packet guard that is later converted into one. A single packet is a batch of one, finished on the spot or on drop; batches that outlive their borrow of the tracker are detached and recorded with `Tracker::insert_batch`, through the same code path.
`Node::commit_decrypted` takes a whole `DecryptedBatch`, looks up its connection once and commits it to the tunnel in one go, so the tunnel and session checks and the timer updates run once per batch.
`CommittedBatch` prefetches each packet one ahead as it is pulled, replacing the `Prefetch` impls on the batch types and the datagram.
`Connection::commit_decrypted` is the event loop's first read of each packet the worker thread wrote, so prefetch the next packet there instead of when iterating the already-committed packets.
Pre-sizes the segment group from the datagram's segment count and picks up sanstun's pre-sized decapsulation batch, so neither grows while a GRO datagram is split into batches.
…e flow Each routing branch now returns whether later packets of the flow may reuse its decision, instead of resetting the batching key to `None`.
The decisions only live as long as one received datagram, so the batch holds them instead of the Client and Gateway state.
A pending encapsulation can now be split at a train boundary, so a TUN batch only ends where its flow does and the encrypt jobs spread it over as many trains as it needs.
# Conflicts: # rust/libs/connlib/tunnel/src/io.rs # rust/libs/connlib/tunnel/src/io/crypto.rs
The crypto worker splits the path agent's probes off a decrypted batch, so committing it hands the probes to the agent and does the per-batch bookkeeping once, for the first accepted data packet. The data packets then go to tunnel-proto lazily, straight from the committed batch, instead of through an intermediate `Vec`.
Packets of a flow that already has a decision in the batch skip `handle_peer_message`, so they are no longer moved through it and back.
The worker extracts each probe as a parsed `Probe`, which the path agent handles directly, so the event loop never inspects an `IpPacket` for one.
Building a typed 5-tuple for every packet only to compare it with the flow of an earlier one dominated the cost of batching. `IpPacket` now classifies TCP segments carrying SYN, FIN or RST once on construction, and a flow holds the address and port bytes exactly as they appear in the packet, so matching a packet against it compares those bytes in place. The gateway's inbound decisions no longer record the connection: a batch of decrypted packets always belongs to a single one.
The key packets are batched by is a flow, but what matters at the call sites is that it selects the decision a packet reuses. Matching the gateway's outbound rewrite inline and finding the reusable decision with a let-else keeps the fast path of a batch unindented.
The single-packet wrappers had no callers left outside the Client's out-of-band path, so fold that path into its one caller and give the batching entry point the name.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consecutive packets of one flow in a TUN read or a received datagram currently repeat the same routing, policy and NAT lookups and reserve a WireGuard counter and clone the session key one by one. With this change, the first packet of a flow in a batch takes the slow path and the rest of the batch reuses its decision: on the way out, a batch of one flow is translated with the cached rewrite, reserves one run of consecutive counters and is encrypted with one key, split across GSO trains where needed; on the way in, the data messages of one received datagram form one decrypt job, are committed to their connection in one go, and the packets of each flow from a Client or Gateway reuse the first packet's authorization and destination rewrite. The flow tracker records a batch once, with a packet count and a byte sum. Packets that change connection-tracking or NAT state, ICMP and control messages stay on the slow path.
This depends on sanstun's
batched-cryptobranch and is not mergeable until that PR lands and the dependency points back atmain.Related: firezone/sanstun#129