fix: WireGuard sessions drop while the tunnel stays up

[kazmosahebi]

Goal

SSH and RDP sessions over the WireGuard tunnel stay connected as reliably as they do over OpenVPN on the same deployment.

Actual behaviour

On 2026-09-17, SSH and RDP sessions over WireGuard dropped repeatedly. The tunnel itself stayed up throughout: the client showed it connected while the sessions inside it died.

Switching the same Windows client to the OpenVPN config for the same deployment fixed it, with no drops since.

A second user on the same deployment is also on WireGuard and suspects the shared-client handling from #27 is still involved.

Not yet measured: how long a session lasts before it drops, and whether the drops track traffic volume.

Expected behaviour

Long-lived TCP sessions survive over WireGuard. docs/vpn-client-setup.md tells users to try WireGuard first, so it needs to be at least as stable as the OpenVPN fallback.

Reproduction steps

1. Windows client, one slot file per device as "Using more than one device" in docs/vpn-client-setup.md describes: *-wg-*.conf on the first device, *-wg2-*.conf on the second.
2. Connect over WireGuard and hold an SSH or RDP session open to a host behind the VPN.
3. The session drops or freezes while the tunnel still shows connected.

Already ruled out

Two devices loading the same key, which is the failure #27 describes. Each device here loads its own slot file. That leaves a fault in slot issuance or the server peer list, which check 2 covers.

Candidate causes, and the check that settles each

1. wg_mtu above what the path carries. fix(network): clamp forwarded TCP MSS to the tunnel MTU #32 states that the MSS clamp reads the configured wg_mtu, so it cannot correct a value that is itself too high. Handshakes and keepalives are small and pass, full-size packets are dropped, and that fits a live tunnel with dying sessions. OpenVPN runs on its own tun_mtu and mssfix, which would explain why it is unaffected. Check from a Windows WireGuard client: ping -f -l <size> <host behind the VPN>, stepping <size> down from the configured wg_mtu minus 28 until replies return, then compare the result with CULVERT_WG_MTU.
2. Slot or peer-list fault on the server. During a drop, run wg show wg0 endpoints and wg show wg0 latest-handshakes for the affected peers. An endpoint that flips between two source addresses means one key is live in two places. wg show wg0 allowed-ips should list each tunnel /32 exactly once.

Environment

• Client: WireGuard on Windows
• Protocol affected: WireGuard only. OpenVPN on the same deployment is stable.
• Device slots: a separate slot file per device

Definition of Done

• Deployed culvert version, CULVERT_WG_MTU and the network profile recorded on this issue
• Both checks above run during a drop, with the results recorded here
• Root cause identified and fixed, or documented as a deployment setting if it turns out to be wg_mtu
• An SSH session held open over WireGuard for 8 hours without a drop

[catinspace-au]

Half of check 2 is answered, and the "already ruled out" line needs reopening.

Ran wg show wg0 on the devex vpn VM (culvert v2.1.15, container up 22h). 30 peers, tunnel IPs 192.168.103.2 through .31, and every /32 appears EXACTLY ONCE. So there is no duplicate allocation and no overlapping AllowedIPs in the server peer list. That part of check 2 is clean.

Four peers had endpoints at sample time:

Tunnel IP	Endpoint	Latest handshake	rx / tx
192.168.103.4	144.6.172.186	Now	2.75 MiB / 4.53 MiB
192.168.103.5	202.128.127.8	4s	2.68 MiB / 1.37 MiB
192.168.103.17	1.145.45.110	1m 29s	10.19 MiB / 103.72 MiB
192.168.103.14	124.170.5.152	10h 45m	18.00 MiB / 72.89 MiB

Two live peers on adjacent slots from DIFFERENT public IPs. That is two people, not one person's two devices.

Which is the problem with "Already ruled out: two devices loading the same key". That was ruled out per-PERSON, not per-deployment. On devex there is ONE client identity (hyperi) and hyperi.zip carries all 10 slots' private keys. Every person gets the same zip and picks a slot by verbal agreement. Nothing in culvert assigns a slot to a person, and nothing detects two people landing on the same one. docs/vpn-client-setup.md says "Load one set per device, or they will keep knocking each other offline" -- that guarantee holds inside one person's kit and nowhere else.

So the shared-key hypothesis is back on the table, and it is a hand-off design problem rather than a slot-issuance bug.

The second thing worth recording is that culvert cannot SEE this even when it happens. start_wg_connection_monitor reads wg show wg0 latest-handshakes and transfer only. Nothing anywhere reads wg show wg0 endpoints. A key live in two places rekeys constantly, so the monitor reports a healthily connected peer while the return traffic is going to whoever handshook last. No log line, no metric, no health check. An operator has no way to find this short of running check 2 by hand at the exact moment someone complains.

On candidate cause 1: devex sets CULVERT_PROFILE=mobile and no CULVERT_WG_MTU, so the server's wg0 is at the profile's 1280 with keepalive 25. Profiles are COPYed into the image and the loader hard-exits on a missing one, so it definitely loads. BUT a client .conf is a static artefact stamped with whatever wg_mtu applied the day it was generated, and nothing warns when the server's value later moves. A config issued before the mobile profile says MTU = 1420 against a server at 1280, and that asymmetry black-holes the server-to-client direction specifically -- which matches the 10:1 tx:rx on .17 above. Worth confirming against the actual file rather than assuming.

Still to run (needs a shell on the vpn VM):

• allocations.json, to name which peer is which person's slot
• the MTU = line in the affected client's .conf
• wg show wg0 endpoints sampled repeatedly DURING a drop, which is the only thing that settles endpoint flapping either way

[catinspace-au]

Correction to my last comment: the drops were a bounded event on Thursday 2026-09-17, not an ongoing condition. So sampling wg show wg0 endpoints during a drop is off the table, and the check that matters now is a static one -- what the affected client's .conf actually says.

There is a much better candidate than either of the two on the issue, and it is a bug this repo already FIXED.

fix(client): separate the address clients dial from the certificate identity (0a66586, v2.1.15, 2026-09-09) exists because a client config naming the VPN's own zone cannot recover. The validator added in that commit states the failure in its own warning text:

The address clients dial ('...') is inside CULVERT_DNS_DOMAIN='...', the zone this VPN serves, so clients cannot resolve it before the tunnel is up. Set CULVERT_EXTERNAL_ENDPOINT to a publicly resolvable name or a literal address.

On the devex deployment that name is split-horizon: public DNS answers 203.217.160.131, the internal CoreDNS answers 10.66.0.1. The tunnel pushes CoreDNS as the client's resolver. So once connected, the client's own endpoint name resolves to an address reachable ONLY through the tunnel it is trying to rebuild.

The WireGuard client on Windows re-resolves endpoint names on network-change events rather than on a timer. Laptop changes network -- wifi to ethernet, sleep and wake, a hotspot -- the endpoint re-resolves to 10.66.0.1, and the tunnel is now aimed at an address that needs the tunnel. It shows connected, sessions inside it die, and it does not come back on its own. That is the reported symptom, and it is random in exactly the way a network change is random.

Timeline:

• 2026-09-09 -- v2.1.15 lands, devex sets CULVERT_EXTERNAL_ENDPOINT=203.217.160.131, converge applied. The live server now ISSUES the literal. Verified in the hyperi-infra commit that did it (50a3020).
• 2026-09-17 -- the drops.

Eight days apart, which is the point. A client .conf is a static artefact. The server-side fix corrects what is issued FROM that day on and does nothing whatever for a zip already in someone's hands. A config handed out before 2026-09-09 still says Endpoint = devex.hyperi.io:9443 and still breaks.

The check, and it needs no shell on the server -- open the imported tunnel in the WireGuard app and read the Endpoint field:

• 203.217.160.131:9443 -- config is post-fix, this is not it, go back to the MTU and shared-slot candidates.
• devex.hyperi.io:9443 -- config predates the fix, and this is the answer. Re-issue and re-import.

Same question applies to the OpenVPN half, and it is worth asking because "switching to OpenVPN fixed it" has two readings. The .ovpn files carried the same broken remote and got the same fix in the same commit. If the switch involved pulling a FRESH zip, then what fixed it was the new endpoint, not the protocol -- which would confirm this rather than pointing at WireGuard.

The general defect behind it is that culvert has no way to tell an operator that the configs people are holding no longer match what the server would issue. Endpoint, MTU and DNS all changed under existing clients at various points, each silently. Worth its own issue.

Sources for the Windows re-resolve behaviour:

• https://www.procustodibus.com/blog/2023/09/troubleshooting-wireguard-dns-issues/
• https://adhdecode.com/debugging/wireguard/wireguard-peer-endpoint-resolution-failed/