feat(templates): bitmagnet — DHT crawler behind a WireGuard kill-switch - #142
Merged
Conversation
…ch (#142) Adds a `bitmagnet` template: a minimal Alpine VM running the bitmagnet BitTorrent DHT crawler and indexer with its own PostgreSQL database, no Docker or Compose involved. The VPN is the reason this is a template rather than a compose file on an existing host. Crawling the DHT continuously announces the guest to the swarm, so running bitmagnet on a bare host publishes that host's IP address to tens of thousands of peers. The guarantee is that with WireGuard down, no software in the guest can reach the internet — not just bitmagnet, anything, as any user. OUTPUT policy is DROP, and the exceptions are pinned rather than opened by port, because a default-deny policy is only as strong as its holes: - the handshake is allowed to the endpoint's resolved addresses, not to the whole internet on UDP/51820 - SSH replies require conntrack ESTABLISHED, so the rule cannot carry newly-opened outbound connections - DHCP is restricted to the broadcast address and source port 68 Both unpinned forms are usable covert channels with the tunnel down; the UDP one was verified to leak before pinning. bitmagnet's own traffic is additionally rejected on any interface other than wg0 by group ownership. With no VPN configured the crawler is disabled outright rather than merely warned about — both dht_crawler.enabled and the dht_crawler worker key, since either alone still joins the swarm. The database and web UI still run, so the VM is a working indexer with an empty index and adding a tunnel is the only step needed to start crawling. The web UI (port 3333) is never exposed: it gets no firewall hole from any source and is reached over `vee tunnel`. `--pg-data-dir` bind-mounts a host directory as PostgreSQL's data directory over virtiofs, so the crawled index outlives the VM. Verified by crawling, destroying the VM, and recreating it against the same directory with the index intact and initdb correctly skipped. Three things about the guest environment that are not obvious: - virtiofs passes host ownership through and vee runs virtiofsd unprivileged, so the guest cannot chown the share. `chown postgres` fails with EPERM and initdb then refuses the directory, leaving a dead cluster. The guest's postgres account is renumbered to the host owner instead — and because that UID is usually 1000, already taken by the Alpine image's login user, the occupant is moved aside first. - bitmagnet locates config.yml relative to its working directory and offers no flag or environment variable to override that. OpenRC /etc/conf.d values are shell variables sourced by the init script, not exported, so credentials placed there were never seen: bitmagnet fell back to its defaults, connected as "postgres" with no password, and panicked on every start while OpenRC reported "started". The service now sets a working directory holding a 0600 config, and is supervised so a startup race with PostgreSQL respawns rather than leaving a permanently dead crawler. - cloud-init renders each single-line runcmd as a bare YAML scalar, so a command starting with "[" parses as a flow sequence and invalidates the entire user-data document. cloud-init then discards every module and the guest boots with no packages, no SSH keys and no services, leaving one warning line in the serial log as the only evidence. TestBitmagnetUserData IsValidYAML renders the real cloud-init and parses it to catch this class; it caught a second instance (`cut -d:`) during development. Wired into `vee create` (--pg-data-dir, --wg-conf), the MCP server, CLI help, README and the docs site. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcuEg4e4xzsUkGA39sh4VC
Adds --nordvpn-token (and optional --nordvpn-country) to the bitmagnet template, so NordVPN users no longer have to export a NordLynx config from the dashboard by hand. NordLynx is WireGuard, so a Nord account already has everything a wg0.conf needs. The two halves come from different places and are assembled in vpn.NordLynxConfig: the account's NordLynx private key from an authenticated endpoint (HTTP Basic with the literal username "token"), and the server's public key, address and port from the public, load-aware recommendations endpoint. The endpoint is recorded as the server's IP rather than its hostname, because the guest's kill-switch pins its handshake hole to resolved addresses and there is no DNS left once OUTPUT is denied. An unknown --nordvpn-country is an error rather than a silent fallback: connecting through a different jurisdiction than the one requested is exactly the surprise this template exists to avoid. Also fixes vpn.ValidateToken, which never validated anything. It fetched the public /servers/countries endpoint and discarded the response, so it reported success for any non-empty string and the first sign of a bad token was a guest that could not connect. It now performs an authenticated request and reports rejection with the URL to generate a new token. This affects the torrent template's prompt as well, where the check has been silently passing bad tokens. Threading a context through templateExtras and the VPN prompts removed the need for the contextcheck suppression at the MCP call site. The credentials endpoint is the one NordVPN's own clients use, not a documented, versioned public API — it can change without notice, which is why the failure messages are specific and --wg-conf remains supported. Verified against the live API: server selection, country filtering, and token rejection. The authenticated success path is covered by tests against a stub server in the shapes observed live; it has not been run with a real token. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcuEg4e4xzsUkGA39sh4VC
Pointing --pg-data-dir at an NFS mount does not fail — it hangs, and it hangs in the worst possible place. PostgreSQL's initdb fsyncs thousands of small files during bootstrap. Over virtiofs onto NFS each one is a full network round trip: measured at 58 kB/s and ~70ms per 4k synchronous write on a real NAS, which turns a ten-second initdb into hours. NFS mounted "hard" (the default) never returns an error either, so the guest blocks rather than failing. The consequence is worse than a slow create. cloud-init stalls mid-initdb, so every runcmd after it never runs — including the firewall rules and the WireGuard tunnel. The VM boots, answers SSH, and looks created, while having no kill-switch at all. Observed on a real create: OUTPUT policy still ACCEPT, wg0 absent, cloud-init wedged in fuse_create_open three minutes in with the guest's initdb blocked on virtiofsd. The docs already said to use local storage. That was not enough, because the failure gives no signal at the moment the mistake is made — so the check now lives in the code, at create time, before the VM or any credentials are touched. It rejects NFS, SMB/CIFS, 9p, Ceph, GlusterFS and FUSE by statfs magic, on both the CLI and MCP paths, in 8ms. An undetectable filesystem is treated as local: failing a create on a statfs error would be a worse outcome than proceeding. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcuEg4e4xzsUkGA39sh4VC
… guest Two separate bugs, both of which made `vee tunnel` unusable against Alpine templates. First, runSSHTunnel read cfg.CloudInit.User directly instead of resolving the account the way vee ssh does. That misses the explicit ssh_user override, and it misses every template that creates no user of its own and injects the SSH keys into the image's default account instead (docker, dns-sink, bitmagnet — the Alpine image ships no bash for useradd). For those the field is empty, so ssh(1) substituted the host username and the tunnel failed with "Permission denied (publickey)" against an account that never existed in the guest — while vee ssh to the same VM worked, which made it look like a key problem rather than a username one. Resolution now goes through cfg.SSHUsername(), extracted into tunnelSSHUser so it can be tested against vee ssh for agreement. Second, the Alpine cloud image ships "AllowTcpForwarding no", so sshd refused every forward and connections reset the moment they were used. The bitmagnet template depends on forwarding: the web UI is deliberately given no firewall hole, so `vee tunnel` over SSH is the only way to reach it, and without this it was unreachable by any means. A drop-in sets AllowTcpForwarding yes, validated with sshd -t before reloading — a bad config would leave sshd dead and a kill-switched guest unrecoverable. Verified against a live guest: the tunnel now returns bitmagnet's UI (301 to /webui) where it previously reset the connection. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcuEg4e4xzsUkGA39sh4VC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
bitmagnettemplate: a minimal Alpine VM running the bitmagnet BitTorrent DHT crawler and indexer with its own PostgreSQL database. No Docker, no Compose, fully headless — 2G / 2 CPUs.Why a template
Crawling the DHT continuously announces the guest to the swarm, so running bitmagnet on a bare host publishes that host's IP address to tens of thousands of peers. Putting it in a VM behind an enforced tunnel is the point.
The guarantee
With WireGuard down, no software in the guest can reach the internet — not just bitmagnet, anything, as any user.
OUTPUTpolicy isDROP, and the exceptions are pinned rather than opened by port, because a default-deny policy is only as strong as its holes:-o wg0— the only unrestricted egress, and only while wg0 is up--sport 22and conntrackESTABLISHED— cannot open new connections--sport 68 --dport 67to the broadcast address onlyBoth unpinned forms are usable covert channels with the tunnel down. The UDP one was verified to leak before pinning:
The endpoint is resolved before
OUTPUTcloses and cached to/etc/wireguard/endpoint-addrs— afterwards there is no DNS left to resolve a hostname with. As a second layer, bitmagnet's own traffic is rejected on any interface other thanwg0by group ownership, placed after the wg0 accept so tunnelled traffic is unaffected.Verified on a live guest with wg0 down:
SSH survived throughout.
No VPN → no crawling
Declining the VPN disables the crawler rather than warning about it — the damage is done long before anyone reads a log line. Both
dht_crawler.enabledand the--keys=dht_crawlerworker key, since either alone still joins the swarm. Verified on a real boot: no UDP socket bound on 3334, zero peer connections, zero torrents indexed, while PostgreSQL and the web UI ran normally with all 226 tables migrated.Web UI
Port 3333 gets no firewall hole from any source. Reached over
vee tunnel.Index outlives the VM
--pg-data-dir <host dir>bind-mounts a host directory as PostgreSQL's data directory over virtiofs. Verified end to end: crawled 169 torrents, destroyed the VM, recreated it against the same directory — index intact,initdbcorrectly skipped.Guest-environment findings
Three non-obvious things, all found by booting rather than by tests:
chown postgreson the share fails withEPERMandinitdbthen refuses the directory, leaving a dead cluster. The guest's postgres account is renumbered to the host owner instead — and because that UID is usually 1000, already taken by Alpine's login user, the occupant is moved aside first./etc/conf.dvalues are not exported. They are shell variables sourced by the init script, so bitmagnet never saw the credentials, fell back to its defaults, connected aspostgreswith no password and panicked on every start — while OpenRC reportedstarted. bitmagnet also locatesconfig.ymlrelative to its working directory with no flag to override it, so the service now sets a working directory holding a0600config, and is supervised so a startup race with PostgreSQL respawns.[invalidates all of cloud-init. It parses as a YAML flow sequence, breaking the whole user-data document; cloud-init discards every module and the guest boots with no packages, no SSH keys and no services, leaving one warning line in the serial log.TestBitmagnetUserDataIsValidYAMLrenders the real cloud-init and parses it — it caught a second instance (cut -d:) during development.Testing
15 unit tests covering kill-switch narrowness and ordering, crawler gating, the initdb reuse guard, UID adoption, loopback-only Postgres, and cloud-init YAML validity. Each security-relevant test was confirmed to fail when its protection is removed. Lint clean, full suite passes, Windows cross-build OK.
Not covered
The
--wg-conffirst boot — wherewg-quick up wg0runs behindOUTPUT DROPwith the handshake pinned to a resolved address — has not been watched end to end; no WireGuard config exists on the dev host. If endpoint resolution ever returns nothing the tunnel cannot come up and the VM stays reachable by SSH but silent, which is the safe direction, but it is worth watching on a first real create.🤖 Generated with Claude Code
https://claude.ai/code/session_01WcuEg4e4xzsUkGA39sh4VC