Skip to content

Commit 49d8fe3

Browse files
TMHSDigitalclaude
andcommitted
feat(dns): DNS over TLS and DNS over HTTPS (-dns-server tls://, https://)
-dns-server now accepts tls://host[:853] and https://host/path. The resolver's Dial hook returns a TLS connection (Go already frames DNS over any stream conn exactly as RFC 7858 requires) or an RFC 8484 adapter that POSTs each framed query. DoT connections are pooled with a 5s idle limit so most queries skip the handshake; DoH uses one keep-alive, HTTP/2-capable client per resolver. One dial is still one query and the RCODE sniffer wraps both, so records, outcome classification (REFUSED, timeouts) and -rate accounting are identical to UDP. The run-quality report gains a transport field. internal/dnstest gains a DoT listener with a generated certificate, a DoH handler and a connection counter, and now answers several queries per stream connection. Two fixes found along the way: a dropped query on a stream now leaves the connection open (closing it read as an error, not a timeout), and UDP is bound before TCP so Windows' reserved UDP port ranges no longer fail server start-up. Fixes #86 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent e2b5667 commit 49d8fe3

13 files changed

Lines changed: 574 additions & 24 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010
### Added
11+
- Encrypted DNS transports: `-dns-server tls://host[:853]` (DNS over TLS, with pooled connections) and `-dns-server https://host/path` (DNS over HTTPS, RFC 8484, keep-alive and HTTP/2). Records, outcome classification (including REFUSED and timeouts) and `-rate` accounting are identical to UDP, and the run-quality report gains a `transport` field (#86).
1112
- Resumable scans: an interrupted run (Ctrl+C, SIGTERM) saves its state to `subenum-resume.json` (or `-state <file>`), and `subenum -resume <file>` replays the original command line, checks the input files are unchanged, restores the earlier results and continues from the first wordlist entry not yet fully looked up. The scan engine tracks that low-water mark exactly even though workers finish out of order, so at most a worker pool's worth of lookups is repeated (#73).
1213
- Permutation mode: `-permute` follows each target's scan with a second pass over permutations of the names found (word joins such as `api-dev`, `dev-api` and `dev.api`, and number increments), and `-seeds <file>` adds seeds from an earlier run in any format. The pass is fully guarded (preflight, wildcard filtering, `-rate`, `-max-queries`, `-exclude`), skips wordlist entries, marks its hits `"permutation": true`, and reports seeds, candidates and hits in the run-quality report (#72).
1314
- Defaults from a config file and the environment: `config.json` in the user config directory (or `$SUBENUM_CONFIG`) and `SUBENUM_<FLAG>` variables, with precedence flag > env > config > built-in. `-print-config` shows each effective setting and its source. `subenum completion bash|zsh|fish|powershell` and `subenum man` generate shell completions and a man page from the flags, and release archives ship them (#88).

‎README.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -367,6 +367,17 @@ The resumed run refuses to start if an input file changed, re-emits the earlier
367367

368368
With `-dL`, each domain is scanned in turn as an independent scan: `-max-queries`, `-rate` and the reliability guard apply per target. A failed target does not stop the others, but a reliability abort (the resolver looks overloaded) skips the remaining targets unless `-no-abort` is set. A per-target status list is printed at the end.
369369

370+
### Encrypted DNS: DoT and DoH
371+
372+
Where outbound UDP/53 is blocked or intercepted, point `-dns-server` at an encrypted resolver:
373+
374+
```bash
375+
subenum -dns-server tls://1.1.1.1 example.com # DNS over TLS (port 853)
376+
subenum -dns-server https://cloudflare-dns.com/dns-query example.com # DNS over HTTPS
377+
```
378+
379+
DoT connections are pooled and reused, and DoH uses one keep-alive HTTP client (HTTP/2 where the server offers it), so most queries skip the TLS handshake. Outcome classification and `-rate` accounting are identical to UDP, and the run-quality report names the `transport`. `-r` resolver pools are plain UDP; with `-r`, an encrypted `-dns-server` is the trusted resolver that re-validates every hit.
380+
370381
### Defaults, completions and the man page
371382

372383
Settings you repeat on every run can live in a config file or the environment. Precedence, highest first: **command-line flag > `SUBENUM_*` environment variable > config file > built-in default.**

‎docs/ROADMAP.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,6 @@ Ideas we want, with no date attached.
3434

3535
- A stable Go library API, `pkg/subenum` ([#77](https://github.com/TMHSDigital/subenum/issues/77)).
3636
- Try AXFR and detect NSEC-walkable zones before brute-forcing ([#85](https://github.com/TMHSDigital/subenum/issues/85)).
37-
- DNS-over-TLS and DNS-over-HTTPS transports ([#86](https://github.com/TMHSDigital/subenum/issues/86)).
3837
- A teaching / lab mode built on `-simulate` and the TUI ([#76](https://github.com/TMHSDigital/subenum/issues/76)).
3938
- Community and discoverability: Discussions, devcontainer, demo GIF, launch posts
4039
([#75](https://github.com/TMHSDigital/subenum/issues/75), [#74](https://github.com/TMHSDigital/subenum/issues/74)).

‎docs/_includes/flags.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
| `-dL <string>` | - | File of apex domains to scan, one per line (- for stdin); replaces the &lt;domain&gt; argument |
77
| `-depth <int>` | 1 | Max recursion depth when -recursive is set (1 = no recursion) |
88
| `-diff <string>` | - | Previous results file (any -format); report only names added or removed since then, and exit 4 when there are changes |
9-
| `-dns-server <string>` | 8.8.8.8:53 | DNS server to use (format: ip:port) |
9+
| `-dns-server <string>` | 8.8.8.8:53 | DNS server: ip:port (UDP, TCP fallback), tls://host[:853] (DNS over TLS) or https://host/path (DNS over HTTPS) |
1010
| `-exclude <string>` | - | Comma-separated out-of-scope names and \*.parent patterns; never queried or expanded |
1111
| `-exclude-file <string>` | - | File of out-of-scope names and \*.parent patterns, one per line (# comments allowed) |
1212
| `-force` | - | Continue scanning even if wildcard DNS is detected |

‎internal/dns/resolver.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,7 @@ func ParseTypes(s string) ([]string, error) {
124124
// ResolveTypes, a dial beyond the slots reserved up front takes another rate
125125
// slot, which keeps -rate exact however many queries Go decides to send (#50).
126126
func NewResolver(timeout time.Duration, dnsServer string) *net.Resolver {
127+
dl := newDialer(timeout, dnsServer) // UDP/TCP, DoT or DoH (#86)
127128
return &net.Resolver{
128129
PreferGo: true,
129130
Dial: func(dialCtx context.Context, network, _ string) (net.Conn, error) {
@@ -139,8 +140,7 @@ func NewResolver(timeout time.Duration, dnsServer string) *net.Resolver {
139140
}
140141
}
141142
}
142-
d := net.Dialer{Timeout: timeout}
143-
c, err := d.DialContext(dialCtx, network, dnsServer)
143+
c, err := dl.dial(dialCtx, network)
144144
if err != nil || b == nil {
145145
return c, err
146146
}

‎internal/dns/transport.go‎

Lines changed: 260 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,260 @@
1+
package dns
2+
3+
import (
4+
"bytes"
5+
"context"
6+
"crypto/tls"
7+
"crypto/x509"
8+
"encoding/binary"
9+
"errors"
10+
"fmt"
11+
"io"
12+
"net"
13+
"net/http"
14+
"net/url"
15+
"strings"
16+
"sync"
17+
"time"
18+
)
19+
20+
// Transports a -dns-server can use (#86).
21+
const (
22+
TransportUDP = "udp" // ip:port; UDP with TCP fallback on truncation
23+
TransportTLS = "tls" // tls://host[:853]; DNS over TLS (RFC 7858)
24+
TransportHTTPS = "https" // https://host/path; DNS over HTTPS (RFC 8484)
25+
)
26+
27+
// Transport names the transport a -dns-server value selects.
28+
func Transport(server string) string {
29+
switch {
30+
case strings.HasPrefix(server, "tls://"):
31+
return TransportTLS
32+
case strings.HasPrefix(server, "https://"):
33+
return TransportHTTPS
34+
}
35+
return TransportUDP
36+
}
37+
38+
// testRootCAs, when set by tests, replaces the system roots for DoT and DoH.
39+
var testRootCAs *x509.CertPool
40+
41+
// dialer opens one connection per DNS query, the way Go's resolver expects.
42+
// Go frames queries with a two-byte length prefix on any connection that is
43+
// not a net.PacketConn, which is exactly the DoT wire format; DoH adapts the
44+
// same framing to HTTP requests.
45+
type dialer interface {
46+
dial(ctx context.Context, network string) (net.Conn, error)
47+
}
48+
49+
// newDialer returns the dialer for a -dns-server value. It is validated
50+
// earlier (validate.DNSServer), so errors here fall back to UDP.
51+
func newDialer(timeout time.Duration, server string) dialer {
52+
switch Transport(server) {
53+
case TransportTLS:
54+
host := strings.TrimPrefix(server, "tls://")
55+
if _, _, err := net.SplitHostPort(host); err != nil {
56+
host = net.JoinHostPort(host, "853")
57+
}
58+
name, _, _ := net.SplitHostPort(host)
59+
return &dotDialer{
60+
addr: host,
61+
tls: &tls.Config{ServerName: name, RootCAs: testRootCAs, MinVersion: tls.VersionTLS12},
62+
timeout: timeout,
63+
idle: make(chan idleConn, 16),
64+
}
65+
case TransportHTTPS:
66+
return &dohDialer{
67+
url: server,
68+
client: &http.Client{Transport: &http.Transport{
69+
TLSClientConfig: &tls.Config{RootCAs: testRootCAs, MinVersion: tls.VersionTLS12},
70+
ForceAttemptHTTP2: true,
71+
MaxIdleConnsPerHost: 64,
72+
IdleConnTimeout: 30 * time.Second,
73+
}},
74+
}
75+
}
76+
return &udpDialer{addr: server, timeout: timeout}
77+
}
78+
79+
type udpDialer struct {
80+
addr string
81+
timeout time.Duration
82+
}
83+
84+
func (d *udpDialer) dial(ctx context.Context, network string) (net.Conn, error) {
85+
nd := net.Dialer{Timeout: d.timeout}
86+
return nd.DialContext(ctx, network, d.addr)
87+
}
88+
89+
// dotIdleMax bounds how long a DoT connection is kept for reuse. Servers
90+
// close idle connections after some seconds; reusing one they already closed
91+
// would fail a query, so connections older than this are discarded.
92+
const dotIdleMax = 5 * time.Second
93+
94+
type idleConn struct {
95+
conn *tls.Conn
96+
since time.Time
97+
}
98+
99+
// dotDialer keeps a small pool of TLS connections so most queries skip the
100+
// handshake.
101+
type dotDialer struct {
102+
addr string
103+
tls *tls.Config
104+
timeout time.Duration
105+
idle chan idleConn
106+
}
107+
108+
func (d *dotDialer) dial(ctx context.Context, _ string) (net.Conn, error) {
109+
for {
110+
select {
111+
case ic := <-d.idle:
112+
if time.Since(ic.since) > dotIdleMax {
113+
_ = ic.conn.Close()
114+
continue
115+
}
116+
return &pooledConn{Conn: ic.conn, pool: d}, nil
117+
default:
118+
}
119+
break
120+
}
121+
td := tls.Dialer{NetDialer: &net.Dialer{Timeout: d.timeout}, Config: d.tls}
122+
c, err := td.DialContext(ctx, "tcp", d.addr)
123+
if err != nil {
124+
return nil, err
125+
}
126+
return &pooledConn{Conn: c.(*tls.Conn), pool: d}, nil
127+
}
128+
129+
// pooledConn returns its TLS connection to the pool on Close, unless a read
130+
// or write failed (the stream may then hold a partial message).
131+
type pooledConn struct {
132+
*tls.Conn
133+
pool *dotDialer
134+
bad bool
135+
}
136+
137+
func (c *pooledConn) Read(p []byte) (int, error) {
138+
n, err := c.Conn.Read(p)
139+
if err != nil {
140+
c.bad = true
141+
}
142+
return n, err
143+
}
144+
145+
func (c *pooledConn) Write(p []byte) (int, error) {
146+
n, err := c.Conn.Write(p)
147+
if err != nil {
148+
c.bad = true
149+
}
150+
return n, err
151+
}
152+
153+
func (c *pooledConn) Close() error {
154+
if c.bad {
155+
return c.Conn.Close()
156+
}
157+
_ = c.SetDeadline(time.Time{})
158+
select {
159+
case c.pool.idle <- idleConn{conn: c.Conn, since: time.Now()}:
160+
return nil
161+
default:
162+
return c.Conn.Close()
163+
}
164+
}
165+
166+
// dohDialer turns each query into an HTTPS POST (RFC 8484). The HTTP client
167+
// keeps connections alive (HTTP/2 where the server offers it), so "dialing"
168+
// is cheap.
169+
type dohDialer struct {
170+
url string
171+
client *http.Client
172+
}
173+
174+
func (d *dohDialer) dial(ctx context.Context, _ string) (net.Conn, error) {
175+
return &dohConn{ctx: ctx, d: d}, nil
176+
}
177+
178+
// dohConn accepts length-prefixed DNS messages on Write, exchanges each one
179+
// over HTTPS, and serves the length-prefixed answers on Read.
180+
type dohConn struct {
181+
ctx context.Context
182+
d *dohDialer
183+
mu sync.Mutex
184+
in, out bytes.Buffer
185+
deadline time.Time
186+
}
187+
188+
func (c *dohConn) Write(p []byte) (int, error) {
189+
c.mu.Lock()
190+
defer c.mu.Unlock()
191+
c.in.Write(p)
192+
for c.in.Len() >= 2 {
193+
size := int(binary.BigEndian.Uint16(c.in.Bytes()[:2]))
194+
if c.in.Len() < 2+size {
195+
break
196+
}
197+
msg := append([]byte(nil), c.in.Next(2 + size)[2:]...)
198+
answer, err := c.exchange(msg)
199+
if err != nil {
200+
return 0, err
201+
}
202+
c.out.Write(binary.BigEndian.AppendUint16(nil, uint16(len(answer)))) //nolint:gosec // DNS messages are under 64 KiB
203+
c.out.Write(answer)
204+
}
205+
return len(p), nil
206+
}
207+
208+
func (c *dohConn) exchange(msg []byte) ([]byte, error) {
209+
ctx := c.ctx
210+
if !c.deadline.IsZero() {
211+
var cancel context.CancelFunc
212+
ctx, cancel = context.WithDeadline(ctx, c.deadline)
213+
defer cancel()
214+
}
215+
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.d.url, bytes.NewReader(msg))
216+
if err != nil {
217+
return nil, err
218+
}
219+
req.Header.Set("Content-Type", "application/dns-message")
220+
req.Header.Set("Accept", "application/dns-message")
221+
resp, err := c.d.client.Do(req)
222+
if err != nil {
223+
var ue *url.Error
224+
if errors.As(err, &ue) && ue.Timeout() {
225+
return nil, &net.DNSError{Err: "i/o timeout", IsTimeout: true}
226+
}
227+
return nil, err
228+
}
229+
defer func() { _ = resp.Body.Close() }()
230+
if resp.StatusCode != http.StatusOK {
231+
return nil, fmt.Errorf("DNS-over-HTTPS server returned %s", resp.Status)
232+
}
233+
return io.ReadAll(io.LimitReader(resp.Body, 65535))
234+
}
235+
236+
func (c *dohConn) Read(p []byte) (int, error) {
237+
c.mu.Lock()
238+
defer c.mu.Unlock()
239+
if c.out.Len() == 0 {
240+
return 0, io.EOF
241+
}
242+
return c.out.Read(p)
243+
}
244+
245+
func (c *dohConn) Close() error { return nil }
246+
func (c *dohConn) LocalAddr() net.Addr { return dohAddr{} }
247+
func (c *dohConn) RemoteAddr() net.Addr { return dohAddr{} }
248+
func (c *dohConn) SetReadDeadline(time.Time) error { return nil }
249+
func (c *dohConn) SetWriteDeadline(t time.Time) error { return c.SetDeadline(t) }
250+
func (c *dohConn) SetDeadline(t time.Time) error {
251+
c.mu.Lock()
252+
c.deadline = t
253+
c.mu.Unlock()
254+
return nil
255+
}
256+
257+
type dohAddr struct{}
258+
259+
func (dohAddr) Network() string { return "https" }
260+
func (dohAddr) String() string { return "doh" }

0 commit comments

Comments
 (0)