Skip to content

Commit 3057e1e

Browse files
TMHSDigitalclaude
andcommitted
fix(dns): DoT pool liveness, DoH redirect/throttle handling, IPv6 tls://
- DoT: probe a pooled connection before reuse so one the server closed no longer wastes a counted, paced query; keep up to 128 idle connections; close idle DoT/DoH connections when a scan ends (dns.CloseIdleConnections, weakly registered per resolver). - DoH: never follow redirects (no cleartext downgrade); HTTP 429/503 classify as refused. - tls://[IPv6] without a port is accepted; validate and the dialer share validate.DoTAddress. Lab zone CNAME targets are validated and "@" means the scanned domain. - dnsserver.Close tracks and closes stream connections and waits for every handler; CloseStreamsAfter for tests. New tests for untrusted DoT certs, DoH redirects/throttling, stale pooled connections and the server lifecycle. Fixes #113 Fixes #114 Fixes #115 Fixes #122 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent f6bedf2 commit 3057e1e

12 files changed

Lines changed: 456 additions & 29 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
3030
- The minimum Go version is now 1.26 (`go 1.26.0` in go.mod), required by current `golang.org/x/sys` and `golang.org/x/text`; both were bumped and `govulncheck -show verbose` now reports no vulnerable modules. Dependabot also updates indirect modules, and CI lints with errorlint, nilerr, contextcheck, noctx, copyloopvar, usestdlibvars and revive, with errcheck exemptions narrowed to stdout prints (#83).
3131

3232
### Fixed
33+
- DoT connection reuse is accurate: a pooled connection the server already closed is detected before reuse instead of failing a query that had already been counted and paced (a server closing after each answer cost 19 dials for 10 lookups; now 10). Up to 128 connections are kept, matching the default `-t`, and a scan closes its idle DoT and DoH connections when it ends, so a program that scans repeatedly through `pkg/subenum` does not hold them open (#113).
34+
- DNS over HTTPS never follows a redirect, so a misconfigured endpoint or proxy cannot move the query elsewhere or downgrade it to cleartext `http://`; HTTP 429 and 503 count as `refused` (the rate-limit signal) instead of `other` (#114).
35+
- `-dns-server tls://[2001:db8::1]` (a bracketed IPv6 host without a port) is accepted; the validator and the dialer now share one parser. Lab zone CNAME targets are validated, and `CNAME @` points at the scanned domain instead of the bogus `@.<domain>` (#115).
36+
- The test DNS server's `Close` waits for every handler, closes open stream connections, and no longer returns while delayed UDP answers or connections parked on a dropped query are still running. New tests cover an untrusted DoT certificate, DoH redirects and throttling, stale pooled connections, a lying pool member and the server's own lifecycle (#122).
3337
- `pkg/subenum.Run` no longer leaks the scan's worker pool when a caller breaks out of the event loop and cancels without draining (about 100 goroutines per run before): after cancellation the forwarder never waits for a reader, so every goroutine exits within about a second. `DefaultWords` parses the bundled list once, and a positive `Timeout` below 1ms is rounded up instead of being rejected as 0 (#112).
3438
- A resolver pool member that answers NXDOMAIN for names that exist can no longer silently hide them. Each member is checked with canaries (names the trusted resolver confirmed, starting with the apex), and an NXDOMAIN from a member that has not passed a canary is confirmed by a second member. A member contradicted three times (a hidden name, a failed canary, or a hit the trusted resolver denies) is flagged and dropped for the rest of the run; the per-resolver report gains `canaries`, `contradicted` and `flagged`. In a hermetic test, one always-NXDOMAIN member previously hid half the real names; now none are lost. In a zone with no known name to use as a canary, NXDOMAIN answers cost a second pool query until one is found (#105).
3539
- `-diff` no longer reports names this run never tested as removed. Every previous name the run did not find is looked up again with the trusted resolver, and only NXDOMAIN counts as removed; names that still resolve (a smaller wordlist, no `-recursive` or `-permute`, now wildcard-filtered) or whose lookup fails are counted as `still_resolving` and `unverified` in the report's `diff` object instead (#110).

‎internal/dns/resolver.go‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ func ParseTypes(s string) ([]string, error) {
125125
// slot, which keeps -rate exact however many queries Go decides to send (#50).
126126
func NewResolver(timeout time.Duration, dnsServer string) *net.Resolver {
127127
dl := newDialer(timeout, dnsServer) // UDP/TCP, DoT or DoH (#86)
128-
return &net.Resolver{
128+
r := &net.Resolver{
129129
PreferGo: true,
130130
Dial: func(dialCtx context.Context, network, _ string) (net.Conn, error) {
131131
if c, ok := dialCtx.Value(queryCounterKey{}).(*atomic.Int64); ok {
@@ -152,6 +152,8 @@ func NewResolver(timeout time.Duration, dnsServer string) *net.Resolver {
152152
return &rcodeTCPConn{Conn: c, b: b}, nil
153153
},
154154
}
155+
registerDialer(r, dl)
156+
return r
155157
}
156158

157159
// wireBudget follows one per-type lookup through Go's resolver: the rate

‎internal/dns/transport.go‎

Lines changed: 113 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,13 @@ import (
1212
"net"
1313
"net/http"
1414
"net/url"
15+
"runtime"
1516
"strings"
1617
"sync"
1718
"time"
19+
"weak"
20+
21+
"github.com/TMHSDigital/subenum/internal/validate"
1822
)
1923

2024
// Transports a -dns-server can use (#86).
@@ -46,31 +50,79 @@ type dialer interface {
4650
dial(ctx context.Context, network string) (net.Conn, error)
4751
}
4852

53+
// idleCloser is a dialer that keeps connections between queries.
54+
type idleCloser interface {
55+
closeIdle()
56+
}
57+
58+
// pooledDialers maps a resolver from NewResolver to its DoT or DoH dialer,
59+
// so CloseIdleConnections can find it. Keys are weak, and an entry is
60+
// removed (and its connections closed) once the resolver is collected.
61+
var pooledDialers sync.Map // weak.Pointer[net.Resolver] -> idleCloser
62+
63+
func registerDialer(r *net.Resolver, d dialer) {
64+
c, ok := d.(idleCloser)
65+
if !ok {
66+
return
67+
}
68+
key := weak.Make(r)
69+
pooledDialers.Store(key, c)
70+
runtime.AddCleanup(r, func(k weak.Pointer[net.Resolver]) {
71+
if v, ok := pooledDialers.LoadAndDelete(k); ok {
72+
v.(idleCloser).closeIdle()
73+
}
74+
}, key)
75+
}
76+
77+
// CloseIdleConnections closes the DoT connections and idle DoH connections
78+
// a resolver from NewResolver keeps for reuse. The resolver stays usable. A
79+
// scan calls it when it ends, so a long-lived program that scans repeatedly
80+
// does not hold connections open between scans.
81+
func CloseIdleConnections(r *net.Resolver) {
82+
if r == nil {
83+
return
84+
}
85+
if v, ok := pooledDialers.Load(weak.Make(r)); ok {
86+
v.(idleCloser).closeIdle()
87+
}
88+
}
89+
90+
// dotIdleConns bounds the DoT connections kept for reuse; it matches the
91+
// default -t, so a default scan rarely closes one only to dial it again.
92+
const dotIdleConns = 128
93+
4994
// newDialer returns the dialer for a -dns-server value. It is validated
5095
// earlier (validate.DNSServer), so errors here fall back to UDP.
5196
func newDialer(timeout time.Duration, server string) dialer {
5297
switch Transport(server) {
5398
case TransportTLS:
54-
host := strings.TrimPrefix(server, "tls://")
55-
if _, _, err := net.SplitHostPort(host); err != nil {
56-
host = net.JoinHostPort(host, "853")
99+
addr, name, err := validate.DoTAddress(server)
100+
if err != nil {
101+
break
57102
}
58-
name, _, _ := net.SplitHostPort(host)
59103
return &dotDialer{
60-
addr: host,
104+
addr: addr,
61105
tls: &tls.Config{ServerName: name, RootCAs: testRootCAs, MinVersion: tls.VersionTLS12},
62106
timeout: timeout,
63-
idle: make(chan idleConn, 16),
107+
idle: make(chan idleConn, dotIdleConns),
64108
}
65109
case TransportHTTPS:
66110
return &dohDialer{
67111
url: server,
68-
client: &http.Client{Transport: &http.Transport{
69-
TLSClientConfig: &tls.Config{RootCAs: testRootCAs, MinVersion: tls.VersionTLS12},
70-
ForceAttemptHTTP2: true,
71-
MaxIdleConnsPerHost: 64,
72-
IdleConnTimeout: 30 * time.Second,
73-
}},
112+
client: &http.Client{
113+
Transport: &http.Transport{
114+
TLSClientConfig: &tls.Config{RootCAs: testRootCAs, MinVersion: tls.VersionTLS12},
115+
ForceAttemptHTTP2: true,
116+
MaxIdleConnsPerHost: 64,
117+
IdleConnTimeout: 30 * time.Second,
118+
},
119+
// A redirect could send the query somewhere else, or in
120+
// cleartext over http://; a DoH server has no reason to
121+
// redirect, so none is followed (#114).
122+
CheckRedirect: func(req *http.Request, _ []*http.Request) error {
123+
return fmt.Errorf("DNS-over-HTTPS server redirected to %s; refusing to follow", req.URL.Redacted())
124+
},
125+
},
74126
}
75127
}
76128
return &udpDialer{addr: server, timeout: timeout}
@@ -88,8 +140,13 @@ func (d *udpDialer) dial(ctx context.Context, network string) (net.Conn, error)
88140

89141
// dotIdleMax bounds how long a DoT connection is kept for reuse. Servers
90142
// close idle connections after some seconds; reusing one they already closed
91-
// would fail a query, so connections older than this are discarded.
92-
const dotIdleMax = 5 * time.Second
143+
// would fail a query, so connections older than this are discarded. A
144+
// younger one is still probed before reuse (see alive).
145+
const dotIdleMax = 4 * time.Second
146+
147+
// dotProbe is how long alive waits for a pooled connection to show it was
148+
// closed. A closed one reports EOF at once; a live one has nothing to read.
149+
const dotProbe = time.Millisecond
93150

94151
type idleConn struct {
95152
conn *tls.Conn
@@ -109,7 +166,10 @@ func (d *dotDialer) dial(ctx context.Context, _ string) (net.Conn, error) {
109166
for {
110167
select {
111168
case ic := <-d.idle:
112-
if time.Since(ic.since) > dotIdleMax {
169+
// A connection the server already closed would fail this query
170+
// after it had been counted and paced, so it is checked first
171+
// (#113).
172+
if time.Since(ic.since) > dotIdleMax || !alive(ic.conn) {
113173
_ = ic.conn.Close()
114174
continue
115175
}
@@ -126,6 +186,35 @@ func (d *dotDialer) dial(ctx context.Context, _ string) (net.Conn, error) {
126186
return &pooledConn{Conn: c.(*tls.Conn), pool: d}, nil
127187
}
128188

189+
// alive reports whether an idle DoT connection is still open. Between
190+
// queries a server sends nothing, so a read that times out means open, and
191+
// EOF, an error or unexpected data means it must not be reused. A read
192+
// timeout leaves a tls.Conn usable.
193+
func alive(c *tls.Conn) bool {
194+
if err := c.SetReadDeadline(time.Now().Add(dotProbe)); err != nil {
195+
return false
196+
}
197+
var b [1]byte
198+
_, err := c.Read(b[:])
199+
var ne net.Error
200+
if !errors.As(err, &ne) || !ne.Timeout() {
201+
return false
202+
}
203+
return c.SetReadDeadline(time.Time{}) == nil
204+
}
205+
206+
// closeIdle closes every pooled connection.
207+
func (d *dotDialer) closeIdle() {
208+
for {
209+
select {
210+
case ic := <-d.idle:
211+
_ = ic.conn.Close()
212+
default:
213+
return
214+
}
215+
}
216+
}
217+
129218
// pooledConn returns its TLS connection to the pool on Close, unless a read
130219
// or write failed (the stream may then hold a partial message).
131220
type pooledConn struct {
@@ -175,6 +264,8 @@ func (d *dohDialer) dial(ctx context.Context, _ string) (net.Conn, error) {
175264
return &dohConn{ctx: ctx, d: d}, nil
176265
}
177266

267+
func (d *dohDialer) closeIdle() { d.client.CloseIdleConnections() }
268+
178269
// dohConn accepts length-prefixed DNS messages on Write, exchanges each one
179270
// over HTTPS, and serves the length-prefixed answers on Read.
180271
type dohConn struct {
@@ -227,7 +318,13 @@ func (c *dohConn) exchange(msg []byte) ([]byte, error) {
227318
return nil, err
228319
}
229320
defer func() { _ = resp.Body.Close() }()
230-
if resp.StatusCode != http.StatusOK {
321+
switch resp.StatusCode {
322+
case http.StatusOK:
323+
case http.StatusTooManyRequests, http.StatusServiceUnavailable:
324+
// The HTTP form of REFUSED: the server is shedding load. Worded so
325+
// Classify counts it as refused, the rate-limit signal (#114).
326+
return nil, fmt.Errorf("DNS-over-HTTPS server refused the query (HTTP %s)", resp.Status)
327+
default:
231328
return nil, fmt.Errorf("DNS-over-HTTPS server returned %s", resp.Status)
232329
}
233330
return io.ReadAll(io.LimitReader(resp.Body, 65535))

‎internal/dns/transport_test.go‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,11 @@ package dns
22

33
import (
44
"context"
5+
"crypto/x509"
6+
"net/http"
57
"net/http/httptest"
68
"slices"
9+
"sync/atomic"
710
"testing"
811
"time"
912

@@ -120,3 +123,102 @@ func TestDoTReusesConnections(t *testing.T) {
120123
t.Fatalf("server saw %d A queries, want 20", n)
121124
}
122125
}
126+
127+
// TestDoTDiscardsClosedConnections covers #113: a server that closes each
128+
// connection after one answer costs one dial per query, not two, because a
129+
// pooled connection the server closed is detected before it is reused.
130+
func TestDoTDiscardsClosedConnections(t *testing.T) {
131+
srv, servers := startAllTransports(t)
132+
srv.CloseStreamsAfter(1)
133+
r := NewResolver(2*time.Second, servers[TransportTLS])
134+
var sent atomic.Int64
135+
ctx := WithQueryCounter(context.Background(), &sent)
136+
before := srv.Connections()
137+
for i := 0; i < 10; i++ {
138+
if _, _, err := ResolveTypes(ctx, r, "www.example.com", 2*time.Second, []string{"A"}); err != nil {
139+
t.Fatal(err)
140+
}
141+
time.Sleep(5 * time.Millisecond) // let the server's close arrive
142+
}
143+
if opened := srv.Connections() - before; opened != 10 {
144+
t.Errorf("10 lookups opened %d connections, want 10", opened)
145+
}
146+
if sent.Load() != 10 {
147+
t.Errorf("counted %d queries, want 10", sent.Load())
148+
}
149+
}
150+
151+
// TestCloseIdleConnections: a scan's DoT connections are closed when it
152+
// ends, and the resolver still works afterwards.
153+
func TestCloseIdleConnections(t *testing.T) {
154+
srv, servers := startAllTransports(t)
155+
r := NewResolver(2*time.Second, servers[TransportTLS])
156+
if _, _, err := ResolveTypes(context.Background(), r, "www.example.com", 2*time.Second, []string{"A"}); err != nil {
157+
t.Fatal(err)
158+
}
159+
CloseIdleConnections(r)
160+
before := srv.Connections()
161+
if _, _, err := ResolveTypes(context.Background(), r, "www.example.com", 2*time.Second, []string{"A"}); err != nil {
162+
t.Fatal(err)
163+
}
164+
if srv.Connections() == before {
165+
t.Error("the lookup after CloseIdleConnections reused a connection that should have been closed")
166+
}
167+
}
168+
169+
// TestDoHRefusesRedirects covers #114: a DoH server that redirects, even to
170+
// plain http://, never gets the query sent there.
171+
func TestDoHRefusesRedirects(t *testing.T) {
172+
var plainHits atomic.Int64
173+
plain := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { plainHits.Add(1) }))
174+
defer plain.Close()
175+
redirect := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
176+
http.Redirect(w, r, plain.URL+"/dns-query", http.StatusTemporaryRedirect)
177+
}))
178+
defer redirect.Close()
179+
trustOnly(t, redirect)
180+
181+
r := NewResolver(2*time.Second, redirect.URL+"/dns-query")
182+
if _, _, err := ResolveTypes(context.Background(), r, "www.example.com", 2*time.Second, []string{"A"}); err == nil {
183+
t.Fatal("lookup through a redirecting server succeeded")
184+
}
185+
if plainHits.Load() != 0 {
186+
t.Errorf("the query was sent in cleartext %d times", plainHits.Load())
187+
}
188+
}
189+
190+
// TestDoHThrottlingIsRefused covers #114: HTTP 429 and 503 count as
191+
// REFUSED, the rate-limit signal, not as "other".
192+
func TestDoHThrottlingIsRefused(t *testing.T) {
193+
for _, status := range []int{http.StatusTooManyRequests, http.StatusServiceUnavailable} {
194+
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(status) }))
195+
trustOnly(t, srv)
196+
r := NewResolver(2*time.Second, srv.URL+"/dns-query")
197+
_, _, err := ResolveTypes(context.Background(), r, "www.example.com", 2*time.Second, []string{"A"})
198+
srv.Close()
199+
if outcome := Classify(err); outcome != OutcomeRefused {
200+
t.Errorf("HTTP %d: outcome %v, want refused", status, outcome)
201+
}
202+
}
203+
}
204+
205+
// TestDoTRejectsUntrustedCertificate: certificate verification is on, so a
206+
// server whose certificate is not trusted never answers a lookup.
207+
func TestDoTRejectsUntrustedCertificate(t *testing.T) {
208+
_, servers := startAllTransports(t)
209+
testRootCAs = x509.NewCertPool() // trusts nothing; startAllTransports restores it
210+
r := NewResolver(2*time.Second, servers[TransportTLS])
211+
if _, _, err := ResolveTypes(context.Background(), r, "www.example.com", 2*time.Second, []string{"A"}); err == nil {
212+
t.Fatal("lookup over an untrusted DoT certificate succeeded")
213+
}
214+
}
215+
216+
// trustOnly makes DoT and DoH trust srv's certificate for this test.
217+
func trustOnly(t *testing.T, srv *httptest.Server) {
218+
t.Helper()
219+
pool := x509.NewCertPool()
220+
pool.AddCert(srv.Certificate())
221+
old := testRootCAs
222+
testRootCAs = pool
223+
t.Cleanup(func() { testRootCAs = old })
224+
}

0 commit comments

Comments
 (0)