Skip to content

Commit 0233ee2

Browse files
TMHSDigitalclaude
andcommitted
fix: filter wildcards served from a rotating address pool
The root wildcard fingerprint came from two probes, so a wildcard behind a CDN or load balancer that rotates through a pool slipped most answers past the subset filter even under -force. FingerprintWildcard now starts with five probes and, when their answers differ, keeps probing until the fingerprint stops growing (max(8, 4x size) stale probes, cap 128). A result that partly overlaps the fingerprint is re-validated with fresh random labels under its parent, whose answers join a shared fingerprint. Branch checks in recursion keep the two-probe check. 8-address pool, 50 wildcard names: 12 filtered before, 50 after; no real names lost. Fixes #48 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 0c53d37 commit 0233ee2

6 files changed

Lines changed: 225 additions & 50 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1919
- Target domains are normalized the same way in the positional argument, `-dL` and the TUI: trimmed, lowercased, trailing dot dropped, so `Example.COM.` no longer yields `api.Example.COM.`. A pasted URL, path, `:port` or leading `*.` is stripped with a one-line note, IDNs such as `bücher.de` are converted to punycode, underscores are accepted as in wordlist entries, and an email address gets a targeted hint instead of `invalid domain format` (#54).
2020
- TUI: a `-` wordlist is rejected on the form instead of freezing the UI on stdin, the wordlist loads in the background with a "Loading wordlist..." line so large lists no longer block input, a scan that ends with an error (preflight failure, wildcard abort, reliability abort) shows "Failed: <reason>" instead of "Aborted", and the TUI exits 1 when the last scan failed (#55).
2121
- `-rate` now charges every DNS query that reaches the wire. A CNAME lookup costs three slots (Go sends A, AAAA and CNAME queries), and queries Go's resolver retries internally on SERVFAIL, REFUSED or socket errors are charged as they are dialed, so a struggling resolver no longer gets more traffic than `-rate` allows. Responses with RCODE REFUSED are now counted as `refused` instead of `other`, and lookup errors name the configured `-dns-server` instead of the system nameserver (#50).
22+
- Wildcards that answer from a rotating CDN or load-balancer pool are now filtered under `-force`. The root fingerprint starts from five probes instead of two and, when their answers differ, keeps probing until the pool stops growing; results that share only some records with the fingerprint are re-validated with fresh probes under the same parent. In a hermetic 8-address pool test, filtering went from 12 of 50 wildcard answers to all 50, with no real names lost (#48).
2223
- Release binaries are built with `CGO_ENABLED=0 -trimpath -s -w`, so `subenum-linux-amd64` is statically linked again and CI fails if a Linux asset is not (#46).
2324

2425
## [0.8.0] - 2026-10-04

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -262,7 +262,7 @@ make help # list all targets
262262
<br>
263263

264264
> [!NOTE]
265-
> Wildcard DNS is detected automatically before scanning begins. If the target resolves wildcard records, or the wildcard probes fail so the check is inconclusive, the tool exits with a warning, since all subdomains would match, making results meaningless. Pass `-force` to override. With `-force`, results whose records are a subset of the wildcard fingerprint are still dropped and counted as `wildcard-filtered`. Recursive scans probe each new parent and skip expanding wildcard branches.
265+
> Wildcard DNS is detected automatically before scanning begins. If the target resolves wildcard records, or the wildcard probes fail so the check is inconclusive, the tool exits with a warning, since all subdomains would match, making results meaningless. Pass `-force` to override. With `-force`, results whose records are a subset of the wildcard fingerprint are still dropped and counted as `wildcard-filtered`. The fingerprint comes from several random probes, and when the wildcard answers from a rotating CDN or load-balancer pool, subenum keeps probing until it has learned the pool; a result that shares only some records with the fingerprint is re-checked against fresh probes. Recursive scans probe each new parent and skip expanding wildcard branches.
266266
267267
> [!CAUTION]
268268
> Simulation mode (`-simulate`) generates synthetic results and performs zero network I/O. Do not confuse simulated output with real DNS data. Simulated output marks itself: JSON and JSONL results carry `"simulated": true`, CSV gets a `simulated` column, and `-o` text files start with a `# SIMULATED` comment line. Bare names piped to stdout stay unmarked so they still pipe cleanly.

‎internal/dns/resolver.go‎

Lines changed: 69 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -380,6 +380,10 @@ func ResolveDomainWithRetry(ctx context.Context, resolver *net.Resolver, domain
380380
return nil, last
381381
}
382382

383+
// RandomLabel returns a 32-character random hex label, the kind wildcard
384+
// probes use: a name no real zone contains.
385+
func RandomLabel() string { return randomHex(32) }
386+
383387
// randomHex returns n random hex characters.
384388
func randomHex(n int) string {
385389
b := make([]byte, (n+1)/2)
@@ -403,43 +407,83 @@ var ErrWildcardInconclusive = errors.New("wildcard check inconclusive")
403407
// Otherwise the error wraps ErrWildcardInconclusive. Returns (isWildcard,
404408
// fingerprint, error).
405409
func CheckWildcard(ctx context.Context, resolver *net.Resolver, domain string, timeout time.Duration, types []string, attempts int) (bool, []Record, error) {
410+
return checkWildcard(ctx, resolver, domain, timeout, types, attempts, 2, false)
411+
}
412+
413+
// Fingerprint probes sizing for FingerprintWildcard (#48).
414+
const (
415+
fingerprintProbes = 5 // initial probes
416+
fingerprintMaxProbes = 128 // hard cap when the answers rotate
417+
)
418+
419+
// FingerprintWildcard is CheckWildcard for the scan root, where the
420+
// fingerprint filters every result. It starts with five probes, and when
421+
// their answers differ (a wildcard served from a rotating CDN or
422+
// load-balancer pool) it keeps probing until the fingerprint has not grown
423+
// for max(8, 4 x its size) consecutive probes, so the whole pool is learned
424+
// rather than the two addresses two probes happened to see (#48).
425+
func FingerprintWildcard(ctx context.Context, resolver *net.Resolver, domain string, timeout time.Duration, types []string, attempts int) (bool, []Record, error) {
426+
return checkWildcard(ctx, resolver, domain, timeout, types, attempts, fingerprintProbes, true)
427+
}
428+
429+
func checkWildcard(ctx context.Context, resolver *net.Resolver, domain string, timeout time.Duration, types []string, attempts, probes int, saturate bool) (bool, []Record, error) {
406430
if len(types) == 0 {
407431
types = DefaultTypes
408432
}
409433
if attempts < 1 {
410434
attempts = 1
411435
}
412-
probe1 := randomHex(32) + "." + domain
413-
probe2 := randomHex(32) + "." + domain
414-
415-
r1, o1 := ResolveDomainWithRetry(ctx, resolver, probe1, timeout, nil, attempts, types)
416-
r2, o2 := ResolveDomainWithRetry(ctx, resolver, probe2, timeout, nil, attempts, types)
417-
418-
if ctx.Err() != nil {
419-
return false, nil, ctx.Err()
436+
var fp []Record
437+
seen := map[string]struct{}{}
438+
add := func(recs []Record) (grew bool) {
439+
for _, r := range recs {
440+
k := r.Type + "\x00" + r.Value
441+
if _, ok := seen[k]; !ok {
442+
seen[k] = struct{}{}
443+
fp = append(fp, r)
444+
grew = true
445+
}
446+
}
447+
return grew
420448
}
421-
for _, o := range []Outcome{o1, o2} {
449+
probe := func() error {
450+
recs, o := ResolveDomainWithRetry(ctx, resolver, RandomLabel()+"."+domain, timeout, nil, attempts, types)
451+
if ctx.Err() != nil {
452+
return ctx.Err()
453+
}
422454
if o != OutcomeFound && o != OutcomeNXDomain {
423-
return false, nil, fmt.Errorf("%w for %s: probe lookup failed (%s) after %d attempt(s)", ErrWildcardInconclusive, domain, o, attempts)
455+
return fmt.Errorf("%w for %s: probe lookup failed (%s) after %d attempt(s)", ErrWildcardInconclusive, domain, o, attempts)
424456
}
457+
add(recs)
458+
return nil
425459
}
426460

427-
fp := unionRecords(r1, r2)
428-
return len(r1) > 0 || len(r2) > 0, fp, nil
429-
}
430-
431-
func unionRecords(a, b []Record) []Record {
432-
seen := make(map[string]struct{}, len(a)+len(b))
433-
out := make([]Record, 0, len(a)+len(b))
434-
for _, recs := range [][]Record{a, b} {
435-
for _, r := range recs {
436-
k := r.Type + "\x00" + r.Value
437-
if _, ok := seen[k]; ok {
438-
continue
461+
firstSize := -1
462+
for i := 0; i < probes; i++ {
463+
if err := probe(); err != nil {
464+
return false, nil, err
465+
}
466+
if firstSize < 0 {
467+
firstSize = len(fp)
468+
}
469+
}
470+
if len(fp) == 0 {
471+
return false, nil, nil
472+
}
473+
// The fingerprint grew after the first probe, so the answers rotate.
474+
if saturate && len(fp) > firstSize {
475+
stale := 0
476+
for n := probes; n < fingerprintMaxProbes && stale < max(8, 4*len(fp)); n++ {
477+
before := len(fp)
478+
if err := probe(); err != nil {
479+
return false, nil, err
480+
}
481+
if len(fp) > before {
482+
stale = 0
483+
} else {
484+
stale++
439485
}
440-
seen[k] = struct{}{}
441-
out = append(out, r)
442486
}
443487
}
444-
return out
488+
return true, fp, nil
445489
}

‎internal/dns/resolver_test.go‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -121,10 +121,11 @@ func TestCheckWildcardInconclusive(t *testing.T) {
121121
if is || fp != nil {
122122
t.Fatalf("is=%v fp=%v, want false/nil on an inconclusive check", is, fp)
123123
}
124-
// Two probes, two attempts each. Go's resolver may itself retry a
125-
// SERVFAIL, so only require that the retry happened.
126-
if got := srv.Queries(); got < 4 {
127-
t.Fatalf("queries = %d, want >= 4 (each probe retried)", got)
124+
// The check stops at the first inconclusive probe, after its two
125+
// attempts. Go's resolver may itself retry a SERVFAIL, so only
126+
// require that the retry happened.
127+
if got := srv.Queries(); got < 2 {
128+
t.Fatalf("queries = %d, want >= 2 (the probe retried)", got)
128129
}
129130
})
130131
}

‎internal/scan/runner.go‎

Lines changed: 72 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import (
44
"context"
55
"fmt"
66
"net"
7+
"strings"
78
"sync"
89
"sync/atomic"
910
"time"
@@ -138,20 +139,77 @@ func (c *wildcardCache) isWildcard(ctx context.Context, cfg Config, parent strin
138139
return is, nil
139140
}
140141

141-
func recordsSubset(got, fp []dns.Record) bool {
142-
if len(fp) == 0 || len(got) == 0 {
143-
return false
142+
// fingerprint is the root wildcard's known answers. A result whose records
143+
// all appear in it is a wildcard answer. Near-miss results grow it with fresh
144+
// probes (see revalidate), so it is shared by the workers and locked.
145+
type fingerprint struct {
146+
mu sync.RWMutex
147+
set map[string]struct{}
148+
}
149+
150+
func newFingerprint(recs []dns.Record) *fingerprint {
151+
f := &fingerprint{set: make(map[string]struct{}, len(recs))}
152+
f.add(recs)
153+
return f
154+
}
155+
156+
func recordKey(r dns.Record) string { return r.Type + "\x00" + r.Value }
157+
158+
func (f *fingerprint) add(recs []dns.Record) {
159+
f.mu.Lock()
160+
defer f.mu.Unlock()
161+
for _, r := range recs {
162+
f.set[recordKey(r)] = struct{}{}
144163
}
145-
set := make(map[string]struct{}, len(fp))
146-
for _, r := range fp {
147-
set[r.Type+"\x00"+r.Value] = struct{}{}
164+
}
165+
166+
// match reports whether every record of got is a known wildcard answer
167+
// (covered) and whether at least one is (overlaps). An empty fingerprint or
168+
// result never matches.
169+
func (f *fingerprint) match(got []dns.Record) (covered, overlaps bool) {
170+
f.mu.RLock()
171+
defer f.mu.RUnlock()
172+
if len(f.set) == 0 || len(got) == 0 {
173+
return false, false
148174
}
175+
covered = true
149176
for _, r := range got {
150-
if _, ok := set[r.Type+"\x00"+r.Value]; !ok {
151-
return false
177+
if _, ok := f.set[recordKey(r)]; ok {
178+
overlaps = true
179+
} else {
180+
covered = false
181+
}
182+
}
183+
return covered, overlaps
184+
}
185+
186+
// nearMissProbes is how many fresh random labels re-validate a result that
187+
// shares some but not all records with the wildcard fingerprint (#48).
188+
const nearMissProbes = 2
189+
190+
// isWildcardAnswer reports whether records are a wildcard answer for name. A
191+
// result that only partly matches the fingerprint is a near miss, typical of
192+
// a wildcard served from a rotating pool: fresh random labels under the same
193+
// parent are probed, their answers join the fingerprint, and the result is
194+
// checked again. Results that share nothing with the fingerprint (real names)
195+
// cost no extra queries.
196+
func isWildcardAnswer(ctx context.Context, cfg Config, fp *fingerprint, name string, records []dns.Record) bool {
197+
covered, overlaps := fp.match(records)
198+
if covered || !overlaps || cfg.Simulate || cfg.resolveHook != nil {
199+
return covered
200+
}
201+
parent := name
202+
if i := strings.IndexByte(name, '.'); i >= 0 {
203+
parent = name[i+1:]
204+
}
205+
for i := 0; i < nearMissProbes; i++ {
206+
recs, _ := dns.ResolveDomainWithRetry(ctx, cfg.Resolver, dns.RandomLabel()+"."+parent, cfg.Timeout, nil, cfg.Attempts, cfg.Types)
207+
fp.add(recs)
208+
if covered, _ = fp.match(records); covered {
209+
return true
152210
}
153211
}
154-
return true
212+
return false
155213
}
156214

157215
// RecursionCeiling returns the theoretical job count for a recursive scan:
@@ -273,12 +331,12 @@ func Run(ctx context.Context, cfg Config, events chan<- Event) {
273331
}
274332
}
275333

276-
var fingerprint []dns.Record
334+
fingerprint := newFingerprint(nil)
277335
wc := &wildcardCache{known: make(map[string]bool)}
278336

279337
// Wildcard detection (skip in simulation mode).
280338
if !cfg.Simulate {
281-
isWildcard, fp, err := dns.CheckWildcard(ctx, cfg.Resolver, cfg.Domain, cfg.Timeout, cfg.Types, cfg.Attempts)
339+
isWildcard, fp, err := dns.FingerprintWildcard(ctx, cfg.Resolver, cfg.Domain, cfg.Timeout, cfg.Types, cfg.Attempts)
282340
if ctx.Err() != nil {
283341
return // interrupted during wildcard probes
284342
}
@@ -292,7 +350,7 @@ func Run(ctx context.Context, cfg Config, events chan<- Event) {
292350
if err != nil {
293351
events <- Event{Kind: EventWildcard, Message: "WARNING: wildcard detection failed (" + err.Error() + "); scanning without wildcard filtering because of -force"}
294352
}
295-
fingerprint = fp
353+
fingerprint.add(fp)
296354
if isWildcard {
297355
msg := "WARNING: Wildcard DNS detected - all subdomains resolve for " + cfg.Domain
298356
events <- Event{Kind: EventWildcard, Message: msg}
@@ -496,7 +554,7 @@ func Run(ctx context.Context, cfg Config, events chan<- Event) {
496554

497555
// processJob resolves a single job and, on success, optionally enqueues
498556
// depth-capped children for recursive enumeration.
499-
func processJob(ctx context.Context, cfg Config, j job, maxDepth int, limiter *dns.RateLimiter, events chan<- Event, enqueue chan<- job, processed, found *int64, st *counters, cancel context.CancelFunc, guardOnce *sync.Once, fp []dns.Record, wc *wildcardCache) {
557+
func processJob(ctx context.Context, cfg Config, j job, maxDepth int, limiter *dns.RateLimiter, events chan<- Event, enqueue chan<- job, processed, found *int64, st *counters, cancel context.CancelFunc, guardOnce *sync.Once, fp *fingerprint, wc *wildcardCache) {
500558
if ctx.Err() != nil {
501559
return
502560
}
@@ -528,7 +586,7 @@ func processJob(ctx context.Context, cfg Config, j job, maxDepth int, limiter *d
528586
return
529587
}
530588

531-
if outcome == dns.OutcomeFound && recordsSubset(records, fp) {
589+
if outcome == dns.OutcomeFound && isWildcardAnswer(ctx, cfg, fp, j.domain, records) {
532590
st.wildcardFiltered.Add(1)
533591
n := atomic.AddInt64(processed, 1)
534592
checkReliability(cfg, n, st, events, cancel, guardOnce)

‎internal/scan/runner_test.go‎

Lines changed: 77 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -610,19 +610,90 @@ func TestRunPreflightFailsOnBlackHole(t *testing.T) {
610610
}
611611
}
612612

613-
func TestRecordsSubset(t *testing.T) {
614-
fp := []dns.Record{{Type: "A", Value: "192.0.2.99"}, {Type: "AAAA", Value: "::1"}}
615-
if !recordsSubset([]dns.Record{{Type: "A", Value: "192.0.2.99"}}, fp) {
616-
t.Error("A-only should be a subset of A+AAAA fingerprint")
613+
func TestFingerprintMatch(t *testing.T) {
614+
recs := []dns.Record{{Type: "A", Value: "192.0.2.99"}, {Type: "AAAA", Value: "::1"}}
615+
fp := newFingerprint(recs)
616+
if covered, _ := fp.match([]dns.Record{{Type: "A", Value: "192.0.2.99"}}); !covered {
617+
t.Error("A-only should be covered by an A+AAAA fingerprint")
617618
}
618-
if recordsSubset([]dns.Record{{Type: "A", Value: "192.0.2.1"}}, fp) {
619+
if covered, overlaps := fp.match([]dns.Record{{Type: "A", Value: "192.0.2.1"}}); covered || overlaps {
619620
t.Error("different IP must not match")
620621
}
621-
if recordsSubset(fp, nil) {
622+
if covered, overlaps := fp.match([]dns.Record{{Type: "A", Value: "192.0.2.99"}, {Type: "A", Value: "192.0.2.1"}}); covered || !overlaps {
623+
t.Error("partial match must be an overlap, not covered")
624+
}
625+
if covered, _ := newFingerprint(nil).match(recs); covered {
622626
t.Error("empty fingerprint must not filter")
623627
}
624628
}
625629

630+
// TestRunRotatingPoolWildcardFiltered covers #48: a wildcard that answers
631+
// from a rotating pool of 8 addresses is filtered under -force (at least 95%
632+
// of wildcard names), and real names with their own addresses are all kept.
633+
func TestRunRotatingPoolWildcardFiltered(t *testing.T) {
634+
var next atomic.Uint32
635+
real := map[string][4]byte{}
636+
var entries []string
637+
for i := 0; i < 5; i++ {
638+
name := fmt.Sprintf("real%d", i)
639+
entries = append(entries, name)
640+
real[name+".example.com"] = [4]byte{198, 51, 100, byte(i + 1)}
641+
}
642+
for i := 0; i < 50; i++ {
643+
entries = append(entries, fmt.Sprintf("w%d", i))
644+
}
645+
addr, stop := startUDPDNSAction(t, func(name string) dnsAction {
646+
if ip, ok := real[name]; ok {
647+
return dnsAction{ip: ip, hit: true}
648+
}
649+
if strings.HasSuffix(name, ".example.com") {
650+
n := next.Add(1)
651+
return dnsAction{ip: [4]byte{203, 0, 113, byte(n%8 + 1)}, hit: true}
652+
}
653+
return dnsAction{}
654+
})
655+
defer stop()
656+
657+
cfg := Config{
658+
Domain: "example.com",
659+
Entries: entries,
660+
Concurrency: 4,
661+
Timeout: time.Second,
662+
Attempts: 1,
663+
Force: true,
664+
Types: []string{"A"},
665+
Resolver: dns.NewResolver(time.Second, addr),
666+
DNSServer: addr,
667+
}
668+
events := make(chan Event, 256)
669+
go Run(context.Background(), cfg, events)
670+
found := map[string]bool{}
671+
var done *Event
672+
for ev := range events {
673+
switch ev.Kind {
674+
case EventResult:
675+
found[ev.Domain] = true
676+
case EventDone:
677+
e := ev
678+
done = &e
679+
case EventError:
680+
t.Fatalf("unexpected error: %s", ev.Message)
681+
}
682+
}
683+
if done == nil {
684+
t.Fatal("no EventDone")
685+
}
686+
for name := range real {
687+
if !found[name] {
688+
t.Errorf("real name %s was filtered", name)
689+
}
690+
}
691+
leaked := len(found) - len(real)
692+
if filtered := 50 - leaked; filtered*100 < 50*95 {
693+
t.Errorf("filtered %d/50 rotating-pool wildcard answers, want >= 95%%; stats=%+v", filtered, done.Stats)
694+
}
695+
}
696+
626697
func skipDNSName(msg []byte, off int) (int, bool) {
627698
for off < len(msg) {
628699
l := int(msg[off])

0 commit comments

Comments
 (0)