Skip to content

Commit 43579aa

Browse files
TMHSDigitalclaude
andcommitted
Account for the -r pool in -dry-run's preflight and estimate (#376)
-dry-run built the -r pool but never used it: the preflight went only to -dns-server and the duration estimate used only -rate, so a run capped by -rate-per-resolver was planned as unlimited. Each pool resolver now gets the target's preflight query too (dns.Pool.Probe, outside the rotation and its accounting), reported per resolver as text "pool" lines and JSON pool_preflight. The estimate uses min(-rate, -rate-per-resolver x pool size), either term 0 meaning no bound, exposed as rate_effective. A failing pool resolver does not fail the dry run, as a scan benches it. The issue's -permute items (a seeds-only pass and custom permutation words) need a flag name and behaviour decision and are left open. Refs #376 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 3cc069c commit 43579aa

7 files changed

Lines changed: 147 additions & 20 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
4646
- The GitHub Action gains three outputs: `changes`, the path of the JSONL file listing the added, removed and changed names (empty when there were no previous results to diff against), so a later step can open an issue or post them; `takeovers`, the number of takeover candidates found; and `internal`, the number of names answering with an address that is not public. The job summary table has a column for each count and lists the takeover candidates under it. `docs/monitoring.md` shows a step that opens an issue from `changes` (#311).
4747
- The TUI's finished screen shows the run-quality verdict (`complete`, `degraded` or `unreliable`) with its reason above the counters, colored green, orange or red, so a scan that lost lookups or stopped at Max Duration no longer looks like a clean one. The counters line adds skipped candidates, retried and recovered names, takeover candidates and rate back-offs when there are any. The screen is the same height as before (#312).
4848
- `pkg/subenum` exports `ProbeZone(ctx, domain, ZoneOptions)`, which tries a zone transfer (AXFR) from the domain's name servers with `-axfr`'s limits (8 servers, 2 addresses each) and reports the names, the servers that allowed it and whether the zone uses plain NSEC (`ZoneInfo`), and `ZoneTransfer(ZoneOptions)`, a `Source` for `Config.Sources` that scans the transferred names. Its queries are not paced by `Rate`, as `-axfr`'s are not. The CLI and the library share the attempt in a new `internal/axfr` package (#314).
49+
- `-dry-run` takes the `-r` resolver pool into account: each pool resolver gets the target's preflight query too and has its own `pool` line (`pool_preflight` in JSON, with each resolver's `address`, `preflight` and `ok`), and the duration estimate uses `-rate-per-resolver` times the pool size when that is below `-rate` or `-rate` is unlimited, naming the flag that bounds it (`rate_effective` in JSON). A failing pool resolver does not fail the dry run, as a scan benches it and carries on (#376).
4950

5051
### Changed
5152
- A `-recursive` scan over a wordlist of single labels no longer remembers every candidate it generated (about 75 MB per million), and one that reaches `-max-queries` or `-max-duration` counts the untested candidates without building each name, so it stops on time (#301).

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -290,7 +290,7 @@ This table describes `main`, which can be ahead of the latest release: a release
290290
| `-simulate-zone <file>` | n/a | Lab mode: answer every query from a scenario file, or a built-in scenario such as `lab1` (`lab1`-`lab6`, with their own wordlist when `-w` is not given), via a local DNS server on `127.0.0.1`; no traffic leaves the machine. See [Labs](docs/labs.md) |
291291
| **Modes** | | |
292292
| `-tui` | `false` | Launch the interactive Terminal UI |
293-
| `-dry-run` | `false` | Print the plan without scanning: candidates after dedupe and `-exclude`, query and duration estimates, and the resolver preflight (its only DNS query, paced by `-rate` and skipped for a target `-exclude` rules out), then exit; JSON with `-format json` or `jsonl` |
293+
| `-dry-run` | `false` | Print the plan without scanning: candidates after dedupe and `-exclude`, query and duration estimates (at `-rate`, or `-rate-per-resolver` times the `-r` pool size when that is lower), and the resolver preflight (its only DNS query, also sent once to each `-r` pool resolver, paced by `-rate` and skipped for a target `-exclude` rules out; a failing pool resolver is reported but does not fail the run), then exit; JSON with `-format json` or `jsonl` |
294294
| `-print-config` | `false` | Print every setting's effective value and source (`flag`, env, config or default), then exit |
295295
| `-version` | n/a | Print version and exit |
296296

‎docs/_includes/flags.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
| `-diff <string>` | - | Previous results file (any -format); report only names added or removed since then, and exit 4 when there are changes |
1212
| `-diff-records` {% include unreleased.html flag="diff-records" %} | - | With -diff, also report names whose records changed (a re-pointed CNAME, moved addresses), compared over the -type record types; needs a json, jsonl or csv -diff file |
1313
| `-dns-server <string>` | 8.8.8.8:53 | DNS server: ip:port (UDP, TCP fallback), tls://host[:853] (DNS over TLS) or https://host/path (DNS over HTTPS) |
14-
| `-dry-run` {% include unreleased.html flag="dry-run" %} | - | Print the plan without scanning: candidates after dedupe and -exclude, query and duration estimates, and the resolver preflight (its only DNS query); then exit |
14+
| `-dry-run` {% include unreleased.html flag="dry-run" %} | - | Print the plan without scanning: candidates after dedupe and -exclude, query and duration estimates, and the resolver preflight (its only DNS query, also sent to each -r pool resolver); then exit |
1515
| `-exclude <string>` | - | Comma-separated out-of-scope names and \*.parent patterns; never queried or expanded |
1616
| `-exclude-file <string>` | - | File of out-of-scope names and \*.parent patterns, one per line (# comments allowed) |
1717
| `-force` | - | Continue scanning even if wildcard DNS is detected |

‎internal/cli/cli.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -217,7 +217,7 @@ func parseFlagsWith(args []string, applyDefaults func(*flag.FlagSet) error, stde
217217
}
218218
fs.BoolVar(&f.tui, "tui", false, "Launch the interactive terminal UI, its form pre-filled from the other flags and the domain argument")
219219
fs.BoolVar(&f.printConfig, "print-config", false, "Print every setting's effective value and its source (flag, env, config or default), then exit")
220-
fs.BoolVar(&f.dryRun, "dry-run", false, "Print the plan without scanning: candidates after dedupe and -exclude, query and duration estimates, and the resolver preflight (its only DNS query); then exit")
220+
fs.BoolVar(&f.dryRun, "dry-run", false, "Print the plan without scanning: candidates after dedupe and -exclude, query and duration estimates, and the resolver preflight (its only DNS query, also sent to each -r pool resolver); then exit")
221221
fs.Var(&f.wordlists, "w", "Path to a wordlist `file` (- for stdin); repeat to combine several, deduplicated; omitted: the bundled top-5000 list")
222222
fs.StringVar(&f.domainList, "dL", "", "File of apex domains to scan, one per line (- for stdin); replaces the <domain> argument")
223223
fs.IntVar(&f.concurrency, "t", scan.DefaultConcurrency, "Number of concurrent workers")

‎internal/cli/dryrun.go‎

Lines changed: 67 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -24,19 +24,55 @@ type dryRunTarget struct {
2424
// counts every retry -attempts allows. Both respect -max-queries.
2525
Queries int64 `json:"queries"`
2626
QueriesMax int64 `json:"queries_max"`
27-
// Seconds and SecondsMax are the time those queries take at -rate; 0
28-
// when -rate is unlimited.
27+
// RateEffective is the query rate a scan is held to: -rate, or with a -r
28+
// pool -rate-per-resolver times the pool's size, whichever is lower; 0
29+
// when neither bounds it (#376).
30+
RateEffective int `json:"rate_effective,omitempty"`
31+
// Seconds and SecondsMax are the time those queries take at
32+
// RateEffective; 0 when it is unbounded.
2933
Seconds float64 `json:"seconds,omitempty"`
3034
SecondsMax float64 `json:"seconds_max,omitempty"`
3135
// Preflight is the resolver's answer for the domain itself, as a scan's
3236
// preflight check gets it; empty in simulation mode.
3337
Preflight string `json:"preflight,omitempty"`
3438
PreflightOK bool `json:"preflight_ok"`
39+
// PoolPreflight is each -r pool member's answer to the same query
40+
// (#376). A failing member does not fail the dry run: a scan benches it
41+
// and carries on with the others.
42+
PoolPreflight []dryRunResolver `json:"pool_preflight,omitempty"`
3543
// OutOfScope is set for a target -exclude rules out: a scan refuses it,
3644
// so it has no candidates and gets no preflight query (#372).
3745
OutOfScope bool `json:"out_of_scope,omitempty"`
3846
}
3947

48+
// dryRunResolver is one -r pool member's preflight answer (#376).
49+
type dryRunResolver struct {
50+
Addr string `json:"address"`
51+
Preflight string `json:"preflight"`
52+
OK bool `json:"ok"`
53+
}
54+
55+
// preflightOK reports whether a preflight answer shows a working resolver.
56+
func preflightOK(o dns.Outcome) bool {
57+
return o == dns.OutcomeFound || o == dns.OutcomeNXDomain
58+
}
59+
60+
// dryRunRate returns the query rate a scan is held to and the flags that set
61+
// it: -rate, or with a -r pool -rate-per-resolver times the pool's size,
62+
// whichever is lower (#376). 0 means neither bounds it.
63+
func dryRunRate(f cliFlags, pool *dns.Pool) (int, string) {
64+
rate, by := f.rate, fmt.Sprintf("-rate %d", f.rate)
65+
if pool != nil && f.ratePerRes > 0 {
66+
if poolRate := f.ratePerRes * pool.Size(); rate <= 0 || poolRate < rate {
67+
rate, by = poolRate, fmt.Sprintf("-rate-per-resolver %d x %d resolvers", f.ratePerRes, pool.Size())
68+
}
69+
}
70+
if rate <= 0 {
71+
return 0, ""
72+
}
73+
return rate, by
74+
}
75+
4076
// dryRun prints what a scan would do without scanning: the candidates per
4177
// target, query and duration estimates, and the resolver preflight, which is
4278
// its only DNS traffic (#192). It fails only if a preflight fails.
@@ -50,6 +86,8 @@ func (s *session) dryRun() int {
5086
excluded := scan.Excluder(s.in.excludes)
5187
// The preflights are paced by -rate like a scan's queries (#372).
5288
ctx := dns.WithLimiter(context.Background(), dns.NewRateLimiter(f.rate))
89+
rate, rateBy := dryRunRate(f, s.in.pool)
90+
defer s.in.pool.CloseIdleConnections()
5391
for _, target := range s.targets {
5492
entries, duplicates, skipped := s.words.For(target)
5593
p := dryRunTarget{Domain: target, Duplicates: duplicates, Skipped: skipped, PreflightOK: true}
@@ -73,16 +111,22 @@ func (s *session) dryRun() int {
73111
// One lookup per name sends a query per type (#223, #249).
74112
p.Queries = names * int64(dns.QueriesFor(s.in.recordTypes))
75113
p.QueriesMax = p.Queries * int64(max(s.in.maxAttempts, 1))
76-
if f.rate > 0 {
77-
p.Seconds = float64(p.Queries) / float64(f.rate)
78-
p.SecondsMax = float64(p.QueriesMax) / float64(f.rate)
114+
p.RateEffective = rate
115+
if rate > 0 {
116+
p.Seconds = float64(p.Queries) / float64(rate)
117+
p.SecondsMax = float64(p.QueriesMax) / float64(rate)
79118
}
80119
if !f.testMode {
81120
lookup := dns.Lookup{Resolver: resolver, Timeout: timeout, Attempts: s.in.maxAttempts, Types: s.in.recordTypes}
82121
_, outcome := lookup.Resolve(ctx, target)
83122
p.Preflight = outcome.String()
84-
p.PreflightOK = outcome == dns.OutcomeFound || outcome == dns.OutcomeNXDomain
123+
p.PreflightOK = preflightOK(outcome)
85124
failed = failed || !p.PreflightOK
125+
if s.in.pool != nil {
126+
for _, r := range s.in.pool.Probe(ctx, lookup, target) {
127+
p.PoolPreflight = append(p.PoolPreflight, dryRunResolver{Addr: r.Addr, Preflight: r.Outcome.String(), OK: preflightOK(r.Outcome)})
128+
}
129+
}
86130
}
87131
plans = append(plans, p)
88132
}
@@ -105,7 +149,7 @@ func (s *session) dryRun() int {
105149
}
106150
}
107151
default:
108-
s.printDryRun(plans)
152+
s.printDryRun(plans, rateBy)
109153
}
110154
if notes := dryRunNotes(f, s.in.sources); len(notes) > 0 {
111155
s.out.Info("Not included in the estimate: %s", strings.Join(notes, "; "))
@@ -116,8 +160,9 @@ func (s *session) dryRun() int {
116160
return exitOK
117161
}
118162

119-
// printDryRun writes the plan as text.
120-
func (s *session) printDryRun(plans []dryRunTarget) {
163+
// printDryRun writes the plan as text; rateBy names the flags that bound
164+
// the duration.
165+
func (s *session) printDryRun(plans []dryRunTarget, rateBy string) {
121166
w := s.a.stdout
122167
for _, p := range plans {
123168
_, _ = fmt.Fprintf(w, "%s\n", p.Domain)
@@ -128,20 +173,27 @@ func (s *session) printDryRun(plans []dryRunTarget) {
128173
_, _ = fmt.Fprintf(w, " candidates %d (%d duplicates removed, %d invalid skipped, %d excluded)\n", p.Candidates, p.Duplicates, p.Skipped, p.Excluded)
129174
_, _ = fmt.Fprintf(w, " queries %d (up to %d with retries)\n", p.Queries, p.QueriesMax)
130175
if p.Seconds > 0 {
131-
_, _ = fmt.Fprintf(w, " duration %s (up to %s) at -rate %d\n", dryRunDuration(p.Seconds), dryRunDuration(p.SecondsMax), s.f.rate)
176+
_, _ = fmt.Fprintf(w, " duration %s (up to %s) at %d/s (%s)\n", dryRunDuration(p.Seconds), dryRunDuration(p.SecondsMax), p.RateEffective, rateBy)
132177
} else {
133178
_, _ = fmt.Fprintf(w, " duration not bounded by -rate (unlimited)\n")
134179
}
135180
if p.Preflight != "" {
136-
status := "ok"
137-
if !p.PreflightOK {
138-
status = "FAILED"
139-
}
140-
_, _ = fmt.Fprintf(w, " preflight %s via %s: %s\n", status, s.f.dnsServer, p.Preflight)
181+
_, _ = fmt.Fprintf(w, " preflight %s via %s: %s\n", dryRunStatus(p.PreflightOK), s.f.dnsServer, p.Preflight)
182+
}
183+
for _, r := range p.PoolPreflight {
184+
_, _ = fmt.Fprintf(w, " pool %s via %s: %s\n", dryRunStatus(r.OK), r.Addr, r.Preflight)
141185
}
142186
}
143187
}
144188

189+
// dryRunStatus labels a preflight answer.
190+
func dryRunStatus(ok bool) string {
191+
if ok {
192+
return "ok"
193+
}
194+
return "FAILED"
195+
}
196+
145197
// dryRunDuration rounds an estimate for display.
146198
func dryRunDuration(seconds float64) string {
147199
return time.Duration(seconds * float64(time.Second)).Round(time.Second).String()

‎internal/cli/dryrun_test.go‎

Lines changed: 55 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import (
44
"encoding/json"
55
"os"
66
"path/filepath"
7+
"reflect"
78
"strings"
89
"testing"
910
"time"
@@ -40,8 +41,8 @@ func TestDryRunPlan(t *testing.T) {
4041
}
4142
p := plans[0]
4243
want := dryRunTarget{Domain: "example.com", Candidates: 3, Duplicates: 1, Skipped: 1, Excluded: 1,
43-
Queries: 6, QueriesMax: 18, Seconds: 1.5, SecondsMax: 4.5, PreflightOK: true}
44-
if p != want {
44+
Queries: 6, QueriesMax: 18, RateEffective: 4, Seconds: 1.5, SecondsMax: 4.5, PreflightOK: true}
45+
if !reflect.DeepEqual(p, want) {
4546
t.Errorf("plan = %+v\nwant %+v", p, want)
4647
}
4748
if left, _ := os.ReadDir(dir); len(left) != 0 {
@@ -149,6 +150,58 @@ func TestDryRunOutOfScopeTarget(t *testing.T) {
149150
}
150151
}
151152

153+
// TestDryRunResolverPool covers #376: with -r, the plan preflights every
154+
// pool member with the target's own query, reports a failing member without
155+
// failing the run, and bounds the duration by -rate-per-resolver times the
156+
// pool's size when that is below -rate.
157+
func TestDryRunResolverPool(t *testing.T) {
158+
listen := func(reply dnsserver.Reply) *dnsserver.Server {
159+
srv, err := dnsserver.Listen(dnsserver.Options{CountNames: true}, func(dnsserver.Query) dnsserver.Reply { return reply })
160+
if err != nil {
161+
t.Fatal(err)
162+
}
163+
t.Cleanup(srv.Close)
164+
return srv
165+
}
166+
answers := dnsserver.Reply{A: []string{"192.0.2.1"}}
167+
trusted, good, bad := listen(answers), listen(answers), listen(dnsserver.ServFail)
168+
wl := writeFile(t, "wl.txt", e2eWords)
169+
list := writeFile(t, "resolvers.txt", good.Addr+"\n"+bad.Addr+"\n")
170+
args := []string{"-dns-server", trusted.Addr, "-r", list, "-rate-per-resolver", "3", "-timeout", "10000",
171+
"-attempts", "1", "-type", "A", "-w", wl}
172+
173+
for _, tc := range []struct {
174+
rate string
175+
want int
176+
}{{"0", 6}, {"4", 4}} {
177+
code, plans := dryRunPlans(t, append(args, "-rate", tc.rate, "example.com")...)
178+
if code != exitOK || len(plans) != 1 {
179+
t.Fatalf("-rate %s: exit %d, %d plans; want 0 and 1 (a failing pool member does not fail the run)", tc.rate, code, len(plans))
180+
}
181+
p := plans[0]
182+
if p.RateEffective != tc.want || p.Seconds != float64(p.Queries)/float64(tc.want) {
183+
t.Errorf("-rate %s: rate_effective %d, seconds %v; want %d/s for %d queries", tc.rate, p.RateEffective, p.Seconds, tc.want, p.Queries)
184+
}
185+
want := []dryRunResolver{{Addr: good.Addr, Preflight: "found", OK: true}, {Addr: bad.Addr, Preflight: "error"}}
186+
if !p.PreflightOK || !reflect.DeepEqual(p.PoolPreflight, want) {
187+
t.Errorf("-rate %s: preflight ok=%v, pool preflight %+v; want ok and %+v", tc.rate, p.PreflightOK, p.PoolPreflight, want)
188+
}
189+
}
190+
for _, srv := range []*dnsserver.Server{good, bad} {
191+
if apex, all := srv.QueriesFor("example.com", dnsmessage.TypeA), srv.Queries(); apex == 0 || apex != all {
192+
t.Errorf("pool member %s got %d queries, %d of them the apex preflight; want only the preflight", srv.Addr, all, apex)
193+
}
194+
}
195+
196+
_, stdout := runCLI(t, "", append(args, "-dry-run", "-progress=false", "example.com")...)
197+
for _, want := range []string{"at 6/s (-rate-per-resolver 3 x 2 resolvers)", "pool ok via " + good.Addr + ": found",
198+
"pool FAILED via " + bad.Addr + ": error"} {
199+
if !strings.Contains(stdout, want) {
200+
t.Errorf("text plan lacks %q:\n%s", want, stdout)
201+
}
202+
}
203+
}
204+
152205
func TestDryRunText(t *testing.T) {
153206
wl := writeFile(t, "wl.txt", e2eWords)
154207
code, stdout := runCLI(t, "", "-dry-run", "-simulate", "-progress=false", "-w", wl, "-permute", "example.com")

‎internal/dns/pool.go‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -482,6 +482,27 @@ func (p *Pool) secondOpinion(ctx context.Context, m *PoolResolver, l Lookup, dom
482482
return nx
483483
}
484484

485+
// ProbeResult is one member's answer to Probe.
486+
type ProbeResult struct {
487+
Addr string
488+
Outcome Outcome
489+
}
490+
491+
// Probe looks domain up once on every member, in pool order, outside the
492+
// rotation and its accounting. Each lookup takes its slots of the member's
493+
// rate cap. -dry-run uses it as a per-resolver preflight (#376).
494+
func (p *Pool) Probe(ctx context.Context, l Lookup, domain string) []ProbeResult {
495+
out := make([]ProbeResult, 0, len(p.members))
496+
for _, m := range p.members {
497+
outcome := OutcomeCanceled
498+
if p.reserve(ctx, m, QueriesFor(l.Types)) == nil {
499+
_, outcome = m.lookup(l).Resolve(ctx, domain)
500+
}
501+
out = append(out, ProbeResult{Addr: m.Addr, Outcome: outcome})
502+
}
503+
return out
504+
}
505+
485506
// lookup is l as one try on m's resolver.
486507
func (m *PoolResolver) lookup(l Lookup) Lookup {
487508
l.Resolver, l.Attempts = m.resolver, 1

0 commit comments

Comments
 (0)