You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4b22c95
Browse filesBrowse the repository at this point in the historyBrowse files
Share scan option checks, pool building and defaults (#262)
The CLI, TUI and pkg/subenum each built scan.Options by hand, with some
range checks, the resolver pool and the defaults repeated per front end.
- scan.Options gains RatePerResolver and MaxDuration, and Validate checks
both (new {rate_per_resolver} setting), replacing the separate checks
in main.go, the TUI form and pkg/subenum.
- dns.NewPoolFromLines parses a resolver list and builds the capped
pool; all three front ends use it.
- scan.DefaultConcurrency, DefaultTimeoutMs and DefaultHitRate feed the
CLI flag defaults and the TUI form; pkg/subenum keeps its documented
literals, now pinned to them by TestDefaultsMatchTheCLI.
- Each front end has a drift-guard test: with every input set, every
scan.Options field must be set or listed as unmapped with a reason.
The TUI's list records the gaps #274 is about (AutoRate,
RetryInconclusive, MaxDuration).
- Tests pin the new messages for a negative rate per resolver and max
duration in each front end.
Fixes#262
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
*`https://host/path` - DNS over HTTPS (RFC 8484), each query sent as an HTTP POST of `application/dns-message`.
148
148
149
149
`scan.Run` builds one resolver per scan (unless `Config.Resolver` is set) and reuses it for every lookup; `dns.CloseIdleConnections` releases its pooled connections when the scan ends.
150
-
***Resolver pool (`-r`)**: `dns.ParseResolverList` reads the `-r` file (`ip` or `ip:port`, port 53 by default) and `dns.NewPool`builds a `dns.Pool` over it. Candidate lookups go round-robin over the healthy members; the `-dns-server` resolver stays the *trusted* resolver and runs the preflight and wildcard probes. Safeguards:
150
+
***Resolver pool (`-r`)**: `dns.NewPoolFromLines` parses the `-r` file (`ip` or `ip:port`, port 53 by default) and builds a `dns.Pool` over it with the `-rate-per-resolver` cap; the TUI and `pkg/subenum` build their pools with it too. Candidate lookups go round-robin over the healthy members; the `-dns-server` resolver stays the *trusted* resolver and runs the preflight and wildcard probes. Safeguards:
151
151
***Health benching**: over each window of 20 lookups, a member that failed (timeout, refused, other) more than half of them is benched for 30 seconds, then tried again.
152
152
***Per-resolver rate cap (`-rate-per-resolver`)**: `Pool.SetRatePerResolver` spaces the queries sent to each member like `-rate` does globally. A lookup prefers a member with room, waits for the one that frees up soonest when every cap is spent (before its timeout starts), and books one slot per record type; canaries and second opinions book slots too. Each spent cap a lookup runs into counts as `throttled`. The global `-rate` limiter still applies, so the stricter of the two wins.
153
153
***Hit re-validation**: every pool hit is resolved again on the trusted resolver, and the trusted answer is the one reported (`resolveViaPool` in `internal/scan/runner.go`), so a poisoned member cannot inject names.
***Purpose**: To efficiently perform DNS lookups for a large number of potential subdomains, `subenum` employs a worker pool pattern. This allows multiple DNS queries to be in flight concurrently, significantly speeding up the enumeration process compared to sequential lookups.
172
172
***Implementation**: The scan engine lives in `internal/scan/runner.go` as `scan.Run`, which takes a context, a `scan.Config`, and an events channel. Both the CLI (`scanTarget` in `main.go`) and the TUI call it.
173
-
***`scan.Options` / `scan.Config`**: `Options` holds the user-facing settings (timeout in milliseconds, attempts, rate, and so on). Its `Validate` method checks ranges and the resolver address, and its `Config` method builds the `scan.Config` the engine runs on. The CLI and the TUI each fill in an `Options`, so they cannot drift apart in what they accept.
173
+
***`scan.Options` / `scan.Config`**: `Options` holds the user-facing settings (timeout in milliseconds, attempts, rate, and so on). Its `Validate` method checks ranges and the resolver address, and its `Config` method builds the `scan.Config` the engine runs on. The CLI, the TUI and `pkg/subenum`each fill in an `Options`, so they cannot drift apart in what they accept, and each has a test that fails when an `Options` field is left unset without a listed reason. `scan.DefaultConcurrency`, `DefaultTimeoutMs` and `DefaultHitRate` are the defaults all three use.
174
174
***`scan.Event` / `scan.EventKind`**: Typed events emitted on the channel: `EventResult` (a `scan.Result`: subdomain, records, simulated flag, takeover hint and internal address scope), `EventProgress`, `EventNotice`, `EventError`, and `EventDone`. An `EventNotice` carries a `scan.NoticeKind` (`NoticeWildcard`, `NoticeCap`, `NoticeCeiling`, `NoticeSkip`, `NoticeRate`) so consumers can style or filter notices without parsing the message. `EventDone` carries a `scan.Stats` snapshot with per-outcome counters (found, nxdomain, timeout, refused, other, wildcard-filtered).
175
175
***Preflight and wildcard check**: Before any candidate is dispatched, `scan.Run` resolves the apex (with retries) and runs `Lookup.FingerprintWildcard`; see the Data Flow section.
176
176
***Dispatcher and lazy candidate generation**: A dispatcher goroutine owns the internal `jobs` channel, which carries chunks of jobs rather than single jobs: `nextChunk` hands each send up to one worker's share of what is left, at most `maxChunk` (8), so a large scan pays one channel operation per chunk and the tail still spreads over every worker. `jobs` and `completed` are buffered (`handoffPerWorker` = 4 sends per worker), so workers and the dispatcher rarely wait on each other. The dispatcher also keeps a queue of *expansions* (a parent name plus a cursor into `Config.Entries`), a visited set, and a pending-work counter. Candidate names are built only as they are dispatched, so a found parent costs one queue entry rather than one per wordlist entry, and the queue is compacted as it is consumed. `-max-queries` stops admission once the cap is reached; the remaining candidates are counted, never built, and reported in a `NoticeCap` notice.
@@ -249,7 +249,7 @@ Visually, this can be seen as:
249
249
250
250
***Missing Required Arguments**: When `-w` or a target (domain or `-dL`) is missing, the tool says what is missing, prints the usage message and flag list, and exits with status 2.
251
251
***Validation** (status 2 on failure): the tool validates:
252
-
* Concurrency, timeout, attempts and depth must be at least 1; `-rate` and`-max-queries` must be 0 (unlimited) or positive. These checks live in `scan.Options.Validate`.
252
+
* Concurrency, timeout, attempts and depth must be at least 1; `-rate`, `-rate-per-resolver`,`-max-queries` and `-max-duration` must be 0 (unlimited) or positive. These checks live in `scan.Options.Validate`.
253
253
* The DNS server must be a valid `ip:port` with an IP host and a port in 1-65535 (`validate.DNSServer`; not checked in simulation mode).
254
254
* The target domain must conform to DNS naming rules after normalization (`validate.NormalizeDomain`, `validate.Domain`). In a `-dL` list, invalid domains are skipped with a message instead.
// Checked once here for every front end, not by each one (#262).
31
+
{"max duration", func(o*Options) { o.MaxDuration=-time.Second }, "max duration ({max_duration}) must be 0 (unlimited) or positive, got -1s"},
32
+
{"rate per resolver", func(o*Options) { o.RatePerResolver=-1 }, "rate per resolver ({rate_per_resolver}) must be 0 (unlimited) or a positive integer, got -1"},
0 commit comments