Skip to content

Commit 8575aea

Browse files
TMHSDigitalclaude
andcommitted
Add -hosts to look up a list of full host names (#351)
-hosts <file> (or - for stdin) replaces the wordlist with full names, such as another tool's output, so they go through the same wildcard filtering, pool, pacing and verdict. Targets still come from <domain> or -dL: each looks up only the names under it, the most specific target when targets nest, and names under no target are skipped, counted in one note. It is refused with -w, -recursive and -tui. The filter sits in wordlist.Prepared.For, so -dry-run and the resume digests agree with it. Fixes #351 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 9e70b67 commit 8575aea

13 files changed

Lines changed: 184 additions & 12 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Added
11+
- `-hosts <file>` looks up a list of full host names, such as another tool's output or an old results file, instead of a wordlist, through the same wildcard filtering, resolver pool, pacing, verdict, `-diff` and output formats (`-` reads stdin). Targets still come from the `<domain>` argument or `-dL`: each looks up only the names under it, the most specific target when targets nest, and names under no target are out of scope, never queried, and counted in one note. It cannot be combined with `-w`, `-recursive` or `-tui` (#351).
12+
1013
## [0.11.0] - 2026-11-18
1114

1215
### Added

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -237,6 +237,7 @@ This table describes `main`, which can be ahead of the latest release: a release
237237
| :--- | :---: | :--- |
238238
| **Targets and scope** | | |
239239
| `-w <file>` | bundled top-5000 | Wordlist file, one prefix per line; `#` comments allowed; `-` reads stdin. Repeat it to combine lists (`-w general.txt -w target.txt`); entries found in more than one are scanned once. Omit it to use the built-in list (see [Choosing a wordlist](#choosing-a-wordlist)) |
240+
| `-hosts <file>` | n/a | File of full host names to look up instead of a wordlist, such as another tool's output; `#` comments allowed; `-` reads stdin. Each target looks up only the names under it (the most specific one, when `-dL` targets nest), through the same wildcard filtering, pacing and verdict as a wordlist scan; names under no target are out of scope, never queried, and counted in one note. Cannot be combined with `-w`, `-recursive` or `-tui` |
240241
| `-dL <file>` | n/a | File of apex domains to scan, one per line (`-` reads stdin), instead of a `<domain>` argument. Each domain is an independent scan; results share one output |
241242
| `-type <list>` | `A,AAAA` | Comma-separated record types to look up: `A`, `AAAA`, `CNAME`, `NS`, `MX`, `TXT`. A name is found when it has a record of any listed type. `CNAME` and `NS` also check for [takeovers](#subdomain-takeover-hints); `MX` values are `<preference> <host>`; a `TXT` value shows a backslash and non-printable bytes as `\DDD` |
242243
| `-exclude <list>` | n/a | Comma-separated out-of-scope names: exact (`vpn.example.com`) or `*.parent` (every name below `parent`, not `parent` itself). Excluded names are never queried, probed or expanded, and are counted as `excluded`. A target that is itself out of scope is refused before `-axfr`, `-ct`/`-sources` or the `-dry-run` preflight touch it, and `-axfr` never resolves or contacts an excluded name server |

‎docs/_data/unreleased.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@
55
# In the release commit, empty these lists and run `make docs-flags`.
66
# subcommands (#245)
77
commands: []
8-
flags: []
8+
flags:
9+
- hosts
910
# new values of a flag that is itself released, as flag=VALUE (#306)
1011
values: []
1112
# action.yml inputs and outputs

‎docs/_includes/flags.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
| `-force` | - | Continue scanning even if wildcard DNS is detected |
1818
| `-format <string>` | text | Output format: text, json, jsonl, or csv |
1919
| `-hit-rate <int>` | 15 | In simulation mode, percentage of names that resolve (1-100) |
20+
| `-hosts <file>` {% include unreleased.html flag="hosts" %} | - | A file of full host names to resolve instead of a wordlist, one per line (- for stdin); each target looks up only the names under it, names under no target are skipped |
2021
| `-inconclusive <string>` | - | Write the names still without an answer (timeout, refused, other) to this file, one per line, to rescan with -w |
2122
| `-log-format <string>` | text | Diagnostics format on stderr: text, or json for one JSON object per line (level, kind, target, message); results on stdout are unchanged |
2223
| `-max-duration <duration>` | 0s | Time budget for the whole run, e.g. 20m (0 = unlimited); then no new names are tested, lookups in flight finish, and the run ends with its report, a degraded verdict and saved state for -resume to continue from |

‎internal/cli/cli.go‎

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ type cliFlags struct {
116116
printConfig bool
117117
dryRun bool // -dry-run: print the plan, scan nothing (#192)
118118
wordlists wordlistPaths // -w, repeatable (#199)
119+
hostsFile string // -hosts: full host names to resolve instead of a wordlist (#351)
119120
domainList string
120121
concurrency int
121122
timeoutMs int
@@ -219,6 +220,7 @@ func parseFlagsWith(args []string, applyDefaults func(*flag.FlagSet) error, stde
219220
fs.BoolVar(&f.printConfig, "print-config", false, "Print every setting's effective value and its source (flag, env, config or default), then exit")
220221
fs.BoolVar(&f.dryRun, "dry-run", false, "Print the plan without scanning: candidates after dedupe and -exclude, query and duration estimates, and the resolver preflight (its only DNS query, also sent to each -r pool resolver); then exit")
221222
fs.Var(&f.wordlists, "w", "Path to a wordlist `file` (- for stdin); repeat to combine several, deduplicated; omitted: the bundled top-5000 list")
223+
fs.StringVar(&f.hostsFile, "hosts", "", "A `file` of full host names to resolve instead of a wordlist, one per line (- for stdin); each target looks up only the names under it, names under no target are skipped")
222224
fs.StringVar(&f.domainList, "dL", "", "File of apex domains to scan, one per line (- for stdin); replaces the <domain> argument")
223225
fs.IntVar(&f.concurrency, "t", scan.DefaultConcurrency, "Number of concurrent workers")
224226
fs.IntVar(&f.timeoutMs, "timeout", scan.DefaultTimeoutMs, "DNS lookup timeout in milliseconds")
@@ -318,6 +320,20 @@ func validateArgs(f cliFlags, positionals []string, fs *flag.FlagSet, out *outpu
318320
out.Error("%s cannot all read standard input (-); it can only be read once, so give all but one a file", strings.Join(readers, ", "))
319321
return false
320322
}
323+
if f.hostsFile != "" {
324+
// A -w from the config file or SUBENUM_W is a default -hosts replaces.
325+
switch {
326+
case f.wordlists.chosen && !f.wordlists.replace:
327+
out.Error("use either -w or -hosts, not both: -hosts names the hosts to look up instead of a wordlist")
328+
return false
329+
case f.recursive:
330+
out.Error("-recursive cannot be used with -hosts: it scans a wordlist under each name found, and -hosts has none")
331+
return false
332+
case f.tui:
333+
out.Error("-hosts is not available in the TUI")
334+
return false
335+
}
336+
}
321337
return true
322338
}
323339

@@ -328,6 +344,7 @@ func stdinReaders(f cliFlags) []string {
328344
names = append(names, "-w")
329345
}
330346
for _, c := range []struct{ name, path string }{
347+
{"-hosts", f.hostsFile},
331348
{"-dL", f.domainList},
332349
{"-r", f.resolvers},
333350
{"-exclude-file", f.excludeFile},
@@ -662,9 +679,12 @@ func logVerboseStart(f cliFlags, domain string, maxAttempts int, out *output.Wri
662679
out.Info("Mode: LIVE DNS RESOLUTION")
663680
}
664681
out.Info("Target domain: %s", domain)
665-
if len(f.wordlists.paths) == 0 {
682+
switch {
683+
case f.hostsFile != "":
684+
out.Info("Hosts: %s", f.hostsFile)
685+
case len(f.wordlists.paths) == 0:
666686
out.Info("Wordlist: %s", defaultWordlistName(f))
667-
} else {
687+
default:
668688
out.Info("Wordlist: %s", f.wordlists.describe())
669689
}
670690
out.Info("Concurrency: %d workers", f.concurrency)

‎internal/cli/completion.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ import (
2121
// fileFlags take a path, so shells should complete file names for them. A
2222
// test fails when a flag whose usage mentions a file is missing here (#117).
2323
var fileFlags = map[string]bool{
24-
"w": true, "dL": true, "o": true, "stats": true, "inconclusive": true, "takeover-providers": true, "diff": true, "r": true, "exclude-file": true,
24+
"w": true, "hosts": true, "dL": true, "o": true, "stats": true, "inconclusive": true, "takeover-providers": true, "diff": true, "r": true, "exclude-file": true,
2525
"simulate-zone": true, "seeds": true, "resume": true, "state": true, "sarif": true,
2626
}
2727

‎internal/cli/config_test.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -395,6 +395,7 @@ var tuiOmittedFlags = map[string]bool{
395395
"sarif": true, "exclude-file": true, "ct": true, "axfr": true, "seeds": true, "state": true,
396396
"diff": true, "diff-records": true, "dL": true, "seed": true, "resume": true, "log-format": true,
397397
"inconclusive": true, "takeover-providers": true, "takeover-providers-only": true,
398+
"hosts": true,
398399
}
399400

400401
// TestSettingNamesAgreeAcrossFrontEnds covers #160: every flag is either wired

‎internal/cli/e2e_test.go‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -269,6 +269,34 @@ func TestE2EFullNamesInWordlist(t *testing.T) {
269269
}
270270
}
271271

272+
// TestE2EHosts covers #351: -hosts resolves full names, each under the
273+
// most specific -dL target it belongs to, and skips names under none.
274+
func TestE2EHosts(t *testing.T) {
275+
t.Parallel()
276+
hosts := writeFile(t, "hosts.txt", "# from another tool\napi.lab.example\nwww.lab.example\nvpn.dev.lab.example\nmail.other.example\n")
277+
targets := writeFile(t, "targets.txt", "lab.example\ndev.lab.example\n")
278+
code, stdout, stderr := runCLIStreams(t, "-simulate", "-seed", "1", "-hit-rate", "100", "-progress=false", "-hosts", hosts, "-dL", targets)
279+
if code != exitOK {
280+
t.Fatalf("exit %d\n%s", code, stderr)
281+
}
282+
lines := strings.Fields(stdout)
283+
sort.Strings(lines)
284+
if strings.Join(lines, ",") != "api.lab.example,vpn.dev.lab.example,www.lab.example" {
285+
t.Errorf("stdout = %q, want the in-scope names once each", stdout)
286+
}
287+
if !strings.Contains(stderr, "Note: 1 -hosts names are under none of the targets; skipped them") {
288+
t.Errorf("missing out-of-scope note:\n%s", stderr)
289+
}
290+
for _, args := range [][]string{
291+
{"-hosts", hosts, "-w", hosts, "lab.example"},
292+
{"-hosts", hosts, "-recursive", "lab.example"},
293+
} {
294+
if code, _, stderr := runCLIStreams(t, append([]string{"-simulate"}, args...)...); code != exitUsage {
295+
t.Errorf("%v: exit %d, want usage error\n%s", args, code, stderr)
296+
}
297+
}
298+
}
299+
272300
func TestE2ESeedReproducible(t *testing.T) {
273301
t.Parallel()
274302
wl := writeFile(t, "wl.txt", e2eWords+"vpn\ncdn\nadmin\nportal\n")

‎internal/cli/resume.go‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -424,7 +424,7 @@ func inputDigests(f cliFlags) (map[string]string, error) {
424424
if _, ok := builtinLab(simZone); ok {
425425
simZone = "" // compiled in, not a file; it does not move a saved position (#354)
426426
}
427-
paths := append(slices.Clone(f.wordlists.paths), f.domainList, f.excludeFile, f.resolvers, simZone, f.seedsFile, f.diff)
427+
paths := append(slices.Clone(f.wordlists.paths), f.hostsFile, f.domainList, f.excludeFile, f.resolvers, simZone, f.seedsFile, f.diff)
428428
for _, p := range paths {
429429
d, err := fileDigest(p)
430430
if err != nil {
@@ -535,8 +535,8 @@ func savedSources(sources []targetSource, scanned bool) map[string][]string {
535535
// state was saved.
536536
func saveResumeState(s *session) bool {
537537
f, out, prev := s.f, s.out, s.resume
538-
if f.wordlists.readsStdin() || f.domainList == "-" {
539-
out.Info("Not saving resume state: the wordlist or domain list came from stdin")
538+
if f.wordlists.readsStdin() || f.hostsFile == "-" || f.domainList == "-" {
539+
out.Info("Not saving resume state: the wordlist, host list or domain list came from stdin")
540540
return false
541541
}
542542
inputs, err := inputDigests(f)

‎internal/cli/session.go‎

Lines changed: 21 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -418,7 +418,7 @@ func (s *session) loadInputs() bool {
418418
f, out := s.f, s.out
419419
// The wordlist is read once (stdin can only be read once) and normalized
420420
// per target, since the name-length limit depends on the domain.
421-
if len(f.wordlists.paths) == 0 {
421+
if len(f.wordlists.paths) == 0 && f.hostsFile == "" {
422422
out.Info("No -w given: using the %s. Pass -w for a larger or target-specific list.", defaultWordlistName(f))
423423
}
424424
s.settings = effectiveSettings(s.fs)
@@ -442,9 +442,21 @@ func (s *session) loadInputs() bool {
442442
}
443443
s.words = words
444444
if probe, _, _ := s.words.For(""); len(probe) == 0 {
445-
out.Error("wordlist %s has no valid entries", f.wordlists.describe())
445+
if f.hostsFile != "" {
446+
out.Error("-hosts %s has no valid host names", f.hostsFile)
447+
} else {
448+
out.Error("wordlist %s has no valid entries", f.wordlists.describe())
449+
}
446450
return false
447451
}
452+
if f.hostsFile != "" {
453+
// Each target looks up only the names under it; the rest are out of
454+
// scope and never queried (#351).
455+
s.words.LimitTo(s.targets)
456+
if n := s.words.Outside(); n > 0 {
457+
out.Info("Note: %d -hosts names are under none of the targets; skipped them", n)
458+
}
459+
}
448460
if s.resume != nil {
449461
// The bundled wordlist changes with the binary, so the file digests
450462
// alone cannot tell that a saved position still means the same entry.
@@ -660,7 +672,7 @@ func (s *session) scanOneTarget(ctx context.Context, i int, target string) (next
660672
out.Info("=== [%d/%d] %s ===", i+1, len(s.targets), target)
661673
}
662674
entries, duplicates, skipped := s.words.For(target)
663-
if n := s.words.Suffixed(target); n > 0 {
675+
if n := s.words.Suffixed(target); n > 0 && s.f.hostsFile == "" {
664676
out.Info("Note: %d wordlist entries ended in %s; used them as prefixes", n, target)
665677
}
666678
if skipped > 0 {
@@ -673,10 +685,14 @@ func (s *session) scanOneTarget(ctx context.Context, i int, target string) (next
673685
}
674686
}
675687
if len(entries) == 0 {
676-
out.Error("no valid wordlist entries for %s", target)
688+
reason := "no valid wordlist entries"
689+
if s.f.hostsFile != "" {
690+
reason = "no -hosts names under this target"
691+
}
692+
out.Error("%s for %s", reason, target)
677693
s.anyFailed = true
678694
s.status[i] = report.StatusFailed
679-
s.results[i] = targetResult{failed: true, err: "no valid wordlist entries"}
695+
s.results[i] = targetResult{failed: true, err: reason}
680696
s.failedTargets++
681697
return true
682698
}

0 commit comments

Comments
 (0)