Skip to content

Commit 1cbee27

Browse files
TMHSDigitalclaude
andcommitted
feat(cli): -silent pipeline mode and a Pipelines guide
-silent prints bare result names on stdout and only errors on stderr, plus a one-line warning when results are simulated so a test run cannot pass for a real one. docs/pipelines.md shows dnsx/httpx/nuclei chains, verdict gating via -stats or the JSONL summary, and takeover-first triage. Fixes #128 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent ffa092c commit 1cbee27

8 files changed

Lines changed: 152 additions & 2 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010
### Added
11+
- `-silent` for pipelines: bare result names on stdout and nothing on stderr but errors and a one-line warning when results are simulated. A new [Pipelines](docs/pipelines.md) page chains subenum with dnsx, httpx and nuclei and gates each step on the run-quality verdict (#128).
1112
- A Go library API, `pkg/subenum`, so other programs can embed the engine: `Config` (every zero value takes the CLI's default), `Scan` for a finished run, `Run` to stream `Event` values, and `Result`, `Record` and `Stats`. The DNS internals stay unexported, engine messages name `Config` fields instead of CLI flags, and `Run` always ends with one terminal event carrying the final `Stats`. Runnable examples ship for pkg.go.dev (#77).
1213
- Lab mode: `-simulate-zone <file>` answers every query from a scenario file (names, A/AAAA/CNAME records, wildcards, TIMEOUT/SERVFAIL/REFUSED/NXDOMAIN failures, `$DELAY` and `$REFUSE-ABOVE` rate limiting) through a DNS server on `127.0.0.1`. The real resolver, wildcard detection, `-rate`, takeover hints and the run-quality report run unchanged, deterministically and without traffic leaving the machine. A new [Labs](docs/labs.md) page has five guided exercises with answers, and the scenarios ship in `examples/labs/`. The run-quality report gains a `zone` field (#76).
1314
- Contributor experience: a devcontainer (Go 1.26 plus the CI golangci-lint) with an "Open in Codespaces" badge, YAML issue forms with blank issues disabled and contact links to private vulnerability reporting and Discussions, `FUNDING.yml` and `CITATION.cff` (#75).

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -275,6 +275,7 @@ make help # list all targets
275275
| `-depth <n>` | `1` | Max recursion depth when `-recursive` is set (1 = no recursion) |
276276
| `-v` | `false` | Verbose output: IPs, timings, per-query detail (stderr) |
277277
| `-progress` | `true` | Live progress line on stderr. Off by default when stderr is not a terminal; passing `-progress` there prints whole progress lines instead of redrawing one |
278+
| `-silent` | `false` | Pipeline mode: bare result names on stdout, only errors (and a one-line simulation warning) on stderr. See [Pipelines](https://tmhsdigital.github.io/subenum/pipelines.html) |
278279
| `-simulate` | `false` | Simulation mode: no real DNS queries |
279280
| `-hit-rate <n>` | `15` | Simulated resolution rate, percent (1-100), applied uniformly to every name |
280281
| `-seed <n>` | `0` | Simulation seed; the same seed reproduces the same results (line order can vary unless `-t 1`). `0` picks a random seed and prints it |

‎docs/_data/nav.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@
1212
- section: Guides
1313
footer: true
1414
pages:
15+
- title: Pipelines
16+
url: /pipelines.html
1517
- title: Monitoring
1618
url: /monitoring.html
1719
- title: Labs

‎docs/_includes/flags.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626
| `-seed <uint>` | - | In simulation mode, seed for reproducible results (0 = random; the seed used is printed) |
2727
| `-seeds <string>` | - | Results file (any -format) whose names also seed -permute; implies -permute |
2828
| `-show-records` | - | In text format, append each result's records (TYPE=value) |
29+
| `-silent` | - | Pipeline mode: bare result names on stdout, and only errors (and a simulation warning) on stderr |
2930
| `-simulate` | - | Run in simulation mode without actual DNS queries (for testing) |
3031
| `-simulate-zone <string>` | - | Lab mode: answer every query from this scenario file via a local DNS server, so no traffic leaves the machine (see the Labs page) |
3132
| `-state <string>` | subenum-resume.json | Where an interrupted run saves its state for -resume |

‎docs/pipelines.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
---
2+
layout: default
3+
title: Pipelines
4+
description: Chain subenum with dnsx, httpx and nuclei, and gate each step on the run-quality verdict.
5+
---
6+
7+
# Pipelines
8+
9+
subenum is built to sit at the start of a recon pipeline. With `-silent`,
10+
stdout carries one resolved name per line and nothing else, and stderr carries
11+
only errors, so its output pipes straight into the next tool. Even without
12+
`-silent`, results go to stdout and everything else to stderr, and piped text
13+
output is already bare names.
14+
15+
Only scan domains you own or are authorized to test. Every recipe below works
16+
offline against a [lab zone](labs.html) first:
17+
18+
```bash
19+
subenum -silent -simulate-zone examples/labs/lab1-first-scan.zone lab.example
20+
```
21+
22+
## Resolve, probe, scan
23+
24+
```bash
25+
# Names with their A records (dnsx re-resolves them; subenum already did,
26+
# so this is for formatting or a second resolver's opinion).
27+
subenum -silent example.com | dnsx -silent -a -resp
28+
29+
# Live web services: status, title and technologies.
30+
subenum -silent example.com | httpx -silent -status-code -title -tech-detect
31+
32+
# Subdomain takeover checks on every name found.
33+
subenum -silent -type A,AAAA,CNAME example.com | nuclei -silent -tags takeover
34+
```
35+
36+
## Gate the pipeline on the verdict
37+
38+
A scan that hit a rate-limiting resolver can return a fraction of the real
39+
names and still exit 0. The [run-quality report](cli.html) says whether the
40+
result set can be trusted, so check it before spending time downstream:
41+
42+
```bash
43+
subenum -silent -stats run.json -o names.txt example.com
44+
if ! jq -e '.verdict == "complete"' run.json > /dev/null; then
45+
jq -r '"subenum run was \(.verdict): \(.reason)"' run.json >&2
46+
exit 1
47+
fi
48+
httpx -silent -l names.txt
49+
```
50+
51+
With `-format jsonl`, the same report is the last line of the output, so one
52+
file holds both:
53+
54+
```bash
55+
subenum -silent -format jsonl -o results.jsonl example.com > /dev/null
56+
tail -n 1 results.jsonl | jq -e '.verdict == "complete"' > /dev/null || exit 1
57+
jq -r 'select(.type != "summary") | .subdomain' results.jsonl | httpx -silent
58+
```
59+
60+
## Takeover candidates first
61+
62+
With CNAME lookups, subenum marks names whose alias target does not resolve or
63+
points at a takeover-prone service. Send those to a closer look before the
64+
rest:
65+
66+
```bash
67+
subenum -silent -format jsonl -type A,AAAA,CNAME example.com \
68+
| jq -r 'select(.takeover_candidate) | "\(.subdomain)\t\(.takeover_candidate)"'
69+
```
70+
71+
## Many targets
72+
73+
`-dL` scans a list of apex domains with one combined output; each target gets
74+
its own preflight, wildcard check and verdict in the report:
75+
76+
```bash
77+
subenum -silent -dL scope.txt -exclude-file out-of-scope.txt -stats run.json \
78+
| httpx -silent
79+
```
80+
81+
## Tips
82+
83+
- Pace the scan with `-rate` when the next tool shares the same resolver or
84+
network budget; subenum counts every wire query against it.
85+
- `-silent` still prints a one-line warning when results are simulated
86+
(`-simulate`), so a test run cannot be mistaken for a real one.
87+
- For change detection over time, see [Monitoring](monitoring.html).

‎internal/output/writer.go‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ type Writer struct {
8282
stderr io.Writer // diagnostics destination; nil means os.Stderr
8383
progressLen int // width of the progress line currently on screen
8484
lineMode bool // progress as whole lines, never carriage-return overwrites (#78)
85+
quiet bool // -silent: Info and Progress print nothing (#128)
8586

8687
seed uint64 // simulation seed, named in the text file header
8788
takeoverColumn bool // CSV carries a takeover_candidate column
@@ -119,6 +120,10 @@ func (w *Writer) SetPlain(plain bool) { w.plain = plain }
119120
// result glued onto a progress line (#78).
120121
func (w *Writer) SetProgressLines(lines bool) { w.lineMode = lines }
121122

123+
// SetQuiet makes Info and Progress print nothing, for -silent pipelines
124+
// (#128). Error and Notice still print.
125+
func (w *Writer) SetQuiet(quiet bool) { w.quiet = quiet }
126+
122127
// SetDiff switches to -diff output (#84): text lines get a "+ " or "- "
123128
// prefix and CSV gains a change column. JSON and JSONL carry Change as is.
124129
func (w *Writer) SetDiff(on bool) { w.diff = on }
@@ -351,6 +356,9 @@ func (w *Writer) errOut() io.Writer {
351356
func (w *Writer) Progress(pct float64, processed, total, found int64) {
352357
w.mu.Lock()
353358
defer w.mu.Unlock()
359+
if w.quiet {
360+
return
361+
}
354362
line := fmt.Sprintf("Progress: %.1f%% (%d/%d) | Found: %d ", pct, processed, total, found)
355363
if w.lineMode {
356364
_, _ = fmt.Fprintln(w.errOut(), strings.TrimSpace(line))
@@ -385,6 +393,15 @@ func (w *Writer) clearProgressLocked() {
385393
// Info writes an informational line to stderr. It is serialized with the
386394
// progress line, so concurrent verbose logging cannot splice into it (#36).
387395
func (w *Writer) Info(format string, a ...any) {
396+
if w.quiet {
397+
return
398+
}
399+
w.Notice(format, a...)
400+
}
401+
402+
// Notice writes a line to stderr like Info, even under SetQuiet: for what a
403+
// user must not miss, such as results being simulated.
404+
func (w *Writer) Notice(format string, a ...any) {
388405
w.mu.Lock()
389406
defer w.mu.Unlock()
390407
w.clearProgressLocked()

‎main.go‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,7 @@ type cliFlags struct {
125125
verbose bool
126126
showVersion bool
127127
showProgress bool
128+
silent bool // -silent: results only on stdout, errors only on stderr (#128)
128129
testMode bool
129130
testHitRate int
130131
seed uint64
@@ -178,6 +179,7 @@ func parseFlagsWith(args []string, applyDefaults func(*flag.FlagSet) error) (cli
178179
fs.BoolVar(&f.verbose, "v", false, "Enable verbose output")
179180
fs.BoolVar(&f.showVersion, "version", false, "Show version information")
180181
fs.BoolVar(&f.showProgress, "progress", true, "Show progress during scanning")
182+
fs.BoolVar(&f.silent, "silent", false, "Pipeline mode: bare result names on stdout, and only errors (and a simulation warning) on stderr")
181183
fs.BoolVar(&f.testMode, "simulate", false, "Run in simulation mode without actual DNS queries (for testing)")
182184
fs.IntVar(&f.testHitRate, "hit-rate", 15, "In simulation mode, percentage of names that resolve (1-100)")
183185
fs.Uint64Var(&f.seed, "seed", 0, "In simulation mode, seed for reproducible results (0 = random; the seed used is printed)")
@@ -589,6 +591,7 @@ func run() (code int) {
589591
recordTypes, typesErr := dns.ParseTypes(f.recordTypes)
590592
maxAttempts, err := resolveAttempts(f.attempts, f.retries, defaults.source(fs, "attempts"), defaults.source(fs, "retries"))
591593
out := output.New(nil, f.testMode, format)
594+
out.SetQuiet(f.silent)
592595
if formatErr != nil {
593596
out.Error("%v", formatErr)
594597
return exitUsage
@@ -623,6 +626,9 @@ func run() (code int) {
623626
if f.seed == 0 {
624627
f.seed = rand.Uint64()
625628
}
629+
if f.silent {
630+
out.Notice("SIMULATION MODE: results are artificially generated (seed %d)", f.seed)
631+
}
626632
out.Info("")
627633
out.Info("╔════════════════════════════════════════════════════════════════════╗")
628634
out.Info("║ SIMULATION MODE ACTIVE - NO ACTUAL DNS QUERIES WILL BE PERFORMED ║")
@@ -729,8 +735,10 @@ func run() (code int) {
729735
if w == nil {
730736
continue
731737
}
732-
// Bare names when piped (no "Found:" banner), human-friendly on a terminal.
733-
w.SetPlain(!stdoutIsTerminal())
738+
// Bare names when piped or -silent (no "Found:" banner), human-friendly
739+
// on a terminal.
740+
w.SetPlain(f.silent || !stdoutIsTerminal())
741+
w.SetQuiet(f.silent)
734742
w.SetShowRecords(f.showRecords)
735743
w.SetSeed(f.seed)
736744
w.SetTakeoverColumn(slices.Contains(recordTypes, "CNAME"))
@@ -770,6 +778,9 @@ func run() (code int) {
770778
// stderr is a pipe or file (2>&1, tee, CI logs) progress is off unless
771779
// -progress was given explicitly, and then printed as whole lines (#78).
772780
// SUBENUM_PROGRESS or a config value counts as explicit too (#116).
781+
if f.silent {
782+
f.showProgress = false
783+
}
773784
if !isTerminal(os.Stderr) {
774785
if defaults.source(fs, "progress") != sourceDefault {
775786
out.SetProgressLines(true)

‎main_e2e_test.go‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -698,3 +698,33 @@ func TestE2EResume(t *testing.T) {
698698
t.Errorf("-resume with extra arguments: exit %d, want 2", code)
699699
}
700700
}
701+
702+
// TestE2ESilent covers #128: -silent prints bare names on stdout and, on
703+
// stderr, nothing but the one-line simulation warning.
704+
func TestE2ESilent(t *testing.T) {
705+
wl := writeFile(t, "wl.txt", e2eWords)
706+
args := []string{"-silent", "-simulate", "-seed", "1", "-hit-rate", "100", "-w", wl, "example.com"}
707+
code, stdout := runCLI(t, "", args...)
708+
if code != 0 {
709+
t.Fatalf("exit %d", code)
710+
}
711+
names := strings.Split(strings.TrimSpace(stdout), "\n")
712+
if len(names) != 6 {
713+
t.Fatalf("stdout = %q, want 6 bare names", stdout)
714+
}
715+
for _, n := range names {
716+
if !strings.HasSuffix(n, ".example.com") || strings.Contains(n, " ") {
717+
t.Errorf("stdout line %q is not a bare name", n)
718+
}
719+
}
720+
_, merged := runCLIMerged(t, args...)
721+
var other []string
722+
for _, line := range strings.Split(strings.TrimSpace(merged), "\n") {
723+
if !strings.HasSuffix(line, ".example.com") {
724+
other = append(other, line)
725+
}
726+
}
727+
if len(other) != 1 || !strings.Contains(other[0], "SIMULATION") {
728+
t.Errorf("stderr under -silent = %q, want only the simulation warning", other)
729+
}
730+
}

0 commit comments

Comments
 (0)