Skip to content

Commit 239de37

Browse files
TMHSDigitalclaude
andcommitted
feat: run-quality report with a verdict (-stats, jsonl summary)
-stats <file> writes a schema-versioned JSON summary of the run, and -format jsonl ends with the same object ("type": "summary"): per-target outcomes, DNS queries actually dialed (counted in the resolver's Dial hook, retries included), achieved query rate, wildcard fingerprint, candidates skipped by -max-queries, and a complete / degraded / unreliable verdict with its reason, using the reliability guard's threshold. Failed and skipped targets are reported as unreliable, so CI can gate on the verdict even when a scan cannot run. README documents the schema and a jq CI gate. Fixes #70 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 410c498 commit 239de37

11 files changed

Lines changed: 528 additions & 16 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Added
11+
- Run-quality report: `-stats <file>` writes a schema-versioned JSON summary of a run (outcomes per target, DNS queries actually sent, achieved query rate, wildcard fingerprint, candidates skipped by `-max-queries`) with a `complete` / `degraded` / `unreliable` verdict and its reason. `-format jsonl` ends with the same object as its last line. The README documents the schema and how to gate CI on the verdict (#70).
12+
1013
### Changed
1114
- The minimum Go version is now 1.26 (`go 1.26.0` in go.mod), required by current `golang.org/x/sys` and `golang.org/x/text`; both were bumped and `govulncheck -show verbose` now reports no vulnerable modules. Dependabot also updates indirect modules, and CI lints with errorlint, nilerr, contextcheck, noctx, copyloopvar, usestdlibvars and revive, with errcheck exemptions narrowed to stdout prints (#83).
1215

‎README.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -255,6 +255,7 @@ make help # list all targets
255255
| `-simulate` | `false` | Simulation mode: no real DNS queries |
256256
| `-hit-rate <n>` | `15` | Simulated resolution rate, percent (1-100), applied uniformly to every name |
257257
| `-seed <n>` | `0` | Simulation seed; the same seed reproduces the same results (line order can vary unless `-t 1`). `0` picks a random seed and prints it |
258+
| `-stats <file>` | n/a | Write a JSON run-quality report (outcomes, queries sent, verdict); see [Run-quality report](#run-quality-report) |
258259
| `-tui` | `false` | Launch the interactive Terminal UI |
259260
| `-version` | n/a | Print version and exit |
260261
| `-retries <n>` | n/a | **Deprecated** - alias for `-attempts`, prints a warning |
@@ -332,6 +333,56 @@ Press `Ctrl+C` at any time to abort. In-flight queries drain, partial results ar
332333

333334
With `-dL`, each domain is scanned in turn as an independent scan: `-max-queries`, `-rate` and the reliability guard apply per target. A failed target does not stop the others, but a reliability abort (the resolver looks overloaded) skips the remaining targets unless `-no-abort` is set. A per-target status list is printed at the end.
334335

336+
### Run-quality report
337+
338+
Every lookup is accounted for, and subenum says how far the results can be trusted. `-stats run.json` writes a report in any format, and `-format jsonl` ends with the same object as its last line (`"type": "summary"`):
339+
340+
```json
341+
{
342+
"type": "summary",
343+
"schema": 1,
344+
"tool": "subenum",
345+
"version": "vX.Y.Z",
346+
"started": "2026-10-04T21:00:00Z",
347+
"duration_ms": 41230,
348+
"simulated": false,
349+
"resolver": "1.1.1.1:53",
350+
"rate_limit": 200,
351+
"queries_sent": 8214,
352+
"achieved_qps": 199.2,
353+
"verdict": "degraded",
354+
"reason": "61 of 4096 lookups failed (timeout 58, refused 3, other 0)",
355+
"targets": [
356+
{
357+
"domain": "example.com",
358+
"status": "ok",
359+
"processed": 4096, "total": 4096, "found": 37,
360+
"outcomes": { "resolved": 37, "nxdomain": 3998, "timeout": 58, "refused": 3, "other": 0, "wildcard_filtered": 0 },
361+
"queries_sent": 8214,
362+
"skipped_by_cap": 0,
363+
"wildcard": false,
364+
"verdict": "degraded",
365+
"reason": "61 of 4096 lookups failed (timeout 58, refused 3, other 0)"
366+
}
367+
]
368+
}
369+
```
370+
371+
| Verdict | Meaning |
372+
|---------|---------|
373+
| `complete` | Under 1% of lookups failed and nothing was skipped |
374+
| `degraded` | Usable, but some names are unknown: 1-20% of lookups failed, `-max-queries` skipped candidates, or the run was interrupted |
375+
| `unreliable` | Over 20% of lookups failed (the reliability guard's threshold), the guard aborted the scan, or the target could not be scanned |
376+
377+
The overall `verdict` is the worst target verdict. `queries_sent` counts DNS messages actually dialed, including retries; it is 0 in simulation mode. SERVFAIL and other errors Go's resolver does not distinguish are counted under `other`. `status` is one of `ok`, `failed`, `interrupted`, `skipped` or `not_run`. The `schema` number changes only when a field is renamed, removed or changes meaning; new fields can appear at any time.
378+
379+
Gate a CI job on the verdict with `jq`:
380+
381+
```bash
382+
subenum -w wordlist.txt -stats run.json example.com > found.txt
383+
jq -e '.verdict == "complete"' run.json > /dev/null || { jq -r .reason run.json; exit 1; }
384+
```
385+
335386
<br>
336387

337388
### Interactive TUI

‎docs/ROADMAP.md‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,6 @@ Work in progress or next up.
2626
Planned features that make results more trustworthy and the tool easier to get.
2727

2828
- **Resolver pool with trusted re-validation** ([#69](https://github.com/TMHSDigital/subenum/issues/69)), `-r resolvers.txt`.
29-
- **Machine-readable run-quality report** ([#70](https://github.com/TMHSDigital/subenum/issues/70)):
30-
per-query stats, wildcard drops and a confidence verdict.
3129
- **Scope control for bug-bounty programs** ([#87](https://github.com/TMHSDigital/subenum/issues/87)):
3230
`-exclude` patterns and an out-of-scope file.
3331
- **Monitoring mode** ([#84](https://github.com/TMHSDigital/subenum/issues/84)):
@@ -64,8 +62,8 @@ surprises:
6462
- **Stable output schemas.** The JSON, JSONL and CSV fields are documented and
6563
only ever gain fields, never lose or rename them.
6664
- **Documented exit codes**, already in place (0, 1, 2, 3, 130, 143).
67-
- **A run-quality report** ([#70](https://github.com/TMHSDigital/subenum/issues/70)),
68-
so a result set says how far it can be trusted.
65+
- **A run-quality report**, already in place (`-stats`, schema 1), so a result
66+
set says how far it can be trusted.
6967
- **Verifiable releases** ([#61](https://github.com/TMHSDigital/subenum/issues/61)):
7068
checksums, SBOM and signatures.
7169

‎internal/dns/resolver.go‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,9 @@ func NewResolver(timeout time.Duration, dnsServer string) *net.Resolver {
127127
return &net.Resolver{
128128
PreferGo: true,
129129
Dial: func(dialCtx context.Context, network, _ string) (net.Conn, error) {
130+
if c, ok := dialCtx.Value(queryCounterKey{}).(*atomic.Int64); ok {
131+
c.Add(1)
132+
}
130133
b, _ := dialCtx.Value(budgetKey{}).(*wireBudget)
131134
if b != nil {
132135
b.server.Store(dnsServer)
@@ -163,6 +166,15 @@ type wireBudget struct {
163166

164167
type budgetKey struct{}
165168

169+
type queryCounterKey struct{}
170+
171+
// WithQueryCounter attaches c to ctx; every DNS query a resolver from
172+
// NewResolver dials under that ctx (retries included) increments it, so a
173+
// scan can report the queries it actually sent (#70).
174+
func WithQueryCounter(ctx context.Context, c *atomic.Int64) context.Context {
175+
return context.WithValue(ctx, queryCounterKey{}, c)
176+
}
177+
166178
// fixErr rewrites what Go's resolver cannot report: it names the system
167179
// nameserver instead of the one dialed, and reports REFUSED as a generic
168180
// "server misbehaving", hiding the most common rate-limit signal (#50).

‎internal/output/summary.go‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
package output
2+
3+
import (
4+
"encoding/json"
5+
"fmt"
6+
"time"
7+
8+
"github.com/TMHSDigital/subenum/internal/dns"
9+
)
10+
11+
// SummarySchema versions the Summary object. Fields may be added without a
12+
// bump; renaming or removing one, or changing its meaning, bumps it (#70).
13+
const SummarySchema = 1
14+
15+
// Summary is the machine-readable run-quality report: what was scanned, how
16+
// many queries were sent, how every lookup ended, and a verdict on how far the
17+
// results can be trusted. It is written by -stats and, in jsonl format, as the
18+
// last line of the output.
19+
type Summary struct {
20+
Type string `json:"type"` // always "summary"
21+
Schema int `json:"schema"`
22+
Tool string `json:"tool"`
23+
Version string `json:"version"`
24+
Started time.Time `json:"started"`
25+
DurationMs int64 `json:"duration_ms"`
26+
Simulated bool `json:"simulated"`
27+
Seed uint64 `json:"seed,omitempty"`
28+
Resolver string `json:"resolver,omitempty"`
29+
RateLimit int `json:"rate_limit"`
30+
QueriesSent int64 `json:"queries_sent"`
31+
AchievedQPS float64 `json:"achieved_qps"`
32+
Verdict string `json:"verdict"`
33+
Reason string `json:"reason"`
34+
Targets []TargetSummary `json:"targets"`
35+
}
36+
37+
// TargetSummary is one scanned domain's part of a Summary.
38+
type TargetSummary struct {
39+
Domain string `json:"domain"`
40+
Status string `json:"status"` // ok, failed, skipped, interrupted, not_run
41+
Error string `json:"error,omitempty"`
42+
Processed int64 `json:"processed"`
43+
Total int64 `json:"total"`
44+
Found int64 `json:"found"`
45+
Outcomes Outcomes `json:"outcomes"`
46+
QueriesSent int64 `json:"queries_sent"`
47+
SkippedCap int64 `json:"skipped_by_cap"`
48+
Wildcard bool `json:"wildcard"`
49+
Fingerprint []dns.Record `json:"wildcard_fingerprint,omitempty"`
50+
Verdict string `json:"verdict"`
51+
Reason string `json:"reason"`
52+
}
53+
54+
// Outcomes counts how each lookup ended. SERVFAIL and other errors Go's
55+
// resolver does not distinguish are counted under Other.
56+
type Outcomes struct {
57+
Resolved int64 `json:"resolved"`
58+
NXDomain int64 `json:"nxdomain"`
59+
Timeout int64 `json:"timeout"`
60+
Refused int64 `json:"refused"`
61+
Other int64 `json:"other"`
62+
WildcardFiltered int64 `json:"wildcard_filtered"`
63+
}
64+
65+
// Summary writes s as the final line of jsonl output (stdout and the -o
66+
// file). Other formats ignore it; use -stats for those.
67+
func (w *Writer) Summary(s Summary) {
68+
if w.format != FormatJSONL {
69+
return
70+
}
71+
w.mu.Lock()
72+
defer w.mu.Unlock()
73+
data, err := json.Marshal(s)
74+
if err != nil {
75+
_, _ = fmt.Fprintf(w.errOut(), "Error: encoding summary: %v\n", err)
76+
return
77+
}
78+
if w.stdout {
79+
w.clearProgressLocked()
80+
fmt.Printf("%s\n", data)
81+
}
82+
if w.outWriter != nil {
83+
_, _ = fmt.Fprintf(w.outWriter, "%s\n", data)
84+
}
85+
}
86+
87+
// WriteSummaryFile writes s as indented JSON to path, atomically.
88+
func WriteSummaryFile(path string, s Summary) error {
89+
f, err := CreateFile(path)
90+
if err != nil {
91+
return err
92+
}
93+
data, err := json.MarshalIndent(s, "", " ")
94+
if err != nil {
95+
_ = f.Close(false)
96+
return err
97+
}
98+
_, werr := f.Write(append(data, '\n'))
99+
if cerr := f.Close(werr == nil); werr == nil {
100+
werr = cerr
101+
}
102+
return werr
103+
}

‎internal/scan/runner.go‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ import (
77
"context"
88
"fmt"
99
"net"
10+
"sort"
1011
"strings"
1112
"sync"
1213
"sync/atomic"
@@ -67,6 +68,13 @@ type Stats struct {
6768
Refused int64
6869
Other int64
6970
WildcardFiltered int64
71+
72+
// Run facts for the quality report (#70); not part of Sum.
73+
QueriesSent int64 // DNS queries dialed, retries included (0 when simulated)
74+
Skipped int64 // candidates not tested because -max-queries was reached
75+
Aborted bool // the reliability guard cancelled the scan
76+
Wildcard bool // the root domain is a wildcard (scanned under -force)
77+
Fingerprint []dns.Record // final wildcard fingerprint, empty when none
7078
}
7179

7280
// Sum returns the number of classified queries (found + negatives + failures +
@@ -86,6 +94,8 @@ type counters struct {
8694
refused atomic.Int64
8795
other atomic.Int64
8896
wildcardFiltered atomic.Int64
97+
skipped atomic.Int64
98+
aborted atomic.Bool
8999
}
90100

91101
func (c *counters) add(o dns.Outcome) {
@@ -113,6 +123,8 @@ func (c *counters) snapshot() Stats {
113123
Refused: c.refused.Load(),
114124
Other: c.other.Load(),
115125
WildcardFiltered: c.wildcardFiltered.Load(),
126+
Skipped: c.skipped.Load(),
127+
Aborted: c.aborted.Load(),
116128
}
117129
}
118130

@@ -166,6 +178,23 @@ func (f *fingerprint) add(recs []dns.Record) {
166178
}
167179
}
168180

181+
// records returns the fingerprint's records in a stable order.
182+
func (f *fingerprint) records() []dns.Record {
183+
f.mu.RLock()
184+
keys := make([]string, 0, len(f.set))
185+
for k := range f.set {
186+
keys = append(keys, k)
187+
}
188+
f.mu.RUnlock()
189+
sort.Strings(keys)
190+
out := make([]dns.Record, 0, len(keys))
191+
for _, k := range keys {
192+
typ, val, _ := strings.Cut(k, "\x00")
193+
out = append(out, dns.Record{Type: typ, Value: val})
194+
}
195+
return out
196+
}
197+
169198
// match reports whether every record of got is a known wildcard answer
170199
// (covered) and whether at least one is (overlaps). An empty fingerprint or
171200
// result never matches.
@@ -347,6 +376,9 @@ func Run(ctx context.Context, cfg Config, events chan<- Event) {
347376
} else {
348377
ctx = dns.WithLimiter(ctx, limiter)
349378
}
379+
var queriesSent atomic.Int64
380+
ctx = dns.WithQueryCounter(ctx, &queriesSent)
381+
wildcardRoot := false
350382

351383
if cfg.Recursive {
352384
ceiling := RecursionCeiling(len(cfg.Entries), maxDepth)
@@ -401,6 +433,7 @@ func Run(ctx context.Context, cfg Config, events chan<- Event) {
401433
}
402434
fingerprint.add(fp)
403435
if isWildcard {
436+
wildcardRoot = true
404437
msg := "WARNING: Wildcard DNS detected - all subdomains resolve for " + cfg.Domain
405438
events <- Event{Kind: EventNotice, Notice: NoticeWildcard, Message: msg}
406439
if !cfg.Force {
@@ -519,6 +552,7 @@ func Run(ctx context.Context, cfg Config, events chan<- Event) {
519552
return job{}, false
520553
}
521554
closeJobs := func() {
555+
stats.skipped.Store(int64(skipped))
522556
if skipped > 0 {
523557
select {
524558
case events <- Event{Kind: EventNotice, Notice: NoticeCap, Message: fmt.Sprintf(
@@ -589,6 +623,9 @@ func Run(ctx context.Context, cfg Config, events chan<- Event) {
589623
<-tickerStopped
590624

591625
snap := stats.snapshot()
626+
snap.QueriesSent = queriesSent.Load()
627+
snap.Wildcard = wildcardRoot
628+
snap.Fingerprint = fingerprint.records()
592629
// EventDone is the abort/finish signal. CLI Finish and TUI aborted=true both
593630
// wait for it after cancel, so this send must not be select-guarded on
594631
// ctx.Done(): once ctx is cancelled that select is a coin-flip drop.
@@ -698,6 +735,7 @@ func checkReliability(cfg Config, processed int64, st *counters, events chan<- E
698735
verb, rate, processed, cfg.Concurrency, cfg.Rate)
699736
events <- Event{Kind: EventError, Message: msg}
700737
if !cfg.NoAbort {
738+
st.aborted.Store(true)
701739
cancel()
702740
}
703741
})

‎internal/scan/verdict.go‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
package scan
2+
3+
import "fmt"
4+
5+
// Run-quality verdicts, from best to worst (#70).
6+
const (
7+
VerdictComplete = "complete" // every candidate got a definitive answer, or nearly so
8+
VerdictDegraded = "degraded" // usable, but some candidates are unknown
9+
VerdictUnreliable = "unreliable" // too many failures to trust the absence of a name
10+
)
11+
12+
// degradedFailPercent is the share of failed lookups at which a finished scan
13+
// is degraded. Above reliabilityFailPercent it is unreliable, the same line
14+
// the reliability guard aborts at.
15+
const degradedFailPercent = 1
16+
17+
// VerdictRank orders verdicts so the worst of several can be picked.
18+
func VerdictRank(v string) int {
19+
switch v {
20+
case VerdictComplete:
21+
return 0
22+
case VerdictDegraded:
23+
return 1
24+
}
25+
return 2
26+
}
27+
28+
// Verdict grades a scan that reached EventDone. interrupted is set when the
29+
// user stopped it (Ctrl+C, SIGTERM), which Stats cannot know.
30+
func (s Stats) Verdict(interrupted bool) (verdict, reason string) {
31+
processed := s.Sum()
32+
fails := s.failures()
33+
detail := fmt.Sprintf("%d of %d lookups failed (timeout %d, refused %d, other %d)",
34+
fails, processed, s.Timeout, s.Refused, s.Other)
35+
switch {
36+
case s.Aborted:
37+
return VerdictUnreliable, "the reliability guard aborted the scan: " + detail
38+
case processed > 0 && fails*100 > processed*reliabilityFailPercent:
39+
return VerdictUnreliable, detail
40+
case interrupted:
41+
return VerdictDegraded, fmt.Sprintf("interrupted after %d lookups", processed)
42+
case fails > 0 && fails*100 >= processed*degradedFailPercent:
43+
return VerdictDegraded, detail
44+
case s.Skipped > 0:
45+
return VerdictDegraded, fmt.Sprintf("-max-queries reached; %d candidates not tested", s.Skipped)
46+
case fails > 0:
47+
return VerdictComplete, detail + ", under 1%"
48+
}
49+
return VerdictComplete, fmt.Sprintf("all %d lookups got a definitive answer", processed)
50+
}

0 commit comments

Comments
 (0)