Skip to content

Commit 4c12fe8

Browse files
TMHSDigitalclaude
andcommitted
Derive the MCP verdict from the library's Stats.Verdict (#263)
runMCPScan rebuilt a scan.Stats from the public Stats field by field to reach the engine's verdict, a reverse mapping no test guarded. The library now exports Stats.Verdict and the Verdict* constants, through engineStats, the inverse of stats, which a round-trip reflection test covers. The MCP server uses it via mcpVerdict, keeping its two documented additions (not_run, and unreliable after an error), and a table test checks its verdicts against internal/report for the same scans. Reasons now name max_queries and the server's -max-duration instead of raw setting tokens. Fixes #263 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 8c42262 commit 4c12fe8

7 files changed

Lines changed: 156 additions & 13 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2222
- Takeover hints recognize two more providers: a CNAME to `*.animaapp.io` is marked `provider:anima` and one to `*.trydiscourse.com` is marked `provider:discourse` (#134, thanks @jdotc1).
2323
- `-log-format json` writes the diagnostics on stderr (notices, warnings, errors, `-v` lines) as JSON Lines, one object per line with `time`, `level` (`error`, `warn`, `notice`, `info`, `debug`), `kind` (`message`, `scan`, `verbose`, `progress`, `simulation`, or an engine notice's category such as `wildcard` or `cap`), the `target` being scanned, the `message`, and the `settings` codes an engine message names (`["max_queries"]`). The simulation banner becomes one `simulation` notice, progress is an event with its numbers and only with an explicit `-progress`, and `-silent` keeps its meaning; results on stdout are unchanged. `SUBENUM_LOG_FORMAT` and the config file set it like any flag (#198).
2424
- `-dry-run` prints what a scan would do without scanning: per target the candidates left after dedupe, invalid entries and `-exclude`, the queries (one per name and record type, and the most with every `-attempts` retry, both capped by `-max-queries`), how long they take at `-rate`, and whether the resolver passes the preflight, which is its only DNS query. `-format json` or `jsonl` prints the plan as JSON. No `-o` file is written, and the run exits 1 only if a preflight fails. `-ct`, `-permute` and `-recursive` add work it cannot count without querying, and it says so (#192).
25+
- `pkg/subenum` exports `Stats.Verdict(interrupted)` and the `VerdictComplete`, `VerdictDegraded` and `VerdictUnreliable` constants: a finished scan's verdict and reason by the same rules as the CLI's run-quality report. `subenum mcp` now takes its verdict from it rather than rebuilding the engine's counters by hand, and its reasons name `max_queries` and "the server's -max-duration" instead of raw setting tokens (#263).
2526
- `pkg/subenum` exports `Permutations(seeds, words []string) []string`, the candidate generator behind `-permute`: from names already found it returns the nearby variants (`api-dev`, `dev-api`, `dev.api`, `api2`), relative to the apex and ready to pass as a second scan's `Words`. `nil` words selects the CLI's built-in list. It sends no DNS queries (#197).
2627
- `-tui` pre-fills its form from the command line: `subenum -tui example.com -w big.txt -r resolvers.txt` opens with the domain, wordlist and resolver file filled in, over `SUBENUM_*`, config file and remembered values. Flags the form has no field for are named in a warning instead of being dropped silently; the form takes the first `-w`. The form gains **Resolver Pool** (`-r`) and **Rate/Resolver** (`-rate-per-resolver`) fields, remembered in `last.json`. After a scan, `r` restores the values just scanned rather than re-applying the launch flags over them (#195).
2728
- `-rate-per-resolver N` caps the DNS queries per second sent to each `-r` pool resolver, so a high global `-rate` no longer lands in uneven bursts on single members. Lookups go to resolvers with room first and wait only when every cap is spent; `-rate` still caps the total, and the stricter limit wins. The end-of-run resolver table and the report's `resolvers` entries gain a `throttled` count, and `pkg/subenum` gains `Config.RatePerResolver` (#196).

‎docs/_data/unreleased.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,3 +34,4 @@ library:
3434
- Sources
3535
- RetryInconclusive
3636
- TakeoverProviders
37+
- Verdict

‎docs/library.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@ for _, c := range changes.Changed {
117117

118118
## Accounting
119119

120-
`Stats` puts each candidate name into exactly one outcome: `Found`, `NXDomain`, `Timeout`, `Refused`, `Other` or `WildcardFiltered`. Alongside are `QueriesSent`, `Skipped` (left untested by `MaxQueries` or `MaxDuration`, with `Expired` set for the latter), `Excluded`, the result counters `Takeover` and `Internal`, and the resolver-pool counters `PoolHits`, `Confirmed` and `Unconfirmed`. `Aborted` means the reliability guard stopped the scan, so the negatives mean nothing. `Wildcard` and `WildcardFingerprint` describe a wildcard zone. `RetryInconclusive` fills `Retried`, `Recovered` and `Inconclusive` (the names still without an answer), and `AutoRate` fills `RateBackoffs` and `FinalRate`. `Total()` sums the outcomes.
120+
`Stats` puts each candidate name into exactly one outcome: `Found`, `NXDomain`, `Timeout`, `Refused`, `Other` or `WildcardFiltered`. Alongside are `QueriesSent`, `Skipped` (left untested by `MaxQueries` or `MaxDuration`, with `Expired` set for the latter), `Excluded`, the result counters `Takeover` and `Internal`, and the resolver-pool counters `PoolHits`, `Confirmed` and `Unconfirmed`. `Aborted` means the reliability guard stopped the scan, so the negatives mean nothing. `Wildcard` and `WildcardFingerprint` describe a wildcard zone. `RetryInconclusive` fills `Retried`, `Recovered` and `Inconclusive` (the names still without an answer), and `AutoRate` fills `RateBackoffs` and `FinalRate`. `Total()` sums the outcomes. `Verdict(interrupted)`{% include unreleased.html library="Verdict" %} applies the CLI run-quality report's rules: `VerdictComplete`, `VerdictDegraded` or `VerdictUnreliable`, with the reason in plain words (naming `Config` fields such as `MaxQueries`).
121121

122122
## Testing your integration
123123

‎main_mcp.go‎

Lines changed: 20 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -530,21 +530,29 @@ func runMCPScan(ctx context.Context, cfg subenum.Config) *mcpReport {
530530
report.Outcomes = mcpOutcomes{NXDomain: stats.NXDomain, Timeout: stats.Timeout, Refused: stats.Refused, Other: stats.Other, WildcardFiltered: stats.WildcardFiltered}
531531
report.Skipped = stats.Skipped
532532
report.Wildcard = stats.Wildcard
533-
verdict, reason := scan.Stats{
534-
Found: stats.Found, NXDomain: stats.NXDomain, Timeout: stats.Timeout, Refused: stats.Refused,
535-
Other: stats.Other, WildcardFiltered: stats.WildcardFiltered,
536-
Skipped: stats.Skipped, Aborted: stats.Aborted, Expired: stats.Expired,
537-
}.Verdict(ctx.Err() != nil)
538-
report.Verdict = verdict
539-
report.Reason = scan.Render(reason, func(s scan.Setting) string { return string(s) })
533+
report.Verdict, report.Reason = mcpVerdict(stats, report.Error, ctx.Err() != nil)
534+
return report
535+
}
536+
537+
// mcpReasonTerms renders the Config fields a verdict reason names in the
538+
// terms an agent sees: its own max_queries argument, the operator's
539+
// -max-duration.
540+
var mcpReasonTerms = strings.NewReplacer("MaxQueries", "max_queries", "MaxDuration", "the server's -max-duration")
541+
542+
// mcpVerdict is a scan's verdict for the agent. It is the library's verdict
543+
// (the CLI report's rules, #263) with two additions for a report that has
544+
// no status field: an error before any name was tested is not_run, and an
545+
// error after an otherwise complete scan is unreliable.
546+
func mcpVerdict(stats subenum.Stats, errMsg string, interrupted bool) (verdict, reason string) {
547+
verdict, reason = stats.Verdict(interrupted)
540548
switch {
541-
case report.Error != "" && stats.Total() == 0:
549+
case errMsg != "" && stats.Total() == 0:
542550
// A wildcard zone, a failed resolver check: nothing was tested.
543-
report.Verdict, report.Reason = verdictNotRun, "the scan stopped before testing any name; see error"
544-
case report.Error != "" && verdict == scan.VerdictComplete:
545-
report.Verdict, report.Reason = scan.VerdictUnreliable, "the scan stopped early; see error"
551+
return verdictNotRun, "the scan stopped before testing any name; see error"
552+
case errMsg != "" && verdict == subenum.VerdictComplete:
553+
return subenum.VerdictUnreliable, "the scan stopped early; see error"
546554
}
547-
return report
555+
return verdict, mcpReasonTerms.Replace(reason)
548556
}
549557

550558
func (r *mcpReport) result() mcp.Result {

‎main_mcp_test.go‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ import (
1515
"github.com/TMHSDigital/subenum/internal/dnsserver"
1616
"github.com/TMHSDigital/subenum/internal/labzone"
1717
"github.com/TMHSDigital/subenum/internal/output"
18+
"github.com/TMHSDigital/subenum/internal/report"
19+
"github.com/TMHSDigital/subenum/internal/scan"
1820
"github.com/TMHSDigital/subenum/pkg/subenum"
1921
)
2022

@@ -311,6 +313,53 @@ func TestMCPExplainStatsBranches(t *testing.T) {
311313
}
312314
}
313315

316+
// TestMCPVerdictMatchesReport is #263's drift guard: for the same finished
317+
// scan the agent gets the verdict the CLI's run-quality report gives, except
318+
// for the two cases the MCP report covers without a status field.
319+
func TestMCPVerdictMatchesReport(t *testing.T) {
320+
type counts struct {
321+
found, nx, timeout, skipped int64
322+
expired, aborted, wildcard bool
323+
}
324+
for _, tc := range []struct {
325+
name string
326+
c counts
327+
errMsg string
328+
interrupted bool
329+
want string // "" = the report's verdict
330+
}{
331+
{name: "complete", c: counts{found: 2, nx: 8}},
332+
{name: "query cap", c: counts{nx: 8, skipped: 2}},
333+
{name: "time cap", c: counts{nx: 8, skipped: 2, expired: true}},
334+
{name: "failures", c: counts{nx: 1, timeout: 9}},
335+
{name: "aborted", c: counts{nx: 3, timeout: 7, aborted: true}},
336+
{name: "wildcard", c: counts{found: 1, nx: 9, wildcard: true}},
337+
{name: "interrupted", c: counts{nx: 5}, interrupted: true},
338+
{name: "error before testing", errMsg: "wildcard zone", want: verdictNotRun},
339+
{name: "error after a complete scan", c: counts{nx: 10}, errMsg: "boom", want: "unreliable"},
340+
} {
341+
c := tc.c
342+
pub := subenum.Stats{Found: c.found, NXDomain: c.nx, Timeout: c.timeout, Skipped: c.skipped, Expired: c.expired, Aborted: c.aborted, Wildcard: c.wildcard}
343+
eng := scan.Stats{Found: c.found, NXDomain: c.nx, Timeout: c.timeout, Skipped: c.skipped, Expired: c.expired, Aborted: c.aborted, Wildcard: c.wildcard}
344+
status := "ok"
345+
if tc.interrupted {
346+
status = "interrupted"
347+
}
348+
rep := report.Target("x.example", status, "", true, scan.Event{Kind: scan.EventDone, Stats: eng}, scan.FieldName)
349+
want := tc.want
350+
if want == "" {
351+
want = rep.Verdict
352+
}
353+
got, reason := mcpVerdict(pub, tc.errMsg, tc.interrupted)
354+
if got != want {
355+
t.Errorf("%s: MCP verdict %s (%s), want %s (report: %s, %s)", tc.name, got, reason, want, rep.Verdict, rep.Reason)
356+
}
357+
if strings.ContainsAny(reason, "{}") || strings.Contains(reason, "MaxDuration") || strings.Contains(reason, "MaxQueries") {
358+
t.Errorf("%s: reason %q names a setting the agent does not know", tc.name, reason)
359+
}
360+
}
361+
}
362+
314363
func TestMCPFlagErrors(t *testing.T) {
315364
for _, tc := range []struct {
316365
args []string

‎pkg/subenum/stats_test.go‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ package subenum
22

33
import (
44
"reflect"
5+
"strings"
56
"testing"
67

78
"github.com/TMHSDigital/subenum/internal/dns"
@@ -57,4 +58,32 @@ func TestStatsCoversEveryEngineField(t *testing.T) {
5758
t.Errorf("subenum.Stats.%s is never set from scan.Stats", got.Type().Field(i).Name)
5859
}
5960
}
61+
// engineStats is stats' inverse (#263): every field survives the round
62+
// trip, so Stats.Verdict sees what the engine saw.
63+
all.RootDone = 0
64+
if back := engineStats(stats(all)); !reflect.DeepEqual(back, all) {
65+
t.Errorf("engineStats(stats(s)) = %+v, want %+v", back, all)
66+
}
67+
}
68+
69+
// TestStatsVerdict: the public verdict follows the engine's rules and names
70+
// Config fields in its reason (#263).
71+
func TestStatsVerdict(t *testing.T) {
72+
for _, tc := range []struct {
73+
s Stats
74+
interrupted bool
75+
want string
76+
reason string
77+
}{
78+
{Stats{NXDomain: 10, Found: 2}, false, VerdictComplete, ""},
79+
{Stats{NXDomain: 8, Skipped: 2}, false, VerdictDegraded, "MaxQueries"},
80+
{Stats{NXDomain: 8, Skipped: 2, Expired: true}, false, VerdictDegraded, "MaxDuration"},
81+
{Stats{NXDomain: 1, Timeout: 9}, false, VerdictUnreliable, ""},
82+
{Stats{NXDomain: 5}, true, VerdictDegraded, ""},
83+
} {
84+
v, reason := tc.s.Verdict(tc.interrupted)
85+
if v != tc.want || !strings.Contains(reason, tc.reason) || strings.Contains(reason, "{") {
86+
t.Errorf("%+v (interrupted %v): %s (%s), want %s naming %q", tc.s, tc.interrupted, v, reason, tc.want, tc.reason)
87+
}
88+
}
6089
}

‎pkg/subenum/subenum.go‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -777,6 +777,61 @@ func notice(n scan.NoticeKind) Notice {
777777
return NoticeNone
778778
}
779779

780+
// Verdicts a finished scan can get from Stats.Verdict, the same ones the
781+
// CLI's run-quality report gives.
782+
const (
783+
VerdictComplete = scan.VerdictComplete // a name missing from the results most likely does not exist
784+
VerdictDegraded = scan.VerdictDegraded // some names were not tested or got no answer
785+
VerdictUnreliable = scan.VerdictUnreliable // too many lookups failed: a missing name means nothing
786+
)
787+
788+
// Verdict says how far the results of a finished scan can be trusted, by
789+
// the rules of the CLI's run-quality report: VerdictComplete,
790+
// VerdictDegraded or VerdictUnreliable, with the reason in plain words.
791+
// interrupted reports that the scan was cancelled before it ended. The
792+
// reason names Config fields, such as MaxQueries.
793+
func (s Stats) Verdict(interrupted bool) (verdict, reason string) {
794+
v, r := engineStats(s).Verdict(interrupted)
795+
return v, scan.Render(r, scan.FieldName)
796+
}
797+
798+
// engineStats is the inverse of stats, for the engine's verdict rules.
799+
func engineStats(s Stats) scan.Stats {
800+
var fp []dns.Record
801+
if len(s.WildcardFingerprint) > 0 {
802+
fp = dnsRecords(s.WildcardFingerprint)
803+
}
804+
return scan.Stats{
805+
Found: s.Found,
806+
NXDomain: s.NXDomain,
807+
Timeout: s.Timeout,
808+
Refused: s.Refused,
809+
Other: s.Other,
810+
WildcardFiltered: s.WildcardFiltered,
811+
812+
QueriesSent: s.QueriesSent,
813+
Skipped: s.Skipped,
814+
Excluded: s.Excluded,
815+
Takeover: s.Takeover,
816+
Internal: s.Internal,
817+
818+
PoolHits: s.PoolHits,
819+
Confirmed: s.Confirmed,
820+
Unconfirmed: s.Unconfirmed,
821+
822+
Aborted: s.Aborted,
823+
Expired: s.Expired,
824+
Wildcard: s.Wildcard,
825+
Fingerprint: fp,
826+
827+
Retried: s.Retried,
828+
Recovered: s.Recovered,
829+
Inconclusive: s.Inconclusive,
830+
RateBackoffs: s.RateBackoffs,
831+
FinalRate: s.FinalRate,
832+
}
833+
}
834+
780835
func stats(s scan.Stats) Stats {
781836
return Stats{
782837
Found: s.Found,

0 commit comments

Comments
 (0)