Skip to content

Commit 70b1c06

Browse files
TMHSDigitalclaude
andcommitted
Refuse shared stdin, clashing output files; spare MCP budget (#378)
Count every flag that can read standard input (-w, -dL, -r, -exclude-file, -takeover-providers) before any input is read, so a second reader is a usage error instead of failing on an empty list. Compare -o files with samePath (case-insensitive on Windows and macOS for files not yet created), and refuse an -inconclusive, -stats or -sarif file that is also an -o file or another of them. subenum mcp validates a scan call with the new subenum.Config.Validate before counting it, so a call that cannot start no longer spends one of the session's -max-calls. Fixes #378 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 330ff05 commit 70b1c06

13 files changed

Lines changed: 233 additions & 23 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010
### Added
11+
- `pkg/subenum`: `Config.Validate()` reports the error `Run` would return for a config, such as an unsupported type, an invalid `Exclude` pattern or `ErrNoWords`, without starting a scan or sending a query (#378).
1112
- `-simulate-zone` accepts the lab scenarios built into every binary by name, so release, container and `go install` users run the labs without cloning the repository: `subenum -simulate-zone lab1 lab.example` works from any directory (`lab1`, `lab1-first-scan` and `lab1-first-scan.zone` all name it), and without `-w` the scan uses the labs' own wordlist (`-w ""` keeps the bundled 5,000-word list). A path, or a file that exists under the given name, is still read from disk, and an unknown name lists the built-in scenarios (#354).
1213
- `pkg/subenum`: `Result.Sources` names what proposed each result, as the CLI's `sources` field does: `wordlist`, and each `Config.Sources` entry that returned it, named by a `Name()` method when it has one (`ct`, `wayback`, `axfr` for the built-in sources) or as `Sources[i]` (#273).
1314
- `-o` can be given more than once, and a format prefix gives a file its own format: `-o names.txt -o jsonl:results.jsonl` writes bare names for the next tool and JSONL with the run-quality report in one run, where a pipeline used to scan twice or extract names with `jq`. A file without a prefix uses `-format`, as before. Every file gets the full current results, also under `-diff`; on a run stopped with state for `-resume`, only an `-o` file that is also the `-diff` file is left unchanged. The same path given twice is a usage error, and `-tui` takes the first file (#334).
@@ -84,6 +85,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
8485
- The GitHub Action passes every line of a multi-line `args: |` block to subenum instead of only the first. `fail-on: never` no longer fails the step when a single-domain scan fails (exit 1 with an `unreliable` verdict); a usage error or a run with no report still does. A second use in the same job no longer reports the first use's verdict and counts when its own scan exits before writing a report. The `found` output and the summary table count `-permute` hits too (#363).
8586
- The GitHub Action's job summary cuts its takeover and change lists to 50 entries in `jq` itself instead of piping into `head -50`, which under the step's `pipefail` could fail the step on a very long list after the scan had succeeded (#345).
8687
- The TUI's finished screen wraps its counters between counters to the terminal width and gives the extra lines to the layout, so with every counter set it still fits an 80×24 terminal instead of the terminal wrapping the line and pushing the screen one row down. ctrl+c during the permutation pass after a `-no-abort` reliability warning shows an abort, not a failure (#343).
88+
- Standard input can be named by only one input flag: `-w -`, `-dL -`, `-r -`, `-exclude-file -` and `-takeover-providers -` are all counted, and a second one is a usage error naming them, before anything is read. Only `-w` with `-dL` used to be refused; with the others the first reader drained standard input and a later one failed on an empty list, with an error about that list instead of the shared input. One `-o` file named twice in different spellings (a relative and an absolute path, `x` and `sub/../x`, another case on Windows and macOS), or an `-inconclusive`, `-stats` or `-sarif` file that is also an `-o` file or another of them, is a usage error too, where only identical spellings were refused and the last file written silently replaced the others. A `subenum mcp` `scan` call that cannot start (an unsupported type, an invalid `exclude` pattern, no usable words) is reported `not_run` as before but no longer spends one of the session's `-max-calls` (#378).
8789
- A `-diff` baseline that is also an `-o` file is recognised however the two paths are spelled (`x` and `./x`, a relative and an absolute path, another case on Windows), so a stopped run leaves it unchanged for the resumed run, as it already did for identical spellings (#342).
8890
- A `CNAME` lookup of a name with an empty label or one over 63 bytes is reported as not found without a query, as a name over 255 bytes already was, instead of dialling, counting a query and reporting a failure that the reliability guard counts (#344).
8991
- Under `-no-abort`, the reliability guard's warning is printed as `Warning: ...` (level `warn` with `-log-format json`) instead of `Error: warning: ...`, and a target that finished no longer carries it as its `error` in the `-stats` report next to `"status": "ok"`. The TUI shows it as a warning too (#308).

‎docs/mcp.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ subenum mcp -allow example.com,example.org # adds scan, for these domains
9696
| `-max-duration` | 2m | Longest one call may scan |
9797
| `-timeout` | 1000 | Live-scan lookup timeout in milliseconds |
9898
| `-sources` | none | Passive sources (`ct`, `wayback`) a `scan` call may ask for |
99-
| `-max-calls` | 100 | Most `scan` calls one session may make; with `-max-queries` it caps the session's live lookups |
99+
| `-max-calls` | 100 | Most `scan` calls one session may make; with `-max-queries` it caps the session's live lookups. A call refused for its arguments (`not_run` before any query) does not count |
100100

101101
Passive sources are off unless you enable them, because they send the domain
102102
to third parties (crt.sh, the Wayback Machine). With `-sources ct,wayback`,

‎docs/pipelines.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,10 @@ tail -n 1 results.jsonl | jq -e '.verdict == "complete"' > /dev/null || exit 1
6060
httpx -silent -l names.txt
6161
```
6262

63+
Each `-o`, `-stats`, `-sarif` and `-inconclusive` file must be its own: one
64+
file named twice, however it is spelled, is a usage error rather than one
65+
output silently replacing the other.
66+
6367
## Takeover candidates first
6468

6569
With CNAME lookups, subenum marks names whose alias target does not resolve or

‎examples/advanced_usage.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ Pass a file of apex domains (one per line, `#` comments allowed) with `-dL`. Eac
102102
./subenum -w wordlist.txt -dL examples/sample_domains.txt -format jsonl -o results.jsonl
103103
```
104104

105-
Either input can come from standard input with `-` (but not both):
105+
Either input can come from standard input with `-`, but only one flag can read it (this holds for `-r`, `-exclude-file` and `-takeover-providers` too):
106106

107107
```bash
108108
cat targets.txt | ./subenum -w wordlist.txt -dL -

‎internal/cli/cli.go‎

Lines changed: 58 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,6 @@ import (
2929
"fmt"
3030
"io"
3131
"os"
32-
"path/filepath"
3332
"runtime/debug"
3433
"slices"
3534
"strings"
@@ -309,20 +308,41 @@ func validateArgs(f cliFlags, positionals []string, fs *flag.FlagSet, out *outpu
309308
out.Error("expected exactly one domain, got %d arguments: %s (use -dL for several)", len(positionals), strings.Join(positionals, " "))
310309
return false
311310
}
311+
// Standard input can only be read once: the first reader drains it and
312+
// every later one would see an empty file (#378).
313+
if n := countStdin(f.wordlists.paths); n > 1 {
314+
out.Error("-w - is given %d times; standard input can only be read once", n)
315+
return false
316+
}
317+
if readers := stdinReaders(f); len(readers) > 1 {
318+
out.Error("%s cannot all read standard input (-); it can only be read once, so give all but one a file", strings.Join(readers, ", "))
319+
return false
320+
}
312321
return true
313322
}
314323

324+
// stdinReaders names the flags set to "-", standard input.
325+
func stdinReaders(f cliFlags) []string {
326+
var names []string
327+
if f.wordlists.readsStdin() {
328+
names = append(names, "-w")
329+
}
330+
for _, c := range []struct{ name, path string }{
331+
{"-dL", f.domainList},
332+
{"-r", f.resolvers},
333+
{"-exclude-file", f.excludeFile},
334+
{"-takeover-providers", f.providersFile},
335+
} {
336+
if c.path == wordlist.Stdin {
337+
names = append(names, c.name)
338+
}
339+
}
340+
return names
341+
}
342+
315343
// validateFlags checks options after validateArgs has checked the domain
316344
// arguments. configure does that before reading any input files.
317345
func validateFlags(f cliFlags, in *runtimeInputs, positionals []string, fs *flag.FlagSet, out *output.Writer) (string, bool) {
318-
if f.wordlists.readsStdin() && f.domainList == wordlist.Stdin {
319-
out.Error("-w and -dL cannot both read standard input")
320-
return "", false
321-
}
322-
if n := countStdin(f.wordlists.paths); n > 1 {
323-
out.Error("-w - is given %d times; standard input can only be read once", n)
324-
return "", false
325-
}
326346
if f.simZone != "" {
327347
// Effective values, not just command-line flags: a resolver list or
328348
// simulation mode from the config file or SUBENUM_* would otherwise
@@ -559,11 +579,10 @@ type outputTarget struct {
559579
}
560580

561581
// parseOutputs resolves the -o values; a file without a prefix takes def,
562-
// the -format of stdout. The same path given twice is refused: both would
563-
// replace one file.
582+
// the -format of stdout. The same file given twice, however it is spelled,
583+
// is refused: both would replace one file (#378).
564584
func parseOutputs(specs []string, def output.Format) ([]outputTarget, error) {
565585
targets := make([]outputTarget, 0, len(specs))
566-
seen := map[string]bool{}
567586
for _, spec := range specs {
568587
name, path := splitOutputSpec(spec)
569588
format := def
@@ -573,16 +592,39 @@ func parseOutputs(specs []string, def output.Format) ([]outputTarget, error) {
573592
return nil, err
574593
}
575594
}
576-
key := filepath.Clean(path)
577-
if seen[key] {
578-
return nil, fmt.Errorf("-o %s is given more than once", path)
595+
for _, t := range targets {
596+
if samePath(t.path, path) {
597+
return nil, fmt.Errorf("-o %s and -o %s name the same file; each -o needs its own", t.path, path)
598+
}
579599
}
580-
seen[key] = true
581600
targets = append(targets, outputTarget{path: path, format: format})
582601
}
583602
return targets, nil
584603
}
585604

605+
// checkOutputFiles refuses an -inconclusive, -stats or -sarif file that is
606+
// also an -o file or another of them: the last one written would replace the
607+
// others (#378).
608+
func checkOutputFiles(f cliFlags, outputs []outputTarget) error {
609+
type named struct{ flag, path string }
610+
var files []named
611+
for _, t := range outputs {
612+
files = append(files, named{"-o", t.path})
613+
}
614+
for _, n := range []named{{"-inconclusive", f.inconFile}, {"-stats", f.statsFile}, {"-sarif", f.sarifFile}} {
615+
if n.path == "" {
616+
continue
617+
}
618+
for _, prev := range files {
619+
if samePath(prev.path, n.path) {
620+
return fmt.Errorf("%s %s and %s %s name the same file; give each its own", prev.flag, prev.path, n.flag, n.path)
621+
}
622+
}
623+
files = append(files, n)
624+
}
625+
return nil
626+
}
627+
586628
// isTerminal reports whether w is a character device (a terminal) rather than
587629
// a pipe, file or in-memory buffer.
588630
func isTerminal(w io.Writer) bool {

‎internal/cli/cli_test.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ func TestValidateFlagsDomainListConflicts(t *testing.T) {
9898
t.Error("domain argument plus -dL should be rejected")
9999
}
100100
f.wordlists.paths, f.domainList = []string{"-"}, "-"
101-
if _, ok := validateFlags(f, &runtimeInputs{maxAttempts: 1}, nil, fs, out); ok {
101+
if validateArgs(f, nil, fs, out) {
102102
t.Error("-w - with -dL - should be rejected")
103103
}
104104
f = base()

‎internal/cli/mcp_report.go‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -58,17 +58,23 @@ type mcpOutcomes struct {
5858
WildcardFiltered int64 `json:"wildcard_filtered"`
5959
}
6060

61+
// notRunReport reports a scan that cfg's error kept from starting.
62+
func notRunReport(cfg subenum.Config, err error) *mcpReport {
63+
report := &mcpReport{Domain: cfg.Domain, Results: []mcpResult{}, MaxQueries: cfg.MaxQueries, Rate: cfg.Rate}
64+
report.Verdict, report.Reason, report.Error = verdictNotRun, "the scan did not start; see error", err.Error()
65+
return report
66+
}
67+
6168
// runMCPScan runs one scan and reports it. offset is where cfg.Words starts
6269
// in the call's word list, for the report's next_offset.
6370
func runMCPScan(ctx context.Context, cfg subenum.Config, offset int) *mcpReport {
64-
report := &mcpReport{Domain: cfg.Domain, Results: []mcpResult{}, MaxQueries: cfg.MaxQueries, Rate: cfg.Rate}
6571
cfg.Logf = func(string, ...any) {} // standard output carries the protocol
6672
// The report never lists inconclusive names, so the scan keeps none (#349).
6773
events, err := subenum.Run(scan.WithoutInconclusiveList(ctx), cfg)
6874
if err != nil {
69-
report.Verdict, report.Reason, report.Error = verdictNotRun, "the scan did not start; see error", err.Error()
70-
return report
75+
return notRunReport(cfg, err)
7176
}
77+
report := &mcpReport{Domain: cfg.Domain, Results: []mcpResult{}, MaxQueries: cfg.MaxQueries, Rate: cfg.Rate}
7278
var stats subenum.Stats
7379
for ev := range events {
7480
switch ev.Kind {

‎internal/cli/mcp_test.go‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import (
1212
"slices"
1313
"strings"
1414
"testing"
15+
"time"
1516

1617
"github.com/TMHSDigital/subenum/internal/dns"
1718
"github.com/TMHSDigital/subenum/internal/dnsserver"
@@ -804,6 +805,40 @@ func TestE2EMCPSessionBudget(t *testing.T) {
804805
}
805806
}
806807

808+
// TestMCPInvalidCallsSpendNoBudget covers #378: a scan call that cannot
809+
// start sends no query, so it does not use up the session's -max-calls.
810+
func TestMCPInvalidCallsSpendNoBudget(t *testing.T) {
811+
t.Parallel()
812+
zone, err := labzone.Load(filepath.Join("examples", "labs", "lab1-first-scan.zone"))
813+
if err != nil {
814+
t.Fatal(err)
815+
}
816+
srv, err := dnsserver.Listen(dnsserver.Options{}, zone.Handler([]string{"lab.example"}))
817+
if err != nil {
818+
t.Fatal(err)
819+
}
820+
defer srv.Close()
821+
tool := scanTool(mcpConfig{allow: []string{"lab.example"}, dnsServer: srv.Addr, maxQueries: 10, rate: 1000,
822+
maxDuration: time.Minute, timeout: 5 * time.Second, maxCalls: 1})
823+
for _, bad := range []string{
824+
`{"domain": "lab.example", "words": ["www"], "types": ["SRV"]}`,
825+
`{"domain": "lab.example", "words": []}`,
826+
`{"domain": "lab.example", "words": ["www"], "exclude": ["*."]}`,
827+
} {
828+
r := tool.Call(t.Context(), json.RawMessage(bad))
829+
if !r.IsError || !strings.Contains(r.Text, verdictNotRun) {
830+
t.Errorf("%s: want a not_run error, got %+v", bad, r)
831+
}
832+
}
833+
r := tool.Call(t.Context(), json.RawMessage(`{"domain": "lab.example", "words": ["www"]}`))
834+
if r.IsError {
835+
t.Errorf("valid call after invalid ones: %s", r.Text)
836+
}
837+
if r := tool.Call(t.Context(), json.RawMessage(`{"domain": "lab.example", "words": ["www"]}`)); !strings.Contains(r.Text, "allows 1 scan calls per session") {
838+
t.Errorf("second valid call: want the budget spent, got %s", r.Text)
839+
}
840+
}
841+
807842
func TestMCPFlagErrors(t *testing.T) {
808843
for _, tc := range []struct {
809844
args []string

‎internal/cli/mcp_tools.go‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -292,6 +292,12 @@ func scanTool(cfg mcpConfig) mcp.Tool {
292292
if err := args.apply(&scanCfg); err != nil {
293293
return mcp.Errorf("%v", err)
294294
}
295+
// A call that cannot start (bad types, exclude patterns, no
296+
// usable words) sends no query, so it spends none of the
297+
// session's scan calls (#378).
298+
if err := scanCfg.Validate(); err != nil {
299+
return notRunReport(scanCfg, err).result()
300+
}
295301
if calls.Add(1) > int64(cfg.maxCalls) {
296302
return mcp.Errorf("session budget spent: the operator allows %d scan calls per session (-max-calls)", cfg.maxCalls)
297303
}

‎internal/cli/session.go‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import (
1010
"os"
1111
"os/signal"
1212
"path/filepath"
13+
"runtime"
1314
"slices"
1415
"strings"
1516
"sync/atomic"
@@ -300,6 +301,9 @@ func (s *session) configure() (domain string, code int, ok bool) {
300301
return "", exitUsage, false
301302
}
302303
outputs, outputsErr := parseOutputs(f.outputs.specs, format)
304+
if outputsErr == nil {
305+
outputsErr = checkOutputFiles(*f, outputs)
306+
}
303307
if outputsErr != nil {
304308
out.Error("%v", outputsErr)
305309
return "", exitUsage, false
@@ -868,7 +872,15 @@ func samePath(a, b string) bool {
868872
}
869873
absA, errA := filepath.Abs(a)
870874
absB, errB := filepath.Abs(b)
871-
return errA == nil && errB == nil && absA == absB
875+
if errA != nil || errB != nil {
876+
return false
877+
}
878+
// Files that do not exist yet: Windows and macOS file systems ignore
879+
// case by default (#378).
880+
if runtime.GOOS == "windows" || runtime.GOOS == "darwin" {
881+
return strings.EqualFold(absA, absB)
882+
}
883+
return absA == absB
872884
}
873885

874886
// finish saves or removes the resume state, writes the run-quality report

0 commit comments

Comments
 (0)