Skip to content

Commit 70a023f

Browse files
TMHSDigitalclaude
andcommitted
Accept the built-in lab scenarios in -simulate-zone (#354)
Every binary embeds the lab scenarios for subenum mcp, but -simulate-zone only opened a file, so release, container and go install users had to clone the repository to run the labs. A value with no path separator that is not an existing file now names a built-in scenario (lab1, lab1-first-scan or lab1-first-scan.zone), and without -w such a run uses the labs' embedded wordlist; -w "" still picks the bundled list. Paths keep loading from disk, an unknown name lists the built-in scenarios, and -resume skips the compiled-in scenario when fingerprinting input files. The Labs page now runs every exercise as -simulate-zone labN. Fixes #354 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent e50737a commit 70a023f

11 files changed

Lines changed: 191 additions & 32 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010
### Added
11+
- `-simulate-zone` accepts the lab scenarios built into every binary by name, so release, container and `go install` users run the labs without cloning the repository: `subenum -simulate-zone lab1 lab.example` works from any directory (`lab1`, `lab1-first-scan` and `lab1-first-scan.zone` all name it), and without `-w` the scan uses the labs' own wordlist (`-w ""` keeps the bundled 5,000-word list). A path, or a file that exists under the given name, is still read from disk, and an unknown name lists the built-in scenarios (#354).
1112
- `pkg/subenum`: `Result.Sources` names what proposed each result, as the CLI's `sources` field does: `wordlist`, and each `Config.Sources` entry that returned it, named by a `Name()` method when it has one (`ct`, `wayback`, `axfr` for the built-in sources) or as `Sources[i]` (#273).
1213
- `-o` can be given more than once, and a format prefix gives a file its own format: `-o names.txt -o jsonl:results.jsonl` writes bare names for the next tool and JSONL with the run-quality report in one run, where a pipeline used to scan twice or extract names with `jq`. A file without a prefix uses `-format`, as before. Every file gets the full current results, also under `-diff`; on a run stopped with state for `-resume`, only an `-o` file that is also the `-diff` file is left unchanged. The same path given twice is a usage error, and `-tui` takes the first file (#334).
1314
- The TUI's finished screen can save the results of a scan run without an Output file: `s` asks for a path (starting from the Output file, or `subenum-<domain>` with the Format's extension), and the file is written as the Output file would have been, in the format its extension names (`.txt`, `.json`, `.jsonl`, `.csv`, otherwise the form's Format). With a filter on, only the results it shows are saved. `v` toggles each result's records under it, during or after the scan, and a text file saved while they are shown carries them, as with `-show-records`. Results past the view's 10,000 are not kept and cannot be saved (#332).

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -287,7 +287,7 @@ This table describes `main`, which can be ahead of the latest release: a release
287287
| `-simulate` | `false` | Simulation mode: no real DNS queries |
288288
| `-hit-rate <n>` | `15` | Simulated resolution rate, percent (1-100), applied uniformly to every name |
289289
| `-seed <n>` | `0` | Simulation seed; the same seed reproduces the same results (line order can vary unless `-t 1`). `0` picks a random seed and prints it |
290-
| `-simulate-zone <file>` | n/a | Lab mode: answer every query from a scenario file via a local DNS server on `127.0.0.1`; no traffic leaves the machine. See [Labs](docs/labs.md) |
290+
| `-simulate-zone <file>` | n/a | Lab mode: answer every query from a scenario file, or a built-in scenario such as `lab1` (`lab1`-`lab6`, with their own wordlist when `-w` is not given), via a local DNS server on `127.0.0.1`; no traffic leaves the machine. See [Labs](docs/labs.md) |
291291
| **Modes** | | |
292292
| `-tui` | `false` | Launch the interactive Terminal UI |
293293
| `-dry-run` | `false` | Print the plan without scanning: candidates after dedupe and `-exclude`, query and duration estimates, and the resolver preflight (its only DNS query), then exit; JSON with `-format json` or `jsonl` |
@@ -303,7 +303,7 @@ This table describes `main`, which can be ahead of the latest release: a release
303303
> Simulation mode (`-simulate`) generates synthetic results and performs zero network I/O. Do not confuse simulated output with real DNS data. Simulated output marks itself: JSON and JSONL results carry `"simulated": true`, CSV gets a `simulated` column, and `-o` text files start with a `# SIMULATED` comment line. Bare names piped to stdout stay unmarked so they still pipe cleanly.
304304
305305
> [!TIP]
306-
> For teaching, `-simulate-zone lab.zone` goes further than `-simulate`: a scenario file defines exact names, records, wildcards, failures and resolver rate limits, and a DNS server on `127.0.0.1` answers from it. The real resolver, wildcard filtering, `-rate` and the run-quality report all run unchanged, deterministically and with no traffic leaving the machine. [docs/labs.md](docs/labs.md) has six guided exercises with answers, built on the scenarios in `examples/labs/`.
306+
> For teaching, `-simulate-zone lab.zone` goes further than `-simulate`: a scenario file defines exact names, records, wildcards, failures and resolver rate limits, and a DNS server on `127.0.0.1` answers from it. The real resolver, wildcard filtering, `-rate` and the run-quality report all run unchanged, deterministically and with no traffic leaving the machine. [docs/labs.md](docs/labs.md) has six guided exercises with answers, built on the scenarios in `examples/labs/`, which every binary carries: `subenum -simulate-zone lab1 lab.example` runs the first one from any directory.
307307
308308
### Defaults, completions and the man page
309309

‎docs/ARCHITECTURE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -346,7 +346,7 @@ With `-ct`, `ct.Client.Names` makes one HTTPS request per target to crt.sh (no A
346346

347347
### 5.6. Lab Mode
348348

349-
`-simulate-zone <file>` loads a scenario with `labzone.Load` (exact names, records, `*.name` wildcards, and `TIMEOUT`, `SERVFAIL`, `REFUSED` or `NXDOMAIN` entries, plus `$DELAY` and `$REFUSE-ABOVE` directives), serves it with `dnsserver.Listen` on a loopback port and points `-dns-server` at it (`startLabZone`). Unlike `-simulate`, the scan then runs the real resolver, wildcard detection, `-rate` pacing and reliability guard, and no traffic leaves the machine. The same `internal/dnsserver` backs the hermetic tests through `internal/dnstest`.
349+
`-simulate-zone <file>` loads a scenario with `labzone.Load`, or, for a name such as `lab1` that is not a file, the scenario compiled in from `examples/labs` with its wordlist as the `-w` default (`builtinLab`) (exact names, records, `*.name` wildcards, and `TIMEOUT`, `SERVFAIL`, `REFUSED` or `NXDOMAIN` entries, plus `$DELAY` and `$REFUSE-ABOVE` directives), serves it with `dnsserver.Listen` on a loopback port and points `-dns-server` at it (`startLabZone`). Unlike `-simulate`, the scan then runs the real resolver, wildcard detection, `-rate` pacing and reliability guard, and no traffic leaves the machine. The same `internal/dnsserver` backs the hermetic tests through `internal/dnstest`.
350350

351351
### 5.7. SARIF Output
352352

‎docs/_includes/flags.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@
3939
| `-show-records` | - | In text format, append each result's records (TYPE=value) |
4040
| `-silent` | - | Pipeline mode: bare result names on stdout, and only errors (and a simulation warning) on stderr |
4141
| `-simulate` | - | Run in simulation mode without actual DNS queries (for testing) |
42-
| `-simulate-zone <string>` | - | Lab mode: answer every query from this scenario file via a local DNS server, so no traffic leaves the machine (see the Labs page) |
42+
| `-simulate-zone <string>` | - | Lab mode: answer every query from this scenario file, or a built-in scenario such as lab1 (lab1-lab6), via a local DNS server, so no traffic leaves the machine (see the Labs page) |
4343
| `-sources <string>` {% include unreleased.html flag="sources" %} | - | Comma-separated keyless passive sources whose names become candidates and -permute seeds: ct (Certificate Transparency, crt.sh), wayback (Wayback Machine); one HTTPS request per source and target |
4444
| `-state <string>` | subenum-resume.json | Where a run that was interrupted, ran out of -max-duration or was aborted saves its state for -resume |
4545
| `-stats <string>` | - | Write a JSON run-quality report (outcomes, queries sent, verdict) to this file |

‎docs/labs.md‎

Lines changed: 20 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -18,19 +18,18 @@ against a live zone, and the results are the same on every run.
1818

1919
## Setup
2020

21-
You need a subenum binary (see [Installation](https://github.com/TMHSDigital/subenum#installation))
22-
and a copy of the repository for the scenario files. For a zero-install
23-
classroom, [open the labs in GitHub Codespaces](https://codespaces.new/TMHSDigital/subenum?devcontainer_path=.devcontainer/labs/devcontainer.json&quickstart=1):
21+
You need a subenum binary (see [Installation](https://github.com/TMHSDigital/subenum#installation)).
22+
The scenarios and their wordlist are built into it, so the labs run from any
23+
directory: `-simulate-zone lab1` (or `lab1-first-scan`) picks the built-in
24+
scenario, and without `-w` the scan uses the labs' short wordlist. For a
25+
zero-install classroom, [open the labs in GitHub Codespaces](https://codespaces.new/TMHSDigital/subenum?devcontainer_path=.devcontainer/labs/devcontainer.json&quickstart=1):
2426
the container builds `./subenum`, opens this page and the first scenario, and
2527
prints the lab 1 command.
2628

27-
Every lab scans `lab.example` with the short wordlist `examples/labs/words.txt`.
28-
Run the commands from the repository root:
29-
30-
```bash
31-
L=examples/labs
32-
W="-w $L/words.txt"
33-
```
29+
Every lab scans `lab.example`. To edit a scenario or write your own, copy one
30+
from [`examples/labs`](https://github.com/TMHSDigital/subenum/tree/main/examples/labs)
31+
and pass its path instead, for example `-simulate-zone ./my-lab.zone`; a path
32+
uses the bundled 5,000-word list unless you add `-w`.
3433

3534
Labs run from the command line. Combining `-tui` with an active `-simulate-zone`
3635
setting (including `SUBENUM_SIMULATE_ZONE` or a config-file value) exits with
@@ -41,7 +40,7 @@ Each lab ends with questions. The answers are folded underneath; try first.
4140
## Lab 1: a first scan
4241

4342
```bash
44-
subenum $W -simulate-zone $L/lab1-first-scan.zone -show-records lab.example
43+
subenum -simulate-zone lab1 -show-records lab.example
4544
```
4645

4746
`-show-records` prints the records behind each name.
@@ -73,7 +72,7 @@ subenum $W -simulate-zone $L/lab1-first-scan.zone -show-records lab.example
7372
## Lab 2: spot the wildcard
7473

7574
```bash
76-
subenum $W -simulate-zone $L/lab2-wildcard.zone lab.example
75+
subenum -simulate-zone lab2 lab.example
7776
```
7877

7978
1. What does subenum report, and why does it refuse to continue?
@@ -100,8 +99,8 @@ This resolver answers REFUSED once it gets more than 40 queries in a second,
10099
as many public and corporate resolvers do.
101100

102101
```bash
103-
subenum $W -simulate-zone $L/lab3-rate-limit.zone -stats lab3.json lab.example
104-
subenum $W -simulate-zone $L/lab3-rate-limit.zone -stats lab3-paced.json -rate 30 lab.example
102+
subenum -simulate-zone lab3 -stats lab3.json lab.example
103+
subenum -simulate-zone lab3 -stats lab3-paced.json -rate 30 lab.example
105104
```
106105

107106
1. Compare the `refused` counts of the two runs.
@@ -126,14 +125,14 @@ subenum $W -simulate-zone $L/lab3-rate-limit.zone -stats lab3-paced.json -rate 3
126125
## Lab 4: recursive discovery
127126

128127
```bash
129-
subenum $W -simulate-zone $L/lab4-recursive.zone lab.example
130-
subenum $W -simulate-zone $L/lab4-recursive.zone -recursive -depth 3 lab.example
128+
subenum -simulate-zone lab4 lab.example
129+
subenum -simulate-zone lab4 -recursive -depth 3 lab.example
131130
```
132131

133132
1. How many names does the flat scan find? How many does the recursive scan
134133
find?
135134
2. Which name needed `-depth 3`?
136-
3. Try the recursive scan with the bundled 5,000-word list (drop `$W`). What
135+
3. Try the recursive scan with the bundled 5,000-word list (add `-w ""`). What
137136
happens, and why?
138137

139138
<details markdown="1">
@@ -153,7 +152,7 @@ subenum $W -simulate-zone $L/lab4-recursive.zone -recursive -depth 3 lab.example
153152
## Lab 5: when lookups fail
154153

155154
```bash
156-
subenum $W -simulate-zone $L/lab5-failures.zone -timeout 300 -stats lab5.json lab.example
155+
subenum -simulate-zone lab5 -timeout 300 -stats lab5.json lab.example
157156
```
158157

159158
1. Which outcome lines in the breakdown are non-zero besides `resolved` and
@@ -175,9 +174,9 @@ subenum $W -simulate-zone $L/lab5-failures.zone -timeout 300 -stats lab5.json la
175174
## Lab 6: IPv6-only services
176175

177176
```bash
178-
subenum $W -simulate-zone $L/lab6-ipv6.zone -show-records lab.example
179-
subenum $W -simulate-zone $L/lab6-ipv6.zone -type A -show-records lab.example
180-
subenum $W -simulate-zone $L/lab6-ipv6.zone -type AAAA -show-records lab.example
177+
subenum -simulate-zone lab6 -show-records lab.example
178+
subenum -simulate-zone lab6 -type A -show-records lab.example
179+
subenum -simulate-zone lab6 -type AAAA -show-records lab.example
181180
```
182181

183182
1. Which three names does the default scan find, and which address families

‎internal/cli/cli.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -231,7 +231,7 @@ func parseFlagsWith(args []string, applyDefaults func(*flag.FlagSet) error, stde
231231
fs.BoolVar(&f.testMode, "simulate", false, "Run in simulation mode without actual DNS queries (for testing)")
232232
fs.IntVar(&f.testHitRate, "hit-rate", scan.DefaultHitRate, "In simulation mode, percentage of names that resolve (1-100)")
233233
fs.Uint64Var(&f.seed, "seed", 0, "In simulation mode, seed for reproducible results (0 = random; the seed used is printed)")
234-
fs.StringVar(&f.simZone, "simulate-zone", "", "Lab mode: answer every query from this scenario file via a local DNS server, so no traffic leaves the machine (see the Labs page)")
234+
fs.StringVar(&f.simZone, "simulate-zone", "", "Lab mode: answer every query from this scenario file, or a built-in scenario such as lab1 (lab1-lab6), via a local DNS server, so no traffic leaves the machine (see the Labs page)")
235235
fs.Var(&f.outputs, "o", "Write results to `file` (in addition to stdout); repeat for several files, each in -format or in the format named as a prefix, format:path (jsonl:results.jsonl)")
236236
fs.IntVar(&f.attempts, "attempts", 0, "Total DNS resolution attempts per subdomain, 1 = no retry (default 1)")
237237
fs.IntVar(&f.retries, "retries", 0, "Deprecated: use -attempts instead")
@@ -607,7 +607,7 @@ func logVerboseStart(f cliFlags, domain string, maxAttempts int, out *output.Wri
607607
}
608608
out.Info("Target domain: %s", domain)
609609
if len(f.wordlists.paths) == 0 {
610-
out.Info("Wordlist: %s", bundledWordlistName)
610+
out.Info("Wordlist: %s", defaultWordlistName(f))
611611
} else {
612612
out.Info("Wordlist: %s", f.wordlists.describe())
613613
}

‎internal/cli/lab.go‎

Lines changed: 54 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,69 @@
11
package cli
22

33
import (
4+
"bytes"
5+
"errors"
6+
"fmt"
7+
"io/fs"
8+
"os"
9+
"path"
10+
"strings"
11+
412
"github.com/TMHSDigital/subenum/internal/dnsserver"
513
"github.com/TMHSDigital/subenum/internal/labzone"
614
"github.com/TMHSDigital/subenum/internal/output"
715
)
816

17+
// builtinLab resolves a -simulate-zone value to a scenario compiled into the
18+
// binary (#354), so a release, container or go install user can run the labs
19+
// without the repository: "lab1", "lab1-first-scan" and
20+
// "lab1-first-scan.zone" all name examples/labs/lab1-first-scan.zone. A value
21+
// with a path separator, or one naming a file that exists, stays a path, so
22+
// custom scenarios keep working.
23+
func builtinLab(value string) (name string, ok bool) {
24+
if value == "" || strings.ContainsAny(value, `/\`) {
25+
return "", false
26+
}
27+
if _, err := os.Stat(value); err == nil {
28+
return "", false
29+
}
30+
value = strings.TrimSuffix(value, ".zone")
31+
for _, name := range labScenarios() {
32+
if name == value || strings.HasPrefix(name, value+"-") {
33+
return name, true
34+
}
35+
}
36+
return "", false
37+
}
38+
39+
// loadLabZone reads the -simulate-zone scenario, from the binary when the
40+
// value names a built-in one and from the file system otherwise. desc names
41+
// it for the LAB MODE notice.
42+
func loadLabZone(value string) (zone *labzone.Zone, desc string, err error) {
43+
if name, ok := builtinLab(value); ok {
44+
b, err := labFiles.ReadFile(path.Join(labDir, name+".zone"))
45+
if err != nil {
46+
return nil, "", err
47+
}
48+
zone, err := labzone.Parse(bytes.NewReader(b))
49+
if err != nil {
50+
return nil, "", fmt.Errorf("built-in scenario %s: %w", name, err)
51+
}
52+
return zone, "the built-in scenario " + name, nil
53+
}
54+
zone, err = labzone.Load(value)
55+
if errors.Is(err, fs.ErrNotExist) && !strings.ContainsAny(value, `/\`) {
56+
err = fmt.Errorf("%w; the built-in scenarios are %s", err, strings.Join(labScenarios(), ", "))
57+
}
58+
return zone, value, err
59+
}
60+
961
// startLabZone serves the -simulate-zone scenario from a loopback DNS server
1062
// and points -dns-server at it (#76). Unlike -simulate, the scan then runs
1163
// the real resolver, wildcard detection, -rate pacing and reliability guard.
1264
// It reports its own errors; stop shuts the server down.
1365
func startLabZone(f *cliFlags, targets []string, out *output.Writer) (stop func(), ok bool) {
14-
zone, err := labzone.Load(f.simZone)
66+
zone, desc, err := loadLabZone(f.simZone)
1567
if err != nil {
1668
out.Error("-simulate-zone: %v", err)
1769
return nil, false
@@ -23,7 +75,7 @@ func startLabZone(f *cliFlags, targets []string, out *output.Writer) (stop func(
2375
}
2476
f.dnsServer = srv.Addr
2577
out.Info("")
26-
out.Info("LAB MODE: answers come from %s, served on %s.", f.simZone, srv.Addr)
78+
out.Info("LAB MODE: answers come from %s, served on %s.", desc, srv.Addr)
2779
out.Info("No DNS traffic leaves this machine.")
2880
out.Info("")
2981
return srv.Close, true

‎internal/cli/lab_test.go‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,72 @@ func TestE2ELabs(t *testing.T) {
117117
}
118118
}
119119

120+
// TestE2EBuiltinLabs runs the scenarios compiled into the binary by name, as
121+
// a release binary does away from a checkout (#354): without -w the lab
122+
// wordlist is the default, and an unknown name lists the built-in ones.
123+
func TestE2EBuiltinLabs(t *testing.T) {
124+
t.Parallel()
125+
for _, name := range []string{"lab1", "lab1-first-scan", "lab1-first-scan.zone"} {
126+
t.Run(name, func(t *testing.T) {
127+
t.Parallel()
128+
args := append(append([]string{"-progress=false"}, labTimingArgs...), "-simulate-zone", name, "lab.example")
129+
code, out := runCLIMerged(t, args...)
130+
if code != exitOK {
131+
t.Fatalf("exit %d\n%s", code, out)
132+
}
133+
for _, want := range []string{"built-in scenario lab1-first-scan", "No -w given: using the " + labWordlistName} {
134+
if !strings.Contains(out, want) {
135+
t.Errorf("output lacks %q:\n%s", want, out)
136+
}
137+
}
138+
var got []string
139+
for _, line := range strings.Split(strings.TrimSpace(out), "\n") {
140+
line = strings.TrimSpace(line)
141+
if !strings.ContainsAny(line, " \t") && strings.HasSuffix(line, ".lab.example") {
142+
got = append(got, strings.TrimSuffix(line, ".lab.example"))
143+
}
144+
}
145+
slices.Sort(got)
146+
if want := []string{"jenkins", "mail", "shop", "vpn", "www"}; !slices.Equal(got, want) {
147+
t.Errorf("found %v, want %v", got, want)
148+
}
149+
})
150+
}
151+
t.Run("-w empty keeps the bundled list", func(t *testing.T) {
152+
t.Parallel()
153+
code, out := runCLIMerged(t, "-dry-run", "-w", "", "-simulate-zone", "lab4", "lab.example")
154+
if code != exitOK || !strings.Contains(out, "No -w given: using the "+bundledWordlistName) {
155+
t.Fatalf("exit %d, want the bundled list\n%s", code, out)
156+
}
157+
})
158+
t.Run("unknown", func(t *testing.T) {
159+
t.Parallel()
160+
code, out := runCLIMerged(t, "-simulate-zone", "lab9", "lab.example")
161+
if code != exitFailure || !strings.Contains(out, "lab1-first-scan, lab2-wildcard") {
162+
t.Fatalf("exit %d, want %d and the built-in names\n%s", code, exitFailure, out)
163+
}
164+
})
165+
}
166+
167+
// TestBuiltinLabKeepsPaths checks that a value with a path separator, or a
168+
// file that exists, is still read from disk (#354).
169+
func TestBuiltinLabKeepsPaths(t *testing.T) {
170+
t.Parallel()
171+
for value, want := range map[string]string{
172+
"lab2": "lab2-wildcard",
173+
"lab6-ipv6": "lab6-ipv6",
174+
"lab": "",
175+
"lab1-first": "lab1-first-scan",
176+
"./lab1": "",
177+
filepath.Join("x", "lab1"): "",
178+
filepath.Join("examples", "labs", "lab1-first-scan.zone"): "",
179+
} {
180+
if got, _ := builtinLab(value); got != want {
181+
t.Errorf("builtinLab(%q) = %q, want %q", value, got, want)
182+
}
183+
}
184+
}
185+
120186
// TestE2ELabVerdicts checks the run-quality verdicts of the failure labs.
121187
func TestE2ELabVerdicts(t *testing.T) {
122188
t.Parallel()

‎internal/cli/resume.go‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -382,7 +382,11 @@ func fileDigest(path string) (string, error) {
382382
// state leaves it as it was (#297).
383383
func inputDigests(f cliFlags) (map[string]string, error) {
384384
out := map[string]string{}
385-
paths := append(slices.Clone(f.wordlists.paths), f.domainList, f.excludeFile, f.resolvers, f.simZone, f.seedsFile, f.diff)
385+
simZone := f.simZone
386+
if _, ok := builtinLab(simZone); ok {
387+
simZone = "" // compiled in, not a file; it does not move a saved position (#354)
388+
}
389+
paths := append(slices.Clone(f.wordlists.paths), f.domainList, f.excludeFile, f.resolvers, simZone, f.seedsFile, f.diff)
386390
for _, p := range paths {
387391
d, err := fileDigest(p)
388392
if err != nil {

0 commit comments

Comments
 (0)