Skip to content

Commit 22c4211

Browse files
TMHSDigitalclaude
andcommitted
fix(wordlist-gen): no silent overwrite, no empty output, validate prefixes
Refuse to replace an existing output file without -f, exit 1 when no entries are generated, reject -combine prefixes that are not valid DNS labels, and only emit entries that pass subenum's own label rules. main is now a testable run(args, stdout, stderr). Fixes #59 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent aedda94 commit 22c4211

6 files changed

Lines changed: 117 additions & 20 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2121
- `-rate` now charges every DNS query that reaches the wire. A CNAME lookup costs three slots (Go sends A, AAAA and CNAME queries), and queries Go's resolver retries internally on SERVFAIL, REFUSED or socket errors are charged as they are dialed, so a struggling resolver no longer gets more traffic than `-rate` allows. Responses with RCODE REFUSED are now counted as `refused` instead of `other`, and lookup errors name the configured `-dns-server` instead of the system nameserver (#50).
2222
- Wildcards that answer from a rotating CDN or load-balancer pool are now filtered under `-force`. The root fingerprint starts from five probes instead of two and, when their answers differ, keeps probing until the pool stops growing; results that share only some records with the fingerprint are re-validated with fresh probes under the same parent. In a hermetic 8-address pool test, filtering went from 12 of 50 wildcard answers to all 50, with no real names lost (#48).
2323
- A wordlist line longer than 64 KB no longer aborts the run with `bufio.Scanner: token too long`. Overlong lines are truncated, skipped and counted with the other invalid entries (#56).
24+
- `tools/wordlist-gen` no longer overwrites an existing output file without the new `-f` flag, exits 1 instead of writing an empty file when nothing is generated, rejects `-combine` prefixes that are not valid DNS labels, and drops generated entries subenum would skip (for example combinations over 63 characters). `make wordlist` and `examples/demo.sh` pass `-f` for the files they own (#59).
2425
- Release binaries are built with `CGO_ENABLED=0 -trimpath -s -w`, so `subenum-linux-amd64` is statically linked again and CI fails if a Linux asset is not (#46).
2526

2627
## [0.8.0] - 2026-10-04

‎Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ wordlist-gen:
6565

6666
# Generate a custom wordlist
6767
wordlist: wordlist-gen
68-
tools/$(WORDLIST_GEN) -domain $(WL_DOMAIN) -combine $(WL_COMBINE) -o $(WL_OUTPUT)
68+
tools/$(WORDLIST_GEN) -f -domain $(WL_DOMAIN) -combine $(WL_COMBINE) -o $(WL_OUTPUT)
6969
@echo "Generated wordlist: $(WL_OUTPUT)"
7070
@echo "Use it with: make WORDLIST=$(WL_OUTPUT) DOMAIN=$(WL_DOMAIN) run-verbose"
7171

‎examples/demo.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ echo ""
8181

8282
# Generate a custom wordlist
8383
echo -e "${YELLOW}[3/7] Generating domain-specific wordlist...${NC}"
84-
tools/wordlist-gen -domain "$TARGET_DOMAIN" -combine "dev,api,staging,test,v1,v2" -o demo-wordlist.txt
84+
tools/wordlist-gen -f -domain "$TARGET_DOMAIN" -combine "dev,api,staging,test,v1,v2" -o demo-wordlist.txt
8585
echo -e "${GREEN}✓ Custom wordlist generated${NC}"
8686
echo ""
8787

‎tools/README.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ A simple utility for generating custom wordlists for subdomain enumeration.
1414
- Ensures no duplicate entries (case-insensitive)
1515
- Deterministic: the same flags always produce a byte-identical file
1616
- Strips the public suffix, including second-level ones such as `.co.uk`
17+
- Writes only valid DNS labels, and exits non-zero when nothing would be written
1718

1819
### Usage
1920

@@ -35,10 +36,11 @@ go build -buildvcs=false -o wordlist-gen wordlist-gen.go
3536

3637
| Flag | Description | Default |
3738
|------|-------------|---------|
38-
| `-o` | Output file path | `wordlist.txt` |
39+
| `-o` | Output file path; an existing file is never overwritten without `-f` | `wordlist.txt` |
40+
| `-f` | Overwrite the output file if it exists | `false` |
3941
| `-common` | Include common subdomain prefixes | `true` |
4042
| `-domain` | Domain to extract terms from | `""` (empty) |
41-
| `-combine` | Comma-separated prefixes to combine with other terms | `""` (empty) |
43+
| `-combine` | Comma-separated prefixes to combine with other terms; each must be a valid DNS label | `""` (empty) |
4244
| `-v` | Print every generated entry to stdout | `false` |
4345

4446
### Example

‎tools/wordlist-gen.go‎

Lines changed: 68 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,14 @@ package main
77

88
import (
99
"bufio"
10+
"errors"
1011
"flag"
1112
"fmt"
1213
"io"
1314
"os"
1415
"strings"
16+
17+
"github.com/TMHSDigital/subenum/internal/wordlist"
1518
)
1619

1720
// commonPrefixes are frequently seen subdomain labels, emitted first when
@@ -72,6 +75,11 @@ func generate(o options) []string {
7275
if w == "" || seen[w] {
7376
return
7477
}
78+
// Only valid labels: a too-long combination or an odd domain term
79+
// would be skipped by subenum anyway.
80+
if _, ok := wordlist.Normalize(w); !ok {
81+
return
82+
}
7583
seen[w] = true
7684
out = append(out, w)
7785
}
@@ -115,40 +123,84 @@ func writeWordlist(w io.Writer, words []string) error {
115123
return bw.Flush()
116124
}
117125

126+
// validateCombine rejects -combine prefixes that are not valid DNS labels,
127+
// which subenum would otherwise skip silently when it loads the list (#59).
128+
func validateCombine(combine string) error {
129+
for _, prefix := range strings.Split(combine, ",") {
130+
p := strings.TrimSpace(prefix)
131+
if p == "" {
132+
continue
133+
}
134+
if _, ok := wordlist.Normalize(p); !ok || strings.Contains(p, ".") {
135+
return fmt.Errorf("invalid -combine prefix %q: use letters, digits, hyphens or underscores", p)
136+
}
137+
}
138+
return nil
139+
}
140+
118141
func main() {
119-
outputFile := flag.String("o", "wordlist.txt", "Path to output wordlist file")
120-
combineWith := flag.String("combine", "", "Combine each word with these prefixes (comma-separated)")
121-
addCommon := flag.Bool("common", true, "Add common subdomain prefixes")
122-
domainInfo := flag.String("domain", "", "Domain to extract potential subdomains from (e.g., company-name.com -> company, name)")
123-
verbose := flag.Bool("v", false, "Print every generated entry")
124-
flag.Parse()
142+
os.Exit(run(os.Args[1:], os.Stdout, os.Stderr))
143+
}
144+
145+
func run(args []string, stdout, stderr io.Writer) int {
146+
fs := flag.NewFlagSet("wordlist-gen", flag.ContinueOnError)
147+
fs.SetOutput(stderr)
148+
outputFile := fs.String("o", "wordlist.txt", "Path to output wordlist file")
149+
force := fs.Bool("f", false, "Overwrite the output file if it already exists")
150+
combineWith := fs.String("combine", "", "Combine each word with these prefixes (comma-separated)")
151+
addCommon := fs.Bool("common", true, "Add common subdomain prefixes")
152+
domainInfo := fs.String("domain", "", "Domain to extract potential subdomains from (e.g., company-name.com -> company, name)")
153+
verbose := fs.Bool("v", false, "Print every generated entry")
154+
if err := fs.Parse(args); err != nil {
155+
if errors.Is(err, flag.ErrHelp) {
156+
return 0
157+
}
158+
return 2
159+
}
125160

126161
if *outputFile == "" {
127-
fmt.Fprintln(os.Stderr, "Error: output file cannot be empty")
128-
os.Exit(1)
162+
fmt.Fprintln(stderr, "Error: output file cannot be empty")
163+
return 1
164+
}
165+
if err := validateCombine(*combineWith); err != nil {
166+
fmt.Fprintf(stderr, "Error: %v\n", err)
167+
return 1
129168
}
130169

131170
words := generate(options{common: *addCommon, domain: *domainInfo, combine: *combineWith})
171+
if len(words) == 0 {
172+
fmt.Fprintln(stderr, "Error: no entries generated; enable -common or pass -domain or -combine")
173+
return 1
174+
}
132175

133-
file, err := os.Create(*outputFile)
176+
flags := os.O_WRONLY | os.O_CREATE | os.O_TRUNC
177+
if !*force {
178+
flags |= os.O_EXCL // never silently replace an existing wordlist (#59)
179+
}
180+
file, err := os.OpenFile(*outputFile, flags, 0o644)
181+
if errors.Is(err, os.ErrExist) {
182+
fmt.Fprintf(stderr, "Error: %s already exists; pass -f to overwrite it or -o to choose another path\n", *outputFile)
183+
return 1
184+
}
134185
if err != nil {
135-
fmt.Fprintf(os.Stderr, "Error creating output file: %v\n", err)
136-
os.Exit(1)
186+
fmt.Fprintf(stderr, "Error creating output file: %v\n", err)
187+
return 1
137188
}
138189
writeErr := writeWordlist(file, words)
139190
closeErr := file.Close()
140191
if writeErr != nil || closeErr != nil {
141-
fmt.Fprintf(os.Stderr, "Error writing %s: %v\n", *outputFile, firstErr(writeErr, closeErr))
142-
os.Exit(1)
192+
fmt.Fprintf(stderr, "Error writing %s: %v\n", *outputFile, firstErr(writeErr, closeErr))
193+
return 1
143194
}
144195

145196
if *verbose {
146197
for _, w := range words {
147-
fmt.Println(w)
198+
fmt.Fprintln(stdout, w)
148199
}
149200
}
150-
fmt.Fprintf(os.Stderr, "Wordlist generated at %s with %d unique entries\n", *outputFile, len(words))
151-
fmt.Fprintln(os.Stderr, "NOTE: Only use this tool to generate wordlists for domains you have explicit permission to test.")
201+
fmt.Fprintf(stderr, "Wordlist generated at %s with %d unique entries\n", *outputFile, len(words))
202+
fmt.Fprintln(stderr, "NOTE: Only use this tool to generate wordlists for domains you have explicit permission to test.")
203+
return 0
152204
}
153205

154206
func firstErr(errs ...error) error {

‎tools/wordlist-gen_test.go‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@ package main
22

33
import (
44
"errors"
5+
"io"
6+
"os"
7+
"path/filepath"
58
"reflect"
69
"strings"
710
"testing"
@@ -69,3 +72,42 @@ func TestWriteWordlistReportsErrors(t *testing.T) {
6972
t.Fatalf("got %q, %v", sb.String(), err)
7073
}
7174
}
75+
76+
// TestRunGuards covers #59: an existing file is not overwritten without -f,
77+
// an empty result exits non-zero, and invalid -combine prefixes are rejected.
78+
func TestRunGuards(t *testing.T) {
79+
dir := t.TempDir()
80+
out := filepath.Join(dir, "wl.txt")
81+
if err := os.WriteFile(out, []byte("keep\n"), 0o644); err != nil {
82+
t.Fatal(err)
83+
}
84+
var stderr strings.Builder
85+
86+
if code := run([]string{"-o", out}, io.Discard, &stderr); code != 1 || !strings.Contains(stderr.String(), "already exists") {
87+
t.Errorf("existing file: exit %d, stderr %q; want 1 and an already-exists error", code, stderr.String())
88+
}
89+
if got, _ := os.ReadFile(out); string(got) != "keep\n" {
90+
t.Errorf("existing file overwritten without -f: %q", got)
91+
}
92+
if code := run([]string{"-f", "-o", out}, io.Discard, io.Discard); code != 0 {
93+
t.Errorf("-f: exit %d, want 0", code)
94+
}
95+
if got, _ := os.ReadFile(out); !strings.Contains(string(got), "www\n") {
96+
t.Errorf("-f did not rewrite the file: %q", got)
97+
}
98+
99+
empty := filepath.Join(dir, "empty.txt")
100+
if code := run([]string{"-common=false", "-o", empty}, io.Discard, io.Discard); code != 1 {
101+
t.Errorf("empty result: exit %d, want 1", code)
102+
}
103+
if _, err := os.Stat(empty); !os.IsNotExist(err) {
104+
t.Errorf("empty result still created %s", empty)
105+
}
106+
107+
for _, bad := range []string{"shop!", "a b", "x.y"} {
108+
stderr.Reset()
109+
if code := run([]string{"-combine", "dev," + bad, "-o", filepath.Join(dir, "c.txt")}, io.Discard, &stderr); code != 1 || !strings.Contains(stderr.String(), "invalid -combine prefix") {
110+
t.Errorf("-combine %q: exit %d, stderr %q", bad, code, stderr.String())
111+
}
112+
}
113+
}

0 commit comments

Comments
 (0)