Skip to content

Commit 2ff594c

Browse files
TMHSDigitalclaude
andcommitted
docs: small drift around NS hints, -auto-rate, labs and Docker (#232)
- start.md: the unreliable verdict stops the scan unless -no-abort, and -auto-rate slows down first - advanced_usage.md: NS, MX and TXT record types with dangling-ns, and -auto-rate as the alternative to aborting - monitoring.md, pipelines.md, action.yml: SARIF and takeover hints cover NS delegations too; the Action also verifies provenance - labs.md: -ct is refused with -simulate-zone; lab 1's zone header uses the lab wordlist like the lab page - docker.md: point the -o example at the nonroot --user workaround - flags.md: -attempts shows its real default, 1 (generator reads a "(default N)" usage suffix for 0-sentinel flags) - ROADMAP.md: Reply lives in internal/dnsserver - DOCUMENTATION_STRUCTURE.md: a map of this repo's docs instead of a generic template Fixes #232 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent fa68aac commit 2ff594c

12 files changed

Lines changed: 77 additions & 123 deletions

File tree

‎action.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ inputs:
3434
required: false
3535
default: ""
3636
sarif:
37-
description: Write takeover candidates to this SARIF file, for github/codeql-action/upload-sarif. Needs CNAME in -type.
37+
description: Write takeover candidates to this SARIF file, for github/codeql-action/upload-sarif. Needs CNAME or NS in -type.
3838
required: false
3939
default: ""
4040
max-duration:

‎docs/DOCUMENTATION_STRUCTURE.md‎

Lines changed: 42 additions & 113 deletions
Original file line numberDiff line numberDiff line change
@@ -1,113 +1,42 @@
1-
# Project Documentation Structure
2-
3-
This document outlines a recommended structure for comprehensive project documentation. While tailored for a Go-based CLI tool like `subenum`, the principles are broadly applicable.
4-
5-
## 1. Core Documentation (`README.md`)
6-
7-
This is the gateway to your project. It should be concise, informative, and guide users to other relevant documentation.
8-
9-
* **Project Title & Tagline:** Clear and descriptive.
10-
* **Badges (Optional but Recommended):**
11-
* Build status (e.g., GitHub Actions)
12-
* Code coverage
13-
* Go Report Card
14-
* License
15-
* Version/Release
16-
* **Brief Description:** What the project does in a few sentences.
17-
* **Cybersecurity Context / Educational Goals (If applicable):** Why this tool/project is important, especially from a learning or security perspective.
18-
* **Features:** Bulleted list of key capabilities.
19-
* **Prerequisites:** What users need to have installed/configured.
20-
* **Installation / Setup:**
21-
* From source (e.g., `go build`, `go install`)
22-
* Pre-compiled binaries (if provided)
23-
* Docker (if applicable)
24-
* **Usage:**
25-
* Basic command-line syntax.
26-
* Explanation of all flags and arguments.
27-
* Clear examples of common use cases.
28-
* **Output Explanation:** Describe what the output means.
29-
* **Contributing:** Guidelines for developers who want to contribute (link to `CONTRIBUTING.md`).
30-
* **License:** Link to `LICENSE.md`.
31-
* **Contact / Support (Optional):** How to get help or report issues.
32-
33-
## 2. Architecture Documentation (`ARCHITECTURE.md` or `/docs/architecture.md`)
34-
35-
This describes the "how" of your project. It's crucial for maintainers and contributors.
36-
37-
* **Overview:** High-level diagram and explanation of the system's components and their interactions.
38-
* **Key Components / Modules:**
39-
* For each major component (e.g., in `subenum`: argument parsing, wordlist processing, DNS resolution, concurrency management, output formatting):
40-
* Purpose and responsibilities.
41-
* Key functions/structs.
42-
* Interactions with other components.
43-
* **Data Flow:** How data moves through the system (e.g., from wordlist to DNS resolver to output).
44-
* **Concurrency Model (If applicable):**
45-
* Explanation of how concurrency is achieved (e.g., goroutines, channels, worker pools).
46-
* Rationale for design choices.
47-
* Potential bottlenecks or considerations.
48-
* **Error Handling Strategy:** How errors are propagated, logged, and handled.
49-
* **External Dependencies:** List of external libraries and why they were chosen.
50-
* **Design Decisions & Rationale:** Important architectural choices made and the reasons behind them. This can include alternatives considered and why they were not chosen.
51-
* **Future Considerations / Potential Improvements:** Areas for future development or refactoring.
52-
53-
## 3. Developer Guide (`DEVELOPER_GUIDE.md` or `/docs/developer_guide.md`)
54-
55-
Information for those who want to build, test, or contribute to the project.
56-
57-
* **Getting Started:**
58-
* Cloning the repository.
59-
* Setting up the development environment (Go version, tools, environment variables).
60-
* Building the project from source.
61-
* **Running Tests:**
62-
* How to execute unit tests, integration tests, etc.
63-
* Test coverage tools.
64-
* **Coding Style / Conventions:** Link to any style guides or linters used.
65-
* **Debugging Tips:** Common issues and how to troubleshoot them.
66-
* **Making Changes:**
67-
* Branching strategy.
68-
* Commit message conventions.
69-
* Pull Request process.
70-
* **Dependencies Management:** How to add or update dependencies (e.g., `go get`, `go mod tidy`).
71-
72-
## 4. Contribution Guidelines (`CONTRIBUTING.md`)
73-
74-
How others can contribute to the project.
75-
76-
* **Code of Conduct:** Link to `CODE_OF_CONDUCT.md`.
77-
* **How to Contribute:**
78-
* Reporting bugs (issue tracker, templates).
79-
* Suggesting enhancements.
80-
* Submitting pull requests.
81-
* **What to Work On:** (Optional) Links to "good first issues" or a project board.
82-
83-
## 5. Code of Conduct (`CODE_OF_CONDUCT.md`)
84-
85-
Sets expectations for community interaction. Adopt a standard one like the Contributor Covenant.
86-
87-
## 6. License (`LICENSE` or `LICENSE.md`)
88-
89-
Specifies how the software can be used, modified, and distributed (e.g., MIT, Apache 2.0, GPL).
90-
91-
## 7. Changelog (`CHANGELOG.md`)
92-
93-
A chronological list of notable changes for each version/release.
94-
95-
* Follows Keep a Changelog format (Added, Changed, Deprecated, Removed, Fixed, Security).
96-
97-
## 8. Examples (`/examples` directory)
98-
99-
* Provide more extensive or varied examples of how to use the tool.
100-
* Sample wordlists, configuration files, or scripts.
101-
102-
## 9. API Documentation (If applicable - for libraries or tools with an API)
103-
104-
* For Go projects, this is often generated using `godoc`.
105-
* Ensure public functions and types are well-commented.
106-
107-
## 10. `/docs` Directory (Optional - for larger projects)
108-
109-
For more complex projects, you might move `ARCHITECTURE.md`, `DEVELOPER_GUIDE.md`, and other detailed documents into a dedicated `/docs` directory to keep the root directory cleaner.
110-
111-
---
112-
113-
This structure provides a solid foundation. Adapt it to the specific needs and complexity of your project.
1+
# Documentation map
2+
3+
Where each piece of subenum's documentation lives and who it is for. The files under `docs/` are also the [project site](https://tmhsdigital.github.io/subenum/); this page is left out of the site build.
4+
5+
## For users
6+
7+
| Document | What it covers |
8+
|---|---|
9+
| [`README.md`](../README.md) | What subenum is, install, quick start, every flag, output formats, the TUI, completions and the man page |
10+
| [`docs/start.md`](start.md) | First scan, the run-quality verdict, what to read next |
11+
| [`docs/cli.md`](cli.md) | CLI reference: flags (generated from the code into [`_includes/flags.md`](_includes/flags.md)), exit codes, config file and `SUBENUM_*` defaults |
12+
| [`docs/pipelines.md`](pipelines.md) | Chaining with dnsx, httpx and nuclei; JSON logs; takeover candidates first |
13+
| [`docs/monitoring.md`](monitoring.md) | Scheduled scans with `-diff`, the GitHub Action ([`action.yml`](../action.yml)) and SARIF alerts |
14+
| [`docs/docker.md`](docker.md) | The container image: volumes, output files, permissions |
15+
| [`docs/labs.md`](labs.md) | Offline exercises against [`examples/labs/`](../examples/labs/) zones with `-simulate-zone` |
16+
| [`docs/library.md`](library.md) | The Go library `pkg/subenum`; the full API reference is on pkg.go.dev |
17+
| [`examples/advanced_usage.md`](../examples/advanced_usage.md) | Longer recipes: wildcards, record types, rate limiting, the reliability guard, CSV |
18+
| [`tools/README.md`](../tools/README.md) | The `wordlist-gen` tool |
19+
20+
## For contributors
21+
22+
| Document | What it covers |
23+
|---|---|
24+
| [`docs/CONTRIBUTING.md`](CONTRIBUTING.md) | Setup, tests, lint, pull requests |
25+
| [`docs/DEVELOPER_GUIDE.md`](DEVELOPER_GUIDE.md) | Project tree, hermetic testing, previewing the site, extension points |
26+
| [`docs/ARCHITECTURE.md`](ARCHITECTURE.md) | How the engine works: packages, resolution, dispatch, output, error handling |
27+
| [`docs/ROADMAP.md`](ROADMAP.md) | Where the project is heading and known constraints |
28+
| [`docs/CODE_OF_CONDUCT.md`](CODE_OF_CONDUCT.md) | Community expectations |
29+
30+
## Project files
31+
32+
| File | What it covers |
33+
|---|---|
34+
| [`CHANGELOG.md`](../CHANGELOG.md) | Changes per release (Keep a Changelog); each release's notes come from its section |
35+
| [`SECURITY.md`](../SECURITY.md) | Supported versions and how to report a vulnerability |
36+
| [`LICENSE`](../LICENSE) | GPL-3.0 |
37+
| [`CITATION.cff`](../CITATION.cff) | How to cite subenum |
38+
39+
## Keeping docs current
40+
41+
- Regenerate the site's flag table with `make docs-flags`; `go test .` fails when it or the README flag table drifts from the CLI.
42+
- List anything on `main` but not yet released in [`_data/unreleased.yml`](_data/unreleased.yml) so the site tags it, and empty the lists in the release commit.

‎docs/ROADMAP.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,8 @@ Constraints to keep in mind when changing the code.
5959
- **The test DNS server (`internal/dnstest`) models A, AAAA, CNAME, NS, MX
6060
and TXT answers,** any RCODE, NODATA with SOA, delays, drops and
6161
truncation. Other record types, referrals and EDNS are not modelled; extend
62-
its `Reply` before testing them.
62+
`Reply` in `internal/dnsserver` (which `internal/dnstest` re-exports) before
63+
testing them.
6364
- **The dangling-NS hint needs the NS records.** A recursive resolver asked
6465
about a subzone whose every name server is gone usually answers SERVFAIL
6566
rather than the delegation, so such a name ends up inconclusive (listed by

‎docs/_includes/flags.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
| Flag | Default | Description |
44
| :--- | :--- | :--- |
5-
| `-attempts <int>` | - | Total DNS resolution attempts per subdomain, 1 = no retry (default 1) |
5+
| `-attempts <int>` | 1 | Total DNS resolution attempts per subdomain, 1 = no retry (default 1) |
66
| `-auto-rate` {% include unreleased.html flag="auto-rate" %} | - | When over 20% of the last 200 lookups fail, halve the query rate and keep going (up to 6 times, down to 5 queries/s) instead of aborting; speed back up once lookups succeed |
77
| `-ct` | - | Add names from Certificate Transparency logs (crt.sh, one HTTPS request per target, no API key) as candidates and -permute seeds |
88
| `-dL <string>` | - | File of apex domains to scan, one per line (- for stdin); replaces the &lt;domain&gt; argument |

‎docs/docker.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,8 @@ docker run --rm -v $(pwd)/data:/data subenum \
9797
example.com
9898
```
9999

100+
The image runs as the unprivileged `nonroot` user (uid 65532). If it cannot write to the mounted directory, add `--user $(id -u):$(id -g)` (see [File Permission Problems](#file-permission-problems)).
101+
100102
### Using Multiple Attempts for Unreliable Networks
101103

102104
The `-attempts` flag sets the total DNS resolution attempts per subdomain:

‎docs/labs.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -198,6 +198,6 @@ lab points at a real host.
198198
Scenarios work with every other flag, such as `-dL` (each listed domain gets
199199
the same zone), `-permute`, `-diff`, `-format json` and `-stats`, which
200200
records the scenario file under `zone`. `-simulate-zone` cannot be combined
201-
with `-simulate`, `-dns-server` or `-r`, whether they come from the command
201+
with `-simulate`, `-dns-server`, `-r` or `-ct`, whether they come from the command
202202
line, a `SUBENUM_*` variable or the config file: a lab run refuses to start
203203
rather than send a query off the machine.

‎docs/monitoring.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ subenum -w wordlist.txt -format jsonl -type A,AAAA,CNAME -diff previous.jsonl -d
6868
## The subenum GitHub Action
6969

7070
The repository is also a GitHub Action. It installs a release (checked against
71-
its `checksums.txt`), diffs against the previous complete run's results kept as
71+
its `checksums.txt` and its build provenance attestation), diffs against the previous complete run's results kept as
7272
an artifact, writes a summary table to the job, and fails the job when the
7373
verdict is not good enough:
7474

@@ -230,8 +230,9 @@ commit `previous.jsonl` to a branch or store it in object storage instead.
230230

231231
## Takeover candidates in code scanning
232232

233-
Add `-type A,AAAA,CNAME -sarif takeovers.sarif` to the scan and upload the file
234-
after it, and every dangling or takeover-prone CNAME becomes an alert in the
233+
Add `-type A,AAAA,CNAME,NS -sarif takeovers.sarif` to the scan and upload the file
234+
after it, and every dangling or takeover-prone CNAME, and every delegation to a
235+
name server that does not exist (`dangling-ns`), becomes an alert in the
235236
repository's Security tab, where it can be triaged and dismissed. Alerts are
236237
fingerprinted per name, so a candidate seen every week stays one alert. The
237238
job needs `security-events: write`:

‎docs/pipelines.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,8 @@ jq -r 'select(.type != "summary") | .subdomain' results.jsonl | httpx -silent
6060
## Takeover candidates first
6161

6262
With CNAME lookups, subenum marks names whose alias target does not resolve or
63-
points at a takeover-prone service. Send those to a closer look before the
63+
points at a takeover-prone service; with NS lookups, names delegated to a name
64+
server that does not exist (`dangling-ns`). Send those to a closer look before the
6465
rest:
6566

6667
```bash

‎docs/start.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ The report ends in a verdict:
6666
|---|---|
6767
| `complete` | Under 1% of lookups failed. A name that isn't listed was asked about and doesn't exist. |
6868
| `degraded` | 1% or more failed, the run was interrupted, or `-max-queries` or `-max-duration`{% include unreleased.html flag="max-duration" %} stopped it early. The result is usable, but some names are unknown. |
69-
| `unreliable` | More than 20% failed. subenum stops the scan rather than return a quietly incomplete list. |
69+
| `unreliable` | More than 20% failed. subenum stops the scan rather than return a quietly incomplete list, unless `-no-abort` is set; `-auto-rate` slows down first and stops only when it cannot slow down further. |
7070

7171
The `reason` field says why, for example `12 of 5000 lookups failed (timeout 9, refused 3, other 0), under 1%`. Gate a pipeline on it:
7272

‎examples/advanced_usage.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,6 +187,12 @@ Find only subdomains that are CNAMEs (useful for spotting potential takeovers):
187187
./subenum -w wordlist.txt -type CNAME -format json example.com
188188
```
189189

190+
`NS`, `MX` and `TXT` work the same way. An `MX` value reads `<preference> <host>`. With `NS`, a delegated subzone whose name server does not exist is marked `dangling-ns`, a takeover that CNAME-only scans miss:
191+
192+
```bash
193+
./subenum -w wordlist.txt -type A,CNAME,NS -show-records example.com
194+
```
195+
190196
The record type is captured per result, so JSON and CSV output carry it in the `records` field / `type` column.
191197

192198
## Rate Limiting
@@ -214,6 +220,15 @@ Pass `-no-abort` to keep the warning but finish the wordlist:
214220
./subenum -w wordlist.txt -no-abort example.com
215221
```
216222

223+
Or pass `-auto-rate` to slow down instead: when more than 20% of the last 200
224+
lookups failed, the query rate is halved (with a `rate` notice) and raised
225+
again once lookups succeed. The guard aborts only after six back-offs, or at 5
226+
queries per second:
227+
228+
```bash
229+
./subenum -w wordlist.txt -rate 200 -auto-rate example.com
230+
```
231+
217232
## Query Cap and Recursion Ceiling
218233

219234
`-max-queries` stops admitting new work once the cap is reached (0 = unlimited).

0 commit comments

Comments
 (0)