Repository navigation
Replace massdns with blastdns 2.0 - #3457
Conversation
Replaces the massdns subprocess in the brute helper with a dedicated blastdns client. It gets its own client rather than sharing the one behind normal resolution, because brute-forcing runs against a large public resolver list instead of the system resolvers and needs its own limits. Uses resolve_batch_full rather than resolve_batch so unanswered queries stay visible and get reported: a query that runs out of retries is a silently missed subdomain, which is the one failure brute-forcing must not hide. Retires dns.brute_threads, which was a massdns passthrough, in favor of brute_concurrency, brute_inflight_per_resolver, and brute_rate_limit. Enables the startup resolver probe so dead entries in the public list are dropped once instead of costing queries all scan. Drops the now-unused module argument from helpers.dns.brute(). Tests move off the run_live massdns monkeypatch: mock_dns now drives brute-forcing directly, since _mock_dns swaps the brute client too. Assertions are unchanged, and the wildcard canary case uses a regex mock key.
Deletes DEP_MASSDNS and the deps_common entries that pulled it in, so BBOT no longer builds massdns from source at install time. Documents the new brute-force knobs in place of dns.brute_threads, and swaps the deps_ansible example in the module howto to masscan, which BBOT still builds the same way.
uv.lock still pins 1.9.1 and cannot be regenerated until 2.0.0 is published, so dependency resolution will fail until then. Regenerate the lock once the release is out.
Supplied nameservers replace the OS-detected ones for normal resolution. DNSHelper.system_resolvers keeps meaning "what the OS reported" and the new DNSHelper.resolvers is what actually gets queried. resolver_file follows the effective set, so external tools pointed at it (nuclei) honor the setting too. The brute-force pool likewise excludes the effective set rather than the system one, preserving the reason that exclusion exists: brute traffic must not rate-limit our own resolution. Brute-forcing is unaffected and keeps its own dns.brute_nameservers list. Validation is blastdns's, so an unusable address is reported once, from one place, rather than reimplementing resolver parsing in Python.
Brute-forcing inherited dns.timeout (5s), which is right for careful resolution of in-scope hosts but wrong here: blastdns abandons an attempt when the timeout expires rather than overlapping retransmissions, so retries x timeout serializes. At 20 retries that meant up to 100s on a single stubborn name, and measured out at 2.4x slower than a 1-retry run. massdns re-asks every 500ms for up to 50 tries, a 25s budget per name. A 1s timeout with 20 retries gives a comparable 20s budget without the serialization cost. Not shorter than 1s, since blastdns discards late answers that massdns would still accept.
A brute run ends when its slowest query ends, and stragglers burned timeout x retries serially: the last 1% of queries took a quarter of each run's wall clock, paid 43 times per scan. Dropping the timeout to 0.5s cuts per-run time 32% (3.26s -> 2.23s over 5 interleaved reps) with the resolved set unchanged, because retries recover anything abandoned early. Going lower shrinks the usable resolver pool by half. Also exposes blastdns's persistent_socket, which trades most of that throughput for ~4x less connection-tracking state, for networks with a small NAT table.
Per-query sockets bind a fresh source port each time, so connection-tracking state grows with query volume: a single-domain scan peaked at 114,342 conntrack entries, 44% of a stock Linux table and several times over what a consumer router holds. One socket per resolver bounds that by the resolver count instead -- same scan peaked at 6,974, and found the identical 154 subdomains in 158s against 168s.
The subdomain-enum preset still set it, so every bundled-preset validation failed and took test_python_api and test_modules_basic_checks with it. dnsbrute also no longer has common deps now that massdns is gone, so the preload assertion moves to portscan, which still uses them.
# Conflicts: # pyproject.toml # uv.lock
Performance Benchmark Report
|
baddns pins blastdns<1.10, so the module-level pip install cannot resolve against the 2.x that #3457 puts in the venv. Revert once baddns widens it.
# Conflicts: # pyproject.toml # uv.lock
We're in a holding pattern here anyway until #3474 merges, which now targets this branch. At that time we will sync up all of the dependencies. |
singlerider
left a comment
singlerider
left a comment
There was a problem hiding this comment.
🟢 Clean massdns removal, conflicts from prior review resolved, brute client mocked so tests stay offline.
singlerider
left a comment
singlerider
left a comment
There was a problem hiding this comment.
Reviewed at 67424f1.
🔴 Custom nameservers with a port leak into the brute pool
--custom-nameservers documents 8.8.8.8:53, but the public brute list is bare IPs, so difference_update(self.parent_helper.dns.resolvers) drops nothing and brute-forcing hammers the scan's own resolver. With dns.nameservers: ["8.8.8.8:53"] and a brute list of 8.8.8.8, 1.0.161.227, resolver_file() returns both. Normalize both sides to host before subtracting.
🟢 LaneQueue
Letting already-resolved children jump ahead of fresh hosts in dnsresolve is a clean fix for the parent/child ordering, and test_dnsresolve_incoming_lanes pins it.
🟢 Delivery logging
_log_delivery separating "no record" from "unanswered" makes silent brute-force loss visible for the first time.
Remove deepdiff dependency
Replaces massdns with blastdns 2.0 for DNS brute-forcing, and drops the massdns dependency entirely.
Changes
--custom-nameserversflag, replacing the system resolvers for normal resolutiondns.nameserversvalue surfaced as a per-event error insidednsresolveafter setup had already reported success, so a typo produced a green "completed" scan with zero results instead of an abort.baddns 2.5.0
2.5.0 is the release that widened baddns's blastdns pin, so the temporary
[tool.uv] override-dependenciesblock is gone. BBOT now requiresbaddns~=2.5.0and capsblastdnsat<2.1to match baddns's own ceiling, so a future blastdns 2.1 cannot silently break module dependency installs the way<1.10did.Adapting to what 2.5.0 changed:
NSgained a lame-delegation finding, raising its ceiling to HIGH/CONFIRMED inSUBMODULE_MAX_SEVERITYandSUBMODULE_MAX_CONFIDENCEDELEGATIONsubmodule is added to both tables at HIGH/CONFIRMEDerror code: 1001)amazonaws.comsubject. Direct mode uses the target as the subject, so that cannot be satisfied against a localhost test server; thebaddns_directfixture now uses a signature with no CNAME identifier, which is the class a Cloudflare-fronted host actually hits. RealSTORAGE_BUCKETdetection is unaffected, since those hosts genuinely containamazonaws.com.Behavior changes
DELEGATIONis enabled by default, both in the module's default submodule list and in thebaddnsandbaddns-heavypresets. It checks_acme-challenge,_dmarcand 15 common DKIM selector labels for dangling delegations and reports at HIGH severity. Two filters keep the volume down: a wildcard CNAME probe per depth drops labels that only resolve because of a wildcard, and generic findings (a CNAME to a name that merely does not exist, which is routine for DKIM selector pairs) are discarded, leaving only claimable outcomes.baddns's
WordMatcheris now passed through. It compiles every signature's word matchers into one YARA ruleset at setup and matches a response against all of them in a single pass, instead of evaluating each signature's words in turn per response. This is what baddns 2.5.0's newyara-pythondependency is for, and baddns's own CLI already does it. It is a cache with the previous path as fallback, and was checked for equivalence against that path across 2,044 signature/response pairs with no divergence.New
DELEGATIONmodule tests cover the three label kinds, wildcard suppression, and the generic-finding filter.docs/scanning/advanced.mdis deliberately not regenerated here. It embedsbbot --help, which CI generates on a newer Python whose argparse formats help text differently, so the daily docs job will pick up--custom-nameservers.