Skip to content

Commit f6bedf2

Browse files
TMHSDigitalclaude
andcommitted
fix(lib): Run no longer leaks the worker pool on break-and-cancel
The forwarder in pkg/subenum.Run did an unguarded send, so the common `for ev := range events { break }` plus deferred cancel left the engine's workers blocked forever (about 100 goroutines per run). After cancellation the forwarder now keeps draining the engine without waiting for a reader: other events are dropped when the buffer is full and the final KindDone waits at most a second. A caller that keeps reading still gets KindDone. Also: DefaultWords parses the bundled list once (sync.OnceValue + clone), and a positive sub-millisecond Timeout rounds up to 1ms instead of being rejected as 0. Fixes #112 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 58f6f51 commit f6bedf2

5 files changed

Lines changed: 125 additions & 8 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
3030
- The minimum Go version is now 1.26 (`go 1.26.0` in go.mod), required by current `golang.org/x/sys` and `golang.org/x/text`; both were bumped and `govulncheck -show verbose` now reports no vulnerable modules. Dependabot also updates indirect modules, and CI lints with errorlint, nilerr, contextcheck, noctx, copyloopvar, usestdlibvars and revive, with errcheck exemptions narrowed to stdout prints (#83).
3131

3232
### Fixed
33+
- `pkg/subenum.Run` no longer leaks the scan's worker pool when a caller breaks out of the event loop and cancels without draining (about 100 goroutines per run before): after cancellation the forwarder never waits for a reader, so every goroutine exits within about a second. `DefaultWords` parses the bundled list once, and a positive `Timeout` below 1ms is rounded up instead of being rejected as 0 (#112).
3334
- A resolver pool member that answers NXDOMAIN for names that exist can no longer silently hide them. Each member is checked with canaries (names the trusted resolver confirmed, starting with the apex), and an NXDOMAIN from a member that has not passed a canary is confirmed by a second member. A member contradicted three times (a hidden name, a failed canary, or a hit the trusted resolver denies) is flagged and dropped for the rest of the run; the per-resolver report gains `canaries`, `contradicted` and `flagged`. In a hermetic test, one always-NXDOMAIN member previously hid half the real names; now none are lost. In a zone with no known name to use as a canary, NXDOMAIN answers cost a second pool query until one is found (#105).
3435
- `-diff` no longer reports names this run never tested as removed. Every previous name the run did not find is looked up again with the trusted resolver, and only NXDOMAIN counts as removed; names that still resolve (a smaller wordlist, no `-recursive` or `-permute`, now wildcard-filtered) or whose lookup fails are counted as `still_resolving` and `unverified` in the report's `diff` object instead (#110).
3536
- Resuming a `-permute` scan no longer skips the permutation pass: the main pass's resume position was applied to the permutation candidates, so a resumed pass queried nothing. The pass now saves and resumes from its own position, and is seeded only with the main pass's names so the resumed candidate list is identical (#107).

‎docs/library.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ for ev := range events {
5757
}
5858
```
5959

60-
Drain the channel, or cancel `ctx` and then drain it. A consumer that stops reading blocks the scan once the buffer fills. Cancelling still delivers `KindDone` with the counts so far.
60+
Read the channel until it closes, or cancel `ctx`. Breaking out of the loop with a deferred `cancel()` is safe: once `ctx` is cancelled the scan stops without waiting for a reader, and its goroutines exit within about a second whether or not the channel is read. A consumer that keeps reading after cancelling still gets `KindDone` with the counts so far. While `ctx` is live, a consumer that stops reading pauses the scan once the buffer fills.
6161

6262
## Configuration
6363

‎pkg/subenum/export_test.go‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
package subenum
2+
3+
import "time"
4+
5+
// SetDoneGrace shortens how long Run waits to deliver the final event after
6+
// cancellation, so leak tests need not wait; it returns a restore function.
7+
func SetDoneGrace(d time.Duration) func() {
8+
old := doneGrace
9+
doneGrace = d
10+
return func() { doneGrace = old }
11+
}

‎pkg/subenum/subenum.go‎

Lines changed: 56 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,9 @@ import (
1919
"errors"
2020
"fmt"
2121
"regexp"
22+
"slices"
2223
"strings"
24+
"sync"
2325
"time"
2426

2527
"github.com/TMHSDigital/subenum/data"
@@ -300,14 +302,23 @@ type Event struct {
300302
// Config.Words defaults to it. The returned slice is a fresh copy, safe to
301303
// append to or filter.
302304
func DefaultWords() []string {
305+
return slices.Clone(defaultWords())
306+
}
307+
308+
// defaultWords parses the bundled list once; callers must not modify it.
309+
var defaultWords = sync.OnceValue(func() []string {
303310
lines, err := wordlist.ReadLinesFrom(strings.NewReader(data.Subdomains5k))
304311
if err != nil {
305312
// The list is compiled into the binary, so this cannot fail.
306313
panic("subenum: reading the bundled wordlist: " + err.Error())
307314
}
308315
words, _, _ := wordlist.Build(lines, "")
309316
return words
310-
}
317+
})
318+
319+
// doneGrace is how long Run's forwarder waits to hand over the final event
320+
// after ctx is cancelled, when the caller may have stopped reading.
321+
var doneGrace = time.Second
311322

312323
// ErrNoWords is returned when a Config's wordlist has no usable entries.
313324
var ErrNoWords = errors.New("subenum: wordlist has no usable entries")
@@ -317,9 +328,14 @@ var ErrNoWords = errors.New("subenum: wordlist has no usable entries")
317328
// final Stats. Problems with cfg itself are reported before the scan starts,
318329
// as an error and no channel.
319330
//
320-
// The caller must drain the channel, or cancel ctx and then drain it, or the
321-
// scan blocks once the buffer fills. Cancelling ctx stops the scan and still
322-
// delivers KindDone with the counts so far.
331+
// Read the channel until it closes, or cancel ctx: the usual
332+
// `for ev := range events { ... break ... }` with a deferred cancel is safe.
333+
// While ctx is live, an unread channel pauses the scan once its buffer
334+
// fills. Once ctx is cancelled the scan stops and never waits for the
335+
// caller: results nobody reads may be dropped, and a caller that keeps
336+
// reading still gets KindDone with the counts so far. Every goroutine the
337+
// scan started exits within about a second of cancellation, whether or not
338+
// the channel is read (#112).
323339
func Run(ctx context.Context, cfg Config) (<-chan Event, error) {
324340
internal, err := cfg.options()
325341
if err != nil {
@@ -335,18 +351,45 @@ func Run(ctx context.Context, cfg Config) (<-chan Event, error) {
335351
for ev := range in {
336352
translated := translate(ev)
337353
done = done || translated.Kind == KindDone
338-
out <- translated
354+
forward(ctx, out, translated)
339355
}
340356
// The engine returns without a final event when a scan never gets
341357
// to start, such as a wildcard zone without Force. Report the same
342358
// terminal event anyway, so consumers need only one shape.
343359
if !done {
344-
out <- Event{Kind: KindDone}
360+
forward(ctx, out, Event{Kind: KindDone})
345361
}
346362
}()
347363
return out, nil
348364
}
349365

366+
// forward hands ev to the caller. Before cancellation it waits for the
367+
// caller to read. After it, it never blocks the engine, which must be
368+
// drained to finish: other events are dropped when the buffer is full, and
369+
// the final event waits at most doneGrace for a reader.
370+
func forward(ctx context.Context, out chan<- Event, ev Event) {
371+
if ctx.Err() == nil {
372+
select {
373+
case out <- ev:
374+
return
375+
case <-ctx.Done():
376+
}
377+
}
378+
if ev.Kind != KindDone {
379+
select {
380+
case out <- ev:
381+
default:
382+
}
383+
return
384+
}
385+
timer := time.NewTimer(doneGrace)
386+
defer timer.Stop()
387+
select {
388+
case out <- ev:
389+
case <-timer.C:
390+
}
391+
}
392+
350393
// Scan runs a scan to completion and returns everything it found, in the
351394
// order the names resolved.
352395
//
@@ -437,7 +480,7 @@ func (c Config) options() (scan.Options, error) {
437480
Domain: domain,
438481
Entries: entries,
439482
Concurrency: orDefault(c.Concurrency, DefaultConcurrency),
440-
TimeoutMs: int(orDefaultDuration(c.Timeout, DefaultTimeout).Milliseconds()),
483+
TimeoutMs: ceilMillis(orDefaultDuration(c.Timeout, DefaultTimeout)),
441484
DNSServer: resolver,
442485
Simulate: c.Simulate,
443486
HitRate: orDefault(c.HitRate, DefaultHitRate),
@@ -487,6 +530,12 @@ func parseTypes(types []string) ([]string, error) {
487530
return parsed, nil
488531
}
489532

533+
// ceilMillis rounds a duration up to whole milliseconds, the engine's unit,
534+
// so a positive Timeout below 1ms becomes 1ms instead of an invalid 0.
535+
func ceilMillis(d time.Duration) int {
536+
return int((d + time.Millisecond - 1) / time.Millisecond)
537+
}
538+
490539
func orDefault(v, def int) int {
491540
if v == 0 {
492541
return def

‎pkg/subenum/subenum_test.go‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ package subenum_test
33
import (
44
"context"
55
"errors"
6+
"runtime"
67
"slices"
78
"strings"
89
"testing"
@@ -390,3 +391,58 @@ func TestRateLimitIsApplied(t *testing.T) {
390391
t.Errorf("40 queries at 20/s took %s, want at least a second", elapsed)
391392
}
392393
}
394+
395+
// TestBreakAndCancelLeaksNothing covers #112: the idiomatic
396+
// `for ev := range events { break }` with a deferred cancel leaves no
397+
// goroutine of the scan behind, even though the channel is never drained.
398+
func TestBreakAndCancelLeaksNothing(t *testing.T) {
399+
defer subenum.SetDoneGrace(10 * time.Millisecond)()
400+
base := runtime.NumGoroutine()
401+
for i := 0; i < 5; i++ {
402+
func() {
403+
ctx, cancel := context.WithCancel(context.Background())
404+
defer cancel()
405+
events, err := subenum.Run(ctx, subenum.Config{Domain: "example.com", Simulate: true, HitRate: 100, Seed: 1})
406+
if err != nil {
407+
t.Fatal(err)
408+
}
409+
for range events {
410+
break
411+
}
412+
}()
413+
}
414+
deadline := time.Now().Add(5 * time.Second)
415+
for runtime.NumGoroutine() > base+2 && time.Now().Before(deadline) {
416+
time.Sleep(20 * time.Millisecond)
417+
}
418+
if n := runtime.NumGoroutine(); n > base+2 {
419+
t.Errorf("%d goroutines remain after 5 abandoned runs, started with %d", n, base)
420+
}
421+
}
422+
423+
// TestCancelThenDrainStillEndsWithDone: a caller that cancels and keeps
424+
// reading still gets the final event.
425+
func TestCancelThenDrainStillEndsWithDone(t *testing.T) {
426+
ctx, cancel := context.WithCancel(context.Background())
427+
events, err := subenum.Run(ctx, subenum.Config{Domain: "example.com", Simulate: true, HitRate: 100, Seed: 1})
428+
if err != nil {
429+
t.Fatal(err)
430+
}
431+
<-events
432+
cancel()
433+
last := subenum.Event{Kind: subenum.KindResult}
434+
for ev := range events {
435+
last = ev
436+
}
437+
if last.Kind != subenum.KindDone {
438+
t.Errorf("last event %v, want KindDone", last.Kind)
439+
}
440+
}
441+
442+
// TestSubMillisecondTimeoutRoundsUp: a positive Timeout below the engine's
443+
// 1ms resolution is rounded up, not rejected as zero.
444+
func TestSubMillisecondTimeoutRoundsUp(t *testing.T) {
445+
if _, _, err := subenum.Scan(context.Background(), subenum.Config{Domain: "example.com", Words: []string{"www"}, Simulate: true, Timeout: 500 * time.Microsecond}); err != nil {
446+
t.Errorf("Timeout 500µs: %v", err)
447+
}
448+
}

0 commit comments

Comments
 (0)