fix(sourcehunt): stop a hunter that makes no new progress for N steps - #202
Open
tsunoda-kazuya wants to merge 1 commit into
Open
fix(sourcehunt): stop a hunter that makes no new progress for N steps#202tsunoda-kazuya wants to merge 1 commit into
tsunoda-kazuya wants to merge 1 commit into
Conversation
Track steps since the last new finding, new potential, or first read of a previously-unread file (constrained read_source_file or deep read_file). When nothing advances for max_steps_without_progress steps (default 8, 0 disables) the hunter stops with stop_reason=stalled. Steps that only reissued an already-throttled call are left to the degenerate_loop terminal, so identical-call loops keep reporting degenerate_loop. Deep hunters read source via read_file/execute rather than the constrained read_source_file that populates ctx.files_read, so the progress signature also tracks ctx.deep_files_read — the set of paths successfully read via the deep read_file tool. Only the first read of a new path counts; a stream of varied failing execute commands or repeated write_file calls does not reset the counter. Without this, deep mode (the production default) either collapses to (findings, potentials) and falsely kills early exploration, or resets on any tool call and lets a wandering agent run to max_steps. The threshold is configurable via SourceHuntConfig.tuning .max_steps_without_progress and the sourcehunt CLI's --max-steps-without-progress flag. Complements degenerate_loop (repeated identical calls) and empty_response (empty turns), which do not catch varied-but-unproductive reading; that case otherwise runs to max_steps.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
stalledstop reason that fires aftermax_steps_without_progressconsecutive steps with no new finding, no new potential, and no first read of a previously unread file.degenerate_loop(repeated identical calls) andempty_response(empty turns) terminals, which do not cover varied-but-unproductive reading.HuntTuning.max_steps_without_progressconfig field (default 8,0disables) and matching--max-steps-without-progressCLI flag.read_filenow populatesctx.deep_files_read: set[str]on first read of an unread file, so the progress signature is meaningful in deep mode (wherectx.files_read— populated only by the constrainedread_source_file— stays empty).executeandwrite_filedo not count as progress.Problem
A deep hunter can keep issuing distinct, schema-valid reads across the same file (varying offsets) or a stream of varied failing commands without ever recording a finding or flagging a potential. None of the current terminals fire: the calls are not identical (so no
degenerate_loop), the turns are not empty (so noempty_response), and the budget is unlimited whenbudget_usd=0, so the loop runs tomax_steps(2000 in deep mode). With output-token caps no longer sent (#182), each such step is also longer than before, so a stuck hunter is now more expensive per iteration.Fix
steps_since_progressinNativeHunter.arun; the per-step signature is(len(findings), len(potentials), len(ctx.files_read), len(ctx.deep_files_read)). Reset on any change, increment when unchanged.read_fileadds the resolved path toctx.deep_files_readon success;executeandwrite_filedo not touch the set.HunterRunResult(stop_reason="stalled")past the threshold, keeping accumulated findings and potentials.prev_step_had_skipgates the counter so steps skipped bydegenerate_loopthrottling do not compete with the new terminal.Validation
uv run --frozen --extra dev pytest -q tests/test_deep_agent_loop.py tests/test_sourcehunt_hunter.py— 109 passed; new regressions: hunter stalls when re-reading the same path 10 times, when running varied failing execs, when spamming write_file; distinct-path reading does not stall;max_steps_without_progress=0disables the guard; degenerate_loop still wins for pure identical-call loops.uv run --frozen --extra dev ruff check clearwing/agent/tools/hunt/sandbox.py clearwing/agent/tools/hunt/deep_agent.py clearwing/sourcehunt/hunter.py clearwing/sourcehunt/config.py clearwing/sourcehunt/runner.py clearwing/sourcehunt/pool.py clearwing/sourcehunt/subsystem.py clearwing/ui/commands/sourcehunt.py tests/test_deep_agent_loop.pygit diff --checkBackward compatibility
max_steps_without_progress = 8is the shipped default;0restores the previous behaviour (no stalled terminal). The new stop reason is additive; existing consumers ofstop_reasonsee the new string only when the guard fires.Independent of the other 4 PRs in this batch; trivial rebase if any lands first.