Repository navigation
feat(decisions): read test outcomes, script edits and subagents (0.23.0) - #58
Merged
Merged
Conversation
The "What the agent decided" section almost never fired on real sessions: agents pipe test runs, edit through scripts, work across several worktrees, and delegate to subagents. - A run's outcome comes from the runner's own summary in its output (vitest/jest, pytest, cargo, go), read and never shown; one run per command; probes against swapped-in code are skipped; variables the command assigns resolve to the files they name. - Files are named only when the run targets nothing else, and a later pass of the same runner outranks an earlier failure. - The repo root also comes from Bash calls run in a known directory (the shell's directory followed across calls) that name a changed file; two roots resolve to the one the last git commit ran in. - Subagent transcripts Entire keeps under tasks/ are read inside the turn that started them, for decisions only. Replayed on 96 checkpointed commits: 1 line before, 13 after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR brief: 3 agent commits · 8 of 25 test files reach this diffRun at least these. Selection is a safe superset; it never marks a test safe to skip.
What each commit did
Claims checked
What each changed file reaches
Paths above are under Tests to run (8)
Blast radius by changed file: 23 dependents
blastline 0.23.0. Selection is advisory unless your workflow gates on it. How selection works. Graph |
…it cannot follow Review of #58: - A bare run that exits non-zero stays failed whatever its summary says; vitest Test Files/Errors and jest Test Suites lines count as failures. - go and cargo lines only ever say a run failed (a pipe can cut the failing package off). - Summary patterns stay on one line; the pytest one backtracked cubically on long = lines (55 s on 30k characters). - cd/pushd are read with redirects stripped and assigned variables substituted (WT=/x; cd "$WT"); popd, a bare cd or one that cannot be read makes the directory unknown, so a run in another repo is not this one's. - Only test files are named (not a --config file); subagent ids may hold dashes; a temp-folder root is never the repo, pinned by a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s use A name assigned again later in the command no longer reaches back to an earlier cd. Adds a test for cd to an assigned variable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
taylorg009
marked this pull request as ready for review
October 21, 2026 01:21
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
"What the agent decided" (0.20.0) showed a line for 1 of 96 checkpointed commits on a real web-app repository since 2026-09-20. The rules assumed sessions that rarely happen:
vitest … 2>&1 | tail), so the status is the pipe's and nothing counted. The status was also wrong both ways: a denied permission read as a failure, and a run whose output saidTests 4 failedexited 0.python3 - <<EOForcat > filehad no root, so nothing was read.tasks/<tool_use_id>/, and nothing read them.What changes
Tests, pytest===, cargotest result:, gook/FAIL), read and never shown. A bare run with no summary still uses its status, unless it never ran.cp,mv,git apply/checkout/restore/stash,patch) is a probe and is skipped.T="path"; vitest run "$T"namespath.cd "$PWD"stays put. Temp folders are never a root.git commitran in wins, if it's among them.Agentcall, for decisions only.Evidence
The same 96 commits were replayed through
checkpointForwith each commit's own files:failed, then passed, 6failed on its last run, 4 with a design note (a commit can have several)I spot-checked lines by hand against the transcripts:
echo/tscsteps now counts, which fixed a "failed on its last run" claim that turned out false during this work.__tests__folder no longer names one file.Subagent transcripts are real. The repository's
entire/checkpoints/v1holds 50tasks/<tool_use_id>/agent-<id>.jsonlfiles across 7 checkpoints written by Entire 0.11.5. So far all of them are read-only helper agents, so none adds a decision yet.An independent review requested changes and then approved at 5540b0c. The fixes, each with a test that fails on the earlier code:
Test Files/Errorsand jestTest Suites:lines count as failures.cd/pushdare read with redirects stripped and with variables assigned earlier in the same command substituted, in order.popd, a barecd, or a step that can't be read makes the directory unknown.--configfile isn't.Tests: 422 passed (16 new cases in
decisions.test.tsandcheckpoint.test.ts, run against real git checkpoints).🤖 Generated with Claude Code