Repository navigation
docs: give a docs-only branch a reader test in place of the adversarial pass - #34
Conversation
…al pass A fresh agent reads the changed files alone and answers questions that the author writes first. A second fresh agent checks each claim against the code. The first trial, on PR 33, found six errors that a read of the diff did not find. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bhj4Ck4y5iwZUsjpgQx8A2
The reader test of this branch found these problems. AGENTS.md did not define a docs-only branch. Both commands said to run a pass before a push, but a pass reads what is pushed. Both named a repository that is not this one. A reader agent loads AGENTS.md before it reads the changed files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bhj4Ck4y5iwZUsjpgQx8A2
The second reader test and fact check of this branch found these problems. AGENTS.md did not say that a revision can need only the fact check, or no pass. It called the reader cold. Neither file said which paths go to scripts/docs-only.sh. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bhj4Ck4y5iwZUsjpgQx8A2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bhj4Ck4y5iwZUsjpgQx8A2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Bhj4Ck4y5iwZUsjpgQx8A2
There was a problem hiding this comment.
🟢 Approval recommended
No blocking or unresolved issues were raised in the review, leaving nothing outstanding to address.
No open issues 🎉
What changed in this PR
This PR extends the repository's pre-PR process so that a docs-only branch gets a "reader test" instead of the adversarial pass, which previously was simply skipped for docs-only branches. A docs-only branch is one whose changed paths scripts/docs-only.sh accepts (Markdown outside crates//taps/). The reader test spawns two fresh subagents at the same pre-PR points as the adversarial pass: one reads only the changed files plus their diff and answers author-written questions, and one fact-checks each claim against the code. The PR also fixes two latent errors in the existing adversarial prompt that were duplicated into the new command.
Changes:
- Add
.claude/commands/reader-test.mddefining the reader-test procedure (question writing, two cold agents, grading, and re-run rules). - Update
AGENTS.mdto require the reader test for docs-only branches "at the same points," list the new prompt in the.claude/table, and clarify that both passes run after the push and before review is (re-)requested. - Correct
.claude/commands/adversarial.md: replace the hardcodedsstriker/durable-memorywith "this repository," and fix the revision-pass timing to "after its push and before review is re-requested."
| File | Description |
|---|---|
.claude/commands/reader-test.md |
New command: full reader-test procedure for docs-only branches, with reader and fact-check prompts and re-run rules. |
AGENTS.md |
Documents the reader test as the docs-only substitute for the adversarial pass, adds the command to the .claude/ table, and corrects both passes' timing. |
.claude/commands/adversarial.md |
Fixes the hardcoded repo name and the revision-pass timing. |
No blocking issues found. The new command is internally consistent with AGENTS.md and with scripts/docs-only.sh (the referenced git diff --name-only --no-renames origin/main...origin/<branch> | scripts/docs-only.sh invocation matches the script's documented usage and intended semantics), and the two adversarial.md corrections are accurate.
Review effort: Balanced
Summary
Before this change, a branch that touched only docs had no pre-PR pass. This change gives such a branch a reader test in place of the adversarial pass.
A docs-only branch is a branch whose changed paths
scripts/docs-only.shaccepts. The reader test has these steps:The command
.claude/commands/reader-test.mdgives the full procedure. It also says which revisions need both agents again, which need only the fact check, and which need no pass.This change also corrects two errors in
.claude/commands/adversarial.md. These errors were also in the first version of the new command:origin/<branch>, so it runs after the push and before review is re-requested. AGENTS.md now says this for both passes.One result of the new definition: a branch that changes only a docs file that is not Markdown, such as
docs/design-book.html, now gets the adversarial pass.Trial
The first trial was PR #33. It ran four rounds. The fact checks found 13 errors in the text: 6, 4, 2 and 1 in the four rounds. The readers found many sentences with two meanings. Details are in #33.
Reader test of this branch
This branch is docs-only, so it got its own reader test. I wrote these questions first:
README.md. What must you do before you open the PR, and in what order?Results:
scripts/docs-only.sh.These points stay open:
.claude/stays ignored", but.gitignoreignores only two paths under.claude/. This change does not touch that line.Posted by Claude on behalf of @sstriker