fix(ally): name the real identity in the self-review guard - #1603
Merged
Conversation
#1599 fixed the same dead identifier in the Step 2 idempotency check but scoped its assertion to that block, so `ally-paperclip[bot]` survived in the "don't review your own work" guard. Ally authors as `allyblockcast[bot]`, so that guard matched nothing and failed open too: one dead identifier, two disabled guards. pim#2251 is the visible result — authored by allyblockcast[bot] and reviewed by allyblockcast[bot]. Widens the contract test to assert the stale identity appears nowhere in the document, and pins the self-review guard to the same exported constants. The command assertions stay fence-scoped, since the prose has to remain free to name an anti-pattern in order to explain it. Verified by mutation: reverting line 185 alone fails tests 3 and 4; fixed file passes 5/5. Positive controls added so an empty read cannot satisfy the file-wide absence assertion. Not changed: whether a clean App self-review should be permitted at all. `isCleanAppSelfReview` in check-ally-review-consistency.mjs deliberately tolerates a non-blocking COMMENTED self-review because GitHub forbids the App approving its own PR, which sits in tension with this guard's blanket skip. Correcting the identifier does not resolve that; flagging rather than silently picking a side.
|
Hey @kkroo! Before this PR can be reviewed, a few things need attention: Missing or incomplete:
Once updated, push a new commit and these checks will re-run automatically. — commitperclip |
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 0be9343
Critical Issues (0)
Important Issues (1)
- [native-codex]
.planning/ally-agent/AGENTS.md:185— the new guard skips PRs authored byallyblockcast(the shared User seat), but the authoritative reviewer policy says that seat must never submit a review or approval; App-authored PRs must instead receive a formalCOMMENTEDreview fromallyblockcast[bot]. This instruction will therefore suppress the required artifact whenever the User seat authors a PR, leaving the review gate without the mandated App review.- Keep the self-review skip scoped to the App author identity (
app/allyblockcast/allyblockcast[bot]) and explicitly route App-authored PRs to the formal AppCOMMENTpath. Do not treat the User seat as an equivalent author/self-review case.
- Keep the self-review skip scoped to the App author identity (
Suggestions (0)
Strengths
- The stale
ally-paperclip[bot]identifier is removed from the executable instruction. - The new contract tests provide positive controls and verify the exact canonical App and User constants.
- The targeted contract test passes with 5 tests.
Recommended Action
- Fix the Important issue before merge.
- Re-run the Ally contract test and verify App-authored PRs still receive formal App comments.
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thinking Path
#1599 fixed a dead reviewer identity in Ally's Step 2 idempotency check, and scoped its contract assertion to that block. That scoping was right for the command assertions — the surrounding prose has to stay free to name an anti-pattern in order to explain it — but it meant the login assertion only covered one of the two places the identity appeared.
ally-paperclip[bot]also sat in the "don't review your own work" guard at line 185. Ally authors asallyblockcast[bot], so that guard matched nothing and failed open exactly like the idempotency check did: one dead identifier, two disabled guards. pim#2251 is the visible consequence — authored byallyblockcast[bot]and reviewed byallyblockcast[bot].I found it verifying #1599 by content on master rather than trusting the merge, which is the only reason it surfaced at all.
What Changed
Line 185 now names
allyblockcast[bot]andallyblockcast, with a pointer to the exported constants so the next reader knows where the source of truth lives.The contract test gains a file-wide assertion that the stale identity appears nowhere, plus one pinning the self-review guard to the same constants. Command assertions stay fence-scoped.
Verification
Mutation-tested, not just run green: reverting line 185 on its own fails tests 3 and 4; the fixed file passes 5/5. Both new assertions carry positive controls, so a failed or empty read cannot satisfy the file-wide absence check.
Risks
Instruction-file change; no runtime code path.
The behavioural consequence is that Ally will now actually skip App-authored PRs, which it has not been doing. If anything downstream has come to depend on those self-reviews existing, it will stop seeing them.
Deliberately not resolved here:
isCleanAppSelfReviewincheck-ally-review-consistency.mjstolerates a non-blockingCOMMENTEDself-review, because GitHub forbids the App approving its own PR. That sits in tension with this guard's blanket skip. Correcting a dead identifier is not the place to decide which behaviour is intended, so I have flagged it rather than quietly picking a side — a reviewer who owns that gate should say which one wins.Model Used
claude-opus-5[1m]
Related work found while searching
fix(ci): ratchet the Ally review-consistency guard so it can signal again (PEN-2847). Addresses the same standing red from the guard side (check-ally-review-consistency.mjs, workflow, baseline). No file overlap with this PR, which changesAGENTS.md— the producer, not the detector. The two are complementary: fix(ci): ratchet the Ally review-consistency guard so it can signal again (PEN-2847) #1585 makes the guard signal usefully again; this stops the producer emitting the duplicates.fix(ally-guard): ignore superseded review status updates. Also guard-side only.Merge-order risk worth flagging: the contract test added here imports
ALLY_APP_REVIEWER_LOGIN/ALLY_USER_REVIEWER_LOGINfromcheck-ally-review-consistency.mjs, which #1585 edits. If #1585 renames or removes those exports, this test fails on master. That coupling is deliberate — pinning prose to the code's own constant is the point — but whichever lands second should re-runscripts/ally-agent-idempotency-contract.test.mjs.AGENTS.md.