fix(running-in-ci): fetch a review's inline comments as part of reading context - #950
Open
tend-agent wants to merge 1 commit into
Open
fix(running-in-ci): fetch a review's inline comments as part of reading context#950tend-agent wants to merge 1 commit into
tend-agent wants to merge 1 commit into
Conversation
…ng context Neither `gh pr view --json reviews` nor `GET /pulls/<n>/reviews/<id>` returns a review's inline comments, and neither signals that anything is missing — so a one-line review body reads as a complete review while the maintainer's actual instruction sits inline, unread. The fetch recipe already existed, but only under "Replying to Comments", which an agent that never intends an inline reply has no reason to open. Move the knowledge to where the read decision is made.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A maintainer's inline review comment told the bot not to do a thing; seven minutes later the bot published that exact thing on the same thread, because the review body it read gave no sign the inline comment existed.
What happened
On PRQL/prql#6181,
kgutwinsubmitted review 4911815208 (COMMENTED, 00:07:20Z). Its body is one line — "Closing this in favor of #6020 which is a superset of this PR and is more likely to be merged." The substance was in the review's single inline comment,discussion_r3762633057onprqlc/prqlc/tests/integration/sql.rs:tend-mentionrun 31549073589 handled the review event and replied at 00:12:37Z — agreeing with the close, then pasting both of the PR's tests so they could be lifted into #6020. One of the two pasted tests istest_aggregate_matching_group_key, carrying the literal// TODO: this is wrongcomment the maintainer had just rejected, offered for adoption into another PR. The inline comment itself was never answered.Root cause
The session log shows the agent read the review and stopped at its body. The prompt named the review ID but not its contents:
The agent's two context reads were
gh pr view 6181 --json title,body,author,state,headRefName,comments,reviews,urlandgh api repos/PRQL/prql/pulls/6181/reviews/4911815208. Neither returns inline comments. The second returned exactly the one-line body, which reads as a complete review. There is no call toreviews/4911815208/comments,pulls/6181/comments, or the comment ID anywhere in the log, and no assistant text mentions the objection.That is the structural part: both obvious ways to read a review omit its inline comments and give no signal that anything is missing.
running-in-cidoes carry the correct fetch, but only under "Replying to Comments" — framed as how to reply inline. An agent that reads the body, concludes a top-level comment is the right response, and never intends an inline reply has no reason to reach that section. The recipe sits behind the decision it was supposed to inform.Fix
One subsection in "Read Context", where the agent learns what to read, stating that the inline comments are a separate fetch and giving the command. It duplicates the
gh api …/reviews/<id>/commentscall already in "Replying to Comments"; that is deliberate — the two are reached at different moments, and the read-time moment is the one that failed here.Gate assessment
PRQL/prqlevidence gists for 2026-07 and 2026-08 forreviews/<id>/comments, unanswered-inline, and contradicted-maintainer shapes — no prior match. This is the first recorded instance, which is consistent with it needing a review that puts its substance inline rather than in the body.Evidence log: https://gist.github.com/192514ea2c36586f9b7f842a482d62ab
Detected by
review-reviewersrun https://github.com/max-sixty/tend/actions/runs/31552948925.