Skip to content

ci: run the Claude review from main, on Dependabot PRs too, with a pass/fail verdict - #215

Merged
schenksj merged 2 commits into
mainfrom
feature/claude-review-from-main
Oct 14, 2026
Merged

schenksj merged 2 commits into
mainfrom
feature/claude-review-from-main

Conversation

@rbrenton

@rbrenton rbrenton commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Ports the review workflow that is live in indextables_spark (indextables/indextables_spark#415). Supersedes #212.

Why

  • The current claude-code-review.yml fails on every pull request (claude-review is red across the repo).
  • It runs on pull_request, so the reviewer's prompt and tools come from the pull request under review, and runs that Dependabot triggers receive no Actions secrets, so it cannot review dependency updates.

What changes

  • Trigger: pull_request_target, which always takes the workflow and its checkout from the default branch.
  • Who gets a review: pull requests whose head is a branch of this repository, whose base is the default branch, that are not drafts, and whose author is dependabot[bot], rbrenton, schenksj or chrisfauerbach. Fork pull requests and other authors skip the job.
  • What the reviewer sees and can do: a checkout of the default branch and the pull request's diff in a file outside the workspace. It can only read those; the pull request's head is never checked out or executed.
  • Result: a structured verdict. A second job with no model and no Claude credential validates it, posts or updates one comment, and ends green on pass, red on fail, and red with a "did not complete" reason for anything else.
  • Instructions written for this repository: the previous prompt's focus (correctness, security, conflict markers, JNI error propagation) plus native resource and memory safety across JNI, and dependency-update guidance for Maven, Cargo under native/ and GitHub Actions: the tantivy and quickwit git revisions and the exact aws-sdk-s3 requirement are held by hand, 0.x minor bumps count as major, arrow/parquet and iceberg crates move together, and build-time code (Maven plugins, build.rs, procedural macros) is flagged.
  • The scripts under .github/scripts/claude-review/ are byte-identical to indextables_spark's, with the same offline test suite.

Behaviour change

  • The claude-review check is replaced by Review and Verdict (plus Offline tests when the scripts or workflows change). None is a required check.
  • The old workflow also started Claude on comments mentioning @claude. That is not carried over (3 successful runs in its last 300). It can be restored as a separate workflow if wanted.
  • The verdict is advisory. A missing or skipped Verdict is not a pass, and on dependency updates a pass checks the version changes against the rules above, not the contents of the new releases.

Verification

  • Offline suite: 203 checks pass in this repository.
  • The adapted instructions were run locally against real diffs from this repo: an arrow-buffer 57→59 bump on its own was failed (family split, probable compile break), and a jackson-databind patch bump and a bytes lock-only bump passed.
  • In indextables_spark the same workflow's first Dependabot-triggered run received the credential, produced a verdict and posted its comment.
  • With the current size caps, none of the last 60 pull requests here would have been too large to review.

Not verifiable before merge

A pull_request_target workflow only takes effect once on the default branch. After merge: close and reopen a same-repo PR for the human path, and @dependabot rebase on one Dependabot PR for the bot path.

The reviewer's read confinement has been probed locally on macOS only and has not been exercised on a Linux runner.

…om-main workflow

The review in claude-code-review.yml ran on `pull_request` with
claude-code-action@beta, a `direct_prompt` and a pinned model, and
currently fails on every pull request. It also had two
structural problems: a `pull_request` workflow is defined by the pull
request itself, so a change could rewrite the reviewer's prompt and tools
while still receiving the credential; and runs started by Dependabot on
`pull_request` receive no repository secrets, so dependency updates were
never reviewed.

Port the workflow already in use in indextables_spark:

- claude-review.yml runs on `pull_request_target`, so the workflow and its
  scripts always come from the default branch. The pull request is never
  checked out; its diff is fetched through the API. The job starts only for
  same-repository, non-draft pull requests against the default branch by an
  allowlisted author.
- The reviewer has Read/Grep/Glob only, confined to the base checkout and the
  diff file, with a read-only token. A second job without the credential
  validates the schema-constrained result, posts or updates one comment and
  is green only for `pass`; anything else, including a failed or timed-out
  review, is red.
- Actions are pinned to commit ids and the runner to ubuntu-24.04.
- .github/scripts/claude-review/ and claude-review-scripts.yml carry the
  scripts and their offline tests. The scripts are byte-identical to the
  sibling repository's.

Adapted for this repository:

- Review instructions keep the previous prompt's points (correctness,
  security, merge-conflict markers and build breaks, error propagation at
  the JNI boundary, API consistency; no style, no pre-existing issues) and
  add native resource and memory safety across JNI.
- Dependency-update guidance covers the three ecosystems here (Maven, Cargo
  under native/, GitHub Actions): the tantivy and quickwit git revisions and
  exact-version requirements that are moved by hand, crate families that
  must move together, Cargo.lock sources and checksums, and build-time code
  (Maven plugins, build-dependencies, procedural macros, build.rs).
- The header and one test no longer refer to a second workflow that does
  not exist here; the README quotes this repository's diff-size history.

Behaviour changes to be aware of: the old workflow also started Claude on
comments that mention it; this one does not. The check `claude-review` is
replaced by `Review` and `Verdict`.

Not verifiable before merge: a pull_request_target workflow only takes
effect once it is on the default branch.

@schenksj schenksj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@schenksj
schenksj enabled auto-merge (squash) October 14, 2026 00:43
@schenksj
schenksj merged commit 0719544 into main Oct 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants