chore: remove PR review GitHub Actions workflow#3393
Conversation
The code review bot is now configured via OpenHands Cloud automation, which triggers when openhands-agent/all-hands-bot is requested as a reviewer, as well as on tags. The GitHub Actions workflow is no longer needed. Co-authored-by: openhands <openhands@all-hands.dev>
Python API breakage checks — ✅ PASSEDResult: ✅ PASSED |
REST API breakage checks (OpenAPI) — ✅ PASSEDResult: ✅ PASSED |
all-hands-bot
left a comment
all-hands-bot
left a comment
There was a problem hiding this comment.
Clean migration — removing the GitHub Actions workflows in favour of the already-running Cloud automation is the right call. The diff is tight and the reasoning is well-documented. One minor documentation gap worth addressing before merging.
This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation.
| - AgentSkills progressive disclosure goes through `AgentContext.get_system_message_suffix()` into `<available_skills>`, and `openhands.sdk.context.skills.to_prompt()` truncates each prompt description to 1024 characters because the AgentSkills specification caps `description` at 1-1024 characters. | ||
| - Workspace-wide uv resolver guardrails belong in the repository root `[tool.uv]` table. When `exclude-newer` is configured there, `uv lock` persists it into the root `uv.lock` `[options]` section as both an absolute cutoff and `exclude-newer-span`, and `uv sync --frozen` continues to use that locked workspace state. | ||
| - `pr-review-by-openhands` delegates to `OpenHands/extensions/plugins/pr-review@main`. Repo-specific reviewer instructions live in `.agents/skills/custom-codereview-guide.md`, and because task-trigger matching is substring-based, that `/codereview` skill is also auto-injected for the workflow's `/codereview-roasted` prompt. | ||
| - PR code review is handled via OpenHands Cloud automation (not a GitHub Actions workflow). Repo-specific reviewer instructions live in `.agents/skills/custom-codereview-guide.md`. The automation triggers when `openhands-agent` or `all-hands-bot` is requested as a reviewer, as well as on tags. |
There was a problem hiding this comment.
🟡 Suggestion: The replacement description is narrower than what the deleted workflow actually covered. The old workflow also triggered on opened (non-draft, non-first-time-contributor) and labeled (for the review-this label). The current wording — "triggers when openhands-agent or all-hands-bot is requested as a reviewer, as well as on tags" — doesn't mention those two conditions, and the event that triggered this very PR's review (ready_for_review) isn't listed either.
If the Cloud automation covers all the same conditions, it would be helpful to document them completely here so future contributors know what to expect. If some conditions were intentionally dropped (e.g., auto-review on opened), noting that is worth a brief comment too.
Suggested expansion:
| - PR code review is handled via OpenHands Cloud automation (not a GitHub Actions workflow). Repo-specific reviewer instructions live in `.agents/skills/custom-codereview-guide.md`. The automation triggers when `openhands-agent` or `all-hands-bot` is requested as a reviewer, as well as on tags. | |
| - PR code review is handled via OpenHands Cloud automation (not a GitHub Actions workflow). Repo-specific reviewer instructions live in `.agents/skills/custom-codereview-guide.md`. The automation triggers on `ready_for_review`, `opened` (non-draft, established contributors), `labeled` (with `review-this`), and when `openhands-agent` or `all-hands-bot` is requested as a reviewer. |
There was a problem hiding this comment.
Good catch — updated in a58f06e. The description now lists all three actual trigger conditions:
The automation triggers on
ready_for_review(established contributors), whenall-hands-botis requested as a reviewer, and when thereview-thislabel is added.
One correction to the suggestion: the Cloud automation does not trigger on opened — only ready_for_review. This is a deliberate difference from the old GitHub Actions workflow. The live automation's event triggers are pull_request.ready_for_review, pull_request.review_requested, and pull_request.labeled.
This reply was posted by an AI agent (OpenHands) on behalf of the user.
all-hands-bot
left a comment
all-hands-bot
left a comment
There was a problem hiding this comment.
✅ QA Report: PASS
The repo-scoped goal was verified: the PR branch removes both redundant PR review workflows, while existing Cloud automation evidence is visible on GitHub.
Does this PR achieve its stated goal?
Yes. The stated goal is to stop carrying the PR Review by OpenHands and PR Review Evaluation GitHub Actions workflows because PR review has moved to OpenHands Cloud automation. GitHub’s workflow registry still shows both workflows active on the default branch, but the PR branch returns HTTP 404 for both workflow paths and the PR branch git tree has no matching workflow files, so merging this PR will remove those workflows from the repository. I also fetched the referenced review on PR #3298 and confirmed it was submitted by all-hands-bot with the OpenHands Automation disclosure.
| Phase | Result |
|---|---|
| Environment Setup | ✅ gh, uv, Python, and the PR checkout were available; no app build was required for this workflow/docs-only change. |
| CI Status | ⏳ Most checks are passing; Agent Server build/publish jobs were still in progress when checked. |
| Functional Verification | ✅ Verified workflow removal through GitHub API + local git tree comparison, and verified the referenced Cloud automation review exists. |
Functional Verification
Test 1: Deleted GitHub Actions workflows disappear from the PR branch
Step 1 — Establish baseline without the fix:
Ran gh workflow list --repo OpenHands/software-agent-sdk --all | grep -E 'PR Review by OpenHands|PR Review Evaluation':
PR Review by OpenHands active 198176755
PR Review Evaluation active 231924678
This confirms the default branch currently registers both GitHub Actions workflows as active.
Step 2 — Apply the PR's changes:
The workspace was checked out at PR head bdbc62c483401759d18a2f00eb939cf56cef7a3c on remove-pr-review-workflow.
Step 3 — Re-run with the fix in place:
Ran GitHub Contents API checks for both workflow paths at ref=remove-pr-review-workflow:
.github/workflows/pr-review-by-openhands.yml: HTTP 404 (absent on PR branch)
.github/workflows/pr-review-evaluation.yml: HTTP 404 (absent on PR branch)
Then compared git trees:
origin/main: .github/workflows/pr-review-by-openhands.yml .github/workflows/pr-review-evaluation.yml
HEAD:
This shows the workflow files exist on the baseline branch and are absent from the PR branch. Since GitHub Actions workflows are registered from files under .github/workflows, this confirms the repo-scoped removal works.
Test 2: Supporting evidence for Cloud automation replacement
Step 1 — Establish baseline / external replacement evidence:
Ran gh api repos/OpenHands/software-agent-sdk/pulls/3298/reviews --paginate filtered to review 4363416738:
{
"id": 4363416738,
"user": "all-hands-bot",
"state": "COMMENTED",
"submitted_at": "2026-05-26T12:01:53Z",
"html_url": "https://github.com/OpenHands/software-agent-sdk/pull/3298#pullrequestreview-4363416738",
"body_preview": "... This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation."
}This confirms the PR description’s referenced example is a real GitHub review from all-hands-bot and explicitly identifies OpenHands Automation as the source.
Step 2 — Apply the PR's changes:
No local software change is needed for Cloud automation; the PR only removes the redundant GitHub Actions definitions.
Step 3 — Re-run with the fix in place:
Ran gh run list --repo OpenHands/software-agent-sdk --branch remove-pr-review-workflow --limit 30 and checked for removed workflow names. The live PR still shows skipped PR Review by OpenHands pull_request_target runs from the base-branch workflow, which is expected before this workflow-removal PR is merged. No PR Review Evaluation runs appeared for the PR branch.
This means the current PR cannot fully suppress base-branch pull_request_target workflow records until merge, but the branch content that will become main no longer contains the workflow definitions.
Unable to Verify
I did not live-trigger the OpenHands Cloud automation by requesting reviewers or creating tags, because doing so would mutate GitHub PR/release state and could start an external automation run. Instead, I verified the referenced existing Cloud automation review through the GitHub API.
Issues Found
None.
This QA review was generated by an AI agent (OpenHands) on behalf of the user.
Address review feedback: the AGENTS.md description was too narrow. Updated to list the actual trigger events: - ready_for_review (established contributors) - review_requested (all-hands-bot) - labeled (review-this) Note: the Cloud automation does NOT trigger on 'opened' — only ready_for_review. This is a deliberate difference from the old GitHub Actions workflow. Co-authored-by: openhands <openhands@all-hands.dev>
|
✅ Review complete. This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here. |
all-hands-bot
left a comment
all-hands-bot
left a comment
There was a problem hiding this comment.
🟢 Good taste — Clean, focused migration PR. Removing redundant GitHub Actions workflows in favour of the already-live Cloud automation is the right call. The diff is minimal and the rationale is solid.
What this PR does
| File | Change |
|---|---|
.github/workflows/pr-review-by-openhands.yml |
Deleted — redundant now that the same functionality runs via Cloud automation |
.github/workflows/pr-review-evaluation.yml |
Deleted — depended on trace artifacts produced by the above workflow; without it the evaluation would always skip silently |
AGENTS.md |
Updated — replaces the GitHub Actions-centric description with accurate Cloud automation trigger conditions |
Review
pr-review-by-openhands.yml / pr-review-evaluation.yml (deleted)
Both deletions are correct and complete:
- The review workflow is fully superseded by Cloud automation already running in production (evidence: PR #3298 review linked in the PR description, and the fact that this very PR is being reviewed by the Cloud automation).
- The evaluation workflow had a hard dependency on
pr-review-by-openhands.ymlviaworkflow: pr-review-by-openhands.ymlon its artifact-download step. Without the parent workflow producing trace artifacts, the evaluation would perpetually hit theif_no_artifact_found: warnbranch and exit early — so keeping it would just be dead code. Deleting it together is the right move.
One thing worth keeping in mind: the evaluation workflow was instrumenting review quality via Laminar traces and uploading pr-review-evaluation-*.log artifacts. That observability surface is now gone. If the Cloud automation has an equivalent evaluation pipeline, no action needed; if not, this is a conscious gap to be aware of.
AGENTS.md (line 118)
The updated description correctly replaces the stale GitHub Actions reference. The prior concern raised in this review thread (incomplete trigger condition list) was addressed in commit a58f06e — the current wording lists all three active triggers: ready_for_review, all-hands-bot reviewer request, and review-this label. The text is accurate and readable.
[RISK ASSESSMENT]
- [Overall PR]
⚠️ Risk Assessment: 🟢 LOW
Deletes two CI workflow files and updates one documentation line. No production code paths are affected. The Cloud automation that replaces the workflows is already verified in production. Rollback is a simple revert if needed.
VERDICT:
✅ Worth merging — the change is well-scoped, the PR description provides concrete evidence (PR #3298 review), and the prior feedback was addressed before this re-review request.
KEY INSIGHT:
Removing the evaluation workflow alongside the review workflow is the correct and complete action — keeping it would leave dead code that silently no-ops on every PR close.
This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation
all-hands-bot
left a comment
all-hands-bot
left a comment
There was a problem hiding this comment.
🟢 Good taste — Clean, focused migration PR. Removing redundant GitHub Actions workflows in favour of the already-live Cloud automation is the right call. The diff is minimal and the rationale is solid.
What this PR does
| File | Change |
|---|---|
.github/workflows/pr-review-by-openhands.yml |
Deleted — redundant now that the same functionality runs via Cloud automation |
.github/workflows/pr-review-evaluation.yml |
Deleted — depended on trace artifacts produced by the above workflow; without it the evaluation would always skip silently |
AGENTS.md |
Updated — replaces the GitHub Actions-centric description with accurate Cloud automation trigger conditions |
Review
pr-review-by-openhands.yml / pr-review-evaluation.yml (deleted)
Both deletions are correct and complete:
- The review workflow is fully superseded by Cloud automation already running in production (evidence: PR #3298 review linked in the PR description, and the fact that this very PR is being reviewed by the Cloud automation).
- The evaluation workflow had a hard dependency on
pr-review-by-openhands.ymlviaworkflow: pr-review-by-openhands.ymlon its artifact-download step. Without the parent workflow producing trace artifacts, the evaluation would perpetually hit theif_no_artifact_found: warnbranch and exit early — so keeping it would just be dead code. Deleting it together is the right move.
One thing worth keeping in mind: the evaluation workflow was instrumenting review quality via Laminar traces and uploading pr-review-evaluation-*.log artifacts. That observability surface is now gone. If the Cloud automation has an equivalent evaluation pipeline, no action needed; if not, this is a conscious gap to be aware of.
AGENTS.md (modified)
The updated description correctly replaces the stale GitHub Actions reference. The prior concern raised in an earlier review thread (incomplete trigger condition list) was addressed in commit a58f06e — the current wording lists all three active triggers: ready_for_review, all-hands-bot reviewer request, and review-this label. The text is accurate and readable.
[RISK ASSESSMENT]
- [Overall PR]
⚠️ Risk Assessment: 🟢 LOW
Deletes two CI workflow files and updates one documentation line. No production code paths are affected. The Cloud automation that replaces the workflows is already verified in production. Rollback is a simple revert if needed.
VERDICT:
✅ Worth merging — the change is well-scoped, the PR description provides concrete evidence (PR #3298 review), and the prior feedback was addressed before this re-review request.
KEY INSIGHT:
Removing the evaluation workflow alongside the review workflow is the correct and complete action — keeping it would leave dead code that silently no-ops on every PR close.
This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation
all-hands-bot
left a comment
all-hands-bot
left a comment
There was a problem hiding this comment.
⚠️ QA Report: PASS WITH ISSUES
Verified the repo-scoped behavior: the redundant PR review workflow files are absent on the PR branch, the old Action is skipped on this PR, and AGENTS guidance now points to Cloud automation; CI is not green at this snapshot.
Does this PR achieve its stated goal?
Yes for the repository change. The two workflow files that exist on main return 404 Not Found on remove-pr-review-workflow, which is the expected user-visible GitHub behavior after removing them. I also confirmed the current PR’s legacy PR Review by OpenHands check is SKIPPED, and the linked example Cloud review exists from all-hands-bot; I did not mutate PR reviewer assignments to live-trigger Cloud automation.
| Phase | Result |
|---|---|
| Environment Setup | ✅ Existing PR checkout and GitHub CLI/API access were sufficient; no test suite, lint, or formatter was run. |
| CI Status | Python API, REST API (OpenAPI), and unresolved-review-threads failing; Agent Server/QA checks still in progress; legacy pr-review skipped. |
| Functional Verification | ✅ GitHub/API-visible workflow removal and AGENTS guidance update verified before/after. |
Functional Verification
Test 1: Removed workflow files are no longer available on the PR branch
Step 1 — Establish baseline on main:
Ran:
python3 - <<'PY'
import subprocess
repo = 'OpenHands/software-agent-sdk'
for path in [
'.github/workflows/pr-review-by-openhands.yml',
'.github/workflows/pr-review-evaluation.yml',
]:
proc = subprocess.run(
['gh', 'api', '-X', 'GET', '-i', f'repos/{repo}/contents/{path}', '-f', 'ref=main'],
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
status = next(line for line in proc.stdout.splitlines() if line.startswith('HTTP/'))
print(f'{path}: exit={proc.returncode}; {status}')
PYObserved:
.github/workflows/pr-review-by-openhands.yml: exit=0; HTTP/2.0 200 OK
.github/workflows/pr-review-evaluation.yml: exit=0; HTTP/2.0 200 OK
This confirms the workflows are present and fetchable on the base branch.
Step 2 — Apply the PR's changes:
Used the checked-out PR branch remove-pr-review-workflow at 7391f7246a8f7760d5df02c572682aea832d31c9.
Step 3 — Re-run against the PR branch:
Ran the same GitHub Contents API check with ref=remove-pr-review-workflow.
Observed:
.github/workflows/pr-review-by-openhands.yml: exit=1; HTTP/2.0 404 Not Found
.github/workflows/pr-review-evaluation.yml: exit=1; HTTP/2.0 404 Not Found
This shows GitHub no longer exposes either workflow file on the PR branch, matching the PR’s stated removal goal.
Test 2: Local git tree and AGENTS guidance reflect the migration
Step 1 — Baseline:
Ran:
git ls-tree --name-only origin/main:.github/workflows | grep -E '^pr-review-(by-openhands|evaluation)\.yml$' || true
git show origin/main:AGENTS.md | grep -F 'pr-review-by-openhands' | head -1Observed:
pr-review-by-openhands.yml
pr-review-evaluation.yml
- `pr-review-by-openhands` delegates to `OpenHands/extensions/plugins/pr-review@main`. Repo-specific reviewer instructions live in `.agents/skills/custom-codereview-guide.md`, and because task-trigger matching is substring-based, that `/codereview` skill is also auto-injected for the workflow's `/codereview-roasted` prompt.
This confirms the old repo guidance referenced the GitHub Actions workflow.
Step 2 — PR branch:
Ran:
git ls-tree --name-only HEAD:.github/workflows | grep -E '^pr-review-(by-openhands|evaluation)\.yml$' || true
grep -F 'PR code review is handled via OpenHands Cloud automation' AGENTS.mdObserved:
- PR code review is handled via OpenHands Cloud automation (not a GitHub Actions workflow). Repo-specific reviewer instructions live in `.agents/skills/custom-codereview-guide.md`. The automation triggers on `ready_for_review` (established contributors), when `all-hands-bot` is requested as a reviewer, and when the `review-this` label is added.
The workflow grep returned no files, and the repo guidance now points users to Cloud automation instead of the removed Action.
Test 3: GitHub status reflects the old Action is not running here
Ran:
gh pr checks 3393 --repo OpenHands/software-agent-sdk --json name,workflow,state,bucket --jq 'map(select(.workflow == "PR Review by OpenHands"))'Observed:
[
{
"bucket": "skipping",
"name": "pr-review",
"state": "SKIPPED",
"workflow": "PR Review by OpenHands"
}
]This is consistent with the workflow being removed for the PR branch: the legacy Action check did not run review work on this PR.
Test 4: Linked Cloud automation example exists
Ran:
gh api repos/OpenHands/software-agent-sdk/pulls/3298/reviews --paginate --jq '.[] | select(.id == 4363416738) | {id,user:.user.login,state,submitted_at,html_url}'Observed:
{
"html_url": "https://github.com/OpenHands/software-agent-sdk/pull/3298#pullrequestreview-4363416738",
"id": 4363416738,
"state": "COMMENTED",
"submitted_at": "2026-05-26T12:01:53Z",
"user": "all-hands-bot"
}This verifies the PR description’s linked example points to a real all-hands-bot review. It does not by itself prove every future Cloud automation trigger path.
Unable to Verify
I did not live-trigger OpenHands Cloud automation by adding/removing reviewers or labels on this PR because that would mutate PR state and potentially start external automation. Future QA guidance could document a safe staging repository or dry-run automation endpoint for validating Cloud reviewer triggers end-to-end without affecting production PRs.
Issues Found
- 🟡 CI status: Manual QA found no functional issue with the workflow removal, but this PR was not merge-green at the time of review (
Python API,REST API (OpenAPI), andunresolved-review-threadswere failing, with additional checks still in progress).
Verdict: PASS WITH ISSUES
This QA review was created by an AI agent (OpenHands) on behalf of the user.
5 participants
Summary
Removes the
pr-review-by-openhands.ymlandpr-review-evaluation.ymlGitHub Actions workflows as the code review bot has been migrated to OpenHands Cloud automation.Why
The PR review functionality is already handled via the OpenHands Cloud automation, which is triggered:
openhands-agentorall-hands-botis requested as a reviewer on a PRHere's an example of the Cloud automation already working correctly:
#3298 (review)
The GitHub Actions workflow is therefore redundant and can be removed.
Changes
.github/workflows/pr-review-by-openhands.yml.github/workflows/pr-review-evaluation.yml(companion evaluation workflow, now obsolete)AGENTS.mdto reflect that PR review is handled via OpenHands Cloud automation@xingyaoww can click here to continue refining the PR
Agent Server images for this PR
• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server
Variants & Base Images
eclipse-temurin:17-jdknikolaik/python-nodejs:python3.13-nodejs22-slimgolang:1.21-bookwormPull (multi-arch manifest)
# Each variant is a multi-arch manifest supporting both amd64 and arm64 docker pull ghcr.io/openhands/agent-server:7391f72-pythonRun
All tags pushed for this build
About Multi-Architecture Support
7391f72-python) is a multi-arch manifest supporting both amd64 and arm647391f72-python-amd64) are also available if needed