Repository navigation
Projects board: failed judge asks the user, remove clears board tags, coordinator never auto-resolves - #7038
Conversation
… coordinator never auto-resolves Three fixes from a review of #6602: - A judge that fails or times out leaves the turn `asking` (Needs you), with the reply's last paragraph as `ask`, and never resolves it. The critic fallback applies only when no judge is available. - Removing a member clears its board tags with the `project` tag, in one tag-locked update, so no stale chip stays. - The coordinator's turns are recorded but never judged or resolved. Co-authored-by: openhands <openhands@all-hands.dev> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwJH94xGSYwfoyccT4E
REST API breakage checks (OpenAPI) — ✅ PASSEDResult: ✅ PASSED |
Agent-server conversation start benchmarkBase
How this is measuredEach create is a Canvas-shaped
A timing is flagged when the 95% CIs do not overlap and the medians differ by at least 50 ms, or when a row with at least 8 samples per side moves by at least 150 ms and 25%. A create stalled the server when its slowest |
Coverage Report •
|
||||||||||||||||||||||||||||||
|
The red "Validate PR description" check comes from the first run (44834372013), which read the original description. That description referenced #6602, a PR, where the check expects an issue. I've updated the description, and the run on the edited description (44834507562) passes. Re-running the old job just re-reads the old description. Generated by Claude Code |
all-hands-bot
left a comment
There was a problem hiding this comment.
This review was posted by an AI agent (OpenHands).
Reviewed the full diff against main@fde86c37 and traced each of the three fixes through the surrounding code.
Verified:
- Failed judge —
ConversationOutcomeSubscriber._recordnow distinguishes "judge failed" (raises, surfaced viajudging.exception()) from "no judge" (None). On failure the turn becomesaskingwithask_line(text)as the ask,_notify_finisheddelivers it asasking, and the earlyturn == "asking"return blocks both the critic fallback and theresolved=autowrite. Cancellation by a new user message still keeps the turn finished, andexc_info=erroron the warning is valid logging. The judge-failure path is reachable only inside the shown/judging block, so there is no un-shown failure state. - Coordinator —
_coordinator()is checked before judging and before the resolve step, so the coordinator is never judged and never auto-resolves; its turns still getturn/askand coordinator delivery._is_coordinatorre-reads live tags and the project store per turn, so it stays correct if membership changes. The_question_open()branch before it preserves prior ask semantics for the coordinator too. - Remove —
ProjectService.release_memberdoes one tag-locked update clearing all ofRESERVED_TAG_KEYS(project,resolved,resolvenote,turn,ask) and preserves other tags; the router's 404 semantics (event_service is Noneor project-tag mismatch) and the coordinator 409 guard are unchanged from the oldset_conversation_tagpath.release_members(project delete) now delegates to the same helper, so both paths behave identically.
The F36 feature map is updated to match the new behavior on all three points. The REST contract and OpenAPI schema are unchanged (breakage check green), no SDK/client/persisted-settings surface is touched, and Projects stays behind its experimental flag.
Evidence: the three new/changed tests are absent on main and pass here; locally tests/agent_server/test_project_board.py + test_projects_router.py pass (135 tests), the wider Projects-related agent-server selection passes (228 tests), and ruff + pyright are clean on the changed files. CI on this head is green; the red "Validate PR description" check is from the pre-edit description and the re-run on this exact SHA (44834507562) passed, as the author's comment explains. No live end-to-end run was made, acceptable here since the failures are in-process logic fully covered by the real test paths.
No material correctness, security, or compatibility findings.
✅ APPROVED
HUMAN:
This is a review of 6602 from another agent. Verify these are real or not or wdyt
AGENT:
Why
A review of PR 6602 found three board bugs. All three were confirmed against the merged code, and none of them were in it.
_judgereturnedNoneon an exception, so the turn stayedfinished. A confident critic score could then auto-resolve a reply that was actually asking the user something.DELETE /api/projects/{id}/conversations/{cid}cleared only theprojecttag.turn,ask,resolvedandresolvenotestayed on the conversation.ProjectService.watchattaches the board subscriber to every member, the coordinator included. That meant the judge could setresolved=autoon the coordinator, the conversation the user steers the project from.Summary
project_board.pynow tells "no judge available" (None) apart from "the judge failed" (it raises).asking(Needs you), with the reply's last paragraph asask, and nothing resolves it.ProjectService.release_member. It dropsprojectand all board tags in one tag-locked update.release_members, used on project delete, now uses the same helper.is_coordinator. The coordinator's turns still getturn(andaskfrom a card), but the judge and the resolve step are skipped.Issue Number
Part of #6637.
How to Test
There are three new or changed tests:
test_a_failed_judge_shows_the_turn_as_needs_you_and_never_resolvestest_the_coordinator_is_never_judged_or_resolvedtest_remove_conversation_clears_the_project_and_board_tagsAll three fail on
mainand pass with this change. The 135 board and router tests pass, along with 237 Projects-related agent-server tests. Ruff and pyright are clean on the changed files. I did not run the live end-to-end check.Type
Notes
Projects is behind its experimental flag, so nothing here changes the app with the flag off. The F36 feature map is updated to match.
Generated by Claude Code
🐳 Agent Server images for this PR — GHCR package, pull/run commands, and all pushed tags (click to expand)
• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server
Variants & Base Images
eclipse-temurin:17-jdkpython-node-runtimepython-node-runtimepython-node-runtimegolang:1.21-bookwormPull (multi-arch manifest)
# Each variant is a multi-arch manifest supporting both amd64 and arm64 docker pull ghcr.io/openhands/agent-server:9655e8d-pythonRun
All tags pushed for this build
About Multi-Architecture Support
9655e8d-python) is a multi-arch manifest supporting both amd64 and arm649655e8d-python-amd64) are also available if needed