Repository navigation
pool: a chat /task never reaches the Model Pool judge #1527
Description
What happened
The Model Pool judge scores tasks when they land (poolJudgeHook in cmd/codeaf/poolrecord.go, wired as taskLanded in cmd/codeaf/chatv3.go). A task started with the chat's ordinary /task never reaches that hook, so its landing is never judged. This was found on 2026-09-25 while fixing the judge's keyless-boot bug in #1519: a hand check of that fix could not reach the judge from /task at all.
Replication
Deterministic: a cmd/codeaf test that lands a /task through the chat's own task road, with a stub worker and the pool in record mode, then checks the pool's judged/ directory, or pool/judge-last.json. Nothing is written today. Compare with the road that does call taskLanded.
By hand (real model, about $0.01):
- Run
/taskon a trivial change withCODEAF_MODEL_POOLrecording. - After it lands,
codeaf pool showlists no new judged landing.
The fix
Every road that lands a task, /task included, reports its landing to the judge hook once.
Acceptance
- e2e: after a
/tasklands,codeaf pool showlists its judged entry. - Unit: every landing road goes through the one hook. A structural test over the landing sites would do.
Activity
santoshkumarradha commentedon Sep 27, 2026
New evidence from dev 837b2b0, found by the docs lanes while writing the public docs. Replication for every item below starts from a clean install:
git checkout 837b2b0 && make build, binarybin/codeaf), or install the dev build withcurl -fsSL https://agentfield.ai/get/devaf | bash.export HOME=$(mktemp -d), exportOPENROUTER_API_KEY, and keep the default model (~deepseek/deepseek-v4-flash-latest, crew on auto).task.max_loadto0(/settings, Tasks) so the busy-machine gate does not hold tasks.Unconfirmed sighting on dev 837b2b0 (B64)
done;logs/calls.jsonlhad no judge call andcodeaf pool statusstill saidlast judge: none yet (no landing judged).cmd/codeaf/poolrecord.go:157-163), so the cause could not be seen; it may also be that a landing with no change is not judged. Consistent with this issue's road finding.Found while writing the public docs; manual text differences are in #1545.
🤖 Generated with Claude Code