Repository navigation
fix(bench): planner timeout configurable, default 600s (M16 slice 2, #211) - #212
Merged
Merged
Conversation
…211) 300s -> 600s already landed in main via PR #41; the remaining gap is that the value is a hardcoded literal. Make it PLANNER_TIMEOUT_S, overridable with EVALID_PLANNER_TIMEOUT_S, default 600s so the planner+executor split inside the CAP_S 1200s cell budget is unchanged. Guard test updated to pin the default (600s) instead of the literal. Local: bench python tests 40/40 pass.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
M16 slice 2 的超时处置(#211,#204)。
背景:先导 openswarm 0/6 的根因是 planner 超时 300 秒硬编码(6 格均 rc 143、wall 约 301 秒、task file 未写出、executor 从未启动)。300→600 秒已于 PR #41 合并进 main,另加 PLANNER 提示词禁探索;本改动补最后一块——把 600 从字面量改成可配置常量。
改动(行为不变):
bench/swebench/harness/arms.py:新增PLANNER_TIMEOUT_S = int(os.environ.get("EVALID_PLANNER_TIMEOUT_S", "600")),planner 的run_pi调用改传该常量。默认 600 秒不变——600 是与 1200s cap 匹配的分账(planner 600+executor 余量约 600,executor 预算由剩余时间倒推);再往上加会 1:1 挤占 executor,故不直接抬到 1200。bench/swebench/test_planner_prompt.py:pinning 测试改锁默认值 600(原锁字面量600),仍禁擅自改默认值;覆盖只走EVALID_PLANNER_TIMEOUT_S。验证:本地 bench python 单测 40/40 通过(一次跑完 7 个文件);导入实测默认
PLANNER_TIMEOUT_S = 600、EVALID_PLANNER_TIMEOUT_S=900时为 900。切片 2 全矩阵(180 格)用默认 600 跑;若仍有 planner 超时,再凭落盘数据决定是否调 env,不在本 PR 改线。