Skip to content

fix(bench): planner timeout configurable, default 600s (M16 slice 2, #211) - #212

Merged
surlymochan merged 1 commit into
mainfrom
m16s2-planner-timeout
Nov 16, 2026
Merged

surlymochan merged 1 commit into
mainfrom
m16s2-planner-timeout

Conversation

@surlymochan

Copy link
Copy Markdown
Owner

M16 slice 2 的超时处置(#211,#204)。

背景:先导 openswarm 0/6 的根因是 planner 超时 300 秒硬编码(6 格均 rc 143、wall 约 301 秒、task file 未写出、executor 从未启动)。300→600 秒已于 PR #41 合并进 main,另加 PLANNER 提示词禁探索;本改动补最后一块——把 600 从字面量改成可配置常量。

改动(行为不变):

  • bench/swebench/harness/arms.py:新增 PLANNER_TIMEOUT_S = int(os.environ.get("EVALID_PLANNER_TIMEOUT_S", "600")),planner 的 run_pi 调用改传该常量。默认 600 秒不变——600 是与 1200s cap 匹配的分账(planner 600+executor 余量约 600,executor 预算由剩余时间倒推);再往上加会 1:1 挤占 executor,故不直接抬到 1200。
  • bench/swebench/test_planner_prompt.py:pinning 测试改锁默认值 600(原锁字面量 600),仍禁擅自改默认值;覆盖只走 EVALID_PLANNER_TIMEOUT_S。

验证:本地 bench python 单测 40/40 通过(一次跑完 7 个文件);导入实测默认 PLANNER_TIMEOUT_S = 600、EVALID_PLANNER_TIMEOUT_S=900 时为 900。

切片 2 全矩阵(180 格)用默认 600 跑;若仍有 planner 超时,再凭落盘数据决定是否调 env,不在本 PR 改线。

…211)

300s -> 600s already landed in main via PR #41; the remaining gap is
that the value is a hardcoded literal. Make it PLANNER_TIMEOUT_S,
overridable with EVALID_PLANNER_TIMEOUT_S, default 600s so the
planner+executor split inside the CAP_S 1200s cell budget is unchanged.
Guard test updated to pin the default (600s) instead of the literal.

Local: bench python tests 40/40 pass.
@surlymochan
surlymochan merged commit f5ea48a into main Nov 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant