Repository navigation
feat(bench): per-cell cost field + uncensored wall-clock protocol (M16 slice 1, #205) - #206
Merged
Merged
Conversation
…6 slice 1, #205) - run.py records cost_usd + cost_price_row on every cell, priced at the new prices.json minimax-m31-flash row (MiniMax-M3.1-Flash list prices, fetched 2026-11-15; closest published PAYG price for the bench model MiniMax-M3.1-Flash-Preview). Computed, not billed (Token Plan). - report.py prefers the recorded cost, falls back to recomputation for older rows; default --price is now minimax-m31-flash (M3 row retained for comparability with pilot2 medians). - Wall clock: keep the 1200s cap (see #205 for the evidence); report.py now reports the median over uncensored cells only, with n, alongside the truncated count. Median over all cells is a cap artifact. - test_cost_field.py: formula, price row, report preference, uncensored wall stats (stdlib only, 5 tests).
This was referenced Nov 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
M16 切片 1(#205):跑批前测量补齐
本 PR 只做测量与口径落地,不跑全矩阵(全矩阵 180 格是切片 2)。
$/task 成本字段
run.py:每格落盘新增cost_usd+cost_price_row(按 token 用量 × MiniMax 价目折算),补 P1-2 的 $/task 腿(上轮总账该腿「无成本字段未测」)。prices.json行minimax-m31-flash(input $0.15 / output $0.60 / cache_read $0.015 per 1M tokens,来源 https://platform.minimax.io/docs/pricing/overview,2026-11-15 实读)。定价页现直接列出 MiniMax-M3.1-Flash,取代旧 M3 Standard($0.30/$1.20/$0.06)代用口径;旧行保留以便复现 pilot2 中位价。Preview 本身无 PAYG 价、走 M Plan / MiniMax Code,故本字段为 computed、not billed(实际走 Token Plan)。report.py:该行优先用落盘值、旧行无字段时回退重算(同一公式);默认--price改为minimax-m31-flash。墙钟截尾口径:选 B——保持 1200s 上限,截尾格单独计数+以未截尾格计中位
理由(135 格实测,非推测):
report.py新增每臂未截尾格中位+n(summary.jsonwall_s_uncensored_*、tables.md 新行);中位比值检验仅在两臂截尾率均 <50% 时可算,否则墙钟腿记「无法判定」、附截尾率与未截尾中位作证据,切片 3 再给下一刀。判定线本身不变。openswarm 臂
验证
test_cost_field.py(stdlib):公式、价目行、report 优先落盘值、未截尾墙钟统计——5/5 通过;旧test_planner_prompt.py7/7、test_diff_filter.py7/7。