[anandhu-eng]

Fixes #178. Stacked on #183.

metric_scores() skips any metric it cannot convert to a number. So "score": {"exact_match": null}, a null metric on a legacy entry with no score key, or a scalar like "N/A" dropped that dataset out of the mean with no error. Only a top-level score: null was reported.

_gate_accuracy now reports each such metric (accuracy-gate / unreadable-metrics, e.g. livecodebench::gptoss (exact_match=null)) and does not compute the aggregate.

• Only gated metrics count: the model's threshold metrics, or the scalar score. A null field that is not gated is ignored, since a legacy entry's extra fields may be bookkeeping.
• The inline entry is still not gated.
• The new AccuracyResult.unreadable_metrics() uses the same metric lookup as metric_scores(), so the two cannot disagree on which fields are metrics.
• The agentic gates read a scalar score, which fix: reject a null SWE-bench score in a mandatory band #181 (null) and fix: reject non-finite accuracy scores in the single-turn gate #183 (non-finite) already cover.

Tests:

• A null named metric (both report formats), a null metric on a legacy entry, and a non-numeric scalar each give one error naming the metric.
• A null ungated metric and a null legacy bookkeeping field are ignored.
• A null named inline metric does not block the gate.
• A unit test for unreadable_metrics().

The four rejection tests fail without the gate change. Full suite: 1,848 passed. Ruff and mypy pass.

🤖 Generated with Claude Code

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[Copilot]

🟢 Approval recommended

No active review issues block the change to reject unreadable gated accuracy metrics before aggregation.

No open issues 🎉

What changed in this PR

Addresses #178 by rejecting null or non-numeric gated accuracy metrics before computing an aggregate, preventing datasets with unreadable scores from silently dropping out of the mean.

Changes:

• Shares metric lookup between numeric score extraction and unreadable-metric detection.
• Adds an accuracy-gate error for unreadable gated metrics while preserving exclusions for inline entries and ungated fields.
• Adds regression tests for named, scalar, and legacy metric formats and the intended exclusions.

File	Description
tests/​submission_checker/​test_checks_file.py	Tests unreadable-metric detection alongside numeric score extraction.
tests/​submission_checker/​test_checks_aggregate.py	Tests gate rejection and exclusions across supported report formats.
src/​submission_checker/​models/​file/​accuracy.py	Adds unreadable-metric detection using shared metric lookup.
src/​submission_checker/​models/​aggregate/​context.py	Blocks aggregation when gated metrics are unreadable.
src/​submission_checker/​data/​messages.yaml	Adds the unreadable-metric error and remediation guidance.

_{Review effort: Balanced}