[anandhu-eng]

Fixes #177. Stacked on #182.

A NaN score compares false with every bound, so _gate_accuracy reported a pass. +inf passed a lower bound. The client can write both: it writes accuracy_results.json with json.dumps, which emits a float NaN as the literal NaN, and RougeScorer writes "nan" for an empty set.

The gate now reports each non-finite metric (accuracy-gate / non-finite-scores, e.g. rouge1=nan) and does not compute the aggregate. The inline entry is still not gated.

The agentic gates already reject these values, because NaN and inf fail the [0, 1] check in _dataset_score. New tests cover that; that code is unchanged.

Tests:

• NaN, "nan", inf and -inf as a scalar score, plus a report parsed from JSON text with a bare NaN, as the client writes it.
• ROUGE "nan" strings: the message names only the non-finite metrics.
• A NaN inline score does not block the gate.
• Agentic inline NaN and inf give score-not-fraction, not a pass.

The six single-turn rejection tests fail without the fix. Full suite: 1,839 passed. Ruff and mypy pass.

🤖 Generated with Claude Code

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[Copilot]

🟢 Approval recommended

No active review issues were identified for the change to reject non-finite accuracy metrics.

No open issues 🎉

What changed in this PR

This PR closes the single-turn accuracy gate’s non-finite score loophole by rejecting NaN and infinite metrics before computing aggregate accuracy. It preserves the exclusion of inline scores from that gate.

Changes:

• Reject non-finite accuracy metrics and report the affected datasets and metrics.
• Add a diagnostic message with rescoring guidance.
• Add regression tests for single-turn rejection, inline-score exclusion, and existing agentic rejection behavior.

File	Description
tests/​submission_checker/​test_checks_aggregate.py	Tests non-finite scalar and ROUGE scores, JSON NaN, and inline-score exclusion.
tests/​submission_checker/​test_checks_agentic_accuracy.py	Tests existing agentic rejection of NaN and infinity.
src/​submission_checker/​models/​aggregate/​context.py	Rejects non-finite metrics before aggregate calculation.
src/​submission_checker/​data/​messages.yaml	Adds the non-finite score error and rescoring guidance.

_{Review effort: Balanced}