EvalResult.scores: dict invariance rejects dict[str, float] in constructor

[willfrey]

Problem

EvalResult.scores is typed as dict[str, float | None] (framework.py:104). Because dict is invariant in both key and value types, constructing an EvalResult with dict[str, float] (no None values) is rejected by type checkers:

# Type error: dict[str, float] is not assignable to dict[str, float | None]
result = EvalResult(input=..., output=..., scores={"accuracy": 0.95})

This is the same invariance class as #280 (EvalParameters) and the Metadata alias.

Why this one is different

EvalResult is a dataclass, not a function parameter. The type annotation serves as both the constructor parameter type and the attribute type. Simply changing to Mapping[str, float | None] would:

• Fix the constructor, but
• Change the attribute type to read-only, which could break internal code that mutates scores after construction

Options

1. Accept Mapping in a custom __init__, store as dict — most correct, but adds boilerplate to a dataclass
2. Change to Mapping[str, float | None] — if the framework never mutates scores after construction (needs verification)
3. Document as output-only — if callers are not expected to construct EvalResult directly

Context

Related PRs: #281 (EvalParameters → Mapping), #285 (Metadata/ParametersSchema → Mapping)

[AbhiPrasad]

Let’s merge in #282 and then we can take a look at this

[AbhiPrasad]

released with https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.18.0