to_openeval()/from_openeval(): portable eval dataset conversion for ResultMessage / AssistantMessage

[adhabnr-ux]

Hi maintainers,

I maintain EvalPort (https://github.com/adhabnr-ux/evalport), an open, provider-neutral JSON Schema spec for portable LLM eval test suites and result sets, so eval data isn't locked to one framework's format. Spec: https://github.com/adhabnr-ux/evalport/blob/main/SPEC.md

For context on why I'm raising this: openai/openai-python#3619 (openai/openai-python#3619 — open, not yet merged) has an OpenAI contributor building native to_openeval() / from_openeval() helpers directly onto that SDK's dataset/response types. I think the same pattern would fit claude-agent-sdk-python well, given how much of what an eval result needs is already sitting on your message types.

From src/claude_agent_sdk/types.py: AssistantMessage.content is a list[ContentBlock] (TextBlock, ToolUseBlock, ToolResultBlock, ...), and ResultMessage already carries result, structured_output, total_cost_usd, usage, num_turns, and session_id. A conversion helper could look roughly like:

# sketch, not a PR
def to_openeval_result(
    messages: list[AssistantMessage | UserMessage],
    result: ResultMessage,
    *,
    test_case_id: str,
) -> dict:
    final_text = "".join(
        block.text
        for msg in messages if isinstance(msg, AssistantMessage)
        for block in msg.content if isinstance(block, TextBlock)
    )
    return {
        "test_case_id": test_case_id,
        "actual_output": final_text or (result.result or ""),
        "metadata": {
            "num_turns": result.num_turns,
            "total_cost_usd": result.total_cost_usd,
            "session_id": result.session_id,
        },
    }

which lines up with an EvalPort ResultSet entry: https://github.com/adhabnr-ux/evalport/blob/main/spec/schemas/resultset.json (test_case_id, actual_output, grader_results, passed).

No pressure if this isn't a priority right now — just wanted it in front of you given the precedent forming in openai-python. Happy to sketch a fuller PR if there's interest.