Skip to content

Update: card metrics query#420

Merged
ibraheem-abe merged 6 commits into
stagingfrom
update/evaluation-query-FE
Jun 23, 2026
Merged

Update: card metrics query#420
ibraheem-abe merged 6 commits into
stagingfrom
update/evaluation-query-FE

Conversation

@ibraheem-abe

@ibraheem-abe ibraheem-abe commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR updates the evaluation detail metrics to only take into account non disqualified agents.

It also updates the GET /evaluation-sets/{ID} endpoint to include the agent_id for the lowest average cost and lowest runtime:

...
  "efficiency": {
    "lowest_average_cost_usd_top_agents": {
      "agent_id": "c9d5efbc-99c3-58a0-9b00-764f0fc2c9f9",
      "value": 0.0368
    },
    "lowest_average_runtime_seconds_top_agents": {
      "agent_id": "b189c272-78ab-5de1-a169-03bb8788fa42",
      "value": 338.5569
    },
    "average_agent_cost_usd": 0.0947,
    "average_agent_runtime_seconds": 563.4667
  }
...

The GET /evaluation-sets/{ID}/leaderboard endpoint was updated so that cancelled agents do not have a rank and their final score reflects the projected maximum score.

@jmnmv12 jmnmv12 requested a review from camfairchild June 19, 2026 15:56
@ibraheem-abe ibraheem-abe changed the base branch from main to staging June 23, 2026 20:08
@ibraheem-abe ibraheem-abe merged commit 0b114ee into staging Jun 23, 2026
3 checks passed
@ibraheem-abe ibraheem-abe deleted the update/evaluation-query-FE branch June 23, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants