Skip to content

Python: move Foundry eval serialization out of core #8031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Eduard van Valkenburg (eavanvalkenburg) merged 6 commits into from
Sep 4, 2026
Merged

Python: move Foundry eval serialization out of core #8031

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
db135ab
Python: move Foundry eval serialization out of core
eavanvalkenburg Sep 3, 2026
64de3c8
Python: mark eval item helper experimental
eavanvalkenburg Sep 3, 2026
cbf67b7
Python: normalize tools for eval items
eavanvalkenburg Sep 3, 2026
2394401
Merge remote-tracking branch 'upstream/main' into foundry-eval-ownership
eavanvalkenburg Sep 3, 2026
977ebfc
Python: address Foundry eval review feedback
eavanvalkenburg Sep 3, 2026
5cb3206
Python: deprecate AgentEvalConverter compatibly
eavanvalkenburg Sep 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions python/packages/core/AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,17 @@ agent_framework/
hosts remain responsible for authorizing and tenant-scoping access to any shared checkpoint adapter.
- **Orchestrators**: `SequentialOrchestrator`, `ConcurrentOrchestrator`, `GroupChatOrchestrator`, `MagenticOrchestrator`, `HandoffOrchestrator`

## Evaluation (`_evaluation.py`)

- Core owns provider-neutral evaluation types, local evaluators and checks, `EvalItem` construction, and the
`evaluate_agent` / `evaluate_workflow` orchestration functions.
- Provider packages own their service-specific evaluator implementations and wire serialization. Core evaluation
code must not emit a provider's request schema. The deprecated `AgentEvalConverter` remains as a temporary
compatibility shim for released Foundry packages whose declared core range still imports it; new code must not use
the shim.
- Core orchestration builds `EvalItem` instances through private helpers; callers needing manual control construct
the public `EvalItem` directly.

## Built-in Providers

### OpenAI (`openai/`)
Expand Down
Loading
Loading