Skip to content

feat: add AG2 target for evaluating AG2/AutoGen agents#120

Open
faridun-ag2 wants to merge 1 commit intoawslabs:mainfrom
faridun-ag2:feat/add-ag2-target
Open

feat: add AG2 target for evaluating AG2/AutoGen agents#120
faridun-ag2 wants to merge 1 commit intoawslabs:mainfrom
faridun-ag2:feat/add-ag2-target

Conversation

@faridun-ag2
Copy link
Copy Markdown

Summary

  • Adds AG2 (formerly AutoGen) as a new target type, enabling evaluation of AG2-based agents through the agent-evaluation framework
  • First non-AWS target, demonstrating the framework's extensibility
  • AG2 is an open-source multi-agent framework with 500K+ monthly PyPI downloads and 4,300+ GitHub stars

What's included

  • src/agenteval/targets/ag2/ — AG2Target implementation extending BaseTarget
  • Unit tests with full mocking of AG2 components (6 tests)
  • Documentation at docs/targets/ag2.md
  • AG2 dependency added to requirements.txt (ag2[openai]>=0.11.4,<1.0)
  • CHANGELOG.md updated under Unreleased

Configuration Example

target:
  type: ag2
  model: gpt-4o-mini
  api_type: openai
  system_message: "You are a helpful assistant."
  max_turns: 3

Test Plan

  • black formatting passes
  • isort passes
  • flake8 passes
  • All unit tests pass (pytest tests/src/agenteval/targets/ag2/)
  • Full test suite passes (pytest tests/ — 80/80)
  • AG2Target.invoke() returns TargetResponse correctly
  • Agent reset between invocations works
  • Documentation follows existing target docs pattern
  • CHANGELOG.md updated
  • No existing targets/tests modified

Add AG2 (formerly AutoGen) as a new target type, enabling evaluation of
AG2-based agents through the agent-evaluation framework. This is the first
non-AWS target, demonstrating the framework's extensibility.

- AG2Target implementation extending BaseTarget
- Unit tests with full mocking of AG2 components
- Documentation at docs/targets/ag2.md
- AG2 dependency added to requirements.txt
- CHANGELOG.md updated
@faridun-ag2
Copy link
Copy Markdown
Author

Hi @bobbywlindsey @sharonxiaohanli — would appreciate your review on this PR. It adds AG2 (formerly AutoGen) as the first non-AWS target, demonstrating the framework's extensibility. Happy to address any feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant