[mking-keeper]

Hello!

I'm Matt, CTO at Keeper (a YC startup). Big fan of stirrup and how it's making benchmarking easier for everyone! I saw you are using litellm, and given their recent security incidents I wanted to try and provide an alternative with a different llm proxy library.

Motivation

• In mar and oct 2026, litellm was compromised by hackers due to supply chain issues
• A few alternatives to litellm are available
  • Mozilla maintains any-llm
  • Andrew Ng maintains aisuite

AnyLLMClient

• This PR adds AnyLLMClient (equivalent to LiteLLMClient)
  • We add an anthropic adapter code to retain thinking blocks
  • We add a gemini adapter code to retain tool call signatures
  • For openai and anthropic, I set max_retries=0. That way, we don't have layered retries when calling the wrapped sdks

LiteLLMClient

• For backwards compatibility, LiteLLMClient code is not modified. We may want to remove it in the future

Dependencies

Gemini sdk version

• Issue: Browser use requires google-genai<2.0.0. However, anyllm 1.40.1 requires google-genai>=2.17.0.
• As a workaround, I pinned any-llm-sdk[gemini]>=1.27.0,<1.29.0

Openai, anthropic

• I updated openai, anthropic versions (see uv.lock) to be compatible with anyllm

Testing

• I did a smoke test with each of the following keys using AnyLLMClient (session.run(prompt)):
  • ANTHROPIC_API_KEY (claude-sonnet-4-6, with thinking)
  • GEMINI_API_KEY (gemini-3-flash-preview)
  • OPENAI_API_KEY (gpt-4.1-mini)
  • OPENROUTER_API_KEY (deepseek-v4.1-flash-1017)