Support Anthropic's native server-side tools in AnthropicChatGenerator#3555
Support Anthropic's native server-side tools in AnthropicChatGenerator#3555ErenAta16 wants to merge 13 commits into
Conversation
Anthropic offers server-side tools like web_search, code_execution, and
bash that are defined as plain dicts (e.g. {"type": "web_search_20250305",
"name": "web_search"}) rather than Haystack Tool objects, since Anthropic
runs them itself instead of calling back into user code. There was no way
to pass these through AnthropicChatGenerator: the tools parameter only
accepted Tool/Toolset instances, and passing a raw dict tripped the
Tool/Toolset flattening and duplicate-name check.
OpenAIResponsesChatGenerator already solves the equivalent problem for
OpenAI's own native tools by accepting list[dict] alongside its normal
ToolsType and branching on whether the first element is a dict. This
mirrors that approach for Anthropic: __init__, to_dict/from_dict, and
_prepare_request_params (used by both run and run_async) now detect a
list of dicts and pass it straight through to the API unchanged, instead
of routing it through the Haystack tool machinery.
Covers initialization, serialization round-trip, and that run() passes native tool dicts through to the Anthropic client unchanged, plus a regression test confirming native tools skip the duplicate-name check that only makes sense for Haystack Tool objects.
|
|
github-actions
Bot
added
integration:anthropic
type:documentation
|
Heads-up for maintainers This PR is from a fork and touches integrations whose integration tests require API keys. Affected integrations:
Please run the integration tests locally ( |
The Anthropic SDK's messages.create() accepts a narrower tools type than the list[ToolParam] | list[dict] union we now support, so mypy flags both call sites even though the native tool dicts are already in the exact shape the API expects.
AnthropicVertexChatGenerator and AnthropicFoundryChatGenerator each reimplement __init__, to_dict, and from_dict rather than delegating to the base class, so they still assumed tools was always a Haystack Tool/Toolset. Since both classes inherit the base class's tools attribute type, mypy was flagging the mismatch in their own serialization code and in Foundry's run/run_async overrides. Mirror the same guard used in AnthropicChatGenerator: skip Haystack's tool validation and serialization when tools is a plain list of dicts, and widen the run/run_async signatures on Foundry to match the base class.
Coverage report (anthropic)
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||
|
Could you please update the comment and fill in our PR template to help the review? Also, please sign the CLA and then ping me again for a review |
Updated the description to follow the PR template. The CLA is signed. Ready for another look whenever you have time — thanks! |
|
There's a bug on the from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
gen = AnthropicChatGenerator(tools=[])
d = gen.to_dict()
AnthropicChatGenerator.from_dict(d)
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Cell In[5], line 1
----> 1 AnthropicChatGenerator.from_dict(d)
File ~/haystack-core-integrations/integrations/anthropic/src/haystack_integrations/components/generators/anthropic/chat/chat_generator.py:253, in AnthropicChatGenerator.from_dict(cls, data)
249 tools = data["init_parameters"].get("tools")
250 is_haystack_toolset = isinstance(tools, dict) and tools.get("type") == "haystack.tools.toolset.Toolset"
251 is_haystack_tool_list = (
252 isinstance(tools, list)
--> 253 and isinstance(tools[0], dict)
254 and tools[0].get("type") == "haystack.tools.tool.Tool"
255 )
256 if tools and (is_haystack_toolset or is_haystack_tool_list):
257 deserialize_tools_or_toolset_inplace(data["init_parameters"], key="tools")
IndexError: list index out of range
|
AnthropicChatGenerator(tools=[]).to_dict() serializes the tools
parameter as [], since serialize_tools_or_toolset([]) returns [] rather
than None. Feeding that back into from_dict() raised IndexError: list
index out of range, because the Haystack-tool-list check indexed into
tools[0] before confirming the list was non-empty:
is_haystack_tool_list = (
isinstance(tools, list)
and isinstance(tools[0], dict)
...
)
The outer `if tools and (...)` guard at the call site did not help,
since is_haystack_tool_list was already evaluated eagerly above it.
The same pattern (a list/dict shape check used to distinguish Anthropic's
native, server-side tool dicts from Haystack Tool/Toolset objects) was
duplicated across chat_generator.py, foundry_chat_generator.py, and
vertex_chat_generator.py, in __init__, to_dict, and from_dict. Moved it
into two shared helpers in utils.py:
- _is_native_tools_list(tools): the existing, already-correct
isinstance(tools, list) and tools and isinstance(tools[0], dict) check,
used in __init__, to_dict, and _prepare_request_params.
- _should_deserialize_tools_or_toolset(tools): the from_dict check,
now guarding the list-emptiness before indexing.
All three generators import and use both helpers instead of repeating
the logic inline, so a future change to how native tools are detected
only needs to happen in one place.
Added a regression test (test_serde_with_empty_tools_list) that
round-trips a generator initialized with tools=[] through to_dict/
from_dict and confirms it no longer raises. Verified it fails against
the previous implementation and passes against this change, and that
the existing native-tools serde/init/duplicate-name tests are unaffected.
|
Good catch, thanks. Confirmed the exact issue: Pushed a fix. Also did the refactor you suggested: moved the native-tools-list check and the deserialize-or-not check into two shared helpers in Added Let me know if you'd rather this be two separate commits (bugfix, then refactor) instead of one — happy to split it if that's easier to review. |
The docstring's summary wrapped onto a second physical line before the blank line separating it from the description, which fails this project's D205 rule (summary and description must be separated by exactly one blank line). Put the summary on a single line instead. Confirmed with the project's own ruff config (pyproject.toml select list, line-length 120): ruff check --select D205 and ruff format --check both pass on the changed file after this fix, matching what CI's fmt-check step runs.
Moving the isinstance(tools, list) and tools and isinstance(tools[0], dict) check out of chat_generator.py and into a plain bool-returning helper lost mypy's type narrowing at the call site in _prepare_request_params: `anthropic_tools = tools` inside the `if _is_native_tools_list(tools):` branch no longer narrows `tools` from its declared ToolsType | list[dict] | None to list[dict], so mypy flagged an incompatible assignment. Annotated the helper's return type as TypeGuard[list[dict[str, Any]]] instead of bool, which restores narrowing for mypy while keeping the single shared implementation. Confirmed with mypy -p haystack_ integrations.components.generators.anthropic (as run by this project's test:types script): no errors in 7 source files. Also re-ran ruff check/format and the full non-integration test suite to confirm nothing else regressed.
|
CI is green now (the two intermediate failures were a lint docstring formatting issue and a mypy type-narrowing regression from the refactor, both fixed in the follow-up commits). |
3 participants
Related Issues
Proposed Changes:
Anthropic offers server-side tools like
web_search,code_execution, andbashthat are defined as plain dicts (e.g.{"type": "web_search_20250305", "name": "web_search"}) rather than HaystackToolobjects, since Anthropic runs them itself instead of calling back into user code. There was no way to pass these throughAnthropicChatGenerator: thetoolsparameter only acceptedTool/Toolsetinstances, and passing a raw dict tripped the Tool/Toolset flattening and duplicate-name check withTypeError: Items in the tools list must be Tool or Toolset instances.OpenAIResponsesChatGeneratoralready solves the equivalent problem for OpenAI's own native tools by acceptinglist[dict]alongside its normalToolsTypeand branching on whether the first element is a dict. This mirrors that approach for Anthropic:__init__,to_dict/from_dict, and_prepare_request_params(used by bothrunandrun_async) now detect a list of dicts and pass it straight through to the Anthropic API unchanged, instead of routing it through the Haystack tool machinery.AnthropicVertexChatGeneratorandAnthropicFoundryChatGeneratormirror the same handling in their own__init__/to_dict/from_dictandrun/run_asyncoverrides, since they don't delegate to the base class for these.Usage:
How did you test it?
run()forwards native tool dicts unchanged, and that native tools correctly skip the duplicate-name check (which only makes sense for HaystackToolobjects with a.name).ruff check/ruff format --checkare clean, andmypyreports no new errors (the two pre-existinganthropic.Anthropic/AsyncAnthropicattr-defined errors are unrelated to this change and present onmaintoo).Notes for the reviewer
The Vertex and Foundry generators subclass
AnthropicChatGeneratorbut reimplement their own__init__/to_dict/from_dictinstead of delegating to it, so they needed the same native-tools guard applied separately. That part is in the second and third commits.Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:.