feat: native tool calling, end to end

.changeset/native-tool-calling.md

@@ -0,0 +1,45 @@
+---
+'@ts-dspy/anthropic': minor
+'@ts-dspy/gemini': minor
+'@ts-dspy/openai': minor
+'@ts-dspy/core': minor
+---
+
+Native tool calling, end to end.
+
+All three providers reported `supportsFunctionCalling: true` while implementing
+nothing, and `RespAct` ran ReAct purely by text prompting — regex-extracting
+`Action:`/`Action Input:` from raw completions. That capped every tool at exactly
+one string argument, ruled out parallel calls, and left the loop at the mercy of
+the model formatting its output correctly. The flag is now honest.
+
+`LLMCallOptions` gains `tools` and `toolChoice`, and `ILanguageModel` gains
+`chatWithTools`, which returns text, tool calls, and a normalised finish reason
+from one turn. `BaseLM` supplies a text-only default, so the capability flag —
+not feature detection — is what callers branch on. Each provider translates the
+declarations into its own request shape (OpenAI `tools`/`tool_calls`, Anthropic
+`input_schema`/`tool_use`, Gemini `functionDeclarations`/`functionCall`) and
+reads the calls back out.
+
+`RespAct` uses that path whenever the model supports it and tools are declared,
+and keeps the text-parsing loop as the fallback for local models and providers
+without native tool calling — the same task completes either way. Tools can now
+declare a JSON Schema or Zod schema for their arguments and receive a validated
+object instead of a single string; bare functions and `{ description, function }`
+keep working unchanged. Parallel tool calls in one turn are executed and reported
+individually, and the whole `RespActEvent` surface stays meaningful on both
+paths. `forceTextMode` pins a tool-capable model to the text loop.
+
+**Breaking:** `ToolCall` is reshaped for cross-provider use. It was a copy of
+OpenAI's encoding — a required `id`, a `type: 'function'` literal, and a nested
+`function.arguments` JSON *string* — which no other provider can populate
+faithfully. It is now `{ id?, name, arguments, rawArguments? }`, where
+`arguments` is always a parsed object and `id` is optional because Gemini's
+function calls have none. The dead `ChatMessage.functionCall` field is removed;
+`ChatMessage` gains `toolCallId` to correlate a tool result with its call.
+
+That correlation also fixes a silent role collapse in all three converters:
+`tool` and `function` turns were downgraded to `user` text, and Anthropic could
+then merge a tool result into the preceding user turn. Anthropic additionally
+dropped `tool_use` blocks on the floor (`textOf` keeps only `text` blocks) and
+ignored `input_json_delta` while streaming; both are now surfaced.

README.md

@@ -430,6 +430,29 @@ const agent = new RespAct(AnswerQuestion, {
 });
 ```
 
+Give a tool a `parameters` schema — JSON Schema or Zod — and it takes named,
+validated arguments instead of one string:
+
+```ts
+const agent = new RespAct(AnswerQuestion, {
+  tools: {
+    flights: {
+      description: 'Find flights between two airports on a date.',
+      parameters: z.object({ from: z.string(), to: z.string(), date: z.string() }),
+      function: ({ from, to, date }) => search(from, to, date),
+    },
+  },
+});
+```
+
+`RespAct` picks its execution path from the model's capabilities. Against a
+provider reporting `supportsFunctionCalling: true` — all three of ours do — tools
+are declared in the request and the model's calls come back as structured data,
+so several tools can run in one turn. Against anything else, the loop falls back
+to prompting for `Action:` / `Action Input:` and parsing the reply, which works on
+any completion model. Both paths run the same tools and emit the same events; pass
+`forceTextMode: true` to pin a tool-capable model to the text loop.
+
 Tool descriptions are what the model uses to decide when to call each tool, so
 they earn the detail. Never pass model output to `eval()` — see
 [`examples/utils.ts`](examples/utils.ts) for a bounded arithmetic evaluator.

packages/anthropic/src/anthropic-lm.test.ts

@@ -312,6 +312,133 @@ describe('AnthropicLM', () => {
             await expect(consume()).rejects.toBeInstanceOf(ContentFilterError);
             expect(lm.getUsage().errorCount).toBe(1);
         });
+
+        it('forwards tool-argument deltas and the assembled calls', async () => {
+            const events = [
+                { type: 'content_block_delta', delta: { type: 'text_delta', text: 'ok' } },
+                {
+                    type: 'content_block_delta',
+                    index: 1,
+                    delta: { type: 'input_json_delta', partial_json: '{"a":' },
+                },
+                {
+                    type: 'content_block_delta',
+                    index: 1,
+                    delta: { type: 'input_json_delta', partial_json: '1}' },
+                },
+            ];
+            mocks.stream.mockReturnValue({
+                async *[Symbol.asyncIterator]() {
+                    yield* events;
+                },
+                finalMessage: async () =>
+                    message('ok', {
+                        content: [
+                            { type: 'text', text: 'ok' },
+                            { type: 'tool_use', id: 'toolu_1', name: 'add', input: { a: 1 } },
+                        ],
+                        stop_reason: 'tool_use',
+                    }),
+            });
+
+            const chunks = [];
+            for await (const chunk of new AnthropicLM({ apiKey: 'k' }).generateStream('Hi')) {
+                chunks.push(chunk);
+            }
+
+            // Argument fragments are not text, so they travel in metadata.
+            expect(chunks.filter((c) => !c.done).map((c) => c.content)).toEqual(['ok', '', '']);
+            expect(chunks[1].metadata).toEqual({
+                toolInputDelta: { index: 1, partialJson: '{"a":' },
+            });
+            expect(chunks.at(-1)?.metadata).toEqual({
+                toolCalls: [{ id: 'toolu_1', name: 'add', arguments: { a: 1 } }],
+            });
+        });
+    });
+
+    describe('tool calling', () => {
+        it('sends tool declarations as name/description/input_schema', async () => {
+            mocks.create.mockResolvedValue(message('ok'));
+
+            await new AnthropicLM({ apiKey: 'k' }).chatWithTools(
+                [{ role: 'user', content: 'hi' }],
+                {
+                    tools: [
+                        {
+                            name: 'add',
+                            description: 'Add two numbers',
+                            parameters: { type: 'object', properties: {} },
+                        },
+                    ],
+                }
+            );
+
+            expect(mocks.create.mock.calls[0][0].tools).toEqual([
+                {
+                    name: 'add',
+                    description: 'Add two numbers',
+                    input_schema: { type: 'object', properties: {} },
+                },
+            ]);
+        });
+
+        it('spells a forced tool call as tool_choice any', async () => {
+            mocks.create.mockResolvedValue(message('ok'));
+
+            await new AnthropicLM({ apiKey: 'k' }).chatWithTools(
+                [{ role: 'user', content: 'hi' }],
+                { tools: [{ name: 'add', parameters: {} }], toolChoice: 'required' }
+            );
+
+            expect(mocks.create.mock.calls[0][0].tool_choice).toEqual({ type: 'any' });
+        });
+
+        it('names a specific tool when the choice is an object', async () => {
+            mocks.create.mockResolvedValue(message('ok'));
+
+            await new AnthropicLM({ apiKey: 'k' }).chatWithTools(
+                [{ role: 'user', content: 'hi' }],
+                { tools: [{ name: 'add', parameters: {} }], toolChoice: { name: 'add' } }
+            );
+
+            expect(mocks.create.mock.calls[0][0].tool_choice).toEqual({
+                type: 'tool',
+                name: 'add',
+            });
+        });
+
+        it('surfaces tool_use blocks alongside the text', async () => {
+            mocks.create.mockResolvedValue(
+                message('ignored', {
+                    content: [
+                        { type: 'text', text: 'Let me add those.' },
+                        { type: 'tool_use', id: 'toolu_1', name: 'add', input: { a: 1, b: 2 } },
+                    ],
+                    stop_reason: 'tool_use',
+                })
+            );
+
+            const result = await new AnthropicLM({ apiKey: 'k' }).chatWithTools([
+                { role: 'user', content: 'hi' },
+            ]);
+
+            // textOf() keeps only text blocks; the tool_use block used to be
+            // silently discarded here with nothing to replace it.
+            expect(result.content).toBe('Let me add those.');
+            expect(result.finishReason).toBe('tool_calls');
+            expect(result.toolCalls).toEqual([
+                { id: 'toolu_1', name: 'add', arguments: { a: 1, b: 2 } },
+            ]);
+        });
+
+        it('omits tool parameters entirely when no tools are offered', async () => {
+            mocks.create.mockResolvedValue(message('ok'));
+            await new AnthropicLM({ apiKey: 'k' }).chat([{ role: 'user', content: 'hi' }]);
+
+            expect(mocks.create.mock.calls[0][0]).not.toHaveProperty('tools');
+            expect(mocks.create.mock.calls[0][0]).not.toHaveProperty('tool_choice');
+        });
     });
 
     describe('toAnthropicMessages', () => {
@@ -347,6 +474,83 @@ describe('AnthropicLM', () => {
             expect(input).toHaveLength(2);
         });
 
+        it('turns an assistant tool call into text and tool_use blocks', () => {
+            const { messages } = toAnthropicMessages([
+                {
+                    role: 'assistant',
+                    content: 'Let me add those.',
+                    toolCalls: [{ id: 'toolu_1', name: 'add', arguments: { a: 1, b: 2 } }],
+                },
+            ]);
+
+            expect(messages).toEqual([
+                {
+                    role: 'assistant',
+                    content: [
+                        { type: 'text', text: 'Let me add those.' },
+                        { type: 'tool_use', id: 'toolu_1', name: 'add', input: { a: 1, b: 2 } },
+                    ],
+                },
+            ]);
+        });
+
+        it('gives parallel calls to one tool distinct synthesized ids', () => {
+            // A Gemini-sourced turn carries no ids at all; two calls to the same
+            // tool must not collapse onto one tool_use_id.
+            const { messages } = toAnthropicMessages([
+                {
+                    role: 'assistant',
+                    content: '',
+                    toolCalls: [
+                        { name: 'lookup', arguments: { id: 1 } },
+                        { name: 'lookup', arguments: { id: 2 } },
+                    ],
+                },
+            ]);
+
+            const ids = (messages[0].content as any[]).map((block) => block.id);
+            expect(new Set(ids).size).toBe(2);
+        });
+
+        it('turns a tool result into a user turn holding a tool_result block', () => {
+            const { messages } = toAnthropicMessages([
+                { role: 'tool', name: 'add', toolCallId: 'toolu_1', content: '3' },
+            ]);
+
+            expect(messages).toEqual([
+                {
+                    role: 'user',
+                    content: [{ type: 'tool_result', tool_use_id: 'toolu_1', content: '3' }],
+                },
+            ]);
+        });
+
+        it('merges a tool result onto a preceding user turn as blocks, not text', () => {
+            // String merging is still the behaviour for two plain user turns
+            // (see above), but concatenating a tool result into prose would
+            // destroy the `tool_use_id` the API correlates on, so a mixed pair
+            // is promoted to block form instead.
+            const { messages } = toAnthropicMessages([
+                { role: 'user', content: 'context' },
+                { role: 'tool', name: 'add', toolCallId: 'toolu_1', content: '3' },
+            ]);
+
+            expect(messages).toEqual([
+                {
+                    role: 'user',
+                    content: [
+                        { type: 'text', text: 'context' },
+                        { type: 'tool_result', tool_use_id: 'toolu_1', content: '3' },
+                    ],
+                },
+            ]);
+        });
+
+        it('degrades an uncorrelated tool result to plain user text', () => {
+            const { messages } = toAnthropicMessages([{ role: 'tool', content: '3' }]);
+            expect(messages).toEqual([{ role: 'user', content: '3' }]);
+        });
+
         it('passes the system parameter through on a chat call', async () => {
             mocks.create.mockResolvedValue(message('ok'));
             await new AnthropicLM({ apiKey: 'k' }).chat([