From 2711378a33c40d937d838120a5adeac56d37f437 Mon Sep 17 00:00:00 2001 From: Sebastian Husch Lee Date: Thu, 9 Jul 2026 07:49:12 +0200 Subject: [PATCH 1/3] Update tutorial 40 to no longer use tool invoker --- ...at_Application_with_Function_Calling.ipynb | 171 ++---------------- 1 file changed, 14 insertions(+), 157 deletions(-) diff --git a/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb b/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb index c0c05c85..20ca9f66 100644 --- a/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb +++ b/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb @@ -5,35 +5,14 @@ "metadata": { "id": "HJsHqaTksDHt" }, - "source": [ - "# Tutorial: Building a Chat Agent with Function Calling\n", - "\n", - "- **Level**: Advanced\n", - "- **Time to complete**: 20 minutes\n", - "- **Components Used**: [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder), [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder), [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [ToolInvoker](https://docs.haystack.deepset.ai/docs/toolinvoker)\n", - "- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys) and be familiar with [creating pipelines](https://docs.haystack.deepset.ai/docs/creating-pipelines)\n", - "- **Goal**: After completing this tutorial, you will have learned how to build chat applications that demonstrate agent-like behavior using OpenAI's function calling feature.\n" - ] + "source": "# Tutorial: Building a Chat Agent with Function Calling\n\n- **Level**: Advanced\n- **Time to complete**: 20 minutes\n- **Components Used**: [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder), [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder), [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [Agent](https://docs.haystack.deepset.ai/docs/agent)\n- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys) and be familiar with [creating pipelines](https://docs.haystack.deepset.ai/docs/creating-pipelines)\n- **Goal**: After completing this tutorial, you will have learned how to build chat applications that demonstrate agent-like behavior using OpenAI's function calling feature." }, { "cell_type": "markdown", "metadata": { "id": "PWXXqq_MPn7y" }, - "source": [ - "## Overview\n", - "\n", - "\n", - "\n", - "📚 Useful Sources:\n", - "* [OpenAIChatGenerator Docs](https://docs.haystack.deepset.ai/docs/openaichatgenerator)\n", - "* [OpenAIChatGenerator API Reference](https://docs.haystack.deepset.ai/reference/generators-api#openaichatgenerator)\n", - "* [🧑‍🍳 Cookbook: Function Calling with OpenAIChatGenerator](https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/function_calling_with_OpenAIChatGenerator.ipynb)\n", - "\n", - "[OpenAI's function calling](https://platform.openai.com/docs/guides/function-calling) connects large language models to external tools. By providing a `tools` list with functions and their specifications to the OpenAI API calls, you can easily build chat assistants that can answer questions by calling external APIs or extract structured information from text.\n", - "\n", - "In this tutorial, you'll learn how to convert your Haystack pipeline into a function-calling tool and how to implement applications using OpenAI's Chat Completion API through `OpenAIChatGenerator` for agent-like behavior." - ] + "source": "## Overview\n\n\n\n📚 Useful Sources:\n* [OpenAIChatGenerator Docs](https://docs.haystack.deepset.ai/docs/openaichatgenerator)\n* [OpenAIChatGenerator API Reference](https://docs.haystack.deepset.ai/reference/generators-api#openaichatgenerator)\n* [Agent Docs](https://docs.haystack.deepset.ai/docs/agent)\n* [🧑‍🍳 Cookbook: Function Calling with OpenAIChatGenerator](https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/function_calling_with_OpenAIChatGenerator.ipynb)\n\n[OpenAI's function calling](https://platform.openai.com/docs/guides/function-calling) connects large language models to external tools. By providing a `tools` list with functions and their specifications to the OpenAI API calls, you can easily build chat assistants that can answer questions by calling external APIs or extract structured information from text.\n\nIn this tutorial, you'll learn how to convert a Haystack pipeline into a function-calling tool and how to implement chat applications that show agent-like behavior. You'll first see the raw tool-calling mechanics through `OpenAIChatGenerator`, and then use Haystack's [`Agent`](https://docs.haystack.deepset.ai/docs/agent) component, which automatically runs the tool-calling loop for you." }, { "cell_type": "markdown", @@ -606,13 +585,11 @@ "metadata": { "id": "_ulFfHcnGfsw" }, - "source": [ - "Next, use [ToolInvoker](https://docs.haystack.deepset.ai/docs/toolinvoker) to process `ChatMessage` object containing tool calls, invoke the corresponding tools and return the results as a list of `ChatMessage`. " - ] + "source": "## Letting the Agent Run the Tool-Calling Loop\n\nAs you saw above, the OpenAI Chat Completions API doesn't call the tool itself. It only tells you *which* tool to call and with *which* arguments. To get a final answer, you'd have to execute the tool yourself, append the result to the message list, and call the generator again, possibly several times, until the model stops requesting tools.\n\nInstead of writing this loop by hand, use Haystack's [`Agent`](https://docs.haystack.deepset.ai/docs/agent) component. The `Agent` wraps a `ChatGenerator` together with a list of `tools` and automatically runs the full tool-calling loop for you: it calls the LLM, invokes any requested tools, feeds the results back to the LLM, and repeats until the model returns a plain text answer.\n\nCreate an `Agent` with the two tools you defined and run it on the same query:" }, { "cell_type": "code", - "execution_count": 19, + "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -634,51 +611,19 @@ "id": "sijMc9fhR7ej", "outputId": "bd126d0f-3204-4089-a786-f0b233621149" }, - "outputs": [ - { - "data": { - "application/vnd.jupyter.widget-view+json": { - "model_id": "466461528ca1456481e860f6202df70c", - "version_major": 2, - "version_minor": 0 - }, - "text/plain": [ - "Batches: 0%| | 0/1 [00:00, _content=[ToolCallResult(result=\"{'reply': 'Mark lives in Berlin.'}\", origin=ToolCall(tool_name='rag_pipeline_tool', arguments={'query': 'Where does Mark live?'}, id='call_kbdd34iZhkVAdIj2ZThTbZdV'), error=False)], _name=None, _meta={})]\n" - ] - } - ], - "source": [ - "from haystack.components.tools import ToolInvoker\n", - "\n", - "tool_invoker = ToolInvoker(tools=[rag_pipeline_tool, weather_tool])\n", - "\n", - "if response[\"replies\"][0].tool_calls:\n", - " tool_result_messages = tool_invoker.run(messages=response[\"replies\"])[\"tool_messages\"]\n", - " print(f\"tool result messages: {tool_result_messages}\")" - ] + "outputs": [], + "source": "from haystack.components.agents import Agent\nfrom haystack.components.generators.chat import OpenAIChatGenerator\nfrom haystack.dataclasses import ChatMessage\n\nagent = Agent(\n chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n tools=[rag_pipeline_tool, weather_tool],\n system_prompt=\"Use the tools that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\",\n)\n\nresult = agent.run(messages=[ChatMessage.from_user(\"Can you tell me where Mark lives?\")])\nprint(result[\"messages\"][-1].text)" }, { "cell_type": "markdown", "metadata": { "id": "I6zam1cfbuZv" }, - "source": [ - "As the last step, run the `OpenAIChatGenerator` again with the tool call results and get the final answer." - ] + "source": "Besides the final answer, the `Agent` returns the full list of messages exchanged during the run under the `\"messages\"` key. This lets you see the tool call and the tool result that the `Agent` handled for you under the hood, exactly the steps you would have had to orchestrate manually:" }, { "cell_type": "code", - "execution_count": 22, + "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -686,42 +631,15 @@ "id": "TdwkbMPLPO_L", "outputId": "0ee96a46-16f3-4a03-ce54-a69bedde24ae" }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Mark lives in Berlin." - ] - } - ], - "source": [ - "# Pass all the messages to the ChatGenerator with the correct order\n", - "messages = user_messages + response[\"replies\"] + tool_result_messages\n", - "final_replies = chat_generator.run(messages=messages, tools=[rag_pipeline_tool, weather_tool])[\"replies\"]" - ] + "outputs": [], + "source": "for message in result[\"messages\"]:\n print(message)\n print(\"-\" * 10)" }, { "cell_type": "markdown", "metadata": { "id": "5UkjGYEfwPzS" }, - "source": [ - "## Building the Chat Agent\n", - "\n", - "As you notice above, OpenAI Chat Completions API does not call the tool; instead, the model generates JSON that you can use to call the tool in your code. That's why, to build an end-to-end chat agent, you need to check if the OpenAI response is a `tool_calls` for every message. If so, you need to call the corresponding tool with the provided arguments and send the tool response back to OpenAI. Otherwise, append both user and messages to the `messages` list to have a regular conversation with the model. Let's build an application that handles all cases.\n", - "\n", - "To build a nice UI for your application, you can use [Gradio](https://www.gradio.app/) that comes with a chat interface. Install `gradio`, run the code cell below and use the input box to interact with the chat application that has access to two tools you've created above. \n", - "\n", - "Example queries you can try:\n", - "* \"***What is the capital of Sweden?***\": A basic query without any tool calls\n", - "* \"***Can you tell me where Giorgio lives?***\": A basic query with one tool call\n", - "* \"***What's the weather like in Berlin?***\", \"***Is it sunny there?***\": To see the messages are being recorded and sent\n", - "* \"***What's the weather like where Jean lives?***\": To force two tool calls\n", - "* \"***What's the weather like today?***\": To force OpenAI to ask more clarification\n", - "\n", - "> Keep in mind that OpenAI models can sometimes hallucinate answers or tools and might not work as expected." - ] + "source": "## Building the Chat Application\n\nNow let's turn the `Agent` into an interactive, multi-turn chat application. Because the `Agent` already handles the tool-calling loop internally, all you need to do to hold a conversation is keep track of the message history: append each new user message, run the `Agent`, and carry the returned `messages` over to the next turn so the model remembers the context.\n\nTo build a nice UI for your application, you can use [Gradio](https://www.gradio.app/) that comes with a chat interface. Install `gradio`, run the code cell below and use the input box to interact with the chat application that has access to the two tools you've created above.\n\nExample queries you can try:\n* \"***What is the capital of Sweden?***\": A basic query without any tool calls\n* \"***Can you tell me where Giorgio lives?***\": A basic query with one tool call\n* \"***What's the weather like in Berlin?***\", \"***Is it sunny there?***\": To see the messages are being recorded and sent\n* \"***What's the weather like where Jean lives?***\": To force two tool calls\n* \"***What's the weather like today?***\": To force OpenAI to ask more clarification\n\n> Keep in mind that OpenAI models can sometimes hallucinate answers or tools and might not work as expected." }, { "cell_type": "code", @@ -743,75 +661,14 @@ "id": "sK_JeKZLhXcy" }, "outputs": [], - "source": [ - "import gradio as gr\n", - "\n", - "from haystack.dataclasses import ChatMessage\n", - "from haystack.components.generators.chat import OpenAIChatGenerator\n", - "from haystack.components.tools import ToolInvoker\n", - "\n", - "tool_invoker = ToolInvoker(tools=[rag_pipeline_tool, weather_tool])\n", - "chat_generator = OpenAIChatGenerator(model=\"gpt-4o-mini\", tools=[rag_pipeline_tool, weather_tool])\n", - "response = None\n", - "messages = [\n", - " ChatMessage.from_system(\n", - " \"Use the tool that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\"\n", - " )\n", - "]\n", - "\n", - "\n", - "def chatbot_with_tc(message, history):\n", - " global messages\n", - " messages.append(ChatMessage.from_user(message))\n", - " response = chat_generator.run(messages=messages)\n", - "\n", - " while True:\n", - " # if OpenAI response is a function call\n", - " if response and response[\"replies\"][0].tool_calls:\n", - " tool_result_messages = tool_invoker.run(messages=response[\"replies\"])[\"tool_messages\"]\n", - " messages = messages + response[\"replies\"] + tool_result_messages\n", - " response = chat_generator.run(messages=messages)\n", - "\n", - " # Regular Conversation\n", - " else:\n", - " messages.append(response[\"replies\"][0])\n", - " break\n", - " return response[\"replies\"][0].text\n", - "\n", - "\n", - "demo = gr.ChatInterface(\n", - " fn=chatbot_with_tc,\n", - " examples=[\n", - " \"Can you tell me where Giorgio lives?\",\n", - " \"What's the weather like in Madrid?\",\n", - " \"Who lives in London?\",\n", - " \"What's the weather like where Mark lives?\",\n", - " ],\n", - " title=\"Ask me about weather or where people live!\",\n", - ")\n", - "\n", - "## Uncomment the line below to launch the chat app with UI\n", - "# demo.launch()" - ] + "source": "import gradio as gr\n\nfrom haystack.components.agents import Agent\nfrom haystack.components.generators.chat import OpenAIChatGenerator\nfrom haystack.dataclasses import ChatMessage\n\nagent = Agent(\n chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n tools=[rag_pipeline_tool, weather_tool],\n system_prompt=\"Use the tools that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\",\n)\nagent.warm_up()\n\nmessages = [ChatMessage.from_system(agent.system_prompt)]\n\n\ndef chatbot_with_tc(message, history):\n global messages\n messages.append(ChatMessage.from_user(message))\n # The Agent runs the full tool-calling loop and returns all messages exchanged during the run\n result = agent.run(messages=messages)\n # Carry the updated conversation history over to the next turn\n messages = result[\"messages\"]\n return result[\"messages\"][-1].text\n\n\ndemo = gr.ChatInterface(\n fn=chatbot_with_tc,\n examples=[\n \"Can you tell me where Giorgio lives?\",\n \"What's the weather like in Madrid?\",\n \"Who lives in London?\",\n \"What's the weather like where Mark lives?\",\n ],\n title=\"Ask me about weather or where people live!\",\n)\n\n## Uncomment the line below to launch the chat app with UI\n# demo.launch()" }, { "cell_type": "markdown", "metadata": { "id": "rgGKjm7cpv1_" }, - "source": [ - "## What's next\n", - "\n", - "🎉 Congratulations! You've learned how to build chat applications that demonstrate agent-like behavior using OpenAI function calling and Haystack Pipelines.\n", - "\n", - "If you liked this tutorial, there's more to learn about Haystack:\n", - "- [Create a Swarm of Agents](https://haystack.deepset.ai/cookbook/swarm)\n", - "- [Evaluating RAG Pipelines](https://haystack.deepset.ai/tutorials/35_evaluating_rag_pipelines)\n", - "\n", - "To stay up to date on the latest Haystack developments, you can [sign up for our newsletter](https://landing.deepset.ai/haystack-community-updates) or [join Haystack discord community](https://discord.gg/Dr63fr9NDS).\n", - "\n", - "Thanks for reading!" - ] + "source": "## What's next\n\n🎉 Congratulations! You've learned how to build chat applications that demonstrate agent-like behavior using OpenAI function calling, Haystack Pipelines, and the `Agent` component.\n\nIf you liked this tutorial, there's more to learn about Haystack:\n- [Building a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent)\n- [Create a Swarm of Agents](https://haystack.deepset.ai/cookbook/swarm)\n- [Evaluating RAG Pipelines](https://haystack.deepset.ai/tutorials/35_evaluating_rag_pipelines)\n\nTo stay up to date on the latest Haystack developments, you can [sign up for our newsletter](https://landing.deepset.ai/haystack-community-updates) or [join Haystack discord community](https://discord.gg/Dr63fr9NDS).\n\nThanks for reading!" } ], "metadata": { @@ -837,4 +694,4 @@ }, "nbformat": 4, "nbformat_minor": 0 -} +} \ No newline at end of file From 9f9daf59fc7fe0340da471cbaa23258403b9aa01 Mon Sep 17 00:00:00 2001 From: Sebastian Husch Lee Date: Thu, 9 Jul 2026 08:09:42 +0200 Subject: [PATCH 2/3] use builtin openai embedders --- index.toml | 2 +- ...at_Application_with_Function_Calling.ipynb | 259 ++++++++++++------ 2 files changed, 175 insertions(+), 86 deletions(-) diff --git a/index.toml b/index.toml index 8e9e02df..7521b302 100644 --- a/index.toml +++ b/index.toml @@ -143,7 +143,7 @@ notebook = "40_Building_Chat_Application_with_Function_Calling.ipynb" aliases = [] completion_time = "20 min" created_at = 2024-03-05 -dependencies = ["sentence-transformers>=4.1.0", "gradio", "pytz"] +dependencies = ["gradio", "pytz"] [[tutorial]] title = "Query Classification with TransformersTextRouter and TransformersZeroShotTextRouter" diff --git a/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb b/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb index 20ca9f66..5f7ccf96 100644 --- a/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb +++ b/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb @@ -5,14 +5,36 @@ "metadata": { "id": "HJsHqaTksDHt" }, - "source": "# Tutorial: Building a Chat Agent with Function Calling\n\n- **Level**: Advanced\n- **Time to complete**: 20 minutes\n- **Components Used**: [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder), [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder), [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [Agent](https://docs.haystack.deepset.ai/docs/agent)\n- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys) and be familiar with [creating pipelines](https://docs.haystack.deepset.ai/docs/creating-pipelines)\n- **Goal**: After completing this tutorial, you will have learned how to build chat applications that demonstrate agent-like behavior using OpenAI's function calling feature." + "source": [ + "# Tutorial: Building a Chat Agent with Function Calling\n", + "\n", + "- **Level**: Advanced\n", + "- **Time to complete**: 20 minutes\n", + "- **Components Used**: [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [OpenAIDocumentEmbedder](https://docs.haystack.deepset.ai/docs/openaidocumentembedder), [OpenAITextEmbedder](https://docs.haystack.deepset.ai/docs/openaitextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder), [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [Agent](https://docs.haystack.deepset.ai/docs/agent)\n", + "- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys) and be familiar with [creating pipelines](https://docs.haystack.deepset.ai/docs/creating-pipelines)\n", + "- **Goal**: After completing this tutorial, you will have learned how to build chat applications that demonstrate agent-like behavior using OpenAI's function calling feature." + ] }, { "cell_type": "markdown", "metadata": { "id": "PWXXqq_MPn7y" }, - "source": "## Overview\n\n\n\n📚 Useful Sources:\n* [OpenAIChatGenerator Docs](https://docs.haystack.deepset.ai/docs/openaichatgenerator)\n* [OpenAIChatGenerator API Reference](https://docs.haystack.deepset.ai/reference/generators-api#openaichatgenerator)\n* [Agent Docs](https://docs.haystack.deepset.ai/docs/agent)\n* [🧑‍🍳 Cookbook: Function Calling with OpenAIChatGenerator](https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/function_calling_with_OpenAIChatGenerator.ipynb)\n\n[OpenAI's function calling](https://platform.openai.com/docs/guides/function-calling) connects large language models to external tools. By providing a `tools` list with functions and their specifications to the OpenAI API calls, you can easily build chat assistants that can answer questions by calling external APIs or extract structured information from text.\n\nIn this tutorial, you'll learn how to convert a Haystack pipeline into a function-calling tool and how to implement chat applications that show agent-like behavior. You'll first see the raw tool-calling mechanics through `OpenAIChatGenerator`, and then use Haystack's [`Agent`](https://docs.haystack.deepset.ai/docs/agent) component, which automatically runs the tool-calling loop for you." + "source": [ + "## Overview\n", + "\n", + "\n", + "\n", + "📚 Useful Sources:\n", + "* [OpenAIChatGenerator Docs](https://docs.haystack.deepset.ai/docs/openaichatgenerator)\n", + "* [OpenAIChatGenerator API Reference](https://docs.haystack.deepset.ai/reference/generators-api#openaichatgenerator)\n", + "* [Agent Docs](https://docs.haystack.deepset.ai/docs/agent)\n", + "* [🧑‍🍳 Cookbook: Function Calling with OpenAIChatGenerator](https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/function_calling_with_OpenAIChatGenerator.ipynb)\n", + "\n", + "[OpenAI's function calling](https://platform.openai.com/docs/guides/function-calling) connects large language models to external tools. By providing a `tools` list with functions and their specifications to the OpenAI API calls, you can easily build chat assistants that can answer questions by calling external APIs or extract structured information from text.\n", + "\n", + "In this tutorial, you'll learn how to convert a Haystack pipeline into a function-calling tool and how to implement chat applications that show agent-like behavior. You'll first see the raw tool-calling mechanics through `OpenAIChatGenerator`, and then use Haystack's [`Agent`](https://docs.haystack.deepset.ai/docs/agent) component, which automatically runs the tool-calling loop for you." + ] }, { "cell_type": "markdown", @@ -22,22 +44,21 @@ "source": [ "## Setting up the Development Environment\n", "\n", - "Install Haystack and [sentence-transformers](https://pypi.org/project/sentence-transformers/) using pip:" + "Install Haystack using pip:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "collapsed": true, - "id": "zNyqNVFaPN1A" + "id": "zNyqNVFaPN1A", + "scrolled": true }, "outputs": [], "source": [ "%%bash\n", "\n", - "pip install haystack-ai\n", - "pip install \"sentence-transformers>=4.1.0\"" + "pip install haystack-ai" ] }, { @@ -51,7 +72,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -61,10 +82,10 @@ }, "outputs": [ { - "name": "stdout", + "name": "stdin", "output_type": "stream", "text": [ - "Enter OpenAI API key:··········\n" + "Enter OpenAI API key: ········\n" ] } ], @@ -92,7 +113,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -104,10 +125,10 @@ { "data": { "text/plain": [ - "{'replies': [ChatMessage(_role=, _content=[TextContent(text='Natural Language Processing (NLP) ist ein Teilbereich der Künstlichen Intelligenz, der sich mit der Interaktion zwischen Computern und menschlicher Sprache beschäftigt. Ziel ist es, natürliche Sprache zu verstehen, zu interpretieren und zu erzeugen, um Kommunikation zwischen Mensch und Maschine zu erleichtern.')], _name=None, _meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 62, 'prompt_tokens': 33, 'total_tokens': 95, 'completion_tokens_details': CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), 'prompt_tokens_details': PromptTokensDetails(audio_tokens=0, cached_tokens=0)}})]}" + "{'replies': [ChatMessage(_role=, _content=[TextContent(text='Natural Language Processing (NLP) ist ein Teilbereich der Künstlichen Intelligenz, der sich mit der Interaktion zwischen Computern und Menschen durch natürliche Sprache beschäftigt. Ziel ist es, Computern zu ermöglichen, menschliche Sprache zu verstehen, zu interpretieren und darauf zu reagieren.')], _name=None, _meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 58, 'prompt_tokens': 33, 'total_tokens': 91, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}})]}" ] }, - "execution_count": 4, + "execution_count": 2, "metadata": {}, "output_type": "execute_result" } @@ -138,7 +159,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -151,7 +172,9 @@ "name": "stdout", "output_type": "stream", "text": [ - "Natural Language Processing (NLP) ist ein Teilbereich der Informatik und Linguistik, der sich mit der Interaktion zwischen Computern und Menschen in natürlicher Sprache befasst. Ziel ist es, Computer zu befähigen, menschliche Sprache zu verstehen, zu interpretieren und zu erzeugen. Anwendungen von NLP umfassen Spracherkennung, maschinelles Übersetzen, Sentiment-Analyse und Chatbots." + "[ASSISTANT]\n", + "Natural Language Processing (NLP) ist ein Teilbereich der Künstlichen Intelligenz, der sich mit der Interaktion zwischen Computern und Menschen in natürlicher Sprache beschäftigt. Ziel ist es, Maschinen das Verstehen, Interpretieren und Generieren von menschlicher Sprache zu ermöglichen.\n", + "\n" ] } ], @@ -184,7 +207,7 @@ "source": [ "### Index Documents with a Pipeline\n", "\n", - "Create a pipeline to store the small example dataset in the [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore) with their embeddings. You will use [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder) to generate embeddings for your Documents and write them to the document store with the [DocumentWriter](https://docs.haystack.deepset.ai/docs/documentwriter).\n", + "Create a pipeline to store the small example dataset in the [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore) with their embeddings. You will use [OpenAIDocumentEmbedder](https://docs.haystack.deepset.ai/docs/openaidocumentembedder) to generate embeddings for your Documents and write them to the document store with the [DocumentWriter](https://docs.haystack.deepset.ai/docs/documentwriter).\n", "\n", "After adding these components to your pipeline, connect them and run the pipeline.\n", "\n", @@ -193,7 +216,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -220,35 +243,18 @@ "name": "stderr", "output_type": "stream", "text": [ - "/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: \n", - "The secret `HF_TOKEN` does not exist in your Colab secrets.\n", - "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n", - "You will be able to reuse this secret in all of your notebooks.\n", - "Please note that authentication is recommended but still optional to access public models or datasets.\n", - " warnings.warn(\n" + "Calculating embeddings: 1it [00:01, 1.34s/it]\n" ] }, - { - "data": { - "application/vnd.jupyter.widget-view+json": { - "model_id": "81e44362f359418f96a7fb064124e04c", - "version_major": 2, - "version_minor": 0 - }, - "text/plain": [ - "Batches: 0%| | 0/1 [00:00 For a step-by-step guide to create a RAG pipeline with Haystack, follow the [Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline) tutorial." ] }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -307,25 +311,25 @@ { "data": { "text/plain": [ - "\n", + "\n", "🚅 Components\n", - " - embedder: SentenceTransformersTextEmbedder\n", + " - embedder: OpenAITextEmbedder\n", " - retriever: InMemoryEmbeddingRetriever\n", " - prompt_builder: ChatPromptBuilder\n", " - llm: OpenAIChatGenerator\n", "🛤️ Connections\n", - " - embedder.embedding -> retriever.query_embedding (List[float])\n", - " - retriever.documents -> prompt_builder.documents (List[Document])\n", - " - prompt_builder.prompt -> llm.messages (List[ChatMessage])" + " - embedder.embedding -> retriever.query_embedding (list[float])\n", + " - retriever.documents -> prompt_builder.documents (list[Document])\n", + " - prompt_builder.prompt -> llm.messages (list[ChatMessage])" ] }, - "execution_count": 7, + "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "from haystack.components.embedders import SentenceTransformersTextEmbedder\n", + "from haystack.components.embedders import OpenAITextEmbedder\n", "from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever\n", "from haystack.components.builders import ChatPromptBuilder\n", "from haystack.dataclasses import ChatMessage\n", @@ -346,7 +350,7 @@ " )\n", "]\n", "rag_pipe = Pipeline()\n", - "rag_pipe.add_component(\"embedder\", SentenceTransformersTextEmbedder(model=\"sentence-transformers/all-MiniLM-L6-v2\"))\n", + "rag_pipe.add_component(\"embedder\", OpenAITextEmbedder(model=\"text-embedding-3-small\"))\n", "rag_pipe.add_component(\"retriever\", InMemoryEmbeddingRetriever(document_store=document_store))\n", "rag_pipe.add_component(\"prompt_builder\", ChatPromptBuilder(template=template))\n", "rag_pipe.add_component(\"llm\", OpenAIChatGenerator(model=\"gpt-4o-mini\"))\n", @@ -368,7 +372,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 6, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -391,27 +395,15 @@ "outputId": "805cd637-e7f6-4099-baa0-93257b7e8aa0" }, "outputs": [ - { - "data": { - "application/vnd.jupyter.widget-view+json": { - "model_id": "d2e3e46dec734d848382ccde2f753a2c", - "version_major": 2, - "version_minor": 0 - }, - "text/plain": [ - "Batches: 0%| | 0/1 [00:00, _content=[TextContent(text='Mark lives in Berlin.')], _name=None, _meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 6, 'prompt_tokens': 83, 'total_tokens': 89, 'completion_tokens_details': CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), 'prompt_tokens_details': PromptTokensDetails(audio_tokens=0, cached_tokens=0)}})]}}" + "{'embedder': {'meta': {'model': 'text-embedding-3-small',\n", + " 'usage': {'prompt_tokens': 5, 'total_tokens': 5}}},\n", + " 'llm': {'replies': [ChatMessage(_role=, _content=[TextContent(text='Mark lives in Berlin.')], _name=None, _meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 5, 'prompt_tokens': 83, 'total_tokens': 88, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}})]}}" ] }, - "execution_count": 8, + "execution_count": 6, "metadata": {}, "output_type": "execute_result" } @@ -436,7 +428,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 7, "metadata": { "id": "hCUFdKV5_Z7k" }, @@ -484,7 +476,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 8, "metadata": { "id": "XaQgnmPoJD13" }, @@ -531,7 +523,7 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": null, "metadata": { "id": "OEScMyqctzFN" }, @@ -585,7 +577,15 @@ "metadata": { "id": "_ulFfHcnGfsw" }, - "source": "## Letting the Agent Run the Tool-Calling Loop\n\nAs you saw above, the OpenAI Chat Completions API doesn't call the tool itself. It only tells you *which* tool to call and with *which* arguments. To get a final answer, you'd have to execute the tool yourself, append the result to the message list, and call the generator again, possibly several times, until the model stops requesting tools.\n\nInstead of writing this loop by hand, use Haystack's [`Agent`](https://docs.haystack.deepset.ai/docs/agent) component. The `Agent` wraps a `ChatGenerator` together with a list of `tools` and automatically runs the full tool-calling loop for you: it calls the LLM, invokes any requested tools, feeds the results back to the LLM, and repeats until the model returns a plain text answer.\n\nCreate an `Agent` with the two tools you defined and run it on the same query:" + "source": [ + "## Letting the Agent Run the Tool-Calling Loop\n", + "\n", + "As you saw above, the OpenAI Chat Completions API doesn't call the tool itself. It only tells you *which* tool to call and with *which* arguments. To get a final answer, you'd have to execute the tool yourself, append the result to the message list, and call the generator again, possibly several times, until the model stops requesting tools.\n", + "\n", + "Instead of writing this loop by hand, use Haystack's [`Agent`](https://docs.haystack.deepset.ai/docs/agent) component. The `Agent` wraps a `ChatGenerator` together with a list of `tools` and automatically runs the full tool-calling loop for you: it calls the LLM, invokes any requested tools, feeds the results back to the LLM, and repeats until the model returns a plain text answer.\n", + "\n", + "Create an `Agent` with the two tools you defined and run it on the same query:" + ] }, { "cell_type": "code", @@ -612,14 +612,29 @@ "outputId": "bd126d0f-3204-4089-a786-f0b233621149" }, "outputs": [], - "source": "from haystack.components.agents import Agent\nfrom haystack.components.generators.chat import OpenAIChatGenerator\nfrom haystack.dataclasses import ChatMessage\n\nagent = Agent(\n chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n tools=[rag_pipeline_tool, weather_tool],\n system_prompt=\"Use the tools that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\",\n)\n\nresult = agent.run(messages=[ChatMessage.from_user(\"Can you tell me where Mark lives?\")])\nprint(result[\"messages\"][-1].text)" + "source": [ + "from haystack.components.agents import Agent\n", + "from haystack.components.generators.chat import OpenAIChatGenerator\n", + "from haystack.dataclasses import ChatMessage\n", + "\n", + "agent = Agent(\n", + " chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n", + " tools=[rag_pipeline_tool, weather_tool],\n", + " system_prompt=\"Use the tools that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\",\n", + ")\n", + "\n", + "result = agent.run(messages=[ChatMessage.from_user(\"Can you tell me where Mark lives?\")])\n", + "print(result[\"messages\"][-1].text)" + ] }, { "cell_type": "markdown", "metadata": { "id": "I6zam1cfbuZv" }, - "source": "Besides the final answer, the `Agent` returns the full list of messages exchanged during the run under the `\"messages\"` key. This lets you see the tool call and the tool result that the `Agent` handled for you under the hood, exactly the steps you would have had to orchestrate manually:" + "source": [ + "Besides the final answer, the `Agent` returns the full list of messages exchanged during the run under the `\"messages\"` key. This lets you see the tool call and the tool result that the `Agent` handled for you under the hood, exactly the steps you would have had to orchestrate manually:" + ] }, { "cell_type": "code", @@ -632,20 +647,40 @@ "outputId": "0ee96a46-16f3-4a03-ce54-a69bedde24ae" }, "outputs": [], - "source": "for message in result[\"messages\"]:\n print(message)\n print(\"-\" * 10)" + "source": [ + "for message in result[\"messages\"]:\n", + " print(message)\n", + " print(\"-\" * 10)" + ] }, { "cell_type": "markdown", "metadata": { "id": "5UkjGYEfwPzS" }, - "source": "## Building the Chat Application\n\nNow let's turn the `Agent` into an interactive, multi-turn chat application. Because the `Agent` already handles the tool-calling loop internally, all you need to do to hold a conversation is keep track of the message history: append each new user message, run the `Agent`, and carry the returned `messages` over to the next turn so the model remembers the context.\n\nTo build a nice UI for your application, you can use [Gradio](https://www.gradio.app/) that comes with a chat interface. Install `gradio`, run the code cell below and use the input box to interact with the chat application that has access to the two tools you've created above.\n\nExample queries you can try:\n* \"***What is the capital of Sweden?***\": A basic query without any tool calls\n* \"***Can you tell me where Giorgio lives?***\": A basic query with one tool call\n* \"***What's the weather like in Berlin?***\", \"***Is it sunny there?***\": To see the messages are being recorded and sent\n* \"***What's the weather like where Jean lives?***\": To force two tool calls\n* \"***What's the weather like today?***\": To force OpenAI to ask more clarification\n\n> Keep in mind that OpenAI models can sometimes hallucinate answers or tools and might not work as expected." + "source": [ + "## Building the Chat Application\n", + "\n", + "Now let's turn the `Agent` into an interactive, multi-turn chat application. Because the `Agent` already handles the tool-calling loop internally, all you need to do to hold a conversation is keep track of the message history: append each new user message, run the `Agent`, and carry the returned `messages` over to the next turn so the model remembers the context.\n", + "\n", + "To build a nice UI for your application, you can use [Gradio](https://www.gradio.app/) that comes with a chat interface. Install `gradio`, run the code cell below and use the input box to interact with the chat application that has access to the two tools you've created above.\n", + "\n", + "Example queries you can try:\n", + "* \"***What is the capital of Sweden?***\": A basic query without any tool calls\n", + "* \"***Can you tell me where Giorgio lives?***\": A basic query with one tool call\n", + "* \"***What's the weather like in Berlin?***\", \"***Is it sunny there?***\": To see the messages are being recorded and sent\n", + "* \"***What's the weather like where Jean lives?***\": To force two tool calls\n", + "* \"***What's the weather like today?***\": To force OpenAI to ask more clarification\n", + "\n", + "> Keep in mind that OpenAI models can sometimes hallucinate answers or tools and might not work as expected." + ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "id": "mmOBVDvmOPWe" + "id": "mmOBVDvmOPWe", + "scrolled": true }, "outputs": [], "source": [ @@ -661,14 +696,67 @@ "id": "sK_JeKZLhXcy" }, "outputs": [], - "source": "import gradio as gr\n\nfrom haystack.components.agents import Agent\nfrom haystack.components.generators.chat import OpenAIChatGenerator\nfrom haystack.dataclasses import ChatMessage\n\nagent = Agent(\n chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n tools=[rag_pipeline_tool, weather_tool],\n system_prompt=\"Use the tools that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\",\n)\nagent.warm_up()\n\nmessages = [ChatMessage.from_system(agent.system_prompt)]\n\n\ndef chatbot_with_tc(message, history):\n global messages\n messages.append(ChatMessage.from_user(message))\n # The Agent runs the full tool-calling loop and returns all messages exchanged during the run\n result = agent.run(messages=messages)\n # Carry the updated conversation history over to the next turn\n messages = result[\"messages\"]\n return result[\"messages\"][-1].text\n\n\ndemo = gr.ChatInterface(\n fn=chatbot_with_tc,\n examples=[\n \"Can you tell me where Giorgio lives?\",\n \"What's the weather like in Madrid?\",\n \"Who lives in London?\",\n \"What's the weather like where Mark lives?\",\n ],\n title=\"Ask me about weather or where people live!\",\n)\n\n## Uncomment the line below to launch the chat app with UI\n# demo.launch()" + "source": [ + "import gradio as gr\n", + "\n", + "from haystack.components.agents import Agent\n", + "from haystack.components.generators.chat import OpenAIChatGenerator\n", + "from haystack.dataclasses import ChatMessage\n", + "\n", + "agent = Agent(\n", + " chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n", + " tools=[rag_pipeline_tool, weather_tool],\n", + " system_prompt=\"Use the tools that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\",\n", + ")\n", + "agent.warm_up()\n", + "\n", + "messages = [ChatMessage.from_system(agent.system_prompt)]\n", + "\n", + "\n", + "def chatbot_with_tc(message, history):\n", + " global messages\n", + " messages.append(ChatMessage.from_user(message))\n", + " # The Agent runs the full tool-calling loop and returns all messages exchanged during the run\n", + " result = agent.run(messages=messages)\n", + " # Carry the updated conversation history over to the next turn\n", + " messages = result[\"messages\"]\n", + " return result[\"messages\"][-1].text\n", + "\n", + "\n", + "demo = gr.ChatInterface(\n", + " fn=chatbot_with_tc,\n", + " examples=[\n", + " \"Can you tell me where Giorgio lives?\",\n", + " \"What's the weather like in Madrid?\",\n", + " \"Who lives in London?\",\n", + " \"What's the weather like where Mark lives?\",\n", + " ],\n", + " title=\"Ask me about weather or where people live!\",\n", + ")\n", + "\n", + "## Uncomment the line below to launch the chat app with UI\n", + "demo.launch()" + ] }, { "cell_type": "markdown", "metadata": { "id": "rgGKjm7cpv1_" }, - "source": "## What's next\n\n🎉 Congratulations! You've learned how to build chat applications that demonstrate agent-like behavior using OpenAI function calling, Haystack Pipelines, and the `Agent` component.\n\nIf you liked this tutorial, there's more to learn about Haystack:\n- [Building a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent)\n- [Create a Swarm of Agents](https://haystack.deepset.ai/cookbook/swarm)\n- [Evaluating RAG Pipelines](https://haystack.deepset.ai/tutorials/35_evaluating_rag_pipelines)\n\nTo stay up to date on the latest Haystack developments, you can [sign up for our newsletter](https://landing.deepset.ai/haystack-community-updates) or [join Haystack discord community](https://discord.gg/Dr63fr9NDS).\n\nThanks for reading!" + "source": [ + "## What's next\n", + "\n", + "🎉 Congratulations! You've learned how to build chat applications that demonstrate agent-like behavior using OpenAI function calling, Haystack Pipelines, and the `Agent` component.\n", + "\n", + "If you liked this tutorial, there's more to learn about Haystack:\n", + "- [Building a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent)\n", + "- [Create a Swarm of Agents](https://haystack.deepset.ai/cookbook/swarm)\n", + "- [Evaluating RAG Pipelines](https://haystack.deepset.ai/tutorials/35_evaluating_rag_pipelines)\n", + "\n", + "To stay up to date on the latest Haystack developments, you can [sign up for our newsletter](https://landing.deepset.ai/haystack-community-updates) or [join Haystack discord community](https://discord.gg/Dr63fr9NDS).\n", + "\n", + "Thanks for reading!" + ] } ], "metadata": { @@ -676,7 +764,8 @@ "provenance": [] }, "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", + "language": "python", "name": "python3" }, "language_info": { @@ -689,9 +778,9 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.6" + "version": "3.12.11" } }, "nbformat": 4, - "nbformat_minor": 0 -} \ No newline at end of file + "nbformat_minor": 4 +} From dde3be9e23f1c46c33bc75b4905e2a91ca3ac768 Mon Sep 17 00:00:00 2001 From: Sebastian Husch Lee Date: Thu, 9 Jul 2026 08:16:36 +0200 Subject: [PATCH 3/3] move back to using sentence transformers --- index.toml | 2 +- ...at_Application_with_Function_Calling.ipynb | 204 ++---------------- 2 files changed, 15 insertions(+), 191 deletions(-) diff --git a/index.toml b/index.toml index af8d6c10..5a290264 100644 --- a/index.toml +++ b/index.toml @@ -143,7 +143,7 @@ notebook = "40_Building_Chat_Application_with_Function_Calling.ipynb" aliases = [] completion_time = "20 min" created_at = 2024-03-05 -dependencies = ["gradio", "pytz"] +dependencies = ["sentence-transformers-haystack", "gradio", "pytz"] [[tutorial]] title = "Query Classification with TransformersTextRouter and TransformersZeroShotTextRouter" diff --git a/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb b/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb index 5f7ccf96..5843b9f9 100644 --- a/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb +++ b/tutorials/40_Building_Chat_Application_with_Function_Calling.ipynb @@ -5,15 +5,7 @@ "metadata": { "id": "HJsHqaTksDHt" }, - "source": [ - "# Tutorial: Building a Chat Agent with Function Calling\n", - "\n", - "- **Level**: Advanced\n", - "- **Time to complete**: 20 minutes\n", - "- **Components Used**: [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [OpenAIDocumentEmbedder](https://docs.haystack.deepset.ai/docs/openaidocumentembedder), [OpenAITextEmbedder](https://docs.haystack.deepset.ai/docs/openaitextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder), [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [Agent](https://docs.haystack.deepset.ai/docs/agent)\n", - "- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys) and be familiar with [creating pipelines](https://docs.haystack.deepset.ai/docs/creating-pipelines)\n", - "- **Goal**: After completing this tutorial, you will have learned how to build chat applications that demonstrate agent-like behavior using OpenAI's function calling feature." - ] + "source": "# Tutorial: Building a Chat Agent with Function Calling\n\n- **Level**: Advanced\n- **Time to complete**: 20 minutes\n- **Components Used**: [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder), [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder), [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [Agent](https://docs.haystack.deepset.ai/docs/agent)\n- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys) and be familiar with [creating pipelines](https://docs.haystack.deepset.ai/docs/creating-pipelines)\n- **Goal**: After completing this tutorial, you will have learned how to build chat applications that demonstrate agent-like behavior using OpenAI's function calling feature." }, { "cell_type": "markdown", @@ -41,11 +33,7 @@ "metadata": { "id": "K04cnh_IleMV" }, - "source": [ - "## Setting up the Development Environment\n", - "\n", - "Install Haystack using pip:" - ] + "source": "## Setting up the Development Environment\n\nInstall Haystack and [sentence-transformers-haystack](https://pypi.org/project/sentence-transformers-haystack/) using pip:" }, { "cell_type": "code", @@ -55,11 +43,7 @@ "scrolled": true }, "outputs": [], - "source": [ - "%%bash\n", - "\n", - "pip install haystack-ai" - ] + "source": "%%bash\n\npip install haystack-ai\npip install sentence-transformers-haystack" }, { "cell_type": "markdown", @@ -204,19 +188,11 @@ "metadata": { "id": "FWkDXKbeoNqZ" }, - "source": [ - "### Index Documents with a Pipeline\n", - "\n", - "Create a pipeline to store the small example dataset in the [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore) with their embeddings. You will use [OpenAIDocumentEmbedder](https://docs.haystack.deepset.ai/docs/openaidocumentembedder) to generate embeddings for your Documents and write them to the document store with the [DocumentWriter](https://docs.haystack.deepset.ai/docs/documentwriter).\n", - "\n", - "After adding these components to your pipeline, connect them and run the pipeline.\n", - "\n", - "> If you'd like to learn about preprocessing files before you index them to your document store, follow the [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline) tutorial." - ] + "source": "### Index Documents with a Pipeline\n\nCreate a pipeline to store the small example dataset in the [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore) with their embeddings. You will use [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder) to generate embeddings for your Documents and write them to the document store with the [DocumentWriter](https://docs.haystack.deepset.ai/docs/documentwriter).\n\nAfter adding these components to your pipeline, connect them and run the pipeline.\n\n> If you'd like to learn about preprocessing files before you index them to your document store, follow the [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline) tutorial." }, { "cell_type": "code", - "execution_count": 4, + "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -238,68 +214,19 @@ "id": "ZE0SEGY92GHJ", "outputId": "5d7d3003-971b-453f-fc47-de612993d1aa" }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Calculating embeddings: 1it [00:01, 1.34s/it]\n" - ] - }, - { - "data": { - "text/plain": [ - "{'doc_embedder': {'meta': {'model': 'text-embedding-3-small',\n", - " 'usage': {'prompt_tokens': 53, 'total_tokens': 53}}},\n", - " 'doc_writer': {'documents_written': 5}}" - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from haystack import Pipeline, Document\n", - "from haystack.document_stores.in_memory import InMemoryDocumentStore\n", - "from haystack.components.writers import DocumentWriter\n", - "from haystack.components.embedders import OpenAIDocumentEmbedder\n", - "\n", - "documents = [\n", - " Document(content=\"My name is Jean and I live in Paris.\"),\n", - " Document(content=\"My name is Mark and I live in Berlin.\"),\n", - " Document(content=\"My name is Giorgio and I live in Rome.\"),\n", - " Document(content=\"My name is Marta and I live in Madrid.\"),\n", - " Document(content=\"My name is Harry and I live in London.\"),\n", - "]\n", - "\n", - "document_store = InMemoryDocumentStore()\n", - "\n", - "indexing_pipeline = Pipeline()\n", - "indexing_pipeline.add_component(instance=OpenAIDocumentEmbedder(model=\"text-embedding-3-small\"), name=\"doc_embedder\")\n", - "indexing_pipeline.add_component(instance=DocumentWriter(document_store=document_store), name=\"doc_writer\")\n", - "\n", - "indexing_pipeline.connect(\"doc_embedder.documents\", \"doc_writer.documents\")\n", - "\n", - "indexing_pipeline.run({\"doc_embedder\": {\"documents\": documents}})" - ] + "outputs": [], + "source": "from haystack import Pipeline, Document\nfrom haystack.document_stores.in_memory import InMemoryDocumentStore\nfrom haystack.components.writers import DocumentWriter\nfrom haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersDocumentEmbedder\n\ndocuments = [\n Document(content=\"My name is Jean and I live in Paris.\"),\n Document(content=\"My name is Mark and I live in Berlin.\"),\n Document(content=\"My name is Giorgio and I live in Rome.\"),\n Document(content=\"My name is Marta and I live in Madrid.\"),\n Document(content=\"My name is Harry and I live in London.\"),\n]\n\ndocument_store = InMemoryDocumentStore()\n\nindexing_pipeline = Pipeline()\nindexing_pipeline.add_component(\n instance=SentenceTransformersDocumentEmbedder(model=\"sentence-transformers/all-MiniLM-L6-v2\"), name=\"doc_embedder\"\n)\nindexing_pipeline.add_component(instance=DocumentWriter(document_store=document_store), name=\"doc_writer\")\n\nindexing_pipeline.connect(\"doc_embedder.documents\", \"doc_writer.documents\")\n\nindexing_pipeline.run({\"doc_embedder\": {\"documents\": documents}})" }, { "cell_type": "markdown", "metadata": { "id": "dwC6fWQy-7pI" }, - "source": [ - "### Build a RAG Pipeline\n", - "\n", - "Build a basic retrieval augmented generative pipeline with [OpenAITextEmbedder](https://docs.haystack.deepset.ai/docs/openaitextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder) and [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator).\n", - "\n", - "> For a step-by-step guide to create a RAG pipeline with Haystack, follow the [Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline) tutorial." - ] + "source": "### Build a RAG Pipeline\n\nBuild a basic retrieval augmented generative pipeline with [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder) and [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator).\n\n> For a step-by-step guide to create a RAG pipeline with Haystack, follow the [Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline) tutorial." }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" @@ -307,58 +234,8 @@ "id": "23JuUuql7PZ8", "outputId": "1ab36b2a-5f95-4580-ab3b-e60c8807795a" }, - "outputs": [ - { - "data": { - "text/plain": [ - "\n", - "🚅 Components\n", - " - embedder: OpenAITextEmbedder\n", - " - retriever: InMemoryEmbeddingRetriever\n", - " - prompt_builder: ChatPromptBuilder\n", - " - llm: OpenAIChatGenerator\n", - "🛤️ Connections\n", - " - embedder.embedding -> retriever.query_embedding (list[float])\n", - " - retriever.documents -> prompt_builder.documents (list[Document])\n", - " - prompt_builder.prompt -> llm.messages (list[ChatMessage])" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from haystack.components.embedders import OpenAITextEmbedder\n", - "from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever\n", - "from haystack.components.builders import ChatPromptBuilder\n", - "from haystack.dataclasses import ChatMessage\n", - "from haystack.components.generators.chat import OpenAIChatGenerator\n", - "\n", - "template = [\n", - " ChatMessage.from_system(\n", - " \"\"\"\n", - "Answer the questions based on the given context.\n", - "\n", - "Context:\n", - "{% for document in documents %}\n", - " {{ document.content }}\n", - "{% endfor %}\n", - "Question: {{ question }}\n", - "Answer:\n", - "\"\"\"\n", - " )\n", - "]\n", - "rag_pipe = Pipeline()\n", - "rag_pipe.add_component(\"embedder\", OpenAITextEmbedder(model=\"text-embedding-3-small\"))\n", - "rag_pipe.add_component(\"retriever\", InMemoryEmbeddingRetriever(document_store=document_store))\n", - "rag_pipe.add_component(\"prompt_builder\", ChatPromptBuilder(template=template))\n", - "rag_pipe.add_component(\"llm\", OpenAIChatGenerator(model=\"gpt-4o-mini\"))\n", - "\n", - "rag_pipe.connect(\"embedder.embedding\", \"retriever.query_embedding\")\n", - "rag_pipe.connect(\"retriever\", \"prompt_builder.documents\")\n", - "rag_pipe.connect(\"prompt_builder.prompt\", \"llm.messages\")" - ] + "outputs": [], + "source": "from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder\nfrom haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever\nfrom haystack.components.builders import ChatPromptBuilder\nfrom haystack.dataclasses import ChatMessage\nfrom haystack.components.generators.chat import OpenAIChatGenerator\n\ntemplate = [\n ChatMessage.from_system(\n \"\"\"\nAnswer the questions based on the given context.\n\nContext:\n{% for document in documents %}\n {{ document.content }}\n{% endfor %}\nQuestion: {{ question }}\nAnswer:\n\"\"\"\n )\n]\nrag_pipe = Pipeline()\nrag_pipe.add_component(\"embedder\", SentenceTransformersTextEmbedder(model=\"sentence-transformers/all-MiniLM-L6-v2\"))\nrag_pipe.add_component(\"retriever\", InMemoryEmbeddingRetriever(document_store=document_store))\nrag_pipe.add_component(\"prompt_builder\", ChatPromptBuilder(template=template))\nrag_pipe.add_component(\"llm\", OpenAIChatGenerator(model=\"gpt-4o-mini\"))\n\nrag_pipe.connect(\"embedder.embedding\", \"retriever.query_embedding\")\nrag_pipe.connect(\"retriever\", \"prompt_builder.documents\")\nrag_pipe.connect(\"prompt_builder.prompt\", \"llm.messages\")" }, { "cell_type": "markdown", @@ -372,7 +249,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", @@ -394,20 +271,7 @@ "id": "h4bXXC2Z7eaj", "outputId": "805cd637-e7f6-4099-baa0-93257b7e8aa0" }, - "outputs": [ - { - "data": { - "text/plain": [ - "{'embedder': {'meta': {'model': 'text-embedding-3-small',\n", - " 'usage': {'prompt_tokens': 5, 'total_tokens': 5}}},\n", - " 'llm': {'replies': [ChatMessage(_role=, _content=[TextContent(text='Mark lives in Berlin.')], _name=None, _meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 5, 'prompt_tokens': 83, 'total_tokens': 88, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}})]}}" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "query = \"Where does Mark live?\"\n", "rag_pipe.run({\"embedder\": {\"text\": query}, \"prompt_builder\": {\"question\": query}})" @@ -696,47 +560,7 @@ "id": "sK_JeKZLhXcy" }, "outputs": [], - "source": [ - "import gradio as gr\n", - "\n", - "from haystack.components.agents import Agent\n", - "from haystack.components.generators.chat import OpenAIChatGenerator\n", - "from haystack.dataclasses import ChatMessage\n", - "\n", - "agent = Agent(\n", - " chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n", - " tools=[rag_pipeline_tool, weather_tool],\n", - " system_prompt=\"Use the tools that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\",\n", - ")\n", - "agent.warm_up()\n", - "\n", - "messages = [ChatMessage.from_system(agent.system_prompt)]\n", - "\n", - "\n", - "def chatbot_with_tc(message, history):\n", - " global messages\n", - " messages.append(ChatMessage.from_user(message))\n", - " # The Agent runs the full tool-calling loop and returns all messages exchanged during the run\n", - " result = agent.run(messages=messages)\n", - " # Carry the updated conversation history over to the next turn\n", - " messages = result[\"messages\"]\n", - " return result[\"messages\"][-1].text\n", - "\n", - "\n", - "demo = gr.ChatInterface(\n", - " fn=chatbot_with_tc,\n", - " examples=[\n", - " \"Can you tell me where Giorgio lives?\",\n", - " \"What's the weather like in Madrid?\",\n", - " \"Who lives in London?\",\n", - " \"What's the weather like where Mark lives?\",\n", - " ],\n", - " title=\"Ask me about weather or where people live!\",\n", - ")\n", - "\n", - "## Uncomment the line below to launch the chat app with UI\n", - "demo.launch()" - ] + "source": "import gradio as gr\n\nfrom haystack.components.agents import Agent\nfrom haystack.components.generators.chat import OpenAIChatGenerator\nfrom haystack.dataclasses import ChatMessage\n\nagent = Agent(\n chat_generator=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n tools=[rag_pipeline_tool, weather_tool],\n system_prompt=\"Use the tools that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\",\n)\nagent.warm_up()\n\nmessages = [ChatMessage.from_system(agent.system_prompt)]\n\n\ndef chatbot_with_tc(message, history):\n global messages\n messages.append(ChatMessage.from_user(message))\n # The Agent runs the full tool-calling loop and returns all messages exchanged during the run\n result = agent.run(messages=messages)\n # Carry the updated conversation history over to the next turn\n messages = result[\"messages\"]\n return result[\"messages\"][-1].text\n\n\ndemo = gr.ChatInterface(\n fn=chatbot_with_tc,\n examples=[\n \"Can you tell me where Giorgio lives?\",\n \"What's the weather like in Madrid?\",\n \"Who lives in London?\",\n \"What's the weather like where Mark lives?\",\n ],\n title=\"Ask me about weather or where people live!\",\n)\n\n## Uncomment the line below to launch the chat app with UI\n# demo.launch()" }, { "cell_type": "markdown",