diff --git a/cookbook/azure_openai_getting_started.ipynb b/cookbook/azure_openai_getting_started.ipynb new file mode 100644 index 0000000000000..61424fa5059eb --- /dev/null +++ b/cookbook/azure_openai_getting_started.ipynb @@ -0,0 +1,1963 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "d808a92c-d36d-4291-a4ad-b12bceb62fec", + "metadata": {}, + "source": [ + "# Getting Started Azure OpenAI with LangChain in Python" + ] + }, + { + "cell_type": "markdown", + "id": "6689d50a-d463-4ad1-8968-180f93bd2f8e", + "metadata": {}, + "source": [ + "The purpose of this notebook is to provide a step-by-step guide on how to set up with Azure OpenAI and use the available models through the LangChain framework in Python. Additionally, it will demonstrate, through examples, how to implement models and utilize available features." + ] + }, + { + "cell_type": "markdown", + "id": "b8202de6-7d14-44e6-b378-879d26f95ce3", + "metadata": {}, + "source": [ + "## Obtaining Keys and Endpoints" + ] + }, + { + "cell_type": "markdown", + "id": "3b21c348-6a01-41d4-9df5-0d1d633367ef", + "metadata": {}, + "source": [ + "Using the [**Azure sign up page**](https://azure.microsoft.com/pricing/purchase-options/azure-account) one can sign up and create an Azure OpenAI resource, giving access to all necessary credentials." + ] + }, + { + "cell_type": "markdown", + "id": "ff61b81a-5c29-4da1-818f-fe3c2b1f9280", + "metadata": {}, + "source": [ + "## Setup\n", + "#### Installing and Importing Dependencies\n", + "Microsoft [**recommends**](https://github.com/microsoft/vscode-jupyter/wiki/Installing-Python-packages-in-Jupyter-Notebooks) using:\n", + "\n", + "- %pip for installing within Jupyter or IDE environments.\n", + "- %conda for installing within Conda environments.\n", + " " + ] + }, + { + "cell_type": "markdown", + "id": "448ce3be-b98f-4d46-8815-36fda94e9d30", + "metadata": {}, + "source": [ + "#### Install the most recent version of `langchain` and `langchain_openai`." + ] + }, + { + "cell_type": "markdown", + "id": "23716db9-d8c8-4682-b662-0de8c185bd07", + "metadata": {}, + "source": [ + "`%pip install -U langchain langchain_openai`" + ] + }, + { + "cell_type": "markdown", + "id": "18b3c17d", + "metadata": {}, + "source": [ + "#### Install the most recent version of `pandas` and `numpy`." + ] + }, + { + "cell_type": "markdown", + "id": "6eee1915", + "metadata": {}, + "source": [ + "`%pip install -U pandas numpy` " + ] + }, + { + "cell_type": "markdown", + "id": "7f634643-841b-47fb-ba55-ff058e863528", + "metadata": {}, + "source": [ + "#### Install Packages in a Virtual Environment (Optional)\n", + "Set up a virtual environment by going to your project directory and executing the following command. This will create a new virtual environment in a folder named `.venv`." + ] + }, + { + "cell_type": "markdown", + "id": "59fa41c8-3535-4d9b-9c30-59e3416ff875", + "metadata": {}, + "source": [ + "**MacOS/UNIX**\n", + "- `python3 -m venv .venv`\n", + "\n", + "**Windows**\n", + "- `py -m venv .venv`" + ] + }, + { + "cell_type": "markdown", + "id": "4679ec04-7fe4-49db-9887-2a821943f029", + "metadata": {}, + "source": [ + "#### Activating the Virtual Environment" + ] + }, + { + "cell_type": "markdown", + "id": "74c7218d-e68c-4af6-8600-9bb523490392", + "metadata": {}, + "source": [ + "To use the virtual environment, you must first activate it by executing the following command." + ] + }, + { + "cell_type": "markdown", + "id": "38cba946-94fd-445c-b23e-eea68661826d", + "metadata": { + "jp-MarkdownHeadingCollapsed": true + }, + "source": [ + "**MacOS/UNIX**\n", + "- `source .venv/bin/activate`\n", + "\n", + " \n", + "**Windows**\n", + "- `.venv\\Scripts\\activate`" + ] + }, + { + "cell_type": "markdown", + "id": "994e5683-d8d4-4f37-945e-19b1d9aab967", + "metadata": {}, + "source": [ + "#### Deactivating the Virtual Environment" + ] + }, + { + "cell_type": "markdown", + "id": "5c481e4e-4c39-46ba-98c1-92448da66e93", + "metadata": {}, + "source": [ + "If you want to leave the virtual environment in MacOS/UNIX and windows, simply execute the following command in the terminal:\n", + "\n", + "`deactivate`" + ] + }, + { + "cell_type": "markdown", + "id": "128f4b14", + "metadata": {}, + "source": [ + "#### Import Packages" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "7444cab5", + "metadata": {}, + "outputs": [], + "source": [ + "# Standard library imports\n", + "import json\n", + "import os\n", + "\n", + "# Third-party imports\n", + "import numpy as np\n", + "import pandas as pd\n", + "from dotenv import load_dotenv, set_key\n", + "\n", + "# callbacks\n", + "from langchain_community.callbacks import get_openai_callback\n", + "\n", + "# messages\n", + "from langchain_core.messages import AIMessage, HumanMessage, SystemMessage\n", + "\n", + "# output parsers\n", + "from langchain_core.output_parsers import StrOutputParser\n", + "\n", + "# prompts\n", + "from langchain_core.prompts import (\n", + " AIMessagePromptTemplate,\n", + " ChatPromptTemplate,\n", + " FewShotPromptTemplate,\n", + " HumanMessagePromptTemplate,\n", + " MessagesPlaceholder,\n", + " PromptTemplate,\n", + " SystemMessagePromptTemplate,\n", + ")\n", + "\n", + "# pydantic\n", + "from langchain_core.pydantic_v1 import BaseModel, Field\n", + "\n", + "# langchain Azure OpenAI\n", + "from langchain_openai import AzureChatOpenAI, AzureOpenAIEmbeddings" + ] + }, + { + "cell_type": "markdown", + "id": "46149db2-b280-45d5-b910-df1db63040bf", + "metadata": {}, + "source": [ + "#### Set all required Environment Variables" + ] + }, + { + "cell_type": "markdown", + "id": "4bf3454d", + "metadata": {}, + "source": [ + "Create a `.env` file and store your credentials like follows:\n", + "\n", + "`AZURE_OPENAI_API_KEY` = <\"Your key here\">\n", + "\n", + "`AZURE_OPENAI_ENDPOINT` = <\"Your endpoint here\">" + ] + }, + { + "cell_type": "markdown", + "id": "7723ed6a", + "metadata": {}, + "source": [ + "\n", + "Or use dotenv to set and load all required env variables into/from your `.env` file.\n", + "\n", + "`%pip install python-dotenv`" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "471ae21c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(True, 'COMPLETIONS_MODEL', 'Your model here')" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "set_key(\".env\", \"OPENAI_API_VERSION\", \"Your api version here\")\n", + "set_key(\".env\", \"COMPLETIONS_MODEL\", \"Your model here\")" + ] + }, + { + "cell_type": "markdown", + "id": "4e595a7e-9b33-472d-96ff-e3229d723642", + "metadata": {}, + "source": [ + "#### Get all required Environment Variables" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "fb74be9e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "load_dotenv()" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "b456baeb-fe51-45c8-940e-7f6ec52da0df", + "metadata": {}, + "outputs": [], + "source": [ + "# Setting up the deployment name\n", + "DEPLOYMENT_NAME = os.environ[\"COMPLETIONS_MODEL\"]\n", + "\n", + "# The API key for your Azure OpenAI resource.\n", + "API_KEY = os.environ[\"AZURE_OPENAI_API_KEY\"]\n", + "\n", + "# The base URL for your Azure OpenAI resource. e.g. \"https://.openai.azure.com\"\n", + "ENDPOINT = os.environ[\"AZURE_OPENAI_ENDPOINT\"]\n", + "\n", + "# The API version required\n", + "VERSION = os.environ[\"OPENAI_API_VERSION\"]" + ] + }, + { + "cell_type": "markdown", + "id": "a43b7040-d6ab-4df8-8432-907eb1ca4099", + "metadata": {}, + "source": [ + "## Creating an AzureChatOpenAI Model\n", + "\n", + "More information on LangChain's AzureChatOpenAI support can be found in [**the integration documentation**](https://python.langchain.com/v0.2/docs/integrations/chat/azure_chat_openai/)." + ] + }, + { + "cell_type": "markdown", + "id": "172e4ff3-b3fd-4f10-bd2b-18ac1e259973", + "metadata": {}, + "source": [ + "- Environment variable values can be passed as parameters.\n", + "- Alternatively, if not passed in, the constructor will search for environment variables with corresponding names." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "74baa57a-6b35-403c-9902-2f5a9c7e2bd4", + "metadata": {}, + "outputs": [], + "source": [ + "model = AzureChatOpenAI(\n", + " openai_api_version=VERSION,\n", + " azure_deployment=DEPLOYMENT_NAME,\n", + " azure_endpoint=ENDPOINT,\n", + " temperature=0.5,\n", + " max_tokens=200,\n", + " timeout=60,\n", + " max_retries=10,\n", + " # model=\"gpt-35-turbo\",\n", + " # model_version=\"0125\",\n", + " # other params...\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "23b80bbe-442c-4374-a885-87bd6408dc28", + "metadata": {}, + "source": [ + "In the above code sample, **OPENAI_API_VERSION** and **AZURE_OPENAI_ENDPOINT** are both being passed in, but **AZURE_OPENAI_API_KEY** is being retrieved within the constructor." + ] + }, + { + "cell_type": "markdown", + "id": "9b85766e-2581-4f24-86c1-cde10a0601c0", + "metadata": {}, + "source": [ + "#### Other Optional Parameters\n", + "\n", + "- `temperature` determines how creative and unpredictable, or how deterministic and predictable, the model should be in its responses. A temperature of 0 would be predictable, while anything higher would make responses more random.\n", + "\n", + "- `max_tokens` defines the maximum number of tokens (words or pieces of words) the model can generate in its response.\n", + "\n", + "- `timeout` specifies the maximum amount of time (in seconds) to wait for a response from the API before timing out. An `APITimeoutError` will be raised in the case of a timeout.\n", + "\n", + "- `max_retries` sets the number of times the API request should be retried in case of retriable failure before giving up.\n", + "\n", + "- `model` specifies the model to be used.\n", + "\n", + "- `model_version` indicates the specific version of the chosen model to use. This is useful for maintaining consistency in testing and for tracing purposes, such as tracking API calls or diagnosing issues related to specific model versions.\n", + "\n", + "- See the [**API Reference**](https://api.python.langchain.com/en/latest/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html) for more details.\n", + "\n", + "- Other parameters may be available in different SDK's." + ] + }, + { + "cell_type": "markdown", + "id": "b9edabc3", + "metadata": {}, + "source": [ + "### Wait in between API calls" + ] + }, + { + "cell_type": "markdown", + "id": "239cdb0b", + "metadata": {}, + "source": [ + "The number of API requests a user can make depends on their Azure plan and account settings. If too many requests are sent in a short period, an error may occur, prompting the user to wait for **x** amount of time before sending another request.\n", + "\n", + "When creating a model, one of the key parameters is the `max_retries` setting. The underlying Python OpenAI library will automatically wait and retry the call on your behalf at least 2 times by default before raising a `RateLimitError`. This behavior can be adjusted by setting a different value for `max_retries`.\n", + "\n", + "Visit the [**quotas and limits**](https://learn.microsoft.com/azure/ai-services/openai/quotas-limits) page to view detailed information related to account limits and restrictions." + ] + }, + { + "cell_type": "markdown", + "id": "28a397b3-27b7-49f0-bccf-108f39196ee6", + "metadata": {}, + "source": [ + "## Model Usage" + ] + }, + { + "cell_type": "markdown", + "id": "1a152eab-52e5-48f3-bc8f-0b410d65c7d0", + "metadata": {}, + "source": [ + "#### Using Messages from the `langchain_core.messages` Library\n", + "\n", + "The `langchain_core.messages` library allows the user to define messages for the model and assign roles to each message." + ] + }, + { + "cell_type": "markdown", + "id": "259a462a-4a83-4f17-a337-be04d67d3020", + "metadata": {}, + "source": [ + "- LangChain-compatible chat models take a list of `messages` as `input` and return the AI message as `output`.\n", + "\n", + "- All messages have `role` and `content` properties. In the sample below, the roles are set by using the `SystemMessage` and `HumanMessage` classes. [**We'll cover more on this later**](#assigning-roles-using-langchain-messages) .\n", + "\n", + "- Additional provider-specific information can be incorporated using the `additional_kwargs` parameter. This could include provider-specific metadata or custom settings and flags." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "57c6f2ae-b6d0-46e3-831a-ad5f77daf0bd", + "metadata": {}, + "outputs": [], + "source": [ + "messages = [\n", + " SystemMessage(content=\"Translate the following from German into English\"),\n", + " HumanMessage(\n", + " content=\"Sie haben gerade Ihr erstes Kunstliche Itelligenz Model erstellt!\"\n", + " ),\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "b6a40126-bede-46dc-8922-8721ac2f9c22", + "metadata": {}, + "outputs": [], + "source": [ + "response = model.invoke(messages)" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "id": "876d40ef-a75f-4d62-bc0a-43de6b55ce30", + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'You have just created your first artificial intelligence model!'" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "a0c37fe3-ed5d-4633-8e97-79cdba0e0365", + "metadata": {}, + "source": [ + "### Prompting\n", + "\n", + "- Prompts are the inputs to language models, refined from raw user inputs to be ready for processing by the models.\n", + "\n", + "- [**Prompting**](https://www.datacamp.com/tutorial/prompt-engineering-with-langchain) involves crafting text inputs that clearly communicate with the models, outlining the specific task we want it to accomplish. This can include:\n", + " - Selecting the appropriate wording and setting a particular tone or style.\n", + " - Providing necessary context.\n", + " - Assigning a role, such as asking it to respond as if it were a native speaker of a certain language." + ] + }, + { + "cell_type": "markdown", + "id": "2ee1ff1f-dbfc-445f-807e-4682db59460f", + "metadata": {}, + "source": [ + "#### Prompt Templates\n", + "\n", + "- LangChain allows developers to design parameterized [**Prompt Templates**](https://python.langchain.com/v0.2/docs/concepts/#prompt-templates) that are reusable and easily transferable between different models for integration.\n", + "\n", + "- It takes user input and inserts said input into the prompt to feed into the language models.\n", + "\n", + "#### `PromptTemplate`\n", + "`PromptTemplate` is used to create an instance of [**Prompt**](https://python.langchain.com/v0.2/api_reference/core/prompts/langchain_core.prompts.prompt.PromptTemplate.html#prompttemplate), and this is `invoked` by sending it to a model, which produces a `PromptValue`.\n", + "\n", + "The example code uses `.from_template`, which handles a single string template with placeholders for dynamic inputs." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "04a11bb8-e4d9-4687-ab89-210aa1b7b51b", + "metadata": {}, + "outputs": [], + "source": [ + "prompt_template = PromptTemplate.from_template(\n", + " \"What vegetable crops can I grow in {month} in {city}, New Zealand?\"\n", + ")\n", + "\n", + "prompt_value = prompt_template.format(month=\"December\", city=\"Rotorua\")\n", + "\n", + "\n", + "# print(prompt_template) # <- uncomment to see\n", + "# print(prompt_value) # <- uncomment to see" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "f92db0c1-3741-42be-9d1e-edbe5010b61e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\"In Rotorua, New Zealand, December falls in the Southern Hemisphere's summer, which is a great time for growing a variety of vegetables. Here are some vegetable crops you can plant in December:\\n\\n1. **Tomatoes**: Ideal for summer planting, they thrive in the warm weather.\\n2. **Capsicums (Bell Peppers)**: These also enjoy the summer heat.\\n3. **Zucchini**: Fast-growing and productive during warm months.\\n4. **Cucumbers**: Perfect for summer salads and pickling.\\n5. **Beans**: Both bush and pole beans grow well in the warm season.\\n6. **Sweet Corn**: Requires warm temperatures and plenty of sunlight.\\n7. **Pumpkins**: Plant now for a harvest in autumn.\\n8. **Eggplants**: Another heat-loving crop.\\n9. **Lettuce**: Opt for heat-tolerant varieties to avoid bolting.\\n10. **Radishes**: Fast-growing and can\"" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response = model.invoke(prompt_value)\n", + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "1ca904f4-faf1-4b95-bf77-7c7a7fa64515", + "metadata": {}, + "source": [ + "#### `ChatPromptTemplate`\n", + "\n", + "This is optimized for a conversation-like format. The prompt is a list of chat messages. Each chat message is associated with `role` and `content`. In the example code, `.from_messages` is used to include multiple messages.\n", + "\n", + "Here, we will hardcode roles in the chat prompt, as opposed to using the pre-built roles `SystemMessage` or `HumanMessage` like earlier." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "46e869bf-370d-4742-837f-c7d5eb76a891", + "metadata": {}, + "outputs": [], + "source": [ + "chat_template = ChatPromptTemplate.from_messages(\n", + " [\n", + " (\n", + " \"system\",\n", + " \"\"\"\n", + " You're a travel agent helping customers plan their trips.\n", + " Offer recommendations on natural features to visit, local cuisine, and activities based on the country the customer is asking about.\n", + " \"\"\",\n", + " ),\n", + " (\"ai\", \"Hi there, What can I help you with today?\"),\n", + " (\n", + " \"human\",\n", + " \"Hi I'm {name}, I'm planning a trip to {country}. Any recommendations\",\n", + " ),\n", + " ]\n", + ")\n", + "\n", + "prompt_value = chat_template.format_messages(name=\"Lucy\", country=\"New Zealand\")\n", + "\n", + "# print(chat_template) # <- uncomment to see\n", + "# print(prompt_value) # <- uncomment to see" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "c87adc53-1248-441d-aa46-44f410c24796", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\"Hi Lucy! New Zealand is a fantastic choice with its stunning landscapes, rich culture, and exciting activities. Here are some recommendations to make your trip memorable:\\n\\n### Natural Features\\n1. **Fiordland National Park**: Home to the famous Milford Sound and Doubtful Sound, this area offers breathtaking fjords, waterfalls, and rainforests.\\n2. **Tongariro National Park**: Known for its dramatic volcanic landscape, you can hike the Tongariro Alpine Crossing, one of the best one-day hikes in the world.\\n3. **Rotorua**: Famous for its geothermal activity, you can see geysers, hot springs, and mud pools. Don't miss the Wai-O-Tapu Thermal Wonderland.\\n4. **Aoraki/Mount Cook**: The highest mountain in New Zealand, offering stunning views, glaciers, and excellent hiking trails.\\n5. **Bay of Islands**: A beautiful coastal area with over 140 subtropical islands, perfect for sailing, fishing,\"" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response = model.invoke(prompt_value)\n", + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "d83fef4b", + "metadata": {}, + "source": [ + "#### Assigning Roles Using LangChain Messages" + ] + }, + { + "cell_type": "markdown", + "id": "cb6794d6-ba1f-4367-bae6-af0136212713", + "metadata": {}, + "source": [ + "Compared to hardcoding the roles like above, LangChain Messages allow for more flexibility and better management, especially with complex conversations involving multiple roles. It also simplifies the visualization of the conversation flow.\n", + "\n", + "It is therefore recommended to use LangChain messages where possible.\n", + "\n", + "**Basic Message Types**\n", + "\n", + "| | |\n", + "|-------------|-----------------------------------------------------------------|\n", + "| `SystemMessage` | Set how the AI should behave (appropriate wording, tone, style, etc.) |\n", + "| `HumanMessage` | Message sent from the user |\n", + "| `AIMessage` | Message from the AI chat model (context setting, guidance for response) |\n", + "\n", + "For more info, see [**Message Types**](https://python.langchain.com/v0.1/docs/modules/model_io/chat/message_types/) and [**API Reference**](https://api.python.langchain.com/en/latest/core_api_reference.html#module-langchain_core.messages)." + ] + }, + { + "cell_type": "markdown", + "id": "95152680-ebc2-4106-99bf-679547d0d1fe", + "metadata": {}, + "source": [ + "#### `base message` and `MessagePromptTemplate`\n", + "We can also pass a `base message` or `MessagePromptTemplate` instead of tuples." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "b956d584-881a-4e9b-ae81-565ffdd7bf4a", + "metadata": {}, + "outputs": [], + "source": [ + "chat_template = ChatPromptTemplate.from_messages(\n", + " [\n", + " SystemMessage(\n", + " content=(\n", + " \"You are a translator. You are to translate the text into English.\"\n", + " )\n", + " ),\n", + " HumanMessagePromptTemplate.from_template(\"{text}\"),\n", + " ]\n", + ")\n", + "\n", + "prompt_value = chat_template.format_messages(text=\"ゆずは日本で人気の果物です\")\n", + "\n", + "# print(chat_template) # <- uncomment to see\n", + "# print(prompt_value) # <- uncomment to see" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "648579a0-ca55-4885-bf70-47c0e08e066d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Yuzu is a popular fruit in Japan.'" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response = model.invoke(prompt_value)\n", + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "33f05d01-151b-4cda-a386-567a5523bb03", + "metadata": {}, + "source": [ + "#### `MessagePlaceHolder`\n", + "This is used to select which messages to include when formatting." + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "id": "0f7c266b-4438-4912-9b21-b0611960f296", + "metadata": {}, + "outputs": [], + "source": [ + "# SYSTEM ROLE Prompt\n", + "system_template = SystemMessagePromptTemplate.from_template(\"\"\"\n", + " You are a precise assistant who knows the schedule of the team.\n", + " Schedule details are as follows: {schedule}.\n", + " Only provide information to the team members.\n", + " Strictly only provide information specific to what is asked, Do not give extra information.\n", + " \"\"\")\n", + "# HUMAN ROLE Prompt\n", + "human_template = HumanMessagePromptTemplate.from_template(\"My name is {user_name}.\")\n", + "# AI ROLE Prompt\n", + "ai_template = AIMessagePromptTemplate.from_template(\n", + " \"Hello {user_name}, how can I help you today?\"\n", + ")\n", + "\n", + "chat_prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " # this has essentially created a 'conversation history'\n", + " system_template,\n", + " human_template,\n", + " ai_template,\n", + " MessagesPlaceholder(variable_name=\"conversation\"),\n", + " ]\n", + ")\n", + "\n", + "# print(chat_prompt) # <- uncomment to see the chat prompt" + ] + }, + { + "cell_type": "markdown", + "id": "6ab8c223-97d6-4a5c-83a5-24eeba7c47f3", + "metadata": {}, + "source": [ + "We can then input more prompts, which will take the `MessagePlaceholders`' place and create lines of sentences or a conversation." + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "id": "cea2fce5-2692-4df1-bac1-80bb9472d7f0", + "metadata": {}, + "outputs": [], + "source": [ + "schedule = \"\"\"\n", + " Team Members: Alice, Bob, Carol, David, Emily\n", + " Team Meeting Schedule: Every Tuesday at 11:00 AM\n", + " Topic: LangChain with Azure OpenAI Integration\n", + "\"\"\"\n", + "# these messages will take MESSAGEPLACEHOLDERS place\n", + "human_query = HumanMessage(\"When is the next team meeting and who is attending?\")\n", + "ai_message = AIMessage(\"Hold on a second, let me check the schedule for you.\")\n", + "\n", + "prompt_value = chat_prompt.format_messages(\n", + " conversation=[human_query, ai_message], user_name=\"David\", schedule=schedule\n", + ")\n", + "\n", + "# print(prompt_value) # <- uncomment to see the prompt" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "id": "1558cfbf-5806-4a24-a0fa-8a1fb90a00d8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'The next team meeting is on Tuesday at 11:00 AM. The attendees are Alice, Bob, Carol, David, and Emily.'" + ] + }, + "execution_count": 53, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response = model.invoke(prompt_value)\n", + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "34ec2ed1-83f1-4331-97fb-a6319f53e1fd", + "metadata": {}, + "source": [ + "#### `FewShotPrompt`\n", + "\n", + "We can use examples (shots) to condition the model for a better response by including some example input and output in the prompt. This will inform the model about the context and how we want the output to be formatted." + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "id": "fcaa0701-6f2c-4ffb-b011-75b6849d2ebe", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Currency Unit Conversion: [Input] one dollar => [Output] $1\n", + "Currency Unit Conversion: [Input] one hundred yen => [Output] ¥100\n" + ] + } + ], + "source": [ + "examples = [\n", + " {\"input\": \"one dollar\", \"output\": \"$1\"},\n", + " {\"input\": \"thirty five euros\", \"output\": \"€35\"},\n", + "]\n", + "\n", + "example_prompt = PromptTemplate(\n", + " input_variables=[\"input\", \"output\"],\n", + " template=\"Currency Unit Conversion: [Input] {input} => [Output] {output}\",\n", + ")\n", + "\n", + "# unpack the first example dictionary and feed it to the prompt template to format\n", + "print(example_prompt.format(**examples[0]))\n", + "\n", + "# feed examples to FewShotPromptTemplate to generate a final prompt\n", + "fewshot_prompt = FewShotPromptTemplate(\n", + " examples=examples,\n", + " example_prompt=example_prompt,\n", + " suffix=\"Convert the currency units: {input}\",\n", + " input_variables=[\"input\"],\n", + ")\n", + "\n", + "prompt_value = fewshot_prompt.format(input=\"one hundred yen\")\n", + "\n", + "response = model.invoke(prompt_value)\n", + "print(response.content)" + ] + }, + { + "cell_type": "markdown", + "id": "e52b27f1-90a9-46c7-8a43-a8ae356235ef", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "- Many LangChain components implement the [**Runnable**](https://python.langchain.com/v0.2/docs/concepts/#runnable-interface) protocol, which allows them to be easily chained together. These components can be combined in a sequence of calls, which we refer to as a chain.\n", + "\n", + "- Chaining `Runnables` in sequence." + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "id": "7bfdc351-1901-4fc4-bceb-067a68b54a30", + "metadata": {}, + "outputs": [], + "source": [ + "system_template = SystemMessagePromptTemplate.from_template(\"\"\"\n", + " You are an expert in {country} cuisine. \n", + " Keep it simple and short.\n", + " \"\"\")\n", + "\n", + "chat_prompt = ChatPromptTemplate.from_messages(\n", + " [\n", + " system_template,\n", + " (\"human\", \"I'd like to find out about {country} cuisine.\"),\n", + " (\"human\", \"{question}\"),\n", + " ]\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3f6682cc-576c-45f0-8b5b-52674a0f30a8", + "metadata": {}, + "source": [ + "This is how we have been using prompts, but now we will skip this step and invoke using the chain." + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "id": "761b48c5-92e5-4857-b280-cdd486c29b02", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'In Japan, maguro (tuna) is the most popular sashimi. Globally, salmon sashimi tends to be more popular.'" + ] + }, + "execution_count": 59, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "prompt_value = chat_prompt.format_messages(\n", + " country=\"Japanese\",\n", + " question=\"What is the most popular Sashimi in Japan vs the rest of the world?\",\n", + ")\n", + "\n", + "response = model.invoke(prompt_value)\n", + "response.content" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "id": "3893edc8-acc0-418e-ac5d-31a65b00b8b8", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "In Japan, the most popular sashimi is often Maguro (tuna), specifically the fatty part known as Otoro. Globally, Salmon sashimi tends to be more popular due to its rich flavor and widespread availability.\n" + ] + } + ], + "source": [ + "chain = chat_prompt | model | StrOutputParser()\n", + "\n", + "print(\n", + " chain.invoke(\n", + " {\n", + " \"country\": \"Japanese\",\n", + " \"question\": \"What is the most popular Sashimi in Japan vs the rest of the world?\",\n", + " }\n", + " )\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "f4079c76", + "metadata": {}, + "source": [ + "## Streaming Chat" + ] + }, + { + "cell_type": "markdown", + "id": "e6f6a55b", + "metadata": {}, + "source": [ + "Azure OpenAI chat models also support a _Streaming Chat_ feature. This feature allows for text to be received sentence by sentence, rather than waiting for the entire response to arrive." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "b77d7db6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Certainly! The story of Papatuanuku and Ranginui is a central creation myth in Māori mythology, the indigenous belief system of the Māori people of New Zealand. It explains the origins of the world and the natural phenomena within it.\n", + "\n", + "### The Story of Papatuanuku and Ranginui\n", + "\n", + "In the beginning, there was nothing but darkness, a void known as Te Kore. From this void emerged two primordial beings: Ranginui, the Sky Father, and Papatuanuku, the Earth Mother. They lay tightly embraced, their union so close that no light could penetrate between them, and their many children were born in this eternal night.\n", + "\n", + "Their children, who were gods of various natural elements and aspects of life, grew frustrated with the darkness. They longed for space and light. Among these children were Tāne Mahuta, the god of forests and birds; Tangaroa, the god of the sea; Tāwhirimātea, the god of storms and winds; and Tū" + ] + } + ], + "source": [ + "for chunk in model.stream(\"Tell me the story of Papatuanuku and Ranginui\"):\n", + " print(chunk.content, end=\"\", flush=True)" + ] + }, + { + "cell_type": "markdown", + "id": "c4a85fe6-ee18-4fc8-88c2-b67297ac8ef8", + "metadata": {}, + "source": [ + "### Check the costs and token usage of a given model API call" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "id": "c9081591-0b58-44b3-89c7-c3a512f84f4c", + "metadata": {}, + "outputs": [], + "source": [ + "messages = [\n", + " SystemMessage(content=\"Translate the following from German into English\"),\n", + " HumanMessage(content=\"What's the first planet in the Solar System?\"),\n", + "]" + ] + }, + { + "cell_type": "markdown", + "id": "61c9a128-f702-4f48-b695-436629e5ac1a", + "metadata": {}, + "source": [ + "`get_openai_callback()` is a context manager of the [**OpenAICallbackHandler**](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/callbacks/openai_info.py) class, meaning it calls this class and creates an instance when used.\n", + "\n", + "Below is an example of how to use the `get_openai_callback()`. However, to get an accurate estimation of cost, you must pass the model and model version as parameters to the `AzureChatOpenAI` [**constructor**](#creating-an-azurechatopenai-model). " + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "id": "178153e8-c37c-4ce2-bbb4-15ec6feb58f6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Total tokens used: 37\n", + "Total prompt tokens: 27\n", + "Total prompt tokens: 10\n", + "Total cost (in dollars): $0.000285\n", + "Total successful requests): 1\n" + ] + } + ], + "source": [ + "with get_openai_callback() as cb:\n", + " model.invoke(messages)\n", + " print(f\"Total tokens used: {cb.total_tokens}\")\n", + " print(f\"Total prompt tokens: {cb.prompt_tokens}\")\n", + " print(f\"Total prompt tokens: {cb.completion_tokens}\")\n", + " print(f\"Total cost (in dollars): ${cb.total_cost}\")\n", + " print(f\"Total successful requests): {cb.successful_requests}\")" + ] + }, + { + "cell_type": "markdown", + "id": "ab9705c6-0512-48b3-b117-8e1bc385cf86", + "metadata": {}, + "source": [ + "## How to use structured outputs" + ] + }, + { + "cell_type": "markdown", + "id": "e62770fa-c0e4-45ad-8f3b-5fd64e835857", + "metadata": {}, + "source": [ + "Import the required packages. `BaseModel` is a parent class that all tools will inherit from, and `Field` is used to define all properties of the tool." + ] + }, + { + "cell_type": "markdown", + "id": "e7b68b06-ccd1-40f1-9201-9814de191254", + "metadata": {}, + "source": [ + "#### Tools\n", + "\n", + "Tools are essentially classes that can be passed to a chosen model to influence or structure how the response should be formatted or generated.\n", + "\n", + "**For example:**\n", + "\n", + "- A Weather tool with a specific API call could be passed so the model knows to use this specific API for data retrieval.\n", + "- A City tool with fields like `population`, `size`, and `main_language` so the model can return any city-related queries with an object containing the corresponding filled fields.\n", + "- An Image tool with a `url` field to be returned when asked to search for an image containing a dog, with the field containing the URL of the image." + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "05f0ff77-07bd-4041-9ab5-c2ba13ca644c", + "metadata": {}, + "outputs": [], + "source": [ + "class Person(BaseModel):\n", + " \"\"\"Information about a given person\"\"\"\n", + "\n", + " name: str = Field(..., description=\"The name of a person\")\n", + " alive: bool = Field(..., description=\"Whether the person is alive or not\")\n", + " place_of_birth: str = Field(..., description=\"Where the person was born\")\n", + " noteable_features: str = Field(\n", + " ..., description=\"Any noteworthy features/achievements about the person\"\n", + " )\n", + " hobbies: str = Field(..., description=\"Any hobbies the person may have\")" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "id": "c5d9b7cc-d127-4f28-ab8f-829f236ef419", + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Person(name='Kate Sheppard', alive=False, place_of_birth='Liverpool, England', noteable_features=\"Leader of the women's suffrage movement in New Zealand, instrumental in making New Zealand the first country to grant women the right to vote in 1893.\", hobbies='Activism, writing, public speaking')" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "structured_model = model.with_structured_output(Person)\n", + "response = structured_model.invoke(\"Tell me about Kate Sheppard\")\n", + "response" + ] + }, + { + "cell_type": "markdown", + "id": "52eb76bf-283c-4095-aa33-27e772f09292", + "metadata": {}, + "source": [ + "##### As the response of the invocation has been structured using the Person tool, the response can be accessed like a `Person` object." + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "id": "63f05dc8-7832-429c-b710-ceb72a56b492", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Kate Sheppard\n", + "False\n", + "Liverpool, England\n", + "Leader of the women's suffrage movement in New Zealand, instrumental in making New Zealand the first country to grant women the right to vote in 1893.\n" + ] + } + ], + "source": [ + "print(response.name)\n", + "print(response.alive)\n", + "print(response.place_of_birth)\n", + "print(response.noteable_features)" + ] + }, + { + "cell_type": "markdown", + "id": "4d5247e4-c810-4a61-98ee-9534ae10d300", + "metadata": {}, + "source": [ + "#### JSON" + ] + }, + { + "cell_type": "markdown", + "id": "350506c9-8d59-43c5-a5ba-cc2493da2faf", + "metadata": {}, + "source": [ + "Models can also be explicitly told to respond in a JSON structured format. This could then be used for future API calls or for easier access to information. However, **the word \"json\" must be included in the message string.**" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "id": "42478d15-0aa2-4e47-b279-e807397d9f31", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'{\\n \"name\": \"Jane Doe\",\\n \"alive\": true,\\n \"place_of_birth\": \"Springfield, Illinois, USA\"\\n}'" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "json_model = model.bind(response_format={\"type\": \"json_object\"})\n", + "\n", + "response = json_model.invoke(\n", + " \"\"\"Return a JSON object of a random person with features like name,\n", + " alive (if they're alive or not) and their place of birth.\"\"\"\n", + ")\n", + "\n", + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "f9b35d39", + "metadata": {}, + "source": [ + "The response can then be formatted into a JSON object and accessed using normal JSON notation." + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "id": "2dde7260", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'name': 'Jane Doe',\n", + " 'alive': True,\n", + " 'place_of_birth': 'Springfield, Illinois, USA'}" + ] + }, + "execution_count": 47, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "person = json.loads(response.content)\n", + "person" + ] + }, + { + "cell_type": "markdown", + "id": "535b4a4c-b435-40d1-9145-3918661fff7d", + "metadata": {}, + "source": [ + "## Image input" + ] + }, + { + "cell_type": "markdown", + "id": "8d150455-42c3-4943-89fa-d916ef76060b", + "metadata": {}, + "source": [ + "Models can be fed image files as their inputs." + ] + }, + { + "cell_type": "markdown", + "id": "65f20898-73d6-4fed-9433-bb086019f1a7", + "metadata": {}, + "source": [ + "When using data of different types, such as text in the `SystemMessage` and a file in the `HumanMessage`, it's necessary to specify a type header so the model knows how to interpret the data.\n", + "\n", + "Additionally, the URL must be passed directly under the `url` content header, allowing the model to retrieve the image autonomously.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "1536321b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\"The image depicts a stunning mountain landscape featuring a snow-capped peak reflected in a calm, clear lake. The mountains are rugged and majestic, with a mix of snow and rocky terrain. The lake in the foreground is serene, with the reflection of the mountains creating a mirror-like effect on the water's surface. The sky is clear with a few scattered clouds, adding to the overall tranquility and beauty of the scene. The area appears remote and untouched, emphasizing the natural beauty of the mountainous region.\"" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "url = \"https://upload.wikimedia.org/wikipedia/commons/b/bf/Aoraki_Mount_Cook.JPG\"\n", + "\n", + "messages = [\n", + " SystemMessage(content=[{\"type\": \"text\", \"text\": \"describe the image location\"}]),\n", + " HumanMessage(\n", + " content=[\n", + " {\n", + " \"type\": \"image_url\",\n", + " \"image_url\": {\"url\": f\"{url}\"},\n", + " },\n", + " ]\n", + " ),\n", + "]\n", + "response = model.invoke(messages)\n", + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "257b4d2a-5dd4-4487-824f-f19c45595f95", + "metadata": {}, + "source": [ + "## Embeddings" + ] + }, + { + "cell_type": "markdown", + "id": "c5e24a75-b54e-4449-a788-4791047fb5d9", + "metadata": {}, + "source": [ + "Embeddings, particularly `AzureOpenAIEmbeddings`, are a natural language processing technique that converts text into mathematical or vector representations. These representations capture the semantic meaning of words, phrases, or entire texts. This transformation enables Azure OpenAI search services to utilize numerical similarities between texts, returning the most relevant search results for a given query." + ] + }, + { + "cell_type": "markdown", + "id": "9a2589e4-fc30-4216-9ddd-b932715cdc1c", + "metadata": {}, + "source": [ + "#### Setup" + ] + }, + { + "cell_type": "markdown", + "id": "afbc35d6-563a-4e59-9a0b-ab2e4b604c5c", + "metadata": {}, + "source": [ + "Embeddings models use a different Azure resource, this model name will be set below as follows." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "c248118a-9f5d-4720-b7e9-614a2eabe8b4", + "metadata": {}, + "outputs": [], + "source": [ + "EMBEDDINGS_MODEL = os.environ[\"EMBEDDINGS_MODEL\"]" + ] + }, + { + "cell_type": "markdown", + "id": "84777968-457c-4e7d-9386-ef79cf103b3a", + "metadata": {}, + "source": [ + "#### Create a model" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "id": "8d296775-37c1-47f4-a318-cdeb5b4c777c", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[0.029870394617319107, -0.004167019855231047, 0.008153270930051804, -0.011176464147865772, -0.015433868393301964]\n" + ] + } + ], + "source": [ + "E_model = AzureOpenAIEmbeddings(model=EMBEDDINGS_MODEL, azure_endpoint=ENDPOINT)\n", + "\n", + "# random generated text\n", + "text = [\n", + " \"The sun sets behind the mountains, casting a warm glow over the valley below.\",\n", + " \"Advances in artificial intelligence are transforming industries across the globe.\",\n", + " \"The recipe calls for two cups of flour, one teaspoon of baking powder, and a pinch of salt.\",\n", + " \"Exercise is essential for maintaining a healthy body and mind, promoting overall well-being.\",\n", + " \"Space exploration continues to reveal the mysteries of the universe, pushing the boundaries of human knowledge.\",\n", + " \"The novel’s protagonist faces a moral dilemma that challenges their deepest beliefs.\",\n", + " \"Sustainable practices in agriculture are crucial for preserving our environment for future generations.\",\n", + "]\n", + "\n", + "embeddings = E_model.embed_documents(text)\n", + "print(embeddings[0][:5])" + ] + }, + { + "cell_type": "markdown", + "id": "88a151d4-6d6c-4b27-bd87-ce26ea44b096", + "metadata": {}, + "source": [ + "#### Searching" + ] + }, + { + "cell_type": "markdown", + "id": "9a2b8abe-397b-43c9-accc-c589d62bbb61", + "metadata": {}, + "source": [ + "To leverage the power of embeddings we will transform the text into an easily accessable data structure known as a `DataFrame` from the `pandas` library." + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "id": "765025f8-15de-4eee-88d2-6fb6dfd31920", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
0
0The sun sets behind the mountains, casting a w...
1Advances in artificial intelligence are transf...
2The recipe calls for two cups of flour, one te...
3Exercise is essential for maintaining a health...
4Space exploration continues to reveal the myst...
5The novel’s protagonist faces a moral dilemma ...
6Sustainable practices in agriculture are cruci...
\n", + "
" + ], + "text/plain": [ + " 0\n", + "0 The sun sets behind the mountains, casting a w...\n", + "1 Advances in artificial intelligence are transf...\n", + "2 The recipe calls for two cups of flour, one te...\n", + "3 Exercise is essential for maintaining a health...\n", + "4 Space exploration continues to reveal the myst...\n", + "5 The novel’s protagonist faces a moral dilemma ...\n", + "6 Sustainable practices in agriculture are cruci..." + ] + }, + "execution_count": 77, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_text = pd.DataFrame(text)\n", + "df_text" + ] + }, + { + "cell_type": "markdown", + "id": "48b3ad3e-f17c-448a-9a23-d67bc0762709", + "metadata": {}, + "source": [ + "Rename the columns and index to make the `DataFrame` clearer." + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "id": "711f849b-6012-4439-b33a-8b5d0f7d6c41", + "metadata": {}, + "outputs": [], + "source": [ + "df_text.rename(columns={0: \"text\"}, inplace=True)\n", + "df_text.index.name = \"index\"" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "id": "f4113077-f4b2-47ea-96b8-9a9bd2ec17b2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
text
index
0The sun sets behind the mountains, casting a w...
1Advances in artificial intelligence are transf...
2The recipe calls for two cups of flour, one te...
3Exercise is essential for maintaining a health...
4Space exploration continues to reveal the myst...
5The novel’s protagonist faces a moral dilemma ...
6Sustainable practices in agriculture are cruci...
\n", + "
" + ], + "text/plain": [ + " text\n", + "index \n", + "0 The sun sets behind the mountains, casting a w...\n", + "1 Advances in artificial intelligence are transf...\n", + "2 The recipe calls for two cups of flour, one te...\n", + "3 Exercise is essential for maintaining a health...\n", + "4 Space exploration continues to reveal the myst...\n", + "5 The novel’s protagonist faces a moral dilemma ...\n", + "6 Sustainable practices in agriculture are cruci..." + ] + }, + "execution_count": 81, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_text" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "id": "7073a830-2f7d-4af9-9e68-f029ecde4aa8", + "metadata": {}, + "outputs": [], + "source": [ + "df_text[\"Embeddings\"] = df_text[\"text\"].apply(E_model.embed_query)" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "id": "16da7bc1-9a72-4b5e-a309-c1e07af3b82e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textEmbeddings
index
0The sun sets behind the mountains, casting a w...[0.029870394617319107, -0.004167019855231047, ...
1Advances in artificial intelligence are transf...[-0.012673170305788517, -0.020036686211824417,...
2The recipe calls for two cups of flour, one te...[0.012168226763606071, 0.018271654844284058, 0...
3Exercise is essential for maintaining a health...[-0.01102688442915678, -0.007886004634201527, ...
4Space exploration continues to reveal the myst...[0.028899280354380608, -0.01474663894623518, -...
5The novel’s protagonist faces a moral dilemma ...[0.013899882324039936, -0.02602808177471161, -...
6Sustainable practices in agriculture are cruci...[-0.006976415403187275, -0.018390081822872162,...
\n", + "
" + ], + "text/plain": [ + " text \\\n", + "index \n", + "0 The sun sets behind the mountains, casting a w... \n", + "1 Advances in artificial intelligence are transf... \n", + "2 The recipe calls for two cups of flour, one te... \n", + "3 Exercise is essential for maintaining a health... \n", + "4 Space exploration continues to reveal the myst... \n", + "5 The novel’s protagonist faces a moral dilemma ... \n", + "6 Sustainable practices in agriculture are cruci... \n", + "\n", + " Embeddings \n", + "index \n", + "0 [0.029870394617319107, -0.004167019855231047, ... \n", + "1 [-0.012673170305788517, -0.020036686211824417,... \n", + "2 [0.012168226763606071, 0.018271654844284058, 0... \n", + "3 [-0.01102688442915678, -0.007886004634201527, ... \n", + "4 [0.028899280354380608, -0.01474663894623518, -... \n", + "5 [0.013899882324039936, -0.02602808177471161, -... \n", + "6 [-0.006976415403187275, -0.018390081822872162,... " + ] + }, + "execution_count": 85, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_text" + ] + }, + { + "cell_type": "markdown", + "id": "501b5125-e306-44a2-b1ab-d8ad4e4b8f51", + "metadata": {}, + "source": [ + "Here you can see that for each `text` it's corresponding embeddings have been set in the `Embeddings` column in the same row." + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "id": "3351dd97-c5d8-4a5c-b2c8-3b3c6a884538", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[-0.012673170305788517, -0.020036686211824417]" + ] + }, + "execution_count": 89, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "E_model.embed_query(text[1])[:2]" + ] + }, + { + "cell_type": "markdown", + "id": "702e72bb-be6b-477a-9b73-80a419b759ac", + "metadata": {}, + "source": [ + "Now that we have the numerical representation, various Azure services can use this for searching. However, to demonstrate what happens 'under the hood,' we will implement a basic vector search manually." + ] + }, + { + "cell_type": "markdown", + "id": "a2bab3fa-bb18-4228-82fc-6459e8b4ca0f", + "metadata": {}, + "source": [ + "#### Cosine similarity" + ] + }, + { + "cell_type": "markdown", + "id": "aac27f37-e119-4b9a-bf29-be6f68129328", + "metadata": {}, + "source": [ + "Cosine similarity measures the similarity between two vectors by evaluating the cosine of the angle between them. In essence, it determines how close two vector points or lines are to each other. Vectors that are closer in space typically share a closer semantic meaning according to the model. This principle forms the core functionality of vector-based search using embeddings." + ] + }, + { + "cell_type": "markdown", + "id": "bc883247-6423-477d-ad96-18438f4b2156", + "metadata": {}, + "source": [ + "#### Setup\n", + "\n", + "The `numpy` library introduces some conveniant mathematical functions." + ] + }, + { + "cell_type": "code", + "execution_count": 100, + "id": "6655272c-87fa-47d8-b2d3-d97c09d1fec8", + "metadata": {}, + "outputs": [], + "source": [ + "def cosine_similarity(text, query):\n", + " return np.dot(text, query) / (np.linalg.norm(text) * np.linalg.norm(query))" + ] + }, + { + "cell_type": "markdown", + "id": "30a2778c-9068-416e-835d-78935f7b7c9f", + "metadata": {}, + "source": [ + "#### Search" + ] + }, + { + "cell_type": "markdown", + "id": "a325870f-a0fc-4a38-807e-bab86d44d87d", + "metadata": {}, + "source": [ + "Use embeddings to get the vector representation of a query." + ] + }, + { + "cell_type": "code", + "execution_count": 105, + "id": "03325c0a-18a7-40d6-8eed-73ca94094fcb", + "metadata": {}, + "outputs": [], + "source": [ + "query = \"What text talks about space or stars?\"\n", + "query_embedding = E_model.embed_query(query)" + ] + }, + { + "cell_type": "code", + "execution_count": 111, + "id": "2c2ae0a5-c5a2-4587-b414-389a65a3c74e", + "metadata": {}, + "outputs": [], + "source": [ + "df_text[\"Similarity\"] = df_text[\"Embeddings\"].apply(\n", + " lambda text_embedding: cosine_similarity(text_embedding, query_embedding)\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 119, + "id": "f7978cb2-a878-4acd-8ae7-e6abfbf56e16", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textEmbeddingsSimilarity
index
4Space exploration continues to reveal the myst...[0.028899280354380608, -0.01474663894623518, -...0.825995
0The sun sets behind the mountains, casting a w...[0.029870394617319107, -0.004167019855231047, ...0.759168
\n", + "
" + ], + "text/plain": [ + " text \\\n", + "index \n", + "4 Space exploration continues to reveal the myst... \n", + "0 The sun sets behind the mountains, casting a w... \n", + "\n", + " Embeddings Similarity \n", + "index \n", + "4 [0.028899280354380608, -0.01474663894623518, -... 0.825995 \n", + "0 [0.029870394617319107, -0.004167019855231047, ... 0.759168 " + ] + }, + "execution_count": 119, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df_text.sort_values(by=\"Similarity\", ascending=False).head(2)" + ] + }, + { + "cell_type": "markdown", + "id": "8a84d962-f036-442b-a781-b07a872cab42", + "metadata": {}, + "source": [ + "Et voilà! The text with the highest cosine similarity discusses space and space exploration, which closely aligns with our query. We also observe that the second-most similar text mentions the Sun. While it’s not an exact match to our query, the Sun is a star, making it contextually similar and resulting in a relatively high cosine similarity." + ] + }, + { + "cell_type": "markdown", + "id": "771ed802", + "metadata": {}, + "source": [ + "## Next Steps/Additional resources" + ] + }, + { + "cell_type": "markdown", + "id": "db29b1ea", + "metadata": {}, + "source": [ + "#### Azure OpenAI Service\n", + "\n", + "- LangChain Azure OpenAI [Docs](https://python.langchain.com/v0.2/docs/integrations/llms/azure_openai/)\n", + "- LangChain AzureChatOpenAI [Docs](https://python.langchain.com/v0.2/docs/integrations/chat/azure_chat_openai/)\n", + "- LangChain AzureChatOpenAI [API](https://python.langchain.com/v0.2/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html#langchain_openai.chat_models.azure.AzureChatOpenAI)\n", + "- LangChain AzureOpenAIEmbeddings [Docs](https://python.langchain.com/v0.2/docs/integrations/text_embedding/azureopenai/)\n", + "- LangChain AzureOpenAIEmbeddings [API](https://api.python.langchain.com/en/latest/embeddings/langchain_openai.embeddings.base.OpenAIEmbeddings.html)\n", + "\n", + "#### Azure AI Search\n", + "\n", + "- LangChain Azure AI Search [Docs](https://python.langchain.com/v0.2/docs/integrations/vectorstores/azuresearch/)\n", + "- LangChain Azure AI Search [API](https://python.langchain.com/v0.2/api_reference/community/vectorstores/langchain_community.vectorstores.azuresearch.AzureSearch.html)\n", + "\n", + "#### Azure Cosmos DB\n", + "\n", + "- LangChain AzureCosmosDBMongovCore [Docs](https://python.langchain.com/v0.2/docs/integrations/vectorstores/azure_cosmos_db/)\n", + "- LangChain AzureCosmosDBMongovCore [API](https://python.langchain.com/v0.2/api_reference/community/vectorstores/langchain_community.vectorstores.azure_cosmos_db.AzureCosmosDBVectorSearch.html)\n", + "- LangChain AzureCosmosDBNoSQL [Docs](https://python.langchain.com/v0.2/docs/integrations/vectorstores/azure_cosmos_db_no_sql/)\n", + "- LangChain AzureCosmosDBNoSQL [API](https://python.langchain.com/v0.2/api_reference/community/vectorstores/langchain_community.vectorstores.azure_cosmos_db_no_sql.AzureCosmosDBNoSqlVectorSearch.html)\n", + "\n", + "#### Azure AI Document Intelligence\n", + "\n", + "- LangChain AzureAIDocumentIntelligenceLoader [Docs](https://python.langchain.com/docs/integrations/document_loaders/azure_document_intelligence/)\n", + "- LangChain AzureAIDocumentIntelligenceLoader [API](https://python.langchain.com/v0.2/api_reference/community/document_loaders/langchain_community.document_loaders.doc_intelligence.AzureAIDocumentIntelligenceLoader.html)\n", + "\n", + "#### Azure AI Services/Azure AI Vision/Azure AI Speech\n", + "\n", + "- LangChain AzureAIServicesToolkit [Docs](https://python.langchain.com/docs/integrations/tools/azure_ai_services/)\n", + "- LangChain AzureAIServicesToolkit [API](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.azure_ai_services.AzureAiServicesToolkit.html)" + ] + }, + { + "cell_type": "markdown", + "id": "b108a82b", + "metadata": {}, + "source": [ + "## References\n", + "\n", + "- Azure Open AI Embeddings Section [source](https://learn.microsoft.com/en-us/azure/ai-services/openai/tutorials/embeddings?tabs=python-new%2Ccommand-line&pivots=programming-language-python)\n", + "\n", + "- AzureChatOpenAI Structured Output [source](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.6" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}