05-Adding_Memory.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "01a8b5c0-87cb-4302-8e3c-dc809d0039fb",
   "metadata": {},
   "source": [
    "# Understanding Memory in LLMs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2f73380-6395-4e9f-9c83-3f47a5d7e292",
   "metadata": {},
   "source": [
    "In the previous Notebook, we successfully explored how OpenAI models can enhance the results from Azure Cognitive Search. \n",
    "\n",
    "However, we have yet to discover how to engage in a conversation with the LLM. With [Bing Chat](http://chat.bing.com/), for example, this is possible, as it can understand and reference the previous responses.\n",
    "\n",
    "There is a common misconception that GPT models have memory. This is not true. While they possess knowledge, they do not retain information from previous questions asked to them.\n",
    "\n",
    "In this Notebook, our goal is to illustrate how we can effectively \"endow the LLM with memory\" by employing prompts and context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "733c782e-204c-47d0-8dae-c9df7091ab23",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import random\n",
    "from langchain.chat_models import AzureChatOpenAI\n",
    "from langchain.chains import LLMChain\n",
    "from langchain.prompts import PromptTemplate\n",
    "from langchain.memory import ConversationBufferMemory\n",
    "from openai.error import OpenAIError\n",
    "from langchain.embeddings import OpenAIEmbeddings\n",
    "from langchain.docstore.document import Document\n",
    "from langchain.memory import CosmosDBChatMessageHistory\n",
    "\n",
    "from IPython.display import Markdown, HTML, display  \n",
    "\n",
    "def printmd(string):\n",
    "    display(Markdown(string))\n",
    "\n",
    "#custom libraries that we will use later in the app\n",
    "from common.utils import (\n",
    "    get_search_results,\n",
    "    update_vector_indexes,\n",
    "    model_tokens_limit,\n",
    "    num_tokens_from_docs,\n",
    "    num_tokens_from_string,\n",
    "    get_answer,\n",
    ")\n",
    "\n",
    "from common.prompts import COMBINE_CHAT_PROMPT_TEMPLATE\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(\"credentials.env\")\n",
    "\n",
    "import logging\n",
    "\n",
    "# Get the root logger\n",
    "logger = logging.getLogger()\n",
    "# Set the logging level to a higher level to ignore INFO messages\n",
    "logger.setLevel(logging.WARNING)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6bc63c55-a57d-49a7-b6c7-0f18bca8199e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the ENV variables that Langchain needs to connect to Azure OpenAI\n",
    "os.environ[\"OPENAI_API_BASE\"] = os.environ[\"AZURE_OPENAI_ENDPOINT\"]\n",
    "os.environ[\"OPENAI_API_KEY\"] = os.environ[\"AZURE_OPENAI_API_KEY\"]\n",
    "os.environ[\"OPENAI_API_VERSION\"] = os.environ[\"AZURE_OPENAI_API_VERSION\"]\n",
    "os.environ[\"OPENAI_API_TYPE\"] = \"azure\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dc72b22-11c2-4df0-91b8-033d01829663",
   "metadata": {},
   "source": [
    "### Let's start with the basics\n",
    "Let's use a very simple example to see if the GPT model of Azure OpenAI have memory. We again will be using langchain to simplify our code "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3eef5dc9-8b80-4085-980c-865fa41fa1f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "QUESTION = \"Tell me some use cases for reinforcement learning\"\n",
    "FOLLOW_UP_QUESTION = \"Give me the main points of our conversation\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a00181d5-bd76-4ce4-a256-75ac5b58c60f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define model\n",
    "MODEL = \"gpt-35-turbo\"\n",
    "COMPLETION_TOKENS = 500\n",
    "# Create an OpenAI instance\n",
    "llm = AzureChatOpenAI(deployment_name=MODEL, temperature=0.5, max_tokens=COMPLETION_TOKENS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "9502d0f1-fddf-40d1-95d2-a1461dcc498a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We create a very simple prompt template, just the question as is:\n",
    "prompt = PromptTemplate(\n",
    "    input_variables=[\"question\"],\n",
    "    template=\"{question}\",\n",
    ")\n",
    "\n",
    "chain = LLMChain(llm=llm, prompt=prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c5c9903e-15c7-4e05-87a1-58e5a7917ba2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "1. Game playing: Reinforcement learning has been successfully applied to games such as chess, Go, and poker, where the agent learns to make optimal decisions by playing against itself or human opponents.\n",
       "\n",
       "2. Robotics: Reinforcement learning can be used to train robots to perform complex tasks, such as grasping objects, navigating through environments, or even playing sports.\n",
       "\n",
       "3. Autonomous driving: Reinforcement learning can be used to train self-driving cars to make decisions in real-time, such as lane changing, merging, and avoiding obstacles.\n",
       "\n",
       "4. Recommendation systems: Reinforcement learning can be used to personalize recommendations for users in various domains, such as movies, music, or online shopping, by learning from user feedback and optimizing for user satisfaction.\n",
       "\n",
       "5. Resource management: Reinforcement learning can be used to optimize the allocation of resources in various domains, such as energy management, traffic control, or supply chain management, by learning to make decisions that maximize efficiency and minimize costs.\n",
       "\n",
       "6. Finance: Reinforcement learning can be used to develop trading strategies in financial markets, where the agent learns to make buy/sell decisions based on market data and maximize profit.\n",
       "\n",
       "7. Healthcare: Reinforcement learning can be used to develop personalized treatment plans for patients, where the agent learns to make decisions about medication dosages or treatment options based on patient data and optimize for health outcomes.\n",
       "\n",
       "8. Advertising: Reinforcement learning can be used to optimize online advertising campaigns, where the agent learns to make decisions about ad placements and targeting based on user interactions and maximize click-through rates or conversions.\n",
       "\n",
       "9. Natural language processing: Reinforcement learning can be used to develop conversational agents or chatbots, where the agent learns to generate responses or carry out tasks based on user interactions and optimize for user satisfaction.\n",
       "\n",
       "10. Industrial control: Reinforcement learning can be used to optimize control systems in industries such as manufacturing, energy, or chemical processing, where the agent learns to make decisions that maximize production efficiency and minimize downtime or waste."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Let's see what the GPT model responds\n",
    "response = chain.run(QUESTION)\n",
    "printmd(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "99acaf3c-ce68-4b87-b24a-6065b15ff9a8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'I apologize, but as an AI, I do not have the capability to recall the specific details of our conversation. However, if you provide me with the key topics discussed, I can try to summarize the main points related to those topics.'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Now let's ask a follow up question\n",
    "chain.run(FOLLOW_UP_QUESTION)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3e1c143-c95f-4566-a8b4-af8c42f08dd2",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "As you can see, it doesn't remember what it just responded, sometimes it responds based only on the system prompt, or just randomly. This proof that the LLM does NOT have memory and that we need to give the memory as a a conversation history as part of the prompt, like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0946ce71-6285-432e-b011-9c2dc1ba7b8a",
   "metadata": {},
   "outputs": [],
   "source": [
    "hist_prompt = PromptTemplate(\n",
    "    input_variables=[\"history\", \"question\"],\n",
    "    template=\"\"\"\n",
    "                {history}\n",
    "                Human: {question}\n",
    "                AI:\n",
    "            \"\"\"\n",
    "    )\n",
    "chain = LLMChain(llm=llm, prompt=hist_prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6d088e51-e5eb-4143-b87d-b2be429eb864",
   "metadata": {},
   "outputs": [],
   "source": [
    "Conversation_history = \"\"\"\n",
    "Human: {question}\n",
    "AI: {response}\n",
    "\"\"\".format(question=QUESTION, response=response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d99e34ad-5539-44dd-b080-3ad05efd2f01",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "- Reinforcement learning can be used in various domains such as game playing, robotics, autonomous driving, recommendation systems, resource management, finance, healthcare, advertising, natural language processing, and industrial control.\n",
       "- In game playing, reinforcement learning can help agents make optimal decisions by playing against themselves or human opponents.\n",
       "- In robotics, reinforcement learning can train robots to perform complex tasks like grasping objects or navigating through environments.\n",
       "- In autonomous driving, reinforcement learning can help self-driving cars make real-time decisions like lane changing or avoiding obstacles.\n",
       "- In recommendation systems, reinforcement learning can personalize recommendations for users based on their feedback and optimize for user satisfaction.\n",
       "- In resource management, reinforcement learning can optimize the allocation of resources in domains like energy management or supply chain management.\n",
       "- In finance, reinforcement learning can develop trading strategies based on market data to maximize profit.\n",
       "- In healthcare, reinforcement learning can develop personalized treatment plans for patients based on their data and optimize for health outcomes.\n",
       "- In advertising, reinforcement learning can optimize online advertising campaigns based on user interactions to maximize click-through rates or conversions.\n",
       "- In natural language processing, reinforcement learning can develop conversational agents or chatbots that generate responses or carry out tasks based on user interactions and optimize for user satisfaction.\n",
       "- In industrial control, reinforcement learning can optimize control systems in industries like manufacturing or energy to maximize production efficiency and minimize downtime or waste."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "printmd(chain.run({\"history\":Conversation_history, \"question\": FOLLOW_UP_QUESTION}))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "045e5af6-55d6-4353-b3f6-3275c95db00a",
   "metadata": {},
   "source": [
    "**Bingo!**, so we now know how to create a chatbot using LLMs, we just need to keep the state/history of the conversation and pass it as context every time"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eafd1694-0077-4aa8-bd01-e9f763ce36a3",
   "metadata": {},
   "source": [
    "## Now that we understand the concept of memory via adding history as a context, let's go back to our GPT Smart Search engine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ba257e86-fd90-4a51-a72d-27000913e8c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Since Memory adds tokens to the prompt, we would need a better model that allows more space on the prompt\n",
    "MODEL = \"gpt-35-turbo-16k\"\n",
    "COMPLETION_TOKENS = 1000\n",
    "llm = AzureChatOpenAI(deployment_name=MODEL, temperature=0.5, max_tokens=COMPLETION_TOKENS)\n",
    "embedder = OpenAIEmbeddings(deployment=\"text-embedding-ada-002\", chunk_size=1) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ef9f459b-e8b8-40b9-a94d-80c079968594",
   "metadata": {},
   "outputs": [],
   "source": [
    "index1_name = \"cogsrch-index-files\"\n",
    "index2_name = \"cogsrch-index-csv\"\n",
    "index3_name = \"cogsrch-index-books-vector\"\n",
    "text_indexes = [index1_name, index2_name]\n",
    "vector_indexes = [index+\"-vector\" for index in text_indexes] + [index3_name]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b01852c2-6192-496c-adff-4270f9380469",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of results: 5\n",
      "CPU times: user 1.78 s, sys: 40.2 ms, total: 1.82 s\n",
      "Wall time: 7.8 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "# Search in text-based indexes first and update vector indexes\n",
    "k=10 # Top k results per each text-based index\n",
    "ordered_results = get_search_results(QUESTION, text_indexes, k=k, reranker_threshold=1, vector_search=False)\n",
    "update_vector_indexes(ordered_search_results=ordered_results, embedder=embedder)\n",
    "\n",
    "# Search in all vector-based indexes available\n",
    "similarity_k = 5 # top results from multi-vector-index similarity search\n",
    "ordered_results = get_search_results(QUESTION, vector_indexes, k=k, vector_search=True,\n",
    "                                        similarity_k=similarity_k,\n",
    "                                        query_vector = embedder.embed_query(QUESTION))\n",
    "print(\"Number of results:\",len(ordered_results))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ca500dd8-148c-4d8a-b58b-2df4c957459d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment the below line if you want to inspect the ordered results\n",
    "# ordered_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9b2a3595-c3b7-4376-b9c5-0db7a42b3ee4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of chunks: 5\n"
     ]
    }
   ],
   "source": [
    "top_docs = []\n",
    "for key,value in ordered_results.items():\n",
    "    location = value[\"location\"] if value[\"location\"] is not None else \"\"\n",
    "    top_docs.append(Document(page_content=value[\"chunk\"], metadata={\"source\": location+os.environ['BLOB_SAS_TOKEN']}))\n",
    "        \n",
    "print(\"Number of chunks:\",len(top_docs))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "c26d7540-feb8-4581-849e-003f4bf2a601",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "System prompt token count: 2464\n",
      "Max Completion Token count: 1000\n",
      "Combined docs (context) token count: 924\n",
      "--------\n",
      "Requested token count: 4388\n",
      "Token limit for gpt-35-turbo-16k : 16384\n",
      "Chain Type selected: stuff\n"
     ]
    }
   ],
   "source": [
    "# Calculate number of tokens of our docs\n",
    "if(len(top_docs)>0):\n",
    "    tokens_limit = model_tokens_limit(MODEL) # this is a custom function we created in common/utils.py\n",
    "    prompt_tokens = num_tokens_from_string(COMBINE_CHAT_PROMPT_TEMPLATE) # this is a custom function we created in common/utils.py\n",
    "    context_tokens = num_tokens_from_docs(top_docs) # this is a custom function we created in common/utils.py\n",
    "    \n",
    "    requested_tokens = prompt_tokens + context_tokens + COMPLETION_TOKENS\n",
    "    \n",
    "    chain_type = \"map_reduce\" if requested_tokens > 0.9 * tokens_limit else \"stuff\"  \n",
    "    \n",
    "    print(\"System prompt token count:\",prompt_tokens)\n",
    "    print(\"Max Completion Token count:\", COMPLETION_TOKENS)\n",
    "    print(\"Combined docs (context) token count:\",context_tokens)\n",
    "    print(\"--------\")\n",
    "    print(\"Requested token count:\",requested_tokens)\n",
    "    print(\"Token limit for\", MODEL, \":\", tokens_limit)\n",
    "    print(\"Chain Type selected:\", chain_type)\n",
    "        \n",
    "else:\n",
    "    print(\"NO RESULTS FROM AZURE SEARCH\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "3ce6efa9-2b8f-4810-904d-5986b4ae0372",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Reinforcement learning can be used in various use cases, including:\n",
       "1. Learning prevention strategies in the context of pandemic influenza<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[1]</a></sup>.\n",
       "2. Personalized music recommendation based on capturing changes in listeners' preferences<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[2]</a></sup>.\n",
       "3. Fairness-aware recommendation systems that dynamically maintain a balance between accuracy and fairness<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[3]</a></sup>.\n",
       "4. Computing lockdown decisions for individual cities or regions, considering health and economic considerations<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[4]</a></sup>.\n",
       "5. Modeling epidemics and predicting the spread of diseases, considering individual decisions and external interventions<sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[5]</a></sup>.\n",
       "\n",
       "References:\n",
       "[1] Source: [1]\n",
       "[2] Source: [2]\n",
       "[3] Source: [3]\n",
       "[4] Source: [4]\n",
       "[5] Source: [5]"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 7.51 ms, sys: 347 µs, total: 7.86 ms\n",
      "Wall time: 16.1 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Get the answer\n",
    "response = get_answer(llm=llm, docs=top_docs, query=QUESTION, language=\"English\", chain_type=chain_type)\n",
    "printmd(response['output_text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27501f1b-7db0-4ee3-9cb1-e609254ffa3d",
   "metadata": {},
   "source": [
    "And if we ask the follow up question:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "5cf5b323-3b9c-479b-8502-acfc4f7915dd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "The main points of our conversation are about the use of reinforcement learning in different domains. In the context of pandemic influenza, a deep reinforcement learning approach is used to automatically learn prevention strategies, and collaboration between districts is shown to be advantageous in designing prevention strategies<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[1]</a></sup>. In the domain of music recommendation, a personalized hybrid recommendation algorithm based on reinforcement learning is proposed to recommend song sequences that match listeners' preferences better<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[2]</a></sup>. In the field of interactive recommender systems, a reinforcement learning-based framework called FairRec is proposed to dynamically maintain a balance between accuracy and fairness in recommendation systems<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[3]</a></sup>. Additionally, a quantitative approach is presented for computing lockdown decisions for individual cities or regions, taking into account health and economic considerations, and the policies are learned automatically using reinforcement learning<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[4]</a></sup>. Finally, a microscopic approach is introduced to model epidemics, where individual decisions can affect the spread of the disease, and game theory and multi-agent reinforcement learning are used to optimize individual decisions and make predictions about the spread of the disease<sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[5]</a></sup>."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "response = get_answer(llm=llm, docs=top_docs,  query=FOLLOW_UP_QUESTION, language=\"English\", chain_type=chain_type)\n",
    "printmd(response['output_text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "035fa6e6-226c-400f-a504-30255385f43b",
   "metadata": {},
   "source": [
    "You might get a different response from above, but it doesn't matter what response you get, it will be based on the context given, not on previous answers.\n",
    "\n",
    "Until now we just have the same as the prior Notebook 03: results from Azure Search enhanced by OpenAI model, with no memory\n",
    "\n",
    "**Now let's add memory to it:**\n",
    "\n",
    "Reference: https://python.langchain.com/docs/modules/memory/how_to/adding_memory_chain_multiple_inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "d98b876e-d264-48ae-b5ed-9801d6a9152b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Reinforcement learning has several use cases in various domains. Here are a few examples:\n",
       "\n",
       "1. **Epidemiology**: Reinforcement learning can be used to automatically learn prevention strategies in the context of epidemics, such as pandemic influenza. It can help in designing mitigation policies in complex epidemiological models with a large state space, considering collaboration between districts or regions when designing prevention strategies<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[1]</a></sup><sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[5]</a></sup>.\n",
       "\n",
       "2. **Recommendation Systems**: Reinforcement learning can be applied to personalized recommendation systems, such as music recommendation. It can capture the changes in users' preferences sensitively and enhance the simulation of the interaction process to continuously update the recommendation model based on user preferences for songs and song transitions<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[2]</a></sup>.\n",
       "\n",
       "3. **Fairness in Recommendation**: Reinforcement learning can be used to dynamically maintain a long-term balance between accuracy and fairness in interactive recommender systems. It compresses user preferences and the system's fairness status into a state representation to generate recommendations and aims to maximize a cumulative reward that combines accuracy and fairness<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[3]</a></sup>.\n",
       "\n",
       "4. **Policy Making**: Reinforcement learning can assist in policy-making decisions, such as computing lockdown decisions for individual cities or regions. By balancing health and economic considerations, reinforcement learning algorithms can automatically learn lockdown policies based on disease parameters and population characteristics<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[4]</a></sup>.\n",
       "\n",
       "These are just a few examples, and reinforcement learning can be applied to many other domains and use cases. Let me know if you have any other questions!"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# memory object, which is neccessary to track the inputs/outputs and hold a conversation.\n",
    "memory = ConversationBufferMemory(memory_key=\"chat_history\",input_key=\"question\")\n",
    "\n",
    "response = get_answer(llm=llm, docs=top_docs, query=QUESTION, language=\"English\", chain_type=chain_type, \n",
    "                        memory=memory)\n",
    "printmd(response['output_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "bf28927b-d9ee-4412-bb07-13e055e832a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Here are the main points of our conversation:\n",
       "\n",
       "1. Reinforcement learning has several use cases in various domains.\n",
       "2. In epidemiology, reinforcement learning can be used to automatically learn prevention strategies in the context of epidemics, such as pandemic influenza. It can help design mitigation policies in complex epidemiological models with a large state space, considering collaboration between districts or regions when designing prevention strategies<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[1]</a></sup><sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[5]</a></sup>.\n",
       "3. In recommendation systems, reinforcement learning can be applied to personalized recommendation systems, such as music recommendation. It can capture changes in users' preferences sensitively and enhance the simulation of the interaction process to continuously update the recommendation model based on user preferences for songs and song transitions<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[2]</a></sup>.\n",
       "4. In fairness in recommendation, reinforcement learning can be used to dynamically maintain a long-term balance between accuracy and fairness in interactive recommender systems. It compresses user preferences and the system's fairness status into a state representation to generate recommendations and aims to maximize a cumulative reward that combines accuracy and fairness<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[3]</a></sup>.\n",
       "5. Reinforcement learning can assist in policy-making decisions, such as computing lockdown decisions for individual cities or regions. By balancing health and economic considerations, reinforcement learning algorithms can automatically learn lockdown policies based on disease parameters and population characteristics<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[4]</a></sup>.\n",
       "\n",
       "These are just a few examples, and reinforcement learning can be applied to many other domains and use cases. Let me know if you have any other questions!"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Now we add a follow up question:\n",
    "response = get_answer(llm=llm, docs=top_docs, query=FOLLOW_UP_QUESTION, language=\"English\", chain_type=chain_type, \n",
    "                      memory=memory)\n",
    "printmd(response['output_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "3830b0b8-0ca2-4d0a-9747-f6273368002b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Based on our conversation, here are the main points:\n",
       "\n",
       "1. Reinforcement learning has several use cases in various domains.\n",
       "2. In epidemiology, reinforcement learning can be used to automatically learn prevention strategies in the context of epidemics, such as pandemic influenza. It can help design mitigation policies in complex epidemiological models with a large state space, considering collaboration between districts or regions when designing prevention strategies<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[1]</a></sup><sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[5]</a></sup>.\n",
       "3. In recommendation systems, reinforcement learning can be applied to personalized recommendation systems, such as music recommendation. It can capture changes in users' preferences sensitively and enhance the simulation of the interaction process to continuously update the recommendation model based on user preferences for songs and song transitions<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[2]</a></sup>.\n",
       "4. In fairness in recommendation, reinforcement learning can be used to dynamically maintain a long-term balance between accuracy and fairness in interactive recommender systems. It compresses user preferences and the system's fairness status into a state representation to generate recommendations and aims to maximize a cumulative reward that combines accuracy and fairness<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[3]</a></sup>.\n",
       "5. Reinforcement learning can assist in policy-making decisions, such as computing lockdown decisions for individual cities or regions. By balancing health and economic considerations, reinforcement learning algorithms can automatically learn lockdown policies based on disease parameters and population characteristics<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[4]</a></sup>.\n",
       "\n",
       "These points summarize our discussion on the use cases of reinforcement learning. Let me know if there's anything else I can help with!"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Another follow up query\n",
    "response = get_answer(llm=llm, docs=top_docs, query=\"Thank you\", language=\"English\", chain_type=chain_type,  \n",
    "                      memory=memory)\n",
    "printmd(response['output_text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "111e732b-3c8c-4df3-8fcb-c3d01e7bec74",
   "metadata": {},
   "source": [
    "You might get a different answer on the above cell, and it is ok, this bot is not yet well configured to answer any question that is not related to its knowledge base, including salutations.\n",
    "\n",
    "Let's check our memory to see that it's keeping the conversation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "1279692c-7eb0-4300-8a66-c7025f02c318",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Human: Tell me some use cases for reinforcement learning\\nAI: Reinforcement learning has several use cases in various domains. Here are a few examples:\\n\\n1. **Epidemiology**: Reinforcement learning can be used to automatically learn prevention strategies in the context of epidemics, such as pandemic influenza. It can help in designing mitigation policies in complex epidemiological models with a large state space, considering collaboration between districts or regions when designing prevention strategies<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[1]</a></sup><sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[5]</a></sup>.\\n\\n2. **Recommendation Systems**: Reinforcement learning can be applied to personalized recommendation systems, such as music recommendation. It can capture the changes in users\\' preferences sensitively and enhance the simulation of the interaction process to continuously update the recommendation model based on user preferences for songs and song transitions<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[2]</a></sup>.\\n\\n3. **Fairness in Recommendation**: Reinforcement learning can be used to dynamically maintain a long-term balance between accuracy and fairness in interactive recommender systems. It compresses user preferences and the system\\'s fairness status into a state representation to generate recommendations and aims to maximize a cumulative reward that combines accuracy and fairness<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[3]</a></sup>.\\n\\n4. **Policy Making**: Reinforcement learning can assist in policy-making decisions, such as computing lockdown decisions for individual cities or regions. By balancing health and economic considerations, reinforcement learning algorithms can automatically learn lockdown policies based on disease parameters and population characteristics<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[4]</a></sup>.\\n\\nThese are just a few examples, and reinforcement learning can be applied to many other domains and use cases. Let me know if you have any other questions!\\nHuman: Give me the main points of our conversation\\nAI: Here are the main points of our conversation:\\n\\n1. Reinforcement learning has several use cases in various domains.\\n2. In epidemiology, reinforcement learning can be used to automatically learn prevention strategies in the context of epidemics, such as pandemic influenza. It can help design mitigation policies in complex epidemiological models with a large state space, considering collaboration between districts or regions when designing prevention strategies<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[1]</a></sup><sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[5]</a></sup>.\\n3. In recommendation systems, reinforcement learning can be applied to personalized recommendation systems, such as music recommendation. It can capture changes in users\\' preferences sensitively and enhance the simulation of the interaction process to continuously update the recommendation model based on user preferences for songs and song transitions<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[2]</a></sup>.\\n4. In fairness in recommendation, reinforcement learning can be used to dynamically maintain a long-term balance between accuracy and fairness in interactive recommender systems. It compresses user preferences and the system\\'s fairness status into a state representation to generate recommendations and aims to maximize a cumulative reward that combines accuracy and fairness<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[3]</a></sup>.\\n5. Reinforcement learning can assist in policy-making decisions, such as computing lockdown decisions for individual cities or regions. By balancing health and economic considerations, reinforcement learning algorithms can automatically learn lockdown policies based on disease parameters and population characteristics<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[4]</a></sup>.\\n\\nThese are just a few examples, and reinforcement learning can be applied to many other domains and use cases. Let me know if you have any other questions!\\nHuman: Thank you\\nAI: Based on our conversation, here are the main points:\\n\\n1. Reinforcement learning has several use cases in various domains.\\n2. In epidemiology, reinforcement learning can be used to automatically learn prevention strategies in the context of epidemics, such as pandemic influenza. It can help design mitigation policies in complex epidemiological models with a large state space, considering collaboration between districts or regions when designing prevention strategies<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[1]</a></sup><sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[5]</a></sup>.\\n3. In recommendation systems, reinforcement learning can be applied to personalized recommendation systems, such as music recommendation. It can capture changes in users\\' preferences sensitively and enhance the simulation of the interaction process to continuously update the recommendation model based on user preferences for songs and song transitions<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[2]</a></sup>.\\n4. In fairness in recommendation, reinforcement learning can be used to dynamically maintain a long-term balance between accuracy and fairness in interactive recommender systems. It compresses user preferences and the system\\'s fairness status into a state representation to generate recommendations and aims to maximize a cumulative reward that combines accuracy and fairness<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[3]</a></sup>.\\n5. Reinforcement learning can assist in policy-making decisions, such as computing lockdown decisions for individual cities or regions. By balancing health and economic considerations, reinforcement learning algorithms can automatically learn lockdown policies based on disease parameters and population characteristics<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\" target=\"_blank\">[4]</a></sup>.\\n\\nThese points summarize our discussion on the use cases of reinforcement learning. Let me know if there\\'s anything else I can help with!'"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "memory.buffer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87405173",
   "metadata": {},
   "source": [
    "## Using CosmosDB as persistent memory\n",
    "\n",
    "In previous cell we have added local RAM memory to our chatbot. However, it is not persistent, it gets deleted once the app user's session is terminated. It is necessary then to use a Database for persistent storage of each of the bot user conversations, not only for Analytics and Auditing, but also if we wisg to provide recommendations. \n",
    "\n",
    "Here we will store the conversation history into CosmosDB for future auditing purpose.\n",
    "We will use a class in LangChain use CosmosDBChatMessageHistory, see [HERE](https://python.langchain.com/en/latest/_modules/langchain/memory/chat_message_histories/cosmos_db.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "c7131daa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create CosmosDB instance from langchain cosmos class.\n",
    "cosmos = CosmosDBChatMessageHistory(\n",
    "    cosmos_endpoint=os.environ['AZURE_COSMOSDB_ENDPOINT'],\n",
    "    cosmos_database=os.environ['AZURE_COSMOSDB_NAME'],\n",
    "    cosmos_container=os.environ['AZURE_COSMOSDB_CONTAINER_NAME'],\n",
    "    connection_string=os.environ['AZURE_COMOSDB_CONNECTION_STRING'],\n",
    "    session_id=\"Agent-Test-Session\" + str(random.randint(1, 1000)),\n",
    "    user_id=\"Agent-Test-User\" + str(random.randint(1, 1000))\n",
    "    )\n",
    "\n",
    "# prepare the cosmosdb instance\n",
    "cosmos.prepare_cosmos()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "d87cc7c6-5ef1-4492-b133-9f63a392e223",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create or Memory Object\n",
    "memory = ConversationBufferMemory(memory_key=\"chat_history\",input_key=\"question\",chat_memory=cosmos)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "27ceb47a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Reinforcement learning has several use cases in different domains. Here are a few examples:\n",
       "\n",
       "1. **Epidemic Prevention**: Reinforcement learning can be used to automatically learn prevention strategies in the context of pandemics, such as influenza. By constructing epidemiological models and using deep reinforcement learning algorithms, researchers have shown that reinforcement learning can be used to learn mitigation policies in complex epidemiological models with a large state space<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[1]</a></sup>.\n",
       "\n",
       "2. **Personalized Music Recommendation**: Reinforcement learning can be used to improve personalized music recommendation systems. By simulating the interaction process between listeners and continuously updating the model based on their preferences, reinforcement learning algorithms can recommend song sequences that better match listeners' preferences<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[2]</a></sup>.\n",
       "\n",
       "3. **Fairness in Recommendation Systems**: Reinforcement learning can also be used to improve fairness in recommendation systems. By dynamically maintaining a long-term balance between accuracy and fairness, reinforcement learning algorithms can generate recommendations that consider both user preferences and the system's fairness status<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[3]</a></sup>.\n",
       "\n",
       "4. **Optimizing Lockdown Decisions**: Reinforcement learning can help compute lockdown decisions for individual cities or regions, taking into account health and economic considerations. By automatically learning policies based on disease parameters and population characteristics, reinforcement learning algorithms can provide a quantitative approach towards lockdown decisions<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[4]</a></sup>.\n",
       "\n",
       "5. **Modeling and Predicting Epidemics**: Reinforcement learning can be used to model and predict the spread of infectious diseases, such as COVID-19. By formulating microscopic multi-agent epidemic models and solving for optimal decisions using game theory and multi-agent reinforcement learning, researchers can make predictions about the spread of diseases and identify interventions to regulate agents' behaviors<sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[5]</a></sup>.\n",
       "\n",
       "These are just a few examples of the diverse applications of reinforcement learning. It is a powerful technique that can be applied to various problem domains to learn optimal policies and make informed decisions."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Testing using our Question\n",
    "response = get_answer(llm=llm, docs=top_docs, query=QUESTION, language=\"English\", chain_type=chain_type, \n",
    "                        memory=memory)\n",
    "printmd(response['output_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "1a5ff826",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Based on our conversation, here are the main points:\n",
       "\n",
       "1. Reinforcement learning has several use cases in different domains, including epidemic prevention, personalized music recommendation, fairness in recommendation systems, optimizing lockdown decisions, and modeling and predicting epidemics.\n",
       "\n",
       "2. In the context of epidemic prevention, reinforcement learning can be used to automatically learn prevention strategies in the context of pandemics, such as influenza. Researchers have shown that reinforcement learning can be used to learn mitigation policies in complex epidemiological models with a large state space<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[1]</a></sup>.\n",
       "\n",
       "3. In the field of personalized music recommendation, reinforcement learning algorithms can simulate the interaction process between listeners and continuously update the model based on their preferences, leading to better song sequence recommendations<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[2]</a></sup>.\n",
       "\n",
       "4. Fairness in recommendation systems can be improved using reinforcement learning algorithms that dynamically maintain a balance between accuracy and fairness. These algorithms generate recommendations that consider both user preferences and the system's fairness status<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[3]</a></sup>.\n",
       "\n",
       "5. Reinforcement learning can be used to compute lockdown decisions for individual cities or regions, taking into account health and economic considerations. These decisions are based on disease parameters and population characteristics, and reinforcement learning algorithms provide a quantitative approach towards lockdown decisions<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[4]</a></sup>.\n",
       "\n",
       "6. Reinforcement learning can be used to model and predict the spread of infectious diseases, such as COVID-19. By formulating microscopic multi-agent epidemic models and solving for optimal decisions using game theory and multi-agent reinforcement learning, researchers can make predictions about the spread of diseases and identify interventions to regulate agents' behaviors<sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[5]</a></sup>.\n",
       "\n",
       "These points summarize the main applications of reinforcement learning discussed in our conversation. Let me know if there's anything else I can help with."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Now we add a follow up question:\n",
    "response = get_answer(llm=llm, docs=top_docs, query=FOLLOW_UP_QUESTION, language=\"English\", chain_type=chain_type, \n",
    "                      memory=memory)\n",
    "printmd(response['output_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "be1620fa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Based on the extracted parts from the documents, here are the main points of our conversation:\n",
       "\n",
       "1. Reinforcement learning has several use cases in different domains, including epidemic prevention, personalized music recommendation, fairness in recommendation systems, optimizing lockdown decisions, and modeling and predicting epidemics.\n",
       "\n",
       "2. In the context of epidemic prevention, reinforcement learning can be used to automatically learn prevention strategies in the context of pandemics, such as influenza. Researchers have shown that reinforcement learning can be used to learn mitigation policies in complex epidemiological models with a large state space[1].\n",
       "\n",
       "3. In the field of personalized music recommendation, reinforcement learning algorithms can simulate the interaction process between listeners and continuously update the model based on their preferences, leading to better song sequence recommendations[2].\n",
       "\n",
       "4. Fairness in recommendation systems can be improved using reinforcement learning algorithms that dynamically maintain a balance between accuracy and fairness. These algorithms generate recommendations that consider both user preferences and the system's fairness status[3].\n",
       "\n",
       "5. Reinforcement learning can be used to compute lockdown decisions for individual cities or regions, taking into account health and economic considerations. These decisions are based on disease parameters and population characteristics, and reinforcement learning algorithms provide a quantitative approach towards lockdown decisions[4].\n",
       "\n",
       "6. Reinforcement learning can be used to model and predict the spread of infectious diseases, such as COVID-19. By formulating microscopic multi-agent epidemic models and solving for optimal decisions using game theory and multi-agent reinforcement learning, researchers can make predictions about the spread of diseases and identify interventions to regulate agents' behaviors[5].\n",
       "\n",
       "These points summarize the main applications of reinforcement learning discussed in our conversation. Let me know if there's anything else I can help with.\n",
       "\n",
       "Sources:\n",
       "[1]<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">1</a></sup>\n",
       "[2]<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">2</a></sup>\n",
       "[3]<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">3</a></sup>\n",
       "[4]<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">4</a></sup>\n",
       "[5]<sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">5</a></sup>\n",
       "\n",
       "Let me know if there's anything else I can assist you with."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Another follow up query\n",
    "response = get_answer(llm=llm, docs=top_docs, query=\"Thank you\", language=\"English\", chain_type=chain_type,  \n",
    "                      memory=memory)\n",
    "printmd(response['output_text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdc5ac98",
   "metadata": {},
   "source": [
    "Let's check our Azure CosmosDB to see the whole conversation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "e1d7688a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[HumanMessage(content='Tell me some use cases for reinforcement learning'),\n",
       " AIMessage(content='Reinforcement learning has several use cases in different domains. Here are a few examples:\\n\\n1. **Epidemic Prevention**: Reinforcement learning can be used to automatically learn prevention strategies in the context of pandemics, such as influenza. By constructing epidemiological models and using deep reinforcement learning algorithms, researchers have shown that reinforcement learning can be used to learn mitigation policies in complex epidemiological models with a large state space<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[1]</a></sup>.\\n\\n2. **Personalized Music Recommendation**: Reinforcement learning can be used to improve personalized music recommendation systems. By simulating the interaction process between listeners and continuously updating the model based on their preferences, reinforcement learning algorithms can recommend song sequences that better match listeners\\' preferences<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[2]</a></sup>.\\n\\n3. **Fairness in Recommendation Systems**: Reinforcement learning can also be used to improve fairness in recommendation systems. By dynamically maintaining a long-term balance between accuracy and fairness, reinforcement learning algorithms can generate recommendations that consider both user preferences and the system\\'s fairness status<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[3]</a></sup>.\\n\\n4. **Optimizing Lockdown Decisions**: Reinforcement learning can help compute lockdown decisions for individual cities or regions, taking into account health and economic considerations. By automatically learning policies based on disease parameters and population characteristics, reinforcement learning algorithms can provide a quantitative approach towards lockdown decisions<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[4]</a></sup>.\\n\\n5. **Modeling and Predicting Epidemics**: Reinforcement learning can be used to model and predict the spread of infectious diseases, such as COVID-19. By formulating microscopic multi-agent epidemic models and solving for optimal decisions using game theory and multi-agent reinforcement learning, researchers can make predictions about the spread of diseases and identify interventions to regulate agents\\' behaviors<sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[5]</a></sup>.\\n\\nThese are just a few examples of the diverse applications of reinforcement learning. It is a powerful technique that can be applied to various problem domains to learn optimal policies and make informed decisions.'),\n",
       " HumanMessage(content='Give me the main points of our conversation'),\n",
       " AIMessage(content='Based on our conversation, here are the main points:\\n\\n1. Reinforcement learning has several use cases in different domains, including epidemic prevention, personalized music recommendation, fairness in recommendation systems, optimizing lockdown decisions, and modeling and predicting epidemics.\\n\\n2. In the context of epidemic prevention, reinforcement learning can be used to automatically learn prevention strategies in the context of pandemics, such as influenza. Researchers have shown that reinforcement learning can be used to learn mitigation policies in complex epidemiological models with a large state space<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[1]</a></sup>.\\n\\n3. In the field of personalized music recommendation, reinforcement learning algorithms can simulate the interaction process between listeners and continuously update the model based on their preferences, leading to better song sequence recommendations<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[2]</a></sup>.\\n\\n4. Fairness in recommendation systems can be improved using reinforcement learning algorithms that dynamically maintain a balance between accuracy and fairness. These algorithms generate recommendations that consider both user preferences and the system\\'s fairness status<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[3]</a></sup>.\\n\\n5. Reinforcement learning can be used to compute lockdown decisions for individual cities or regions, taking into account health and economic considerations. These decisions are based on disease parameters and population characteristics, and reinforcement learning algorithms provide a quantitative approach towards lockdown decisions<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[4]</a></sup>.\\n\\n6. Reinforcement learning can be used to model and predict the spread of infectious diseases, such as COVID-19. By formulating microscopic multi-agent epidemic models and solving for optimal decisions using game theory and multi-agent reinforcement learning, researchers can make predictions about the spread of diseases and identify interventions to regulate agents\\' behaviors<sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">[5]</a></sup>.\\n\\nThese points summarize the main applications of reinforcement learning discussed in our conversation. Let me know if there\\'s anything else I can help with.'),\n",
       " HumanMessage(content='Thank you'),\n",
       " AIMessage(content='Based on the extracted parts from the documents, here are the main points of our conversation:\\n\\n1. Reinforcement learning has several use cases in different domains, including epidemic prevention, personalized music recommendation, fairness in recommendation systems, optimizing lockdown decisions, and modeling and predicting epidemics.\\n\\n2. In the context of epidemic prevention, reinforcement learning can be used to automatically learn prevention strategies in the context of pandemics, such as influenza. Researchers have shown that reinforcement learning can be used to learn mitigation policies in complex epidemiological models with a large state space[1].\\n\\n3. In the field of personalized music recommendation, reinforcement learning algorithms can simulate the interaction process between listeners and continuously update the model based on their preferences, leading to better song sequence recommendations[2].\\n\\n4. Fairness in recommendation systems can be improved using reinforcement learning algorithms that dynamically maintain a balance between accuracy and fairness. These algorithms generate recommendations that consider both user preferences and the system\\'s fairness status[3].\\n\\n5. Reinforcement learning can be used to compute lockdown decisions for individual cities or regions, taking into account health and economic considerations. These decisions are based on disease parameters and population characteristics, and reinforcement learning algorithms provide a quantitative approach towards lockdown decisions[4].\\n\\n6. Reinforcement learning can be used to model and predict the spread of infectious diseases, such as COVID-19. By formulating microscopic multi-agent epidemic models and solving for optimal decisions using game theory and multi-agent reinforcement learning, researchers can make predictions about the spread of diseases and identify interventions to regulate agents\\' behaviors[5].\\n\\nThese points summarize the main applications of reinforcement learning discussed in our conversation. Let me know if there\\'s anything else I can help with.\\n\\nSources:\\n[1]<sup><a href=\"https://arxiv.org/pdf/2003.13676v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">1</a></sup>\\n[2]<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206183/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">2</a></sup>\\n[3]<sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206277/?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">3</a></sup>\\n[4]<sup><a href=\"https://arxiv.org/pdf/2003.14093v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">4</a></sup>\\n[5]<sup><a href=\"https://arxiv.org/pdf/2004.12959v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rl&se=2025-11-06T23:27:04Z&st=2023-11-06T15:27:04Z&spr=https&sig=IxmYt1nWtSI0MtBHeQBC1t%2F4VeoN19HqQM1Xu6tvacU%3D\">5</a></sup>\\n\\nLet me know if there\\'s anything else I can assist you with.')]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#load message from cosmosdb\n",
    "cosmos.load_messages()\n",
    "cosmos.messages"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5e30694-ae2a-47bb-a5c7-db51ecdbba1e",
   "metadata": {},
   "source": [
    "![CosmosDB Memory](./images/cosmos-chathistory.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6789cada-23a3-451a-a91a-0906ceb0bd14",
   "metadata": {},
   "source": [
    "# Summary\n",
    "##### Adding memory to our application allows the user to have a conversation, however this feature is not something that comes with the LLM, but instead, memory is something that we must provide to the LLM in form of context of the question.\n",
    "\n",
    "We added persitent memory using CosmosDB.\n",
    "\n",
    "We also can notice that the current chain that we are using is smart, but not that much. Although we have given memory to it, it searches for similar docs everytime, regardless of the input and it struggles to respond to prompts like: Hello, Thank you, Bye, What's your name, What's the weather and any other task that is not search in the knowledge base.\n",
    "\n",
    "\n",
    "## <u>Important Note</u>:<br>\n",
    "As we proceed, while all the code will remain compatible with GPT-3.5 models, we highly recommend transitioning to GPT-4. Here's why:\n",
    "\n",
    "**GPT-3.5-Turbo** can be likened to a 7-year-old child. You can provide it with concise instructions, but it frequently struggles to follow them accurately. Additionally, its limited memory can make sustained conversations challenging.\n",
    "\n",
    "**GPT-3.5-Turbo-16k** resembles the same 7-year-old, but with an increased attention span for longer instructions. However, it still faces difficulties accurately executing them about half the time.\n",
    "\n",
    "**GPT-4** exhibits the capabilities of a 10-12-year-old child. It possesses enhanced reasoning skills and more consistently adheres to instructions. While its memory retention for instructions is moderate, it excels at following them.\n",
    "\n",
    "**GPT-4-32k** is akin to the 10-12-year-old child with an extended memory. It comprehends lengthy sets of instructions and engages in meaningful conversations. Thanks to its robust memory, it offers detailed responses.\n",
    "\n",
    "Understanding this analogy above will become clearer as you complete the final notebook.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c629ebf4-aced-45b7-a6a2-315810d37d48",
   "metadata": {},
   "source": [
    "# NEXT\n",
    "We know now how to do a Smart Search Engine that can power a chatbot!! great!\n",
    "\n",
    "But, does this solve all the possible scenarios that a virtual assistant will require?  **What about if the answer to the Smart Search Engine is not related to text, but instead requires to look into tabular data?** The next notebook explains and solves the tabular problem and the concept of Agents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d4d9da4-3918-4da6-b235-a3320f0dcb12",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}