Introduction_to_Retrieval_Argumented_Generation_(RAG).ipynb

{
  "metadata": {
    "kernelspec": {
      "language": "python",
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.10.13",
      "mimetype": "text/x-python",
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "pygments_lexer": "ipython3",
      "nbconvert_exporter": "python",
      "file_extension": ".py"
    },
    "kaggle": {
      "accelerator": "gpu",
      "dataSources": [],
      "dockerImageVersionId": 30646,
      "isInternetEnabled": true,
      "language": "python",
      "sourceType": "notebook",
      "isGpuEnabled": true
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat_minor": 0,
  "nbformat": 4,
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/nathan-young1/Introduction-to-Retrieval-Argumented-Generation-RAG-/blob/main/Introduction_to_Retrieval_Argumented_Generation_(RAG).ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Retrieval Augmented Generation (RAG)\n",
        "\n",
        "Large language models (LLMs) like ChatGPT are great at understanding language and generating fluent text. However, sometimes they struggle with factual accuracy or keeping information up to date. Retrieval augmented generation (RAG) solves this by adding a \"research assistant\" step:\n",
        "\n",
        "1. **Retrieval**: When you give the LLM a prompt or question, RAG first searches through a database of texts 📜 - like having access to a giant virtual library! It retrieves relevant snippets of information that could be useful for composing its response.\n",
        "\n",
        "2. **Augmentation**: Those retrieved context passages are then incorporated into the prompt to the LLM 📝, giving it an information source to base the answer on. Just like reading research notes from a database and integrating them into your understanding before writing on a topic.\n",
        "\n",
        "3. **Generation**: Finally, the LLM leverages the augmented context to expand its knowledge and language capabilities to generate a response. Making the text produced not just fluent, but also accurate and factual, since it's based on relevant reference material.\n",
        "\n",
        "In essence, RAG reduces the LLM's chance of hallucinating because now it gets to consult a knowledge base before responding. This makes responses more reliable and trustworthy, especially for topics requiring specific up-to-date facts."
      ],
      "metadata": {
        "id": "tk4fMl1KiE5z"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://python.langchain.com/assets/images/vector_stores-125d1675d58cfb46ce9054c9019fea72.jpg\" height=400 width=800/>\n",
        "\n",
        "⭐ Photo credits: [Langchain](https://python.langchain.com/docs/modules/data_connection/vectorstores/)"
      ],
      "metadata": {
        "id": "-KsvDD0jiE53"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Retrieval**\n",
        "\n",
        "To use RAG, we need to have a database of documents that can provide relevant information for our queries. In this tutorial, we will create a database from the book \"How to Build a Career in AI\" by Andrew NG. We will use Langchain, Chroma, and Hugging Face to perform RAG on this book.\n",
        "\n",
        "The process of creating a database involves the following steps:\n",
        "\n",
        "- **Chunking**: We divide the book into smaller pieces, such as paragraphs or sentences, that can be easily indexed and retrieved.\n",
        "\n",
        "- **Embedding**: We use a pre-trained model from Hugging Face to convert each chunk into a vector representation, also known as a sentence embedding. This captures the semantic meaning of the chunk and allows us to compare it with other chunks or queries.\n",
        "\n",
        "💡: For more information on vector embeddings check out the word embeddings section in my last lesson at [Notebook Link](https://www.kaggle.com/code/nathanyoung1/transformer-based-language-translation-in-pytorch). The word embeddings are combined to form sentence embeddings which we will refer to as vector embeddings throughout this tutorial.\n",
        "\n",
        "- **Indexing**: We store the vector embeddings in a vector database, such as Chroma, that can efficiently perform similarity search. This means that given a query vector, we can find the most similar vectors in the database, and retrieve the corresponding chunks.\n",
        "\n",
        "When we want to use RAG to generate a response for a query, we first embed the query using the same model as before. Then, as shown in the image above 👆 we use the vector database to find the most similar embeddings to the query embedding. These similar embeddings are linked to particular chunks of our document. We then fed this chunks as context to the LLM, enabling it to generate a coherent and informative answer."
      ],
      "metadata": {
        "id": "m_WRX5EaiE54"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# install the vector database, langchain, pypdf, hugging face sentence_transformers\n",
        "!pip install chromadb langchain pypdf sentence_transformers"
      ],
      "metadata": {
        "trusted": true,
        "id": "10wKzYNUiE55"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# install langchain experimental features (this will likely be moved to stable in the future).\n",
        "!pip install --quiet langchain_experimental"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:13:33.386151Z",
          "iopub.execute_input": "2024-02-16T01:13:33.386526Z",
          "iopub.status.idle": "2024-02-16T01:13:46.771803Z",
          "shell.execute_reply.started": "2024-02-16T01:13:33.386495Z",
          "shell.execute_reply": "2024-02-16T01:13:46.770295Z"
        },
        "trusted": true,
        "id": "LrLLgwSJiE56"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain_community.document_loaders import PyPDFLoader\n",
        "from langchain.docstore.document import Document\n",
        "\n",
        "# Load the pdf file... By default will split into pages\n",
        "loader = PyPDFLoader(\"How to Build a Career in AI.pdf\")\n",
        "pages = loader.load_and_split()"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:13:58.226771Z",
          "iopub.execute_input": "2024-02-16T01:13:58.227146Z",
          "iopub.status.idle": "2024-02-16T01:14:00.477452Z",
          "shell.execute_reply.started": "2024-02-16T01:13:58.22711Z",
          "shell.execute_reply": "2024-02-16T01:14:00.476578Z"
        },
        "trusted": true,
        "id": "eHJEA7G8iE57"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.embeddings import HuggingFaceBgeEmbeddings\n",
        "\n",
        "# Load an embedding model from hugging face.\n",
        "model_name = \"BAAI/bge-large-en-v1.5\"\n",
        "model_kwargs = {'device': 'cuda'}\n",
        "encode_kwargs = {'normalize_embeddings': True}\n",
        "\n",
        "embed_model = HuggingFaceBgeEmbeddings(\n",
        "    model_name=model_name,\n",
        "    model_kwargs=model_kwargs,\n",
        "    encode_kwargs=encode_kwargs\n",
        ")"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:14:02.502954Z",
          "iopub.execute_input": "2024-02-16T01:14:02.503662Z",
          "iopub.status.idle": "2024-02-16T01:14:20.072514Z",
          "shell.execute_reply.started": "2024-02-16T01:14:02.503627Z",
          "shell.execute_reply": "2024-02-16T01:14:20.071661Z"
        },
        "trusted": true,
        "id": "AAfaxiZyiE57",
        "outputId": "419f8edf-3e77-4d09-f6a6-a3f042be99f8",
        "colab": {
          "referenced_widgets": [
            "d0363b1837804854be0623dc0cf6d64b",
            "d1dc14522e26472486bc025110f5d852",
            "b71258a85aac455b91da31d4484dfa66",
            "f71c65b091524687aa356b16c6e5a57f",
            "beb37a51c913413cbacf79e9378b6bb4",
            "b001c85dfb1540dc8547252b96fd1ee1",
            "e25aa33ddb9147e89390c0f489ae4c2f",
            "83582e9959dd4cdfb42c7455b51d8a54",
            "c239f45729334add8ac1cc5404c42dd1",
            "155f82e51c8844abbec112ff44568c66",
            "3c2679f7d3c1459c93789c230498feb1"
          ]
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "modules.json:   0%|          | 0.00/349 [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d0363b1837804854be0623dc0cf6d64b"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "config_sentence_transformers.json:   0%|          | 0.00/124 [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d1dc14522e26472486bc025110f5d852"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "README.md:   0%|          | 0.00/92.9k [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "b71258a85aac455b91da31d4484dfa66"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "sentence_bert_config.json:   0%|          | 0.00/52.0 [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f71c65b091524687aa356b16c6e5a57f"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "config.json:   0%|          | 0.00/779 [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "beb37a51c913413cbacf79e9378b6bb4"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "model.safetensors:   0%|          | 0.00/1.34G [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "b001c85dfb1540dc8547252b96fd1ee1"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "tokenizer_config.json:   0%|          | 0.00/366 [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "e25aa33ddb9147e89390c0f489ae4c2f"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "83582e9959dd4cdfb42c7455b51d8a54"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "tokenizer.json:   0%|          | 0.00/711k [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c239f45729334add8ac1cc5404c42dd1"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "special_tokens_map.json:   0%|          | 0.00/125 [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "155f82e51c8844abbec112ff44568c66"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "1_Pooling/config.json:   0%|          | 0.00/191 [00:00<?, ?B/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "3c2679f7d3c1459c93789c230498feb1"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain_experimental.text_splitter import SemanticChunker\n",
        "\n",
        "# create a Semantic text splitter for the document, At a high level,\n",
        "# this splits into sentences, then groups into groups of 3 sentences,\n",
        "# and then merges ones that are similar in the embedding space.\n",
        "text_splitter = SemanticChunker(embed_model)\n",
        "\n",
        "# split the pages using Semantic Chunker.\n",
        "documents = text_splitter.split_documents(pages)"
      ],
      "metadata": {
        "trusted": true,
        "id": "ewW7Tt98iE58"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain_community.vectorstores import Chroma\n",
        "\n",
        "# embed and insert all chunks of the documents into the vector database\n",
        "vector_db = Chroma.from_documents(\n",
        "    documents,\n",
        "    embed_model, # model to use for embedding the document chunks before storing.\n",
        "    persist_directory='vector_db', # persist the database in memory.\n",
        "    collection_name='ai_career' # name of the collection to store the chunks in.\n",
        ")"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:15:38.832354Z",
          "iopub.execute_input": "2024-02-16T01:15:38.832711Z",
          "iopub.status.idle": "2024-02-16T01:15:42.705172Z",
          "shell.execute_reply.started": "2024-02-16T01:15:38.83268Z",
          "shell.execute_reply": "2024-02-16T01:15:42.704368Z"
        },
        "trusted": true,
        "id": "nB0v5zgniE59",
        "outputId": "c5a8b681-246a-4c4a-d3d7-b089ca9abac1",
        "colab": {
          "referenced_widgets": [
            "e609aa495ddd4751bad3164ee76ad295"
          ]
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "Batches:   0%|          | 0/3 [00:00<?, ?it/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "e609aa495ddd4751bad3164ee76ad295"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# perform a vector similarity search on a query.\n",
        "query = \"how do i start a career in ai?\"\n",
        "\n",
        "# return the chunks of the most similar five embeddings in the db\n",
        "docs = vector_db.similarity_search(query, k=5)\n",
        "\n",
        "print(docs[4].page_content)"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:15:49.246632Z",
          "iopub.execute_input": "2024-02-16T01:15:49.247473Z",
          "iopub.status.idle": "2024-02-16T01:15:49.327884Z",
          "shell.execute_reply.started": "2024-02-16T01:15:49.247432Z",
          "shell.execute_reply": "2024-02-16T01:15:49.326824Z"
        },
        "trusted": true,
        "id": "evGmPItRiE59",
        "outputId": "408d9e57-01fd-4cf9-a93a-299e1f1184ea",
        "colab": {
          "referenced_widgets": [
            "bfa691ab2dd648658d67da99fbb54800"
          ]
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "Batches:   0%|          | 0/1 [00:00<?, ?it/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "bfa691ab2dd648658d67da99fbb54800"
            }
          },
          "metadata": {}
        },
        {
          "name": "stdout",
          "text": "PAGE 9In the previous chapter, I introduced three key steps for building a career in AI: learning \nfoundational technical skills, working on projects, and finding a job, all of which is supported \nby being part of a community. In this chapter, I’d like to dive more deeply into the first step: \nlearning foundational skills. More research papers have been published on AI than anyone can read in a lifetime. So, when \nlearning, it’s critical to prioritize topic selection. I believe the most important topics for a technical \ncareer in machine learning are:\nFoundational machine learning skills: For example, it’s important to understand models such \nas linear regression, logistic regression, neural networks, decision trees, clustering, and anomaly \ndetection. Beyond specific models, it’s even more important to understand the core concepts \nbehind how and why machine learning works, such as bias/variance, cost functions, regularization, \noptimization algorithms, and error analysis. Deep learning: This has become such a large fraction of machine learning that it’s hard to excel \nin the field without some understanding of it! It’s valuable to know the basics of neural networks, \npractical skills for making them work (such as hyperparameter tuning), convolutional networks, \nsequence models, and transformers. Software development: While you can get a job and make huge contributions with only machine \nlearning modeling skills, your job opportunities will increase if you can also write good software \nto implement complex AI systems. These skills include programming fundamentals, data \nstructures (especially those that relate to machine learning, such as data frames), algorithms \n(including those related to databases and data manipulation), software design, familiarity with \nPython, and familiarity with key libraries such as TensorFlow or PyTorch, and scikit-learn.Math relevant to machine learning:  Key areas include linear algebra (vectors, matrices, and various \nmanipulations of them) as well as probability and statistics (including discrete and continuous \nprobability, standard probability distributions, basic rules such as independence and Bayes’ rule, \nand hypothesis testing). In addition, exploratory data analysis (EDA) — using visualizations and other \nmethods to systematically explore a dataset — is an underrated skill. I’ve found EDA particularly \nuseful in data-centric AI development, where analyzing errors and gaining insights can really help \ndrive progress! Finally, a basic intuitive understanding of calculus will also help.\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Argumentation** ➕\n",
        "\n",
        "Now that we have setup a vector database and can retrieve similar chunks to our query, we are going to combine this chunks together to form a context. This context is then passed together with our query as the prompt to our LLM."
      ],
      "metadata": {
        "id": "57DBvbE8iE59"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# util function to join all retrieved documents chunks together to form a context.\n",
        "def join_retrieved_docs(docs):\n",
        "    return \"\\n\\n\".join([doc.page_content for doc in docs])"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:16:19.258198Z",
          "iopub.execute_input": "2024-02-16T01:16:19.258591Z",
          "iopub.status.idle": "2024-02-16T01:16:19.263506Z",
          "shell.execute_reply.started": "2024-02-16T01:16:19.25856Z",
          "shell.execute_reply": "2024-02-16T01:16:19.262397Z"
        },
        "trusted": true,
        "id": "e1qIOw7SiE5-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain_core.prompts import ChatPromptTemplate\n",
        "\n",
        "# Template so we can attach our context and query as prompt to the LLM on the fly.\n",
        "template = \"\"\"Answer the question using vital information from the following context,\n",
        "if the context is relevant, if context given is not relevant,\n",
        "you should reply >>> 'Sorry, But the context provided doesn't contain enough or relevant\n",
        "information to answer your question'\n",
        "\n",
        ">>> 'Context : {context}'\n",
        "\n",
        ">>> 'Question: {question}'\n",
        "\"\"\"\n",
        "\n",
        "prompt = ChatPromptTemplate.from_template(template)"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:21:41.145318Z",
          "iopub.execute_input": "2024-02-16T01:21:41.146201Z",
          "iopub.status.idle": "2024-02-16T01:21:41.151148Z",
          "shell.execute_reply.started": "2024-02-16T01:21:41.146165Z",
          "shell.execute_reply": "2024-02-16T01:21:41.150279Z"
        },
        "trusted": true,
        "id": "2CAfQAKOiE5-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Generation** ✍️"
      ],
      "metadata": {
        "id": "-gilvu95iE5-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.llms import HuggingFaceHub\n",
        "import os\n",
        "\n",
        "os.environ[\"HUGGINGFACEHUB_API_TOKEN\"] = \"<Your Hugging Face API Token Here>\"\n",
        "# Note: To get an API Token, sign up to hugging face -> go to settings -> access token\n",
        "# -> create new token -> copy the token and paste above 👆."
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:21:43.903142Z",
          "iopub.execute_input": "2024-02-16T01:21:43.90406Z",
          "iopub.status.idle": "2024-02-16T01:21:43.909088Z",
          "shell.execute_reply.started": "2024-02-16T01:21:43.904018Z",
          "shell.execute_reply": "2024-02-16T01:21:43.908041Z"
        },
        "trusted": true,
        "id": "4iPdKOupiE5-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "repo_id = \"google/flan-t5-xxl\" # we will be using google flan LLM from hugging face.\n",
        "\n",
        "llm = HuggingFaceHub(\n",
        "    repo_id=repo_id,\n",
        "    # params for LLM text generation\n",
        "    model_kwargs={\n",
        "        \"temperature\": 0.1 # low values means more precise generation.\n",
        "    }\n",
        ")"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:21:46.296521Z",
          "iopub.execute_input": "2024-02-16T01:21:46.296884Z",
          "iopub.status.idle": "2024-02-16T01:21:46.628371Z",
          "shell.execute_reply.started": "2024-02-16T01:21:46.296857Z",
          "shell.execute_reply": "2024-02-16T01:21:46.627159Z"
        },
        "trusted": true,
        "id": "f6kax7yuiE5-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Import the LLMChain class from langchain.chains module\n",
        "from langchain.chains import LLMChain\n",
        "\n",
        "# Define a function that takes a question, a language model, and a prompt template as arguments\n",
        "def query_llm_with_context(question, llm, prompt_template):\n",
        "\n",
        "    # Use the vector database to find the most similar document chunks to the question\n",
        "    # The parameter k specifies the number of document chunks to retrieve\n",
        "    context = vector_db.similarity_search(question, k=5)\n",
        "\n",
        "    # Create an instance of the LLMChain class\n",
        "    # The prompt parameter specifies the format of the input for the language model\n",
        "    # The llm parameter specifies the name of the language model to use\n",
        "    llm_chain = LLMChain(\n",
        "        prompt=prompt_template,\n",
        "        llm=llm\n",
        "    )\n",
        "\n",
        "    # Invoke the LLMChain instance with the input dictionary\n",
        "    # The input dictionary contains the context and the question keys\n",
        "    # The context key contains the concatenated document chunks retrieved from the vector database\n",
        "    # The question key contains the original question to the LLM\n",
        "    llm_response = llm_chain.invoke(\n",
        "        input = {\n",
        "            'context' : join_retrieved_docs(context),\n",
        "            'question' : question\n",
        "                })\n",
        "\n",
        "    # Return the text of the response generated by the language model\n",
        "    return llm_response['text']\n"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:21:49.432681Z",
          "iopub.execute_input": "2024-02-16T01:21:49.433054Z",
          "iopub.status.idle": "2024-02-16T01:21:49.439682Z",
          "shell.execute_reply.started": "2024-02-16T01:21:49.433025Z",
          "shell.execute_reply": "2024-02-16T01:21:49.438591Z"
        },
        "trusted": true,
        "id": "UOr0mrE6iE5-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Testing**"
      ],
      "metadata": {
        "id": "pbkCIbCQiE5_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "query = 'what is recommended for new ai startups to do?'\n",
        "\n",
        "query_llm_with_context(question=query, llm=llm, prompt_template=prompt)"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:21:52.937877Z",
          "iopub.execute_input": "2024-02-16T01:21:52.93859Z",
          "iopub.status.idle": "2024-02-16T01:21:56.795055Z",
          "shell.execute_reply.started": "2024-02-16T01:21:52.938556Z",
          "shell.execute_reply": "2024-02-16T01:21:56.794092Z"
        },
        "trusted": true,
        "id": "7RH7Hwu0iE5_",
        "outputId": "2700535c-548d-4c2e-90cf-f63fa74df461",
        "colab": {
          "referenced_widgets": [
            "2afd29bce9274588bce569cf7152beca"
          ]
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "Batches:   0%|          | 0/1 [00:00<?, ?it/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "2afd29bce9274588bce569cf7152beca"
            }
          },
          "metadata": {}
        },
        {
          "execution_count": 33,
          "output_type": "execute_result",
          "data": {
            "text/plain": "'Identify a business problem (not an AI problem). I like to find a domain expert and ask, “What are the top three things that you wish worked better? Why aren’t they working yet?” For example, if you want to apply AI to climate change, you might discover that power-grid operators can’t accurately predict how much power intermittent sources like wind and solar might generate in the future. Brainstorm AI solutions.'"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "query2 = 'What is Russian Roulette'\n",
        "\n",
        "query_llm_with_context(question=query2, llm=llm, prompt_template=prompt)"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2024-02-16T01:22:00.0385Z",
          "iopub.execute_input": "2024-02-16T01:22:00.038862Z",
          "iopub.status.idle": "2024-02-16T01:22:01.181826Z",
          "shell.execute_reply.started": "2024-02-16T01:22:00.038834Z",
          "shell.execute_reply": "2024-02-16T01:22:01.180845Z"
        },
        "trusted": true,
        "id": "oKN5QL4eiE5_",
        "outputId": "66b2074b-1fd7-4cf0-d247-790190ca1aea",
        "colab": {
          "referenced_widgets": [
            "4e00998979034fe99a432ad701a8d40c"
          ]
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "Batches:   0%|          | 0/1 [00:00<?, ?it/s]",
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "4e00998979034fe99a432ad701a8d40c"
            }
          },
          "metadata": {}
        },
        {
          "execution_count": 34,
          "output_type": "execute_result",
          "data": {
            "text/plain": "\"Sorry, But the context provided doesn't contain enough or relevant information to answer your question\""
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "👆👆 As you can see in the first example the LLM used our context as the source to generate an answer for us. While in the second example, instead of hallucinating the LLM simply replied that there isn't enough or revelant context to answer us."
      ],
      "metadata": {
        "id": "JOCH5UFIiE5_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Final Words**\n",
        "This tutorial simply introduced you to a RAG techniques & implementation, in production more complex RAG techinques like Sentence-Window retrival, Auto-merging retrival e.t.c are used to improve context relevance. We also use tools 🏹 like TruERA for LLM response Evaluation.\n",
        "\n",
        "**Congratulations** 🎉🎉\n",
        "You can now use fundermental RAG techniques. 😊\n",
        "\n",
        "Follow me on:\n",
        "\n",
        "* **[LinkedIn Profile](https://www.linkedin.com/in/jonathan-okorie-843126216/)** for questions, deep learning projects, chat e.t.c.\n",
        "\n",
        "* **[Twitter Profile](https://twitter.com/Nathan_Young_1)** for bite-sized knowledge & (questionable) puns.\n",
        "\n",
        "* **[Kaggle Profile](https://www.kaggle.com/nathanyoung1)** to be notified when i create a new detailed notebook explanation."
      ],
      "metadata": {
        "id": "UiHyhdQaiE5_"
      }
    }
  ]
}