Deeplake packs

llama_hub/llama_packs/arize_phoenix_query_engine/README.md

@@ -0,0 +1,92 @@
+<center>
+    <p style="text-align:center">
+        <img alt="phoenix logo" src="https://storage.googleapis.com/arize-assets/phoenix/assets/phoenix-logo-light.svg" width="200"/>
+        <br>
+        <a href="https://docs.arize.com/phoenix/">Docs</a>
+        |
+        <a href="https://github.com/Arize-ai/phoenix">GitHub</a>
+        |
+        <a href="https://join.slack.com/t/arize-ai/shared_invite/zt-1px8dcmlf-fmThhDFD_V_48oU7ALan4Q">Community</a>
+    </p>
+</center>
+<h1 align="center">Arize-Phoenix LlamaPack</h1>
+
+This LlamaPack instruments your LlamaIndex app for LLM tracing with [Phoenix](https://github.com/Arize-ai/phoenix), an open-source LLM observability library from [Arize AI](https://phoenix.arize.com/).
+
+## Install and Import Dependencies
+
+
+```python
+!pip install "arize-phoenix[llama-index]" llama-hub html2text
+```
+
+
+```python
+import os
+
+from llama_hub.llama_packs.arize_phoenix_query_engine import ArizePhoenixQueryEnginePack
+from llama_index.node_parser import SentenceSplitter
+from llama_index.readers import SimpleWebPageReader
+from tqdm.auto import tqdm
+```
+
+Configure your OpenAI API key.
+
+
+```python
+os.environ["OPENAI_API_KEY"] = "copy-your-openai-api-key-here"
+```
+
+Parse your documents into a list of nodes and pass to your LlamaPack. In this example, use nodes from a Paul Graham essay as input.
+
+
+```python
+documents = SimpleWebPageReader().load_data(
+    [
+        "https://raw.githubusercontent.com/jerryjliu/llama_index/adb054429f642cc7bbfcb66d4c232e072325eeab/examples/paul_graham_essay/data/paul_graham_essay.txt"
+    ]
+)
+parser = SentenceSplitter()
+nodes = parser.get_nodes_from_documents(documents)
+phoenix_pack = ArizePhoenixQueryEnginePack(nodes=nodes)
+```
+
+Run a set of queries via the pack's `run` method, which delegates to the underlying query engine.
+
+
+```python
+queries = [
+    "What did Paul Graham do growing up?",
+    "When and how did Paul Graham's mother die?",
+    "What, in Paul Graham's opinion, is the most distinctive thing about YC?",
+    "When and how did Paul Graham meet Jessica Livingston?",
+    "What is Bel, and when and where was it written?",
+]
+for query in tqdm(queries):
+    print("Query")
+    print("=====")
+    print(query)
+    print()
+    response = phoenix_pack.run(query)
+    print("Response")
+    print("========")
+    print(response)
+    print()
+```
+
+View your trace data in the Phoenix UI.
+
+
+```python
+phoenix_session_url = phoenix_pack.get_modules()["session_url"]
+print(f"Open the Phoenix UI to view your trace data: {phoenix_session_url}")
+```
+
+You can access the internals of the LlamaPack, including your Phoenix session and your query engine, via the `get_modules` method.
+
+
+```python
+phoenix_pack.get_modules()
+```
+
+Check out the [Phoenix documentation](https://docs.arize.com/phoenix/) for more information!

llama_hub/llama_packs/arize_phoenix_query_engine/__init__.py

@@ -0,0 +1,3 @@
+from llama_hub.llama_packs.arize_phoenix_query_engine.base import ArizePhoenixQueryEnginePack
+
+__all__ = ["ArizePhoenixQueryEnginePack"]

llama_hub/llama_packs/arize_phoenix_query_engine/arize_phoenix_llama_pack.ipynb

@@ -0,0 +1,186 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<center>\n",
+    "    <p style=\"text-align:center\">\n",
+    "        <img alt=\"phoenix logo\" src=\"https://storage.googleapis.com/arize-assets/phoenix/assets/phoenix-logo-light.svg\" width=\"200\"/>\n",
+    "        <br>\n",
+    "        <a href=\"https://docs.arize.com/phoenix/\">Docs</a>\n",
+    "        |\n",
+    "        <a href=\"https://github.com/Arize-ai/phoenix\">GitHub</a>\n",
+    "        |\n",
+    "        <a href=\"https://join.slack.com/t/arize-ai/shared_invite/zt-1px8dcmlf-fmThhDFD_V_48oU7ALan4Q\">Community</a>\n",
+    "    </p>\n",
+    "</center>\n",
+    "<h1 align=\"center\">Arize-Phoenix LlamaPack</h1>\n",
+    "\n",
+    "This LlamaPack instruments your LlamaIndex app for LLM tracing with [Phoenix](https://github.com/Arize-ai/phoenix), an open-source LLM observability library from [Arize AI](https://phoenix.arize.com/)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Install and Import Dependencies"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install \"arize-phoenix[llama-index]\" llama-hub html2text"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "from llama_hub.llama_packs.arize_phoenix_query_engine import ArizePhoenixQueryEnginePack\n",
+    "from llama_index.node_parser import SentenceSplitter\n",
+    "from llama_index.readers import SimpleWebPageReader\n",
+    "from tqdm.auto import tqdm"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Configure your OpenAI API key."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "os.environ[\"OPENAI_API_KEY\"] = \"copy-your-openai-api-key-here\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Parse your documents into a list of nodes and pass to your LlamaPack. In this example, use nodes from a Paul Graham essay as input."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "documents = SimpleWebPageReader().load_data(\n",
+    "    [\n",
+    "        \"https://raw.githubusercontent.com/jerryjliu/llama_index/adb054429f642cc7bbfcb66d4c232e072325eeab/examples/paul_graham_essay/data/paul_graham_essay.txt\"\n",
+    "    ]\n",
+    ")\n",
+    "parser = SentenceSplitter()\n",
+    "nodes = parser.get_nodes_from_documents(documents)\n",
+    "phoenix_pack = ArizePhoenixQueryEnginePack(nodes=nodes)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Run a set of queries via the pack's `run` method, which delegates to the underlying query engine."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "queries = [\n",
+    "    \"What did Paul Graham do growing up?\",\n",
+    "    \"When and how did Paul Graham's mother die?\",\n",
+    "    \"What, in Paul Graham's opinion, is the most distinctive thing about YC?\",\n",
+    "    \"When and how did Paul Graham meet Jessica Livingston?\",\n",
+    "    \"What is Bel, and when and where was it written?\",\n",
+    "]\n",
+    "for query in tqdm(queries):\n",
+    "    print(\"Query\")\n",
+    "    print(\"=====\")\n",
+    "    print(query)\n",
+    "    print()\n",
+    "    response = phoenix_pack.run(query)\n",
+    "    print(\"Response\")\n",
+    "    print(\"========\")\n",
+    "    print(response)\n",
+    "    print()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "View your trace data in the Phoenix UI."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "phoenix_session_url = phoenix_pack.get_modules()[\"session_url\"]\n",
+    "print(f\"Open the Phoenix UI to view your trace data: {phoenix_session_url}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can access the internals of the LlamaPack, including your Phoenix session and your query engine, via the `get_modules` method."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "phoenix_pack.get_modules()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Check out the [Phoenix documentation](https://docs.arize.com/phoenix/) for more information!"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "llmapps",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

llama_hub/llama_packs/arize_phoenix_query_engine/base.py

@@ -0,0 +1,73 @@
+"""
+Arize-Phoenix LlamaPack.
+"""
+
+from typing import TYPE_CHECKING, Any, Dict, List
+
+from llama_index import set_global_handler
+from llama_index.indices.vector_store import VectorStoreIndex
+from llama_index.llama_pack.base import BaseLlamaPack
+from llama_index.schema import TextNode
+
+if TYPE_CHECKING:
+    from phoenix import Session as PhoenixSession
+
+
+class ArizePhoenixQueryEnginePack(BaseLlamaPack):
+    """
+    The Arize-Phoenix LlamaPack show how to instrument your LlamaIndex query
+    engine with tracing. It launches Phoenix in the background, builds an index
+    over an input list of nodes, and instantiates and instruments a query engine
+    over that index so that trace data from each query is sent to Phoenix.
+
+    Note: Using this LlamaPack requires that your OpenAI API key is set via the
+    OPENAI_API_KEY environment variable.
+    """
+
+    def __init__(
+        self,
+        nodes: List[TextNode],
+        **kwargs: Any,
+    ) -> None:
+        """
+        Initializes a new instance of ArizePhoenixQueryEnginePack.
+
+        Args:
+            nodes (List[TextNode]): An input list of nodes over which the index
+            will be built.
+        """
+        try:
+            import phoenix as px
+        except ImportError:
+            raise ImportError(
+                "The arize-phoenix package could not be found. "
+                "Please install with `pip install arize-phoenix`."
+            )
+        self._session: "PhoenixSession" = px.launch_app()
+        set_global_handler("arize_phoenix")
+        self._index = VectorStoreIndex(nodes, **kwargs)
+        self._query_engine = self._index.as_query_engine()
+
+    def get_modules(self) -> Dict[str, Any]:
+        """
+        Returns a dictionary containing the internals of the LlamaPack.
+
+        Returns:
+            Dict[str, Any]: A dictionary containing the internals of the
+            LlamaPack.
+        """
+        return {
+            "session": self._session,
+            "session_url": self._session.url,
+            "index": self._index,
+            "query_engine": self._query_engine,
+        }
+
+    def run(self, *args: Any, **kwargs: Any) -> Any:
+        """
+        Runs queries against the index.
+
+        Returns:
+            Any: A response from the query engine.
+        """
+        return self._query_engine.query(*args, **kwargs)

llama_hub/llama_packs/arize_phoenix_query_engine/requirements.txt

@@ -0,0 +1 @@
+arize-phoenix