LLM_Inference_with_llama_cpp_python__Llama_2_13b_chat.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/R3gm/InsightSolver-Colab/blob/main/LLM_Inference_with_llama_cpp_python__Llama_2_13b_chat.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ihaZyg2aYQpH"
      },
      "source": [
        "# llama-cpp-python\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7MDuvLwarQO8"
      },
      "source": [
        "The Python package provides simple bindings for the llama.cpp library, offering access to the C API via ctypes interface, a high-level Python API for text completion, OpenAI-like API, and LangChain compatibility. It supports multiple BLAS backends for faster processing and includes both high-level and low-level APIs, along with web server functionality.\n",
        "\n",
        "`llama.cpp`'s objective is to run the LLaMA model with 4-bit integer quantization on MacBook. It is a plain C/C++ implementation optimized for Apple silicon and x86 architectures, supporting various integer quantization and BLAS libraries. Originally a web chat example, it now serves as a development playground for ggml library features.\n",
        "\n",
        "`GGML`, a C library for machine learning, facilitates the distribution of large language models (LLMs). It utilizes quantization to enable efficient LLM execution on consumer hardware. GGML files contain binary-encoded data, including version number, hyperparameters, vocabulary, and weights. The vocabulary comprises tokens for language generation, while the weights determine the LLM's size. Quantization reduces precision to optimize resource usage."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The GGML format has been replaced by GGUF, effective as of August 21st, 2023. Starting from this date, llama.cpp will no longer provide compatibility with GGML models. This notebook uses `llama-cpp-python==0.1.78, which is compatible with GGML Models`."
      ],
      "metadata": {
        "id": "V2WYqinVm8DB"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OBnzlAqYZM6_"
      },
      "source": [
        "| Code Credits | Link |\n",
        "| ----------- | ---- |\n",
        "| 🎉 llama-cpp-python | [![GitHub Repository](https://img.shields.io/github/stars/abetlen/llama-cpp-python?style=social)](https://github.com/abetlen/llama-cpp-python) |\n",
        "| 🎉 llama.cpp | [![GitHub Repository](https://img.shields.io/github/stars/ggerganov/llama.cpp?style=social)](https://github.com/ggerganov/llama.cpp) |\n",
        "| 🎉 GGML | [![GitHub Repository](https://img.shields.io/github/stars/ggerganov/ggml?style=social)](https://github.com/ggerganov/ggml) |\n",
        "| 🚀 Online inference | [![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/ysharma/Explore_llamav2_with_TGI) |\n",
        "| 🚀 Online inference | [![Replicate](https://replicate.com/google-research/frame-interpolation/badge)](https://replicate.com/replicate/llama70b-v2-chat)\n",
        " |\n",
        "| 🔥 Discover More Colab Notebooks | [![GitHub Repository](https://img.shields.io/badge/GitHub-Repository-black?style=flat-square&logo=github)](https://github.com/R3gm/InsightSolver-Colab/) |\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6rkl4zg7nLhJ"
      },
      "source": [
        "\n",
        "The library works the same with a CPU, but the inference can take about three times longer compared to using it on a GPU.\n",
        "\n",
        "If you want to use only the CPU, you can replace the content of the cell below with the following lines.\n",
        "```\n",
        "# CPU llama-cpp-python\n",
        "!pip install llama-cpp-python==0.1.78\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rkBmY3vQvRSw"
      },
      "outputs": [],
      "source": [
        "# GPU llama-cpp-python\n",
        "!CMAKE_ARGS=\"-DLLAMA_CUBLAS=on\" FORCE_CMAKE=1 pip install llama-cpp-python==0.1.78 --force-reinstall --upgrade --no-cache-dir --verbose"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b_nBHTYSoIWV",
        "outputId": "f504e262-9fc8-4f45-e87d-827a2a7c707e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting huggingface_hub\n",
            "  Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface_hub) (3.12.2)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface_hub) (2023.6.0)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface_hub) (2.31.0)\n",
            "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub) (4.66.1)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub) (6.0.1)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub) (4.7.1)\n",
            "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub) (23.1)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub) (3.2.0)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub) (3.4)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub) (2.0.4)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub) (2023.7.22)\n",
            "Installing collected packages: huggingface_hub\n",
            "Successfully installed huggingface_hub-0.16.4\n"
          ]
        }
      ],
      "source": [
        "# For download the models\n",
        "!pip install huggingface_hub"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R11KqY7lW0yv"
      },
      "source": [
        "# Select the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CzJkCoRRbICP"
      },
      "source": [
        "\n",
        "First, we need to specify the model to use. In Colab with T4 GPU, we can run models of up to 20B of parameters with all optimizations, but this may degrade the quality of the model's inference. The library can run GGML models on a CPU."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hHAz1yGZb4lq"
      },
      "source": [
        "In this case, we will use a [Llama 2 13B-chat](https://huggingface.co/meta-llama/Llama-2-13b-chat) The Llama 2 is a collection of pretrained and fine-tuned generative text models, ranging from 7 billion to 70 billion parameters, designed for dialogue use cases. It outperforms open-source chat models on most benchmarks and is on par with popular closed-source models in human evaluations for helpfulness and safety.\n",
        "\n",
        "![asd](https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc24dac6d-6b5e-4b5f-938c-05951c938a9e_1085x543.png)\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PdmNwEqjRcPY"
      },
      "source": [
        "# Model quantization"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tx-5Fgx80UXj"
      },
      "source": [
        "We can quantize the model using this library, but for practical purposes, it is better to use pre-quantized models. The resulting model is only compatible with libraries that support GGML.\n",
        "\n",
        "\n",
        "```\n",
        "# obtain the original LLaMA model weights and place them in ./models\n",
        "ls ./models\n",
        "65B 30B 13B 7B tokenizer_checklist.chk tokenizer.model\n",
        "\n",
        "# install Python dependencies\n",
        "python3 -m pip install -r requirements.txt\n",
        "\n",
        "# convert the 7B model to ggml FP16 format\n",
        "python3 convert.py models/7B/\n",
        "\n",
        "# quantize the model to 4-bits (using q4_0 method)\n",
        "./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin q4_0\n",
        "\n",
        "# run the inference\n",
        "./main -m ./models/7B/ggml-model-q4_0.bin -n 128\n",
        "```\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GSeedwAFSay9"
      },
      "source": [
        "#  Quantized Models from the Hugging Face Community"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4QwfjGFYRM4g"
      },
      "source": [
        "The Hugging Face community provides quantized models, which allow us to efficiently and effectively utilize the model on the T4 GPU. It is important to consult reliable sources before using any model.\n",
        "\n",
        "There are several variations available, but the ones that interest us are based on the GGLM library.\n",
        "\n",
        "We can see the different variations that Llama-2-13B-GGML has [here](https://huggingface.co/models?search=llama%202%20ggml).\n",
        "\n",
        "\n",
        "\n",
        "In this case, we will use the model called [Llama-2-13B-chat-GGML](https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4TbhbyD9wIdy"
      },
      "source": [
        "![22.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAToAAALICAIAAABgkquEAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAHSmSURBVHhe7b17XFTHuvepmHuMgDdQQUUFFCJKwASNgpeAF4gIijbh5o0QiSSKbHaI7kSIF2IIURQNihIwwtHIRuXdBHICmy2EKFv0AHFgkzl9eOfF15nx837eOfOZmX/3PFW17r266W4a6NU8+Xw/lepnVdWqVat+66nqxvWMe9V5BoIgmgDliiCaAeWKIJoB5YogmgHliiCaAeWKIJphFOT6PzwWIIijopjttsU2clX0GEGQQVGIyBwslqvilAiC2BCF3BSYK1dFowiCDCsKATLMkquiIQRBRgCFDIGRlquiZTPxXPNB+JplCqMBvn7RH6wM8TWwm8WbKSd2RSiNapTeffrsicAvpQYFEOvI+vM/yJDePa+wawnFbB8iisYBpVxfmeT+wisuE+a+On7LC07JL0x485XnJ7soWjHk0Gdfiry/HtpZk1dTmpcibXkw1m/mWjiqS/ogPHK9u7t41LzWQncV13x5INTAbg4JJffuFqwWPjJNtp8VCwjYo1zTb/+72KV/1KUbFDCO5HKU12L0Ss/+IrFTbKGxIcqVVf/3P2co7IMivUyD6ufb+UOygeVHgJ8hGXU9tAzXeVZrsLmhEJEhivKATK4vTZ36vPOrTu8+P/7W+PF948cPOI3/DyensglGWhEEVvTllZoLxUVDk2vKH6/WFH7FNXi0pKb0aqkumDs67HKNuNJ07/to0WJCrjzm3ZIRwFA8ZiuWUwiH9Fr4+SeDL6ByRmt0omBU5Cp/JCkuRKpVBj+wwgj03M6SfrRIrsCyFeH5py+npB5kGBGaiCjXV9zcJsS+OGHp8+OLncZ3EqGO/69ORLH/1WmwVlR0Yp1c/7hN+LhMd7rmwmdx7ONwy3Xl6butlxMkFu3IldOVkce8SQQX8WfmnCXXwg6xuUjgJi53FnZUOIWysJWMgly5VYlChNw4cP3hr4sTNuseP3Ssrqh5K+QKKmWZ4BXvDCY0iVyf/+DVCRufH7/eafxVJ6cHE8b/L07j/+H0/L+/OOnMpMFaMSbXD3ySvgTHW3r1xpc5ce7C0SlLQw4UFV4l9sKT2Uu4Ra9Crp7hJ6GF99hHuVznLdldeJw0W3PhdOHmVfN4u6wbPh9eKDp9bKUXPeQTs+vkNShfeuXakQ/XO3PlBd468fNvZalSyxDkKvNLykf13fNCy4rZAIh3nZbkK5ok/XwpTJc/i42YPek5BZKS3Kw1Mb1YYfm0Fk6h+Dgo4lyX1RJ7LhSQPwKMjA9/IQrMeHwYjhWc4t//fLuU3HfFcxCAE/2j7s/nySjRHnKPuZ5/QAq12P3lS0Jds+Uq+FiF0MCuKC/KdULIy04uL41fMt4tfPo7F9+Zfs9t6oNpGysj5wXNU7QiVOFRl+uFKze+zPvAz2+ZT1Jh0dUbH0V60kO+K/NulBYfDVni6+yxbvPJGxfyUqYSu0Su7v4+kUe/vHotdQ2rIpWrp9+B70qvfpcaGerpFxqSVQr5XSGsmKQbIYe/FOwe7x26UvNlVpyXx7ypIRlHrtz44zb511GLi+of3tgltVgtV5U1JK9YWp7eWmK/+4swydhZZHORIZlGZsN1wCwnAyplc9qoXKWXI1lgS/XGMLurnELU6vLPL36IKMKFGB8f6+XK2jQyVqxZyVVLYXI9K4zPP+rO0jG0Wq7MxyqEZkquThtedHJ+ZfyGCeNeG8f+Gz9+PMsoWhGq8BjxricFf7hOLBAMQirV+fEl/TKPXy0KnwJ5IlfiAHkKwTnz3zaJcvX44MjVmqO7/bnqzv6bvoITfUAFz3fDK+WPRJNcGR+Q9+lsL678DM/3SyUdIyw69nP79UNSi9VyTT/f3iO5/WxaS28hmUPsHtOK3F0nR+t6/iFOeqYf6YRTKEQoKYeb07KZqpjNanPILLkaXJcc6VgpNCnRA9cZvjBrn5MEL1eu8/KPg42PSe0Z64+0Cjd0FHZHaFdZ36RjKN44UlG4xaxL0nutOtQKrJfr82kvOQW8+PykieM9nhv32vhxL4yb8Py4ZZMmzp75kqIVoQqPEbmKy1exwNTdF0qvfPdH7jsq4EIhuEHylZJsMezstR4cb2nxYT+iZElra44WceU5iBpLji4Rz5ICaelXmZ5cAV/Qs/g1GPDVNWjWh6/+qrP3pzW/XftQ+MiwUq6A4VSW3kKap42Ld507C6cZCZbJVZhSii5Jp5rhUYpRuQpwjXBdFeYoO2pQ3ahc1WQmwNVSNCsUNj0+oyVXeoiUZ92T3mtT48kzbeaCud4BVsn1ZZfnX5g0ecrMCS+84jTlRad5z73g9fwHU17OdJ2iaEWowmOBXIlzu1KUmPRBuMh7fh5QRrF3BUeacYSsh0lebC3yywuGcr3yZQjJk7OAWy7KO/bHKzVHOA9MvrIqOpktOd0H4dHrqDemzDrxw+O6T2fxHzmslKvsnimmNS1P82pypUcVU9PItFZBmMrmV5HCVVdcS4ZRMSjkyg2FkXWjFJPXZVKug4+PrIfmoTyj7L5zKwvJHJDccVGuzllnz5NusC5J77U5cmVYI9eJzu5v+U9/fY7z67MnrVs6dcmCKb6erqtnu+Qvna1oRajCY4Fcqd4ubGbfAMkwkKtf9nGy/yR5sTWyeBa2wYDnys9ulJ5mvpTKlTpk921FF64UhdOzhECBrzLEL7oUfHirveaEm8JopVwVt1/+kZaneRW5MgEI80/x0TSssDlqMYaBXLmeix1gF2tErlx1czrAtcMPrGRfYCgeqSbNGB8r5GowdLLusQaVJ5XcONm5RlqugPdctyO61w5FT86Idtkf6xy3ZsqhmClFu10VrUirUCyRq3NoYjG4uwxuX+q+zC+YuUGJXKf4uq967yOivWy2apW0xr6pOhbiQxTrHnH0y6s3DkWzr46kZyF72qK894gXDc6GMkd2h7IvhJ391vtIHhZp13+7c+wt4SMPd6vkyO6iAnafuDsqGPnvk8hRk3LlZjxPzy+wB6YZ2YxUQ7a9lGDGXFGclAdmYVY6/Z5TCd+m4jIZgsxMwmlSSs8vpVQtpuRqxvgoWx586AD10eOeJurjY75cFRi/IyDLQ5+eMPwhZxC5ApOnuEctd1nhPzF146trl0zavnKibrVt5TrjVa+Y1NM3YNV6oQTSG19mxVAhEbkK3zOVXrlx/DPxqyYvWEJfrTm0jTrVKcs35dFfZQjXjry/jmtWcRbqh9kXTqBq+sPPjSJIr3yXGiH89pNR9vDuicUsL8VKuUonDZ0uXGFy1KRc+TyFHOLaGSW50gLylgUVAYZyNUsbPNLqkmZNyRUYfHzkIjG3S2ItuHB2v7g7AognBQ/MSoo3zmZyBUCZgEJosLNVFFPKVRVFK4qjVuKx1NNv6VT6TZIZeE71WSb9s0QLqwOkBU8f+U84qTfaf/52pdSCIKPHoEIbPbnaAdGXH9af3qgwIshoMajQrJGrKooqWmBd1uUbn5r1r3AQxMYo5KOKogpgM7lah+JECKI5FFPahihOBJglV0DREIIgw41Cg4C5cgUUbSEIMkwopCdggVwFFE0jCGITFEIzxBq5GkNxbgRBVFEIx3xsKVcrUFwGgmgOxZQeVkZZrgiCmA/KFUE0A8oVQTQDyhVBNAPKFUE0A8oVQTSDilwnuni4TPeePMPP1X3RVI8AxExc3BfCoDlP957oMksxpIaM5UF2+IEavgtUynXSlLlTZvq7uvm4TF/gMn0+YhluC2DoYABhGBUDKwUH2fEHanguUCbXSVPnTZ6xSFECsQIYRhhM6dgK4CBLcfiBsu0FinIFpwxCVxxGrAYG03AthINsiMMPlA0vUJQrLKCpU1aWQKwDBhN2L8Lw4iAbw+EHyoYXKMoVNru4X7UlbgtgSIXhxUE2isMPlO0uUJSrq/tCxTFkiMCQCsNr5iBHhnh8ssUdgIzi0PATPM9XYRkhrBgoYPbM+dt9vRVG+8SKC5w0f9GkpYGE+eL+VpTrVI8AwSpl5tzQrTvaN8c0CcBHvyV7FcXMIuiD7JOn80SO6YIMygxOQEBSDqmem7HaS3HI1oSfv9NYHqswmg0MqTC8pgcZiArx+PnI5K7yGf/smv2fLR6QuZ09ddXS2YpiAhH7pCN5Om/fdkUBy4i8VP/o0Z0fy9MV9hHBooFigFZb1/hlLvZV2FXhx+rU3pjweQZHhzotd1S13j6lNMqx6AJBny/veffFP+gYLx3YCh+ZaAeXq7ffjuWhst4YWsyFG5fqpsf3vzNrXE7dvF8lV0tk9g+P2hur86H62fqmH8+vkB21Cbsv3q8/wvJeiem5GVafwvybBLJsPDEdhKqg6ejUAN+5isIMNgXP/fi4/cdyMsmGJtfVZ1tbyz5QGEcMi2YzwLR66S3ioD4P9AWkRw05cvv3pmswXOcv3r7f2mLw/LV4WsqxqVwnLQp4MXvHxJVvvZy4iXlXlgEjHAK5zoTKE11mGqvPxDnHexM41fjEniXBB5kdLEIZCzl18zGvh+nzPWKOXfyhvrLsWJTX/NV7TqXHkG54xOQc2fO+7nB106OmcydP743k6358p72lPIKvKyBtBD5G7DsU4bs775poAZ8cdbiq8vadi4cTPcjH7Xv3bV+xr7zy2umo6cER+85/d7u+siQH3HXEvvI7j8hty04KI8UOfxBATzFvxyl6ilOxbMUY9MHepMjVH5dX3q7O3xNJT6GEu0kuZITZOBsb5MrMaf985PG3Cr8vDr3JhPrH9JD//td5YPx+/zRFYSmxZY9ay3azfEDSIV1YIlz1uY/fCUgivb157bTx3gbH5tIByd09L+iDfCZ7qnnVK406WX2zJGchy8NI/lCVHRNASwpDOt8lLCP/Gpz0fHKYoj8rPML5Q+EqIwDDAjNQGCUYMWMDxagL47QqcNCkmwW53sxl+ZjCxkcXszOOCE+3yENH9sTQvGxaKgdBNmHm85dDh5fI9bStZgI40lfXriTe9UgCyUevJZl9W17ZHg4fmXc1VV/wpVKnum791Xc2fC9I10Ik47KjvKmlOj0mJupkfesPOR6wJGup0nltL2y8f25HmO7rpna5XGF21p9lgytB0Qi9Pa0/VqXHJObdfszKx5bcb7qWExUOlkeVH8OVgt9+dOeHU3tjYjx2XKq8dkoXnph+7VFTyfaIfdxTlsqVd+9wikdN+UkxUR9XN5EekpsExW6e/GD1vqqmR3dU15D0JrE7xFCfhXPmeLV8Oe2ff3P/f1o8/+97c5lc++q8//lgFhhbj09VlJcilWtsWWtrS33hvg8itsNUOw+XpgOf+cMxoiXD3n58p/XH8zAg6V+f1q2SyFXtSuvv37959pAuMlJoJyr3TuujR/W3T+tiTt981FoYDqqGsSIVySkaL8EjVexPGBHJd/tiViedzj+sMqG52cyNlanZCMAC+Je1sp8rwdmCZa+f0S9aRbl6HfoOerv5WOV97pal/wDzgRWTT0vFIMgmjHD0fF5SAJHr40c2mQlkGbxvC8mAU4UFMPOu+7ZACkbIjGNPtUG9qzQDWgVPCx8hFYpZgjgue689vnN2++rwmNXhOd+1kEslumpsrS/7gE2yVvliOP2Hx2zco862tj963P7oTrZaI0duP7q4g1bJre8kCxW4SU2FkVAgZjXcgB9ySB8eVe8VWw4Ohv1wWSstLH3KcnKFU9R/DeoFSwB378UlECye+dPJYbNQGGFIVQcZlrv3zroxlZYeD7x+egnwL6eXMAsoWVFeilyuj/hOAgEBMYeOnK3nBtCwt2SdcudIUhhzjEI76lfaeJ4tMSTtiKPEigV83dp6LYfehe2FPxIBS/oDcoV7BEse2ogBMCzSUTIxG4EfQ/1WzFV+p7rd1/v620b/5IA8vhvrb/7YCo+YO2fJ1CIqPQynyBF0q5iWykGQTBjJUYrtZgJZ9B5JMAFdDJP6g8uVAVoFpzpzbmjyrgFr18OyO03G8TbjUjIsMw7faX8MT2LaGRgIuVyjSu63XzvEf+Q2mYaNGMgVzgiugC9zNoNY+JY9ksrr7zd9d/IQPErpuKvIlb9nBF3ZIwvkSseW3iEyzqqDDHL9P0qd//njlH/+ZfL9Cp/qNaurw1f/R60XfATjs4uTjG1fAYVcuU6GQbfBH+bows/fMSZXtt778X57S9XeILEdk1cqzSvlCi20tzTxd6HqCJWr0JSLV0zy13ea7oMXUtlj09ksjpKJ2QgoXCsDHCzIWGEUkF4Uxx5yrz1giSFOJ+UVMTs3CJIJo2zNdjPBTO9KWjE2QNNmBW5893bYuguQZ1oFy+aYJpvsXck3HNcOsQc8AdYqLfVHktiSeL5L+KX6+1U64ShAvsBsyuP2P5xclY2QAVXIlS3GpBcojj547DsnySGPfdXtpHBOJZyC2/FyxSSnILfkuz1m3yT+DrFUdZB/ypn8H5Vu/9/dGf/8+6x/NkxlTpWo9++z/t+/ztD/izsUUFQRUJXrCuF7o7DzMIDqcvUNZt+Rgq+AWkI7Jq/UlFyJu26UffMnkWvAPF964UJ/5MCwSEfJxGwEQJaGv9/ASti0d5UJjLD9XGNT5Q/3v9snWFSnJT8IkgkjORrgQTcLtpoJANmvrn375cSN4Evle9cIund1ofVNDhAjePlnttAqII6Li9cHIKTWxqabP8Le8vO0svv1X5O9DVkSk/Xw7nMtj5t+bDpHdptc9RWH7zQ9etza2ErSHy+REZQ1krNQRa7gQkH5j+p/rL/Tcv+7j1dIRz8gF1aM9+/82Fr/YyuVa9iR29B+feVJmL58MXqKJqh+/3FTCb9QN+MmsbEVUtVB/uGT6S3nZ/7Pv3oQlf6b5z//dRoBMl2z/6+/esAhKKCoIqAqV7r1enTndhNckbHFcNTXTa3EGUKB+iMS72rySqV5Q18Umf0D+Gq4C61wX+jele/PYtik0DYhLeF6K4XMZskoQWpiNoJWu8L9wJ0KFlgb92/w37DA6G+wanKdH3CyqfN+NazFeItyWsoGQTJhhPkGl1N/NtGGMwEg3wz/QafyzfAfdOSbYfrXjFB/lstgv9vCkniB3/Yha1UFj6DIFUHGbk9wcBi3v5IQEBAWEyzfCJlshAG1IgM4tynHN1zeGpRU+YEOTqE46SC4L2JjS+8QGWewKMtMn58b7956dkbXTaLP/7PG7cFJV+B//zPZzYLxLydnQAFFlcHxCgseZDRImRUqY0uw+EoFfMNXLw9WGinzlitvmYj7QukoDTobv17mC/rc4+cDEoUU8oP+lmMIyLX1GmyLlHYB04Ng2RCZNxMY3O+u2TsYLx3YRhbG7HdXqMxaoX8VpawpBfarsI+FVGFH1KF/eibcIQYZZDeVPz2bM8crKsQj/73pwt9FQAY+ghEOCcUcE/Y3epJR4maj2kAJgFAvvbUQFsYgXRN+VZ3Fu7O/rq6HZQX9wWnYsWQmCDh7+DDvChnBSL0rtOI6y8XN29VNG3/SpQnoH3B7s7Fl9wlSFzcfh/nLdVvB/gKeGyU+pbNx2AbKKyY5N0c36OrDRthwJnDeFXjN1XPKDPy3XTZjyqzXYUiF4RUHeebripJjHIcfKBte4LiJrh6vQn1X0sSkqfNc3fHZbwPYP0rmx9aD3CF+nB3jX13bCu5fb/MzUDpijjFQtp0JIFdSk7VCm/Aiq2r8R17WwV75Met1GEY2ntKxFfJwFJ6sZC1kcvfiyIgDBVNZfZQgo+GBGp6ZQLwr1HyNplx+8mxYak+e6TcZX61mCa4zFtHvD3xgnSMbT7U8lIHtGZSH56uiHYfH1X0RzC6X6T4w00yPEslrcDYO30ygckUQRAuMoyoHiZMPIHRJHu2OZze/JNrt0S56V/YZ846dt64W5u0kP+61yZ4khymmmNp9SrwrzcGemFpJyufR7mh280ui3R7tUu/qifkxkLeuFubtIk/kSqBWzDt43rpamLebPMh1NlUwpphiau8p866zMcUUU/tPiXeVfmY6NiyHdsewG6aj1RO0W2FnckUQRAOMe20K+d8kgxTtjmof+TOi3VZ26l2Z1RC0MxzPPvJnZKCdYa193KQpc+B/mGKKqf2nsBieQ2CfMe/YeetqYd5u8obedY6BBe2OZDdMR6snaLfYTrwryUlSBtodz25oQbu27OBd59IcTadK8mh3PLthOlo9QbtVdrIYhpwkZUgtaHcku2HKQLsG7ONAxyTH1CxP0e6o9pE/I9ptYieLYfpZnjLQ7nh2QwvatWMH7+ol+Yx5x85bVwvz9pKni2HOyoN5R81bVwvzdpMf50y1iymmmNp/ShfDIqBg6UcBtDO0bjdktHqCdoZldupdp1LtTvVynibJo93x7IaW0eoJ2q2yE+8qsXJILWh3JLthykC7JuzjnKfNIzlMMcXU7lPqXafNA+EKVsw7at66Wpi3nzx6V0wx1UxK5EqYSlPMO3beulqYt5s8lSvNTWLHLMnP9Qma6x1kusxQ814hHvN81MvMWDIrIGiKiboW5qd4r/TwtLKuVvLW1XLU/MqAWSuXkCjmJ+Nd7376EsubKD/qed67Woibp9/ioNVvhIQDkIGPigJS3FambNi60U20LFm8dX/YyiVCARMsyiw/lRmuMHKEfHbq6ul1CuMQWPdFzSc7lMahA0MEKXm0SZAWEAjf903eSZHspFDnj++0P/795mFy9Mjt3ztvn5KWHy52VLbCSXMN7ENEci32wHthbv+8PA4AofYVvfzr6cn9Xz0HilUUk+G/ed2+vKzDn21YE6I8NCKMc5k+n+ZIamZ+3qJlb4REwCwMWk5Slp+36E1j5UFypVcvxy/h7Us+PX615lRmhLHy0jyVKympUobK9R3jdS3NU7kOUsaKfOnVGkjf2bRjHQ/kFWVYngjy8e/tjx4z6s8mOIcdu3i7MjuMP3r7lPnnNcybW1KUq/EyVuQl1zKkdmyU/9++eq6l2A34zwtOAz97/lo2g6m35dOXVMt76k6fvVL+SebBDTv/kFV845Md4KKUZYY7D94V/id8Hjw/1yc4PCo+eMV6qlJG+NtrN2fmnIC8al2Q3PEzl49nrGIfvTMunypmIqRlZixdvDljw+ZNbjMkdT1Xvrk1Y92akMWcd2XG4MWb968LX+M2g37kvCurItT1dVuZsGFnwuJ5vpx93hrSVPgqsmwGy4ylHv5LnWeEvLF152IfWmXexrd3Zrwd4s97V9YOrWuLPJWrit0wTwR5vypGavcKXR4eudiL5HnvSu2+u/adPJ2Xeyjcl6s7K+gDasmkFgZrJ2BxUg5x1x9/sNiLWrzCloeHe00Livn4dN7JnBi+BedpweH7TuWd/CRmTxWVq2gPCo9ZHrSY5uFEkWHh73jRlsP2QPlv9iWFzeIObQpbHuzsuxt6sicSNjJhO8gpTu2JhNUEOy9ci6xXQl3naUFB5Chr81RSeAA1khHgGwnm+8MYUn7fhmmg0v/rl9mtRVNbS9x/r/UcOPdCC82AYg3LO8/L+NPV8t0rYVJR+wx+dk2bP2XJ1nU7d765hKkX8PdYAns0/7nhOzdsjiCzjky2jLdXLuXqzoP9nS+ddTvf8CftTAlJ2LB161xP7lxTfFZ5cHkfN/8QNtun+MBObT7zrvOZgs3Je/kGxyXui9WlrlwXTb1rxI7kD8+WVMKkhLxqXZDriczP/nTmU29iidhTfC4tm5Ori+/eTy5WnsgvyMq/fPbi6XU+tK7P3pwrNafyv8rKLymEDPOutGRO5sH4w5cLCw56Q93lgncVzhW87ovKC8Xnsj4/d+LiuSj/+VN1p+nHgk/O3Dh7bCcsyEmt4nM5Z8r/9HneO0vnO6+ERiqPk3NdvnC1JkfH9Vn12i3Knzpz2W8peUIx7xqj2wvjtj1pH6SQV63L5Borte8A5Ty6SHw+J1ewz0oqb3r8eyfxwJDeSQcxh31zByz3m+60QFpJW2BtLie1HhNfTfz27VOLoWXaZn3jY2gBjJ0t5bT8piO3aRlmpHJlvXKZHpbf+Htn43lSd3pA9g+POxsvhXHluZbrz26HM8aWPepsrL9JetV6LiYy70do59GdH+9DenEHd156LbJzQa+Wk96euvn4cX3jI3Jd5OrqjwRBSUUjrD+Qqoye+fkA39n/85wTqLTrB48n51548rPn72df7rvjCZl/FL0Mcp3tOVdZd8e50oKDbso2fcGdsAmWe/HG8YyN1L4352p5bv63fyLGmsLPv8g6823O5+dOXb2RtZUo1i+z/OyZktyCc2TOXy1Jyy6B+f/JmcrS4k8X0fbBbdB5CPmI3cXle5aT/DvHak4VfEsfJPTcZqbgXePonIM0MjYl71QxTEfGG8upwzSoRRe0W6Pyy3eHzHde+8WpzxOoz4xwnu4LPTv+EfW6033fyK688HkCuMd10LPMTZK6XMk/pcLzlZTckF+ZFiHxrsK5NhdcKP5sMTyNBAs8BVl+/sE/XT0dBnlSiz4miR2GA1Y1wawMnJcshln5IadpHx0GxUKeyNVkSSGl0hIWw3eywS5McXaUeNcwMoMbzy+Ho16Hvnv0e/3XYc659Z2PWwvDoZ0AL1/ql/g2Z4VvD6POObwEZnxTHuRJm4/vnIyZNZ0ZacU91eBR67+OhJKzaJ56V66dxV+3krpEPzmV7IykzKPv9pFzLYejj6r3TJ8fA3J9/PhmbiStBfIDGceQPvgGw7nEayF1oQPkXMtPNrU/flz5MVe+9VqGF5Q/DLtcaiQapo1Ar6ARyXUNJe364oVfz0z5Z9dsWAMP/KsnOFVwsy3n3f5+2rntygzYx34SM0VRC+Zh6Rd7Fe2QuXSlIIzNNx+YYLDjg/zeT65WfriJut+IL85ePPn2bK4FMr3Z3rDgoAdpYWlUQU3uPtgGz3eeCbUux4P/kM1DIleiGqrhs4d3gHddACXMT718l1Evkc5SQatErsS7qtRiknPbea4wO/qN7PID0Qs5ERJPC8rhS0YTsfmxJ8oqWV1a8sYpcMKfE3KIxsC7fk69q3gueG4VZm4SzkvT1xdt/SyrAJbf4P+JtuW1Uj+5em7DTK48fapJ6w4ppd41FPJErtPmx+hShXGL2ZFqWB5SIshH9+/crr9JKE0COzfF+aPEu5IZzNwa806tZbudg3K+u0+k3vRj+b7wAFnLYR/l/9DaCh6P1CJqkbbpQnRO8pG8bkktUoB5V76doNPgvW/mBriQr4ta84NYeeHJAi3XH2He9X6Vjqu1et81UC9476ZzH8eA0oTz0rrkwUFa9iItwyWw67qZS+tyHYB8WLqkEQ+hP0NI3wtzZ8tgkKvAr2UzQaj/2Ta79Vt38K6Rb85U1PLOKC89ZnDXdKdL8/fyltD4M2z+EO8KGiN20HPx59RnwqieZi3ALAXls1rgM8lMJnmqBVpLMg+Zd+VKghDGgXCJByfpAoNUxT7XN5is6JLozEvaJ5UrrI0Ny0MK/YMzucxM/Li4JLe44G3BQnoDfpIvufU0rAe8p0O/ifOU1aUlP4xeNXfpKq+loZB6zl7gTBfD60hdrp+g7bPZW4Xzwlrl7c9vnDqcOHc25EGZVKViLSgDI1sSNZ8rT8ZOJ9SVphaMj6EdRgZS99n+sI+A0YPnHeRVyxNBwt5VauemOH8UvOt04t9ar+WsDo8JC4+BdHkQ1ef04Ih952+CaB/d2ceUQFr4CGTc/uP5HVCGV6a0TeqWSX71WfCfJENqCWoRe0JcevsPOXuuwUr4/OLprHxTYSTXh7Dw8HnTFxDvyvWfuyKvyEP5t4nLrfx4iXBeWhceDbTloPP18JQpSWSOlHtGcB3g2pkXmcU3wq5UbN8gHdyeEzO168xrUq0yYG0MLrfrhgdsYtM3TFO2AL7k4hdBivZBrgWZbiQP9ojk4pqsrZCHmUY1Bnay8wIPRPNQ+BhRKfGux4ijhjw4UqJMkmfKJO1L5iHV8HLSPl1yRlDvSmtSNZuVf3PVJkPvmpKaaaw87yF9V39+ozB7s2iZPv+twzfOfp4wlZQP3pDPji4MI8W20rrLwEjrkpKFh3eQkmCfuZC0L/hJ4VybTp69cvodX5p/fbkHu1rmvYM/PaHiXckT8U+py2ibkbuLJd7VyLVYkYfBgfw7m3T0O2EuVS3P5BortXNTnOTJ0cbzAdOX7IPd46P6vMhgGJ/wpN0roOSOU/kf74Bl5LyTTaISSDvcYtLDKyb7B5jxVKVCm3AWQcN0AQxr0QDf1UklreQXF6l3nT7fA0o+aq2/T1bCxBJ5icisLCPAa8GsoA90MUugDPOupP/k7LuPfJ0TAffCl/hP6IN4XlqXOxdxnq2FkVCe967cVbM8aSTcd74L34hixKzIr1ziAd4VHKlCruBvYe/6929c/vOc02Lf2QZ1iRpPZW5mM3Dq2k9364JdlmbmXr38XjCdjatgXp17dz6Ul3hXIlfeuxK58t6V6hbyojKZdyXKXBD2Rc2Jj0JJm8GZf7rCKZ/zrnBYwFmSl2JoBxfx7rZdcVSuX525HLR8vfSoojzvIRc4z14GXlFqAZG8V1B54Url2Ss3zhZkB8ykVXxTsy7WEOPFyx9+XiIveTm3oPJsMewHwJJ44ErNiX30KMV5+sKAjy5fgKYu3rhw8dy7yxf4kY+Vx8+Un/j886xiolJBrrT8ApdVIOOaCxehzXMH8stBrpxdDevsbHBgxATm+i4TykgR5Coa+SkO+Vi6/qz/erWL70fnGskymPCoKT9mwercOzC/mQUc72KxhdXsSx1wTfW3QYS0KUmbvHeF/BIdbR9o/7GpiaqFb4TixRbhsINlliWrD4snbbqWtRB6yMuVFAg/VUnX54T71elQS3Le5WLdRzdPklWuIFdSl5erSiOscR7r7kjr4Zdhg/rg2iwQLdD1gwe4VrDAMhgOBfjOUZTn8NWlnblRys2ukvfWvg5Gz+iTp5jlSvmH0cto+VSQK2iMVIHJVvw59a4LqHdNBSOVK8kAzLvSPJErVyv65Nmr0GZlYX72nnzOyMvVzZv1iaQW5oNWrAfFWldXmp+6MNRj/kK5faHb68vdZxmUn798ru/rYt3Zy7wWLlaWmb1s7uuBYpn5yz0XLFKWkeVf91i6bKqK3d7ySxaHRQXM4+yzgqJWr3hTUubNoPCYYPBmYnku77Wctw92FtLmcjrnjJeR55cFh0ct9jJaZt6K2LDl5NmksJtTV8hD/+VXOkj5QfNzZnud3+UMXhT0yQCVggWEqlpelocZKJ1dkM4KJPPHWHnr8nPeJPPcwC71rt6SvBS0MxzPPvJnZKCdYbGdeFeSk6dM02h3PPvInxHtNrSDd/WmOUwxxdTe03GuJMe06415x85bVwvz9pOni2EEQbQAeFcf+B9JiY75vDRFu6PaR/6MaB+anXhX9tlFcgzzjpq3rhbm7SQ/ztVd/Ix5B89bVwvzdpMH7+pDcz5ghQyXl6ZodxS7oQUyMgva7dvOvCvkfNlngxTtjmof+TOifah28lUTfMYUU0ztPwXvytQsS+kxtDugfeTPiHYb2olcKfQY5h08b10tzNtLXpAr91kNtDM4u/BPfBE7RHqnDNC8fdxUjwAEQTTBuFedZyAIoglQrgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlU5DrRxcNluvfkGX6u7osULxHXEC7uC+ESnKd7T3SZpbhABNEoSrlOmuo11WPxzPlvePosm73wzTmL3tJu6uEdPHNBEFwOXJTiMhFEi8jk6uruPX32ktm+INRljpTCRcGlSa8UQbSIKFdYA4Mj8vQJ9vQNhhQmuiPl4dJwVYxoHVGusF9191rq4RPk4R3kSVNHyrvPWwr7WOFiEUSLiHKdPMNv5vzAWQsCPbzfcLx0xrylcIHCxSKIFhHl6uq+cNb8pTC5Z1HRylNHsMMFCheLIFpElOtUj4CZ85fOnLfEUVO4QOFiEUSLyOQ6wytgxjyWLnG8/NDk6r909bpFsxRGzeMZvPHNxbil1wxKubrPXWw63RST/FXRlbxT5+cuXGa6pPXp3pq/93d/v9dkGctTS+QavetYxlqZ5cQPjx+WbJVabIrXqg2pGXEr3nJT2EXgebHKU2LxDI7b9WHGhmB/3gJ9LjhK+TQ1zswny6c1v7deTlAYLcR7Ufyhgqt1P1z9Nm2r0BnAf21qQcn1umuXCxJWc0+ERfFHj36ya5FY5q24Twqy4t8i+YiMo6nRvB1RRyZX9zmvu8993XR6OLdwU0wSpGs2xoFl7sLgvC/Pg4A3bUkyUcuyNCTnzL9c/jDEZBnLU0vkCuKs+1RpGS65vvnJraZ79+7U1N25+1t7zYk3DQoknP656fHvnWKXAtOuPmy/9/MPNT83Pfy96cIuKnLo4b0yItdvy+oedj6s+3QxV90E5sp16/et976PVhgJ6z6t+a3z3s8lpwqOnvr+zr2H1z4MJPZZu87e/b397q0C6M9puLq7BVGkfPTlh52P7xas5qtHXKl/zHfgs7rOmhOcHTGCTK5uc/ynz/aD1G22v7H8xi2JZy9WfnHq/ByfILBsiE78NPfrjTRVLb80Jm37lrVLYz4/U3712J51YF914PKVS2f2viOWWbXj+OnyfzmTnbCU1X3rvR17dq2CfEjC9sT3loYkHCy+fqU4Z6Va++bnbShXz+BdaccK0uJ5ZzjrrTfBMc56K+7Dgk9TNhKjV0LasRO7IkRvo6wiwS3+0K4VND8ro+zhvbN0ZkuBJevKD2+0Srrktnojp+r471sf3thF8tI+v3Xi599/+Izl5XhF7/oM3G80c9Qg16YLcZ4RGZ8eOyH4QEDWW7g6OPu9G2lwUvnKedFnde33vo8TPPksb3Z1IMv2ugLhueM2i6sF9vqf7zZd4Lzo2gt3m+4+RLmaj0yu0z39YFoPmoIy3wqNZPnZPm98caoYBAy6VS1/4L/o9b91df72oPXXXv3Trh//tbe340HrbwNP+n46QMv8kRSAow/+3ves9798Rmr94afepw8uwFHI9Pd2/tbLjur/9Zhh++antpJr3OV7rXfBs9XV34O5TqcaOJ+Hd2Ei3qHurv7qjXooUHev/fG9s8aqqLOr5J4RHw6nUHaJ8Oaxn9t//nYlyUv67JVRdu9hWYpYjOEWf6Xp8W9NdXU/1D1svZoGFuJdQT8/gwV6+/NR6pCVvV19ouRncNfE/5d8tlFoDZbBR+t+rz9Fl7Iy4Cp+K0tVGAlErpe/r797hW40Ekru/lx2FeVqAQq5LprmsXDQlMp1E8v7LF6e92Wxbud+1ZKQglyfdP95C8nrvu941ln14RLILytvfdp7Yxcts2w1sUD5/GZ930/pkM+icgUL0W3v93Hi0QOsJJ/uO/Cn0qs1DMhLz2uY2sy78j7k1dQb7XevbIAM0dLdE9RDLjp1txP0QxyON4ih/jSd34ZV1HBLBSemuuZUkevKz74ni+e7N9KYZyY9/L394W9A5+Pfm64ekmwRGdFn7/5259g6qRF62H41g/ZN4pANewtaUulYQsk9vsriXVl020w3opKxgh0pte+KIB9Brq2XM47W3SNHU240XT8aRywoV3ORyXXqLF+Y1tNoaiIPcj34ybFTZ66ASMCv7krL8lyw1Fj5j0Guv5az/Plfn7VeZPY/3erT386i+WUfflHV/PdufW8fcbkfg4WX61SWoXVJHuTK8jRlbe47cIRq9YjCbpi3nVw3Jpy6cefuw6Z7sG2jk1iqJcm0o7OTzkWDKgmn68B9AQWCD1xx4oeHD0vi6boRHBo9+sPlE9RzqsjVbfG6lVEZJ2oe8ttdsc9ui+OO1vzWfv2Qm6wdKPDzUflXUNK9K+R5uRpcoCjXtALWYM23Cc5xsEHlqtATweODtiYZq5RvoTB4aVaMDciiYz+Db0+4eu/aJ97iEKFczUAp1ykzfSCdSlNj+ZyjBXmnit9cucFEGSFP5NpWzvJUrsx+BOR66xDkD9zoHmivOrI+yAcE2dPXAHKdcqih5+mD89ACWCDD2uSPGp4r4t14IW+qP7aRa3TBz7/Vn95FvnoFCZklV5UqZDu6mrDUi9aatasEXN+pOM6twXaRHl0pfFdsIFce2O6yjsn7zE4kawcK3D0h//5JTa5qFyjK1X8pa5B+TZ12/TdwzkJr/PVuLPhZtkgWHgRcgVlHr929W3/3Vpr0iYZyNQOZXKfM8J4y09t0GhEVf7akMje/eNa8xaZLsvSjWpDrdyx/vg3kyuyHqVwh/13r067SN8jRzWcf6PsaPoKjnFz5DGsN8uyo2lnMSW0kV8hwPurNT8gXLWbIVa2KFPo9atNl9gWvEaSnmLXr6KlDdL0NPrmonvuuVdpn/4SrD9uvH5U3SFTEn8V/5WoiJzW5qvUWVsX3vo/j2pGwFTbD90pSuG/UhJUt+QrqYd1R7rsrssWVydXZO+v6b2zzjHK1CIVcF0x2X2A6BdcaEaWDNPjt9aZLspSXK8kTuZYwO5VrJuTjitsGnvTre7r17bW/dBJBLpicCSr9ezG0QOT69/OsNU6uYsuWphbK9XfYAXKQaQQbv987rx+FPR756eLhvaa79+5cvlFvllzVqkgAHwVnaX8oPZ20wNFrZEdKDpGtKenDuqzr99g2tf0x8YRUgaTPnPHhb011RQnMb0uB9fa93ztpgdarh+ary1Wtt7MOXYMOQLOXicakvJn6fT09RHr48O7ZVOZUvTccq2vl+vN75726T8W9Kz2d1yq2rJDJVRjwx5y8EQUyubq6zZ/sPt90Gh6546uiy3lfnps51990SfPTkJi0d0MHKTP01BK5qsF+raF5WMpyi1izsaLKIJCFrhV/aOW9aMXgPVHrrfLvNKSQtb34BxsC5Fz4V1M2RCHXeS7TvRw1HapcEWS0kcnVZdpcmNaOmqJcEa0jytXFfSFMa+epcxw2xX9Ah2gcUa7kvYFTYVrPnjRltgOm0+biP09HtI7Eu073dp4277XJnpOmeBqks7Vuh0vDl78gWkeU60SXWZNn+r822eM1Vw/HS6fM9MdXqyFaR5QrMGnKXFc3n9dcZ010mTmRpiwvTbVod3XzhUuTXimCaBGZXAGY1sTHTpk70dUDJvqrzjNpCr6XpK+KeQ3YYRkMFwKXg1pFHAOlXAFYNMI2b7LGg264YtANxOFQkSuCIPYJyhVBNAPKFUE0A8oVQTQDyhVBNAPKFUE0A8oVQTQDyhVBNAPKFUE0A8oVQTQDyhVBNAPKFUE0g4pcnafOdZvjP2tB4JxFbyEIYj/I5PraZA8QqqfvsulzXp8yc6HL9Pk8C8S8mySPdg6t2A0tlraA9tG0y+Tq7hUww2uJixtfDkEQe0KUq6v7fA+fIGJlapamTOto17rdMB2tnqDdKrso1xleAdM8/LhymGKKqf2lolzBtU6e4UusVMeuEk1j3jHy1tXCvP3kRbnO9n2Tilg4hnnHy1tXC/P2khflOmfRW0zHmGKKqX2mMrkSERMrpphiao+pwrt6g5XqGFLMO17eulqYtyA/1cNv1oI3PHyC2B822Ba5d4VTcmelGcw7WN66Wpg3Oz/Nw2/m/DemzvJzdfcdDlS9K6aYYmpNCn51+LQKKL0rOSt5WpDUWH7SlLkvOk957o2JTvEvPhcx8cXZU16bMtdEeVP5wLjY5MS3rKvrtsAvcnd8zEajZQK3xyYnvGVoH8N562ph3sy8h3eQQmC2xQzvynrDWyZOmjFh9ktOXzmN/7fxTv/NyemJk1Ov04S0519xdlctL6QZRZWlZZWXysQ0d+8C18yGnqd/L1YrL28nDqpzdYtyo0I5e8Yd/ZO279TK0/QgNP6g2Eh/jPVzDNlH/oyObiciMtCYDVF4Vzgl3xu1/MtLpj038WWnLCene07j/6vT+CcTCP/dafy/P+cU/sLLr7mbqFvc8KCl7UFL98CTp/p2yLQ9qPjE24XIFRSlUl6eP1zTN9ByExR781Zbr/6pviaT2DNqqVxVytOM8CxQ2IeWf+dYTelVgdPhamWEvF9mRWnx5/5Su+70qcz10jKU1JyrNTk6lh/kvLSYehkuv+LzU6x7xZ/78fZwWXUgYk9xDdcTUr5izwpih7PwBQCTZ7E0z124ehkLrm448twISOxgOZYqK2NGXk2ukVEp6fGUrZFhroG6rfxHzuLu6x+5V8irE5rMChh4V9YD6VOETye9Ofv5oy9MCHJyKpkwvsNp/D+cxv+vTk7/QRX735zGl42fMP2FSd5zZLUM2yn5+5O+hgzBzhQVuJ343qI/vCXYQxMPFVUWH/84PJBaph++1QcS5Y7mNgz01B6GPOddufZXRmRevFR2MSNyFXde7llA8mTZrNvuR+ybIj+5WFpSmBK5krVGUsN+DmKHuS7cXandwGJol85ai89LVJejU7FLLKB8rm9+meWlZM4t8IenxrFUWlI4uh7kSvUMR0+fKi6ncjVo3/IeGrWTC49QsUss4tlteF7r7FSuKnZFKreryDXzp57+XuKo2h7UlaS7ppTX0TzQ2T/QUqLbWdbV0912/WYbfGwsUFNsbHlLX29dZW1Ln17pXelZ1dMXdzlPWPzC+LcmjL8w/vmWF8Z3O41/PGH8Pya89rvzzIUznTKdnpv2wgsHnE20QNKSB0SuggUU1d/b2U2up73vmb4hl9hjvmvp17fXVl9v7tV3V6eQkuBd9beIR/V2DdxX0fGsvWw75Jl3pS0nFrcN6LuFUUgkJTm5ervQBmsyV7nSYj0dDRU3f+ns761IVeuhWSnMdc4dUUtqTvHn74AkqENjvhRmJ3MXnE7AIjg9DtYClQ3/UTgLccu0GHOAgl3wfoJFmYrzjFpWrIf+vHNM2tv1/tx5T+/JPL1nxfo9xz7fQwvA0cHbpy1A3Rzwh6B2eplQhbcrr4U8KZgFHgrCtQiDQ/2/0LJZZxeGkXo/ZpeuHagF7sjpHOjMsVQ6krQ/8Lw4dhrq5uhoP7k7RdY1tO7pd7jWBAsH3+bgqbpc28qVRiDzp86OqijIBIb5U4t/WZe+IY8cCj2QW9nc0tZ2vSAdDqXV6tsrdcSeWqvwrj7iud0leWp//vWJTs+9Mj5g/PgZ4z7KyljYtOiVf3vV95dF2XmfjBs/zun18RNeePm5N18Ryqu3w8uVsxNF9VbE0Pzxv+rhkHtocduz9kqqN7fQSx0DdcehLnjXZ0/6B/TA02f6jqqUQNImlWs5lPSHZrtrd9Lz+kM7T3/Jh/aZXAOhLhEwlPeDYh3VkaRlbz8yOoX0LOrXa9IO91uqrvfhBrO5SP2DpDzMSzarhHaIk9nA5UU7a5AvQyT3Ps3D1IH1ttgHZfu8Xdp+KVdXsMt7K57xdPiKz3OOnc7JXE/1TOx8+2otk1SoW5MT7wO1TmWmClfE1SXlSbfJ7CfSYvsFUgWGiLQjXh0djUPiaMiuTuW8kIeWzzBdgQ7ZeeGJQFqWXSncEcjAk6gi59AGKAnNkpGBpwNcMjyn4FAx1w5tGcpz+xruvNxTT9IH9f7I7CpyLWjr6fjpekPz9aLsENGuK+3ovZ4qfAyL0OVVdOjB2cLHjMq2upK8+P2XGvtIGZjk7WXU6wYWKxfDfG84pHmnDS86vfqqU9xz4yaPA33Cf+PpfyQ3bpyT34QJE16dsPJlobx6O4JcmZ13gFyeHCLK7OngFgzgcltK4ChZDHPeNTQxF1TaXQteV/CukGHLY9KO27nGp6QwfRZ0NTYPPOn7a1YgqUvK93Wxlls6BM/MdcaSPDcteDvxrsxLsKkjlqdy5fKsLpncdNbyRpoX5tkgdjah5WXkef6M3G6Q+BC+t7xfoi2AESYodyhc7l3F1qQti3lWl6/FXRFnZGWYP2ed4erSYlwPecdFkKwFBr86TkUyO3cummfK5O8I1yW4KcTI+sm1wB3i2+GfLzxs78rlpXaTeRW5ppZfL8uLT8kubNbrm4uZI3VNqe3sro0XyuRUt3TrQdX5sbwlNDmrqKquAyY/+OFaWAk21v7U0t2r5l3Z00KaUvuLf3zVacULE156xWnB8+OncCp9Ydy41VOnTpg3wcn1hQkfPP/86olCefV2VLwryJXPE+96pKZvoK5oX3wyR1QoHGV7V6Gd8hao5e4jeFfIgKvkzwtH9bf2M+/6TN9WWdpM97q0vL7hImk2hTYes4mWZ7UkqYn+c3lulvP292Fy+NO8/yHhSU/Lk/lKPQmzQEomDfUngoWkG2iDgkW6HpPaef/Daqn2U3pGmHPcrBV7y7cAZ4T56uOnS4Weq3tXRcuQcnZaV6jFXRHv97iz0DPqzhA1srrxZ9jICF5R1jJXS3J10lToD1GRZDyJXXZ1MP60/+yOcP0kGo7n+8m1ALXE3nLeleVZyslVYpGmRuwqchUAifb9lEHzEZVdPbVHZEdhMVzQBsvGCPewjNreno7m0uPZ+Q3gb+lR8lXT3giFd2X9oE8Lrk/S/EuTpj733MTZc3wmPP+Kk8uLTgFO44OdXvOfcN/V2ee5l59/7tXnJr784rzJqnXFPCdX3s7Jlc9TJec3D+ibz4WwMoGhsMFwdQMNg8Nk7YTGl3XRpa/gXX1cC37R97cVhpJ2ouBo/18PQUnSeFdpoI9LTGV7P6wrvGFlou//JT+Ua8ePrKiN9HOQPKcu3k6e5bSf3FwUy7P5Kq2ro7NW2SY/56idW90Z2CGl3k9RV5Fn60Cah7OTjvmQDeSx9+lR6DlrgZ+vtBbvXX2oYIy1LO0tqStokvVWUpetLam6rp6hLlcyMsRIjkJecqWkjHlXp6xLM2xFzd0X/o5w/YQCRMOsn5wOqZLF9smzhvWTOxenapqX2k3mDeUalXqArYGJGnmPCvOWW99Cgf3ZZAcLHG+mBYob+2HSgiUMhEDlGuZPPvrCxJZ7V4OnhTSdNNVrwvMTn3vuNWACAfKvApFOLzx+663Pprk99/xrUMZECyRV9a5CnnhXH5fAw9c7YJuq7+zW6/t+KQyHo5K9a/9AT/dfC5NXQQt+Rb88edp7/UNoLTTjZhc7+qS/63omOSqutN19Iiu76K51FSvW093b069vLIpR6eFgKUxKcSHH+RM17ypf8nE7NLCTycqMdK4TDyAWI78MkXYE73rmHWiHrrTl52V1lX3j0vgzXLHiCupdpWcBC0iCWUR/IvWu/Cn474FUzsLVFb0rd3XiWVhrUB5Uylo7BZ1hIwN2sYfcuFlwdcIACn6b9ISrS7wosRjxrtBPTof80wpKciel8P2R2vk2B08N5BoG862H7L96Ib2eyUm0uI2ucsUCvS1tXT39vbeOw95Vx31p2t3b3k3kGna8mX63DC20KRfD3NPCnWQM8+tC5u9YNTkq0Dlr8/TLf1z08guvfvW+9+tzXPMi3DI3zQzynWWirqV5/8g9sFg1XQbyITFstUztoUnxuh1shhkrT/KBO2KTE0NMl3HEvHW1MG9+3tC7EshvrcmS75kMMCgQEisvDwViIyFjgXdladxat+8OTPOeOfHvJS9NeGHinZNu+9ZNuvuVi+4dNxO1MMV0LKSzFrwhamwYMPCu7GlhMvVbOPelV6d//f7EV16esiv8tbAlUxf5epkojymmYySdtSBo5P7EHz670LPSFPOOl7euFubNzU/z9J85nP8oR7kYJtBzY94B89bVwrwleVAs+NiR+OfpIF/J0wJTTDG1r9TQu4LPxRRTTO0xVfeu1LPzeVoO7Y5hH/kzot2GdqVcmZXPYN7B8tbVwry95A3lSktIVC5L0a51+8ifEe22s6t6VwRB7BGZXBEEsWfQuyKIZkC5IohmQLkiiGZAuSKIZkC5IohmQLkiiGZAuSKIZkC5IohmQLkiiGZAuSKIZrBYrh4L3li9fqvAwqWhigL2Q0isyaheCKI1LJbrwT/mfll4AVKgvPLPv9z/t/iUdEUZy0ipbu/vqkgxsFtLfoO+p6EYMsVtzwzflT58zPVdtnZD3JYdu0PCIt1nL1YcRZChY41c12zYJuTf3ZoCoh2SYgOzCysvpdH3lEtel6xGyYMnQjAvaV5OVEFVaU4yZCyQq/HWzGGm15It2/fEJ+/bsCUhLDxmc1xK0p6MiKj4YX0pHjIGGapc96b/AZbEN2/Xg26FMqYJSf2muKw8d780QC15CTKsXUkkyEqws3ciR0blXCotu5TBFrShyfH0TfxQKypJkg+lL03W6fwD03NLyrNixTUwkys9Hd8IPQupQvJhEbq9EfCYkLbMDtGAQsXHD5CjtJZ/5JHCsqrCHPWXO29LSH1nU9zbazdLiY7buWpttKIkggyFIckV9rHwEWDLY6GMCXZW9j7pe3C9rLal75m+mwaozfyJxt3wzarsIhGoSIzW6iz3vFt9A50kVBwY9bcySZSudhJ3g0SCrbgpyefQEJp9XS3d+va25uIU0alCRt+np8HsSMD1WzTkgcSBH6FR7eQtQ2ux5dLosjuhJI2oW1dZfatN33nzALsQKdsS0lati1bIdf3m+DUR3EAhiE0YklwFwGKeXLM5hUD+eLO+o4rEz+PlSvUjLob9Azl/mEtipWdDhsWbY0ZpnrbAqRGQylVYDO+82Qvl4XQqcpW1FgYF2ivJWhrypSS6LImHK4b6U0OQ6+a4nZB+e6VSl7wP5YrYnBGWa3pF90BjEdEVCaDMFGJMrinfVDR0dYJ77OdUZ1KurAWCqlxp5DsSrm8wuZI+yKPLgr+91AjuFxYFRVw0MQWxur2h4THhUbpfH3RVVNXcvN1AvOu7sI+NVZREkKEwwnL19T/eDJ6ws4OE0ypkwWdV5Zpa29nfVZGpA58GWrKNXGm4PjPkOlBXRPbVDHGjm1le1z0AbpZ+lLEhOjE8agdINCvni780/A10C/nouF1vhIQrSiLIUBghuUbllOfvZ1tHfR3sD6VHRbHpyFdNLO4lqIstld2TC9tgMUxUF0G+EGJGWd6UXBvyWCOlHc/YthOMLOO/v7qd7YrlrdHossWcFw0MA6N/yoF49p0TDdFJMnKClq8HcYJEpcTvTPdcEKQoiSBDYWTkClvWZ8x9ZcAe8ikfprXjJ+JgJWIjX0SRow9KM8tb+skXRZ19XbeaezknmVrd+fQZVGwp08nyJuTa0dVDzvVM31GbQSVHouLSij0dtbc6OLnKWgs8Ioku21YYHpNFQsKSFQGkjUVsW6tEl/yh9NsmcLYR7+oUZRBkiFgj1/zC8weyj0oBi2nv6hqqoz+K5N3qflAaGxlF15m5DZIFLY9/5F7+T5GgmMEPJ+w3G8O8CUKT42lsTFMWQN4adENWhvzaRH/1ESxyZnsHJ+zcv5IqNuLd+LiE9/FHV8TmWCxXjwVvgC81BOyKkipEVrX3d5WyLWtgemnbgOrvIhplptfSxN0fbY7b+e7WFNQqMhxYLNehERZf1Ey+7O3u7enrbSzLVv2iVbtMmblo4dJVCiOC2IoRliuCINaDckUQzYByRRDNgHJFEM2AckUQzYByRRDNgHJFEM2AckUQzSCTK4Ig9gx6VwTRDChXBNEMKFcE0QwoVwTRDChXBNEMKFcE0QwoVwTRDChXBNEMKFcE0QwoVwTRDBbLVUPxXRHEwbBYrgetiu8akXmpMFP+3t3Ub0qPi69B9E/5pvhmc0vzT6XH0+nbQ3UZRVWlZQLfkNBS7gdyy0iMOb4WfGR2IDKtqLaure0WWLj37qu2oIL7nIBlb2/asmPvxujExUFrFEcRxH6wRq5r+NeCQ97M+K5C4AwRSUjVqJIH+qf6ltrqUhKZbqClJEYRq4bGpIOS5S1Pn0miS8FH7vX/xW0D+u7mirIqEjaOxsIx0oKM6Z6vb9iSlLT3o+i4XWERW8OjdmxLeD9x10dvr47GN48idshQ5WpmfFdTco2sosEvuPhxLM6FNICNhPKWvq520S7IFTJdpUIsVi50nWoLMiJjkjdtSRRevc9YtS5623tpb6yIUBRGkFFnSHI1P76rCbn6l5FQyBHSQwQjcn36oLSgTd9RHcV/pHLNq+t/1llbzEWy4Rhcrlt27Fm7URlGGXh3286312AkZcTuGJJcBcBitVwl0d+kkKXsExpKB2gsYkamz+SKDlgww05YkKuv//6qFij/dKC9oTyN27uqtiBDkCuLy6pL3tfUch/litgtoy/XNKNyVfeuRJ+ZP/X0N+cGinJlhOwvr+sYeNL/oDgSPg7uXWExvHYjCfRYUVUD/PqgKyvnC/gYvW1n8IoNisIIMuqMvlxdC9qegPakhwgm5eoeltug77xZq5ArhThVGsF1cLmGhcdGxiSBPsOjdH9p+BvTKrA96QPfAIydgdgddiBXKrCe2jwWLyckp/ZWSfJgcvV1jSxv6RvQ04/+mT+1NPAb19Dixv4BGkJ2cLkuXPz29sQ0JlGBleui39u5X1ESQeyBkZPrk6ewt2RQFYly9XWNLa7rJjtP2GQ+6e+6nhPJNKysIpWru+9OEiqWfgzNLm2D9uk2FbavN4/w3y0btqBkU0xSeCRZDwtEb98VFIIrYcQesUau1sR3NQMSUtWceK3GGCwEqypTZ/ltS0hbvzmBaTU2PnXN+jhFGQSxEyyW65Diu9oloNiN0Ylb499P2vvxshXrFUcRxH6wWK6OygK/kBlzFiuMCGJXoFwRRDOgXBFEM6BcEUQzoFwRRDOgXBFEM6BcEUQzoFwRRDOgXBFEM8jkiiCIPYPeFUE0A8oVQTQDyhVBNAPKFUE0A8oVQTQDyhVBNAPKFUE0A8oVQTQDyhVBNAPKFUE0g8VyxfiuCDJaWCzXg9bGdy0tyBZeSqoS7tUI8ZVd+o7qeAO7+WD4VsRhsEaua/jXgkPe/PiuT572Xk8VPypf6i9F8sZw//2XKopEnVsEhm9FHIyhytX8+K6dHV36tnIWGFIm10Aa5rzoCPdG79Bk8KhPwKOmpEeF0neFx0ZyJd3DiJcuu5QRyQWD9Y88kFtSVZiTzAJ2KMDwrYiDMSS5Whjf9dL1bn3dcaI0Ua6x5S39A50kunmvvv9Bcayva051Own02NvS9qAih5TkPW1ycdvAk74usHf2dZXqSGicnv7expvVtzr0kpDqIhi+FXEwhiRXAbCYIdcj/seb9X0/ZYhyDStue9Z58wAtE5bbMPCkuZgVFhbDQt6/qO0J1BVjahy51TfAxE/zXRUkSKQMDN+KOBgjKldXdx14yPbKZP6jPEhcARWkEbnyVfjCJLwVc8uEzn6ViFUYvhVxMEZYrrD6rW7vb7suyhU8JF+s5AHINc24XPUN33AlCSDXrorUdNjiUlSCWWH4VsTBGHG5MuH1P2Mf85sH9M2X6BdFyaUdz/QNeWCMIF81VbG9qCjdgjZucwsL45T0KPf0iu5n7ZXprJh/IPflkxQM34o4GKMgV9fA4kZerq6BR6530Lis/c/0HbXc1jS1uvPpMzC2lOlEubqHZdT2ckFc+7sq9oOjvtTYN6Dv6+2EtLt6J2tcDoZvRRwJa+Rq8/iuIbHpW/nfZjgCdVtVY72GJitiwKrUlYDhWxFHwmK5ai6+K4ZvRRwGi+WqUTB8K+IAjBW5IogDgHJFEM2AckUQzYByRRDNgHJFEM2AckUQzYByRRDNgHJFEM0gkyuCIPYMelcE0QwoVwTRDChXBNEMKFcE0QwoVwTRDChXBNEMKFcE0QwoVwTRDChXBNEMKFcE0QwWy3XsxXeNjEpRD5mFICOMxXI9aHV817IqSnluKv/m0dRveCNBEvGVxKSTBoAVqxflRYVyRsNirpFHCkl8OpKXnJGQy8eqtJzylqcPipVGkbm+y9ZuiNuyY3dIWKT7bHx7GzKMWCPXNfxrwSFvfnxXfVstUc7N5va+Z1zkyJIHLNIco66EbySltrN/QN9RxaJLSqrX1rX16p8OtJQkU/uRW33PnnTXCsGad97sffKUC5ZD3idOw9UxKnK4MpZjVK4zvZZs2b4nPnnfhi0JYeExm+NSkvZkRETFY/BYZJgYqlzNj+8qvsU/srqdCUASdllK/M3entrquv6uUj6onLQ6iRPJRXYGufZ2dgtRng9c79b38DGyZGc0ICSWBI8NSf2muOxSGnHXYfHHy8UYswQulmxaqFG5bktIfWeTMiBldNzOVWsxvB0yLAxJrhbGd+XFE1vdaUqu6aA6kFx+87P2Sm6hK9deWGnHs86b4IqJXG/VdnEhJ0HGHc115sm1uO1ZT3dXZ8eDlg79k/62ujZ9Z9sD0e2TAB9gJ56/vW/giVG5pq1aF62Q6/rN8WsilEFJEMQmDEmuAmAZVK7SxTAIiWxfQa5czBtY+tamscJkJdycBZmCticd1SrR1sWPNN7k8ap2GmYyq0HfWEQtwmJYaLz5klCXAXLlIzh/U9c/0FhAA7QTv92Wz2W6Smn4LBOLYUGuLH7st1cqdcn7UK7I8DHCcq1t6Qefmcx91aTmXcn+s6OZfj/UBpphQZYVcgXHK8o1U1faoa/LyavraysMlMlVWkUB8a7cUbGKqEzoGB8Cz4RcY3V7Q8NjwqN0vz7oqqiquXm7gXjXd2EfG6soiSA2YeTkyuQRAUqgzpDYVeQK+89n7bXc17l1JCQkWQ/LtZdX1/+ssShMUJp/UVtPd28PibxuQ7lyjt2EXDdEJ4ZHkfB2WTlf/KXhb6BbyEfH7XojJFxREkFswkjLlQmS220ayjW1tpMtRyn+vGzE6oHp+Q16XvC80gIvNT4dqCPf/ZqWa3JWSXEa/TJpELmSbnRVsFiyZGGsLteg5etBnCBRKfE70z0XBClKIohNGHm5SnaGoManz0TayslKmDhJvmJkVTv9Ehiqs40opD1ttfncrlJUWkgs+0sGmVyljZOz74fzPmspIUcHkasklmxPR1t7n7pcAV3yh9Jvm8DZRrwr+R0YQWyKNXK1eXzXESMkUi1mrDFCk+Nj6VdQxpntHZywc/9KqtiId+PjEt7HH12R4cNiuWouvutwM9NraeLujzbH7Xx3awpqFRlWLJYrYsiUmYsWLl2lMCKIzUG5IohmQLkiiGZAuSKIZkC5IohmQLkiiGZAuSKIZkC5IohmQLkiiGaQyRVBEHsGvSuCaAaUK4JoBpQrgmgGlCuCaAaUK4JoBpQrgmgGlCuCaAaUK4JoBpQrgmgGlCuCaAaL5Toa8V0jo1LS43WWvMQQQRwRi+V60Kr4rkMgLL95QN/9oKWh6svKLn1HtRAe0hhZtb09zSwylY3BUK7I6GKNXNfwrwWHvJnxXYdAcSMfstV//6WKouxBfWxUQVVpDgsAazMwlCtiDwxVrmbGdyWEHsgtqSo+foC+bh/gAqhmRIZxBQJ1W2HFG0hiopcWZdNiYRG66van+rqC9K2RYf6Re4X3dPuTQOlVhTnp8KRQvLxbrRh7xz+BxnQtz00VqsBKG45G7jxeXlryTbwY31UGhnJF7IEhydX8+K7+mT91Ph3obCPxVHsavnF1Ty5uo0vctged/Xw09MyfevpJeGUaVfWZviHP1T29uKFX//RZTweJrU7iaNCYOv4Fbfr+3rrK6rruARIivRJKiucSirEGoditNj2NykOjttKQ6tB+JwtaSQJt6Ds7etuhb9CaJGS7FAzlitgDQ5KrAFhMypXGlSsTF6gkVlV37U6WP96sF6Oq9rJAUq5g5OJWETmJYW+oDovboDXqkyOrWXBXkucRiqXV6vkgrhTSflsh85+x4LR7r6dAHtofaCzgWzMSvQpDuSL2wMjIFSTRVSpZZ4KiJOHh+N0pkROvFsgbl2t+84AYMb3vJy6OM49QzDX2UmPfsyd9D64X0RU4iWcnRG1NhycIDW8ltk/z6nLFUK6IPTBycmWBlRmgKD1ZEgtH9bf2WyBX15SqdhIbrqunr+t6Jr/15RGLEWCHXA5rZnCz8qit2bf6LJArhnJF7IGRkWt6RTcJ0Eg9W2R8is6VbD7bCkPJ0aiyrif9zVlwyGy5wiq3vUwRG06M3SoU8085wH11BKfrro1Pqe6EBfB+Im8atbWrlDxBzJIrhnJF7IGRkSusS8tb+p896acRUxuKw9zDMm52QZ6EbO3nPaTZco0oatPT4Kukte4HFVBdEruVLxaTRU5BvkaCtLGI7Jyjitp62EmfchYz5QpgKFdk1LFGrtbGdw2L0KVHUY/KEZps1d8q6Uo7em9lRkJr8SnpaeCc6fZViN0qWwwH6ram7I2Q/jxDLOLvOuaDoVyRUcdiudpBfNe8un7wh2zLGpkl+/o3LCRUl988IPkey5ZgKFdkdLFYrvZASE51Sx8sg3t7+vQtN4slf9uQd6sb7M2F7NegYQBDuSKjiCbliiBjE5QrgmgGlCuCaAaUK4JoBpQrgmgGlCuCaAaUK4JoBpQrgmgGmVwRBLFn0LsiiGZAuSKIZkC5IohmQLkiiGZAuSKIZkC5IohmQLkiiGZAuSKIZkC5IohmQLkiiGawWK7WxXcNiSUxqSAjjTdlnwhdNR8rqtgcexxYa186iRjDYrketCq+a3EbeS04ZGRvFbU9YRG6vNzj2bLXo8qBaS2TFkypnEuFOeLLTYWumo8VVWzOIAMbmpx2/FLufvPFM/hIurrrMoouZfDBGfz3F5eW5G2VFpC+OFogpbq9v6uCRCeyAIysy7BGrmv414JD3sz4riMh18Aj1zv0PR0PWtq6esibvg3cXQqJvvHkqURa+2HqcHHxnvQ/KKbvTxxGuYrvOrc9JgY2rKC5p1/Phdjr/ilD+tZlVQYdSY4jt/r4N6qT174blFSVa2B2YeUlFm/BHDCyrpShytXM+K7qcg3UsaiqO/lHOKwq4XFOQ7BeSiPGsHgoUHREfK+3QRUJB/ILuGDNUZW9KnOXrM3SKzqk0kqOT2EzLKy041l7JXkrP+sq64MYe1YBF4SW6xitks1FjuUaJITE5pHospn0feVwdpBNX3N+iuHKmYsum5aSHs+/xNwggK0kDi0LfhuYnlvCNy4OLBQTVwrcCjkQWmAWENhA3XHuqAB3rkxdSORe6k4HG0kOXq6BpFku6qcUJlc6VmJcX3GFjJF1LWZIcjU/vquKXMPJ85g88tt69eDZ6JqKFOvu6oTnOnF3bXVtenB9JNZrWzkJRaVWRRX/si5jk0zoiQw64RoLuAL6PuZbevWqvoUGEGEFejqqYPkndptUYaF3fMn7ymkAW+g/OSPn20n/60qki5EDJIBQW23pzQc9JARuc3GKagBbEhyExaElA9Lc3NLXS70f6IQ8ZfiBJZH1Gou4Z1BhM/cM4oitau/vVSxEabggPvRuPxe4RDxqfCR5uZJQvXCB/JvZJdCWe0g/uZsYxRmZyxWvCCPrmsmQ5CoAFovl6h7mzz1QdeDx2CwhUuFeyf9NXT/oh353Qm4wDQCrVkWN5IoObhIbYiDXvApQ3dOB9jLOn0gL7LxJfIt8IhINKGanpNuSjgWGcWVorFoS1bLkwRPDxfB+cYWc2wDdIEqDBg0C2MLk5uPQQoP9bfl0fUHGs7mYy9CBjSjr0jdfIqcOvNTYzz07yMOiQ6/v19cVKHwgSE4MvQvnlY+qqZFkcm1v08P+4klHNZGiAnLj+ECh0BkWi0gmV4ysaxmjKFffkP3lt0AqfXo9H41KIhX25GbVxUhTyiowC4mzfdDSUB5PC4Cc4BT6jio6e0CK9GhbNYlwR5GcggFLsvSskuZOfuslK8AJTNqOtGMc0iqQ5+WanlvZ3N6t7+kDD0klpyrXlOrOfhYkOkyoqxbAlvgi7rzUyLUDbdLxFAc2ElwoadC/qA3GgXNZZAmanVv5oIffovNAs2LoXbHzBOlIqkKkTvf8JKOialGZACujkKvKLVaAkXWljJpcyXqvr60wlfhPYZYIxVTlqlKFbkfJ7o4PjRVV8kAP+0NuRhIpkqOSnxMkp5ARUcmt+mQFQAzdtfGydqBjyu2ftArXMRIwfqC9km5uBXWpytU9LLdB/6Svlwi7+RKnDZUAtmbLlZN9GGgeUmoRAQcuv3wiVyH0Lt95gnwkVSFDwdwjXb035/Ky5zCQK9luWChXjKwrZdTkChlu3gRm84HMpfNeRa6qVaSQGab0Hkokp/CNKqou3c9+qwTNDLBVJSnQkEfFn1za8YzzchJABtw2zD0MBAwWaZuQpx0jMmBeK6oI5j1VF10V83IVAtKWt/TR8LYS1ALYSib3IHKlfrW5uRGGgrQfllFWnc99AQaLW7gisnOOyinPJ6Fu4bEC+1tyFXQDz42qOSMpv0dkrNgg+O8vLs6hDRJldpXSRTvdIdNFhIVyxci6UkZIrmQTCHeLzqQnJCYy3LbazqfPYFnb0912q01vjlxVq4iQaMvPnjwdgHUy2U2JU4EjjUWUZWX6u6AP/pm14MRonNhnT8Bv09lJ+gBujRr1HbUqP3vAtO7mYtXqO6phpaomV11x28CTfuiqvr22rZOpK7C4kQa57azNkwSkPQKC4Vrrh8LEwaoEsB1MruLAkhPBRvHZE7aDJc8LLqqt5IpI8HiuYuZPMG7kaF9XJ5PrYCPJI71Hvq6pcHe6KmLJCMAigizgoZ/9ve195NRiQF0L5QpgZF0Ba+RqVXzXsAgd9wMD+7WGGGEpa2l8VyuqDAb0R+XvgUKTTf+RkDl/ySReqQj79YLkuYC0OT91tlVFwenIYvubOroJVA1gK29HBbXTCZDgurIOh+qE33uEXvHPmqERqIuQdUO89daBkXUFLJarHcR3dSjIF7n81zn+KVUt/b3XU00EsB1ejMtV+LJNgPzgZFBsuMDIugyL5YrYmMD0wgbyXXdnN1nks7208QC2w0tW5YOKHKXRTsDIugDKFUE0A8oVQTQDyhVBNAPKFUE0A8oVQTQDyhVBNAPKFUE0A8oVQTSDTK4Igtgz6F0RRDOgXBFEM6BcEUQzoFwRRDOgXBFEM6BcEUQzoFwRRDOgXBFEM6BcEUQzoFwRRDNYLFeM72qIFVVsjv0PrDHsYfS0gsVyPYjxXQ2woorNGWRgrYrvWliUZ1JIg8V3FZGVVKA6evGVXfqOaj6Qilm4zwlY9vamLTv2boxOXBy0RnHUMbBGrmv414JDHuO7AuZWkb7O29aYGFiL47u6Z1d0DNDweSzCHX2dtwqS14KrxncVkb9AXI7q6Pnvv1RRxAUZG5Tpnq9v2JKUtPej6LhdYRFbw6N2bEt4P3HXR2+vjnawt5wOVa4Y3xU+0iqOFN/VNyolnTVLAkZ2VKuGchRFSMNtGlc1MLhcFQMurO3ZaojrIRchRUlkTPKmLYnCa/4Zq9ZFxyWmvbF8vaKwphmSXDG+q6PHd6VB6GjoIDWYCI3HdxUZRK6GAy7ME9KB7t5O3tXXqb0GecuOPWs3KkM2A5u3paxYvVlRWNMMSa4CYLFYrhjflWGv8V2zKh90wnOho4rG3VKFiNBUfFeRQeRqOOBSuT7p5ho3CJ/HIciVxYDVJe9rarmPciXYTq4Y35Vir/FdYWMSn0NWBHBp0meTBCJ1U/FdRcyVqzBEMrnyEwby8nvHAYvhtRtJUMmKqhrg1wddWTlfwMfouJ1BIRsUhTXNqMkV47vy2Gl8Vw7jcc3pPTIZ31XEErmSAbdMrmHhsZExSaDP8CjdXxr+xrQK7Eja5xuwUlFY04yaXMWhx/iu9hbfNfZSRRn9Nguagj5zOxFDpPdIjO+qhopc+Q6oD7h0ngwq14WL396emMYkKrByXfR7O/crSmqdEZIrxncl6tJKfFfykxh3djEuqwpyEfLxXSUFBMiDgN4UCum82AHVAbdIrsCmmKTwSLIeFojevsvBVsKANXLF+K6AOX+LI16pCPs9huTtPb4r6ZK4ibA9sg4MPuCmmTrLb1tC2vrNCUyrsfGpa9bHKco4ABbLFeO72hZHi++aUy0vJv0WcHgBxW6MTtwa/37S3o+XrXCon1sFLJYrYmMwvqtNWeAXMmPOYoXRYUC5IohmQLkiiGZAuSKIZkC5IohmQLkiiGZAuSKIZkC5IohmQLkiiGaQyRVBEHsGvSuCaAaUK4JoBpQrgmgGlCuCaAaUK4JoBpQrgmgGlCuCaAaUK4JoBpQrgmgGlCuCaAaL5YrxXQ2xoorNsf+BHRQHuIThxmK5HsT4rgZYUcXmDDKwox/flUS4Ey0kopwy4qvqJWCgVynWyHUN/1pwyGN8V8DcKtLXedsaEwNrH/Fdn7FoGsxCXxMveaU4RfUSMNCrlKHKFeO7wkdaBeO7GlM1AAV6O7t7r6eyjweuk0hf6nKlnSnPpZGQAHGFzN4Iz408FxNEgcMHeh2SXDG+K8Z3NTu+a++t2i4xrF5Hc52qXEmYEhg00auLEwZq9RPNsyHVN8CYyKoDDh/odUhyFQCLxXLF+K6MsRPf9XgVu5asBn1jEZO6rBi5hI4qNmgk5A8tLJPr014uBo8sOJiIwwd6HUW5YnxXyhiK76or7dDX5eTV9UH3VIZROjeoOLmwgxK5ctNANgISHD7Q66jJFeO78oyh+K7wBCGRRMhywBy5kiWVRXJ1+ECvoyZXyHDzBuO7jp34riSS5UAdCcMj1hUCvZJL6Kii5yVLJBbwUrwuM+Tq8IFeR0iuGN+VzK0xHd+VKxkSy1Y6gkXsCShT382NvHA7LJIr4NiBXq2RK8Z3Bcz5SybxSkXY7zEkP9bjuwooAr0O7RY7dqBXi+WK8V1tC8Z3tTkOHOjVYrkiNgbjuw4PDhnoFeWKIJoB5YogmgHliiCaAeWKIJoB5YogmgHliiCaAeWKIJoB5YogmkEmVwRB7Bn0rgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlQrgiiGSyWK8Z3NcSKKjZHy8FRI6VhuBATWCzXgxjf1QArqticQQZ2lOO7ApFpRbV1bW23ygyjB0peoSxSXNenr6NBxsxnru+ytRvituzYHRIW6T7b0d6rBlgj1zX8a8Ehj/FdAXOrGH+Z9dAxMbB2EN+VvCRd391cUVZ1vbnXIBqVqlyT88uqskwGZJAy02vJlu174pP3bdiSEBYeszkuJWlPRkRUvGOEdRUYqlwxvit8pFUwvqsxVQMgSDFwln+g4vKZXNmlCfdavBDJrJANr5RtCanvbFKGioyO27lqbbSipKYZklwxvivGdzUvvmteXf+zzlpjL0yGSxvoIbEnqf8H30uKiS5XdXgVbEtIW7UuWiHX9Zvj10QoA8RomiHJVQAsFssV47syxkR8V9jZVrX0kehE7Q3ldN0kRVQmC7FFA+fJ5KoyvHIEubLIrt9eqdQl70O52lCuGN+VMibiu3LA7avrGCBVZB5SKldhMGVyVRleObG6vaHhMeFRul8fdFVU1dy83UC867uwj41VlNQ0oyZXjO/KMxbiu0ph2pZalHKlIS0tk+uG6MTwKBJ4Livni780/A10C/nouF1vhIQrSmqaUZMrZLh7gPFdHT2+K2yPWxr4jWtocWM/DfEamJ1fkkeHkVxaXQ7tIeyun/ZeT+WM5ss1aPl6ECdIVEr8znTPBUGKkppmhOSK8V2JusZsfNfQ7FKyxaVtwvb1Jl1dQ8/hikiHyaW1k6215KiFcgV0yR9Kv20CZxvx7uDrc21hjVwxvisAtQb9SybxSkVgXc2tzMdcfFeycxF/ZCJdipSNsMn+D85s7+CEnftXUsVGvBsfl/C+g/3oClgsV4zvalswvqsNmem1NHH3R5vjdr67NcXxtApYLFfExmB8V5syZeaihUtXKYwOA8oVQTQDyhVBNAPKFUE0A8oVQTQDyhVBNAPKFUE0A8oVQTQDyhVBNINMrgiC2DPoXRFEM6BcEUQzoFwRRDOgXBFEM6BcEUQzoFwRRDOgXBFEM6BcEUQzoFwRRDOgXBFEM1gsV4zvaogVVWyO3Q0see+hTV+kiFgh14MY39UAK6rYnEEG1uL4rozIwQIlswE/oFIm86cew9f/p1S393cpYmoNisNHbTUfa+S6hn8tOOQxvitgbhXp67xtjYmBtTy+K0dUZZfh29Ul0HBEJMoeDdmqaFZVroHZhZWXjMfIUjJGoraaz1DlivFd4SOt4lDxXQmx8CwDhRuXa0pt51MuGBeNwSV/ZDO50oEqPn6A8+riCllyOSXfGHsz6xiJ2mo+Q5Irxnd13PiuLOjmN7KwGgpKHjzpqOLeVy7NM2izPaST3B0krz4XXa54OcTnC5Hy5IyRqK3mMyS5CoDFYrlifFeGXcZ3hWUwfUTKo+DISRNvJb1Hiksjd40PRUli9tB2ZHLlL8d4kLsxErXVfEZRrhjflWKH8V3JMrgtnxhNyTWePsu4j2I/eURlAuShQFqWyVXl/ioYI1FbzWfU5IrxXXnsLr4rF6qPi0BHouM1FgnlJcCpO6rZ40Bl92EgV7LXsFCuYyRqq/mMmlwhw80bjO9qb/FdWcsE6V0wANw4FyESBgqW8WT34b+/uDiHtkaU2VVKV+x0e0xXEBbKdYxEbTWfEZIrxncl6tJKfFfWMsGkXLnnI1ntwziw/sDlwwqCrIqhk/297XBIGiTWQrkCYyFqq/lYI1eM7wqY85dM4pWKsB8wSN7e47uaiWKgAnURsj6I9906xkLUVvOxWK4Y39W2OFp812HA4aO2mo/FckVsDMZ3NQPHjtpqPihXBNEMKFcE0QwoVwTRDChXBNEMKFcE0QwoVwTRDChXBNEMKFcE0QwyuSIIYs+gd0UQzYByRRDNgHJFEM2AckUQzYByRRDNgHJFEM2AckUQzYByRRDNgHJFEM2AckUQzWCxXDG+qyFWVLE5GN91LGCxXA9ifFcDrKhicwYZ2OGK7wqIr2KVIb5SWALGdx0a1sh1Df9acMhjfFfA3CrS13nbGhMDO2zxXX2zKmGonz1Rfam3qlwxvuvQGKpcMb4rfKRVxl58V3q/4uGiTMgV47valCHJFeO7jun4rgxVLwrQZjG+q20ZklwFwGKxXDG+K0Oz8V05TMkV47vamFGUK8Z3pWg2viuHKbkKdvJQIC3L5KpyfxVgfFcFoyZXjO/Ko9n4rgyz5YrxXYfOqMkVMty8wfiuGo3vypDLFeO7DisjJFeM70rU5WjxXWFvTNqE0SbVmy+BES4fVhAY33WYsEauGN8VMOcvmcQrFWE/YJC8g8R3VRCI8V2HEYvlivFdbQvGdx0UjO8qYLFcERuD8V3NAOO7MlCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlkckUQxJ5B74ogmgHliiCaAeWKIJoB5YogmgHliiCaAeWKIJoB5YogmgHliiCaAeWKIJoB5YogmsFiuWJ8V0OsqGJz7H9gB8UBLmG4sViuBzG+qwFWVLE5gwzssMR31WUUXcrgIwD47y8uLcnbKisgQOLQFWZK3g9MYt6JdRmqlxBf2aXvqOYjqpiF+5yAZW9v2rJj78boxMVBaxRHNY01cl3DvxYc8hjfFTC3ivR13rbGxMAOW3xXyUvDSVQ+tTHnIPEBaPgSzkJfE69sWfUS/Pdfqijioo0NynTP1zdsSUra+1F03K6wiK3hUTu2JbyfuOujt1dHO8YbT4cqV4zvCh9plTEY35WXK4nWMdBSQt/Trw4U6O3s7r2eyj4euE4ifanLlXamPJdGQgLEFTJ7Izw38nQEDIiMSd60JVF45T9j1brouMS0N5avVxTWIkOSK8Z3HdvxXZlck4vbSJgskw6QyPVWbZcYVq+juU5VriRMCQwauy6if3HC0CgeoHk2pPoGGBNZdWDLjj1rNyrDNwObt6WsWL1ZUViLDEmuAmCxWK4Y35Wh4fiu5Gh7m55EJOqo5mLnqUPbOV7FriWrQd9YpNIyuYSOKjZoJOQPLSyT69PeCrZMkAUHExHkyuLB6pL3NbXcR7naRK4Y35Wi4fiucPQZ3fOTjLHnI4W1oyvt0Nfl5NX1QfdUWpbODSpOctPlcuUjX0lHQAIshtduJAEmK6pqgF8fdGXlfAEfo+N2BoVsUBTWIqMmV4zvyqPd+K5kKJjDp6v35lxe9gZwdxOeICSSCFkOmCNXsqSySK5h4bGRMUmgz/Ao3V8a/sa0CuxI2ucbsFJRWIuMmlwhw80bjO+q1fiu0qNkrIRBMIAvSSJZDtSRMDxiXb4n9BI6quiDlSyRWMBL8brMkOvCxW9vT0xjEhVYuS76vZ37FSU1ygjJFeO7krnlaPFd5UdT4e50cXtLJWLJkFi20hEsYk9AmfpubuSF22GRXIFNMUnhkWQ9LBC9fZdjrIQBa+SK8V0Bc/6SSbxSEfZ7DMk7ZnxXK5D1ZKi3eOosv20Jaes3JzCtxsanrlkfpyijXSyWK8Z3tS2OFt81p1peTPot4AgBit0Ynbg1/v2kvR8vW+EIP7cKWCxXxMZgfNfhYYFfyIw5ixVGrYNyRRDNgHJFEM2AckUQzYByRRDNgHJFEM2AckUQzYByRRDNgHJFEM0gkyuCIPYMelcE0QwoVwTRDChXBNEMKFcE0QwoVwTRDChXBNEMKFcE0QwoVwTRDChXBNEMKFcE0QwWyxXjuxpiRRWbo+XgqJEmw1IiIhbL9SDGdzXAiio2Z5CBHeX4rkBkWlFtXVvbrTLD6IGSVyiLFNf16etokDHzmeu7bO2GuC07doeERbrPdrT3qgHWyHUN/1pwyGN8V8DcKsZfZj10TAysHcR3JS9J13c3V5RVXW/uNYhGpSrX5PyyqiyTARmkzPRasmX7nvjkfRu2JISFx2yOS0nakxERFe8YYV0FhipXjO8KH2kVjO9KI3eoA4IUA2f5Byoun8mVXZpwr8ULkcwK2fBK2ZaQ+s4mZajI6Lidq9ZGK0pqmiHJFeO7YnxX8+K75tX1P+usNfbCZLi0gR4Se5L6f/C9pJjoclWHV8G2hLRV66IVcl2/OX5NhDJAjKYZklwFwGKxXDG+K2NMxHeFnW1VSx+JTtTeUE7XTVJEZbIQWzRwnkyuKsMrR5Ari+z67ZVKXfI+lKsN5YrxXSljIr4rB9y+uo4BUkXmIaVyFQZTJleV4ZUTq9sbGh4THqX79UFXRVXNzdsNxLu+C/vYWEVJTTNqcsX4rjxjIb6rFKZtqUUpVxrS0jK5bohODI8igeeycr74S8PfQLeQj47b9UZIuKKkphk1uUKGuwcY39XR47vC9rilgd+4hhY39tMQr4HZ+SV5dBjJpdXl0B7C7vpp7/VUzmi+XIOWrwdxgkSlxO9M91wQpCipaUZIrhjflahrzMZ3Dc0uJVtc2gfYvt6kO3/oOVwR6TC5tHaytZYctVCugC75Q+m3TeBsI94dfH2uLayRK8Z3Bcz5SybxSkVgXc2tzMdcfFeyc5H+0UVYRKRshE32f3Bmewcn7Ny/kio24t34uIT3HexHV8BiuWJ8V9uC8V1tyEyvpYm7P9oct/PdrSmOp1XAYrkiNgbju9qUKTMXLVy6SmF0GFCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlQrgiiGVCuCKIZUK4IohlkckUQxJ5B74ogmgHliiCaAeWKIJoB5YogmgHliiCaAeWKIJoB5YogmgHliiCaAeWKIJoB5YogmsFiuWJ8V0OsqGJz7G5gyXsPxbexIzbBYrkexPiuBlhRxeYMMrAWxncl4ucCF6i91VWEDfgByetIeTJ/6uFDpYikVLf3dyliag2Kw0dtNR9r5LqGfy045DG+K2BuFenrvG2NiYG1Ir4raY1/L7m++ZLiKA8NR0Si7NGQrYpmVeUamF1YeYnGLjCLMRK11XyGKleM7wofaRWHiu8KrcEVScuokFLb+ZQLxkVjcMkf2UyudKCKjx/gvLq4QpZcTsk3xt7MOkaitprPkOSK8V0dNb4rXFFLiUz5KpQ8eNJRxb2vXJpn0GZ7SCe5O0hefS66XPFyiM8XIuXJGSNRW81nSHIVAIvFcsX4rgy7jO8KGfKYIE8fGElFFY408VbSe6S4NHLX+FCUJGYPDXgjkyt/OXCNhstmyhiJ2mo+oyhXjO9KscP4rqQ896zxh0cD96xUEk+fZdxHsZ88ojIB8lAgLcvkqnJ/FYyRqK3mM2pyxfiuPHYX31VyFCAFSsMVRgqcuqOaPQ5Udh8GciV7DQvlOkaitprPqMkVMty8wfiu9hbfNbK4gt8a0CFVdo8D3DgXIRIGCpbxZPfhv7+4OIe2RpTZVUpX7HR7TFcQFsp1jERtNZ8RkivGdyXq0kp819C86x3c2Z/AQBl+2cZDxUxW+zAOrD9w+bCCIKti6GR/bzscIiHY9Y1FgoYtkCswFqK2mo81csX4roA5f8kkXqkI+wGD5O09vivpkriJMIpioAJ1EbI+iPfdOsZC1FbzsViuGN/VtjhafNdhwOGjtpqPxXJFbAzGdzUDx47aaj4oVwTRDChXBNEMKFcE0QwoVwTRDChXBNEMKFcE0QwoVwTRDChXBNEMMrkiCGLPoHdFEM2AckUQzYByRRDNgHJFEM0wanL9Hx4LEK2juKfIcDO8clXcXWTsoJgJiE2wvVwVtw1BAMUkQazDlnJV3CEEkaKYLYgV2EyuinuDIIYo5gxiKaMjV0VdRKMobuugKKojlmKxXI3Fd1XcGIZQS3hvIMZ3HSbsbWAVM4GhKINYisVyPWgkvqvpGyO8iVf69uphAOO7Ku0cwxXfFRBfxSqDvlJYOSswvuvQsEaua/jXgkNeiO+qvDHyWiMhV4zvamRghy2+q29WJQz1syeqL/VWlSvGdx0aQ5WrEN9VeWPktdTlivFdObQZ35Xer3i4KAvkivFdh8SQ5CqN76q8MfJaKnLF+K4cmo3vyqCyNCLXAeWsEAuLl4PxXc1nSHIVAIvyxsgLCCKReFeM70rRbHxXDlNy7VLOCplcMb6rxYyiXDG+K0Wz8V05TMnVYDEsk6vK/VWA8V0VjJpcMb4rj2bjuzKGU64Y31XBqMkVMty8wfiuGo3vypDLVRHfVTkrLJQrxndVMEJyxfiuRF2OFt8V9sakTRhtUpj+3gOXDysIIb6rclZYKFcA47tKsUauqvFdlTdGWRHjuzLYDxgk7yDxXRUEyuK7DjYrBgfju0qxWK7G4rsqbgxDURcxxFHjuypmAkNRxkwwvquAxXI1huLGmI+inTGHNuO7Km6imSgaMR+M78oYfbkiYwfFnEEsxWZyBRT3BkGkKGYLYgW2lCtDcZMQRDFDEKuxvVylKG4bMnZQzATEJgyvXM1BcZsR+0dxB5ERY/TliiCImaBcEUQzoFwRRDPI5IogiD2D3hVBNAPKFUE0A8oVQTQDyhVBNAPKFUE0A8oVQTQDyhVBNAPKFUE0A8oVQTQDyhVBNIPFcjUW39U0wnsDMb7rMGH/AzsoDnAJw43Fcj1oJL6raYQ38UrfXj0MYHxXpZ1jWOK76jKKLmXwEQD89xeXluRtlRUQIAH7CjMl7wcmMe/EugzVS4iv7NJ3VPMRVczCfU7Asrc3bdmxd2N04uKgNYqjmsYaua7hXwsOeSG+q1BAlZGQK8Z3NTKwwxbfVfLqdhKVT23MOUh8ABq+hLPQ18QLrwXnUL0E//2XKoq4kAKDMt3z9Q1bkpL2fhQdtyssYmt41I5tCe8n7vro7dXRjvHG06HKVYjvCroVyhiiLleM78qh0fiuvFxpuE2TgeqgQG9nd+/1VPbxwHUS6UtdrrQz5bk0EhIgrpDZG+G5kacjYEBkTPKmLYnCK/8Zq9ZFxyWmvbF8vaKwFhmSXKXxXSEVyhiiIleM78qh3fiuTK7JxW0kTJZJB0jkequ2Swyr19FcpypXEqYEBo1dF9G/OGFoFA/QPBtSfQOMiaw6sGXHnrUbleGbgc3bUlas3qworEWGJFcBsFgsV4zvytBwfFci1/Y2WDY/e9JRzcXOU4cK+3gVu5asBn1jEZO6rBi5hI4qNmgk5A8tLJPr094KtkyQBQcTEeTK4sHqkvc1tdx3HLm6+/7/6syvpc1kNl0AAAAASUVORK5CYII=)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The prefix 'q5_1' signifies the quantization method we used. I won't delve into too many details, but to determine the best method in each case, I follow the rule that 'q8' yields superior responses at the cost of higher memory usage [slow]. On the other hand, 'q2' may generate subpar responses but requires less RAM [fast].\n",
        "\n",
        "There are other quantization methods available, and you can read about them in the [model card](https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML)"
      ],
      "metadata": {
        "id": "4L7AZFe_7Px0"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "oI-kXwg5bHF-"
      },
      "outputs": [],
      "source": [
        "model_name_or_path = \"TheBloke/Llama-2-13B-chat-GGML\"\n",
        "model_basename = \"llama-2-13b-chat.ggmlv3.q5_1.bin\" # the model is in bin format"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8Td05XSuiWdI"
      },
      "source": [
        "First, we download the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "8271376097504c9899da9229452e6166",
            "8be84652e8b841e690d368db3d1fb47e",
            "be88ec18e1fc4572a9231ba1edf1cdc5",
            "5d56bbe7214c49aa9fbad9ae0c0cf3d8",
            "6e0173da75fd462ea7563da052b73f64",
            "bbc0f3ef435b4fa3bb615fbc43ceca9a",
            "38554d251118436cb32ed3916f452280",
            "9139682a6fc24728b92f68217df8a8c9",
            "a7276dc8b5c14b1a87d66381b45735c8",
            "30b97f6eea224138b91b2a98ed5c0f0c",
            "5366f26373ea4bba813bfebcd1b650fc"
          ]
        },
        "id": "cBEJr-G-2ht4",
        "outputId": "c85c9d35-f3cf-4ae3-de77-74038046e32f"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)chat.ggmlv3.q5_1.bin:   0%|          | 0.00/9.76G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "8271376097504c9899da9229452e6166"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "from huggingface_hub import hf_hub_download\n",
        "\n",
        "model_path = hf_hub_download(repo_id=model_name_or_path, filename=model_basename)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "max1jwxvCSbm"
      },
      "source": [
        "# Inference with llama-cpp-python"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8TOfnZpj394g"
      },
      "source": [
        "Loading the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oY1bItJu4Zfv",
        "outputId": "37fd66d7-e521-49ee-d9ed-d883f5ff7996"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 | \n"
          ]
        }
      ],
      "source": [
        "# GPU\n",
        "from llama_cpp import Llama\n",
        "lcpp_llm = None\n",
        "lcpp_llm = Llama(\n",
        "    model_path=model_path,\n",
        "    n_threads=2, # CPU cores\n",
        "    n_batch=512, # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.\n",
        "    n_gpu_layers=43, # Change this value based on your model and your GPU VRAM pool.\n",
        "    n_ctx=4096, # Context window\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "For run in CPU\n",
        "```\n",
        "# CPU\n",
        "from llama_cpp import Llama\n",
        "\n",
        "lcpp_llm = Llama(\n",
        "    model_path=model_path,\n",
        "    n_threads=2, # CPU cores\n",
        "    )\n",
        "```\n",
        "\n"
      ],
      "metadata": {
        "id": "UdZnPtB8-Bhx"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BeH6eWiKuaxW",
        "outputId": "cf3a1ea5-37ff-48ec-ea06-56d9587e3c2d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "43"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "# See the number of layers in GPU\n",
        "lcpp_llm.params.n_gpu_layers"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qLEEOufGVlID"
      },
      "source": [
        "We will use this prompt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "-NzVIlMCVoVD"
      },
      "outputs": [],
      "source": [
        "prompt = \"Write a linear regression in python\"\n",
        "prompt_template=f'''SYSTEM: You are a helpful, respectful and honest assistant. Always answer as helpfully.\n",
        "\n",
        "USER: {prompt}\n",
        "\n",
        "ASSISTANT:\n",
        "'''"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qaFQjPhcFgfN"
      },
      "source": [
        "Generating response\n",
        "\n",
        "If you only use CPU, the response can take a long time. You can reduce the max_tokens to get a faster response."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R76uxL293jTc",
        "outputId": "c9d109f0-8803-47b1-b297-af2cf2683d6c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Llama.generate: prefix-match hit\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "SYSTEM: You are a helpful, respectful and honest assistant. Always answer as helpfully.\n",
            "\n",
            "USER: Write a linear regression in python\n",
            "\n",
            "ASSISTANT:\n",
            "\n",
            "To write a linear regression in Python, you can use the scikit-learn library. Here is an example of how to do this:\n",
            "```\n",
            "from sklearn.linear_model import LinearRegression\n",
            "import pandas as pd\n",
            "\n",
            "# Load your dataset into a Pandas DataFrame\n",
            "df = pd.read_csv('your_data.csv')\n",
            "\n",
            "# Create a linear regression object and fit the data\n",
            "reg = LinearRegression()\n",
            "reg.fit(df[['x1', 'x2']], df['y'])\n",
            "\n",
            "# Print the coefficients\n",
            "print(reg.coef_)\n",
            "```\n",
            "This will print out the coefficients for your linear regression model. You can also use the `predict()` method to make predictions on new data, like this:\n",
            "```\n",
            "# Make a prediction on some new data\n",
            "new_data = pd.DataFrame({'x1': [2, 3], 'x2': [4, 5]})\n",
            "prediction = reg.predict(new_data)\n",
            "print(prediction)\n",
            "```\n",
            "This will print out the predicted values for your new data.\n",
            "\n",
            "I hope this helps! Let me know if you have any questions or need further assistance.\n"
          ]
        }
      ],
      "source": [
        "response = lcpp_llm(\n",
        "    prompt=prompt_template,\n",
        "    max_tokens=256,\n",
        "    temperature=0.5,\n",
        "    top_p=0.95,\n",
        "    repeat_penalty=1.2,\n",
        "    top_k=50,\n",
        "    stop = ['USER:'], # Dynamic stopping when such token is detected.\n",
        "    echo=True # return the prompt\n",
        ")\n",
        "\n",
        "print(response[\"choices\"][0][\"text\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "no531n827gI9"
      },
      "source": [
        "# Inference with langchain\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zG2uFQFl7jrq",
        "outputId": "b5ae2201-4eb6-49f7-a51b-a5616199111b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip -q install langchain"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-Kl8UYvXHYIt"
      },
      "source": [
        "We can use the model that we loaded earlier. However, for illustrative purposes, we will load one from the 'langchain' library. Due to vRAM limitations, before running these cells, you need to delete the previous model from memory."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "NWaQuYH0AT1z"
      },
      "outputs": [],
      "source": [
        "lcpp_llm.reset()\n",
        "lcpp_llm.set_cache(None)\n",
        "lcpp_llm = None\n",
        "del lcpp_llm"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "Prompt for the model."
      ],
      "metadata": {
        "id": "scri08pVDp8U"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "N5ijTbW6_dI_"
      },
      "outputs": [],
      "source": [
        "from langchain.llms import LlamaCpp\n",
        "from langchain import PromptTemplate, LLMChain\n",
        "from langchain.callbacks.manager import CallbackManager\n",
        "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\n",
        "from huggingface_hub import hf_hub_download\n",
        "\n",
        "template = \"\"\"''SYSTEM: You are a helpful, respectful and honest assistant. Always answer as helpfully.\n",
        "USER: {question}\n",
        "ASSISTANT: \"\"\"\n",
        "prompt = PromptTemplate(template=template, input_variables=[\"question\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hx06Ea1DH8ky"
      },
      "source": [
        "Stream tokens"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "iVIXxNoIHOe_"
      },
      "outputs": [],
      "source": [
        "# Callbacks support token-wise streaming\n",
        "callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])\n",
        "# Verbose is required to pass to the callback manager"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G2DHWIr-IEDE"
      },
      "source": [
        "Load the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iipsXBGhHNjf",
        "outputId": "00fb9a1b-d985-4d70-d23a-7ec79f37f834"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 | \n"
          ]
        }
      ],
      "source": [
        "n_gpu_layers = 40  # Change this value based on your model and your GPU VRAM pool.\n",
        "n_batch = 512  # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.\n",
        "\n",
        "# Loading model,\n",
        "llm = LlamaCpp(\n",
        "    model_path=model_path,\n",
        "    max_tokens=1024,\n",
        "    n_gpu_layers=n_gpu_layers,\n",
        "    n_batch=n_batch,\n",
        "    callback_manager=callback_manager,\n",
        "    verbose=True,\n",
        "    n_ctx=4096, # Context window\n",
        "    stop = ['USER:'], # Dynamic stopping when such token is detected.\n",
        "    temperature = 0.4,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JZ8kbRz2F9vi"
      },
      "source": [
        "Generating response"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 663
        },
        "id": "Lw4MaS4WBlR-",
        "outputId": "722b9e87-f391-4463-aa15-dc545f7a6898"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Llama.generate: prefix-match hit\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Sure thing! Here is an example of how you could write a simple linear regression in Python using scikit-learn library:\n",
            "```\n",
            "from sklearn.linear_model import LinearRegression\n",
            "import pandas as pd\n",
            "\n",
            "# Load your dataset\n",
            "df = pd.read_csv('your_data.csv')\n",
            "\n",
            "# Create X and y vectors\n",
            "X = df[['feature1', 'feature2']]\n",
            "y = df['target']\n",
            "\n",
            "# Create a linear regression object and fit the data\n",
            "reg = LinearRegression()\n",
            "reg.fit(X, y)\n",
            "\n",
            "# Print the coefficients\n",
            "print(reg.coef_)\n",
            "\n",
            "# Print the R-squared value\n",
            "print(reg.score(X, y))\n",
            "```\n",
            "This code assumes that you have a CSV file containing your data, with the features in the first two columns and the target variable in the last column. It creates X and y vectors from the data, fits a linear regression model to the data using the `fit()` method, prints the coefficients of the model using the `coef_` attribute, and prints the R-squared value using the `score()` method.\n",
            "\n",
            "I hope this helps! Let me know if you have any questions or need further assistance.\n",
            "```\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\" Sure thing! Here is an example of how you could write a simple linear regression in Python using scikit-learn library:\\n```\\nfrom sklearn.linear_model import LinearRegression\\nimport pandas as pd\\n\\n# Load your dataset\\ndf = pd.read_csv('your_data.csv')\\n\\n# Create X and y vectors\\nX = df[['feature1', 'feature2']]\\ny = df['target']\\n\\n# Create a linear regression object and fit the data\\nreg = LinearRegression()\\nreg.fit(X, y)\\n\\n# Print the coefficients\\nprint(reg.coef_)\\n\\n# Print the R-squared value\\nprint(reg.score(X, y))\\n```\\nThis code assumes that you have a CSV file containing your data, with the features in the first two columns and the target variable in the last column. It creates X and y vectors from the data, fits a linear regression model to the data using the `fit()` method, prints the coefficients of the model using the `coef_` attribute, and prints the R-squared value using the `score()` method.\\n\\nI hope this helps! Let me know if you have any questions or need further assistance.\\n```\\n\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "llm_chain = LLMChain(prompt=prompt, llm=llm)\n",
        "\n",
        "question = \"Write a simple linear regression in python\"\n",
        "\n",
        "llm_chain.run(question)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Inference with llama.cpp"
      ],
      "metadata": {
        "id": "IdTSFrMqF-yu"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "To use llama.cpp directly, we must clone the repository. In this example, we will use only the CPU."
      ],
      "metadata": {
        "id": "AjwqmnwYZlyR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!git clone https://github.com/ggerganov/llama.cpp\n",
        "%cd llama.cpp\n",
        "!git checkout dadbed99e65252d79f81101a392d0d6497b86caa # For compatibility with GGML"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Xwt3kdw4Rnmt",
        "outputId": "bbbcb889-4b78-4ded-a496-1f17faa43b67"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into 'llama.cpp'...\n",
            "remote: Enumerating objects: 7911, done.\u001b[K\n",
            "remote: Counting objects: 100% (4150/4150), done.\u001b[K\n",
            "remote: Compressing objects: 100% (567/567), done.\u001b[K\n",
            "remote: Total 7911 (delta 3913), reused 3658 (delta 3582), pack-reused 3761\u001b[K\n",
            "Receiving objects: 100% (7911/7911), 7.34 MiB | 21.05 MiB/s, done.\n",
            "Resolving deltas: 100% (5488/5488), done.\n",
            "/content/llama.cpp\n",
            "Note: switching to 'dadbed99e65252d79f81101a392d0d6497b86caa'.\n",
            "\n",
            "You are in 'detached HEAD' state. You can look around, make experimental\n",
            "changes and commit them, and you can discard any commits you make in this\n",
            "state without impacting any branches by switching back to a branch.\n",
            "\n",
            "If you want to create a new branch to retain commits you create, you may\n",
            "do so (now or later) by using -c with the switch command. Example:\n",
            "\n",
            "  git switch -c <new-branch-name>\n",
            "\n",
            "Or undo this operation with:\n",
            "\n",
            "  git switch -\n",
            "\n",
            "Turn off this advice by setting config variable advice.detachedHead to false\n",
            "\n",
            "HEAD is now at dadbed9 metal : fix synchronization in new matrix multiplication kernel (#2686)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Build llama.cpp\n",
        "\n",
        "We will use the model only with the CPU."
      ],
      "metadata": {
        "id": "yjhMVxTVZ9o0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# CPU\n",
        "!make\n",
        "\n",
        "# GPU\n",
        "#!make LLAMA_CUBLAS=1"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8E38icaHSLB_",
        "outputId": "a1f852d7-b838-4084-d0ae-81ffb0d50532"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "I llama.cpp build info: \n",
            "I UNAME_S:  Linux\n",
            "I UNAME_P:  x86_64\n",
            "I UNAME_M:  x86_64\n",
            "I CFLAGS:   -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS\n",
            "I CXXFLAGS: -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS\n",
            "I LDFLAGS:  \n",
            "I CC:       cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0\n",
            "I CXX:      g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0\n",
            "\n",
            "cc  -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS   -c ggml.c -o ggml.o\n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS -c llama.cpp -o llama.o\n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS -c examples/common.cpp -o common.o\n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS -c examples/console.cpp -o console.o\n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS -c examples/grammar-parser.cpp -o grammar-parser.o\n",
            "cc -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS   -c -o k_quants.o k_quants.c\n",
            "cc  -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS   -c ggml-alloc.c -o ggml-alloc.o\n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/main/main.cpp ggml.o llama.o common.o console.o grammar-parser.o k_quants.o ggml-alloc.o -o main \n",
            "\n",
            "====  Run ./main -h for help.  ====\n",
            "\n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/quantize/quantize.cpp ggml.o llama.o k_quants.o ggml-alloc.o -o quantize \n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/quantize-stats/quantize-stats.cpp ggml.o llama.o k_quants.o ggml-alloc.o -o quantize-stats \n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/perplexity/perplexity.cpp ggml.o llama.o common.o k_quants.o ggml-alloc.o -o perplexity \n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/embedding/embedding.cpp ggml.o llama.o common.o k_quants.o ggml-alloc.o -o embedding \n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS pocs/vdot/vdot.cpp ggml.o k_quants.o ggml-alloc.o -o vdot \n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/train-text-from-scratch/train-text-from-scratch.cpp ggml.o llama.o k_quants.o ggml-alloc.o -o train-text-from-scratch \n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp ggml.o llama.o k_quants.o ggml-alloc.o -o convert-llama2c-to-ggml \n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/simple/simple.cpp ggml.o llama.o common.o k_quants.o ggml-alloc.o -o simple \n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS -Iexamples/server examples/server/server.cpp ggml.o llama.o common.o grammar-parser.o k_quants.o ggml-alloc.o -o server  \n",
            "g++ --shared -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/embd-input/embd-input-lib.cpp ggml.o llama.o common.o k_quants.o ggml-alloc.o -o libembdinput.so \n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/embd-input/embd-input-test.cpp ggml.o llama.o common.o k_quants.o ggml-alloc.o -o embd-input-test  -L. -lembdinput\n",
            "g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS examples/llama-bench/llama-bench.cpp ggml.o llama.o common.o k_quants.o ggml-alloc.o -o llama-bench \n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!./main -t 2 -m {model_path} --color -c 128 --temp 0.7 -n 56 -p \"USER: Write a linear regression in python\\nASSISTANT:\""
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oMNdFcnte8-K",
        "outputId": "351ece94-4391-418d-f599-85b202c23ad9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "main: build = 852 (294f424)\n",
            "main: seed  = 1689809714\n",
            "llama.cpp: loading model from /root/.cache/huggingface/hub/models--TheBloke--Llama-2-13B-chat-GGML/snapshots/e3b15539668fb5740b42fa01e0e2f04ce1d0a3ee/llama-2-13b-chat.ggmlv3.q5_1.bin\n",
            "llama_model_load_internal: format     = ggjt v3 (latest)\n",
            "llama_model_load_internal: n_vocab    = 32000\n",
            "llama_model_load_internal: n_ctx      = 128\n",
            "llama_model_load_internal: n_embd     = 5120\n",
            "llama_model_load_internal: n_mult     = 256\n",
            "llama_model_load_internal: n_head     = 40\n",
            "llama_model_load_internal: n_layer    = 40\n",
            "llama_model_load_internal: n_rot      = 128\n",
            "llama_model_load_internal: freq_base  = 10000.0\n",
            "llama_model_load_internal: freq_scale = 1\n",
            "llama_model_load_internal: ftype      = 9 (mostly Q5_1)\n",
            "llama_model_load_internal: n_ff       = 13824\n",
            "llama_model_load_internal: model size = 13B\n",
            "llama_model_load_internal: ggml ctx size =    0.09 MB\n",
            "llama_model_load_internal: mem required  = 11113.06 MB (+ 1608.00 MB per state)\n",
            "llama_new_context_with_model: kv self size  =  100.00 MB\n",
            "\n",
            "system_info: n_threads = 2 / 2 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | \n",
            "sampling: repeat_last_n = 64, repeat_penalty = 1.100000, presence_penalty = 0.000000, frequency_penalty = 0.000000, top_k = 40, tfs_z = 1.000000, top_p = 0.950000, typical_p = 1.000000, temp = 0.700000, mirostat = 0, mirostat_lr = 0.100000, mirostat_ent = 5.000000\n",
            "generate: n_ctx = 128, n_batch = 512, n_predict = 56, n_keep = 0\n",
            "\n",
            "\n",
            "\u001b[33m USER: Write a linear regression in python\\nASSISTANT:\u001b[0m Sure, here's an example of how to implement linear regression in Python using scikit-learn library.\n",
            "\n",
            "import pandas as pd\n",
            "from sklearn.linear_model import LinearRegression\n",
            "from sklearn.model_selection import train_test_split\n",
            "\n",
            "\n",
            "llama_print_timings:        load time = 50655.89 ms\n",
            "llama_print_timings:      sample time =    70.93 ms /    56 runs   (    1.27 ms per token,   789.53 tokens per second)\n",
            "llama_print_timings: prompt eval time = 50291.40 ms /    16 tokens ( 3143.21 ms per token,     0.32 tokens per second)\n",
            "llama_print_timings:        eval time = 1488997.82 ms /    55 runs   (27072.69 ms per token,     0.04 tokens per second)\n",
            "llama_print_timings:       total time = 1539377.00 ms\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Details about the different parameters."
      ],
      "metadata": {
        "id": "wwxfqy_daf5F"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<tbody><tr><td class=\"l\"></td><td class=\"l\"> </td><td class=\"l\"> <b>param value</b> </td><td class=\"l\"></td></tr>\n",
        "  <tr><td class=\"l\">  <code>-h</code> </td><td class=\"l\"> <code>--help</code> </td><td class=\"l\"> </td><td class=\"l\"> Show this help message and exit</td></tr>\n",
        "  <tr><td class=\"l\">  <code>-i</code> </td><td class=\"l\"> <code>--interactive</code> </td><td class=\"l\"> </td><td class=\"l\"> Run in interactive mode</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>--interactive-first</code> </td><td class=\"l\"> </td><td class=\"l\"> Run in interactive mode and wait for input right away</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>-ins</code>, <code>--instruct</code> </td><td class=\"l\">  </td><td class=\"l\"> Run in instruction mode (use with Alpaca models)</td></tr>\n",
        "  <tr><td class=\"l\">  <code>-r</code> </td><td class=\"l\"> <code>--reverse-prompt</code> </td><td class=\"l\"> <code>PROMPT</code> </td><td class=\"l\"> Run in interactive mode and poll user input upon seeing <code>PROMPT</code> (can be specified more than once for multiple prompts).</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>--color</code> </td><td class=\"l\">  </td><td class=\"l\"> Colorise output to distinguish prompt and user input from generations</td></tr>\n",
        "  <tr><td class=\"l\">  <code>-s</code> </td><td class=\"l\"> <code>--seed</code> </td><td class=\"l\"> <code>SEED</code> </td><td class=\"l\"> Seed for random number generator (default: <code>-1</code>, use random seed for &lt;= 0)</td></tr>\n",
        "  <tr><td class=\"l\">  <code>-t</code> </td><td class=\"l\"> <code>--threads</code> </td><td class=\"l\"> <code>N</code> </td><td class=\"l\"> Number of threads to use during computation (default: 12)</td></tr>\n",
        "  <tr><td class=\"l\">  <code>-p</code> </td><td class=\"l\"> <code>--prompt</code> </td><td class=\"l\"> <code>PROMPT</code> </td><td class=\"l\"> Prompt to start generation with (default: empty)</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>--random-prompt</code> </td><td class=\"l\"> </td><td class=\"l\"> Start with a randomized prompt.</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>--in-prefix</code> </td><td class=\"l\"> <code>STRING</code> </td><td class=\"l\"> String to prefix user inputs with (default: empty)</td></tr>\n",
        "  <tr><td class=\"l\">  <code>-f</code> </td><td class=\"l\"> <code>--file</code> </td><td class=\"l\"> <code>FNAME</code> </td><td class=\"l\"> Prompt file to start generation.</td></tr>\n",
        "  <tr><td class=\"l\">  <code>-n</code> </td><td class=\"l\"> <code>--n_predict</code> </td><td class=\"l\"> <code>N</code> </td><td class=\"l\"> Number of tokens to predict (default: 128, -1 = infinity)</td></tr>\n",
        "  <tr><td class=\"l\">   </td><td class=\"l\"> <code>--top_k</code> </td><td class=\"l\"> <code>N</code> </td><td class=\"l\"> Top-k sampling (default: 40)</td></tr>\n",
        "  <tr><td class=\"l\"> </td><td class=\"l\"> <code>--top_p</code> </td><td class=\"l\"> <code>N</code> </td><td class=\"l\"> Top-p sampling (default: 0.9)</td></tr>\n",
        "  <tr><td class=\"l\"> </td><td class=\"l\"> <code>--repeat_last_n</code> </td><td class=\"l\"> <code>N</code> </td><td class=\"l\"> Last n tokens to consider for penalize (default: 64)</td></tr>\n",
        "  <tr><td class=\"l\"> </td><td class=\"l\"> <code>--repeat_penalty</code> </td><td class=\"l\"> <code>N</code> </td><td class=\"l\"> Penalize repeat sequence of tokens (default: 1.1)</td></tr>\n",
        "  <tr><td class=\"l\"> <code>-c</code> </td><td class=\"l\"> <code>--ctx_size</code> </td><td class=\"l\"> <code>N</code> </td><td class=\"l\"> Size of the prompt context (default: <code>512</code>)</td></tr>\n",
        "  <tr><td class=\"l\"> </td><td class=\"l\"> <code>--ignore-eos</code> </td><td class=\"l\"> </td><td class=\"l\"> Ignore end of stream token and continue generating</td></tr>\n",
        "  <tr><td class=\"l\"> </td><td class=\"l\"> <code>--memory_f32</code> </td><td class=\"l\"> </td><td class=\"l\"> Use <code>f32</code> instead of <code>f16</code> for memory key+value</td></tr>\n",
        "  <tr><td class=\"l\"> </td><td class=\"l\"> <code>--temp</code> </td><td class=\"l\"> <code>N</code> </td><td class=\"l\"> Temperature (default: <code>0.8</code>)</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>--n_parts</code> </td><td class=\"l\"> <code>N</code> </td><td class=\"l\"> Number of model parts (default: -1 = determine from dimensions)</td></tr>\n",
        "  <tr><td class=\"l\">  <code>-b</code> </td><td class=\"l\"> <code>--batch_size</code> </td><td class=\"l\"> <code>N</code> </td><td class=\"l\"> Batch size for prompt processing (default: 8)</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>--perplexity</code> </td><td class=\"l\"> </td><td class=\"l\"> Compute perplexity over the prompt</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>--keep</code> </td><td class=\"l\">   </td><td class=\"l\"> Number of tokens to keep from the initial prompt (default: 0, -1 = all)</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>--mlock</code> </td><td class=\"l\">  </td><td class=\"l\"> Force system to keep model in RAM rather than swapping or compressing</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>--mtest</code> </td><td class=\"l\">  </td><td class=\"l\"> Determine the maximum memory usage needed to do inference for the given <code>n_batch</code> and <code>n_predict</code> parameters (uncomment the <code>\"used_mem\"</code> line in <code>llama.cpp</code> to see the results)</td></tr>\n",
        "  <tr><td class=\"l\">  </td><td class=\"l\"> <code>--verbose-prompt</code> </td><td class=\"l\">   </td><td class=\"l\"> Print prompt before generation</td></tr>\n",
        "  <tr><td class=\"l\">  <code>-m</code> </td><td class=\"l\"> <code>--model</code> </td><td class=\"l\"> <code>FNAME</code> </td><td class=\"l\"> Model path (default: <code>models/llama-7B/ggml-model.bin</code>)</td></tr>\n",
        "\n",
        "</tbody>"
      ],
      "metadata": {
        "id": "ZdNp0ixPevHC"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cQcO_xyCJyGZ"
      },
      "source": [
        "\n",
        "\n",
        "\n",
        "# What is better: GPTQ or GGML?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MU6o0MlQJ2K3"
      },
      "source": [
        "- GPTQ is a specific format for GPU only.\n",
        "\n",
        "- GGML is designed for CPU and Apple M series but can also offload some layers on the GPU\n",
        "\n",
        "- GGMLs like q4_2, q4_3, q5_0, q5_1 and q8_0 have superior inference quality and outperform GPTQ on benchmarks"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# References\n",
        "\n",
        "- [renenyffenegger.ch - LLaMA C++ Library](https://renenyffenegger.ch/notes/development/Artificial-intelligence/language-model/LLM/LLaMA/libs/llama_cpp/)\n",
        "- [LLaMA C++ Library Documentation](https://llama-cpp-python.readthedocs.io/en/latest/)\n",
        "- [MacOS Install with Metal GPU - LLaMA C++ Library Documentation](https://llama-cpp-python.readthedocs.io/en/latest/install/macos/)\n"
      ],
      "metadata": {
        "id": "YX8Fx_n8fWwe"
      }
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "authorship_tag": "ABX9TyMe4pV9dRmjVou1yAutKzVp",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "8271376097504c9899da9229452e6166": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_8be84652e8b841e690d368db3d1fb47e",
              "IPY_MODEL_be88ec18e1fc4572a9231ba1edf1cdc5",
              "IPY_MODEL_5d56bbe7214c49aa9fbad9ae0c0cf3d8"
            ],
            "layout": "IPY_MODEL_6e0173da75fd462ea7563da052b73f64"
          }
        },
        "8be84652e8b841e690d368db3d1fb47e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bbc0f3ef435b4fa3bb615fbc43ceca9a",
            "placeholder": "​",
            "style": "IPY_MODEL_38554d251118436cb32ed3916f452280",
            "value": "Downloading (…)chat.ggmlv3.q5_1.bin: 100%"
          }
        },
        "be88ec18e1fc4572a9231ba1edf1cdc5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9139682a6fc24728b92f68217df8a8c9",
            "max": 9763701888,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a7276dc8b5c14b1a87d66381b45735c8",
            "value": 9763701888
          }
        },
        "5d56bbe7214c49aa9fbad9ae0c0cf3d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_30b97f6eea224138b91b2a98ed5c0f0c",
            "placeholder": "​",
            "style": "IPY_MODEL_5366f26373ea4bba813bfebcd1b650fc",
            "value": " 9.76G/9.76G [01:20&lt;00:00, 152MB/s]"
          }
        },
        "6e0173da75fd462ea7563da052b73f64": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bbc0f3ef435b4fa3bb615fbc43ceca9a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "38554d251118436cb32ed3916f452280": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9139682a6fc24728b92f68217df8a8c9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a7276dc8b5c14b1a87d66381b45735c8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "30b97f6eea224138b91b2a98ed5c0f0c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5366f26373ea4bba813bfebcd1b650fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}