LLM_Inference__GGUF__WizardCoder_Python_34B.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/R3gm/InsightSolver-Colab/blob/main/LLM_Inference__GGUF__WizardCoder_Python_34B.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ihaZyg2aYQpH"
      },
      "source": [
        "# WizardCoder-Python-34B on Colab's free tier with GGUF\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "`GGUF` is an enhancement over the \"llama.cpp\" file format, addressing the constraints of the current \".bin\" files. Unlike the existing format, GGUF permits inclusion of supplementary model information in a more adaptable manner and supports a wider range of model types"
      ],
      "metadata": {
        "id": "XLCdN7r6G8fY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "llama-cpp-python allows us to perform inference with quantized language models. For more details on how to use it, you can visit the following notebook at [![Open](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/R3gm/InsightSolver-Colab/blob/main/LLM_Inference_with_llama_cpp_python__Llama_2_13b_chat.ipynb)."
      ],
      "metadata": {
        "id": "70wxZebAKmm1"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OBnzlAqYZM6_"
      },
      "source": [
        "| Code Credits | Link |\n",
        "| ----------- | ---- |\n",
        "| 🎉 llama-cpp-python | [![GitHub Repository](https://img.shields.io/github/stars/abetlen/llama-cpp-python?style=social)](https://github.com/abetlen/llama-cpp-python) |\n",
        "| 🔥 Discover More Colab Notebooks | [![GitHub Repository](https://img.shields.io/badge/GitHub-Repository-black?style=flat-square&logo=github)](https://github.com/R3gm/InsightSolver-Colab/) |\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rkBmY3vQvRSw"
      },
      "outputs": [],
      "source": [
        "# GPU llama-cpp-python; Starting from version llama-cpp-python==0.1.79, it supports GGUF\n",
        "!CMAKE_ARGS=\"-DLLAMA_CUBLAS=on\" FORCE_CMAKE=1 pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir\n",
        "# For download the models\n",
        "!pip install huggingface_hub"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Select the model\n",
        "\n",
        "When we use GGUF, we can offload model layers to the GPU, which facilitates inference time; we can do this with all layers, but what will allow us to run large models on a T4 is the support of RAM on the CPU. Therefore, we will use both the GPU and CPU for inference. Since Colab only provides us with 2 CPU cores, this inference can be quite slow, but it will still allow us to run models like llama 2 70B that have been quantized previously."
      ],
      "metadata": {
        "id": "QclMRDCf2e__"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We will use the quantized model [WizardCoder-Python-34B-V1.0-GGUF](https://huggingface.co/TheBloke/WizardCoder-Python-34B-V1.0-GGUF) from [WizardCoder Python 34B](https://huggingface.co/WizardLM/WizardCoder-Python-34B-V1.0) with the k-quants method Q4_K_M\n"
      ],
      "metadata": {
        "id": "P-NMhJug3Rjo"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "oI-kXwg5bHF-"
      },
      "outputs": [],
      "source": [
        "model_name_or_path = \"TheBloke/WizardCoder-Python-34B-V1.0-GGUF\"\n",
        "model_basename = \"wizardcoder-python-34b-v1.0.Q4_K_M.gguf\""
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "About k-quants\n",
        "\n",
        "`k-quants` are a series of quantization methods ranging from 2 to 6 bits, designed to enhance both model size and performance for language generation tasks. The primary objective is to empower users to choose the most suitable quantized model considering their hardware constraints, all while upholding high-quality generation capabilities. Notably, these methods introduce only marginal quantization errors; for instance, 6-bit quantization maintains a perplexity within 0.1% of the original fp16 model's performance. This suite of quantization techniques empowers models to gracefully adapt to memory constraints on devices, all while delivering remarkable generation performance."
      ],
      "metadata": {
        "id": "P6DtS-N2I5Zq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Table about perplexity on the wikitext dataset as a function of model size\n",
        "\n",
        "\n",
        "![link text](https://user-images.githubusercontent.com/48489457/243093269-07aa49f0-4951-407f-9789-0b5a01ce95b8.png)\n",
        "\n",
        "\n",
        "Note: Perplexity is a metric used to evaluate language models by measuring how well they predict sequences of words, with lower values indicating better predictive performance."
      ],
      "metadata": {
        "id": "-Q_xhDzKGJWG"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8Td05XSuiWdI"
      },
      "source": [
        "First, we download the model"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from huggingface_hub import hf_hub_download\n",
        "\n",
        "model_path = hf_hub_download(\n",
        "    repo_id=model_name_or_path,\n",
        "    filename=model_basename,\n",
        "    cache_dir= '/content/models' # Directory for the model\n",
        ")"
      ],
      "metadata": {
        "id": "KwqiZ_eKUKOl",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "2c6195cbac9e45d0bf43bc0091664f3d",
            "d9d712b86dda4c92bb1a6902ab810507",
            "b32e6a58e7f549c7a612d889512dc3f0",
            "00bbbe5d0fcd4412a1187e7c231ab448",
            "70b1ff012c5e43b38d0e19bfc1c312b3",
            "a06617deb9fe4477933496bfb93aa522",
            "b85ce354b1684778bcc06e6437861267",
            "c5d0b10884ba47d89b22451dd23200c1",
            "60d9b1e2e90d489bb496a295cb46efae",
            "f949ef2676774b358cfb41c903d516dd",
            "90d7e6401ecd4f1ba7a3e9901f5061b0"
          ]
        },
        "outputId": "43684689-a8da-4d91-93a6-7a5b7016e892"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)34b-v1.0.Q4_K_M.gguf:   0%|          | 0.00/20.5G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "2c6195cbac9e45d0bf43bc0091664f3d"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "max1jwxvCSbm"
      },
      "source": [
        "## Stream inference with llama-cpp-python"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8TOfnZpj394g"
      },
      "source": [
        "Loading the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oY1bItJu4Zfv",
        "outputId": "17f75b0e-6d5f-4466-91e0-426fbbf73ac9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | \n"
          ]
        }
      ],
      "source": [
        "from llama_cpp import Llama\n",
        "\n",
        "llm = Llama(\n",
        "    model_path=model_path,\n",
        "    n_threads=2, # CPU cores\n",
        "    n_batch=512, # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.\n",
        "    n_gpu_layers=30, # The max for this model is 30 in a T4, If you use llama 2 70B, you'll need to put fewer layers on the GPU\n",
        "    n_ctx=4096, # Context window\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qLEEOufGVlID"
      },
      "source": [
        "Prompt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-NzVIlMCVoVD"
      },
      "outputs": [],
      "source": [
        "prompt = \"Example of linear regression in python\"\n",
        "prompt_template=f'''Below is an instruction that describes a task. Write a response that appropriately completes the request.\n",
        "\n",
        "### Instruction:\n",
        "{prompt}\n",
        "\n",
        "### Response:'''"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qaFQjPhcFgfN"
      },
      "source": [
        "Stream response\n",
        "\n",
        "We are using both the GPU and the CPU for inference, which will allow us to use the model but with a very slow generation."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R76uxL293jTc",
        "outputId": "31252484-f4a8-41ed-d3c3-edfda0203136"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Llama.generate: prefix-match hit\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[34mS\u001b[0m\u001b[34mure\u001b[0m\u001b[34m!\u001b[0m\u001b[34m Here\u001b[0m\u001b[34m'\u001b[0m\u001b[34ms\u001b[0m\u001b[34m an\u001b[0m\u001b[34m example\u001b[0m\u001b[34m of\u001b[0m\u001b[34m how\u001b[0m\u001b[34m to\u001b[0m\u001b[34m perform\u001b[0m\u001b[34m linear\u001b[0m\u001b[34m regression\u001b[0m\u001b[34m using\u001b[0m\u001b[34m Python\u001b[0m\u001b[34m:\u001b[0m\u001b[34m\u001b[0m\u001b[34m\n",
            "\u001b[0m\u001b[34m\u001b[0m\u001b[34m\n",
            "\u001b[0m\u001b[34m\u001b[0m\u001b[34m\n",
            "\u001b[0m\u001b[34m```\u001b[0m\u001b[34mpython\u001b[0m\u001b[34m\u001b[0m\u001b[34m\n",
            "\u001b[0m\u001b[34mimport\u001b[0m\u001b[34m pandas\u001b[0m\u001b[34m as\u001b[0m\u001b[34m pd\u001b[0m\u001b[34m \u001b[0m\u001b[34m\u001b[0m"
          ]
        }
      ],
      "source": [
        "stream = llm(\n",
        "    prompt_template,\n",
        "    max_tokens=16, # Number of new tokens to be generated, increase it for a longer response\n",
        "    temperature=0.8,\n",
        "    top_p=0.95,\n",
        "    repeat_penalty=1.2,\n",
        "    top_k=50,\n",
        "    echo=False,\n",
        "    stream=True,\n",
        "    stop=[\"Instruction:\", \"Response:\"] # Stop generation when such token appears\n",
        ")\n",
        "\n",
        "response = ''\n",
        "for output in stream:\n",
        "    text_output = output['choices'][0]['text'].replace('\\r', '')\n",
        "    print(\"\\033[34m\" + text_output + \"\\033[0m\", end ='')\n",
        "    response += text_output"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(response)"
      ],
      "metadata": {
        "id": "TEpMj9fupRb4"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Convert a GGML model to GGUF"
      ],
      "metadata": {
        "id": "JY0RwpC-MDVh"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "`Restart the runtime` before running this section."
      ],
      "metadata": {
        "id": "HJBBuxytCcXC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We download a GGML model, which is no longer supported currently, but we can convert them to GGUF."
      ],
      "metadata": {
        "id": "8QPZMtph5q-O"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Requirements\n",
        "!pip install gguf\n",
        "!wget https://raw.githubusercontent.com/ggerganov/llama.cpp/master/convert-llama-ggmlv3-to-gguf.py"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d0fNjiRDqdOQ",
        "outputId": "2178085b-40d5-4c92-e551-aeace84becf3"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting gguf\n",
            "  Downloading gguf-0.3.0-py3-none-any.whl (9.7 kB)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from gguf) (1.25.2)\n",
            "Installing collected packages: gguf\n",
            "Successfully installed gguf-0.3.0\n",
            "--2023-08-30 17:57:31--  https://raw.githubusercontent.com/ggerganov/llama.cpp/master/convert-llama-ggmlv3-to-gguf.py\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.109.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 15633 (15K) [text/plain]\n",
            "Saving to: ‘convert-llama-ggmlv3-to-gguf.py’\n",
            "\n",
            "convert-llama-ggmlv 100%[===================>]  15.27K  --.-KB/s    in 0.001s  \n",
            "\n",
            "2023-08-30 17:57:31 (21.7 MB/s) - ‘convert-llama-ggmlv3-to-gguf.py’ saved [15633/15633]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Select the GGML model"
      ],
      "metadata": {
        "id": "_ymi9x8Ri9Dl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model_name_or_path = \"TheBloke/Llama-2-13B-chat-GGML\"\n",
        "model_basename = \"llama-2-13b-chat.ggmlv3.q5_1.bin\""
      ],
      "metadata": {
        "id": "E7-SavmzniiR"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "cBEJr-G-2ht4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "2212f95b97b046758ef394b862103918",
            "f56a1d0d8c614b5dadd823c5d130f1b3",
            "a8f7b3b1a6074184bff776103d0a5481",
            "670888b1252e4bbabf904711aff4f32d",
            "f228232b2c77405b9c5c2a4b5ac0a409",
            "65ba04bd55894885b8f65a904bf29c55",
            "ed077d0806474535a6ff0ff3da24b60a",
            "19c07fc8f9354e8180ddd0f5000bf739",
            "c0b8e9cccf7740c9af732d55895d00fa",
            "1ff43bcb76104566b0af076d3b857c50",
            "9df6a377c494405e9b3487180cd96046"
          ]
        },
        "outputId": "553b937c-5e85-412b-ea58-fe8d034f5aa3"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)chat.ggmlv3.q5_1.bin:   0%|          | 0.00/9.76G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "2212f95b97b046758ef394b862103918"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "from huggingface_hub import hf_hub_download\n",
        "\n",
        "model_path = hf_hub_download(\n",
        "    repo_id=model_name_or_path,\n",
        "    filename=model_basename,\n",
        "    cache_dir= '/content/models'\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Convert GGML to GGUF using the script provided by llama.cpp."
      ],
      "metadata": {
        "id": "Nj9oCeCt56S5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Experimental\n",
        "!python convert-llama-ggmlv3-to-gguf.py\\\n",
        "  -i $model_path \\\n",
        "  -o llama-2-13b-GGUF.gguf \\\n",
        "  --context-length 4096 \\\n",
        "  --eps 1e-5"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9pHpS-WpnSN_",
        "outputId": "905eb8df-bf87-4124-9163-917694c5fa17"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "* Using config: Namespace(input=PosixPath('models--TheBloke--Llama-2-13B-chat-GGML/snapshots/47d28ef5de4f3de523c421f325a2e4e039035bab/llama-2-13b-chat.ggmlv3.q5_1.bin'), output=PosixPath('llama-2-13b-GGUF.gguf'), name=None, desc=None, gqa=1, eps='1e-5', context_length=4096, model_metadata_dir=None, vocab_dir=None, vocabtype='spm')\n",
            "\n",
            "=== WARNING === Be aware that this conversion script is best-effort. Use a native GGUF model if possible. === WARNING ===\n",
            "\n",
            "* Scanning GGML input file\n",
            "* GGML model hyperparameters: <Hyperparameters: n_vocab=32000, n_embd=5120, n_mult=256, n_head=40, n_layer=40, n_rot=128, n_ff=13824, ftype=9>\n",
            "\n",
            "=== WARNING === Special tokens may not be converted correctly. Use --model-metadata-dir if possible === WARNING ===\n",
            "\n",
            "* Preparing to save GGUF file\n",
            "* Adding model parameters and KV items\n",
            "* Adding 32000 vocab item(s)\n",
            "* Adding 363 tensor(s)\n",
            "    gguf: write header\n",
            "    gguf: write metadata\n",
            "    gguf: write tensors\n",
            "* Successful completion. Output saved to: llama-2-13b-GGUF.gguf\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# We select the directory of our converted model.\n",
        "model_path = 'llama-2-13b-GGUF.gguf'"
      ],
      "metadata": {
        "id": "zzxlolVGuSzO"
      },
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from llama_cpp import Llama\n",
        "\n",
        "llm = Llama(\n",
        "    model_path=model_path,\n",
        "    n_threads=2,\n",
        "    n_batch=512,\n",
        "    n_gpu_layers=43, # On a T4, we can offload all layers of a 13B model.\n",
        "    n_ctx=4096,\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "riIysd-iq5Xr",
        "outputId": "cecc4f66-b8f4-4364-840c-6e35f34a312a"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | \n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "prompt = \"Write a poem about llamas.\"\n",
        "prompt_template=f'''SYSTEM: You are a helpful, respectful and honest assistant. Always answer as helpfully.\n",
        "\n",
        "USER: {prompt}\n",
        "\n",
        "ASSISTANT:\n",
        "'''"
      ],
      "metadata": {
        "id": "caMiNo_mo6iF"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Stream inference"
      ],
      "metadata": {
        "id": "CF-Ef_246HYd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stream = llm(\n",
        "    prompt_template,\n",
        "    max_tokens=1024,\n",
        "    temperature=0.7,\n",
        "    top_p=0.95,\n",
        "    repeat_penalty=1.2,\n",
        "    top_k=50,\n",
        "    echo=False,\n",
        "    stream=True,\n",
        "    stop=[\"USER:\", \"ASSISTANT:\", \"SYSTEM:\"]\n",
        ")\n",
        "\n",
        "response = ''\n",
        "for output in stream:\n",
        "    text_output = output['choices'][0]['text'].replace('\\r', '')\n",
        "    response += text_output\n",
        "    print(\"\\033[31m\" + text_output + \"\\033[0m\", end ='')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "923zeIB_XKUI",
        "outputId": "f7aef406-05bd-4378-9f7d-f53fab344df3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[31m\n",
            "\u001b[0m\u001b[31mOh\u001b[0m\u001b[31m ll\u001b[0m\u001b[31mama\u001b[0m\u001b[31m,\u001b[0m\u001b[31m oh\u001b[0m\u001b[31m so\u001b[0m\u001b[31m fine\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mYour\u001b[0m\u001b[31m w\u001b[0m\u001b[31mool\u001b[0m\u001b[31mly\u001b[0m\u001b[31m coat\u001b[0m\u001b[31m,\u001b[0m\u001b[31m your\u001b[0m\u001b[31m gentle\u001b[0m\u001b[31m eyes\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mYou\u001b[0m\u001b[31m ro\u001b[0m\u001b[31mam\u001b[0m\u001b[31m the\u001b[0m\u001b[31m And\u001b[0m\u001b[31me\u001b[0m\u001b[31man\u001b[0m\u001b[31m high\u001b[0m\u001b[31mlands\u001b[0m\u001b[31m divine\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mA\u001b[0m\u001b[31m symbol\u001b[0m\u001b[31m of\u001b[0m\u001b[31m grace\u001b[0m\u001b[31m and\u001b[0m\u001b[31m pride\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mWith\u001b[0m\u001b[31m ne\u001b[0m\u001b[31mcks\u001b[0m\u001b[31m that\u001b[0m\u001b[31m b\u001b[0m\u001b[31mend\u001b[0m\u001b[31m and\u001b[0m\u001b[31m legs\u001b[0m\u001b[31m that\u001b[0m\u001b[31m stretch\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mYou\u001b[0m\u001b[31m str\u001b[0m\u001b[31mut\u001b[0m\u001b[31m across\u001b[0m\u001b[31m the\u001b[0m\u001b[31m mountains\u001b[0m\u001b[31mide\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mYour\u001b[0m\u001b[31m sp\u001b[0m\u001b[31mit\u001b[0m\u001b[31m-\u001b[0m\u001b[31mcurl\u001b[0m\u001b[31ms\u001b[0m\u001b[31m dri\u001b[0m\u001b[31mpping\u001b[0m\u001b[31m,\u001b[0m\u001b[31m ears\u001b[0m\u001b[31m ere\u001b[0m\u001b[31mct\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mA\u001b[0m\u001b[31m creature\u001b[0m\u001b[31m of\u001b[0m\u001b[31m beauty\u001b[0m\u001b[31m,\u001b[0m\u001b[31m inside\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mIn\u001b[0m\u001b[31m the\u001b[0m\u001b[31m fields\u001b[0m\u001b[31m you\u001b[0m\u001b[31m gra\u001b[0m\u001b[31mze\u001b[0m\u001b[31m all\u001b[0m\u001b[31m day\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mM\u001b[0m\u001b[31munch\u001b[0m\u001b[31ming\u001b[0m\u001b[31m on\u001b[0m\u001b[31m grass\u001b[0m\u001b[31mes\u001b[0m\u001b[31m with\u001b[0m\u001b[31m g\u001b[0m\u001b[31mlee\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mYour\u001b[0m\u001b[31m gentle\u001b[0m\u001b[31m hum\u001b[0m\u001b[31m fills\u001b[0m\u001b[31m the\u001b[0m\u001b[31m air\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mAs\u001b[0m\u001b[31m you\u001b[0m\u001b[31m watch\u001b[0m\u001b[31m the\u001b[0m\u001b[31m world\u001b[0m\u001b[31m go\u001b[0m\u001b[31m by\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mWith\u001b[0m\u001b[31m a\u001b[0m\u001b[31m soft\u001b[0m\u001b[31mness\u001b[0m\u001b[31m in\u001b[0m\u001b[31m your\u001b[0m\u001b[31m eyes\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mYou\u001b[0m\u001b[31m gaz\u001b[0m\u001b[31me\u001b[0m\u001b[31m out\u001b[0m\u001b[31m at\u001b[0m\u001b[31m the\u001b[0m\u001b[31m sky\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mAnd\u001b[0m\u001b[31m though\u001b[0m\u001b[31m you\u001b[0m\u001b[31m may\u001b[0m\u001b[31m not\u001b[0m\u001b[31m speak\u001b[0m\u001b[31m my\u001b[0m\u001b[31m language\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mI\u001b[0m\u001b[31m know\u001b[0m\u001b[31m that\u001b[0m\u001b[31m you\u001b[0m\u001b[31m understand\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mOh\u001b[0m\u001b[31m ll\u001b[0m\u001b[31mama\u001b[0m\u001b[31m,\u001b[0m\u001b[31m oh\u001b[0m\u001b[31m so\u001b[0m\u001b[31m grand\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mA\u001b[0m\u001b[31m symbol\u001b[0m\u001b[31m of\u001b[0m\u001b[31m the\u001b[0m\u001b[31m And\u001b[0m\u001b[31mes\u001b[0m\u001b[31m land\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mF\u001b[0m\u001b[31more\u001b[0m\u001b[31mver\u001b[0m\u001b[31m ro\u001b[0m\u001b[31maming\u001b[0m\u001b[31m,\u001b[0m\u001b[31m free\u001b[0m\u001b[31m and\u001b[0m\u001b[31m wild\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mIn\u001b[0m\u001b[31m this\u001b[0m\u001b[31m poem\u001b[0m\u001b[31m,\u001b[0m\u001b[31m I\u001b[0m\u001b[31m have\u001b[0m\u001b[31m found\u001b[0m\u001b[31m.\u001b[0m"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(response)"
      ],
      "metadata": {
        "id": "CJmVaYKaIXi7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# References\n",
        "- [k-quants](https://github.com/ggerganov/llama.cpp/pull/1684)\n",
        "- [renenyffenegger.ch - LLaMA C++ Library](https://renenyffenegger.ch/notes/development/Artificial-intelligence/language-model/LLM/LLaMA/libs/llama_cpp/)\n",
        "- [LLaMA C++ Library Documentation](https://llama-cpp-python.readthedocs.io/en/latest/)\n",
        "- [MacOS Install with Metal GPU - LLaMA C++ Library Documentation](https://llama-cpp-python.readthedocs.io/en/latest/install/macos/)\n"
      ],
      "metadata": {
        "id": "YX8Fx_n8fWwe"
      }
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "authorship_tag": "ABX9TyMe7gPsi7ranonFc/YWyufC",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU",
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "2c6195cbac9e45d0bf43bc0091664f3d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d9d712b86dda4c92bb1a6902ab810507",
              "IPY_MODEL_b32e6a58e7f549c7a612d889512dc3f0",
              "IPY_MODEL_00bbbe5d0fcd4412a1187e7c231ab448"
            ],
            "layout": "IPY_MODEL_70b1ff012c5e43b38d0e19bfc1c312b3"
          }
        },
        "d9d712b86dda4c92bb1a6902ab810507": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a06617deb9fe4477933496bfb93aa522",
            "placeholder": "​",
            "style": "IPY_MODEL_b85ce354b1684778bcc06e6437861267",
            "value": "Downloading (…)34b-v1.0.Q4_K_M.gguf: 100%"
          }
        },
        "b32e6a58e7f549c7a612d889512dc3f0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c5d0b10884ba47d89b22451dd23200c1",
            "max": 20529068416,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_60d9b1e2e90d489bb496a295cb46efae",
            "value": 20529068416
          }
        },
        "00bbbe5d0fcd4412a1187e7c231ab448": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f949ef2676774b358cfb41c903d516dd",
            "placeholder": "​",
            "style": "IPY_MODEL_90d7e6401ecd4f1ba7a3e9901f5061b0",
            "value": " 20.5G/20.5G [07:43&lt;00:00, 50.5MB/s]"
          }
        },
        "70b1ff012c5e43b38d0e19bfc1c312b3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a06617deb9fe4477933496bfb93aa522": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b85ce354b1684778bcc06e6437861267": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c5d0b10884ba47d89b22451dd23200c1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "60d9b1e2e90d489bb496a295cb46efae": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f949ef2676774b358cfb41c903d516dd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "90d7e6401ecd4f1ba7a3e9901f5061b0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2212f95b97b046758ef394b862103918": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f56a1d0d8c614b5dadd823c5d130f1b3",
              "IPY_MODEL_a8f7b3b1a6074184bff776103d0a5481",
              "IPY_MODEL_670888b1252e4bbabf904711aff4f32d"
            ],
            "layout": "IPY_MODEL_f228232b2c77405b9c5c2a4b5ac0a409"
          }
        },
        "f56a1d0d8c614b5dadd823c5d130f1b3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_65ba04bd55894885b8f65a904bf29c55",
            "placeholder": "​",
            "style": "IPY_MODEL_ed077d0806474535a6ff0ff3da24b60a",
            "value": "Downloading (…)chat.ggmlv3.q5_1.bin: 100%"
          }
        },
        "a8f7b3b1a6074184bff776103d0a5481": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_19c07fc8f9354e8180ddd0f5000bf739",
            "max": 9763701888,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c0b8e9cccf7740c9af732d55895d00fa",
            "value": 9763701888
          }
        },
        "670888b1252e4bbabf904711aff4f32d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1ff43bcb76104566b0af076d3b857c50",
            "placeholder": "​",
            "style": "IPY_MODEL_9df6a377c494405e9b3487180cd96046",
            "value": " 9.76G/9.76G [01:40&lt;00:00, 95.5MB/s]"
          }
        },
        "f228232b2c77405b9c5c2a4b5ac0a409": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "65ba04bd55894885b8f65a904bf29c55": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ed077d0806474535a6ff0ff3da24b60a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "19c07fc8f9354e8180ddd0f5000bf739": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c0b8e9cccf7740c9af732d55895d00fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1ff43bcb76104566b0af076d3b857c50": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9df6a377c494405e9b3487180cd96046": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}