diff --git a/README.md b/README.md index 9106736..9f45550 100644 --- a/README.md +++ b/README.md @@ -133,6 +133,7 @@ | :----------------------------------------------------------- | ---------------------------- | :--: | :--: | :--: | :--: | :--: | :--: | :----------------------------------------------------------: | | [**llama.cpp**](https://github.com/ggerganov/llama.cpp) | 丰富的量化选项和高效本地推理 | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | [link](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/wiki/llamacpp_zh) | | [**🤗Transformers**](https://github.com/huggingface/transformers) | 原生transformers推理接口 | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | [link](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/wiki/inference_with_transformers_zh) | +| [**Colab Demo**](https://colab.research.google.com/drive/1yu0eZ3a66by8Zqm883LLtRQrguBAb9MR?usp=sharing) | 在Colab中启动交互界面 | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | [link](https://colab.research.google.com/drive/1yu0eZ3a66by8Zqm883LLtRQrguBAb9MR?usp=sharing) | | [**仿OpenAI API调用**](https://platform.openai.com/docs/api-reference) | 仿OpenAI API接口的服务器Demo | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | [link](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/wiki/api_calls_zh) | ⚠️ 一代模型相关推理与部署支持将陆续迁移到本项目，届时将同步更新相关教程。 diff --git a/README_EN.md b/README_EN.md index 472b99a..fc04e97 100644 --- a/README_EN.md +++ b/README_EN.md @@ -127,6 +127,7 @@ The models in this project mainly support the following quantization, inference, | :----------------------------------------------------------- | ------------------------------------------------------- | :--: | :--: | :---: | :--: | :--: | :--: | :----------------------------------------------------------: | | [**llama.cpp**](https://github.com/ggerganov/llama.cpp) | Rich quantization options and efficient local inference | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | [link](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/wiki/llamacpp_en) | | [**🤗Transformers**](https://github.com/huggingface/transformers) | Native transformers inference interface | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | [link](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/wiki/inference_with_transformers_en) | +| [**Colab Demo**](https://colab.research.google.com/drive/1yu0eZ3a66by8Zqm883LLtRQrguBAb9MR?usp=sharing) | Running a Gradio web demo in Colab | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | [link](https://colab.research.google.com/drive/1yu0eZ3a66by8Zqm883LLtRQrguBAb9MR?usp=sharing) | | [**OpenAI API Calls**](https://platform.openai.com/docs/api-reference) | A server that implements OpenAI API | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | [link](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/wiki/api_calls_en) | ⚠️ Inference and deployment support related to the first-generation model will be gradually migrated to this project, and relevant tutorials will be updated later. diff --git a/notebooks/gradio_web_demo.ipynb b/notebooks/gradio_web_demo.ipynb new file mode 100644 index 0000000..ba97b2b --- /dev/null +++ b/notebooks/gradio_web_demo.ipynb @@ -0,0 +1,430 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "provenance": [], + "machine_shape": "hm", + "gpuType": "V100" + }, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + }, + "language_info": { + "name": "python" + }, + "accelerator": "GPU" + }, + "cells": [ + { + "cell_type": "markdown", + "source": [ + "## 通过Colab启动Web Demo\n", + "\n", + "本notebook提供了使用Colab中的GPU架设基于Gradio的demo示例。\n", + "\n", + "项目地址：https://github.com/ymcui/Chinese-LLaMA-Alpaca-2\n", + "\n", + "启动前准备：\n", + "- V100/A100：可直接加载7B模型\n", + "- T4：需要启用8bit加载（见以下启动说明）" + ], + "metadata": { + "id": "xidqUxwtsBnl" + } + }, + { + "cell_type": "markdown", + "source": [ + "### 克隆代码并安装依赖" + ], + "metadata": { + "id": "BNc29-zks3p5" + } + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "zPqpnapOr70x", + "outputId": "e297adf1-b934-492d-ff97-4f1c2c8c738a" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Cloning into 'Chinese-LLaMA-Alpaca-2'...\n", + "remote: Enumerating objects: 242, done.\u001b[K\n", + "remote: Counting objects: 100% (84/84), done.\u001b[K\n", + "remote: Compressing objects: 100% (66/66), done.\u001b[K\n", + "remote: Total 242 (delta 37), reused 39 (delta 18), pack-reused 158\u001b[K\n", + "Receiving objects: 100% (242/242), 362.76 KiB | 3.42 MiB/s, done.\n", + "Resolving deltas: 100% (104/104), done.\n", + "Collecting git+https://github.com/huggingface/peft.git@13e53fc (from -r Chinese-LLaMA-Alpaca-2/requirements.txt (line 2))\n", + " Cloning https://github.com/huggingface/peft.git (to revision 13e53fc) to /tmp/pip-req-build-edd6aj5p\n", + " Running command git clone --filter=blob:none --quiet https://github.com/huggingface/peft.git /tmp/pip-req-build-edd6aj5p\n", + "\u001b[33m WARNING: Did not find branch or tag '13e53fc', assuming revision or ref.\u001b[0m\u001b[33m\n", + "\u001b[0m Running command git checkout -q 13e53fc\n", + " Resolved https://github.com/huggingface/peft.git to commit 13e53fc\n", + " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", + " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", + " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", + "Requirement already satisfied: torch==2.0.1 in /usr/local/lib/python3.10/dist-packages (from -r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (2.0.1+cu118)\n", + "Collecting transformers==4.31.0 (from -r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3))\n", + " Downloading transformers-4.31.0-py3-none-any.whl (7.4 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.4/7.4 MB\u001b[0m \u001b[31m30.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting sentencepiece==0.1.97 (from -r Chinese-LLaMA-Alpaca-2/requirements.txt (line 4))\n", + " Downloading sentencepiece-0.1.97-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m57.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting bitsandbytes==0.39.1 (from -r Chinese-LLaMA-Alpaca-2/requirements.txt (line 5))\n", + " Downloading bitsandbytes-0.39.1-py3-none-any.whl (97.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m97.1/97.1 MB\u001b[0m \u001b[31m11.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (3.12.2)\n", + "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (4.7.1)\n", + "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (1.11.1)\n", + "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (3.1)\n", + "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (3.1.2)\n", + "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (2.0.0)\n", + "Collecting huggingface-hub<1.0,>=0.14.1 (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3))\n", + " Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m31.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (1.22.4)\n", + "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (23.1)\n", + "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (6.0.1)\n", + "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (2022.10.31)\n", + "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (2.27.1)\n", + "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3))\n", + " Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m120.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting safetensors>=0.3.1 (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3))\n", + " Downloading safetensors-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m81.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (4.65.0)\n", + "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (3.25.2)\n", + "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (16.0.6)\n", + "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from peft==0.3.0.dev0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 2)) (5.9.5)\n", + "Collecting accelerate (from peft==0.3.0.dev0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 2))\n", + " Downloading accelerate-0.21.0-py3-none-any.whl (244 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m244.2/244.2 kB\u001b[0m \u001b[31m30.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.14.1->transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (2023.6.0)\n", + "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (2.1.3)\n", + "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (1.26.16)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (2023.7.22)\n", + "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (2.0.12)\n", + "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (3.4)\n", + "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (1.3.0)\n", + "Building wheels for collected packages: peft\n", + " Building wheel for peft (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for peft: filename=peft-0.3.0.dev0-py3-none-any.whl size=40651 sha256=b925814e35e3f918ed8cf768c39e74046f9687af061d572ba75f446b8cdb631d\n", + " Stored in directory: /tmp/pip-ephem-wheel-cache-oj3_2rx9/wheels/d9/13/c6/404d5f8a81c5620f65f7fd75b6a66619f013cd79c2875b981c\n", + "Successfully built peft\n", + "Installing collected packages: tokenizers, sentencepiece, safetensors, bitsandbytes, huggingface-hub, transformers, accelerate, peft\n", + "Successfully installed accelerate-0.21.0 bitsandbytes-0.39.1 huggingface-hub-0.16.4 peft-0.3.0.dev0 safetensors-0.3.1 sentencepiece-0.1.97 tokenizers-0.13.3 transformers-4.31.0\n", + "Collecting gradio\n", + " Downloading gradio-3.39.0-py3-none-any.whl (19.9 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.9/19.9 MB\u001b[0m \u001b[31m72.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting aiofiles<24.0,>=22.0 (from gradio)\n", + " Downloading aiofiles-23.1.0-py3-none-any.whl (14 kB)\n", + "Requirement already satisfied: aiohttp~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.8.5)\n", + "Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (4.2.2)\n", + "Collecting fastapi (from gradio)\n", + " Downloading fastapi-0.100.1-py3-none-any.whl (65 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m65.8/65.8 kB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting ffmpy (from gradio)\n", + " Downloading ffmpy-0.3.1.tar.gz (5.5 kB)\n", + " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Collecting gradio-client>=0.3.0 (from gradio)\n", + " Downloading gradio_client-0.3.0-py3-none-any.whl (294 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m294.2/294.2 kB\u001b[0m \u001b[31m31.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting httpx (from gradio)\n", + " Downloading httpx-0.24.1-py3-none-any.whl (75 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.4/75.4 kB\u001b[0m \u001b[31m10.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: huggingface-hub>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (0.16.4)\n", + "Requirement already satisfied: jinja2<4.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.1.2)\n", + "Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.0.0)\n", + "Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.1.3)\n", + "Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.7.1)\n", + "Collecting mdit-py-plugins<=0.3.3 (from gradio)\n", + " Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.5/50.5 kB\u001b[0m \u001b[31m6.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: numpy~=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (1.22.4)\n", + "Collecting orjson~=3.0 (from gradio)\n", + " Downloading orjson-3.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.7/138.7 kB\u001b[0m \u001b[31m16.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gradio) (23.1)\n", + "Requirement already satisfied: pandas<3.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (1.5.3)\n", + "Requirement already satisfied: pillow<11.0,>=8.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (9.4.0)\n", + "Requirement already satisfied: pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,<3.0.0,>=1.7.4 in /usr/local/lib/python3.10/dist-packages (from gradio) (1.10.12)\n", + "Collecting pydub (from gradio)\n", + " Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)\n", + "Collecting python-multipart (from gradio)\n", + " Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.7/45.7 kB\u001b[0m \u001b[31m5.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: pyyaml<7.0,>=5.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (6.0.1)\n", + "Requirement already satisfied: requests~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.27.1)\n", + "Collecting semantic-version~=2.0 (from gradio)\n", + " Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)\n", + "Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (4.7.1)\n", + "Collecting uvicorn>=0.14.0 (from gradio)\n", + " Downloading uvicorn-0.23.2-py3-none-any.whl (59 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m59.5/59.5 kB\u001b[0m \u001b[31m7.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting websockets<12.0,>=10.0 (from gradio)\n", + " Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m14.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (23.1.0)\n", + "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (2.0.12)\n", + "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (6.0.4)\n", + "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (4.0.2)\n", + "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (1.9.2)\n", + "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (1.4.0)\n", + "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (1.3.1)\n", + "Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (0.4)\n", + "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (4.3.3)\n", + "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (0.12.0)\n", + "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client>=0.3.0->gradio) (2023.6.0)\n", + "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.14.0->gradio) (3.12.2)\n", + "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.14.0->gradio) (4.65.0)\n", + "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio) (0.1.2)\n", + "Requirement already satisfied: linkify-it-py<3,>=1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio) (2.0.2)\n", + "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (1.1.0)\n", + "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (0.11.0)\n", + "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (4.41.1)\n", + "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (1.4.4)\n", + "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (3.1.0)\n", + "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (2.8.2)\n", + "INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.\n", + "Collecting mdit-py-plugins<=0.3.3 (from gradio)\n", + " Downloading mdit_py_plugins-0.3.2-py3-none-any.whl (50 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.4/50.4 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading mdit_py_plugins-0.3.1-py3-none-any.whl (46 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.5/46.5 kB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading mdit_py_plugins-0.3.0-py3-none-any.whl (43 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading mdit_py_plugins-0.2.8-py3-none-any.whl (41 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading mdit_py_plugins-0.2.7-py3-none-any.whl (41 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading mdit_py_plugins-0.2.6-py3-none-any.whl (39 kB)\n", + " Downloading mdit_py_plugins-0.2.5-py3-none-any.whl (39 kB)\n", + "INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.\n", + " Downloading mdit_py_plugins-0.2.4-py3-none-any.whl (39 kB)\n", + " Downloading mdit_py_plugins-0.2.3-py3-none-any.whl (39 kB)\n", + " Downloading mdit_py_plugins-0.2.2-py3-none-any.whl (39 kB)\n", + " Downloading mdit_py_plugins-0.2.1-py3-none-any.whl (38 kB)\n", + " Downloading mdit_py_plugins-0.2.0-py3-none-any.whl (38 kB)\n", + "INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n", + " Downloading mdit_py_plugins-0.1.0-py3-none-any.whl (37 kB)\n", + "Collecting markdown-it-py[linkify]>=2.0.0 (from gradio)\n", + " Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading markdown_it_py-2.2.0-py3-none-any.whl (84 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.5/84.5 kB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3.0,>=1.0->gradio) (2022.7.1)\n", + "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio) (1.26.16)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio) (2023.7.22)\n", + "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio) (3.4)\n", + "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio) (8.1.6)\n", + "Collecting h11>=0.8 (from uvicorn>=0.14.0->gradio)\n", + " Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting starlette<0.28.0,>=0.27.0 (from fastapi->gradio)\n", + " Downloading starlette-0.27.0-py3-none-any.whl (66 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting httpcore<0.18.0,>=0.15.0 (from httpx->gradio)\n", + " Downloading httpcore-0.17.3-py3-none-any.whl (74 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m74.5/74.5 kB\u001b[0m \u001b[31m9.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio) (1.3.0)\n", + "Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/lib/python3.10/dist-packages (from httpcore<0.18.0,>=0.15.0->httpx->gradio) (3.7.1)\n", + "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio) (0.19.3)\n", + "Requirement already satisfied: uc-micro-py in /usr/local/lib/python3.10/dist-packages (from linkify-it-py<3,>=1->markdown-it-py[linkify]>=2.0.0->gradio) (1.0.2)\n", + "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib~=3.0->gradio) (1.16.0)\n", + "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5.0,>=3.0->httpcore<0.18.0,>=0.15.0->httpx->gradio) (1.1.2)\n", + "Building wheels for collected packages: ffmpy\n", + " Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for ffmpy: filename=ffmpy-0.3.1-py3-none-any.whl size=5579 sha256=57ade8007aa0450b3a03c935d39c4bf930a2cd6896c0e9d5c0687136d78ccfcc\n", + " Stored in directory: /root/.cache/pip/wheels/01/a6/d1/1c0828c304a4283b2c1639a09ad86f83d7c487ef34c6b4a1bf\n", + "Successfully built ffmpy\n", + "Installing collected packages: pydub, ffmpy, websockets, semantic-version, python-multipart, orjson, markdown-it-py, h11, aiofiles, uvicorn, starlette, mdit-py-plugins, httpcore, httpx, fastapi, gradio-client, gradio\n", + " Attempting uninstall: markdown-it-py\n", + " Found existing installation: markdown-it-py 3.0.0\n", + " Uninstalling markdown-it-py-3.0.0:\n", + " Successfully uninstalled markdown-it-py-3.0.0\n", + " Attempting uninstall: mdit-py-plugins\n", + " Found existing installation: mdit-py-plugins 0.4.0\n", + " Uninstalling mdit-py-plugins-0.4.0:\n", + " Successfully uninstalled mdit-py-plugins-0.4.0\n", + "Successfully installed aiofiles-23.1.0 fastapi-0.100.1 ffmpy-0.3.1 gradio-3.39.0 gradio-client-0.3.0 h11-0.14.0 httpcore-0.17.3 httpx-0.24.1 markdown-it-py-2.2.0 mdit-py-plugins-0.3.3 orjson-3.9.2 pydub-0.25.1 python-multipart-0.0.6 semantic-version-2.10.0 starlette-0.27.0 uvicorn-0.23.2 websockets-11.0.3\n" + ] + } + ], + "source": [ + "!git clone https://github.com/ymcui/Chinese-LLaMA-Alpaca-2.git\n", + "!pip install -r Chinese-LLaMA-Alpaca-2/requirements.txt\n", + "!pip install gradio" + ] + }, + { + "cell_type": "markdown", + "source": [ + "### 下载Alpaca-2-7B完整模型" + ], + "metadata": { + "id": "aqD4OUDBtaKM" + } + }, + { + "cell_type": "code", + "source": [ + "!git clone https://huggingface.co/ziqingyang/chinese-alpaca-2-7b" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "4R7_FhEUtfFq", + "outputId": "321d12ae-9a56-47bc-8d0e-c2adc35e3c1d" + }, + "execution_count": 2, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Cloning into 'chinese-alpaca-2-7b'...\n", + "remote: Enumerating objects: 18, done.\u001b[K\n", + "remote: Counting objects: 5% (1/18)\u001b[K\rremote: Counting objects: 11% (2/18)\u001b[K\rremote: Counting objects: 16% (3/18)\u001b[K\rremote: Counting objects: 22% (4/18)\u001b[K\rremote: Counting objects: 27% (5/18)\u001b[K\rremote: Counting objects: 33% (6/18)\u001b[K\rremote: Counting objects: 38% (7/18)\u001b[K\rremote: Counting objects: 44% (8/18)\u001b[K\rremote: Counting objects: 50% (9/18)\u001b[K\rremote: Counting objects: 55% (10/18)\u001b[K\rremote: Counting objects: 61% (11/18)\u001b[K\rremote: Counting objects: 66% (12/18)\u001b[K\rremote: Counting objects: 72% (13/18)\u001b[K\rremote: Counting objects: 77% (14/18)\u001b[K\rremote: Counting objects: 83% (15/18)\u001b[K\rremote: Counting objects: 88% (16/18)\u001b[K\rremote: Counting objects: 94% (17/18)\u001b[K\rremote: Counting objects: 100% (18/18)\u001b[K\rremote: Counting objects: 100% (18/18), done.\u001b[K\n", + "remote: Compressing objects: 5% (1/17)\u001b[K\rremote: Compressing objects: 11% (2/17)\u001b[K\rremote: Compressing objects: 17% (3/17)\u001b[K\rremote: Compressing objects: 23% (4/17)\u001b[K\rremote: Compressing objects: 29% (5/17)\u001b[K\rremote: Compressing objects: 35% (6/17)\u001b[K\rremote: Compressing objects: 41% (7/17)\u001b[K\rremote: Compressing objects: 47% (8/17)\u001b[K\rremote: Compressing objects: 52% (9/17)\u001b[K\rremote: Compressing objects: 58% (10/17)\u001b[K\rremote: Compressing objects: 64% (11/17)\u001b[K\rremote: Compressing objects: 70% (12/17)\u001b[K\rremote: Compressing objects: 76% (13/17)\u001b[K\rremote: Compressing objects: 82% (14/17)\u001b[K\rremote: Compressing objects: 88% (15/17)\u001b[K\rremote: Compressing objects: 94% (16/17)\u001b[K\rremote: Compressing objects: 100% (17/17)\u001b[K\rremote: Compressing objects: 100% (17/17), done.\u001b[K\n", + "remote: Total 18 (delta 2), reused 0 (delta 0), pack-reused 0\u001b[K\n", + "Unpacking objects: 100% (18/18), 4.58 KiB | 1.14 MiB/s, done.\n", + "Filtering content: 100% (3/3), 4.90 GiB | 24.03 MiB/s, done.\n", + "Encountered 1 file(s) that may not have been copied correctly on Windows:\n", + "\tpytorch_model-00001-of-00002.bin\n", + "\n", + "See: `git lfs help smudge` for more details.\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "### 启动Demo\n" + ], + "metadata": { + "id": "EZCv3HS3tohu" + } + }, + { + "cell_type": "code", + "source": [ + "!python Chinese-LLaMA-Alpaca-2/scripts/inference/gradio_demo.py --base_model /content/chinese-alpaca-2-7b --load_in_8bit" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "l9lURhKSuJVM", + "outputId": "ef5ea01e-e5bb-4ceb-dcc3-4af5e4083027" + }, + "execution_count": 23, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\n", + "===================================BUG REPORT===================================\n", + "Welcome to bitsandbytes. For bug reports, please run\n", + "\n", + "python -m bitsandbytes\n", + "\n", + " and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues\n", + "================================================================================\n", + "bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118_nocublaslt.so\n", + "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /usr/lib64-nvidia did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...\n", + " warn(msg)\n", + "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/sys/fs/cgroup/memory.events /var/colab/cgroup/jupyter-children/memory.events')}\n", + " warn(msg)\n", + "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('8013'), PosixPath('http'), PosixPath('//172.28.0.1')}\n", + " warn(msg)\n", + "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('//colab.research.google.com/tun/m/cc48301118ce562b961b3c22d803539adc1e0c19/gpu-v100-hm-2ecbo83kb4j2y --tunnel_background_save_delay=10s --tunnel_periodic_background_save_frequency=30m0s --enable_output_coalescing=true --output_coalescing_required=true'), PosixPath('--logtostderr --listen_host=172.28.0.12 --target_host=172.28.0.12 --tunnel_background_save_url=https')}\n", + " warn(msg)\n", + "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/env/python')}\n", + " warn(msg)\n", + "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('//ipykernel.pylab.backend_inline'), PosixPath('module')}\n", + " warn(msg)\n", + "CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...\n", + "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0'), PosixPath('/usr/local/cuda/lib64/libcudart.so')}.. We'll flip a coin and try one of these, in order to fail forward.\n", + "Either way, this might cause trouble in the future:\n", + "If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.\n", + " warn(msg)\n", + "CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so.11.0\n", + "CUDA SETUP: Highest compute capability among GPUs detected: 7.0\n", + "CUDA SETUP: Detected CUDA version 118\n", + "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!\n", + " warn(msg)\n", + "CUDA SETUP: Loading binary /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118_nocublaslt.so...\n", + "2023-08-02 06:10:09.023871: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n", + "USE_MEM_EFF_ATTENTION: True\n", + "STORE_KV_BEFORE_ROPE: False\n", + "Apply NTK scaling with ALPHA=1.0\n", + "You are using the legacy behaviour of the . This means that tokens that come after special tokens will not be properly handled. We recommend you to read the related pull request available at https://github.com/huggingface/transformers/pull/24565\n", + "Loading checkpoint shards: 100% 2/2 [01:08<00:00, 34.32s/it]\n", + "Vocab of the base model: 55296\n", + "Vocab of the tokenizer: 55296\n", + "/content/Chinese-LLaMA-Alpaca-2/scripts/inference/gradio_demo.py:444: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.\n", + " user_input = gr.Textbox(\n", + "Running on local URL: http://0.0.0.0:19324\n", + "Running on public URL: https://70d599ba8bb44ae10a.gradio.live\n", + "\n", + "This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)\n", + "Keyboard interruption in main thread... closing server.\n", + "Traceback (most recent call last):\n", + " File \"/usr/local/lib/python3.10/dist-packages/gradio/blocks.py\", line 2133, in block_thread\n", + " time.sleep(0.1)\n", + "KeyboardInterrupt\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/content/Chinese-LLaMA-Alpaca-2/scripts/inference/gradio_demo.py\", line 538, in \n", + " demo.queue().launch(\n", + " File \"/usr/local/lib/python3.10/dist-packages/gradio/blocks.py\", line 2049, in launch\n", + " self.block_thread()\n", + " File \"/usr/local/lib/python3.10/dist-packages/gradio/blocks.py\", line 2137, in block_thread\n", + " self.server.close()\n", + " File \"/usr/local/lib/python3.10/dist-packages/gradio/networking.py\", line 49, in close\n", + " self.thread.join()\n", + " File \"/usr/lib/python3.10/threading.py\", line 1096, in join\n", + " self._wait_for_tstate_lock()\n", + " File \"/usr/lib/python3.10/threading.py\", line 1116, in _wait_for_tstate_lock\n", + " if lock.acquire(block, timeout):\n", + "KeyboardInterrupt\n", + "Killing tunnel 0.0.0.0:19324 <> https://70d599ba8bb44ae10a.gradio.live\n", + "^C\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "启动之后，日志中的\n", + "```\n", + "Running on public URL: https://70d599ba8bb44ae10a.gradio.live\n", + "```\n", + "就是可以在线体验的demo链接（每次都不一样）。用完记得及时关闭和删除运行时，以免不必要的浪费。\n", + "\n", + "\n" + ], + "metadata": { + "id": "S4BLss6S0AuD" + } + } + ] +} \ No newline at end of file