text generation webui_en

Inference with text-generation-webui

A gradio web UI for running Large Language Models like LLaMA, llama.cpp and so on.

Step 1: install text-generation-webui

text-generation-webui currently supports Windows/Linux/macOS/WSLsystems. Refer to webui installation for more details.

Step 2: Preparing model weights

Copy the full weights of chinese-alpaca-2 to the text-generation-webui models folder, as shown below

>>> ls models/chinese-alpaca-2-7b
config.json
generation_config.json
pytorch_model-00001-of-00002.bin
pytorch_model-00002-of-00002.bin
pytorch_model.bin.index.json
special_tokens_map.json
tokenizer_config.json
tokenizer.json
tokenizer.model

Step 3: Load the model and start the webui

Run the following command to talk to chinese-alpaca-2.

python server.py --model chinese-alpaca-2-7b --chat

Please refer to webui using docs for more detailed official instructions. If you encounter installation or running problems, please go to the original repo to ask questions, the process has been run under commit-id 32a2bbe