Making the code VENV friendly

[3dluvr]

Thank you for sharing the code from your research with the public.

Please consider a few suggestions that I think would improve usability of your code, and make it more friendly:

1. Make the code VENV friendly so that all additional files are stored inside the contained virtual environment (your project root folder);
2. Avoid "black box" downloads - personally I really do not appreciate seeing a 10+ GB download of random files coming from someplace on the internet, and being stored somewhere on my drive;
3. Include all of the necessary dependencies in the requirements.txt file rather than only adding basic/partial ones and surprising the users with a list of additional downloads;
4. Better document all of the above in your README;
5. ...

Cheers!

[Yangr116]

Thanks for suggestions! We will continue to improve GPT4Tools.

[3dluvr]

Great !

At the moment, I'm having problems actually getting the code to work.

For whatever reason when I specify --base_model and --lora_model arguments, I get back that the arguments are unrecognized:

`Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
/home/user/Envs/text-generation-webui_env/lib/python3.10/site-packages/bitsandbytes-0.37.1-py3.10.egg/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/home/user/Envs/text-generation-webui_env/lib/python3.10/site-packages/cv2/../../lib64')}
warn(msg)

CUDA SETUP: CUDA runtime path found: /usr/local/cuda-11.8/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 118
CUDA SETUP: Loading binary /home/user/Envs/text-generation-webui_env/lib/python3.10/site-packages/bitsandbytes-0.37.1-py3.10.egg/bitsandbytes/libbitsandbytes_cuda118.so...

usage: gpt4tools.py [-h] [--notebook] [--chat] [--cai-chat] [--model MODEL] [--lora LORA] [--model-dir MODEL_DIR] [--lora-dir LORA_DIR] [--model-menu] [--no-stream] [--settings SETTINGS]
[--extensions EXTENSIONS [EXTENSIONS ...]] [--verbose] [--cpu] [--auto-devices] [--gpu-memory GPU_MEMORY [GPU_MEMORY ...]] [--cpu-memory CPU_MEMORY] [--disk] [--disk-cache-dir DISK_CACHE_DIR]
[--load-in-8bit] [--bf16] [--no-cache] [--xformers] [--sdp-attention] [--trust-remote-code] [--threads THREADS] [--wbits WBITS] [--model_type MODEL_TYPE] [--groupsize GROUPSIZE]
[--pre_layer PRE_LAYER] [--monkey-patch] [--quant_attn] [--warmup_autotune] [--fused_mlp] [--flexgen] [--percent PERCENT [PERCENT ...]] [--compress-weight] [--pin-weight [PIN_WEIGHT]] [--deepspeed]
[--nvme-offload-dir NVME_OFFLOAD_DIR] [--local_rank LOCAL_RANK] [--rwkv-strategy RWKV_STRATEGY] [--rwkv-cuda-on] [--listen] [--listen-host LISTEN_HOST] [--listen-port LISTEN_PORT] [--share]
[--auto-launch] [--gradio-auth-path GRADIO_AUTH_PATH] [--api] [--public-api]
gpt4tools.py: error: unrecognized arguments: --base_model data/models/vicuna-13b-1.1-gptq-4bit-128g --lora_model data/loras/ --llm_device cpu Text2Box_cuda:0,Segmenting_cuda:0,Inpainting_cuda:0,ImageCaptioning_cuda:0`

[Yangr116]

Try this one?

python gpt4tools.py --base_model data/models/vicuna-13b-1.1-gptq-4bit-128g --lora_model data/loras/ --llm_device "cpu" --load "Text2Box_cuda:0,Segmenting_cuda:0,Inpainting_cuda:0,ImageCaptioning_cuda:0"

[3dluvr]

That is basically the same command, which also returns arguments not recognized.

There is another glaringly obvious issue here as well that you do not support safetensors which I was trying to use, but before I can even tackle that I would like to get to a point where at least I get an error about it.

I will deploy a clean VENV and try again on my end to exclude a possibility of corruption.