当前中文llama2中文模型似乎在最新的privateGPT中无法使用

[olimish]

这是我在当前最新版的privateGPT中的模型配置
local:
llm_hf_repo_id: hfl/chinese-alpaca-2-13b
llm_hf_model_file: pytorch_model-00001-of-00003.bin
embedding_hf_model_name: BAAI/bge-reranker-base

安装依赖也是用了他提供的
poetry run python scripts/setup

之后在启动的时候直接报错了
$env:PGPT_PROFILES="local"; make run
poetry run python -m private_gpt
Starting application with profiles: ['default', 'local']
ggml_init_cublas: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 3060, compute capability 8.6
gguf_init_from_file: invalid magic number 04034b50
error loading model: llama_model_loader: failed to load model from C:\Users\Administrator\Documents\projects\privateGPT\models\pytorch_model-00001-of-00003.bin

llama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 798, in get
return self._context[key]
~~~~~~~~~~~~~^^^^^
KeyError: <class 'private_gpt.server.ingest.ingest_service.IngestService'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 798, in get
return self._context[key]
~~~~~~~~~~~~~^^^^^
KeyError: <class 'private_gpt.components.llm.llm_component.LLMComponent'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Users\Administrator\Documents\projects\privateGPT\private_gpt_main.py", line 5, in
from private_gpt.main import app
File "C:\Users\Administrator\Documents\projects\privateGPT\private_gpt\main.py", line 123, in
from private_gpt.ui.ui import mount_in_app
File "C:\Users\Administrator\Documents\projects\privateGPT\private_gpt\ui\ui.py", line 19, in
ingest_service = root_injector.get(IngestService)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W-py3.11\Lib\site-packages\injector_init_.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 974, in get
provider_instance = scope_instance.get(interface, binding.provider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 800, in get
instance = self.get_instance(key, provider, self.injector)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W-py3.11\Lib\site-packages\injector_init_.py", line 811, in get_instance
return provider.get(injector)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W-py3.11\Lib\site-packages\injector_init_.py", line 264, in get
return injector.create_object(self.cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W-py3.11\Lib\site-packages\injector_init_.py", line 998, in create_object
self.call_with_injection(init, self_=instance, kwargs=additional_kwargs)
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 1031, in call_with_injection
dependencies = self.args_to_inject(
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 1079, in args_to_inject
instance: Any = self.get(interface)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 974, in get
provider_instance = scope_instance.get(interface, binding.provider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 91, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 800, in get
instance = self.get_instance(key, provider, self.injector)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W-py3.11\Lib\site-packages\injector_init_.py", line 811, in get_instance
return provider.get(injector)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W-py3.11\Lib\site-packages\injector_init_.py", line 264, in get
return injector.create_object(self.cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W-py3.11\Lib\site-packages\injector_init_.py", line 998, in create_object
self.call_with_injection(init, self_=instance, kwargs=additional_kwargs)
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\injector_init_.py", line 1040, in call_with_injection
return callable(*full_args, **dependencies)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\Documents\projects\privateGPT\private_gpt\components\llm\llm_component.py", line 20, in init
self.llm = LlamaCPP(
^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W_-py3.11\Lib\site-packages\llama_index\llms\llama_cpp.py", line 110, in init
self.model = Llama(model_path=model_path, **model_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\AppData\Local\pypoetry\Cache\virtualenvs\private-gpt-lmSmH4W-py3.11\Lib\site-packages\llama_cpp\llama.py", line 365, in init
assert self.model is not None
^^^^^^^^^^^^^^^^^^^^^^
AssertionError
Makefile:36: recipe for target 'run' failed
make: *** [run] Error 1

能否看看哪里不兼容了，还是我配置错了

[ymcui]

这和是不是中文llama-2关系不大。建议你去privateGPT那边看看原版llama-2是否有类似的问题。

[olimish]

我换一个问题，如果huggingface中的模型文件分拆了
例如：
pytorch_model-00001-of-00003.bin
pytorch_model-00002-of-00003.bin
pytorch_model-00003-of-00003.bin

那配置文件中的llm_hf_model_file应该如何配置
多谢了