You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have few gptq and gguf models, when i try to run gptq i have this:
2024-08-06 00:47:26 ERROR:Failed to load the model.
Traceback (most recent call last):
File "/nix/store/1zzqmn5cl5c8dcbv37xp8xvvii892015-textgen-patchedSrc/modules/ui_model_menu.py", line 201, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/1zzqmn5cl5c8dcbv37xp8xvvii892015-textgen-patchedSrc/modules/models.py", line 79, in load_model
output = load_func_map[loader](model_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/1zzqmn5cl5c8dcbv37xp8xvvii892015-textgen-patchedSrc/modules/models.py", line 318, in AutoGPTQ_loader
import modules.AutoGPTQ_loader
File "/nix/store/1zzqmn5cl5c8dcbv37xp8xvvii892015-textgen-patchedSrc/modules/AutoGPTQ_loader.py", line 3, in <module>
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
ModuleNotFoundError: No module named 'auto_gptq'
Similar with other model loaders, just changes the name of module that not found
I have few gptq and gguf models, when i try to run gptq i have this:
Similar with other model loaders, just changes the name of module that not found
If i try to run gguf model i have
I'm absolutely unsure what i can try to fix this, for me looks like i somehow broke very reproducible installation process and now i'm here
P.S. If anyone can at least recommend me some other ways to run local models, I will be very grateful.
The text was updated successfully, but these errors were encountered: