Issue with NeMo Model Installation on Python 3.10.12: ImportError from huggingface_hub

[shripadalkar]

I encountered an issue when trying to install and import the NeMo ASR model on Python 3.10.12. After following the installation steps, I received the following error during the import of the nemo.collections.asr module:
Steps to Reproduce:

steps 1
git clone https://github.com/AI4Bharat/NeMo.git && cd NeMo && git checkout nemo-v2 && bash reinstall.sh
step 2
import nemo.collections.asr as nemo_asr

Error:

Traceback (most recent call last):
File "", line 1, in
File "/home/shrikant/Project/python/ASR/speech_to_text_setup/dec262024/NeMo/nemo/collections/asr/init.py", line 15, in
from nemo.collections.asr import data, losses, models, modules
File "/home/shrikant/Project/python/ASR/speech_to_text_setup/dec262024/NeMo/nemo/collections/asr/losses/init.py", line 15, in
from nemo.collections.asr.losses.angularloss import AngularSoftmaxLoss
File "/home/shrikant/Project/python/ASR/speech_to_text_setup/dec262024/NeMo/nemo/collections/asr/losses/angularloss.py", line 18, in
from nemo.core.classes import Loss, Typing, typecheck
File "/home/shrikant/Project/python/ASR/speech_to_text_setup/dec262024/NeMo/nemo/core/init.py", line 16, in
from nemo.core.classes import *
File "/home/shrikant/Project/python/ASR/speech_to_text_setup/dec262024/NeMo/nemo/core/classes/init.py", line 20, in
from nemo.core.classes.common import (
File "/home/shrikant/Project/python/ASR/speech_to_text_setup/dec262024/NeMo/nemo/core/classes/common.py", line 40, in
from nemo.core.classes.mixins.hf_io_mixin import HuggingFaceFileIO
File "/home/shrikant/Project/python/ASR/speech_to_text_setup/dec262024/NeMo/nemo/core/classes/mixins/init.py", line 28, in
from nemo.core.classes.mixins.hf_io_mixin import HuggingFaceFileIO
File "/home/shrikant/Project/python/ASR/speech_to_text_setup/dec262024/NeMo/nemo/core/classes/mixins/hf_io_mixin.py", line 19, in
from huggingface_hub import HfApi, ModelCard, ModelCardData, ModelFilter
ImportError: cannot import name 'ModelFilter' from 'huggingface_hub' (/home/shrikant/Project/python/ASR/speech_to_text_setup/dec262024/asr/lib/python3.10/site-packages/huggingface_hub/init.py)

[wannasleepforlong]

Encountered the exact same error. Weirdly enough, it was working earlier in different device

[AK3847]

The issue is that the NeMo library has not been maintained for the past 9 months while there have been major changes in the hugginface-hub repo, so you have to downgrade the package version which can lead to version conflict with other packages (transformers, datasets), I tried my best to find set of versions which would work, you can check out here: https://colab.research.google.com/drive/1umnew1cip7XnQkVJ3Edt1QtZB1ZRaaVK?authuser=1#scrollTo=0t9XnTTkjq5l&line=1&uniqifier=1

fix:

!pip install transformers==4.40.0
!pip install huggingface_hub==0.21.3
!pip install tokenizers>=0.21
!pip install datasets==2.19.0

[shripadalkar]

@AK3847 Thanks.

[bhargaviajaypatel]

The issue is that the NeMo library has not been maintained for the past 9 months while there have been major changes in the hugginface-hub repo, so you have to downgrade the package version which can lead to version conflict with other packages (transformers, datasets), I tried my best to find set of versions which would work, you can check out here: https://colab.research.google.com/drive/1umnew1cip7XnQkVJ3Edt1QtZB1ZRaaVK?authuser=1#scrollTo=0t9XnTTkjq5l&line=1&uniqifier=1

fix:

!pip install transformers==4.40.0
!pip install huggingface_hub==0.21.3
!pip install tokenizers>=0.21
!pip install datasets==2.19.0

it is not working for me... Pls help

[wannasleepforlong]

@bhargaviajaypatel Try running this mate
#!/bin/bash

python: 3.10.12

python3 -m pip install "pydub"
python3 -m pip install "cython==0.29.37"
python3 -m pip install "pyyaml==6.0.2"
python3 -m pip install "argparse==1.4.0"
python3 -m pip install "onnxruntime==1.19.0"
python3 -m pip install "tqdm==4.66.5"
python3 -m pip install "nemo_toolkit @ git+https://github.com/AI4Bharat/NeMo@nemo-v2"
python3 -m pip install "transformers==4.40.0"
python3 -m pip install "huggingface_hub==0.23"
python3 -m pip install "pytorch-lightning==2.4.0"
python3 -m pip install "hydra-core==1.3.2"
python3 -m pip install "librosa==0.10.2.post1"
python3 -m pip install "sentencepiece==0.2.0"
python3 -m pip install "pandas==2.2.2"
python3 -m pip install "lhotse==1.27.0"
python3 -m pip install "editdistance==0.8.1"
python3 -m pip install "jiwer==3.0.4"
python3 -m pip install "pyannote.audio==3.3.1"
python3 -m pip install "webdataset==0.2.100"
python3 -m pip install "datasets==2.21.0"
python3 -m pip install "IPython==8.27.0"
python3 -m pip install "joblib==1.4.2"
python3 -m pip install "numpy==1.26.4 "
python3 -m pip install "argparse==1.4.0"
python3 -m pip install "moviepy==1.0.3"

[AK3847]

@bhargaviajaypatel can you share the versions of installed libraries in your environment? It's most likely a version conflict which can be solved by downgrading python version or some other package.