ImportError: cannot import name '_module_available' from 'pytorch_lightning.utilities'

[AudioEngineer99]

Hello,

After having installed all required packages listed in 'requirements.txt', I am still getting this error, when running the command:

"python ear_to_prtf.py configs/edges_median.json /path/to/pinna_contours.png --nfft 512 --output_path /path/to/prtf.mat"

I tried upgrading from 'pytorch-lightning-bolts' to the more recent 'lightning-bolts' as well as installing different versions of 'pytorch-lightning-bolts'/'pytorch-lightning' without any success unfortunately.

I have the following package versions:
torch 1.13.0
torchvision 0.14.0
torchaudio 0.13.0
pytorch-lightning 1.8.4.post0
pytorch-lightning-bolts 0.3.2.post1
matplotlib 3.5.3
librosa 0.9.2
pandas 1.3.5
tqdm 4.64.1
numpy 1.21.6
scipy 1.7.3
python-sofa 0.2.0
Pillow 9.3.0
lightning-utilities 0.4.2

Maybe you would be so kind to share your list of package versions for which the model is working, as I guess it has something to do with changes to 'pytorch-lightning-bolts'/'pytorch-lightning'.

[miccio-dk]

Hi, thanks for writing this. Unfortunately, I no longer have access to the original python environment, but I can try and help you debug this. Can you post the full stack trace?

[AudioEngineer99]

Thanks for your quick response. Here ist the full stack trace:

Traceback (most recent call last):
  File "ear_to_prtf.py", line 12, in <module>
    from models.vae_resnet_cfg import ResNetVAECfg
  File "XXXXX\itsadive_hybrid_structural_model-main\python\models\vae_resnet_cfg.py", line 3, in <module>
    from pl_bolts.models.autoencoders import VAE
  File "XXXXX\Python\Python37\site-packages\pl_bolts\__init__.py", line 19, in <module>
    from pl_bolts import (  # noqa: E402
  File "XXXXX\Python\Python37\site-packages\pl_bolts\callbacks\__init__.py", line 5, in <module>
    from pl_bolts.callbacks.data_monitor import ModuleDataMonitor, TrainingDataMonitor  # noqa: F401
  File "XXXXX\Python\Python37\site-packages\pl_bolts\callbacks\data_monitor.py", line 14, in <module>
    from pl_bolts.utils import _WANDB_AVAILABLE
  File "XXXXX\Python\Python37\site-packages\pl_bolts\utils\__init__.py", line 2, in <module>
    from pytorch_lightning.utilities import _module_available
ImportError: cannot import name '_module_available' from 'pytorch_lightning.utilities' (XXXXX\Python\Python37\site-packages\pytorch_lightning\utilities\__init__.py)

I had to censor a few paths, but it shouldn't matter for debugging.

[miccio-dk]

alright, try and create a new environment and install these versions for the relevant python packages:

pytorch==1.8
pytorch-lightning==1.0.4
pytorch-lightning-bolts==0.2.5 
torchvision==0.8

Keep me posted, so that I can update the requirements file if this works :)

[AudioEngineer99]

I now have thw following package versions:

torch                         1.8.0
pytorch-lightning             1.0.4
pytorch-lightning-bolts       0.2.5
torchvision                   0.9.0

I was not able to install torchvision 0.8.0 via pip and installed 0.9.0 instead.

Now, after running:

python ear_to_prtf.py configs/edges_median.json /path/to/pinna_contours.png --nfft 512 --output_path /path/to/prtf.mat

I now get the following:

### Loading model VAE_incept from ./checkpoints/vae_incept_edges.ckpt...
### Loading model DNN from ./checkpoints/dnn_edge_median.ckpt...
### Loading model CVAE_dense from ./checkpoints/dnn_edge_median.ckpt...
Traceback (most recent call last):
  File "ear_to_prtf.py", line 168, in <module>
    main()
  File "ear_to_prtf.py", line 72, in main
    model = ModelClass.load_from_checkpoint(model_ckpt_path)
  File "XXXXX\Python\Python37\site-packages\pytorch_lightning\core\saving.py", line 154, in load_from_checkpoint
    model = cls._load_model_state(checkpoint, strict=strict, **kwargs)
  File "XXXXX\Python\Python37\site-packages\pytorch_lightning\core\saving.py", line 194, in _load_model_state
    model = cls(**_cls_kwargs)
TypeError: __init__() missing 1 required positional argument: 'nfft'

[babar414]

Hi, Any update on the above issue regarding the
TypeError: init() missing 1 required positional argument: 'nfft'

[miccio-dk]

Hi everyone, sorry for the late reply. I have been trying to replicate the error but again, unfortunately, I no longer have access to my original setup. Anyways, based on this, it seems that it would be possible to pass nfft=args.nfft to load_model_from_checkpoint, like so:

model = ModelClass.load_from_checkpoint(model_ckpt_path, nfft=args.nfft)

Let me know if that works, and feel free to continue the discussion here! I'd be happy to provide further support.
Update: I pushed a few fixes and it should now work with nfft = 512.

[babar414]

Hi, thank you for your help. I did try to follow the above. But it seems we still need to provide cfg. As the below error occurs:

Traceback (most recent call last):
File "ear_to_prtf.py", line 172, in
main()
File "ear_to_prtf.py", line 73, in main
model = ModelClass.load_from_checkpoint(model_ckpt_path, nfft=args.nfft)
File "/home/tcl/code/python/hrtf-customize/itsadive_hybrid_structural_model/python/env/lib/python3.8/site-packages/pytorch_lightning/core/saving.py", line 154, in load_from_checkpoint
model = cls._load_model_state(checkpoint, strict=strict, kwargs)
File "/home/tcl/code/python/hrtf-customize/itsadive_hybrid_structural_model/python/env/lib/python3.8/site-packages/pytorch_lightning/core/saving.py", line 194, in _load_model_state
model = cls(_cls_kwargs)
File "/home/tcl/code/python/hrtf-customize/itsadive_hybrid_structural_model/python/models/cvae_dense_cfg.py", line 87, in init
self.c_labels = cfg['labels']
KeyError: 'labels'

[AudioEngineer99]

Hi,

So, after downloading and using the latest version of the code (it has been updated by miccio-dk), I am now finally able to create the 'prtf.mat'.

I have one hint: In order to use the command

python ear_to_prtf.py configs/edges_median.json /path/to/pinna_contours.png --nfft 512 --output_path /path/to/prtf.mat

line 39 in 'ear_to_prtf.py', has to be changed to:

parser.add_argument('--output_path', type=str)

@miccio-dk Thanks for your help!

[miccio-dk]

My pleasure, and thanks for the suggestion! i'll apply the change (along with specifying the correct versions of the pkgs in the requirements) and then close this issue :)

[babar414]

@miccio-dk @AudioEngineer99 Thank you very much. I can output prtf.mat file now.
(It would be good to update the requirements file also along with mentioning the python version that was used to run the code)