error when installing mountainsort 4: cl.exe failed with exit code 2

[elduvelle]

Thank you for Mountainsort - great software!

I am having a 'cl.exe failed with exit code 2' error while trying to install mountainsort4 via pip install mountainsort4 in a conda environment.

The relevant part of the error message that I get is:

error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\bin\HostX86\x64\cl.exe' failed with exit code 2
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for isosplit5
Running setup.py clean for isosplit5
Failed to build isosplit5
ERROR: Could not build wheels for isosplit5, which is required to install pyproject.toml-based projects

additional info:

• I previously had an error about pybind11 missing, but I then ran pip install git+https://github.com/magland/isosplit5_python.git and that error was fixed
• I am on Win10
• I have installed visual studio Build Tools (the path that is cited above definitely exists)
• I tried with 2 different versions of python ( 3.8.10 or 3.10.9) and got the same error.
• I read that 'exit code 2' meant that the path wasn't found so I have added it to my windows path environment variable without success (same error)

Thanks in advance for any suggestions!

[elduvelle]

Hi, I'm still getting this error even with other versions of python. It would really help if someone had any insights into how to fix it and install mountainsort4 (my other code works with that and not mountainsort5).

[magland]

Hi @elduvelle, sorry I haven't tested on windows. Have you tried with WSL?

[elduvelle]

Hi! Do you mean Windows Subsystem for Linux? I haven't tried that. I have an Ubuntu machine but every time I try to do things on Linux it takes 10x the time it takes me on windows, so I generally try to avoid it. I guess I could test the installation on it to see if I get the same error.

My code (using spike interface and mountainsort 4, see here for the repo) is working fine on another of my Windows 10 computers, so it seems that there is some subtle but hopefully simple thing to do to make it work... If you have any ideas of what to test I can do the tests!
Thanks!

[elduvelle]

I don't know if it's helpful but I tried to install mountainsort 5 and that worked (the installation didn't have the same error). Of course it did have some problems when I tried to run my code with ms5 but maybe it could help narrow down the problem?
Also, someone suggested to me that the error could be because ms4 needs a specific version of visual studio installed; if so would you know which version?

[elduvelle]

Some additional info: I tried with a different version of python (3.8.10, following this post) and this time the error was when I tried to install isosplit 5, which if I am not mistaken is also yours @magland (thank you for contributing all this great code!)?
It turns out someone had the same error as me (here with the cl.exe file). I tried the proposed solution but it didn't work for me; I got the cl.exe error again:

C:\Users\...\Documents\GitHub\isosplit5_python\src\isocut5.h(19): fatal error C1083: Cannot open include file: 'stdlib.h': No such file or directory
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2

Edit: actually I had tried that python version before, sorry.

[magland]

@elduvelle thanks for that info.

Mountainsort and isosplit were developed and tested in Linux, and I don't have easy access to a windows machine for troubleshooting. I strongly suggest you give WSL (windows subsystem for linux) a try. I think that will be your best bet.

[elduvelle]

@magland That's too bad.. but thanks nonetheless. I will try that for lack of better solution. Hopefully someone else will have some insights on how to make it work on Windows 10!

[elduvelle]

For whoever is reading this, here's an update: the install works on Ubuntu.
In more details:
I tried my install process on linux, ubuntu version 18.04.6 LTS.
I used a fresh install of Anaconda, with conda version 23.7.2 and python version 3.11.4. The first attempt to run pip install mountainsort4 crashed with one of the errors I encountered previously on Win10 related to ModuleNotFoundError: No module named 'pybind11'.
Then I ran pip install git+https://github.com/magland/isosplit5_python.git and then pip install mountainsort4 again and it worked! (while that step didn't work on my more recent Win10).

However, there are now other problems with what I wanted to do with mountainsort, but they are unrelated to mountainsort or this question.

I'd still appreciate any pointers regarding how to solve the 'cl.exe' problem on win 10!

[elduvelle]

Hi @magland I'm really sorry to bother you again with this but is it possible to know which version of Visual Studio tools / suite was used when isosplit v. 0.1.3 was released? I think the problems I'm having are because Visual studio tools have been updated and expect a specific syntax of files that they try to compile which is why it fails.

[magland]

Hi @magland I'm really sorry to bother you again with this but is it possible to know which version of Visual Studio tools / suite was used when isosplit v. 0.1.3 was released? I think the problems I'm having are because Visual studio tools have been updated and expect a specific syntax of files that they try to compile which is why it fails.

I never used Visual Studio. I believe that's only for windows. I have only tested this on Linux.

[SusanL82]

I think I am running into the same problem, but with a windows11 PC.
Weirdly, I am able to install isosplit5 separately, and (part of) mountainsort4 via pip install ml_ms4alg.

This does not get recognized by spikeinterface (which is my final goal). But when I then try to install mountainsort4 for any missing components, it tried to re-install isosplit5 and failed at that.
I've basically installed all available interpreters that come with Visual Studio, and I'm not sure what else to try. I don't really use visual studio directly, but it's an installation requirement for mountainsort4. Switching to linux is not really an option for me. I eventually want to use mountainsort4 on a lab PC, and IT simply will not allow me to choose my own operating system.

Anyway, my error message looks like this:

(SIlatest) C:\Users\susan>pip install mountainsort4
Collecting mountainsort4
  Using cached mountainsort4-1.0.4-py3-none-any.whl
Requirement already satisfied: dask in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from mountainsort4) (2023.5.0)
Requirement already satisfied: pybind11 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from mountainsort4) (2.11.1)
Collecting isosplit5==0.1.3 (from mountainsort4)
  Using cached isosplit5-0.1.3.tar.gz (17 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from mountainsort4) (1.24.4)
Requirement already satisfied: h5py in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from mountainsort4) (3.9.0)
Requirement already satisfied: scikit-learn in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from mountainsort4) (1.3.0)
Requirement already satisfied: spikeextractors>=0.9.5 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from mountainsort4) (0.9.11)
Requirement already satisfied: tqdm in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from spikeextractors>=0.9.5->mountainsort4) (4.66.1)
Requirement already satisfied: joblib in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from spikeextractors>=0.9.5->mountainsort4) (1.3.2)
Requirement already satisfied: click>=8.0 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from dask->mountainsort4) (8.0.4)
Requirement already satisfied: cloudpickle>=1.5.0 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from dask->mountainsort4) (2.2.1)
Requirement already satisfied: fsspec>=2021.09.0 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from dask->mountainsort4) (2023.6.0)
Requirement already satisfied: packaging>=20.0 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from dask->mountainsort4) (23.0)
Requirement already satisfied: partd>=1.2.0 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from dask->mountainsort4) (1.4.0)
Requirement already satisfied: pyyaml>=5.3.1 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from dask->mountainsort4) (6.0)
Requirement already satisfied: toolz>=0.10.0 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from dask->mountainsort4) (0.12.0)
Requirement already satisfied: importlib-metadata>=4.13.0 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from dask->mountainsort4) (6.0.0)
Requirement already satisfied: scipy>=1.5.0 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from scikit-learn->mountainsort4) (1.10.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from scikit-learn->mountainsort4) (3.2.0)
Requirement already satisfied: colorama in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from click>=8.0->dask->mountainsort4) (0.4.6)
Requirement already satisfied: zipp>=0.5 in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from importlib-metadata>=4.13.0->dask->mountainsort4) (3.11.0)
Requirement already satisfied: locket in c:\users\susan\anaconda3\envs\silatest\lib\site-packages (from partd>=1.2.0->dask->mountainsort4) (1.0.0)
Building wheels for collected packages: isosplit5
  Building wheel for isosplit5 (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [34 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-38
      creating build\lib.win-amd64-cpython-38\isosplit5
      copying isosplit5\__init__.py -> build\lib.win-amd64-cpython-38\isosplit5
      running build_ext
      building 'isosplit5_interface' extension
      creating build\temp.win-amd64-cpython-38
      creating build\temp.win-amd64-cpython-38\Release
      creating build\temp.win-amd64-cpython-38\Release\src
      "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\susan\anaconda3\envs\SIlatest\lib\site-packages\pybind11\include -IC:\Users\susan\anaconda3\envs\SIlatest\lib\site-packages\pybind11\include -IC:\Users\susan\anaconda3\envs\SIlatest\include -IC:\Users\susan\anaconda3\envs\SIlatest\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /EHsc /Tpsrc/isocut5.cpp /Fobuild\temp.win-amd64-cpython-38\Release\src/isocut5.obj /EHsc /DVERSION_INFO=\\\"0.1.3\\\"
      isocut5.cpp
      C:\Users\susan\AppData\Local\Temp\pip-install-c072mh1l\isosplit5_752d31fc5ece42db9734fc3c5fdeea45\src\isocut5.h(21): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
      C:\Users\susan\AppData\Local\Temp\pip-install-c072mh1l\isosplit5_752d31fc5ece42db9734fc3c5fdeea45\src\isocut5.h(21): error C2146: syntax error: missing ';' before identifier 'bigint'
      C:\Users\susan\AppData\Local\Temp\pip-install-c072mh1l\isosplit5_752d31fc5ece42db9734fc3c5fdeea45\src\isocut5.h(27): error C2061: syntax error: identifier 'bigint'
      src/isocut5.cpp(39): warning C4244: '=': conversion from 'double' to 'float', possible loss of data
      src/isocut5.cpp(40): error C2660: 'isocut5': function does not take 5 arguments
      C:\Users\susan\AppData\Local\Temp\pip-install-c072mh1l\isosplit5_752d31fc5ece42db9734fc3c5fdeea45\src\isocut5.h(27): note: see declaration of 'isocut5'
      src/isocut5.cpp(40): note: while trying to match the argument list '(double *, double *, int, float *, isocut5_opts)'
      src/isocut5.cpp(55): warning C4244: 'initializing': conversion from 'double' to 'bigint', possible loss of data
      src/isocut5.cpp(57): warning C4244: 'initializing': conversion from 'double' to 'bigint', possible loss of data
      src/isocut5.cpp(62): warning C4244: '=': conversion from 'int' to 'float', possible loss of data
      src/isocut5.cpp(64): warning C4244: '=': conversion from 'int' to 'float', possible loss of data
      src/isocut5.cpp(65): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data
      src/isocut5.cpp(81): warning C4244: '=': conversion from 'bigint' to 'float', possible loss of data
      src/isocut5.cpp(136): warning C4244: 'initializing': conversion from 'double' to 'bigint', possible loss of data
      src/isocut5.cpp(138): warning C4244: 'initializing': conversion from 'double' to 'bigint', possible loss of data
      src/isocut5.cpp(143): warning C4244: '=': conversion from 'int' to 'float', possible loss of data
      src/isocut5.cpp(145): warning C4244: '=': conversion from 'int' to 'float', possible loss of data
      src/isocut5.cpp(146): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data
      src/isocut5.cpp(162): warning C4244: '=': conversion from 'bigint' to 'float', possible loss of data
      error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.37.32822\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for isosplit5
  Running setup.py clean for isosplit5
Failed to build isosplit5
ERROR: Could not build wheels for isosplit5, which is required to install pyproject.toml-based projects 

[elduvelle]

Hi :) I have made some more tests on my side of things:

• the mountainsort4 install still doesn't work on Win10 after trying different versions of the visual studio codes (it would help if I knew what exactly is needed from visual studio)
• the install works on Ubuntu 18.04 and runs fine, albeit with some warnings
• the install didn't originally work on Windows Subsystem for Linux, but it eventually did. Here are the steps I followed if anyone is interested:

1. Install WSL following the instructions here
   In a console run as administrator:

wsl --install

2. Create the account & password
3. Install miniconda as explained here:

3.1. Make sure you move to a folder that you have writing access to, like /home/your_user_name/ by doing cd path_to_that_folder

3.2 Download the miniconda installer

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

3.3. Run the installation, follow the instructions

bash Miniconda3-latest-Linux-x86_64.sh

3.3. Close and reopen

4. Install 'C++ tools':

sudo apt install build-essential

5. Install mountainsort

pip install mountainsort4

and it should work! That's for WSL. I still would like to make it work again in Win10 without the additional WSL steps. Interestingly, it seemed that all the operations done in WSL were faster than the same ones run in Windows...

Note: my pipeline involves installing a few things before mountainsort and maybe these help. Feel free to check it out. It's using spikeinterface to use Phy to process the output of mountainsort.

[magland]

Thanks @elduvelle for the update! Glad you got it working in WSL. Thanks also for posting a link to your pipeline... that could very well be useful for folks.