Python Documentation / Conda Versions.

[gattia]

I went to conda install opensim 4.4 (which is amazing!) and noticed 2 things that I have some feedback/questions about.

1. The description here mentioned recommending installing in the base environment

Conda has a base environment but allows creation of environments with different python versions, it is less likely to have problems if you install the opensim package in the base environment so you have access to it in other created environments

In my experience this is actually the opposite of what most people would want to do and what is generally recommended. My understanding/experience is that having many packages installed in base conda can actually slow down conda itself quite considerably - particularly when its trying to solve to install new packages. Mamba helps speed this up - but I think the problem still exists. I've actually wiped my entire conda folder and re-started for exactly this reason. So, it is normally strongly suggested that all new programs/pacakges/projects create a conda environment specific for that project and work from that. I think that conda still cache's all downloaded versions (e.g., python 3.9, opensim 4.4) for use in other environments (even if you install it in a virtual environment and not base).

2. I notice Mac python 3.8 version of Opensim is missing in the list of available versions. This isn't a big deal, but it sort of failed silently for me, which was a bit strange and might catch some people up. E.g., I conda installed opensim in a 3.8 environment, which seemed to go through successfully and then when I tried to import opensim it just gave a package not found error. Looking back it looked like it downloaded a python 3.9 version, but just didn't even try to use it for whatever reason. I then created a python 3.9 environment, conda installed and everything has been working as expected. Not a big issue, but worth noting.

[aymanhab]

Great feedback @gattia 👍

1. The reason I suggested using base environment is to minimize the number of unknowns, ideally after that you can/should remove from base environment and install as needed in new environments. During testing we found interactions between environments due to changes in python/conda behavior between versions but I definitely would follow the conda user conventions/guidelines. Feel free to update or propose new text on confluence.
2. For Mac, I have one old osx machine that I created a build on but ci builds are failing I'll look into that when I get a chance. If you know somebody who can help/try and create a build with python 3.9, 3.10 that would be awesome.

[gattia]

1. Sound good, I can make an update to that text.
2. I have an older Mac laying around as well - if you want to point me to the build instructions I can run them for the missing ones.

[aymanhab]

Thanks @gattia you can checkout the repository https://github.com/opensim-org/conda-opensim and follow the instructions on the readme. It's supposed to work out of the box for python 3.7, 3.8, 3.9 but may take a while. Let me know if you run into issues and I'll try to help troubleshoot. The resulting bz2 files can be uploaded to our anaconda.org account then. Thank you.

[aymanhab]

@gattia I'm following up on this since Scott Ulrich was having issues using the conda package on osx 12.4 with M1 processor, can you comment on your environment? I also just uploaded conda packages built with python 3.9 to the anconda.org user opensim_admin in case you can try them out. Thank you

[gattia]

@aymanhab Sorry - I dropped the ball on this and haven't had a chance to update the docs. However, I do think I've figured out the issues here.

When you install opensim with conda on mac using: conda install -c opensim-org opensim it is defaulting to install the exact same python build from conda - on my machine its installing the version with build string: py39np120 (it doesn't check what version of python or numpy I actually have installed).

So, if you install this when you are in a python 3.10 environment then it seems to install it, but it fails to actually load in python (import opensim returns: ModuleNotFoundError). This is presumably a result of using the wrong interpreter. You can see the version it is trying to install when you run conda install.

To fix this, you can specify the "build_string" eg:
Python 3.10:
conda install -c opensim-org opensim=4.4=py310np121 numpy=1.21
Python 3.8:
conda install -c opensim-org opensim=4.4=py38np120 numpy=1.20

I've specified the python version as well as numpy here because it doesn't seem like conda is picking up the numpy dependency either.

This stack overflow explains the above syntax where Im specifying a particular build string.

I think the root problem here is that when building the conda package the meta.yml file builds the string manually and doesn't actually specify the version of python/numpy needed. It does, for the build environment, but not for the running environment - therefore, conda thinks it can use any of the packages (because they have no specific python/numpy dependencies). You can see this by clicking the (i) icon next to the file for opensim here where python/numpy are listed but the version isn't. This is in contrast to other libraries where the python version is specified in the dependencies (e.g. SimpleITK here)

I haven't made conda recipes before, so I might be missing something else, but this seems to be what's going on to me. I've made a pull request on the recipes repo to make this initial fix: opensim-org/conda-opensim#24

This might also be related to the preference for using the base environment with conda that I mentioned in the initial post. I'll hold off until this is sorted to make a pull request on the docs above about recommendations on where to build.

For testing purposes, below is code to create environment and test build for python 3.8 & 3.10. Python 3.9 should work out of the box, but that's just because its downloading that version by default.

conda create -n opensim_38 python=3.8 -y                                                      # create env 
conda activate opensim_38                                                                                 # activate 
conda install -c opensim-org opensim=4.4=py38np120 numpy=1.20    -y     # install appropriate verson
python -c "import opensim"                  # inline import of opensim, should raise error if not installed properly. 

conda create -n opensim_310 python=3.10 -y                                                    # create env 
conda activate opensim_310                                                                                 # activate 
conda install -c opensim-org opensim=4.4=py310np121 numpy=1.21    -y      # install appropriate verson
python -c "import opensim"                  # inline import of opensim, should raise error if not installed properly. 

[gattia]

Ahh, I for some reason just loaded back up the docs and noticed that this was already suggested somewhere - someone beat me to it!

I think the root cause is still identified over on my pull request on the recipe. However, it failed because the CONDA_PY and CONDA_NPY weren't quite formatted correctly - they dont have the .. If the dependencies are built then people should be able to install in any conda environment and it will either successfully install, or conda will tell them that there is no combination of packages that can satisfy the requirements.

opensim-org/conda-opensim#24

[jenhicks]

Please re-open if this one isn't actually resolved.