Building locally for ppc64le

[jaimergp]

Comes from openmm/openmm#2722 (comment)

@Socramin, I'll comment on this issue at some point during the week! Thanks for the patience!

[jaimergp]

We have opened a PR to address ppc builds in #23

Right now it's not working due to the compiler differences, but Peter Eastman found the bug and will be fixed soon.

If you want to try now, you can get a copy of this specific commit.

Uncompress, cd into that new directory and run:

conda build -m .ci_support/linux_ppc64le_python3.7.____cpython.yaml recipe

[Socramin]

Thank you very much!
I will try that tomorrow.

[Socramin]

I am afraid I have not been able to make it work yet. Still, I wanted to report some feedback in case it is usefull.

The first issue I was facing was the compiler complaining because it could not find cuda libraries. Manually defining CUDA_HOME in build.sh header did nothing, but copying:

        CMAKE_FLAGS+=" -DCUDA_TOOLKIT_ROOT_DIR=${CUDA_HOME}"
        CMAKE_FLAGS+=" -DCMAKE_LIBRARY_PATH=${CUDA_HOME}/lib64/stubs"
        CMAKE_FLAGS+=" -DOPENMM_BUILD_CUDA_TESTS=OFF"

to the $target_platform" == linux-ppc64le case solved that issue.

With this modification, everything seems to be working except the packaging. When

Packaging openmm
INFO:conda_build.build:Packaging openmm
Packaging openmm-7.5.0-py37h2279694_0
INFO:conda_build.build:Packaging openmm-7.5.0-py37h2279694_0
compiling .pyc files...
number of files: 544

is reached, the rpath calls seem to work ok but something fails after "Fixing permissions" is printed.
The complete log of this error is:

Fixing permissions
Traceback (most recent call last):
  File "/home/aaa/anaconda3/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/home/aaa/anaconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 438, in main
    execute(sys.argv[1:])
  File "/home/aaa/anaconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 429, in execute
    verify=args.verify)
  File "/home/aaa/anaconda3/lib/python3.7/site-packages/conda_build/api.py", line 201, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "/home/aaa/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 2204, in build_tree
    notest=notest,
  File "/home/aaa/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 1599, in build
    built_package = bundlers[output_d.get('type', 'conda')](output_d, m, env, stats)
  File "/home/aaa/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 963, in bundle_conda
    output['checksums'] = create_info_files(metadata, files, prefix=metadata.config.host_prefix)
  File "/home/aaa/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 658, in create_info_files
    copy_license(m)
  File "/home/aaa/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 349, in copy_license
    src_file = join(m.config.work_dir, license_file)
  File "/home/aaa/anaconda3/lib/python3.7/posixpath.py", line 94, in join
    genericpath._check_arg_types('join', a, *p)
  File "/home/aaa/anaconda3/lib/python3.7/genericpath.py", line 149, in _check_arg_types
    (funcname, s.__class__.__name__)) from None
TypeError: join() argument must be str or bytes, not 'list'

[jaimergp]

Hm, that looks like a conda-build bug. What version are you using? Maybe an update (or downgrade 😬) is needed.

[Socramin]

I tried changing to different versions of conda build:

#Python 3.7
conda-build               3.19.2           py37hc8dfbb8_2    conda-forge
conda-build               3.18.12          py37hc8dfbb8_2    conda-forge
#Python 3.6
conda-build               3.19.2           py36h9f0ad1d_2    conda-forge

but none of them worked.

All this didn't make any sense because conda-build was working before, so I ended up taking a more drastic solution. I removed anaconda3 and made a fresh install followed by a conda update --all and this finally fixed the packaging problem. Unfortunately there is a segmentation fault during the final testing testing:

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
export PREFIX=/home/aaa/anaconda3/conda-bld/openmm_1595067390126/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol
export SRC_DIR=/home/aaa/anaconda3/conda-bld/openmm_1595067390126/test_tmp
/home/aaa/anaconda3/conda-bld/openmm_1595067390126/test_tmp/conda_test_runner.sh: line 2: 34252 Segmentation fault      "/home/aaa/anaconda3/conda-bld/openmm_1595067390126/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/bin/python" -s "/home/aaa/anaconda3/conda-bld/openmm_1595067390126/test_tmp/run_test.py"
Tests failed for openmm-7.5.0-py37h1016c5d_0.tar.bz2 - moving package to /home/aaa/anaconda3/conda-bld/broken
WARNING:conda_build.build:Tests failed for openmm-7.5.0-py37h1016c5d_0.tar.bz2 - moving package to /home/aaa/anaconda3/conda-bld/broken
WARNING conda_build.build:tests_failed(2751): Tests failed for openmm-7.5.0-py37h1016c5d_0.tar.bz2 - moving package to /home/aaa/anaconda3/conda-bld/broken
TESTS FAILED: openmm-7.5.0-py37h1016c5d_0.tar.bz2

Thanks a lot for your patience with these issues.

[jaimergp]

I will have to debug this further now that there's cudatoolkit packages for ppc64le. I'll keep you posted, thanks!

[Socramin]

Thank you very much!

I am out on vacation for a couple of weeks or so, so excuse me if it it takes me some time to answer.

By the way, in case it is useful, just before leaving I was able to make a successful compilation (one that passes the python -m simtk.testInstallation tests) with the old recipe using the code modified by Peastman in #2781.

When I come back, I still want to test again that specific commit you sent using the new openmm code. I will provide feedback when I do so.

[jaimergp]

Any updates on this @Socramin? PR #23 is now passing so you should be able to get it working!

[Socramin]

I am sorry for the lack of feedback. I am a part-time university teacher and the course start under COVID19 is getting quite crazy. I will try to make it work this weekend.

Thank you for your patience!

[jaimergp]

No rush! I just wanted to check if I could help somehow. Just tag me whenever you need!