Failure to build from source on MacOS

[tbretl]

Describe the bug

Build from source fails on MacOS.

To Reproduce

Follow the instructions to build from source on a MacBook Pro with M2 running macOS 14.0:

1. Install gmp with brew (docs)
2. Install miniforge for arm64 (docs)
3. Create a python environment with nothing but python:

mamba create -n sf-dev
mamba activate sf-dev
mamba install -y python=3

4. Clone the repository and build with pip:

git clone https://github.com/symforce-org/symforce.git
cd symforce
pip install .

5. See error (below)

Expected behavior

Build from source should complete without error.

Screenshots

See error_log.txt

Environment (please complete the following information):

• MacBook Pro M2
• macOS 14.0
• Python 3.12
• SymForce bdbcbf7e

Additional context

• Python version does not matter (tried 3.11, etc.)
• SymForce version does not matter (tried tagged version v0.9.0, etc.)
• Build using cmake instead of pip does not matter
• Installing pinned versions of dependencies (pip install -r dev_requirements.txt) does not matter
• Using a different machine owned by a different person does not matter (a colleague gets the same error when following the same process on his own Mac w/ M2)
• Installing gmp with mamba instead of brew does not matter

[aaron-skydio]

The relevant factor may be the compiler, which I see is AppleClang 15 - looking at the Eigen source it does look like those are just unused. It might also be that the compilers we test on usually do catch this, but they're system includes so it's fine?

This should make the build work though I think, regardless of why you're actually seeing the warning. Really just the first commit here: #370

[tbretl]

Thanks.

Turns out this was an easy one - the problem was not having first installed eigen. If I either do a global install (brew install eigen) or a local install (mamba install eigen) then building SymForce from source succeeds as expected with pip install ..

It looks like the install script recognized that Eigen was missing and tried to fetch it (see Eigen3 not found, adding with FetchContent in the earlier error log), but this process did not succeed. I suspect the reason is that FetchContent failed to install Eigen inside my mamba environment and did not have permission to install Eigen globally (nor would I have wanted it to do so).

Perhaps the README.md should be modified to say that it is necessary to install Eigen before building from source.

Along these same lines, the README.md should also be modified to say that gmp must be installed with brew and cannot be installed with mamba on macOS. When I install gmp with mamba, I get the following error:

      CMake Error at /private/var/folders/0j/gcrqfhps14b11tyxczhdzdv80000gn/T/pip-build-env-qp81m144/normal/lib/python3.11/site-packages/cmake/data/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find GMP (missing: GMP_LIBRARIES GMP_INCLUDE_DIRS)
      Call Stack (most recent call first):
        /private/var/folders/0j/gcrqfhps14b11tyxczhdzdv80000gn/T/pip-build-env-qp81m144/normal/lib/python3.11/site-packages/cmake/data/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
        cmake/FindGMP.cmake:19 (find_package_handle_standard_args)
        CMakeLists.txt:198 (find_package)

[aaron-skydio]

Did you try doing a clean build with #370 before installing eigen? I expect that to work, unless I'm misunderstanding things.

The fatal errors from the log are these:

      In file included from /Users/timothybretl/Documents/sabbatical/Projects/symforce/symforce/opt/sparse_cholesky/sparse_cholesky_solver.cc:6:
      In file included from /Users/timothybretl/Documents/sabbatical/Projects/symforce/symforce/opt/sparse_cholesky/./sparse_cholesky_solver.h:14:
      In file included from /Users/timothybretl/Documents/sabbatical/Projects/symforce/build/temp.macosx-11.0-arm64-cpython-312/_deps/eigen3-src/Eigen/MetisSupport:11:
      In file included from /Users/timothybretl/Documents/sabbatical/Projects/symforce/build/temp.macosx-11.0-arm64-cpython-312/_deps/eigen3-src/Eigen/SparseCore:61:
      /Users/timothybretl/Documents/sabbatical/Projects/symforce/build/temp.macosx-11.0-arm64-cpython-312/_deps/eigen3-src/Eigen/src/SparseCore/TriangularSolver.h:273:13: error: variable 'count' set but not used [-Werror,-Wunused-but-set-variable]
            Index count = 0;
                  ^
      [ 52%] Building C object _deps/metis-build/programs/CMakeFiles/ndmetis.dir/io.c.o
      In file included from /Users/timothybretl/Documents/sabbatical/Projects/symforce/symforce/opt/sparse_cholesky/sparse_cholesky_solver.cc:6:
      In file included from /Users/timothybretl/Documents/sabbatical/Projects/symforce/symforce/opt/sparse_cholesky/./sparse_cholesky_solver.h:15:
      In file included from /Users/timothybretl/Documents/sabbatical/Projects/symforce/build/temp.macosx-11.0-arm64-cpython-312/_deps/eigen3-src/Eigen/Sparse:29:
      In file included from /Users/timothybretl/Documents/sabbatical/Projects/symforce/build/temp.macosx-11.0-arm64-cpython-312/_deps/eigen3-src/Eigen/SparseLU:35:
      /Users/timothybretl/Documents/sabbatical/Projects/symforce/build/temp.macosx-11.0-arm64-cpython-312/_deps/eigen3-src/Eigen/src/SparseLU/SparseLU_heap_relax_snode.h:78:9: error: variable 'nsuper_et_post' set but not used [-Werror,-Wunused-but-set-variable]
        Index nsuper_et_post = 0; // Number of relaxed snodes in postordered etree
              ^
      /Users/timothybretl/Documents/sabbatical/Projects/symforce/build/temp.macosx-11.0-arm64-cpython-312/_deps/eigen3-src/Eigen/src/SparseLU/SparseLU_heap_relax_snode.h:79:9: error: variable 'nsuper_et' set but not used [-Werror,-Wunused-but-set-variable]
        Index nsuper_et = 0; // Number of relaxed snodes in the original etree
              ^

Which mean that Eigen was fetched successfully, but was not added as a system include path on the compile command, meaning that warnings in Eigen itself were fatal - #370 fixes this. If Eigen is already installed, it's included as a system include path, meaning warnings in Eigen itself are ignored.

I'll take a look at the gmp issue separately