Problems installing firedrake on MacOS

[acse-yw11823]

I have been trying to install firedrake on my Mac, but unfortunately I cannot get the install to complete. I have gone through all common steps trying to fix the problem, including using pyenv but to no success. Would you please be able to help me fix the problem?

Upon using $(brew --prefix)/bin/python3 firedrake-install the install starts and gets to the point of installing petsc, but then interrupts with the following printout in terminal:

yw11823@IC-FVFL80FW1WGC acds3-peer-evaluation-yw11823 % curl -O https://raw.githubusercontent.com/firedrakeproject/firedrake/master/scripts/firedrake-install
python3.11 firedrake-install

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 94492  100 94492    0     0  68016      0  0:00:01  0:00:01 --:--:-- 67979
/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install:20: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pkg_resources.extern.packaging.version import Version, InvalidVersion
Running firedrake-install
Installing command line tools...
Checking brew doctor...
Installing required packages via homebrew...
Required package 'gcc' is already installed via Homebrew.
Required package 'autoconf' is already installed via Homebrew.
Required package 'make' is already installed via Homebrew.
Required package 'automake' is already installed via Homebrew.
Required package 'cmake' is already installed via Homebrew.
Required package 'libtool' is already installed via Homebrew.
Required package 'boost' is already installed via Homebrew.
Creating firedrake venv in '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake'.
BLAS config is {'BLAS': '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/petsc/default', 'OPENBLAS': '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/petsc/default', 'LDFLAGS': '-L/opt/homebrew/opt/gcc/lib/gcc/14'}
Configuration saved to /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/.configuration.json
Cloning firedrake

Failed to clone firedrake using ssh, falling back to https.
Successfully cloned repository firedrake.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Creating firedrake-update script.
Installing numpy using setuptools==59.2.0 and wheel==0.37.0 and Cython==0.29.36
Installing numpy==1.24 due to performance regression
Updating setuptools and wheel to latest versions
Pip installing VTK to venv
Cloning the dependencies of firedrake
Cloning ufl

Failed to clone ufl using ssh, falling back to https.
Successfully cloned repository ufl.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning fiat

Failed to clone fiat using ssh, falling back to https.
Successfully cloned repository fiat.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning FInAT

Failed to clone FInAT using ssh, falling back to https.
Successfully cloned repository FInAT.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning tsfc

Failed to clone tsfc using ssh, falling back to https.
Successfully cloned repository tsfc.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning PyOP2

Failed to clone PyOP2 using ssh, falling back to https.
Successfully cloned repository PyOP2.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning pyadjoint

Failed to clone pyadjoint using ssh, falling back to https.
Successfully cloned repository pyadjoint.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning petsc

Failed to clone petsc using ssh, falling back to https.
Successfully cloned repository petsc.
Checking out branch firedrake
Successfully checked out branch firedrake
Updating submodules.
Successfully updated submodules.
Cloning pytest-mpi

Failed to clone pytest-mpi using ssh, falling back to https.
Successfully cloned repository pytest-mpi.
Checking out branch main
Successfully checked out branch main
Updating submodules.
Successfully updated submodules.
Cloning the dependencies of PyOP2
Cloning loopy

Failed to clone loopy using ssh, falling back to https.
Successfully cloned repository loopy.
Checking out branch main
Successfully checked out branch main
Updating submodules.
Successfully updated submodules.
Installing petsc/
Depending on your platform, PETSc may take an hour or more to build!
Traceback (most recent call last):
  File "/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install", line 1870, in <module>
    install("petsc/")
  File "/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install", line 1064, in install
    build_and_install_petsc()
  File "/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install", line 1176, in build_and_install_petsc
    check_call([python, "./configure", "PETSC_DIR={}".format(petsc_dir), "PETSC_ARCH={}".format(petsc_arch)] + petsc_options)
  File "/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install", line 680, in check_call
    log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=os.environ).decode())
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/bin/python', './configure', 'PETSC_DIR=/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/petsc', 'PETSC_ARCH=default', '--download-metis', '--LDFLAGS=-Wl,-ld_classic', '--download-mpich', '--download-hdf5', '--download-netcdf', '--download-mumps', '--with-c2html=0', '--download-pnetcdf', '--CFLAGS=-Wno-implicit-function-declaration', '--with-debugging=0', '--with-fortran-bindings=0', '--download-mpich-configure-arguments=--disable-opencl', '--with-zlib', '--download-ptscotch', '--download-openblas', '--download-scalapack', '--download-suitesparse', '--download-hwloc', '--download-bison', "--download-openblas-make-options='USE_THREAD=0 USE_LOCKING=1 USE_OPENMP=0'", '--download-hypre', '--download-pastix', '--download-superlu_dist', '--with-shared-libraries=1', '--with-x=0', '--download-hwloc-configure-arguments=--disable-opencl']' returned non-zero exit status 1.


Install log saved in /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install.log
yw11823@IC-FVFL80FW1WGC acds3-peer-evaluation-yw11823 % brew doctor
Your system is ready to brew.

The firedrake install log is here:
firedrake-install.log
and the and the petsc configure log is here:
configure.log

Thank you very much for your help!

[Ig-dolci]

Can you please try the instruction Q2, number 2. described at firedrake FaQ?

[acse-yw11823]

Thanks very much for the swift reply. I have tried to downgrade the Xcode to the 15.2 version and the firedrake-install script by requesting. But I still have the same problem:

yw11823@IC-FVFL80FW1WGC acds3-peer-evaluation-yw11823 % python3.11 firedrake-install
/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install:20: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pkg_resources.extern.packaging.version import Version, InvalidVersion
Running firedrake-install
Installing command line tools...
Checking brew doctor...
Installing required packages via homebrew...
Required package 'gcc' is already installed via Homebrew.
Required package 'autoconf' is already installed via Homebrew.
Required package 'make' is already installed via Homebrew.
Required package 'automake' is already installed via Homebrew.
Required package 'cmake' is already installed via Homebrew.
Required package 'libtool' is already installed via Homebrew.
Required package 'boost' is already installed via Homebrew.
Creating firedrake venv in '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake'.
BLAS config is {'BLAS': '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/petsc/default', 'OPENBLAS': '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/petsc/default', 'LDFLAGS': '-L/opt/homebrew/opt/gcc/lib/gcc/14'}
Configuration saved to /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/.configuration.json
Cloning firedrake

Failed to clone firedrake using ssh, falling back to https.
Successfully cloned repository firedrake.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Creating firedrake-update script.
Installing numpy using setuptools==59.2.0 and wheel==0.37.0 and Cython==0.29.36
Installing numpy==1.24 due to performance regression
Updating setuptools and wheel to latest versions
Pip installing VTK to venv
Cloning the dependencies of firedrake
Cloning ufl

Failed to clone ufl using ssh, falling back to https.
Successfully cloned repository ufl.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning fiat

Failed to clone fiat using ssh, falling back to https.
Successfully cloned repository fiat.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning FInAT

Failed to clone FInAT using ssh, falling back to https.
Successfully cloned repository FInAT.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning tsfc

Failed to clone tsfc using ssh, falling back to https.
Successfully cloned repository tsfc.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning PyOP2

Failed to clone PyOP2 using ssh, falling back to https.
Successfully cloned repository PyOP2.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning pyadjoint

Failed to clone pyadjoint using ssh, falling back to https.
Successfully cloned repository pyadjoint.
Checking out branch master
Successfully checked out branch master
Updating submodules.
Successfully updated submodules.
Cloning petsc

Failed to clone petsc using ssh, falling back to https.
Successfully cloned repository petsc.
Checking out branch firedrake
Successfully checked out branch firedrake
Updating submodules.
Successfully updated submodules.
Cloning pytest-mpi

Failed to clone pytest-mpi using ssh, falling back to https.
Successfully cloned repository pytest-mpi.
Checking out branch main
Successfully checked out branch main
Updating submodules.
Successfully updated submodules.
Cloning the dependencies of PyOP2
Cloning loopy

Failed to clone loopy using ssh, falling back to https.
Successfully cloned repository loopy.
Checking out branch main
Successfully checked out branch main
Updating submodules.
Successfully updated submodules.
Installing petsc/
Depending on your platform, PETSc may take an hour or more to build!
Traceback (most recent call last):
  File "/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install", line 1871, in <module>
    install("petsc/")
  File "/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install", line 1065, in install
    build_and_install_petsc()
  File "/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install", line 1177, in build_and_install_petsc
    check_call([python, "./configure", "PETSC_DIR={}".format(petsc_dir), "PETSC_ARCH={}".format(petsc_arch)] + petsc_options)
  File "/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install", line 680, in check_call
    log.debug(subprocess.check_output(arguments, stderr=subprocess.STDOUT, env=os.environ).decode())
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/bin/python', './configure', 'PETSC_DIR=/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/petsc', 'PETSC_ARCH=default', '--download-netcdf', '--download-scalapack', '--download-hdf5', '--CFLAGS=-Wno-implicit-function-declaration -Wno-int-conversion -Wno-deprecated-non-prototype -Wno-implicit-int', '--download-hwloc', '--download-mpich-configure-arguments=--disable-opencl', '--download-pnetcdf', '--download-bison', '--download-openblas', '--LDFLAGS=-Wl,-ld_classic', '--download-suitesparse', '--with-x=0', '--with-fortran-bindings=0', '--download-pastix', '--with-zlib', '--with-debugging=0', "--download-openblas-make-options='USE_THREAD=0 USE_LOCKING=1 USE_OPENMP=0'", '--with-c2html=0', '--with-shared-libraries=1', '--download-metis', '--download-mpich', '--download-superlu_dist', '--download-hypre', '--download-hwloc-configure-arguments=--disable-opencl', '--download-mumps', '--download-ptscotch']' returned non-zero exit status 1.


Install log saved in /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake-install.log

Here is my firedrake-install. log file:
firedrake-install.log

and here is my configure.log file:configure.log

.

Many thanks for the help.

[connorjward]

This run just appears to have failed to download OpenBLAS - which could either be a temporary server issue or a network connectivity issue. If you try again you should make it further.

The first issue you encountered was a strange one:

src/lapack_wrappers.c:609:81: error: incompatible integer to pointer conversion passing 'blasint' (aka 'int') to parameter of type 'const blasint *' (aka 'const int *'); take the address with & [-Wint-conversion]
    RELAPACK_zgemmt(uplo, transA, transB, n, k, alpha, A, ldA, B, ldB, beta, C, info);

Can you make sure that your system is up-to-date (either Xcode 15.2 or 15.3 and have run brew upgrade)? You can check the Xcode version from xcodebuild -version.

[acse-yw11823]

Hello, thanks for the reply. I have tried upgrading Brew and using Xcode 15.2 to rerun it. It took over two hours and succeeded. Thanks a lot!!

[acse-yw11823]

But it shows the following in the terminal when I run the test suite after installation to check that Firedrake is fully functional:

(firedrake) yw11823@IC-FVFL80FW1WGC acds3-peer-evaluation-yw11823 % cd $VIRTUAL_ENV/src/firedrake
pytest tests/regression/ -k "poisson_strong or stokes_mini or dg_advection"
============================== test session starts ===============================
platform darwin -- Python 3.11.9, pytest-8.2.2, pluggy-1.5.0
rootdir: /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake
configfile: setup.cfg
plugins: mpi-0.1, anyio-4.4.0, nbval-0.11.0, xdist-3.6.1
collected 3556 items / 3529 deselected / 2 skipped / 27 selected                 

tests/regression/test_dg_advection.py .F.F                                 [ 14%]
tests/regression/test_poisson_strong_bcs.py ................F              [ 77%]
tests/regression/test_poisson_strong_bcs_nitsche.py ....                   [ 92%]
tests/regression/test_stokes_mini.py ..                                    [100%]

==================================== FAILURES ====================================
_________________ test_dg_advection_icosahedral_sphere_parallel __________________

args = (), kwargs = {}

    def parallel_callback(*args, **kwargs):
>       subprocess.run(cmd, check=True)

../pytest-mpi/pytest_mpi.py:192: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, capture_output = False, timeout = None, check = True
popenargs = (['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', ...],)
kwargs = {}
process = <Popen: returncode: 15 args: ['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CH...>
stdout = None, stderr = None, retcode = 15

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them,
        or pass capture_output=True to capture both.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.
    
        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel', ':', '-n', '2', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel', '--tb=no', '--no-summary', '--no-header', '--disable-warnings', '--show-capture=no']' returned non-zero exit status 15.

/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py:571: CalledProcessError
____________________ test_dg_advection_cubed_sphere_parallel _____________________

args = (), kwargs = {}

    def parallel_callback(*args, **kwargs):
>       subprocess.run(cmd, check=True)

../pytest-mpi/pytest_mpi.py:192: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, capture_output = False, timeout = None, check = True
popenargs = (['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', ...],)
kwargs = {}
process = <Popen: returncode: 15 args: ['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CH...>
stdout = None, stderr = None, retcode = 15

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them,
        or pass capture_output=True to capture both.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.
    
        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_cubed_sphere_parallel', ':', '-n', '2', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_cubed_sphere_parallel', '--tb=no', '--no-summary', '--no-header', '--disable-warnings', '--show-capture=no']' returned non-zero exit status 15.

/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py:571: CalledProcessError
_____________________ test_poisson_analytic_linear_parallel ______________________

args = (), kwargs = {}

    def parallel_callback(*args, **kwargs):
>       subprocess.run(cmd, check=True)

../pytest-mpi/pytest_mpi.py:192: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, capture_output = False, timeout = None, check = True
popenargs = (['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', ...],)
kwargs = {}
process = <Popen: returncode: 15 args: ['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CH...>
stdout = None, stderr = None, retcode = 15

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them,
        or pass capture_output=True to capture both.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.
    
        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_poisson_strong_bcs.py::test_poisson_analytic_linear_parallel', ':', '-n', '1', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_poisson_strong_bcs.py::test_poisson_analytic_linear_parallel', '--tb=no', '--no-summary', '--no-header', '--disable-warnings', '--show-capture=no']' returned non-zero exit status 15.

/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py:571: CalledProcessError
================================ warnings summary ================================
../fiat/FIAT/__init__.py:5
  /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/fiat/FIAT/__init__.py:5: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================ short test summary info =============================
FAILED tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel - subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYT...
FAILED tests/regression/test_dg_advection.py::test_dg_advection_cubed_sphere_parallel - subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYT...
FAILED tests/regression/test_poisson_strong_bcs.py::test_poisson_analytic_linear_parallel - subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYT...
= 3 failed, 24 passed, 2 skipped, 3529 deselected, 1 warning in 125.40s (0:02:05) =
WARNING! There are options you set that were not used!
WARNING! could be spelling mistake, etc!
There is one unused database option. It is:
Option left: name:-k value: poisson_strong or stokes_mini or dg_advection source: command line

[acse-yw11823]

I tried to modify the hosts database at /etc/hosts to include the following entries:

127.0.0.1       LOCALHOSTNAME.local
127.0.0.1       LOCALHOSTNAME

But still have the following error:

(firedrake) yw11823@IC-FVFL80FW1WGC acds3-peer-evaluation-yw11823 % hostname

IC-FVFL80FW1WGC
(firedrake) yw11823@IC-FVFL80FW1WGC acds3-peer-evaluation-yw11823 % mpiexec -n 4 python test_mpi.py

Hello from process 0 of 4
Hello from process 1 of 4
Hello from process 2 of 4
Hello from process 3 of 4
(firedrake) yw11823@IC-FVFL80FW1WGC acds3-peer-evaluation-yw11823 % cd $VIRTUAL_ENV/src/firedrake
pytest tests/regression/ -k "poisson_strong or stokes_mini or dg_advection"
============================== test session starts ===============================
platform darwin -- Python 3.11.9, pytest-8.2.2, pluggy-1.5.0
rootdir: /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake
configfile: setup.cfg
plugins: mpi-0.1, anyio-4.4.0, nbval-0.11.0, xdist-3.6.1
collected 3556 items / 3529 deselected / 2 skipped / 27 selected                 

tests/regression/test_dg_advection.py .F.^C

==================================== FAILURES ====================================
_________________ test_dg_advection_icosahedral_sphere_parallel __________________

args = (), kwargs = {}

    def parallel_callback(*args, **kwargs):
>       subprocess.run(cmd, check=True)

../pytest-mpi/pytest_mpi.py:192: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, capture_output = False, timeout = None, check = True
popenargs = (['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', ...],)
kwargs = {}
process = <Popen: returncode: 1 args: ['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHI...>
stdout = None, stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them,
        or pass capture_output=True to capture both.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.
    
        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel', ':', '-n', '2', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel', '--tb=no', '--no-summary', '--no-header', '--disable-warnings', '--show-capture=no']' returned non-zero exit status 1.

/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py:571: CalledProcessError
------------------------------ Captured stdout call ------------------------------
FFF
1 failed, 1 warning in 3.98s

1 failed, 1 warning in 3.98s

=================================== FAILURES ===================================
________________ test_dg_advection_icosahedral_sphere_parallel _________________

    @pytest.mark.parallel(nprocs=3)
    def test_dg_advection_icosahedral_sphere_parallel():
>       run_test(UnitIcosahedralSphereMesh(refinement_level=3))

tests/regression/test_dg_advection.py:74: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

mesh = Mesh(VectorElement(FiniteElement('Lagrange', Cell(triangle, 3), 1), dim=3), 1)

    def run_test(mesh):
        x = SpatialCoordinate(mesh)
        mesh.init_cell_orientations(x)
    
        V = FunctionSpace(mesh, "DG", 0)
        M = VectorFunctionSpace(mesh, "CG", 1)
    
        # advecting velocity
        u0 = as_vector((-x[1]*(1 - x[2]*x[2]), x[0]*(1 - x[2]*x[2]), Constant(0)))
        u = Function(M).interpolate(u0)
    
        dt = (pi/3) * 0.006
        Dt = Constant(dt)
    
        phi = TestFunction(V)
        D = TrialFunction(V)
    
        n = FacetNormal(mesh)
    
        un = 0.5 * (dot(u, n) + abs(dot(u, n)))
    
        a_mass = inner(D, phi) * dx
        a_int = inner(-u*D, grad(phi))*dx
        a_flux = inner(un('+')*D('+') - un('-')*D('-'), jump(phi))*dS
        arhs = a_mass - Dt * (a_int + a_flux)
    
        dD1 = Function(V)
        D1 = Function(V)
    
        D0 = conditional(le(real(x[0]), 0), 1, 0.0)
        D = Function(V).interpolate(D0)
    
        t = 0.0
        T = 10*dt
    
        problem = LinearVariationalProblem(a_mass, action(arhs, D1), dD1)
        solver = LinearVariationalSolver(problem, solver_parameters={'ksp_type': 'cg'})
    
        L2_0 = norm(D)
        Dbar_0 = assemble(D*dx)
        while t < (T - dt/2):
            D1.assign(D)
            solver.solve()
            D1.assign(dD1)
    
            solver.solve()
            D1.assign(0.75*D + 0.25*dD1)
            solver.solve()
            D.assign((1.0/3.0)*D + (2.0/3.0)*dD1)
    
            t += dt
    
        L2_T = norm(D)
        Dbar_T = assemble(D*dx)
    
        # L2 norm decreases
        assert L2_T < L2_0
    
        # Mass conserved
>       assert np.allclose(Dbar_T, Dbar_0)
E       assert False
E        +  where False = <function allclose at 0x10a1edd00>(0.00010694428099914108, 6.314952848718281)
E        +    where <function allclose at 0x10a1edd00> = np.allclose

tests/regression/test_dg_advection.py:65: AssertionError
=============================== warnings summary ===============================
../fiat/FIAT/__init__.py:5
  /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/fiat/FIAT/__init__.py:5: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel
1 failed, 1 warning in 3.98s
WARNING! There are options you set that were not used!
WARNING! could be spelling mistake, etc!
There are 3 unused database options. They are:
Option left: name:--runxfail (no value) source: command line
Option left: name:-q value: /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel source: command line
Option left: name:-s (no value) source: command line
------------------------------ Captured stderr call ------------------------------
firedrake:WARNING OMP_NUM_THREADS is not set or is set to a value greater than 1, we suggest setting OMP_NUM_THREADS=1 to improve performance
================================ warnings summary ================================
../fiat/FIAT/__init__.py:5
  /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/fiat/FIAT/__init__.py:5: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================ short test summary info =============================
FAILED tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel - subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYT...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py:2011: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --full-trace)
= 1 failed, 2 passed, 2 skipped, 3529 deselected, 1 warning in 709.42s (0:11:49) =
WARNING! There are options you set that were not used!
WARNING! could be spelling mistake, etc!
There is one unused database option. It is:
Option left: name:-k value: poisson_strong or stokes_mini or dg_advection source: command line
(firedrake) yw11823@IC-FVFL80FW1WGC firedrake % sudo nano /etc/hosts           

Password:
(firedrake) yw11823@IC-FVFL80FW1WGC firedrake % export OMP_NUM_THREADS=1

(firedrake) yw11823@IC-FVFL80FW1WGC firedrake % mpiexec -n 4 python test_mpi.py 2> error.log
cat error.log

Hello from process 2 of 4
Hello from process 3 of 4
Hello from process 0 of 4
Hello from process 1 of 4
(firedrake) yw11823@IC-FVFL80FW1WGC firedrake % mpiexec -n 4 hostname

IC-FVFL80FW1WGC
IC-FVFL80FW1WGC
IC-FVFL80FW1WGC
IC-FVFL80FW1WGC
(firedrake) yw11823@IC-FVFL80FW1WGC firedrake % export OMP_NUM_THREADS=1

(firedrake) yw11823@IC-FVFL80FW1WGC firedrake % cd $VIRTUAL_ENV/src/firedrake
pytest tests/regression/ -k "poisson_strong or stokes_mini or dg_advection"

============================== test session starts ===============================
platform darwin -- Python 3.11.9, pytest-8.2.2, pluggy-1.5.0
rootdir: /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake
configfile: setup.cfg
plugins: mpi-0.1, anyio-4.4.0, nbval-0.11.0, xdist-3.6.1
collected 3556 items / 3529 deselected / 2 skipped / 27 selected                 

tests/regression/test_dg_advection.py .F.F                                 [ 14%]
tests/regression/test_poisson_strong_bcs.py ................F              [ 77%]
tests/regression/test_poisson_strong_bcs_nitsche.py ....                   [ 92%]
tests/regression/test_stokes_mini.py ..                                    [100%]

==================================== FAILURES ====================================
_________________ test_dg_advection_icosahedral_sphere_parallel __________________

args = (), kwargs = {}

    def parallel_callback(*args, **kwargs):
>       subprocess.run(cmd, check=True)

../pytest-mpi/pytest_mpi.py:192: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, capture_output = False, timeout = None, check = True
popenargs = (['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', ...],)
kwargs = {}
process = <Popen: returncode: 1 args: ['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHI...>
stdout = None, stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them,
        or pass capture_output=True to capture both.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.
    
        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel', ':', '-n', '2', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel', '--tb=no', '--no-summary', '--no-header', '--disable-warnings', '--show-capture=no']' returned non-zero exit status 1.

/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py:571: CalledProcessError
------------------------------ Captured stdout call ------------------------------
FFF
1 failed, 1 warning in 6.21s

=================================== FAILURES ===================================
________________ test_dg_advection_icosahedral_sphere_parallel _________________

    @pytest.mark.parallel(nprocs=3)
    def test_dg_advection_icosahedral_sphere_parallel():
>       run_test(UnitIcosahedralSphereMesh(refinement_level=3))

tests/regression/test_dg_advection.py:74: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

mesh = Mesh(VectorElement(FiniteElement('Lagrange', Cell(triangle, 3), 1), dim=3), 1)

    def run_test(mesh):
        x = SpatialCoordinate(mesh)
        mesh.init_cell_orientations(x)
    
        V = FunctionSpace(mesh, "DG", 0)
        M = VectorFunctionSpace(mesh, "CG", 1)
    
        # advecting velocity
        u0 = as_vector((-x[1]*(1 - x[2]*x[2]), x[0]*(1 - x[2]*x[2]), Constant(0)))
        u = Function(M).interpolate(u0)
    
        dt = (pi/3) * 0.006
        Dt = Constant(dt)
    
        phi = TestFunction(V)
        D = TrialFunction(V)
    
        n = FacetNormal(mesh)
    
        un = 0.5 * (dot(u, n) + abs(dot(u, n)))
    
        a_mass = inner(D, phi) * dx
        a_int = inner(-u*D, grad(phi))*dx
        a_flux = inner(un('+')*D('+') - un('-')*D('-'), jump(phi))*dS
        arhs = a_mass - Dt * (a_int + a_flux)
    
        dD1 = Function(V)
        D1 = Function(V)
    
        D0 = conditional(le(real(x[0]), 0), 1, 0.0)
        D = Function(V).interpolate(D0)
    
        t = 0.0
        T = 10*dt
    
        problem = LinearVariationalProblem(a_mass, action(arhs, D1), dD1)
        solver = LinearVariationalSolver(problem, solver_parameters={'ksp_type': 'cg'})
    
        L2_0 = norm(D)
        Dbar_0 = assemble(D*dx)
        while t < (T - dt/2):
            D1.assign(D)
            solver.solve()
            D1.assign(dD1)
    
            solver.solve()
            D1.assign(0.75*D + 0.25*dD1)
            solver.solve()
            D.assign((1.0/3.0)*D + (2.0/3.0)*dD1)
    
            t += dt
    
        L2_T = norm(D)
        Dbar_T = assemble(D*dx)
    
        # L2 norm decreases
        assert L2_T < L2_0
    
        # Mass conserved
>       assert np.allclose(Dbar_T, Dbar_0)
E       assert False
E        +  where False = <function allclose at 0x104a89d00>(0.00010694428099914108, 6.314952848718281)
E        +    where <function allclose at 0x104a89d00> = np.allclose

tests/regression/test_dg_advection.py:65: AssertionError
=============================== warnings summary ===============================
../fiat/FIAT/__init__.py:5
  /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/fiat/FIAT/__init__.py:5: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel
1 failed, 1 warning in 6.21s

1 failed, 1 warning in 6.21s
WARNING! There are options you set that were not used!
WARNING! could be spelling mistake, etc!
There are 3 unused database options. They are:
Option left: name:--runxfail (no value) source: command line
Option left: name:-q value: /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel source: command line
Option left: name:-s (no value) source: command line
____________________ test_dg_advection_cubed_sphere_parallel _____________________

args = (), kwargs = {}

    def parallel_callback(*args, **kwargs):
>       subprocess.run(cmd, check=True)

../pytest-mpi/pytest_mpi.py:192: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, capture_output = False, timeout = None, check = True
popenargs = (['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', ...],)
kwargs = {}
process = <Popen: returncode: 1 args: ['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHI...>
stdout = None, stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them,
        or pass capture_output=True to capture both.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.
    
        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_cubed_sphere_parallel', ':', '-n', '2', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_cubed_sphere_parallel', '--tb=no', '--no-summary', '--no-header', '--disable-warnings', '--show-capture=no']' returned non-zero exit status 1.

/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py:571: CalledProcessError
------------------------------ Captured stdout call ------------------------------
FFF
=================================== FAILURES ===================================
___________________ test_dg_advection_cubed_sphere_parallel ____________________

    @pytest.mark.parallel(nprocs=3)
    def test_dg_advection_cubed_sphere_parallel():
>       run_test(UnitCubedSphereMesh(refinement_level=4))

tests/regression/test_dg_advection.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

mesh = Mesh(VectorElement(FiniteElement('Q', Cell(quadrilateral, 3), 1), dim=3), 1)

    def run_test(mesh):
        x = SpatialCoordinate(mesh)
        mesh.init_cell_orientations(x)
    
        V = FunctionSpace(mesh, "DG", 0)
        M = VectorFunctionSpace(mesh, "CG", 1)
    
        # advecting velocity
        u0 = as_vector((-x[1]*(1 - x[2]*x[2]), x[0]*(1 - x[2]*x[2]), Constant(0)))
        u = Function(M).interpolate(u0)
    
        dt = (pi/3) * 0.006
        Dt = Constant(dt)
    
        phi = TestFunction(V)
        D = TrialFunction(V)
    
        n = FacetNormal(mesh)
    
        un = 0.5 * (dot(u, n) + abs(dot(u, n)))
    
        a_mass = inner(D, phi) * dx
        a_int = inner(-u*D, grad(phi))*dx
        a_flux = inner(un('+')*D('+') - un('-')*D('-'), jump(phi))*dS
        arhs = a_mass - Dt * (a_int + a_flux)
    
        dD1 = Function(V)
        D1 = Function(V)
    
        D0 = conditional(le(real(x[0]), 0), 1, 0.0)
        D = Function(V).interpolate(D0)
    
        t = 0.0
        T = 10*dt
    
        problem = LinearVariationalProblem(a_mass, action(arhs, D1), dD1)
        solver = LinearVariationalSolver(problem, solver_parameters={'ksp_type': 'cg'})
    
        L2_0 = norm(D)
        Dbar_0 = assemble(D*dx)
        while t < (T - dt/2):
            D1.assign(D)
            solver.solve()
            D1.assign(dD1)
    
            solver.solve()
            D1.assign(0.75*D + 0.25*dD1)
            solver.solve()
            D.assign((1.0/3.0)*D + (2.0/3.0)*dD1)
    
            t += dt
    
        L2_T = norm(D)
        Dbar_T = assemble(D*dx)
    
        # L2 norm decreases
        assert L2_T < L2_0
    
        # Mass conserved
>       assert np.allclose(Dbar_T, Dbar_0)
E       assert False
E        +  where False = <function allclose at 0x10a369d00>(0.00010618437443777819, 6.270081126176359)
E        +    where <function allclose at 0x10a369d00> = np.allclose

tests/regression/test_dg_advection.py:65: AssertionError
=============================== warnings summary ===============================
../fiat/FIAT/__init__.py:5
  /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/fiat/FIAT/__init__.py:5: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/regression/test_dg_advection.py::test_dg_advection_cubed_sphere_parallel
1 failed, 1 warning in 7.69s

1 failed, 1 warning in 7.69s

1 failed, 1 warning in 7.70s
WARNING! There are options you set that were not used!
WARNING! could be spelling mistake, etc!
There are 3 unused database options. They are:
Option left: name:--runxfail (no value) source: command line
Option left: name:-q value: /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_dg_advection.py::test_dg_advection_cubed_sphere_parallel source: command line
Option left: name:-s (no value) source: command line
_____________________ test_poisson_analytic_linear_parallel ______________________

args = (), kwargs = {}

    def parallel_callback(*args, **kwargs):
>       subprocess.run(cmd, check=True)

../pytest-mpi/pytest_mpi.py:192: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, capture_output = False, timeout = None, check = True
popenargs = (['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', ...],)
kwargs = {}
process = <Popen: returncode: 1 args: ['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHI...>
stdout = None, stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them,
        or pass capture_output=True to capture both.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.
    
        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYTEST_MPI_CHILD_PROCESS', '1', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_poisson_strong_bcs.py::test_poisson_analytic_linear_parallel', ':', '-n', '1', 'python', '-m', 'pytest', '--runxfail', '-s', '-q', '/Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_poisson_strong_bcs.py::test_poisson_analytic_linear_parallel', '--tb=no', '--no-summary', '--no-header', '--disable-warnings', '--show-capture=no']' returned non-zero exit status 1.

/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py:571: CalledProcessError
------------------------------ Captured stdout call ------------------------------
[1] error: 12.124355652982139
F[0] error: 12.124355652982139
F
1 failed, 1 warning in 1.09s

=================================== FAILURES ===================================
____________________ test_poisson_analytic_linear_parallel _____________________

    @pytest.mark.parallel(nprocs=2)
    def test_poisson_analytic_linear_parallel():
        from mpi4py import MPI
        error = run_test_linear(1, 1)
        print('[%d]' % MPI.COMM_WORLD.rank, 'error:', error)
>       assert error < 5e-6
E       assert 12.124355652982139 < 5e-06

tests/regression/test_poisson_strong_bcs.py:92: AssertionError
=============================== warnings summary ===============================
../fiat/FIAT/__init__.py:5
  /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/fiat/FIAT/__init__.py:5: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/regression/test_poisson_strong_bcs.py::test_poisson_analytic_linear_parallel
1 failed, 1 warning in 1.09s
WARNING! There are options you set that were not used!
WARNING! could be spelling mistake, etc!
There are 3 unused database options. They are:
Option left: name:--runxfail (no value) source: command line
Option left: name:-q value: /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/firedrake/tests/regression/test_poisson_strong_bcs.py::test_poisson_analytic_linear_parallel source: command line
Option left: name:-s (no value) source: command line
================================ warnings summary ================================
../fiat/FIAT/__init__.py:5
  /Users/yw11823/ACSE/acds3-peer-evaluation-yw11823/firedrake/src/fiat/FIAT/__init__.py:5: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================ short test summary info =============================
FAILED tests/regression/test_dg_advection.py::test_dg_advection_icosahedral_sphere_parallel - subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYT...
FAILED tests/regression/test_dg_advection.py::test_dg_advection_cubed_sphere_parallel - subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYT...
FAILED tests/regression/test_poisson_strong_bcs.py::test_poisson_analytic_linear_parallel - subprocess.CalledProcessError: Command '['mpiexec', '-n', '1', '-genv', '_PYT...
====== 3 failed, 24 passed, 2 skipped, 3529 deselected, 1 warning in 38.37s ======
WARNING! There are options you set that were not used!
WARNING! could be spelling mistake, etc!
There is one unused database option. It is:
Option left: name:-k value: poisson_strong or stokes_mini or dg_advection source: command line

Many thanks for the reply. And many thanks for all the help. Thank you so much for helping me resolve the issues with my code.

[connorjward]

This is unfortunately a known issue with PETSc on Mac (e.g. see this discussion). In particular using the MUMPS direct solver in parallel is known to be broken. If you don't need to run in parallel then you won't encounter this and if you really do need a parallel direct solver then you should use SuperLU_DIST instead (by specifying solver options).

Many thanks for the reply. And many thanks for all the help. Thank you so much for helping me resolve the issues with my code.

You are quite welcome :)