Building on MacOS Sierra

Oct 08, 2016

AKA: Installing all petsc dependencies through petsc and not homebrew

[Note: updated Feb 6, 2017 with a workaround for the runaway CPU usage in idle notebooks (new open-mpi version requirement)]

The new releases of MacOS generally bring a new suite of compilers and problems with pre-existing homebrew installed binaries / libraries. This can take several weeks to shake out.

A more robust, if time-consuming, approach is to build the core dependencies through petsc which ensures that only compatible versions are used. This also ensures that we can use the compatible petsc version specified by underworld2. homebrew does not allow us to specify specific versions.

When upgrading, we always recommend purging homebrew altogether

brew update
brew remove --force $(brew list)
brew doctor

or, on a new machine, installing as follows:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

A non-system python installation, libxml2, and swig are the only homebrew requirements, but you might also make sure that you have wget to grab a compatible version of petsc. The gcc requirement is to allow fortran dependencies to be installed. This takes a while but is worth having for later !

brew install python
brew install libxml2
brew install swig
brew install wget
brew install gcc   # this is needed if you choose the --download-scalapack option

We currently test against petsc-3.6.4 and we support 3.4.x, 3.5.x. You can install anywhere, but the following instructions assume /usr/local/src, change as necessary If you need petsc-3.7.x, you will need to access the development branch where we are currently testing compatibility.

wget http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-3.7.5.tar.gz  <!-- 3.6.4 -->

You should set the following environment variables

export PETSC_DIR=/usr/local/src/petsc-3.6.4      # or wherever you chose to untar petsc, and your version number !
export PETSC_ARCH=opt                            # arbitrary, for your own records if you have several builds
                                                 # and you can also let PETSc decide and it will tell you what to use here.

The following is a relatively bare-bones configuration for petsc that should work with all the example notebooks we provide. From the $PETSC_DIR run the configure script and make. Note, we specify a particular version of open-mpi here because the default that is otherwise installed appears to have some problems (idle notebooks run at full CPU load).

./config/configure.py --download-mumps=1 --download-mpi4py=1 --download-hdf5=1 --download-scalapack --download-parmetis --download-metis --download-cmake --with-debugging=0 --download-openmpi=https://www.open-mpi.org/software/ompi/v2.0/downloads/openmpi-2.0.2.tar.gz --enable-shared=1
make all
make test
export PYTHONPATH=$PETSC_DIR/$PETSC_ARCH/lib:$PYTHONPATH    # mpi4py
export PATH=$PETSC_DIR/$PETSC_ARCH/bin:$PATH                # h5pcc and mpicc

To ensure that we use the compatible packages installed by petsc and by homebrew, set the following

export MPI_DIR=$PETSC_DIR/$PETSC_ARCH
export HDF5_DIR=$PETSC_DIR/$PETSC_ARCH
export PYTHON_DIR=/usr/local/opt/python/Frameworks/Python.framework/Versions/Current
export LIBXML2_DIR=/usr/local/opt/libxml2

There are a number of python dependencies which we install via pip

pip install numpy
pip install scipy
pip install matplotlib
pip install jupyter

Obtain the underworld source code from github The installation path for underworld is not especially important but you can specify it explicitly at this stage. Your python path will need to include this directory explicitly so that you can import the underworld modules.

git clone https://github.com/underworldcode/underworld2 underworld2-2016.10.08

The build process for underworld has been simplified slightly from earlier versions

cd underworld2-2016.10.08
cd libUnderworld
./configure.py
./compile.py

Assuming that works flawlessly

export PYTHONPATH=XXX/underworld2-2016.10.08:$PYTHONPATH    # XXX = wherever you put underworld !