Underworld paper in the Journal of Open Source Software
The paper of record for underworld 2 has been published. It does tie to the Underworld 2.9 release but is sufficiently broad to cite for all 2.x releases.
Mansour, John, Julian Giordani, Louis Moresi, Romain Beucher, Owen Kaluza, Mirko Velic, Rebecca Farrington, Steve Quenette, and Adam Beall. “Underworld2: Python Geodynamics Modelling for Desktop, HPC and Cloud.” Journal of Open Source Software 5, no. 47 (March 6, 2020): 1797. https://doi.org/10.21105/joss.01797.
While Underworld2 is a Python API, it relies on a complex stack of C libraries to provide much of its core functionality. Statically typed languages such as C are generally orders of magnitude faster than dynamically typed languages (such as Python) and therefore are usually the best choice for numerically heavy routines. Indeed, most computational Python libraries (such as
scipy use Python for high level orchestration, but outsource all expensive computations to statically typed layers, and
underworld is no different. While this design is critical to achieving high performance, it does come with the necessary complexity of generating library binaries for each platform
underworld needs to run on. ABI compatible third party libraries (such as
petsc) also need to be created if not available.
The Underworld team’s solution to this problem is container virtualisation, and in particular Docker. Containers allow us to generate an image containing the entire runtime stack required to run
underworld, along with all other peripheral tools our users usually wish to run (such as
Jupyter). Containers also provide portability, with the single image being able to be utilised on any platform where a Linux kernel can be made available (which includes OS X and Windows). Indeed there are many other advantages to using containers, such as reproducibility, the ability to switch versions trivially, and even HPC amenability is becoming realistic via tools such as Shifter and Singularity (with potential benefits over bare metal for Python in particular).
However there are certainly shortcomings to container usage. Foremost, it presents a significant learning curve for new users, particularly those not familiar with virtualisation. While front-ends such as Kitematic go a long way to hide much of this difficulty, users will probably still require some understanding of the underlying paradigms for frictionless operation. Image size can also be of annoyance, with our complete images weighing in at almost 1GB, and over 2GB when uncompressed. Finally, running inside a virtualised environment isolates
underworld from the host and its software, such as locally installed Python modules and tools such as IDEs.
It is therefore also nice to have the option of a natively installed
underworld module. We have always provided minimal compilation instructions to allow users to generate local
underworld builds, but the expertise required to successfully compile
underworld and its dependencies generally restricted this to power users. However in recent times the installation of third party libraries has become easier, and we’ve also made some changes to simplify our requirements. As such, we’re now in a position where we can provide
underworld installation via the
pip Python package manager:
pip3 install -v git+https://github.com/underworldcode/underworld2
Note that we unable to provide pre-compiled binaries at the moment, so
pip will attempted to compiled
underworld locally. You therefore will require various tools/libraries for the compilation process, as well as Python3, pip, and any required Python modules. We also recommend the use of
virtualenv Python environments which allow you to run different Python environments concurrently and only install to user space.
Note that the installation of
pip is still experimental, and while we encourage users to give it a shot (and welcome feedback), usage via Docker containers is still the recommendation.
PETSc: PETSc can be installed via
pip these days, or is usually available via platform package managers (such as
apt on Ubuntu as
petsc-dev). If you have PETSc installed in a non-standard location, please set the
PETSC_DIR environment variable to specify the required location.
MPI & mpi4py: You will need an implementation of MPI installed on your system. Underworld is commonly used with MPICH and OpenMPI. You will also need to install the
mpi4py package (via
pip) which provides Python bindings to the MPI library. If non-standard, you can specify the wrapped compilers by setting the
MPICXX environment variables.
h5py: The standard
h5py (installed via
pip) is the recommended version for desktop usage. However, note that it will be the non-parallel enabled version, and for large parallel simulations saving/reading data may become a bottleneck, and collective IO via MPI-enabled
h5py is recommended. The following command may be useful for installed MPI-enabled
h5py where necessary:
CC=mpicc HDF5_MPI="ON" HDF5_DIR=/path/to/your/hdf5/install/ pip install --no-binary=h5py h5py
Note that, for the above, you will also need to have a parallel enabled version of the
HDF5 library available for
h5py to link against.
Lavavu: For rendering of visualisations, you will also need to install
pip). Please check the lavavu page for further installation instructions.
Other: The following should also be installed via a system package manager (such as apt on Ubuntu):
libxml2-dev (or equivalent). The following should be installed via pip:
Please refer to the
Installation.rst file located at the top level of the project repository for the latest installation information and guidelines.