Compiling AMR-Wind

AMR-Wind is primarily written in C++ and requires a modern C++ compiler (that supports C++14 standard) to build. This section describes the dependencies of AMR-Wind as well as the general procedure to compile and execute AMR-Wind on your system. The main dependencies are listed below:

Operating system – AMR-Wind has been tested on Linux and MacOS operating systems.
C++ compiler – AMR-Wind requires a C++ compiler that supports the C++14 standard. The code has been tested on GCC v5.x and higher, LLVM Clang v7.x, and Intel 2019 compiler suites.
MPI – OpenMPI, MPICH, or Intel-MPI.
CMake – Configure and build the code
Python A recent version of python

Optional dependencies

NVIDIA CUDA version 10 or higher required to run on GPUs.
Intel OneAPI for building with SyCL.
AMD GPU for building with ROCM.

Building from source

If you are on an HPC system that provides Modules Environment, load the necessary compiler, MPI, and CMake modules. If targeting GPUs, load CUDA modules. You can also use scripts from exawind-builder.

Clone a local copy of the git repository

git clone --recursive https://github.com/exawind/amr-wind.git

Configure and build
```
mkdir amr-wind/build && cd amr-wind/build
cmake -DAMR_WIND_ENABLE_TESTS:BOOL=ON ../
make
```
Upon successful build, you will end up with an executable amr_wind in your build directory.
(Optional) Test your build
```
ctest --output-on-failure
```

CMake configuration reference

Architecture options

AMR_WIND_ENABLE_MPI: Enable MPI support for parallel executions. Default: OFF

AMR_WIND_ENABLE_OPENMP: Enable OpenMP threading support for CPU builds. It is not recommended to combine this with GPU builds. Default: OFF

AMR_WIND_ENABLE_CUDA: Enable NVIDIA CUDA GPU builds. Default: OFF

AMR_WIND_ENABLE_SYCL: Enable Intel OneAPI SyCL builds. Default: OFF

AMR_WIND_ENABLE_ROCM: Enable AMD ROCM GPU builds. Default: OFF

Dependencies

AMR_WIND_ENABLE_MASA: Enable MASA library. Default: OFF

AMR_WIND_ENABLE_NETCDF: Enable NetCDF outputs. Default: OFF

AMR_WIND_USE_INTERNAL_AMREX

Use a super-build with the AMReX tracked as a submodule. Default: ON

If set to OFF, then provide the the path to the existing AMReX installation either through AMREX_DIR variable or via CMAKE_PREFIX_PATH.

Other AMR-Wind specific options

AMR_WIND_ENABLE_DOCUMENTATION: Enable documentation. Default: OFF

AMR_WIND_ENABLE_TESTS: Enable CTest testing. Default: OFF

AMR_WIND_TEST_WITH_FCOMPARE: Enable checking test results against gold files using fcompare. Default: OFF

AMR_WIND_REFERENCE_GOLDS_DIRECTORY: Directory for reference gold files when using fcompare. Default: “”

AMR_WIND_SAVE_GOLDS: Enable saving of the gold files. Default: OFF

AMR_WIND_SAVED_GOLDS_DIRECTORY: Directory for saving gold files when using fcompare. Default: “”

AMR_WIND_TEST_WITH_PYTHON: Enable performing tests that rely on python, primarily consists of checks for post-processing scripts in tools directory. Default: OFF

AMR_WIND_ENABLE_ALL_WARNINGS: Enable compiler warnings during build. Default: OFF

General CMake options

CMAKE_INSTALL_PREFIX: The directory where the compiled executables and libraries as well as headers are installed. For example, passing -DCMAKE_INSTALL_PREFIX=${HOME}/software will install the executables in ${HOME}/software/bin when the user executes the make install command.

CMAKE_BUILD_TYPE

Controls the optimization levels for compilation. This variable can take the following values:

Value

Typical flags

RELEASE

-O2 -DNDEBUG

DEBUG

-g

RelWithDebInfo

-O2 -g

Example: -DCMAKE_BUILD_TYPE:STRING=RELEASE

CMAKE_CXX_COMPILER

Set the C++ compiler used for compiling the code.

For Intel SyCL builds (see AMR_WIND_ENABLE_SYCL) this should be set to icpx.

CMAKE_C_COMPILER: Set the C compiler used for compiling the code

CMAKE_Fortran_COMPILER: Set the Fortran compiler used for compiling the code

CMAKE_CXX_FLAGS: Additional flags to be passed to the C++ compiler during compilation.

CMAKE_C_FLAGS: Additional flags to be passed to the C compiler during compilation.

CMAKE_Fortran_FLAGS: Additional flags to be passed to the Fortran compiler during compilation.