Installation of the Fortran-ML-Interface

March 28, 2025 · View on GitHub

Dependencies

Python Virtual Environment

Create a Python virtual environment with the following commands:

python3 -m venv venv-fortran-ml-py3113
source ./venv-fortran-ml-py3113/bin/activate

pip install --no-cache-dir --no-binary :all: mpi4py==3.1.6
pip install Cython
pip install tensorflow==2.17.0
pip install tf-keras==2.17

pip install matplotlib
pip install scipy

pip install scorep

Notes:

Disable cache and binary to make sure that mpi4y gets build with the correct MPI implementation from the module system.
newer version of mpi4py (4.0.1) has some binary incompatibility
Cython is needed to compile PhyDLL later on.
matplotlib is only used to visualize results of the testprograms

h5fortran

(https://github.com/geospace-code/h5fortran/blob/main/Install.md)

cd extern
git clone https://github.com/geospace-code/h5fortran.git
mkdir BUILD && cd BUILD
mkdir INSTALL
cmake .. -DCMAKE_INSTALL_PREFIX=${FORTRAN_ML_ROOT}/extern/h5fortran/BUILD/INSTALL
cmake --build . -j && cmake --install .

Note: h5fortran will automatically install its own HDF5 installation (v1.14.3) - no need to load HDF5 module on the cluster!

The AIxeleratorService relies on the C/C++ API of TensorFlow. Thus, it requires an TensorFlow installation outside Python providing the libtensorflow.so. For now we will download a precompiled version from the official TensorFlow releases.

TF_VERSION="2.17.0"
mkdir -p ./extern/tensorflow
cd ./extern/tensorflow
wget "https://storage.googleapis.com/tensorflow/versions/${TF_VERSION}/libtensorflow-gpu-linux-x86_64.tar.gz"
tar -xzvf libtensorflow-gpu-linux-x86_64.tar.gz
rm libtensorflow-gpu-linux-x86_64.tar.gz

Note: You may change the version of TensorFlow here. In this case make sure to also install the same version beforehand into your Python virtual environment and update all paths that refer to the TensorFlow installation and Python version below accordingly!

AIxeleratorService

cd extern/aixeleratorservice
mkdir BUILD && cd BUILD
cmake .. -DWITH_TORCH=OFF -DWITH_TENSORFLOW=ON -DTensorflow_DIR=${FORTRAN_ML_ROOT}/extern/tensorflow/ -DTensorflow_Python_DIR=${FORTRAN_ML_ROOT}/venv-fortran-ml-py3113/lib/python3.11/site-packages/tensorflow/

PhyDLL

make BUILD_DIR=${FORTRAN_ML_ROOT}/extern/phydll/BUILD ENABLE_PYTHON=ON ENABLE_FORTRAN=ON

LD_LIBRARY_PATH=${FORTRAN_ML_ROOT}/extern/phydll/BUILD/lib:$LD_LIBRARY_PATH PYTHONPATH=$PYTHONPATH:${FORTRAN_ML_ROOT}/extern/phydll/src/python make BUILD_DIR=${FORTRAN_ML_ROOT}/extern/phydll/BUILD ENABLE_PYTHON=ON ENABLE_FORTRAN=ON TEST_VERBOSE=ON install

Build ML Interface

mkdir BUILD && cd BUILD
cmake .. -Dh5fortran_ROOT=${FORTRAN_ML_ROOT}/extern/h5fortran/BUILD/INSTALL -DTensorflow_DIR=${FORTRAN_ML_ROOT}/extern/tensorflow/ -DTensorflow_Python_DIR=${FORTRAN_ML_ROOT}/venv-fortran-ml-py3113/lib/python3.11/site-packages/tensorflow/ -DWITH_AIX=ON -DWITH_PHYDLL=ON -DWITH_NCSA=ON
cmake --build . && cmake --install .

Build ML Interface with Score-P

The ML module can also be built with Score-P for profiling and tracing. First create the scorep-wrappers for mpiifort and mpiicpc if they are not provided by the Score-P module from the cluster.

mkdir scorep-wrapper
scorep-wrapper --create mpiifort ./scorep-wrapper
scorep-wrapper --create mpiicpc ./scorep-wrapper

Afterwards built the ML module using the scorep wrappers.

mkdir BUILD-SCOREP && cd BUILD-SCOREP
PATH=$PATH:${FORTRAN_ML_ROOT}/scorep-wrapper SCOREP_WRAPPER=off SCOREP_WRAPPER_INSTRUMENTER_FLAGS="--user --io=none --nomemory" SCOREP_WRAPPER_COMPILER_FLAGS="-g -DSCOREP" cmake .. -Dh5fortran_ROOT=${FORTRAN_ML_ROOT}/extern/h5fortran/BUILD/INSTALL -DTensorflow_DIR=${FORTRAN_ML_ROOT}/extern/tensorflow/ -DTensorflow_Python_DIR=${FORTRAN_ML_ROOT}/venv-fortran-ml-py3113/lib/python3.11/site-packages/tensorflow/ -DWITH_AIX=ON -DWITH_PHYDLL=ON -DWITH_NCSA=ON -DCMAKE_C_COMPILER=scorep-mpiicc -DCMAKE_CXX_COMPILER=scorep-mpiicpc -DCMAKE_Fortran_COMPILER=scorep-mpiifort
cmake --build . && cmake --install .