HDF5

How to build HDF5 with Arm Compiler for HPC.

Hierarchical Data Format (HDF) is a set of file formats that are designed to store and organize large and complex data sets together with API's for Fortran, C, and C++. For more information on HDF, see the HDF website.

For the purposes of this build, the following components are used:

 Component Form
 HDF5  Version 1.10.4
 Arm Compiler for HPC
 Version 19.1
 Open MPI  Version 4.0.0
 Operating system  RHEL 7.5
 Hardware
 Cavium ThunderX2

Before you begin


Procedure

  1. Create build and install locations, and set them using the environment variables build_dir and install_dir, for example:

    mkdir hdf5_build
    mkdir hdf5_install
    export build_dir=/path/to/hdf5_build
    export install_dir=/path/to/hdf_install

    Replace /path/to/ with the path to the new directories.

  2. Download and unpack HDF5:

    cd $build_dir
    wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.4/src/hdf5-1.10.4.tar.gz
    tar -zxvf hdf5-1.10.4.tar.gz
    cd hdf5-1.10.4
  3. To build parallel IO and the Fortran interface, run ./configure with the following arguments:

    ./configure CC=mpicc FC=mpif90 CXX=mpicxx --enable-parallel --enable-fortran --with-pic --prefix=$install_dir
  4. The configure script does not detect armflang. To correct this omission, the libtool script that is generated by the configure script, requires some post-configuration patching as follows:

    sed -i -e 's#wl=""#wl="-Wl,"#g' libtool
    sed -i -e 's#pic_flag=""#pic_flag=" -fPIC -DPIC"#g' libtool
  5. Build and install HDF5, using:

    make -j
    make install
    

Resolving issues with deprecated MPI features 

This section shows how to resolve issues with deprecated calls when building with Open MPI 4.0.0.

HDF5 uses the following deprecated MPI calls and data types which were replaced in MPI2 onwards, and are not supported in the 4.0.0 release of Open MPI: 

Deprecated Replacement
MPI_NULL_COPY_FN MPI_COMM_NULL_COPY_FN
MPI_Address MPI_Get_address
MPI_Type_struct MPI_Type_create_struct
MPI_Type_extent MPI_Type_get_extent
Note:  MPI_Type_get_extent has an extra output parameter lb, for the lower bound of the datatype

All occurrences of these deprecated elements can be replaced with the correct MPI2 form using sed:

sed -i -e 's/MPI_NULL_COPY_FN/MPI_COMM_NULL_COPY_FN/g' ./src/H5.c
sed -i -e 's/MPI_Address/MPI_Get_address/g' ./testpar/t_cache.c
sed -i -e 's/MPI_Type_struct/MPI_Type_create_struct/g' ./testpar/t_cache.c
sed -i -e 's/MPI_Type_extent (inner_type, \&inner_extent)/MPI_Type_get_extent (inner_type, \&lb, \&inner_extent)/g' ./src/H5Smpio.c
sed -i -e 's/MPI_Type_extent (old_type, \&old_extent)/MPI_Type_get_extent (old_type, \&lb, \&old_extent)/g' ./src/H5Smpio.c
sed -i -e 's/disp\[2\], old_extent/disp\[2\], old_extent, lb/g' ./src/H5Smpio.c

For details about the features that are removed in Open MPI 4.0.0, see Removed MPI constructs on the Open MPI website.

You can re-enable the support for these features when building Open MPI. For more information, see Building Open MPI with Arm Compiler.

Related information