How to build MVAPICH with Arm Compiler

Overview Before you begin Procedure Related information


  1. Download and unpack the application source code:

    tar ‑zxvf mvapich2‑2.3.1.tar.gz
  2. Change into the unpacked directory:

    cd mvapich2-2.3.1
  3. Set an install location, INSTALL_DIR, where MVAPICH will be installed to. For example:

    export INSTALL_DIR=/path/to/MVAPICH_install

    Replace /path/to/MVAPICH_install with the path to your installation.

  4. Set the compilers to use for the build:

    export CC=armclang
    export CXX=armclang++
    export FC=armflang
  5. Create a build directory and change into it:

    mkdir build
    cd build
  6. Run configure from the build directory, specifying the devices and protocols to build for, and the language interfaces to include: 

    Note: The default on Linux is OpenFabrics (OFA) IB/iWARP/RoCE with the CH3 channel. Explicitly select it with: --with-device=ch3:mrail --with-rdma=gen2.

    ../configure --prefix=$INSTALL_DIR --with-device=ch3:mrail --with-rdma=gen2 --enable-cxx --enable-fc

    Note: For more configuration options, see the MVAPICH user guide.

  7. Build, test, and install MVAPICH, using:

    make -j
    make install
    make testing
  8. By default, MVAPICH enables CPU affinity. However, in the case of multi‑threaded programs, this can lead to poor thread placement and poor performance. It is recommended to disable CPU affinity in this case:

    export MV2_ENABLE_AFFINITY=0

    You must also disable CPU affinity if you want to over-subscribe the available cores with MPI tasks, for testing and development purposes.

    MVAPICH supports the explicit control of the CPU and thread placement and binding policy via MV2_CPU_BINDING_LEVEL, MV2_CPU_BINDING_POLICY, MV2_HYBRID_BINDING_POLICY and MV2_THREADS_PER_PROCESS.

    mpirun provides details on the mapping between CPUs and processes if MV2_SHOW_CPU_BINDING is set.

    For more information, see the MVAPICH User Guide.

Previous Next