Open MPI

How to build Open MPI with Arm Compiler for HPC.

Introduction

This topic describes how to build Open MPI with Arm Compiler for HPC.

Open MPI is freely available. The Open MPI project is an open source MPI implementation developed and maintained by a consortium of academics, researchers and industry partners.

For more information on Open MPI, visit the Open MPI webpage.

Version information

For the purposes of this build, the following components are used:

Component
 Form
 Open MPI  Version 3.1.2
 Operating System
 RHEL 7.5
 Hardware  Cavium ThunderX2
 Arm Compiler for HPC
 Version 19.0

Download

To download the source code for Open MPI v3.1.2, visit the Open MPI website.

Download the openmpi-3.1.2.tar.bz2 source code tarball and unpack it.

Build and Install Open MPI with Arm Compiler for HPC

To build and install Open MPI with Arm Compiler for HPC, follow these steps:

  1. Ensure that your paths are set up appropriately for Arm Compiler for HPC, for example:

    export PATH=/proj/ArmCompiler/bin:$PATH
    export LD_LIBRARY_PATH=/proj/ArmCompiler/lib:$LD_LIBRARY_PATH
  2. Within the openmpi-3.1.2 directory, create a new directory called build and change into it.

    Keeping builds in separate directories is optional when building open source software, but is considered best practice. Having multiple build directories side-by-side is often useful for testing and debugging.

  3. Run the Open MPI v3.1.2 configure script:

    CC=armclang CXX=armclang++ FC=armflang ../configure --prefix=/var/tmp/openmpi_ArmCompiler --enable-mpirun-prefix-by-default

    The --enable-mpirun-prefix-by-default flag prefixes PATH and LD_LIBRARY_PATH on both local and remote hosts, ensuring the local environment is copied to remote nodes, where applicable.

    Note: /var/tmp/openmpi_ArmCompiler is the directory for the build of Open MPI to be installed into and may be replaced with a directory of choice.

  4. Next, invoke make to build Open MPI with armflang, armclang, and armclang++ for Fortran, C, and C++, respectively.

    Build speed can be improved by parallelizing using the -j flag for make. Common practice is to use 2N+1 parallel jobs, where N=number of threads (or cores) on your system. For example, if the combined total of CPUs on your system is 64 threads, passing -j129 will enable a fast build:

    make -j129 V=1 all

    To show the command lines passed to the compiler for each object file being compiled or linked, the optional V=1 flag is used.

  5. Once built, install the build using:

    make install
  6. Add the --prefix-configured /var/tmp/openmpi_ArmCompiler directory to your $PATH

    export PATH=/var/tmp/openmpi_ArmCompiler/bin:$PATH

    This directory locates where Open MPI was installed.

    The commands mpifort, mpicc and mpicxx are now ready to be used for compiling Fortran, C and C++ MPI code, respectively. The mpiexec command can be used to execute a compiled MPI program.

    Now you are ready to test MPI programs using Arm Compiler for HPC.

Run an example Fortran program

The mpihello.f90 program is a 'Hello World' example Fortran MPI program:

! mpihello.f90
program mpihello   use mpi   implicit none
   integer :: err, rank, size
  call MPI_Init(ierror = err)
  call MPI_Comm_rank(comm = MPI_COMM_WORLD, rank = rank, ierror = err)
  call MPI_Comm_size(comm = MPI_COMM_WORLD, size = size, ierror = err)
  write (*, 10), rank, size
  call MPI_Finalize(ierror = err)
  10 format(' Hello World, rank: ', I5, ' size: ', I5)
end program mpihello

The program comprises four common Open MPI functions:

  • MPI_Init
  • MPI_Comm_Rank
  • MPI_Comm_Size
  • MPI_Finalize

To run this program with the library build:

  1. Compile the mpihello.f90 program:

    mpifort -O3 -o mpihello mpihello.f90
  2. Execute the mpihello program:

    mpiexec -np 4 ./mpihello

    to return:

     Hello World, rank:     0 size:     4
     Hello World, rank:     2 size:     4
     Hello World, rank:     1 size:     4
     Hello World, rank:     3 size:     4

    Note: The ordering of the output may change due to parallelism.

Run an example C/C++ program

The mpihello.c program is a 'Hello World' example C MPI program:

// mpihello.c
#include <stdio.h>
#include <stddef.h>
#include <mpi.h>

int main()
{
int rank, size;

MPI_Init(NULL, NULL);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
printf("Hello World, rank: %5d size: %5d\n", rank, size);
MPI_Finalize();
return 0;
}

The program comprises four common Open MPI functions:

  • MPI_Init
  • MPI_Comm_Rank
  • MPI_Comm_Size
  • MPI_Finalize

To run this program with the library build:

  1. Compile the mpihello.c program:

    mpicc -O3 -o mpihello mpihello.c
    
  2. Execute the mpihello program:

    mpiexec -np 4 ./mpihello

    to return:

     Hello World, rank:     0 size:     4
     Hello World, rank:     2 size:     4
     Hello World, rank:     1 size:     4
     Hello World, rank:     3 size:     4

    Note: The ordering of the output may change due to parallelism.

Related information