Arm provides a simple 1-D wave equation solver that is useful as a profiling example program. Both C and Fortran variants are provided:
Both are built using the same makefile, wave.makefile. To navigate and run wave.makefile, use:
There is also a mixed-mode MPI+OpenMP variant in examples/wave_openmp.c, which is built with the openmp.makefile makefile.
Note: The makefiles for all supplied examples are located in the <INSTALL_DIR>/examples directory.
Depending on the default compiler on your system you may see some errors when running the makefile, for example:
By default, this example makefile is set up for the GNU compilers. To setup the makefile for a different compiler, open the examples/wave.makefile file, uncomment the appriopriate compilation command for the compiler you want to use, and comment those of the GNU compiler.
Note: The compilation commands for other popular compilers are already present within the makefile, separated by compiler.
Note: Although the example makefiles include the -g flag, Arm Performance Reports does not require this and you should not use them in your own makefiles.
On Cray X-series systems the example program must either be dynamically linked (using -dynamic) or explicitly linked with the Arm profiling libraries.
Example how to dynamically link:
Example how to explicitly link with the Arm profiling libraries: First create the libraries using the command make-profiler-libraries --platform=cray --lib-type=static:
Created the libraries in /home/user/examples:
To instrument a program, add these compiler options:
compilation for use with MAP - not required for Performance Reports:
-g (or -G2 for native Cray fortran) (and -O3 etc.)
linking (both MAP and Performance Reports):
-Wl,@/home/user/examplesm/allinea-profiler.ld ... EXISTING_MPI_LIBRARIES
If your link line specifies EXISTING_MPI_LIBRARIES (e.g. -lmpi), then
these must appear *after* the Arm sampler and MPI wrapper libraries in
the link line. There's a comprehensive description of the link ordering
requirements in the 'Preparing a Program for Profiling' section of either
userguide-forge.pdf or userguide-reports.pdf, located in
Then follow the instructions in the output to link the example program with the Arm profiling libraries:
cc -g -O3 wave.c -o wave -g -Wl,@allinea-profiler.ld -lm -lrt
ftn -G2 -O3 wave.f90 -o wave -G2 -Wl,@allinea-profiler.ld -lm -lrt
As this example uses MPI you need to run on a compute node on your cluster. Your site's help pages and support staff can tell you exactly how to do this on your machine. The simplest way when running small programs is often to request an interactive session, as follows:
$ qsub -I
qsub: waiting for job 31337 to start
qsub: job 31337 ready
$ cd arm/reports/examples
$ mpiexec -n 4 ./wave_c
Wave solution running with 4 processes
0: points = 1000000, running for 30 seconds
points / second: 63.9M (16.0M per process)
compute / communicate efficiency: 94% | 97% | 100%
Points for validation:
0:0.00 200000:0.95 400000:0.59 600000:-0.59 800000:-0.95 999999:0.00
Make sure the Arm Performance Reports module for your system has been loaded:
$ perf-report --version
Arm Performance Reports
Copyright (c) 2002-2018 Arm Limited (or its affiliates). All rights reserved.
If this command cannot be found consult the site documentation to find the name of the correct module.
Once the module is loaded, you can add the perf-report command in front of your existing mpiexec command-line:
If your program is submitted through a batch queuing system, then modify your submission script to load the Arm module and add the 'perf-report' line in front of the mpiexec command you want to generate a report for.
The program runs as usual, although startup and shutdown may take a few minutes longer while Arm Performance Reports generates and links the appropriate wrapper libraries before running and collects the data at the end of the run. The runtime of your code (between MPI_Init and MPI_Finalize should not be affected by more than a few percent at most.
After the run finishes, a performance report is saved to the current working directory, using a name based on the application executable:
$ ls -lrt wave_c*
-rwx------ 1 mark mark 403037 Nov 14 03:21 wave_c
-rw------- 1 mark mark 1911 Nov 14 03:28 wave_c_4p_2013-11-14_03-27.txt
-rw------- 1 mark mark 174308 Nov 14 03:28 wave_c_4p_2013-11-14_03-27.html
Note that both .txt and .html versions are automatically generated.