You copied the Doc URL to your clipboard.

7 Controlling program execution

Whether debugging a multi-process or a single process code, the mechanisms for controlling program execution are very similar.

In multi-process mode, most of the features described in this section are applied using Process Groups, which are described in the following sections.

For single process mode, the commands and behaviors are identical, but apply to only a single process, freeing the user from concerns about process groups.

7.1 Process control and process groups

MPI programs are designed to run as more than one process and can span many machines. Arm DDT allows you to group these processes so that actions can be performed on more than one process at a time. The status of processes can be seen at a glance by looking at the Process Group Viewer.

The Process Group Viewer is (by default) at the top of the screen with multi-colored rows. Each row relates to a group of processes and operations can be performed on the currently highlighted group (for example, playing, pausing and stepping) by clicking on the toolbar buttons. Switch between groups by clicking on them or their processes. The highlighted group is indicated by a lighter shade. Groups can be created, deleted, or modified by the user at any time, with the exception of the All group, which cannot be modified.

Groups are added by clicking on the Create Group button or from a context-sensitive menu that appears when you right-click on the process group widget. This menu can also be used to rename groups, delete individual processes from a group and jump to the current position of a process in the code viewer. You can load and save the current groups to a file, and you can create sub-groups from the processes currently playing, paused or finished. You can even create a sub-group excluding the members of another group. For example, to take the complement of the Workers group, select the All group and choose Copy, but without Workers.

You can also use the context menu to switch between the two different methods of viewing the list of groups in Arm DDT. These methods are the detailed view and the summary view.

7.1.1 Detailed view

The detailed view is ideal for working with smaller numbers of processes. If your program has 32 processes or less, Arm DDT defaults to the detailed view. You can switch to this view using the context menu if you wish.


Figure 34: The Detailed Process Group View

In the detailed view, each process is represented by a square containing its MPI rank (0 through n-1). The squares are color-coded; red for a paused process, green for a playing process and gray for a finished/dead process. Selected processes are highlighted with a lighter shade of their color and the current process also has a dashed border.

When a single process is selected the local variables are displayed in the Variable Viewer and displayed expressions are evaluated. You can make the Source Code Viewer jump to the file and line for the current stack frame (if available) by double-clicking on a process.

To copy processes from one group to another, simply click and drag the processes. To delete a process, press the delete key. When modifying groups it is useful to select more than one process by holding down one or more of the following:

Key Description
Control Click to add/remove process from selection
Shift Click to select a range of processes
Alt Click to select an area of processes


Some window managers (such as KDE) use Alt and drag to move a window. You must disable this feature in your window manager if you wish to use the Arm DDT's area select.

7.1.2 Summary view

The summary view is ideal for working with moderate to huge numbers of processes. If your program has 32 processes or more, Arm DDT defaults to this view. You can switch to this view using the context menu if you wish.


Figure 35: The Summary Process Group View

In the summary view, individual processes are not shown. Instead, for each group, Arm DDT shows:

  • The number of processes in the group.
  • The processes belonging that group. Here 1-2048 means processes 1 through 2048 inclusive, and 1-10, 12-1024 means processes 1-10 and processes 12-1024 (but not process 11). If this list becomes too long, it is truncated with a '…'. Hovering the mouse over the list shows more details.
  • The number of processes in each state (playing, paused or finished). Hovering the mouse over each state shows a list of the processes currently in that state.
  • The rank of the currently selected process. You can change the current process by clicking here, typing a new rank and pressing Enter. Only ranks belonging to the current group will be accepted.

The Show processes toggle button allows you to switch a single group into the detailed view and back again. This is useful if you are debugging a 2048 process program, but have narrowed the problem down to just 12 processes, which you have put in a group.

7.2 Focus control

The focus control allows you to focus on individual processes or threads as well as process groups. When focused on a particular process or thread, actions such as stepping, playing/pausing, adding breakpoints, will only apply to that process or thread rather than the entire group.

In addition, the Arm DDT GUI will change depending on whether you are focused on group, process or thread. This allows Arm DDT to display more relevant information about your currently focused object.


Figure 36: Focus options

7.2.1 Overview of changing focus

Focusing in Arm DDT affects a number of different controls in the Arm DDT main window. These are described here:


Focus controls do not affect Arm DDT windows such as the Multi-Dimensional Array Viewer, Memory Debugger, Cross-Process Comparison.

7.2.2 Process group viewer

The changes to the process group viewer amongst the most obvious changes to the Arm DDT GUI. When focus on current group is selected you see your currently created process groups. When switching to focus on current process or thread you see the view change to show the processes in the currently selected group, with their corresponding threads.


Figure 37: The Detailed Process Group View Focused on a Process

If there are 32 threads or more, Arm DDT defaults to showing the threads using a summary view (as in the Process Group View). The view mode can also be changed using the context menu.

During focus on process, a tooltip is shown that identifies the OpenMP thread ID of each thread, if the value exists.

7.2.3 Breakpoints

The breakpoints tab in Arm DDT is filtered to only display breakpoints relevant to your current group, process, thread. When focused on a process, the breakpoint tab displays which thread the breakpoint belongs to. If you are focused on a group, the tab displays both the process and the thread the breakpoint belongs to.

7.2.4 Code viewer

The code viewer in Arm DDT shows a stack back trace of where each thread is in the call stack. This is also filtered by the currently focused item, for example when focused on a particular process, you only see the back trace for the threads in that process.

Also, when adding breakpoints using the code viewer, they are added for the group, process or thread that is currently focused.

7.2.5 Parallel stack view

The parallel stack view can also be filtered by focusing on a particular process group, process or thread.

7.2.6 Playing and stepping

The behavior of playing, stepping and the Run to here feature are also affected by your currently focused item. When focused on a process group, the entire group is affected, whereas focusing on a thread means that only current thread is executed. The same goes for processes, but with an additional option which is explained below.

7.2.7 Step threads together

The step threads together feature in Arm DDT is only available when focused on process. If this option is enabled then Arm DDT attempts to synchronize the threads in the current process when performing actions such as stepping, pausing and using Run to here.

For example, if you have a process with two threads and you choose Run to here, Arm DDT pauses your program when either of the threads reaches the specified line. If Step threads together is selected Arm DDT attempts to play both of the threads to the specified line before pausing the program.


You should always use Step threads together and Run to here to enter or move within OpenMP parallel regions. With many compilers it is also advisable to use Step threads together when leaving a parallel region, otherwise threads can get 'left behind' inside system-specific locking libraries and may not enter the next parallel region on the first attempt.

7.2.8 Stepping threads window

When using the step threads together feature it is not always possible for all threads to synchronize at their target. There are two main reasons for this:

  1. One or more threads may branch into a different section of code (and hence never reach the target). This is especially common in OpenMP codes, where worker threads are created and remain in holding functions during sequential regions.
  2. As most of Arm DDT's supported debug interfaces cannot play arbitrary groups of threads together, Arm DDT simulates this behavior by playing each thread in turn. This is usually not a problem, but can be if, for example, thread 1 is playing, but waiting for thread 2 (which is not currently playing). Arm DDT attempts to resolve this automatically but cannot always do so.

If either of these conditions occur, the Stepping Threads Window appears, displaying the threads which have not yet reached their target.


Figure 38: The Stepping Threads Window

The stepping threads window also displays the status of threads, which may be one of the following:

  • Done: The thread has reached it target (and has been paused).
  • Skipped: The thread has been skipped and paused. Arm DDT no longer waits for it to reach its target.
  • Playing: This is the thread that is currently being executed. Only one thread may be playing at a time while the Stepping Threads Window is open.
  • Waiting: The thread is currently awaiting execution. When the currently playing thread is done or has been skipped, the highest waiting thread in the list is executed.

The Stepping Threads Window also lets you interact with the threads with the following options:

  • Skip: Arm DDT skips and pauses the currently playing thread. If this is the last waiting thread the window is closed.
  • Try Later: The currently playing thread is paused, and added to the bottom of the list of threads to be retried later. This is useful if you have threads which are waiting on each other.
  • Skip All: This skips, and pauses, all of the threads and close the window.

7.3 Starting, stopping and restarting a program

The File menu can be accessed at almost any time while Arm DDT is running. If a program is running you can end it and run it again or run another program. When Arm DDT's start up process is complete your program should automatically stop either at the main function for non-MPI codes, or at the MPI_Init function for MPI.

When a job has run to the end, Arm DDT displays a window box asking if you wish to restart the job. If you select yes then Arm DDT kills any remaining processes and clear up the temporary files and then restart the session from scratch with the same program settings.

When ending a job, Arm DDT attempts to ensure that all the processes are shut down and any temporary files are cleared up. If this fails for any reason you may have to manually kill your processes using kill, or a method provided by your MPI implementation such as lamclean for LAM/MPI.

7.4 Stepping through a program

To continue the program playing click Play/Continue PIC and to stop it at any time click Pause PIC .

For multi-process Arm DDT these start/stop all the processes in the current group (see Process Control and Process Groups).

Like many other debuggers there are three different types of step available. These are enumerated here:

  1. Step Into moves to the next line of source code unless there is a function call in which case it steps to the first line of that function.
  2. Step Over moves to the next line of source code in the bottom stack frame.
  3. Step Out executes the rest of the function and then stop on the next line in the stack frame above. The return value of the function is displayed in the Locals view. When using Step Out be careful not to try and step out of the main function, as doing this ends your program.

7.5 Stop messages

In certain circumstances your program may be automatically paused by the debugger. There are five reasons your program may be paused in this way:

  1. It hit one of Arm DDT's default breakpoints, for example, exit or abort. See section 7.10 Default breakpoints for more information on default breakpoints.
  2. It hit a user-defined breakpoint, that is a breakpoint shown in the Breakpoints view.
  3. The value of a watched variable changed.
  4. It was sent a signal. See section 7.20 Signal handling for more information on signals.
  5. It encountered a Memory Debugging error. See section 12.4 Pointer error detection and validity checking for more information on Memory Debugging errors.

Arm DDT displays a message telling you exactly why the program was paused. To copy the message text to the clipboard select it with the mouse cursor, then right-click and select Copy.

You may want to suppress these messages in certain circumstances, for example if you are playing from one breakpoint to another. Use the Control → Messages menu to enable or disable stop messages.

7.6 Setting breakpoints

7.6.1 Using the source code viewer

First locate the position in your code where you want to place a breakpoint. If you have numerous source code files and wish to search for a particular function you can use the Find/Find In Files window.

Right-clicking in the Source Code Viewer displays a menu showing several options, including one to add or remove a breakpoint.

In multi-process mode this sets the breakpoint for every member of the current group. Breakpoints may also be added by left-clicking the margin to the left of the line number.

Every breakpoint is listed under the breakpoints tab towards the bottom of Arm DDT's window.

If you add a breakpoint at a location where there is no executable code, Arm DDT highlights the line you selected as having a breakpoint. However, when hitting the breakpoint, Arm DDT stops at the next executable line of code.

7.6.2 Using the Add Breakpoint window

You can also add a breakpoint by clicking the Add Breakpoint PIC icon in the toolbar. This opens the Add Breakpoint window.


Figure 39: The Add Breakpoint window

You may wish to add a breakpoint in a function for which you do not have any source code: for example in malloc, exit, or printf from the standard system libraries. Select the Function radio button and enter the name of the function in the box next to it.

You can specify what group/process/thread you want the breakpoint to apply in the Applies To section. You may also make the breakpoint conditional by checking the Condition check box and entering a condition in the box.

7.6.3 Pending breakpoints


This feature is not supported on all platforms.

If you try to add a breakpoint on a function that is not defined, Arm DDT asks you if you want to add it anyway. If you click Yes the breakpoint is applied to any shared objects that are loaded in the future.

7.6.4 Conditional breakpoints


Figure 40: The Breakpoints Table

Select the breakpoints tab to view all the breakpoints in your program. You may add a condition to any of them by clicking on the condition cell in the breakpoint table and entering an expression that evaluates to true or false.

Each time a process (in the group the breakpoint is set for) passes this breakpoint it evaluates the condition and breaks only if it returns true (typically any non-zero value). You can drag an expression from the Evaluate window into the condition cell for the breakpoint and this is set as the condition automatically.


Figure 41: Conditional Breakpoints In Fortran

Conditions may be any valid expression for the language of the file containing the breakpoint. This includes other variables in your program and function calls.

You may want to avoid using functions with side effects as these will be executed every time the breakpoint is reached.

The expression evaluation may be more pedantic than your compiler. To ensure the correct interpretation of, for example, boolean operations, it is advisable to use brackets explicitly, to ensure correct evaluation.

7.7 Suspending breakpoints

To deactivate or reactivate a breakpoint, either:

  • Check or clear the activated column in the breakpoints panel.
  • Right-click the breakpoint icon in the code editor and choose Enable/Disable.
  • Hold SHIFT and select a breakpoint icon in the code editor.

Breakpoints that are disabled are grayed out.

7.8 Deleting a breakpoint

Breakpoints may be deleted by right-clicking on the breakpoint in the breakpoints panel.

They can also be deleted by right-clicking in the file/line of the breakpoint, while in the correct process group, and right-clicking and selecting delete breakpoint.

They may also be deleted by left-clicking the breakpoint icon in the margin, situated to the left of the line number in the code viewer.

7.9 Loading and saving breakpoints

To load or save the breakpoints in a session right-click in the breakpoint panel and select the load/save option. Breakpoints are also loaded and saved as part of the load/save session.

7.10 Default breakpoints

Arm DDT has a number of default breakpoints that stop your program under certain conditions which are described below. You may enable/disable these while your program is running using the Control → Default Breakpoints menu.

  • Stop at exit/_exit

    When enabled, Arm DDT pauses your program as it is about to end under normal exit conditions. Arm DDT pauses both before and after any exit handlers have been executed. (Disabled by default.)

  • Stop at abort/fatal MPI Error

    When enabled, Arm DDT pauses your program as it about to end after an error has been triggered. This includes MPI and non-MPI errors. (Enabled by default.)

  • Stop on throw (C++ exceptions)

    When enabled, Arm DDT pauses your program whenever an exception is thrown (regardless of whether or not it will be caught). Due to the nature of C++ exception handling, you may not be able to step your program properly at this point. Instead, you should play your program or use the Run to here feature in DDT. (Disabled by default.)

  • Stop on catch (C++ exceptions)

    As above, but triggered when your program catches a thrown exception. Again, you may have trouble stepping your program. (Disabled by default.)

  • Stop at fork

    Arm DDT stops whenever your program forks (that is, calls the fork system call to create a copy of the current process). The new process is added to your existing Arm DDT session and can be debugged along with the original process.

  • Stop at exec

    When your program calls the exec system call, Arm DDT stops at the main function (or program body for Fortran) of the new executable.

  • Stop on CUDA kernel launch

    When debugging CUDA GPU code, this pauses your program at the entry point of each kernel launch.

7.11 Synchronizing processes

If the processes in a process group are stopped at different points in the code and you wish to resynchronize them to a particular line of code this can be done by right-clicking on the line at which you wish to synchronize them to and selecting Run To Here. This effectively plays all the processes in the selected group and puts a break point at the line at which you choose to synchronize the processes at, ignoring any breakpoints that the processes may encounter before they have synchronized at the specified line.

If you choose to synchronize your code at a point where all processes do not reach then the processes that cannot get to this point will play to the end.


Though this ignores breakpoints while synchronizing the groups it will not actually remove the breakpoints.


If a process is already at the line which you choose to synchronize at, the process will still be set to play. Be sure that your process will revisit the line, or alternatively synchronize to the line immediately after the current line.

7.12 Setting a watchpoint


Figure 42: The Watchpoints Table

A watchpoint is a variable or expression that will be monitored by the debugger such that when it is changed or accessed the debugger pauses the application.


Figure 43: Program Stopped At Watchpoint being watched

Unlike breakpoints, watchpoints are not displayed in the Source Code Viewer. Instead they are created by right-clicking on the Watchpoints view and selecting the Add Watchpoint menu item.

It is also possible to add watchpoints automatically dragging a variable to the Watchpoints view from the Local Variables, Current Line and Evaluate views, or right-clicking over the variable in the Source Code Viewer and then selecting Add Watchpoint.

The automatic watchpoints are write-only by default.

Upon adding a watchpoint the Add Watchpoint dialog appears allowing you to apply restrictions to the watchpoint:

  • Process Group restricts the watch point to the chosen process group (see 7.1 Process control and process groups).
  • Process restricts the watchpoint to the chosen process.
  • Expression is the variable name in the program to be watched.
  • Language is the language of the portion of the program containing the expression.
  • Trigger On allows you to select whether the watchpoint will trigger when the expression is read, written or both.

You can set a watchpoint for either a single process, or every process in a process group.

Arm DDT automatically removes a watchpoint once the target variable goes out of scope. If you are watching the value pointed to by a variable, that is, *p, you may want to continue watching the value at that address even after p goes out of scope. You can do this by right-clicking on *p in the Watchpoints view and selecting the Pin to address menu item. This replaces the variable p with its address so the watch is not removed when p goes out of scope.

Modern processors have hardware support for a handful of watchpoints that are set to watch the contents of a memory location. Consequently, watchpoints can normally be used with no performance penalty.

Where the number of watchpoints used is over this quantity, or the expression being watched is too complex to tie to a fixed memory address, the implementation is through software monitoring, which imposes significant performance slowdown on the application being debugged.

The number of hardware watchpoints available depends on the system. The read watchpoints are only available as hardware watchpoints.

Consequently, watchpoints should, where possible, be a single value that is stored in a single memory location. While it is possible to watch the whole contents of non-trivial user defined structures or an entire array simultaneously, or complex statements involving multiple addresses, these can cause extreme application slow down during debugging.

7.13 Tracepoints

Tracepoints allow you to see what lines of code your program is executing, and the variables, without stopping it. Whenever a thread reaches a tracepoint it will print the file and line number of the tracepoint to the Input/Output view. You can also capture the value of any number of variables or expressions at that point.

Examples of situations in which this feature will prove invaluable include:

  • Recording entry values in a function that is called many times, but crashes only occasionally. Setting a tracepoint makes it easier to correlate the circumstances that cause a crash.
  • Recording entry to multiple functions in a library, enabling the user or library developer to check which functions are being called, and in which order. An example of this is the MPI History Plugin, which records MPI usage. See section 13.3 Using a plugin.
  • Observing progress of an application and variation of values across processes without having to interrupt the application.

7.13.1 Setting a tracepoint

Tracepoints are added by either right-clicking on a line in the Source Code Viewer and selecting the Add Tracepoint menu item, or by right-clicking in the Tracepoints view and selecting Add Tracepoint. If you right-click in the Source Code Viewer a number of variables based on the current line of code are captured by default.

Tracepoints can lead to considerable resource consumption by the user interface if placed in areas likely to generate a lot of passing. For example, if a tracepoint is placed inside of a loop with N iterations, then N separate tracepoint passings will be recorded.

While Arm DDT attempts to merge such data in a scalable manner, when alike tracepoints are passed in order between processes, where process behavior is likely to be divergent and unmergeable then a considerable load would result.

If it is necessary to place a tracepoint inside a loop, set a condition on the tracepoint to ensure you only log what is of use to you. Conditions may be any valid expression in the language of the file the tracepoint is placed in and may include function calls, although you may want to be careful to avoid functions with side effects as these will be evaluated every time the tracepoint is reached.

Tracepoints also momentarily stop processes at the tracepoint location in order to evaluate the expressions and record their values. This means if they are placed inside (for example) a loop with a very large number of iterations, or a function executed many times per second, then a slowdown in your application will be noticed.

7.13.2 Tracepoint output

The output from the tracepoints can be found in the Tracepoint Output view.


Figure 44: Output from Tracepoints in a Fortran application

Where tracepoints are passed by multiple processes within a short interval, the outputs will be merged. Sparklines of the values recorded are shown for numeric values, along with the range of values obtained, showing the variation across processes.

As alike tracepoints are merged then this can lose the order/causality between different processes in tracepoint output. For example, if process 0 passes a tracepoint at time T, and process 1 passes the tracepoint at T + 0.001, then this will be shown as one passing of both process 0 and process 1, with no ordering inferred.

Sequential consistency is preserved during merging, in that for any process, the sequence of tracepoints for that process will be in order.

To find particular values or interesting patterns, use the Only show lines containing box at the bottom of the panel. Tracepoint lines matching the text entered here will be shown, the rest will be hidden. To search for a particular value, for example, try "my_var: 34". In this case the space at the end helps distinguish between my_var: 34 and my_var: 345.

For more detailed analysis you may wish to export the tracepoints. To do this, right-click and choose Export from the pop-up menu. An HTML tracepoint log will be written using the same format as Arm DDT's offline mode.

7.14 Version control breakpoints and tracepoints

Version control breakpoint/tracepoint insertion allows you to quickly record the state of the parts of the target program that were last modified in a particular revision. The resulting tracepoint output may be viewed in the Tracepoint Output tab or the Logbook tab and may be exported or saved as part of a logbook or offline log.


Figure 45: DDT with version control tracepoints

Version control tracepoints may be inserted either in the graphical interactive mode or in offline mode via a command line argument.

In interactive mode enable Version Control Information from the View menu and wait for the annotation column to appear in the code editor. This does not appear for files that are not tracked by a supported version control system.


Figure 46: Version Control-Enable from Menu

Right-click a line last modified by the revision of interest and choose Trace Variables At This Revision.


Figure 47: Version Control-Trace at this revision

Arm DDT will find all the source files modified in the revision, detect the variables on the lines modified in the revision and insert tracepoints (pending if necessary). A progress dialog may be shown for lengthy tasks.

Both the tracepoints and the tracepoint output in the Tracepoints, Tracepoint Output, and Logbook tabs may be double-clicked during a session to jump to the corresponding line of source in the code viewer.

In offline mode supply the additional argument --trace-changes and Arm DDT applies the same process as in interactive mode using the current revision of the repository.

By default version control tracepoints are removed after 20 hits. To change this hit limit set the environment variable ALLINEA_VCS_TRACEPOINT_HIT_LIMIT to an integer greater than or equal to 0. To configure version control tracepoints to have no hit limit set this to 0.

See also Version control information .

7.15 Examining the stack frame


Figure 48: The Stack Tab

The stack back trace for the current process and thread are displayed under the Stack tab of the Variables Window. When you select a stack frame Arm DDT jumps to that position in the code, if it is available, and will display the local variables for that frame. The toolbar can also be used to step up or down the stack, or jump straight to the bottom-most frame.

7.16 Align stacks

The align stacks button, or CTRL-A hotkey, sets the stack of the current thread on every process in a group to the same level as the current process, where it is possible to do so.

This feature is particularly useful where processes are interrupted, by the pause button, and are at different stages of computation. This enables tools such as the Cross-Process Comparison window to compare equivalent local variables, and also simplifies casual browsing of values.

7.17 Viewing stacks in parallel

7.17.1 Overview

To find out where your program is, in one single view, you can use the Parallel Stack View. It is found in the bottom area of Arm DDT's GUI, tabbed alongside Input/Output, Breakpoints and Watches:


Figure 49: DDT Parallel Stack View

If you want to know where a group's processes are, click on the group and look at the Parallel Stack View. This shows a tree of functions, merged from every process in the group (by default). If there is only one branch in this tree, one list of functions, then all your processes are at the same place.

If there are several different branches, then your group has split up and is in different parts of the code. Click on any branch to see its location in the Source Code Viewer, or hover your mouse over it and a little popup will list the processes at that location. Right-click on any function in the list and select New Group to automatically gather the processes at that function together in a new group, labelled by the function's own name.

The best way to learn about the Parallel Stack View is to simply use it to explore your program. Click on it and see what happens. Create groups with it, and watch what happens to it as you step processes through your code. The Parallel Stack View's ability to display and select large numbers of processes based on their location in your code is invaluable when dealing with moderate to large numbers of processes.

7.17.2 The Parallel Stack View in detail

The Parallel Stack View takes over much of the work of the Stack display, but instead of just showing the current process, this view combines the call trees (commonly called stacks) from many processes and displays them together. The call tree of a process is the list of functions (strictly speaking frames or locations within a function) that lead to the current position in the source code.

For example, if main() calls read_input(), and read_input() calls open_file(), and you stop the program inside open_file(), then the call tree looks like the following:


If a function was compiled with debug information (usually -g) then Arm DDT adds extra information, displaying the exact source file and line number that your code is on.

Any functions without debug information are grayed-out and are not shown by default. Functions without debug information are typically library calls or memory allocation subroutines and are not generally of interest. To see the entire list of functions, right-click on one and choose Show Children from the pop-up menu.

You can click on any function to select it as the 'current' function in Arm DDT. If it was compiled with debug information, then Arm DDT also displays its source code in the main window, and its local variables and so on in the other windows.

One of the most important features of the Parallel Stack View is its ability to show the position of many processes at once. Right-click on the view to toggle between:

  1. Viewing all the processes in your program at once.
  2. Viewing all the processes in the current group at once (default).
  3. Viewing only the current process.

The function that Arm DDT is currently displaying and using for the variable views is highlighted in dark blue. Clicking on another function in the Parallel Stack View selects another frame for the source code and variable views. It also updates the Stack display, since these two controls are complementary. If the processes are at several different locations, then only the current process' location is displayed in dark blue. The other processes' locations are displayed in a light blue:


Figure 50: Current Frame Highlighting in Parallel Stack View

In the example above, the program's processes are at two different locations. 1 process is in the main function, at line 85 of hello.c. The other 15 processes are inside a function called func2, at line 34 of hello.c. The 15 processes reached func2 in the same way, main called func1 on line 123 of hello.c, then func1 called func2 on line 40 of hello.c. Clicking on any of these functions takes you to the appropriate line of source code, and displays any local variables in that stack frame.

There are two optional columns in the Parallel Stack View. The first, Processes shows the number of processes at each location. The second, Threads, shows the number of threads at each location. By default, only the number of processes is shown. Right-click to turn these columns on and off. Note that in a normal, single-threaded MPI application, each process has one thread and these two columns will show identical information.

Hovering the mouse over any function in the Parallel Stack View displays the full path of the filename, and a list of the process ranks that are at that location in the code:


Figure 51: Parallel Stack View tool tip

Arm DDT is at its most intuitive when each process group is a collection of processes doing a similar task. The Parallel Stack View is invaluable in creating and managing these groups.

Right-click on any function in the combined call tree and choose the New Group option. This creates a new process group that contains only the processes sharing that location in code. By default Arm DDT uses the name of the function for the group, or the name of the function with the file and line number if it is necessary to distinguish the group further.

7.18 Browsing source code

Source code is automatically displayed when a process is stopped, when you select a process, or position in the stack changed. If the source file cannot be found you are prompted for its location.

Arm DDT highlights lines of the source code to show the current location of your program's execution. Lines that contain processes from the current group are shaded in that group's color. Lines only containing processes from other groups are shaded in gray.

This pattern is repeated in the focus on process and thread modes. For example, when you focus on a process, Arm DDT highlights lines containing that process in the group color, and other processes from that group in gray.

Arm DDT also highlights lines of code that are on the stack, functions that your program will return to when it has finished executing the current one. These are drawn with a faded look to distinguish them from the currently-executing lines.

You can hover the mouse over any highlighted line to see which processes/threads are currently on that line. This information is presented in a variety of ways, depending on the current focus setting:

Focus on Group

A list of groups that are on the selected line, along with the processes in them on this line, and a list of threads from the current process on the selected line.

Focus on Process

A list of the processes from the current group that are on this line, along with the threads from the current process on the selected line.

Focus on Thread

A list of threads from the current process on the selected line.

The tool tip distinguishes between processes and threads that are currently executing that line, and ones that are on the stack by grouping them under the headings On the stack and On this line.

Variables and Functions

Right-clicking on a variable or function name in the Source Code Viewer causes Arm DDT to check whether there is a matching variable or function, and then to display extra information and options in a sub-menu.

In the case of a variable, the type and value are displayed, along with options to view the variable in the Cross-Process Comparison Window (CPC) or the Multi-Dimensional Array Viewer (MDA), or to drop the variable into the Evaluate Window, each of which are described in the next chapter.


Figure 52: Right-Click Menu-Variable Options

In the case of a function, it is also possible to add a breakpoint in the function, or to the source code of the function when available.


Figure 53: Right-Click Menu-Function Options

7.19 Simultaneously viewing multiple files

Arm DDT presents a tabbed pane view of source files. Occasionally it may be useful to view two files simultaneously, such as when tracking two different processes.

Inside the code viewing panel, right-click to split the view. This displays a second tabbed pane which can be viewed beneath the first one. When viewing additional files, the currently 'active' panel displays the file. Click on one of the views to make it active.

The split view can be reset to a single view by right-clicking in the code panel and deselecting the split view option.


Figure 54: Horizontal Alignment Of Multiple Source Files

7.20 Signal handling

By default Arm DDT will stop a process if it encounters one of the standard signals. See section 7.20.1 Custom signal handling (signal dispositions). The standard signals include:

  • SIGSEGV - Segmentation fault

    The process has attempted to access memory that is not valid for that process. Often this will be caused by reading beyond the bounds of an array, or from a pointer that has not been allocated yet. The DDT Memory Debugging feature may help to resolve this problem.

  • SIGFPE - Floating Point Exception

    This is raised typically for integer division by zero, or dividing the most negative number by -1. Whether or not this occurs is Operating System dependent, and not part of the POSIX standard. Linux platforms will raise this.

    Note that floating point division by zero will not necessarily cause this exception to be raised, behavior is compiler dependent. The special value Inf or -Inf may be generated for the data, and the process would not be stopped.

  • SIGPIPE - Broken Pipe

    A broken pipe has been detected while writing.

  • SIGILL - Illegal Instruction

SIGUSR1, SIGUSR2, SIGCHLD, SIG63 and SIG64 are passed directly through to the user process without being intercepted by DDT.

7.20.1 Custom signal handling (signal dispositions)

You can change the way individual signals are handled using the Signal Handling window. To open the window select the Control → Signal Handling… menu item.


Figure 55: Signal Handling dialog

Set a signal's action to Stop to stop a process whenever it encounters the given signal, or Ignore to let the process receive the signal and continue playing without being stopped by the debugger.

7.20.2 Sending signals

The Send Signal window allows a signal to be sent to the debugged processes. Select the Control → Send Signal… menu item. Select the signal you want to send from the drop-down list and click the Send to process button.