You copied the Doc URL to your clipboard.

Limitations of parallel processing and Amdahl's law

There are limitations of parallel processing that you must consider when developing parallel applications.

For example, if your application parallelizes perfectly, executing the application on ten processors makes it run ten times faster. However, applications rarely parallelize perfectly because part of the application is serial. This serial component imposes a limit on the amount of parallelization the application can use.

Amdahl's law describes the maximum speedup that parallel processing can achieve.

The formula for Amdahl's law is shown in the following figure where the terms in the equation are:

SFraction of the application that is serial.
PFraction of the application that is parallelizable.
NNumber of processors.

Figure 2-4 Formula for Amdahl's law

The following figure shows the speedup that different numbers of processors provide for applications with different serial components.

Figure 2-5 Speedup for an application with different serial components

The biggest speedups are achieved with relatively small numbers of processors. However, as the number of processors rises, the per-processor gains are reduced.

You cannot avoid Amdahl's law in your application but you can reduce the impact.

For high performance with a large number of processors, the application must have a very small serial component. These sorts of applications are said to be embarrassingly parallel.