What is an Arm NN backend?

The Arm NN backend is an abstraction that maps the layers of a network graph to the hardware that is responsible for executing those layers. Arm NN provides ready-made backends to allow workloads to run on Cortex-A CPUs, Mali GPUs, and Arm ML processors. Arm NN also provides an interface so that you can write your own custom backends to interface with third-party devices.

Backends support one or more layers from the network graph, creating backend-specific workloads for the layers that they support, and then executing those workloads.

Each backend identifies the layers that it can process. The Arm NN then divides the original graph into several subgraphs to be assigned to the different backends. For example, in the following diagram, Arm NN divides the graph into three subgraphs. Arm NN does this by selecting the largest contiguous set of layers that can be processed by a single backend.

What is an Arm NN backend layers diagram

Arm NN subgraph and layers

When we look at this diagram, we can see that:

  • Layers 1 and 2 can be executed by the same backend.
  • Layers 4 and 5 can be executed by the same backend. This may be the same backend as Layers 1 and 2, or it may be a different backend.
  • Layer 3 requires a different backend from all the other layers.

All backends must:

  • implement the IBackendInternal interface
  • identify themselves with a string that must be unique across all of the backends
  • register themselves with BackendRegistry, so that Arm NN knows about them
  • implement the ILayerSupport interface for the layers the backend intends to support
  • implement the IWorkloadFactory interface, so that Arm NN can execute layers on the backend

You can learn more about backends in Write your own Arm NN backend plugin.

Previous Next