Arm NN

Software Developer Kit (SDK)

Arm NN is an inference engine for CPUs, GPUs and NPUs. It bridges the gap between existing NN frameworks and the underlying IP. It enables efficient translation of existing neural network frameworks, such as TensorFlow and Caffe, allowing them to run efficiently, without modification, across Arm Cortex CPUs and Arm Mali GPUs.

Arm NN is free of charge.

About Arm NN SDK

Arm NN SDK is a set of open-source Linux software and tools that enables machine learning workloads on power-efficient devices. It provides a bridge between existing neural network frameworks and power-efficient Arm Cortex CPUs, Arm Mali GPUs or the Arm Machine Learning processor.

Arm NN SDK utilizes the Compute Library to target programmable cores, such as Cortex-A CPUs and Mali GPUs, as efficiently as possible. It includes support for the Arm Machine Learning processor. Arm NN does not yet provide support for Cortex-M CPUs.

The latest release supports Caffe, TensorFlow, TensorFlow Lite, and ONNX. Arm NN takes networks from these frameworks, translates them to the internal Arm NN format and then, through the Compute Library, deploys them efficiently on Cortex-A CPUs, and, if present, Mali GPUs such as the Mali-G71 and Mali-G72.

In September 2018, Arm donated Arm NN to the Linaro Machine Intelligence Initiative, where it is now developed fully in open source. 

Arm's Neural Network Framework Diagram.

 

Neural Network SDK Diagram.
Neural Network Diagram for Android.

Arm NN for Android

Also available is Arm NN for NNAPI, Google’s interface for accelerating neural networks on Android devices, made available in Android O. By default, NNAPI runs neural network workloads on the device’s CPU cores, but also provides a Hardware Abstraction Layer (HAL) that can target other processor types, such as GPUs. Arm NN for Android NNAPI provides this HAL for Mali GPUs. A further release adds support for the Arm Machine Learning processor. 

Arm support for Android NNAPI gives >4x performance boost.

Learn more

Download Arm NN for Android sources. 

Download here

Support for Cortex-M CPUs

Arm NN is currently not compatible with Cortex-M microcontrollers. CMSIS-NN is a collection of efficient neural network kernels developed to maximize the performance and minimize the memory footprint of neural networks on Cortex-M processor cores.

Download CMSIS-NN

Arm NN future roadmap

Future releases of Arm NN will enable support for other machine learning frameworks as inputs, and other forms of processor cores as targets. This includes processor cores and accelerators from Arm’s partners, assuming availability of suitable extensions.

Information on the Machine Learning Platform.

Webinar - Project Trillium: Optimizing ML Performance for any Application

Project Trillium is a suite of Arm IP designed to deliver scalable ML and neural network functionality at any point on the performance curve, from sensors, to mobile, and beyond. 

 

Find out more

Community Blogs

Community Forums

Not answered What is the "Integer divide unit with support for operand-dependent early termination"? 0 votes 28 views 0 replies Started 21 hours ago by jing Answer this
Answered Binary Semaphore upset by FIQ
  • Cortex-A
0 votes 808 views 20 replies Latest 3 days ago by 42Bastian Schick Answer this
Not answered Identifying Generic IP Components on an Access Port 0 votes 48 views 0 replies Started 3 days ago by Torsten Robitzki Answer this
Not answered Issue with WatchDog reset De-asserting 0 votes 54 views 0 replies Started 3 days ago by BAB Answer this
Not answered Getting processor and cache details
  • cache
  • Linux
0 votes 103 views 0 replies Started 5 days ago by karthikeyan.d Answer this
Not answered Loading cortex M1 soft processor on Pynq Processor
  • Cortex-M1
  • Cortex-M
0 votes 81 views 0 replies Started 5 days ago by Sivasankar Answer this
Not answered What is the "Integer divide unit with support for operand-dependent early termination"? Started 21 hours ago by jing 0 replies 28 views
Answered Binary Semaphore upset by FIQ Latest 3 days ago by 42Bastian Schick 20 replies 808 views
Not answered Identifying Generic IP Components on an Access Port Started 3 days ago by Torsten Robitzki 0 replies 48 views
Not answered Issue with WatchDog reset De-asserting Started 3 days ago by BAB 0 replies 54 views
Not answered Getting processor and cache details Started 5 days ago by karthikeyan.d 0 replies 103 views
Not answered Loading cortex M1 soft processor on Pynq Processor Started 5 days ago by Sivasankar 0 replies 81 views