Bringing ExecuTorch to Arm Cortex-M CPUs

Authors

Meta: RJ Ascani, Siddartha Pothapragada, Digant Desai

Arm: Adrian Lundell, Zingo Andersen, Måns Nilsson, Erik Lundell

---

Background

Microcontrollers power billions of always-on devices: wearables, industrial sensors, voice interfaces, and smart home appliances. These devices operate under extreme constraints: often less than 1 MB of SRAM, single-digit megabytes of flash, and power budgets measured in milliwatts. Despite these limitations, there is growing demand to run meaningful ML workloads on these platforms and the hardware is evolving to meet it. Modern Cortex-M based microcontroller units (MCUs) now include DSP & vector ISAs, large tightly-coupled memories (TCMs), and power-efficient NPUs such as the Arm Ethos-U NPU that make ML inference more practical in resource constrained devices.

Until now, deploying ML on Cortex-M meant leaving the PyTorch ecosystem. Developers had to convert models to alternative formats, maintain separate toolchains for training and deployment, and debug numerical mismatches introduced during conversion. For teams that train in PyTorch, which accounts for over 90% of the research code published at top ML conferences, this translation step adds friction to every iteration cycle.

ExecuTorch was designed to eliminate this gap. Arm MCU support has been available through the Ethos-U NPU, and the Cortex-M backend extends it to Cortex-M CPUs, so a model can run on parts with or without an accelerator. The trained PyTorch model is exported, quantized and lowered for the device within the same stack that serves models on mobile, desktop and server.

The Cortex-M backend

Figure 1: ExecuTorch model compilation and deployment workflow for Arm Cortex-M CPUs.

The ExecuTorch Cortex-M backend replaces standard PyTorch operators with CMSIS-NN-backed custom operators through ahead-of-time compiler passes. At inference time, these operators call directly into Arm's CMSIS-NN library, an open-source collection of optimized neural network kernels for Cortex-M processors.

Key properties:

• CMSIS-NN accelerated. Quantized operations (linear, conv, add, mul, softmax, activations, etc.) are replaced with optimized implementations that call CMSIS-NN functions and additional optimized Cortex-M implementations directly.
• Target-aware compilation. The CPU variant (M0 through M85) and ISA tier are specified at compile time, so the correct CMSIS-NN backend is selected, eliminating runtime dispatch overhead.
• Purpose-built quantizer. Produces INT8 quantized graphs matching the data layout requirements of CMSIS-NN kernels, using PyTorch's PT2E quantization framework.
• Single compiler toolchain. The Cortex-M lowering path operates directly on the ExecuTorch edge dialect, keeping the compilation pipeline simple and debuggable.

Cortex-M capability tiers

The backend targets the full Cortex-M spectrum:

Tier	Cores	Key Feature	Use Case
Scalar	M0, M0+, M3, M23	Standard ARM instructions	Ultra-low-cost, minimal silicon
DSP	M4, M7, M33, M35P	SIMD multiply-accumulate	Mid-range MCUs, signal processing
MVE (aka Helium)	M55, M85	128-bit vector ops, beat-based execution	High-performance MCU ML inference

 

Arm Helium technology is where the biggest performance gains are. CMSIS-NN leverages Helium's beat-based execution model to pipeline data movement and compute, providing significant throughput gains for the depthwise and pointwise convolutions common in edge ML architectures. That said, the same workflow targets all variants. The target configuration selects the appropriate backend automatically.

ExecuTorch is embedded friendly

Unlike mobile or cloud environments that must accommodate varied hardware at runtime, microcontroller deployments target a fixed hardware profile. Embedded developers possess precise knowledge of their deployment target, including the exact CPU core, available ISA extensions, SRAM allocation, and flash capacity.

ExecuTorch's ahead-of-time stack is designed to exploit this. ExecuTorch makes target-specific decisions at compile time: graph-level optimizations, memory planning, weight packing, and kernel selection are all target-aware at compile time.

Concretely, for Cortex-M this means:

• Graph-level compute elimination. Redundant dtype conversions, quantize/dequantize pairs, and memory format transposes are removed during lowering, so only necessary compute reaches the device.
• Quantization parameter folding. Scale and zero-point arithmetic is precomputed into integer multiplier/shift form at export, so quantized kernel inference stays entirely in fixed-point.
• AOT weight layout transposition. Convolution weights are transposed from PyTorch's native layout (OIHW) to CMSIS-NN's expected format (OHWI) at export time. The permute cost is paid once during compilation and never at inference.
• Static memory planning across heterogeneous memories. All tensor and scratch buffers are allocated and laid out ahead of time, with support for spanning multiple memory regions (e.g., TCM and SRAM). Developers can also customize placement to put latency-critical buffers in fast local memory. No heap allocator, no fragmentation, and no runtime memory management required.
• Target-specific operator replacement. The CortexMPassManager selects CMSIS-NN kernel variants matched to your core’s ISA tier (Scalar, DSP, or MVE) at compile time, not at runtime.

Developer workflow

The resulting .pte binary is linked into firmware alongside the ExecuTorch runtime and CMSIS-NN library, then flashed to the device. At runtime, inference executes the precompiled graph with minimal runtime overhead and no dynamic memory allocation for tensors.

Using the Cortex-M backend alongside Ethos-U

Many Cortex-M MCUs pair the CPU with an Arm Ethos-U NPU such as the U55, U65, or U85. ExecuTorch's EthosUPartitioner delegates supported operations to the NPU via the Vela compiler, and operations left outside the partition run on the Cortex-M CPU within the same graph.

The Cortex-M passes run after Ethos-U partitioning and reuse the quantization parameters already in the graph. This allows a model quantized with EthosUQuantizer to use CMSIS-NN for CPU operations whose quantization, layout and shape constraints meet the Cortex-M backend requirements. Otherwise they fall back to ExecuTorch's portable kernels, which are functional but not accelerated.

The practical benefit is that NPU operator coverage is not an all-or-nothing gate. A model with a handful of undelegated operations can ship today on mixed CPU/NPU execution, and move more of the graph onto the NPU as coverage grows.

MLPerf Tiny Benchmark Suite

We measured inference latency and memory usage across four of the five MLPerf Tiny Inference benchmark models on the Alif Ensemble E8. These benchmark models represent a set of common deep neural networks representative of TinyML workloads. The Alif E8 integrates multiple Cortex-M55 cores alongside Ethos-U55 and U85 NPUs; we measured on a single M55 high performance core running at 400 MHz, with the Ethos-U NPUs unused. ExecuTorch's ahead-of-time memory planning means RAM usage is deterministic and bounded, with no fragmentation risk under sustained workloads. ExecuTorch’s operator selective build was used to only include those necessary for inference, creating a minimal flash footprint. All models were INT8 quantized using the CortexMQuantizer and the Alif ML Embedded Evaluation Kit was used as a test harness. These are our own measurements rather than an official MLPerf Tiny submission, so they are not directly comparable to published results on the MLCommons leaderboard.

All four models run in under 90 KiB of RAM, three of them in under 60 KiB, and the anomaly detection autoencoder in 5.2 KiB. Code footprint stays between 15 and 37 KiB because operator selective build links only the operators each model uses.

Model	Task	Latency (ms)	Inferences per second	Peak RAM (KiB)	Flash (KiB)
					Model	Code *	Total
DS-CNN	Keyword Spotting	5.825	172	26.00	37.67	36.03	73.70
MobileNet v1 0.25	Visual Wake Words	21.450	47	85.15	253.07	36.03	289.10
ResNet-8	Image Classification	15.290	65	55.14	89.77	29.82	119.59
Autoencoder	Anomaly Detection	1.208	828	5.20	272.03	14.97	287.00

*Code compiled with GCC 15.2.1 includes ExecuTorch core runtime (-Oz) and kernels (-O3) with logging disabled.

Comparison with other frameworks

For comparison, the MLPerf Tiny benchmark suite includes reference .tflite model files. We benchmarked these models with TensorFlow Lite for Microcontroller (TFLM) using the same Alif E8 M55. The models derive from trained reference weights, with differences in quantization and calibration between frameworks. The Alif ML Embedded Evaluation Kit was re-used as a test harness with the latest version of tflite-micro. The same GCC 15.2.1, compiler options and CMSIS-NN 8.0.0 release were used. The operator selection was also minimized using custom registration.

Figure 2: ExecuTorch and TensorFlow Lite for Microcontrollers performance comparison on the Alif Ensemble E8

In our measurements, ExecuTorch achieves 2.3% to 10.6% lower inference latency on three of the four models. Autoencoder latency is nearly identical, with ExecuTorch taking 0.27% longer. Memory is where the two frameworks diverge more sharply. The measured model/operator/core footprint is up to 21.1% less flash across all four models, and less RAM on three of the four, including 46.1% less on the Autoencoder. ResNet-8 is the exception, using 1.2% more RAM than TFLM.

Demo: Real-time person detector

Figure 3 Real-time person detection using ExecuTorch on a Renesas EK-RA8D1 board with an Arm Cortex-M85 CPU

To show what the Cortex-M backend enables in practice, we deployed a µYOLO person detection model on a Renesas EK-RA8D1 board, powered by the Arm Cortex-M85. The deployment fits comfortably within MCU-class memory constraints:

• Task: Real-time person detection
• Model: µYOLO, pruned depthwise separable CNN backbone with linear predictor head
• Model size: 452 KiB (INT8 .pte file)
• Total flash: 640KiB
• Peak RAM: 254 KiB
• Inference latency: 24 ms per frame inference @ 480MHz
• End-to-end throughput:7 FPS
• Input: 128×128 RGB camera frame

 

This example of resource-constrained, always-on vision on the edge supports applications such as robotics sensing, occupancy monitoring, safety systems, smart cameras, and presence-triggered devices.

Just as importantly, the example demonstrates the development workflow enabled by ExecuTorch. The model definition, training code, export flow, and embedded deployment live within the same PyTorch-based ecosystem. Intermediate representations remain executable, making it possible to compare model outputs throughout export and lowering rather than treating deployment as a black box. The resulting graphs can also be inspected with tools such as Model Explorer, while ETDump exposes per-operator execution costs for profiling on the target. Together, these tools make it possible to iterate on model architecture, accuracy, memory use, and latency while continuously checking that transformations have preserved model behaviour. Instead of exporting a model and only discovering numerical or performance regressions after deployment, developers can inspect and execute intermediate stages of the transformation pipeline.

The model definition, training and export flow are available in the ExecuTorch repository. To facilitate an easy bring-up of the demo without the need for a physical target, the in-tree application is targeted for the Corstone FVP. Board-specific bring-up for the on-device demo was based on code from the ra-fsp-examples repo, configured with the RA Smart Configurator, built using the Arm GNU Toolchain, and flashed using J-Link Flash. For measurements above, the weights of the most expensive operators were placed in DTCM, and the instruction cache was enabled. This resulted in an inference latency reduction from 191 ms to 24 ms, a nearly 8x improvement.

Getting started

The Cortex-M backend is available today in the ExecuTorch repository:

Resource	Description
Cortex-M Backend Documentation	Supported operators, target configuration, and build setup
Arm Backend Examples	Full example directory with setup scripts
CMSIS-ExecuTorch Project Template	Docker build + Keil Studio + AVH simulation
Zephyr RTOS Integration	ExecuTorch on Zephyr with CMSIS Toolbox
Raspberry Pi Pico 2 Example	Cortex-M33, MNIST digit classification

Platforms

Platform	CPU	Available via
Arm Corstone-300 FVP	Cortex-M55 + Ethos-U55	In-tree examples, exercised in CI
Arm Corstone-320 FVP	Cortex-M85 + Ethos-U85	In-tree examples, exercised in CI
Alif Ensemble E8	Cortex-M55 (MVE)	Alif ML Embedded Evaluation Kit
Raspberry Pi Pico 2	Cortex-M33 (DSP)	In-tree example
Renesas EK-RA8D1	Cortex-M85	Out-of-tree board port; in-tree demo runs on Corstone FVP

What is next

• Expanded mixed-precision support. Extending A8W4 and A16W8 mixed-precision support across the full operator set.
• Floating-point kernels. CMSIS-NN 8.0.0 introduced fp32 and fp16 MVE kernels. Exposing them via the Cortex-M backend enables non-fully quantized models to use the same lowering path without separate deployment flows.
• Broader operator coverage. Adding support for attention operators, layer normalization, and other popular architecture patterns beyond traditional CNNs, along with CMSIS-DSP operators such as FFT so feature extraction can run as part of the graph.
• More end-to-end examples. Voice activity detection, gesture recognition, predictive maintenance, and streaming wake word, the fifth MLPerf Tiny benchmark, which needs test infrastructure for multi-frame inference.
• Additional hardware validation. Expanding the set of validated platforms and development kits.

 

We are excited about what this unlocks for the embedded ML community. If you are deploying ML at the edge, whether on a $1 sensor node or a Cortex-M55 running voice commands, ExecuTorch now provides a direct path from PyTorch training to on-device inference.

Learn more at ExecuTorch Documentation | ExecuTorch repo | Cortex-M Backend | ExecuTorch Discord