Beyond the model: From PyTorch to on-device AI using ExecuTorch

Developing an AI model is only part of the journey. Getting that model to run efficiently on real hardware, often with limited memory, compute resources and power, introduces a whole new set of challenges. As AI moves beyond the cloud to smartphones, laptops, IoT devices, and microcontrollers, developers need an efficient way to bring their models to these platforms.

PyTorch is widely used to build and train machine learning models. ExecuTorch extends that familiar workflow to on-device inference, helping developers take their models from development to deployment on edge devices.

Through our collaboration with Meta and the broader PyTorch ecosystem, Arm is working to make this journey easier with hardware and software optimizations, developer resources and opportunities for developers to learn and build together.

From models to deployment

ExecuTorch is a lightweight runtime that executes PyTorch models on-device. Developers start with a PyTorch model, export it, optimize it for target hardware where appropriate, and prepare it for deployment through a supported backend. Once deployed, they can profile and refine the model to meet their application’s requirements.

Arm supports this workflow across different classes of hardware through optimized software libraries and dedicated execution paths:

• Cortex-A CPUs: ExecuTorch’s XNNPACK backend enables optimized CPU inference on Arm-powered devices. Arm’s KleidiAI library provides optimized computational kernels that XNNPACK can use to accelerate supported machine learning operations.
• Cortex-M CPUs: ExecuTorch’s Cortex-M backend can use CMSIS-NN, Arm’s optimized neural network kernel library, to accelerate supported quantized operations on resource-constrained microcontrollers.
• Ethos-U NPUs: ExecuTorch’s Ethos-U backend enables supported quantized operations to run on dedicated Arm NPUs like Ethos U55, U65 and U85. It uses TOSA representation and Arm’s Vela compiler to prepare workloads for execution.

These integrations allow developers to target different Arm-powered devices while retaining a familiar PyTorch development workflow.

For more information on Arm's technical contributions, read the ExecuTorch 1.0 announcement and explore the details of Arm’s work with PyTorch.

For developers interested in exploring this workflow hands-on, the Optimize and Deploy AI Inference with ExecuTorch on Arm course provides practical exercises covering model export, quantization, backend delegation, performance benchmarking and deployment on Arm hardware.

Putting ExecuTorch into developers’ hands

One of the best ways to understand a technology is to build something with it.

At our recent Arm x Meta ExecuTorch hackathon at San Francisco State University (SFSU), developers came together to explore the possibilities of on-device AI.

Participants used Alif Semiconductor's Ensemble E8 development boards, featuring dual-core Cortex-M55 CPUs and an Ethos-U85 NPU. They explored deploying AI models on resource-constrained embedded hardware to power interactive, real-world applications.

Projects like Alfy and FingerSpell demonstrated what is possible using ExecuTorch. Alfy, a multimodal animated AI companion, ran four machine learning models on-device, while FingerSpell translated American Sign Language fingerspelling to editable text.

Starting with example projects and development resources, participants built their own applications on real hardware. The hackathon demonstrated how hands-on experimentation can help developers move beyond tutorials and turn their ideas into working applications.

Creating more opportunities for developers

Building on the SFSU hackathon, we are introducing Arm ExecuTorch Developer Days, a series of technical talks, hands-on workshops and hackathons designed to help developers move from PyTorch models to working AI applications on Arm-powered hardware. Participants will explore model deployment, hardware-aware optimization and performance evaluation.

This is just the beginning. As the series grows, we plan to introduce new hardware platforms, challenges, and technical tracks covering generative AI, embedded intelligence, and emerging agentic applications.

By connecting developers with technical experts and our ecosystem partners, we aim to make on-device AI development more accessible to the broader developer community.

Get started with ExecuTorch on Arm

Ready to explore what is possible?

Explore upcoming Arm ExecuTorch Developer Days and other resources on our ExecuTorch on Arm page. 

We look forward to seeing what you build next. 

ExecuTorch on Arm