Meet Arm AI Portal: The launchpad for your next AI application

Whether you are adding generative AI to a smartphone, deploying vision at the edge, or running a task-specific model on an Arm-based cloud CPU, choosing the right model is only the beginning. You need to know whether it fits your target, runtime, and performance constraints, and how to get it running efficiently.

Either the full capabilities of the device are not effectively utilized, leaving performance on the table, or the AI feature impedes the rest of the application, slowing it down or drawing too much power.

Arm AI Portal is designed to make these integration and deployment challenges easier, narrowing the gap between idea and working prototype:

• Start with an optimized model: Find and compare validated models for your application and hardware, from cloud to edge and physical AI use cases.
• Bring your own model: Use Arm optimization tooling to meet your performance requirements – sign up for our early access program.
• Build where you build: Access models and resources through AI Portal, Hugging Face, GitHub and agentic development environments that you already use.

At launch, pre-optimized models include Alibaba Qwen, Google Gemma, and Ultralytics YOLO, using runtimes including ExecuTorch, LiteRT, and ONNX-RT, with support from ecosystem partners including Alibaba, Raspberry Pi, and Ultralytics.

---

“As generative AI reaches a growing range of devices, developers need fast and efficient models that can be deployed across a diverse hardware ecosystem. As one of the most popular model families in the developer community, Qwen and Arm are working in close collaboration, to take full advantage of features such as SME2, giving developers a more efficient foundation for building on-device experiences such as speech generation and, omni-modal perception. Through the Arm AI Portal, we are making these optimizations available across the Qwen model family, making it easier for developers throughout the Arm ecosystem to build with Qwen.” – Dong Xu, General Manager of Qwen LLM Business, Alibaba Group

“By partnering with Arm, Ultralytics is helping developers take proven vision models like YOLO and deploy them more easily across the Arm ecosystem. Making optimized, reusable models available through the AI Portal reduces the work needed to tune for individual devices, so developers can move faster from experimentation to production and bring high-performance vision experiences to a much broader range of Arm-based platforms.” – Manuel Calve, Head of Ecosystem, Ultralytics.

---

From standalone model to application feature

With AI Portal, you can filter by task type, device class, runtime, or format, while also using filters to narrow down pre-optimized model selection based on hardware constraints. It is simple to compare multiple models within a single task type, such as object detection, allowing a model to be selected based on several configurable axes including benchmark score, peak memory, average memory, or latency. Of course, there is no universal best model, but the best choice is the one that fits your workload, software stack, and product constraints.

Individual models include provenance information, including training datasets, benchmarking conditions, and performance evaluations compared to the base model. Runtimes and test devices are also recorded.

From a model page, you can either download the model artifact, follow through to Hugging Face, or choose a code example or guide that demonstrates how to deploy the model within an application context. Models also have Learning Paths for more in-depth deployment guides.

Built for developers, discoverable by agents: Integrating AI Portal model selection into agentic development workflows

AI isn't only changing what developers build. It's changing how software gets built. As coding becomes increasingly agentic, the models, performance data and deployment resources developers rely on need to be machine-discoverable too.

AI Portal provides clear, structured signals that help coding agents identify models optimized for the target hardware and find appropriate deployment workflows. Through the Arm MCP Server, agents can access those resources directly from supported development environments.

First, install the MCP server using the appropriate commands for your AI coding tool, for example:

codex mcp add arm-ai --url https://mcp.api.devplatform.arm.com/ai-portal

Once the MCP is enabled, you can use a natural language prompt to identify and deploy a model. For example:

I have a Raspberry Pi 5. Use the AI Portal to find an image classification model and example application suitable for this device.

The MCP server enables your agent to discover Arm-optimized models for your target hardware, and identify suitable deployment model pathways, spanning containerized projects, example applications, and learning pathways.

A brief detour down the stack: Model optimization deep-dive

For the initial models that Arm has made available on AI Portal, optimization techniques have been used to carefully balance accuracy and on-device performance, with the results published for each model on Hugging Face and performance shown compared to the base model on AI Portal.

Model optimization is a process with several potential steps: models are exported from an upstream checkpoint and their operators mapped to ensure compatibility. Often, performance gains are made possible by quantization, or repacking weight values, altering the numerical precision of the model as a trade-off. Also, performance can be improved by altering the graph structure, serializing model shape, or ensuring that performance kernels are selected. Runtimes include Arm KleidiAI for acceleration, but the choice of runtime also changes overall model performance. 

Case study: Making OpenAI Whisper’s decoder cache explicit

Quantization can reduce model size and accelerate matrix operations, but it does not remove unnecessary computation. Whisper’s native autoregressive generation loop could not be represented efficiently in the static LiteRT export used in our initial optimization investigation, causing the encoder work to be repeated during decoding.

Restructuring the model into separate encode and decode signatures allowed the encoder to compute the cross-attention key and value data once, while the decoder carries its self-attention cache explicitly from one token to the next. Arm then applied dynamic-range INT8 quantization, using per-channel INT8 weights and activations quantized at runtime inside the accelerated kernels.

On a vivo X300 using four CPU threads and the LiteRT, XNNPACK, and KleidiAI stack, the combined architectural and quantization changes reduced Whisper Base’s median latency from 16.55 seconds to 1.05 seconds, a 15.7x speedup. Whisper Medium fell from 174.16 seconds to 8.85 seconds, a 19.7x improvement. The real-time factor increased from 0.074 to 1.45, taking the model from far slower than real time to processing audio faster than it is played, which makes it viable for use in real-time applications.

Quality remained close to the FP32 baseline, with Whisper Base WER changed by +0.11 percentage points and Whisper Medium’s measured WER by -0.03 points. The optimization takeaway is that quantization delivers its full value only after the exported graph stops recomputing work that should be cached.

Case study: Turning Mask2Former fallbacks into accelerated kernels

Mask2Former demonstrated another common deployment problem: a model can be quantized successfully, but does not execute efficiently on-device. Profiling the initial ExecuTorch export showed that 28 decoder addmm calls (matrix multiplications with an added bias) were falling back to the portable FP32 backend. Together, these calls accounted for 58% of profiled operator compute. After graph conversion, their original linear structure was no longer clear to the XNNPACK partitioner, preventing it from selecting its optimized INT8 kernels.

Arm added a targeted graph transformation that rewrote the affected addmm patterns as mathematically equivalent linear operations before partitioning. This allowed the XNNPACK partitioner to recognize and accelerate them, while numerically sensitive attention blocks remained in FP32. On an AWS Graviton instance, mean latency for a 640 x 640 input fell from 5.30 seconds to 2.43 seconds, a 2.18 times speedup, and the ExecuTorch artifact shrank from 824 MB to 390 MB. Across all 5,000 COCO validation images, mask mAP changed from 46.82% to 46.69%, a reduction of just 0.13 percentage points. This illustrates why effective optimization must align the model graph with the patterns the deployment runtime can actually accelerate.

Model optimization tools: Early access

If you are deploying your own models to Arm-based devices, many of the optimization techniques outlined above are available in early access. If you would like to join our program and provide feedback, then you can register your interest here and we will be in touch.

Deployment workflows

Developing an application that includes an AI model is not complete without deploying it to your target device. Our approach to deployment differs based on device class:

Android: For Arm-based Android devices, we provide a series of step-by-step guides that walk you through setting up Android Studio or using a CLI approach, preparing a target device and cloning example applications, which act as a harness for a range of models that you can evaluate. Alternatively, if you are completely new to running local models on device, Google also provides the Google AI Edge Gallery app for Android and iOS as well as desktop, which gives you a rapid, zero-code way to evaluate a model.

Edge Linux: For application development on edge devices that run Linux, we have taken the approach of providing containerized examples that combine runtime with task type, allowing you to switch between a range of compatible models easily during the configuration of the project. For example, we have provided an image classifier example that includes ExecuTorch and XNNPACK, plus a simple Gradio web interface, allowing multiple image classifier models to be configured. This workflow is enabled by Topo, our CLI and VS Code extension for multi-container development and deployment. Once installed, it is a simple command, or a single click to launch VS Code, to clone the example and bootstrap any necessary tools dependencies.

Cloud CPU: Learn how to deploy AI workloads to Arm-based servers, including AWS Graviton, Google Axion and Microsoft Azure Cobalt with guides that you can adapt for your use case. Arm also provides a simple Learning Path to guide you through setting up ONNX runtime, loading in a model, linking it, and testing within a simple application that you can build from.

Cortex-M Microcontrollers and Ethos-U NPUs: For embedded use cases, there are a significant range of example projects, including anomaly detection, automated speech recognition, image classification, and object detection. Each project describes required tooling dependencies that can easily be installed, either as VS Code extensions or standalone CLI tools.

For further details on deployment workflows for each device class, check out the Learning Path here.

Your AI app – best on Arm

AI Portal brings together simple pre-optimized model discovery, performance evaluations, and practical deployment workflows, helping you move from an AI feature idea to code running on real Arm hardware, whether the target is a cloud server, smartphone, physical AI solution, edge device, or microcontroller. Explore AI Portal, choose a workload and target, and test a model. And if you build models, devices, or development tools, work with us to make the next generation of AI experiences easier for developers to deploy on Arm.