AI PORTAL
  • Models
  • Code
  • Tools

About the models on this Arm AI Portal

The Arm AI Portal (the “Portal”) provides details of AI models developed, optimized or tested for use on Arm-based platforms. Models listed on the Portal may have been developed by Arm or by third parties. Unless expressly stated otherwise, Arm did not develop or train the underlying third-party model and is not responsible for its original intended behavior or use.

Information accompanies each model listing and describes the model’s provenance, together with relevant information about the original third-party model, its developer and repository, where applicable. Inclusion on the Portal of a model developed by a third party or based on a third-party model does not constitute Arm’s endorsement or certification of that third-party model or any third-party work relating to it.

Except as expressly stated in the information accompanying a specific model, Arm makes no representation as to the accuracy, safety, security, non-infringement, legal compliance, suitability for production use or fitness for any particular purpose of any model, converted or optimized version, or related outputs.

Inclusion of a model on the Portal does not itself grant you any rights to use that model. Use of each model is subject to the applicable license terms, usage restrictions and documentation. You are responsible for reviewing that information and independently evaluating the model, its outputs and its suitability for your intended use, including compliance with applicable legal, regulatory, safety and security requirements. Any use of or reliance on a model, its outputs or related materials is at your own risk. Where the Portal links to repositories or other materials controlled by third parties, those repositories and materials are outside Arm’s control and may change without notice.

Use of the Portal is subject to the Arm Website Terms and Conditions of Use.

Models

Filters

1

Device Class

Edge LinuxMobile CPUCloud CPU

Evaluation Target

vivo X300AWS Graviton G4Raspberry Pi 5

Arm Technology

SME2

Task

Text GenerationImage ClassificationObject DetectionFeature ExtractionImage-to-ImageAutomatic Speech RecognitionImage SegmentationDepth EstimationText-to-SpeechAudio ClassificationKeypoint DetectionImage-Text-to-TextText-to-ImageTranslationZero-Shot Image Classification

Vendor

Runtime

Quantization

Format

Execution Backend

Model Size

≤9 GB

Average Memory

≤5.48 GB

Peak Memory

≤8.79 GB
    50 results

    TinyLlama 1.1B Chat INT8

    Raspberry Pi 5

    An INT8-quantized TinyLlama 1.1B Chat causal language model for conversational text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation1.2 GB
    ExecuTorch

    Gemma 2 2B INT8

    Raspberry Pi 5

    An INT8-quantized Gemma 2 2B causal language model for text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation4 GBExecuTorch

    Qwen2.5 0.5B Instruct INT8

    Raspberry Pi 5

    An INT8-quantized Qwen2.5 0.5B Instruct causal language model for instruction-following and text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation806.7 MBExecuTorch

    Phi-3.5 Mini INT8

    AWS Graviton G4

    An INT8-quantized Phi-3.5 Mini language model for text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation5 GBExecuTorch

    Llama-3.2-1B-Base INT4

    vivo X300SME2

    A compact decoder-only causal language model for text generation, using INT4 weight quantization with dynamic INT8 activations and INT8 embeddings to reduce size and memory use while preserving practical language capabilities. It supports raw prompt continuation and in-context few-shot tasks without chat formatting.

    text-generation1.2 GBONNX

    Llama 3.1 8B Instruct INT8

    Raspberry Pi 5

    An INT8-quantized Llama 3.1 8B Instruct causal language model for instruction-following and text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation9 GBExecuTorch

    Gemma-3-1B-Base INT4

    vivo X300SME2

    A compact decoder-only causal language model for text generation, using INT4 quantized linear weights with dynamic INT8 activations and tied INT8 block-quantized embeddings/LM head. It is designed for efficient continuation and few-shot prompting with a large SentencePiece vocabulary, offering reduced memory footprint and faster CPU inference while retaining the behavior of a base, non-instruction-tuned model.

    text-generation865.5 MBONNX

    Qwen3.5-0.8B Q4_K_M

    vivo X300SME2

    This is a 4-bit weight-quantized version of Qwen/Qwen3.5-0.8B packaged as a single GGUF file for edge deployment on ARM CPUs using llama.cpp. The model was quantized with llama.cpp's native Q4_K_M k-quant recipe, guided by an importance matrix (llama-imatrix) computed over a plain-text calibration corpus, and it runs on device through ggml CPU_REPACK kernels compiled with -march=armv8.6-a+dotprod+i8mm. Norms and the SSM convolution/time-step parameters are kept in FP32. The upstream checkpoint is a vision-language model; the export produced a separate f16 multimodal projector (mmproj) GGUF that is not part of this bundle and was neither benchmarked nor evaluated. The tokenizer, merges, special tokens and chat template are embedded in the GGUF, so a single file is all you need to run the language model.

    text-generation529.3 MBllama.cpp

    Phi-4 Mini INT8

    vivo X300

    An INT8-quantized Phi-4 Mini language model for text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation5.3 GBExecuTorch

    TinyLlama 1.1B Chat INT8

    vivo X300

    An INT8-quantized TinyLlama 1.1B Chat causal language model for conversational text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation1.2 GBExecuTorch

    SmolLM2-360M-Instruct 8da4w

    vivo X300SME2

    A compact 362M-parameter decoder-only causal language model for instruction-following, open-ended text generation, and multiple-choice scoring, using 8-bit dynamic activations and 4-bit grouped weights to reduce size and memory needs while preserving core language capabilities.

    text-generation251.7 MBExecuTorch

    Qwen2.5 0.5B Instruct INT8

    vivo X300

    An INT8-quantized Qwen2.5 0.5B Instruct causal language model for instruction-following and text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation806.4 MBExecuTorch

    Gemma 3 1B Instruct INT8

    vivo X300

    An INT8-quantized Gemma 3 1B Instruct causal language model for instruction-following and text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation1.7 GBExecuTorch

    Llama 3.2 1B Instruct INT8

    AWS Graviton G4

    An INT8-quantized Llama 3.2 1B Instruct causal language model for instruction-following and text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation1.8 GBExecuTorch

    SmolLM2 INT8

    Raspberry Pi 5

    An INT8-quantized SmolLM2 compact language model for text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation202.2 MBExecuTorch

    SmolLM3 INT8

    vivo X300

    An INT8-quantized SmolLM3 language model for text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation3.8 GBExecuTorch

    Llama-3.2-3B-Instruct INT4

    AWS Graviton G4

    Quantized instruction-following causal language models for efficient text generation, using INT4 weights with dynamic INT8 activations plus INT8 embeddings and LM head. The family targets compact storage and low-memory operation while preserving chat-oriented capabilities, long-context handling, and decoder-only transformer behavior.

    text-generation2.7 GBONNX

    DeepSeek-R1-Distill-Qwen-1.5B Q4_K_M

    vivo X300SME2

    DeepSeek-R1-Distill-Qwen-1.5B text generation optimized as a Q4_K_M GGUF model for the llama.cpp runtime on Arm-based Premium Smartphone systems.

    text-generation1.1 GBllama.cpp

    SmolLM3 INT8

    Raspberry Pi 5

    An INT8-quantized SmolLM3 language model for text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation3.8 GBExecuTorch

    Phi-3 Mini 3.8B INT8

    Raspberry Pi 5

    An INT8-quantized Phi-3 Mini 3.8B language model for text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation4.1 GBExecuTorch

    Phi-3.5 Mini INT8

    Raspberry Pi 5

    An INT8-quantized Phi-3.5 Mini language model for text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation5.4 GBExecuTorch

    Qwen3-1.7B-Base Q4_K_M

    vivo X300SME2

    This is a 4-bit weight-quantized version of Qwen/Qwen3-1.7B-Base packaged as a single GGUF file for edge deployment on ARM CPUs using llama.cpp. The model was quantized with llama.cpp's native Q4_K_M k-quant recipe, guided by an importance matrix (llama-imatrix) computed over a plain-text calibration corpus, and it runs on device through ggml CPU_REPACK kernels compiled with -march=armv8.6-a+dotprod+i8mm. Quantization is weight-only — activations stay float at runtime, so there is no activation calibration and no runtime quantization step to configure. The tokenizer, merges, special tokens and chat template are embedded in the GGUF, so a single file is all you need to run the model.

    text-generation1.2 GBllama.cpp

    ERNIE-4.5-0.3B-PT Q4_K_M

    Raspberry Pi 5

    ERNIE-4.5-0.3B-PT optimized for text generation in GGUF format with the llama.cpp runtime, targeting Arm-based Edge Linux systems.

    text-generation240.6 MBllama.cpp

    TinyLlama-1.1B-Chat INT4

    AWS Graviton G4

    A compact chat-oriented causal language model family optimized with INT4/INT8 quantization for efficient text generation and instruction following, offering reduced memory footprint and faster decoding while preserving strong general-purpose conversational capabilities within a 2,048-token context window.

    text-generation763.2 MBONNX

    Phi-3 Mini 3.8B INT8

    vivo X300

    An INT8-quantized Phi-3 Mini 3.8B language model for text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation4.1 GBExecuTorch

    Gemma 2 2B INT8

    AWS Graviton G4

    An INT8-quantized Gemma 2 2B causal language model for text generation. Exported as an ExecuTorch program using the XNNPACK backend for inference on Arm CPUs.

    text-generation4 GBExecuTorch

    Llama-3.2-1B-Instruct INT4

    vivo X300SME2

    Quantized decoder-only causal language model for instruction-following and text generation, packaged in ONNX GenAI format with INT4 weights, dynamic INT8 activations, and INT8 embeddings to reduce memory and compute while preserving practical chat capabilities on ARM CPU environments.

    text-generation1.2 GBONNX

    Gemma-4-E2B 8da4w

    vivo X300

    A compact instruction-tuned autoregressive language model for text generation and multiple-choice scoring, provided as a text-only quantized variant with dynamic 8-bit activations, grouped 4-bit linear weights, and packed 4-bit embeddings; multimodal encoders are not included.

    text-generation2.7 GBExecuTorch

    Develop on Arm with your AI coding assistant

    The Arm AI Portal MCP Server makes it faster to find optimized models, integrate code examples and deploy to your target device from your agentic coding assistant.

    User guide

    Run the following command in your terminal:

    codex mcp add arm-ai --url https://mcp.api.devplatform.arm.com/ai-portal