AI PORTAL
  • Models
  • Code
  • Tools

About the models on this Arm AI Portal

The Arm AI Portal (the “Portal”) provides details of AI models developed, optimized or tested for use on Arm-based platforms. Models listed on the Portal may have been developed by Arm or by third parties. Unless expressly stated otherwise, Arm did not develop or train the underlying third-party model and is not responsible for its original intended behavior or use.

Information accompanies each model listing and describes the model’s provenance, together with relevant information about the original third-party model, its developer and repository, where applicable. Inclusion on the Portal of a model developed by a third party or based on a third-party model does not constitute Arm’s endorsement or certification of that third-party model or any third-party work relating to it.

Except as expressly stated in the information accompanying a specific model, Arm makes no representation as to the accuracy, safety, security, non-infringement, legal compliance, suitability for production use or fitness for any particular purpose of any model, converted or optimized version, or related outputs.

Inclusion of a model on the Portal does not itself grant you any rights to use that model. Use of each model is subject to the applicable license terms, usage restrictions and documentation. You are responsible for reviewing that information and independently evaluating the model, its outputs and its suitability for your intended use, including compliance with applicable legal, regulatory, safety and security requirements. Any use of or reliance on a model, its outputs or related materials is at your own risk. Where the Portal links to repositories or other materials controlled by third parties, those repositories and materials are outside Arm’s control and may change without notice.

Use of the Portal is subject to the Arm Website Terms and Conditions of Use.

Models

Filters

2

Device Class

Mobile CPUCloud CPU

Evaluation Target

AWS Graviton G4vivo X300

Arm Technology

SME2

Task

Text GenerationObject DetectionImage ClassificationImage SegmentationDepth EstimationImage-to-ImageText-to-Speech

Vendor

Runtime

Quantization

Format

Execution Backend

Model Size

≤5.7 GB

Average Memory

≤5.48 GB

Peak Memory

≤5.5 GB
    14 results

    Llama-3.2-1B-Base INT4

    vivo X300SME2

    A compact decoder-only causal language model for text generation, using INT4 weight quantization with dynamic INT8 activations and INT8 embeddings to reduce size and memory use while preserving practical language capabilities. It supports raw prompt continuation and in-context few-shot tasks without chat formatting.

    text-generation1.2 GBONNX

    Gemma-3-1B-Base INT4

    vivo X300SME2

    A compact decoder-only causal language model for text generation, using INT4 quantized linear weights with dynamic INT8 activations and tied INT8 block-quantized embeddings/LM head. It is designed for efficient continuation and few-shot prompting with a large SentencePiece vocabulary, offering reduced memory footprint and faster CPU inference while retaining the behavior of a base, non-instruction-tuned model.

    text-generation865.5 MBONNX

    Llama-3.2-3B-Instruct INT4

    AWS Graviton G4

    Quantized instruction-following causal language models for efficient text generation, using INT4 weights with dynamic INT8 activations plus INT8 embeddings and LM head. The family targets compact storage and low-memory operation while preserving chat-oriented capabilities, long-context handling, and decoder-only transformer behavior.

    text-generation2.7 GBONNX

    TinyLlama-1.1B-Chat INT4

    AWS Graviton G4

    A compact chat-oriented causal language model family optimized with INT4/INT8 quantization for efficient text generation and instruction following, offering reduced memory footprint and faster decoding while preserving strong general-purpose conversational capabilities within a 2,048-token context window.

    text-generation763.2 MBONNX

    Llama-3.2-1B-Instruct INT4

    vivo X300SME2

    Quantized decoder-only causal language model for instruction-following and text generation, packaged in ONNX GenAI format with INT4 weights, dynamic INT8 activations, and INT8 embeddings to reduce memory and compute while preserving practical chat capabilities on ARM CPU environments.

    text-generation1.2 GBONNX

    Qwen2.5-1.5B-Instruct INT4

    AWS Graviton G4

    A compact 1.5B-parameter instruction-tuned decoder-only language model for text generation and general question answering, using GPTQ INT4 quantized weights with dynamic INT8 activations and an INT8 language-model head to reduce size and improve inference efficiency while preserving strong instruction-following capabilities.

    text-generation1.1 GBONNX

    Llama-3.1-8B-Instruct INT4

    AWS Graviton G4

    Instruction-tuned causal language model for chat and text generation, packaged in ONNX format with INT4 weight quantization, dynamic INT8 activations, and INT8 embeddings to reduce size and memory use while preserving general-purpose assistant capabilities.

    text-generation5.4 GBONNX

    Gemma-3-1B-Instruct INT4

    vivo X300SME2

    Instruction-following causal language model packaged for efficient text generation, using INT4 quantized weights, dynamic INT8 activations, and tied INT8 block-quantized embeddings shared with the LM head. It supports chat-style prompting with a large SentencePiece vocabulary and is designed to reduce model size and memory use while preserving strong response quality.

    text-generation865.5 MBONNX

    Llama-3.2-1B-Base INT4

    AWS Graviton G4

    Compact decoder-only causal language models optimized for efficient text generation on ARM CPUs, using ONNX Runtime GenAI with INT4 weights, dynamic INT8 activations, and INT8 embeddings. Designed for low-memory, low-latency inference on Graviton and edge-style aarch64 systems, they support long-context continuation and in-context learning workflows while preserving the behavior of a base, non-instruction-tuned pretrained model.

    text-generation1.2 GBONNX

    Qwen3-1.7B-Base INT4 Dynamic

    vivo X300SME2

    This is a 4-bit quantized version of Qwen/Qwen3-1.7B-Base optimized for edge deployment on ARM devices using ONNX Runtime. The model was quantized using GPTQ (INT4 weights, group size 32, asymmetric, with an INT8 LM head and tied INT8 token embeddings) and exported to the ONNX format with a fused key-value-cache attention graph for efficient autoregressive inference on ARM CPUs (Android smartphones, AWS Graviton). Activations are quantized to INT8 per token dynamically at kernel time (W4A8), so no activation calibration is required.

    text-generation1.3 GBONNX

    Llama-3.2-1B-Instruct INT4

    AWS Graviton G4

    Instruction-following causal language model family with a decoder-only Transformer architecture, packaged in ONNX format and compressed using INT4 groupwise weights, dynamic INT8 activations, and INT8 embeddings/output head for efficient text generation with reduced memory footprint.

    text-generation1.2 GBONNX

    Llama-3.1-8B-Base INT4

    AWS Graviton G4

    A quantized 8B-parameter base causal language model for text completion, using INT4 weights with dynamic INT8 activations plus INT8 embeddings and LM head. It is a decoder-only Transformer with grouped-query attention, a large native context, and no chat or instruction tuning, making it suited to prompt completion and few-shot pattern-following tasks.

    text-generation5.7 GBONNX

    Qwen3-0.6B INT4

    AWS Graviton G4

    A compact decoder-only causal language model for text generation and instruction following, using GPTQ INT4 weight quantization with dynamic INT8 activations to reduce size and memory use while preserving response quality. It supports long-context generation, ChatML-style prompting, optional reasoning-style outputs, and efficient autoregressive decoding through ONNX Runtime GenAI.

    text-generation483.7 MBONNX

    TinyLlama-1.1B-Chat INT4

    vivo X300SME2

    A compact chat-oriented causal language model for text generation and instruction following, using mixed-precision quantization with INT4 transformer weights, INT8 embeddings, dynamic INT8 activations, and a protected INT8 output head to reduce size and memory while preserving conversational quality.

    text-generation763.2 MBONNX

    Develop on Arm with your AI coding assistant

    The Arm AI Portal MCP Server makes it faster to find optimized models, integrate code examples and deploy to your target device from your agentic coding assistant.

    User guide

    Run the following command in your terminal:

    codex mcp add arm-ai --url https://mcp.api.devplatform.arm.com/ai-portal