Gemma-3-1B-Base INT4

Mobile CPUvivo X300SME2

A compact decoder-only causal language model for text generation, using INT4 quantized linear weights with dynamic INT8 activations and tied INT8 block-quantized embeddings/LM head. It is designed for efficient continuation and few-shot prompting with a large SentencePiece vocabulary, offering reduced memory footprint and faster CPU inference while retaining the behavior of a base, non-instruction-tuned model.View on Hugging Face

Accuracyvs base model

HellaSwag (0-shot)57.96%↓0.99 pp

Tokens / sec

35.28 tok/s

Time to First Token

584.2 ms

Latency

4207.35 ms

Peak Memory

1489.25 MB

Get Started with this Model

Code example

Text Generator (ONNX Runtime GenAI)

An on-device evaluation harness for Arm-optimized generative ONNX models.

Model Evaluation

Base model: gemma-3-1b-ptTested on: vivo X300 (arm64, 8-core)

Evaluation Conditions

Text Generation · vivo X300 · INT4 · HellaSwag (validation)

Metric	Resultsvs baseline	Change
End to end latency p50	4207.35ms15725.10ms	−11517.76 ms
End to end latency p90

About this version

Original Model:: gemma-3-1b-pt by Google — Repository
Optimization/conversion:: Arm-Optimized version for execution on Arm-based platforms.
Converted/optimized by:: Arm
License:: Not specified in the model metadata.

This repository contains a converted or optimized version of the Original Model (the “Optimized Model”). The Original Model has been converted or optimized as described above for execution on Arm-based platforms.

No retraining or fine-tuning of the Original Model was performed as part of the conversion or optimization. The conversion or optimization was not intended to change the Original Model’s behavior or intended use.

Original Model and Documentation

For information about the Original Model, including its development, training data, intended uses, limitations and other relevant information, please refer to the Original Model repository. Information in that repository was provided by the original developer or other third parties and, unless expressly stated otherwise, has not been independently verified by Arm.

Licenses and Third-Party Terms

Use of the Original Model and the Optimized Model is subject to the applicable licenses, usage restrictions and other terms identified above and in the relevant repositories. Publication of the Optimized Model does not grant any rights beyond those provided under the applicable license terms.

You are responsible for reviewing those terms and ensuring that your use of the Original Model and the Optimized Model is permitted.

Purpose of this Release

The Optimized Model is provided as a reference implementation to demonstrate and evaluate execution and performance on Arm-based systems. It is not a production-ready or supported solution.

Metric

Resultsvs baseline

Change

End to end latency p50

4207.35ms15725.10ms

−11517.76 ms

End to end latency p90

About this version

Original Model:: gemma-3-1b-pt by Google — Repository
Optimization/conversion:: Arm-Optimized version for execution on Arm-based platforms.
Converted/optimized by:: Arm
License:: Not specified in the model metadata.

Original Model and Documentation

Licenses and Third-Party Terms

You are responsible for reviewing those terms and ensuring that your use of the Original Model and the Optimized Model is permitted.

Purpose of this Release

The Optimized Model is provided as a reference implementation to demonstrate and evaluate execution and performance on Arm-based systems. It is not a production-ready or supported solution.

Configuration	Baseline	Arm Optimized
Dataset	HellaSwag (validation)	HellaSwag (validation)
Sample count	8,000	8,000
Batch size	1	1

Gemma-3-1B-Base INT4

Get Started with this Model

Text Generator (ONNX Runtime GenAI)

Gemma-3-1B-Base INT4

Get Started with this Model

Text Generator (ONNX Runtime GenAI)

Configuration

Comparability