Whisper Small INT8

Mobile CPUvivo X300SME2

A compact INT8-quantized speech recognition model for transcribing 16 kHz audio, using an encoder-decoder Transformer that converts log-mel spectrograms into text tokens. It combines dynamic INT8 linear layers with selective FP32 components to reduce size and improve inference efficiency while preserving transcription quality.View on Hugging Face

Accuracyvs base model

WER3.41%↓0.04 pp

CER1.29%↓0.04 pp

RTFx

0.9

Latency

7802.5ms

Peak Memory

4662.29MB

Get Started with this Model

Learning pathRecommended

Transcribe speech with models from the Arm AI Portal on Android

Download and run Arm-optimized Whisper speech-to-text models from the Arm AI Portal on an Arm-powered Android device.

Learning path

Build a Sentiment-Aware Voice Assistant with On-Device LLMs

Build an end-to-end, on-device voice assistant that understands both speech and emotion using Whisper, HuBERT, ONNX Runtime, and a local LLM with llama.cpp on Arm.

Learning path

Accelerate multimodal Voice Assistant performance with KleidiAI and SME2

Build and benchmark a multimodal Android voice-assistant pipeline, then use KleidiAI and SME2 to accelerate its speech recognition and language-model components.

Model Evaluation

Base model: whisper-smallTested on: vivo X300 (arm64, 8-core)

Evaluation Conditions

Automatic Speech Recognition · vivo X300 · INT8 · LibriSpeech (test-clean)

Metric	Resultsvs baseline	Change
End to end latency p50	7802.50ms10927.00ms	−3124.50 ms
End to end latency p90

About this version

Original Model:: whisper-small by OpenAI — Repository
Optimization/conversion:: Arm-Optimized version for execution on Arm-based platforms.
Converted/optimized by:: Arm
License:: The Original Model and the Optimized Model are subject to Apache-2.0.

This repository contains a converted or optimized version of the Original Model (the “Optimized Model”). The Original Model has been converted or optimized as described above for execution on Arm-based platforms.

No retraining or fine-tuning of the Original Model was performed as part of the conversion or optimization. The conversion or optimization was not intended to change the Original Model’s behavior or intended use.

Original Model and Documentation

For information about the Original Model, including its development, training data, intended uses, limitations and other relevant information, please refer to the Original Model repository. Information in that repository was provided by the original developer or other third parties and, unless expressly stated otherwise, has not been independently verified by Arm.

Licenses and Third-Party Terms

Use of the Original Model and the Optimized Model is subject to the applicable licenses, usage restrictions and other terms identified above and in the relevant repositories. Publication of the Optimized Model does not grant any rights beyond those provided under the applicable license terms.

You are responsible for reviewing those terms and ensuring that your use of the Original Model and the Optimized Model is permitted.

Metric

Resultsvs baseline

Change

End to end latency p50

7802.50ms10927.00ms

−3124.50 ms

End to end latency p90

About this version

Original Model:: whisper-small by OpenAI — Repository
Optimization/conversion:: Arm-Optimized version for execution on Arm-based platforms.
Converted/optimized by:: Arm
License:: The Original Model and the Optimized Model are subject to Apache-2.0.

Original Model and Documentation

Licenses and Third-Party Terms

You are responsible for reviewing those terms and ensuring that your use of the Original Model and the Optimized Model is permitted.

Configuration	Baseline	Arm Optimized
Dataset	LibriSpeech (test-clean)	LibriSpeech (test-clean)
Sample count	2,620	2,620
Batch size	1	1

Whisper Small INT8

Get Started with this Model

Transcribe speech with models from the Arm AI Portal on Android

Build a Sentiment-Aware Voice Assistant with On-Device LLMs

Accelerate multimodal Voice Assistant performance with KleidiAI and SME2

Model Evaluation

Whisper Small INT8

Get Started with this Model

Transcribe speech with models from the Arm AI Portal on Android

Build a Sentiment-Aware Voice Assistant with On-Device LLMs

Accelerate multimodal Voice Assistant performance with KleidiAI and SME2

Model Evaluation

Configuration

Comparability

Purpose of this Release