Supported models
Arm AI Model Optimizer provides 51 preset models. Pass a model name from np models ls to np projects create --model <name>.
Domains
- Computer vision (CV): Classification, object detection, super resolution, face detection, mask generation
- Large language model (LLM): Text generation, text embedding, text classification
- Audio: Speech-to-text, audio classification
- Multimodal: Vision-language, image-text matching
New devices
2 devices were added alongside the original device set (RPi5, M4 MacBook, Alif E8): AWS Graviton4 and vivo X300 Pro, both executorch and xnnpack. See Supported devices for the full matrix.
How to read this page
-
2 header rows. The first names the device, and the second names the 3 columns that device owns: quality metric, latency, and memory. A device with no measurement in a category is left out of that table.
-
2 rows for each model, or 3 where an Arm-tuned build exists.
-
Baseline: the reference model before optimization with Arm AI Model Optimizer. Its precision depends on the model, usually FP32 or FP16. On targets that run quantized models only, for example Alif E8, there is no floating-point baseline, so the baseline is itself a quantized model.
-
Arm-optimized: the standard product path, graph optimization, and 8-bit integer (INT8) quantization.
-
Arm-tuned: additional model-specific engineering on top of that path.
Compare the rows within 1 device to see what optimization changed: quality typically shifts by a small amount, and latency and memory typically drop (a lower value is better for both).
Cell symbols
| Symbol | Meaning |
|---|---|
| number | Measured value |
| — | Not a target for this device |
| WIP | Measurement in progress |
| x | Could not be measured on this hardware |
a / b / c | Several figures, in the order the column header names them |
The column header always names what a cell holds, including quality columns that report more than one figure (mAP / AP50 / AP75) and latency or memory split by pipeline stage (Latency prefill / decode (ms)). Quality metrics are not identical on every device: text embedding reports Recall@10 on RPi5, M4 Macbook, and Alif E8, and nDCG@10 on Graviton4 and vivo X300 Pro. Object detection adds AP50 and AP75 on those 2 devices.
Measurement conditions
- Runtime: executorch v1.1.0
- Devices: Raspberry Pi 5, MacBook M4, Alif Ensemble E8, AWS Graviton4, vivo X300 Pro. See Supported devices for specifications
- To measure on your own hardware, see On-device profiling
CV models
Classification
Dataset: ImageNet
| Model | Device | RPi5 | M4 Macbook | Alif E8 | AWS Graviton4 | vivo X300 Pro | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Type | Top-1 Acc (%) | Latency (ms) | Memory (MB) | Top-1 Acc (%) | Latency (ms) | Memory (MB) | Top-1 Acc (%) | Latency (ms) | Memory (MB) | Top-1 Acc (%) | Latency (ms) | Memory (MB) | Top-1 Acc (%) | Latency (ms) | Memory (MB) | |
convnext | Baseline | 86.12 | 443.25 | 388.72 | 86.12 | 436.37 | 387.75 | — | — | — | — | — | — | — | — | — |
| Arm-optimized | 85.92 | 275.01 | 230.2 | 86.16 | 244.41 | 125.55 | — | — | — | — | — | — | — | — | — | |
| Arm-tuned | 84.24 | 166.542 | 100.98 | — | — | — | — | — | — | — | — | — | — | — | — | |
deit_tiny | Baseline | 71 | 40.605 | 37.62 | — | — | — | 71 | x | x | — | — | — | 71 | 31.698 | 40.83 |
| Arm-optimized | 70 | 26.796 | 23.19 | — | — | — | 65.46 | 299.001 | 2.55 | — | — | — | 71.02 | 22.128 | 24.68 | |
efficientnet_b4 | Baseline | 83.06 | 467.94 | 188.92 | 83.06 | 37.73 | 205.33 | — | — | — | 83.08 | 15.028 | 159.67 | 83.08 | 80.417 | 164.31 |
| Arm-optimized | 82.74 | 431.22 | 132.91 | 83.1 | 38.41 | 183.11 | — | — | — | 82.82 | 12.563 | 86.11 | 82.38 | 63.65 | 99.18 | |
efficientvit | Baseline | 79.36 | 26.01 | 50.92 | 79.36 | 5.06 | 69.39 | 24.3 | 801.96 | 2.68 | — | — | — | — | — | — |
| Arm-optimized | 78.68 | 20.9 | 31.69 | 78.68 | 4.87 | 50.11 | 33.22 | 68.93 | 2.25 | — | — | — | — | — | — | |
googlenet | Baseline | 69.3 | 28.68 | 49.22 | 69.3 | 9.3 | 69.7 | 69.38 | 95.84 | 2.34 | 69.32 | 3.725 | 49.22 | 69.3 | 14.048 | 61.15 |
| Arm-optimized | 69.24 | 8.92 | 20.67 | 69.24 | 3.43 | 30.48 | 69.7 | 83.168 | 2.34 | 68.8 | 2.835 | 24.93 | 68.6 | 5.68 | 32.96 | |
| Arm-tuned | 69.76 | 5.89 | 20.38 | — | — | — | 69.66 | 73.144 | 2.2 | — | — | — | 69.58 | 4.977 | 30.84 | |
inception_v3 | Baseline | 77.8 | 343.13 | 119.25 | 77.8 | 32.94 | 160.75 | — | — | — | 77.78 | 71.097 | 128.82 | 77.78 | 171.229 | 149.9 |
| Arm-optimized | 77.42 | 265.56 | 43.11 | 77.42 | 29.29 | 53.02 | — | — | — | 76.92 | 4.923 | 49.09 | 76.92 | 15.21 | 62 | |
mobile_vit_s | Baseline | 77.3 | 150.08 | 65.14 | 77.3 | 149.1 | 65.11 | — | — | — | 77.32 | 13.499 | 64.52 | 77.32 | 36.429 | 69.16 |
| Arm-optimized | 77.36 | 135.76 | 49.2 | 77.36 | 136.08 | 49.2 | — | — | — | 77.28 | 11.451 | 48.64 | 77.28 | 29.385 | 50.98 | |
| Arm-tuned | 77.08 | 43.796 | 34.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
mobilenet_v3_small | Baseline | 65.64 | 3.98 | 22.3 | 65.64 | 1.71 | 32.53 | 65.7 | x | x | — | — | — | — | — | — |
| Arm-optimized | 65.28 | 2.87 | 13 | 65.72 | 1.86 | 19.12 | 64.82 | 76.819 | 3.73 | — | — | — | — | — | — | |
resnet18 | Baseline | 67.4 | 32.14 | 90.69 | 67.4 | 12.21 | 109.47 | 67.26 | 123.97 | 3.98 | — | — | — | — | — | — |
| Arm-optimized | 67.38 | 9.36 | 26.67 | 67.38 | 2.36 | 41.69 | 66.9 | 63.59 | 3.19 | — | — | — | — | — | — | |
squeezenet1_0 | Baseline | 55.58 | 20.22 | 21.5 | 55.58 | 1.64 | 33.47 | 54.94 | 61.05 | 2.65 | 55.56 | 1.824 | 22.93 | 55.56 | 8.021 | 29.72 |
| Arm-optimized | 54.92 | 5.09 | 14.16 | 54.92 | 1.5 | 22.44 | 54.8 | 22.92 | 2.65 | 55.66 | 1.65 | 21.54 | 55.66 | 5.794 | 25.25 | |
swin_t | Baseline | 80.34 | 149.57 | 154.5 | 80.34 | 19.65 | 175.59 | x | x | x | — | — | — | 80.74 | 72.743 | 147.73 |
| Arm-optimized | 80.58 | 96.87 | 57.89 | 80.58 | 19.23 | 71.95 | 62.04 | 1,581.31 | 8.6 | — | — | — | 80.58 | 35.648 | 66.03 | |
vit_b_16 | Baseline | 80.3 | 456.86 | 391.38 | 80.3 | 59.48 | 408.7 | — | — | — | — | — | — | 80.3 | 157.636 | 358.68 |
| Arm-optimized | 76.5 | 231.82 | 107.61 | 76.5 | 47.68 | 121.72 | — | — | — | — | — | — | 76.28 | 65.559 | 113.98 | |
| Arm-tuned | 78.34 | 124.993 | 72.13 | — | — | — | — | — | — | — | — | — | 78.34 | 54.875 | 75.11 | |
wideresnet50 | Baseline | 80.82 | 292.05 | 516.14 | 80.82 | 16.76 | 523.52 | 77.34 | 456.26 | 3.95 | 80.82 | 17.027 | 516.77 | 80.82 | 77.766 | 518.09 |
| Arm-optimized | 80.7 | 49.43 | 134.19 | 80.7 | 9.14 | 156.09 | 76.68 | 302.59 | 3.68 | 80.32 | 4.595 | 134.61 | 80 | 23.663 | 146.25 | |
| Arm-tuned | — | — | — | — | — | — | — | — | — | 79.62 | 3.28 | 95.18 | 79.62 | 16.85 | 114.42 |
Object detection
Dataset: COCO
| Model | Device | RPi5 | M4 Macbook | Alif E8 | AWS Graviton4 | vivo X300 Pro | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Type | mAP (%) | Latency (ms) | Memory (MB) | mAP (%) | Latency (ms) | Memory (MB) | mAP (%) | Latency (ms) | Memory (MB) | mAP / AP50 / AP75 (%) | Latency (ms) | Memory (MB) | mAP / AP50 / AP75 (%) | Latency (ms) | Memory (MB) | |
detr_resnet_50 | Baseline | 37.29 | 2,682.95 | 937.92 | 37.29 | 230.62 | 700.7 | — | — | — | — | — | — | — | — | — |
| Arm-optimized | 36.68 | 2,123.36 | 745.86 | 34.82 | 212.84 | 363.05 | — | — | — | — | — | — | — | — | — | |
rtmdet | Baseline | 51.59 | x | x | 51.59 | x | x | — | — | — | — | — | — | — | — | — |
| Arm-optimized | 49.95 | 681.44 | 241.88 | 50.35 | 61.61 | 232.5 | — | — | — | — | — | — | — | — | — | |
ssd_mobilenet_v2 | Baseline | 24.92 | 70.49 | 136.14 | 24.92 | 5.41 | 159.94 | 1.33 | 201.25 | 6.46 | — | — | — | — | — | — |
| Arm-optimized | 23.59 | 39.76 | 49.62 | 23.59 | 4.83 | 84.59 | 20.56 | 153.72 | 6.46 | — | — | — | — | — | — | |
yolox_s | Baseline | 36.61 | 347.97 | 123.64 | 36.61 | 29.5 | 160.58 | — | — | — | 37.0289 / 49.1696 / 39.8641 | 24.35 | 131.14 | 37.0289 / 49.1696 / 39.8641 | 120.032 | 140.11 |
| Arm-optimized | 34.99 | 194.32 | 72.58 | 34.99 | 24.15 | 81.34 | — | — | — | 36.1866 / 48.1498 / 39.3667 | 12.527 | 76.31 | 36.1866 / 48.1498 / 39.3667 | 59.751 | 89.74 |
Super resolution
Dataset: Set5
| Model | Device | RPi5 | M4 Macbook | Alif E8 | AWS Graviton4 | vivo X300 Pro | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Type | PSNR (dB) | Latency (ms) | Memory (MB) | PSNR (dB) | Latency (ms) | Memory (MB) | PSNR (dB) | Latency (ms) | Memory (MB) | PSNR (dB) | Latency (ms) | Memory (MB) | PSNR (dB) | Latency (ms) | Memory (MB) | |
qai_quicksrnetsmall | Baseline | 26.55 | 12.83 | 21.19 | 26.55 | 1.21 | 28.28 | 25.85 | 148.38 | 7.69 | 26.545 | 0.923 | 22.22 | 26.545 | 5.939 | 24.41 |
| Arm-optimized | 26.2 | 5.64 | 16.19 | 26.2 | 1.08 | 23.27 | 25.56 | 130.53 | 7.69 | 26.4555 | 0.428 | 18.65 | 26.4555 | 3.01 | 4.86 | |
sesr_m5 | Baseline | 26.52 | 115.27 | 46.53 | 26.52 | 8.7 | 54.03 | 25.95 | 200.44 | 9.44 | 26.5191 | 7.816 | 47.33 | 26.5191 | 47.626 | 50.52 |
| Arm-optimized | 26.27 | 33.76 | 29.19 | 26.52 | 1.92 | 36.67 | 26.48 | 177.15 | 9.44 | 26.4692 | 2.404 | 30.05 | 26.4006 | 6.552 | 28.93 | |
| Arm-tuned | 26.17 | 22.412 | 28.14 | — | — | — | — | — | — | 26.17 | 1.708 | 28.54 | — | — | — |
Face detection
| Model | Device | RPi5 | Alif E8 | ||||
|---|---|---|---|---|---|---|---|
| Type | AP50 / mAP / AP75 (%) | Latency (ms) | Memory (MB) | AP50 / mAP / AP75 (%) | Latency (ms) | Memory (MB) | |
mediapipe_face | Baseline | 13.0871 / 3.4681 / 0.1408 | 23.293 | 57.22 | 13.0871 / 3.4681 / 0.1408 | x | x |
| Arm-optimized | 12.8831 / 3.261 / 0.096 | 10.792 | 20.19 | 12.5236 / 3.123 / 0.066 | 162.663 | 5.88 |
Mask generation
| Model | Device | RPi5 | ||
|---|---|---|---|---|
| Type | mIoU | Latency (ms) | Memory (MB) | |
mobile_sam | Baseline | 0.7309 | 2,070.632 | 381.44 |
| Arm-optimized | 0.7301 | 1,595.461 | 348.56 |
LLM models
Text generation
Dataset: HellaSwag
| Model | Device | RPi5 | M4 Macbook | AWS Graviton4 | vivo X300 Pro | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Type | HellaSwag (%) | TTFT (ms) | Memory (MB) | HellaSwag (%) | TTFT (ms) | Memory (MB) | HellaSwag (%) | Latency prefill / decode (ms) | Memory prefill / decode (MB) | HellaSwag (%) | Latency prefill / decode (ms) | Memory prefill / decode (MB) | |
gemma2_2b | Baseline | 66 | 18,010.23 | 7,515.55 | 66 | 2,564 | 8,164 | 66 | 2,575.959 / 426.235 | 7,497.52 / 7,487.88 | 66 | 7,617.914 / 892.401 | 7,217.68 / 7,159.22 |
| Arm-optimized | 67 | 9,491.552 | 4,368.33 | 67 | 2,572.02 | 4,540.17 | 67 | 1,628.657 / 71.663 | 4,369.62 / 4,451.65 | 67 | 5,566.107 / 288.584 | 4,340.89 / 4,422.4 | |
gemma3_1b_it | Baseline | 60 | 7,535.76 | 3,437.67 | 60 | 1,198.5 | 3,417 | — | — | — | 60 | 2,910.24 / 330.764 | 3,331.9 / 3,110.76 |
| Arm-optimized | 59 | 5,092.782 | 1,938.16 | 59 | 1,122.32 | 1,989.17 | — | — | — | 59 | 2,781.28 / 253.89 | 1,911.45 / 1,884.16 | |
llama3_1_8b_instruct | Baseline | 70 | — | — | 70 | 7,827.6 | 15,767 | 70 | 6,484.742 / 896.85 | 16,766.33 / 16,632.7 | — | — | — |
| Arm-optimized | 70 | 32,427.135 | 8,945.05 | 70 | 10,355.458 | 9,071.62 | 70 | 3,868.159 / 132.523 | 8,945.14 / 8,998.62 | — | — | — | |
llama3_2_1b_instruct | Baseline | 62 | 11,747.31 | 3,587.52 | 62 | 2,278.4 | 3,632 | 62 | 1,506.208 / 315.41 | 3,648.88 / 3,458.31 | 62 | 5,465.451 / 549.061 | 3,639.61 / 3,448.92 |
| Arm-optimized | 61 | 5,372.255 | 2,074.73 | 61 | 1,343.313 | 2,171.72 | 61 | 872.176 / 37.848 | 2,103.63 / 2,040.49 | 61 | 2,934.222 / 162.792 | 2,053.83 / 2,019.06 | |
llama3_8b | Baseline | 70 | — | — | 70 | 8,325.5 | 15,319 | 70 | 6,481.827 / 1,145.043 | 16,767.05 / 16,633.73 | — | — | — |
| Arm-optimized | 70 | 33,048.8 | 8,945.09 | 70 | 10,418.618 | 9,071.28 | 70 | 3,868.765 / 136.446 | 8,945.58 / 8,999.95 | — | — | — | |
ministral_3b | Baseline | 41 | 23,757.14 | 6,720.25 | 41 | 3,544.4 | 7,068 | 41 | 2,842.68 / 392.27 | 6,907.79 / 6,742.66 | 41 | 9,099.26 / 958.772 | 6,481.7 / 6,612.25 |
| Arm-optimized | 41 | 14,690.979 | 3,613.7 | 41 | 4,111.838 | 3,735.47 | 41 | 1,606.985 / 58.307 | 3,615.26 / 3,594.89 | 41 | 6,306.724 / 307.102 | 3,547.59 / 3,558.29 | |
phi3_mini_3_8b | Baseline | 66 | — | — | 66 | 4,899.6 | 7,787 | — | — | — | 66 | 39,403.142 / 23,809.737 | 6,920.5 / 6,380.78 |
| Arm-optimized | 64 | 20,263.64 | 4,196.3 | 64 | 5,677.8 | 4,188 | — | — | — | 64 | 17,359.458 / 1,403.964 | 4,519.93 / 4,955.26 | |
phi_3_5_mini | Baseline | 67 | 33,931.92 | 7,668.7 | 67 | 4,909 | 7,787 | 67 | 6,537.464 / 806.297 | 8,000.85 / 8,312.6 | 67 | 31,384.925 / 11,620.851 | 7,859.54 / 8,301.61 |
| Arm-optimized | 66 | 25,879.87 | 5,780.06 | 66 | 5,136.12 | 5,939.16 | 66 | 4,515.18 / 130.063 | 5,166.86 / 5,755.88 | 66 | 12,746.922 / 559.287 | 5,068.25 / 5,742.02 | |
phi_4_mini | Baseline | 62 | 30,765.52 | 10,283.91 | 62 | 4,636 | 10,057 | — | — | — | 62 | x | x |
| Arm-optimized | 61 | 19,252.34 | 5,892.36 | 61 | 5,965.48 | 6,015.31 | — | — | — | 61 | 12,484.546 / 1,340.319 | 5,800.06 / 5,721.44 | |
qwen2_5_0_5b_instruct | Baseline | 48 | 5,258.66 | 1,661.23 | 48 | 1,178 | 1,687 | — | — | — | 48 | 3,546.845 / 741.85 | 1,627.28 / 1,505.29 |
| Arm-optimized | 51 | 5,094.83 | 1,233.97 | 51 | 1,126.004 | 1,097.88 | — | — | — | 51 | 3,319.055 / 289.218 | 1,028.61 / 908.96 | |
qwen3_1_7b | Baseline | 47 | 16,095.14 | 4,852.91 | 47 | 2,724.8 | 4,732 | — | — | — | 47 | 8,442.312 / 960.314 | 4,769.74 / 4,721.67 |
| Arm-optimized | 47 | 11,061.32 | 2,770.42 | 47 | 3,080.155 | 3,050.81 | — | — | — | 47 | 8,159.073 / 834.423 | 2,832.29 / 2,791 | |
smollm2 | Baseline | 49 | 4,038.865 | 525.83 | 49 | 726.188 | 583.59 | — | — | — | — | — | — |
| Arm-optimized | 47 | 1,943.872 | 273.17 | 47 | 398.71 | 322.84 | — | — | — | — | — | — | |
smollm3 | Baseline | 68 | 23,028.77 | 7,113.31 | 68 | 4,023.8 | 7,208 | 68 | 3,799.291 / 503.508 | 7,152.56 / 7,050.69 | 68 | 11,526.219 / 1,226.412 | 7,060.18 / 7,040.76 |
| Arm-optimized | 69 | 13,386.304 | 3,930.92 | 69 | 3,809.329 | 3,946 | 69 | 2,246.898 / 79.078 | 3,934 / 3,954.05 | 69 | 7,795.432 / 342.594 | 3,839.17 / 3,862.35 | |
tinyllama_1_1b_chat | Baseline | 63 | 11,135.85 | 2,522.75 | 63 | 2,344.2 | 2,507 | 63 | 1,807.454 / 303.46 | 2,519.68 / 2,307.49 | 63 | 5,945.782 / 687.054 | 2,506.92 / 2,294.92 |
| Arm-optimized | 62 | 5,976.751 | 1,321.8 | 62 | 1,377.354 | 1,368.67 | 62 | 872.155 / 42.282 | 1,322.95 / 1,259.1 | 62 | 2,952.479 / 185.398 | 1,294.73 / 1,241.68 |
Text embedding
Dataset: BEIR
| Model | Device | RPi5 | M4 Macbook | vivo X300 Pro | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Type | Recall@10 | Latency (ms) | Memory (MB) | Recall@10 | Latency (ms) | Memory (MB) | nDCG@10 | Latency (ms) | Memory (MB) | |
bge_base_en_v1_5 | Baseline | 87.42 | 1,360.84 | 289.02 | 87.42 | 297.1 | 300 | 0.7404 | 502.616 | 494.05 |
| Arm-optimized | 87.52 | 1,318.27 | 216.16 | 87.52 | 308.1 | 221 | 0.7416 | 205.071 | 244.08 | |
e5_base_v2 | Baseline | 84.65 | 1,377.56 | 289 | 84.65 | 299.2 | 301 | 0.7211 | 509.178 | 494.55 |
| Arm-optimized | 84.14 | 1,309.08 | 216.22 | 84.14 | 319.64 | 227.2 | 0.7194 | 231.821 | 169.98 | |
embeddinggemma_300m | Baseline | — | — | — | — | — | — | 0.7821 | 555.83 | 1,245.62 |
| Arm-optimized | — | — | — | — | — | — | 0.7583 | 273.232 | 922.24 | |
gte_base | Baseline | 89.02 | 1,548.39 | 289.02 | 89.02 | 295.9 | 300 | 0.7613 | 518.018 | 495.14 |
| Arm-optimized | 88.29 | 1,441.4 | 169.75 | 88.29 | 397.6 | 181.94 | 0.7592 | 311.672 | 136.05 | |
minilm_l6_v2 | Baseline | 80.81 | 501.3 | 100.58 | 80.81 | 116.2 | 118 | 0.6543 | 153.629 | 156.01 |
| Arm-optimized | 79.64 | 500.77 | 92.66 | 79.64 | 118.5 | 107 | 0.6536 | 56.67 | 66.52 | |
nomic_embed_text_v1 | Baseline | 83.12 | 2,629.28 | 654.2 | 83.12 | 283.9 | 683 | 0.7045 | 626.472 | 613.57 |
| Arm-optimized | 83.12 | 1,381.42 | 286.77 | 83.12 | 247.6 | 308 | 0.707 | 436.096 | 194.67 |
Text classification
| Model | Device | vivo X300 Pro | ||
|---|---|---|---|---|
| Type | Accuracy (%) | Latency (ms) | Memory (MB) | |
deberta_v3_small_mnli | Baseline | 88.12 | 1,425.423 | 930.91 |
| Arm-optimized | 88.2 | 1,365.011 | 353.68 | |
roberta_base_sst2 | Baseline | 93.578 | 610.265 | 316.56 |
| Arm-optimized | 93.46 | 226.428 | 195.88 |
Audio models
Speech-to-text
Dataset: LibriSpeech
| Model | Device | AWS Graviton4 | vivo X300 Pro | ||||
|---|---|---|---|---|---|---|---|
| Type | WER | Latency encoder / decoder_prefill / decoder_decode (ms) | Memory encoder / decoder_prefill / decoder_decode (MB) | WER | Latency encoder / decoder_prefill / decoder_decode (ms) | Memory encoder / decoder_prefill / decoder_decode (MB) | |
conformer_ctc | Baseline | — | — | — | 0.056 | 11,041.576 | 4,189.36 |
| Arm-optimized | — | — | — | 0.0566 | 7,218.498 | 2,159.28 | |
executorch_conformer | Baseline | — | — | — | 0.2213 | 87.934 | 95.68 |
| Arm-optimized | — | — | — | 0.2215 | 78.245 | 80.25 | |
qwen3_asr_0_6b | Baseline | 0.024 | 219.968 / 1,003.442 / 98.082 | 1,658.22 / 2,159.13 / 4,058.66 | — | — | — |
| Arm-optimized | 0.0245 | 74.836 / 748.278 / 80.534 | 1,083.98 / 670.31 / 1,728.44 | — | — | — | |
whisper_base | Baseline | — | — | — | 0.0485 | 450.377 / 539.387 / 35.631 | 328.22 / 471.32 / 496.85 |
| Arm-optimized | — | — | — | 0.0485 | 282.94 / 501.302 / 17.167 | 276 / 247 / 307.57 |
Audio classification
| Model | Device | RPi5 | AWS Graviton4 | ||||
|---|---|---|---|---|---|---|---|
| Type | mAP (%) | Latency (ms) | Memory (MB) | mAP (%) | Latency (ms) | Memory (MB) | |
yamnet | Baseline | 34.7232 | 2.22 | 28.67 | 34.7232 | 0.801 | 30.43 |
| Arm-optimized | 35.153 | 0.989 | 11.14 | 34.7209 | 0.719 | 30.3 |
Multimodal models
Vision-language
| Model | Device | RPi5 | AWS Graviton4 | ||||
|---|---|---|---|---|---|---|---|
| Type | Pair / Ques | Latency encoder / prefill / decode (ms) | Memory encoder / prefill / decode (MB) | Pair / Ques | Latency encoder / prefill / decode (ms) | Memory encoder / prefill / decode (MB) | |
llava1_5_7b | Baseline | 0.28 / 0.61 | 3,090.06 / 86,312.45 / 4,046.83 | 801.45 / 13,468.97 / 14,040.47 | 0.28 / 0.61 | 452.35 / 12,542.61 / 836.53 | 804.55 / 13,617.37 / 14,107.49 |
| Arm-optimized | 0.3 / 0.6 | 1,780.89 / 43,844.34 / 1,472.42 | 743.3 / 7,026.23 / 7,819.66 | 0.3 / 0.6 | 177.63 / 8,636.66 / 140.49 | 719.7 / 7,027.14 / 7,800.17 | |
qwen2_5_vl_7b_instruct | Baseline | — | — | — | 0.46 / 0.72 | x / 4,042.26 / 313.91 | x / 14,689.82 / 15,663.59 |
| Arm-optimized | — | — | — | 0.46 / 0.72 | 1,704.02 / 2,266.59 / 62.46 | 1,878.84 / 7,410.25 / 8,406.52 |
Image-text matching
Dataset: COCO Captions
| Model | Device | RPi5 | AWS Graviton4 | ||||
|---|---|---|---|---|---|---|---|
| Type | I2T Recall@1 | Latency (ms) | Memory (MB) | I2T Recall@1 | Latency (ms) | Memory (MB) | |
clip | Baseline | 0.1012 | 132.416 | 316.59 | 0.1012 | 39.445 | 318.16 |
| Arm-optimized | 0.1004 | 83.685 | 189.39 | 0.1004 | 25.391 | 193.65 |
Using your own model
Bring your own LLM text-generation model with np models register, or your own dataset for any task with np datasets register. See Bring Your Own Model (BYOM) and Bring Your Own Data (BYOD) formats and Tutorial 4: Bring Your Own Model (BYOM) and Bring Your Own Data (BYOD).