Features

The Arm AI Model Optimizer command-line interface (CLI) uses a modular optimization pipeline. Each module handles a specific stage of the workflow, including model extraction, graph optimization, quantization, profiling, and evaluation.

Pipeline overview

The pipeline processes models through the Graph Extractor (GE), Graph Optimizer (GO), and Graph Quantizer (GQ). For Hugging Face large language models (LLMs), the Advanced Quantizer (AQ) can optionally run first. You can use the measurement and inspection modules at individual stages or after optimization.

Module summary

Module	Role	Use case
Advanced Quantizer	Quantizes model weights and activations at the PyTorch eager level	Activation-aware Weight Quantization (AWQ), AutoRound, Round-To-Nearest (RTN), SmoothQuant, GPTQ
Graph Extractor	Converts models to PyTorch 2 (PT2) format and traces the model structure during model registration	Model input, Bring Your Own Model (BYOM) registration
Graph Optimizer	Applies pattern-based graph transformations to improve hardware efficiency	All models
Graph Quantizer	Reduces precision at the intermediate representation (NPIR) level using calibration data	All models
Evaluator	Compares the accuracy of model checkpoints	Quality validation
Profiler	Measures latency and memory use for an on-device run or a Machine Learning Inference Advisor (MLIA) static estimate	Performance measurement
Visualizer	Provides browser-based graph inspection and checkpoint comparison	Model inspection, debugging
BYOM and BYOD formats	Defines the supported formats for bringing your own models and datasets into Arm AI Model Optimizer	BYOM, Bring Your Own Data (BYOD)

Supported pipelines

Steps	Pipeline
--steps go,gq	Graph Optimize → Graph Quantize
--steps aq,go,gq	Advanced Quantize → Graph Optimize → Graph Quantize