Advanced Quantizer
Advanced Quantizer (AQ) applies quantization algorithms at the PyTorch eager level before the model is converted to intermediate representation (NPIR). It uses weight-aware algorithms that outperform standard post-training quantization in preserving accuracy at aggressive bit widths.
Evaluator
The Evaluator measures the accuracy of model checkpoints produced during the optimization pipeline. It compares model outputs before and after each optimization step to show how optimization affects accuracy.
Graph Extractor
Graph Extractor converts input models into the intermediate representation (PT2 / NPIR). For Hugging Face models, the Graph Extractor runs automatically after the Advanced Quantizer. You do not invoke it directly.
Graph Optimizer
Graph Optimizer (GO) applies pattern-based transformations to the computation graph at the intermediate representation (NPIR) level. It restructures, simplifies, and rearranges operators to improve hardware execution efficiency without changing the mathematical output of the model.
Graph Quantizer
Graph Quantizer (GQ) reduces model precision at the intermediate representation (NPIR) level using calibration data. It converts floating-point weights and activations to integers with lower bit widths. This process enables faster inference and smaller model size while keeping accuracy degradation within acceptable bounds.
Features
The Arm AI Model Optimizer command-line interface (CLI) uses a modular optimization pipeline. Each module handles a specific stage of the workflow, including model extraction, graph optimization, quantization, profiling, and evaluation.
Profiler
The Profiler measures model latency and memory usage. The supported measurement method, --method mlia, provides a static estimate using the Arm Machine Learning Inference Advisor (MLIA).
Visualizer
np visualize renders the model graph in a browser-based viewer, with per-operator detail and side-by-side difference comparison between two model checkpoints.