Profiler

The Profiler measures model latency and memory usage. The supported measurement method, --method mlia, provides a static estimate using the Arm Machine Learning Inference Advisor (MLIA).

How profiling works

Profiling runs automatically as part of np run when you define a profile section in run.yaml. The Profiler uses the target device that you configure when you create the project and follows a compile → profile sequence.

Pipeline position

The Profiler sets compile options automatically based on the target device for the project. These options are not exposed in run.yaml. You can configure profile execution options in run.detail.yaml, for example the number of runs.

Measurement method

Method	Meaning	Notes
mlia	Static estimate using the Arm Machine Learning Inference Advisor (MLIA)	xnnpack targets report MACs and parameters only, and do not report latency or memory usage. Targets that MLIA can fully estimate, for example Ethos-U, report the full set.

The report omits a column if the backend combination does not provide a value. When a combination cannot produce a metric, a warning explains the supported scope and the missing metrics. See Concepts: Compatibility matrix for method availability by device and backend.

YAML configuration

The profile section lists which model checkpoints to measure. To disable profiling for a specific checkpoint, omit it from the list. To skip profiling entirely, remove the profile section.

profile:
- INPUT_MODEL         # original model before optimization
- ADVANCED_QUANTIZE   # after AQ step
- GRAPH_OPTIMIZE      # after GO step
- GRAPH_QUANTIZE      # after GQ step

Model reference names

Name	Description
INPUT_MODEL	Original model before optimization
ADVANCED_QUANTIZE	After Advanced Quantize
GRAPH_OPTIMIZE	After Graph Optimize
GRAPH_QUANTIZE	After Graph Quantize
OUTPUT_MODEL	Final output of the last step

Reference names are case-insensitive.

Profiling examples

GO + GQ

version: "1.0"

steps:
- graph_optimize: null
- graph_quantize:
  - scheme: W8-A8
    automatic_mixed_precision: null

profile:
- INPUT_MODEL
- GRAPH_OPTIMIZE
- OUTPUT_MODEL

AQ + GO + GQ

version: "1.0"

steps:
- advanced_quantize:
  - algorithm: AWQ
    scheme: W4-A8_dynamic
- graph_optimize: null
- graph_quantize:
  - scheme: W8-A8
    automatic_mixed_precision: null

profile:
- INPUT_MODEL
- ADVANCED_QUANTIZE
- GRAPH_OPTIMIZE
- OUTPUT_MODEL

Selective profiling

To skip profiling for specific checkpoints, omit them from the list:

profile:
- INPUT_MODEL
# - GRAPH_OPTIMIZE   # omitted → not profiled
- GRAPH_QUANTIZE

Configure advanced profile options

Configure profile execution options in run.detail.yaml. Each profile[N] entry corresponds to the Nth item in the profile list in run.yaml.

run.detail.yaml

detail:
  profile[0]:   # INPUT_MODEL
    num_executions: 30
    timeout: 60

  profile[1]:   # GRAPH_OPTIMIZE
    num_executions: 30
    timeout: 60

  profile[2]:   # GRAPH_QUANTIZE
    num_executions: 30
    timeout: 60

Parameter	Default	Description
num_executions	30	Number of inference runs used to compute average latency
timeout	60	Maximum measurement time in seconds before returning partial results

Increasing num_executions reduces measurement variance but increases profiling time.

Log output

============================================================
[Phase 4] Profiling
============================================================
2026-08-20 15:40.00 [INFO ] [Profiler] Target: raspberry_pi_5 / executorch / xnnpack
2026-08-20 15:40.01 [INFO ] [Profiler] Compiling: input_model
2026-08-20 15:40.18 [INFO ] [Profiler] Profiling: input_model (30 runs)
2026-08-20 15:40.45 [INFO ] [Profiler]   Latency: 24.71 ms (avg), 24.13 ms (min), 25.44 ms (max)
2026-08-20 15:40.45 [INFO ] [Profiler]   Memory:  45.2 MB (peak)
2026-08-20 15:40.46 [INFO ] [Profiler] Compiling: graph_optimize
2026-08-20 15:40.59 [INFO ] [Profiler] Profiling: graph_optimize (30 runs)
2026-08-20 15:41.18 [INFO ] [Profiler]   Latency: 18.34 ms (avg), 17.92 ms (min), 18.81 ms (max)
2026-08-20 15:41.18 [INFO ] [Profiler]   Memory:  44.8 MB (peak)
2026-08-20 15:41.19 [INFO ] [Profiler] Compiling: graph_quantize
2026-08-20 15:41.31 [INFO ] [Profiler] Profiling: graph_quantize (30 runs)
2026-08-20 15:41.48 [INFO ] [Profiler]   Latency:  8.92 ms (avg),  8.71 ms (min),  9.14 ms (max)
2026-08-20 15:41.48 [INFO ] [Profiler]   Memory:  12.3 MB (peak)
2026-08-20 15:41.48 [INFO ] [Profiler] Profiling complete. Results saved to report.json

View results

np report

The report table shows latency and memory values for each model checkpoint, measured using MLIA. Post-hoc np profile runs appear in their own table for each execution, with columns for the device, runtime, backend, and method. See CLI reference: np report.

Notes

• Omitting the profile section skips profiling entirely.
• To measure directly on physical hardware outside np profile, see On-device profiling.
• Profiler stores results in report.json at the experiment level and in job.json for each step.
• Arm AI Model Optimizer can reuse an earlier mlia estimate for the same model and device combination rather than recomputing it. Use --force to measure again.
• The timeout applies only to measurement time, not device wait time. If device queuing takes longer than expected, Profiler prints a warning with an estimated wait time.