run.yaml reference

run.yaml defines the optimization pipeline for an experiment. It is auto-generated by np experiments create and you can edit it before running.

For fine-grained parameter overrides, use run.detail.yaml in the same directory.

File structure

run.yaml
version: "1.0"          # required

steps:                  # required — optimization steps
  - ...

profile:                # optional — checkpoints to measure latency
  - ...

evaluate:               # optional — checkpoints to evaluate
  - ...

`steps`

Defines the optimization steps to run. Supported combinations:

`--steps`	Pipeline
`go,gq`	GO → GQ
`aq,go,gq`	AQ → GO → GQ
`aq` / `go` / `gq`	Single module run

`advanced_quantize`

- advanced_quantize:
  - algorithm: AWQ  # Available: RTN, AWQ, AUTOROUND, SMOOTHQUANT, GPTQ
    scheme: W8-A8  # Available: W8-A8, W8-A8_dynamic, W4-A8_dynamic

Field	Required	Options
`algorithm`	Yes	`AWQ`, `AUTOROUND`, `RTN`, `SMOOTHQUANT`, `GPTQ`
`scheme`	Yes	`W8-A8`, `W8-A8_dynamic`, `W4-A8_dynamic`

`graph_optimize`

No parameters required:

- graph_optimize:

`graph_quantize`

- graph_quantize:
  - scheme: W8-A8  # Available (XNNPACK): W8-A8, W8-A8_dynamic, W4-A8_dynamic / (Ethos-U): W8-A8, W8-A16
    automatic_mixed_precision: null  # Available: GREEDY

Field	Required	Options
`scheme`	Yes	`W8-A8`, `W8-A8_dynamic`, `W4-A8_dynamic` (XNNPACK) / `W8-A8`, `W8-A16` (Ethos-U)
`automatic_mixed_precision`	No	`GREEDY` enables AMP; `null` disables it

Available schemes depend on the target hardware backend. See Graph Quantizer.

Multi-component models

Some models trace to more than one independent graph: the clearest example is an LLM whose prefill and decode phases execute along different paths. For these, the graph_quantize step carries a components: override alongside the model-level default, and quantizes each component with its own settings:

- graph_quantize:
  - scheme: W4-A8_dynamic
    automatic_mixed_precision: null
    components:
      prefill:
        scheme: W4-A8_dynamic
        automatic_mixed_precision: null
      decode:
        scheme: W4-A8_dynamic
        automatic_mixed_precision: null

np models register determines the component names and count during registration. You do not define them in run.yaml. Changing the model-level default (scheme) still applies to any component that doesn't have its own override.

advanced_quantize is generated with a components: block too, but it is not honored yet. AQ quantizes the model in one pass using the model-level algorithm and scheme, so changing the values nested under advanced_quantize.components has no effect on the result. Leave them at the model-level default. Per-component AQ settings are planned for a future release. graph_optimize gets no components: block at all. It runs once for each component from a single model-level setting.

Single-component models, for example most computer vision (CV) models, never get a components: block. Everything in this section applies only to models the trace split into more than one component. For a whole file with these blocks in place, see AQ + GO + GQ (Multi-Component) below.

Shape handling

This release supports static shapes only. There is no shape configuration in run.yaml. Graph Extractor resolves every input to a fixed shape during np models register, using the inputs you declared in model_register.yaml. By the time a model reaches run.yaml, its shapes are already fixed. There is no shape-override mechanism here or in run.detail.yaml, and shapes are never echoed back for confirmation. If a model needs different input shapes, reregister it with an updated model_register.yaml.

`profile`

Lists model checkpoints for latency measurement. profile uses Arm Machine Learning Inference Advisor (MLIA) static estimation. Always pass --method mlia on post-hoc np profile calls. See Profiler for more details.

profile:
- INPUT_MODEL
- GRAPH_OPTIMIZE
- OUTPUT_MODEL

Remove the profile section entirely to skip profiling.

`evaluate`

Lists model checkpoints to evaluate. Each entry declares a model and the metrics to compute.

evaluate:
- model: INPUT_MODEL
  metric:
  - SNR
- model: GRAPH_OPTIMIZE
  metric:
  - SNR
- model: OUTPUT_MODEL
  metric:
  - SNR
  - TOP1_ACCURACY

Signal-to-noise ratio (SNR) compares against INPUT_MODEL by default. To change the comparison target, use run.detail.yaml.

Available metrics

The available metrics depend on the model's task. See Evaluator: Supported metrics for the available metrics, applicable tasks, and descriptions.

Comparison metrics (SNR, MSE, and the rest) measure the gap between two checkpoints, need no dataset, and work for any model.
Independent metrics measure a single checkpoint's absolute accuracy, require an evaluation dataset registered with the project, and are only meaningful for the matching task.

Complete examples

GO + GQ

run.yaml
version: '1.0'
steps:
- graph_optimize: null
- graph_quantize:
  - scheme: W8-A8  # Available: W8-A8, W8-A8_dynamic, W4-A8_dynamic
    automatic_mixed_precision: null  # Available: GREEDY
profile:
- INPUT_MODEL
- GRAPH_OPTIMIZE
- OUTPUT_MODEL
evaluate:
- model: INPUT_MODEL
  metric:
  - SNR
- model: GRAPH_OPTIMIZE
  metric:
  - SNR
- model: OUTPUT_MODEL
  metric:
  - SNR

AQ + GO + GQ

run.yaml
version: '1.0'
steps:
- advanced_quantize:
  - algorithm: AWQ  # Available: RTN, AWQ, AUTOROUND, SMOOTHQUANT, GPTQ
    scheme: W8-A8  # Available: W8-A8, W8-A8_dynamic, W4-A8_dynamic
- graph_optimize: null
- graph_quantize:
  - scheme: W8-A8  # Available (XNNPACK): W8-A8, W8-A8_dynamic, W4-A8_dynamic / (Ethos-U): W8-A8, W8-A16
    automatic_mixed_precision: null  # Available: GREEDY
profile:
- INPUT_MODEL
- ADVANCED_QUANTIZE
- GRAPH_OPTIMIZE
- OUTPUT_MODEL
evaluate:
- model: INPUT_MODEL
  metric:
  - SNR
  - PPL
- model: ADVANCED_QUANTIZE
  metric:
  - SNR
  - PPL
- model: GRAPH_OPTIMIZE
  metric:
  - SNR
- model: OUTPUT_MODEL
  metric:
  - SNR
  - PPL

AQ + GO + GQ (multi-component)

The following example shows what np experiments create generates for a model that traces into more than one component, for example an LLM with separate prefill and decode graphs. The model-level algorithm and scheme stay as the default. Each component repeats them so you can diverge per component without touching the other.

run.yaml
version: '1.0'
steps:
- advanced_quantize:
  - algorithm: AWQ  # Available: RTN, AWQ, AUTOROUND, SMOOTHQUANT, GPTQ
    scheme: W4-A8_dynamic  # Available: W4-A8_dynamic, W8-A8, W8-A8_dynamic
    components:
      prefill:
        algorithm: AWQ
        scheme: W4-A8_dynamic
      decode:
        algorithm: AWQ
        scheme: W4-A8_dynamic
- graph_optimize: null
- graph_quantize:
  - scheme: W4-A8_dynamic  # Available: W4-A8_dynamic, W8-A8, W8-A8_dynamic
    automatic_mixed_precision: null  # Available: GREEDY
    components:
      prefill:
        scheme: W4-A8_dynamic
        automatic_mixed_precision: null
      decode:
        scheme: W4-A8_dynamic
        automatic_mixed_precision: null
profile:
- INPUT_MODEL
- ADVANCED_QUANTIZE
- GRAPH_OPTIMIZE
- OUTPUT_MODEL
evaluate:
- model: INPUT_MODEL
  metric:
  - SNR
- model: ADVANCED_QUANTIZE
  metric:
  - SNR
- model: GRAPH_OPTIMIZE
  metric:
  - SNR
- model: OUTPUT_MODEL
  metric:
  - SNR

graph_optimize has no components: block even in this case: it runs once for each component, but takes a single model-level setting. See Multi-component models.

Single step: AQ only

run.yaml
version: '1.0'
steps:
- advanced_quantize:
  - algorithm: AWQ
    scheme: W8-A8
profile:
- input_model
- advanced_quantize
evaluate:
- model: input_model
  metric:
  - SNR
- model: advanced_quantize
  metric:
  - SNR

File structure​

steps​

advanced_quantize​

graph_optimize​

graph_quantize​

Multi-component models​

Shape handling​

profile​

evaluate​

Available metrics​

Complete examples​

GO + GQ​

AQ + GO + GQ​

AQ + GO + GQ (multi-component)​

Single step: AQ only​