Graph Extractor

Graph Extractor converts input models into the intermediate representation (PT2 / NPIR). For Hugging Face models, the Graph Extractor runs automatically after the Advanced Quantizer. You do not invoke it directly.

Overview

When you use a Hugging Face model as input, the SDK automatically runs Graph Extractor to convert it to .pt2 format before passing it through the optimization pipeline. For models already in .pt2 format, no extraction is needed.

When Graph Extractor runs

Model input	Trigger
Hugging Face model	Runs automatically after AQ step
.pt2 file	Skipped: model is already in the required format

Pipeline position

Shape handling

When you run np models register, Graph Extractor resolves every tensor shape to a static value once at trace time. See run.yaml: Shape handling for the full explanation and how to change shapes afterward.

Multi-component models

Most models trace as a single graph, but some do not. When the execution path for a model branches, Graph Extractor traces it into multiple named components instead of one. This happens for models like:

• An LLM, split into prefill and decode
• A vision-language model, split into an image encoder and a language model
• An audio model, split into an encoder and a decoder

This split happens once automatically when you run np models register. You cannot configure the split. The trace determines the component names and number of components for each model.

Where it shows up

run.yaml lists the components that Graph Extractor found. run.detail.yaml includes a components: override block for each per-component step:

detail:
  graph_quantize[0]:
    scheme: W4-A8_dynamic
    components:
      prefill:
        scheme: W4-A8_dynamic
      decode:
        scheme: W4-A8_dynamic

• Only GQ (graph_quantize) applies its per-component settings and quantizes each component with its own values.
• AQ (advanced_quantize) receives an equivalent block, but does not apply it. Every component uses the model-level algorithm and scheme. Per-component AQ settings are planned for a future release.
• GO (graph_optimize) receives no components: block and runs once for each component from a single model-level setting.
• Target a single component directly with np analyze sensitivity <job_id> --component prefill or np visualize --job <job_id> --component prefill. Get the exact name from np jobs info.
• Single-component models never get a components: block. Everything above applies only to models that the trace split into more than one component.

What is not supported

Components come only from what the trace produces. You cannot manually request an additional split. In this release, components are independent and do not share or deduplicate weight tensors between them. A tensor used by both prefill and decode is stored once in each component, and the model exports one artifact for each component. This means that the deployed footprint is the sum across components rather than a deduplicated total.

YAML configuration

Graph Optimize requires no parameters in run.yaml. The Graph Extractor runs automatically based on the input model type.

steps:
- graph_optimize: null
- graph_quantize:
  - scheme: W8-A8
    automatic_mixed_precision: null

Notes

• Graph Extractor runs automatically and requires no configuration in run.yaml.