Tutorial 4: Bring Your Own Model (BYOM) and Bring Your Own Data (BYOD)

This section provides an end-to-end walkthrough for registering a model you provide (BYOM) and a dataset you provide (BYOD), then running the optimization pipeline against them.

What you'll build

A custom Qwen/Qwen2.5-0.5B-Instruct text-generation model and matching calibration and evaluation datasets, registered from scratch and run through the AQ + GO + GQ pipeline.

Step 1. Register your model

Model registration takes a single recipe file, model_register.yaml. model is a Hugging Face repository ID, and the registered model's name is derived from this value and is not a separate field. The model list for this release includes large language model (LLM) text-generation only. Every model on it uses the same generate pipeline. See Bring Your Own Model (BYOM) and Bring Your Own Data (BYOD) formats for the full list.

model_register.yaml

model: Qwen/Qwen2.5-0.5B-Instruct   # HF repo id
domain: LLM
task: TEXT_GENERATION
pipeline: generate                  # Graph Extractor trace entry function (HF only) — always include this for LLMs
inputs:                             # HF only, required
  input_ids:
    dtype: int64
    dims: {0: 1, 1: 128}

Register with --wait so the command blocks until Graph Extractor verification finishes, instead of just uploading and returning immediately:

np models register model_register.yaml --wait

Sample output

2026-08-22 13:15.00 [INFO ] [CLI] Registering Model
2026-08-22 13:15.00 [INFO ] [CLI] Reading model metadata from the Hugging Face Hub...
2026-08-22 13:15.01 [INFO ] [CLI] Registering the model on the server...
2026-08-22 13:15.02 [INFO ] [CLI] Uploading model_register.yaml to the server...
2026-08-22 13:15.02 [INFO ] [CLI] Validating model_register.yaml on the server...
2026-08-22 13:15.03 [INFO ] [CLI] Downloading model weights from the Hugging Face Hub (several GB — this can take a while)...
2026-08-22 13:16.30 [INFO ] [CLI] Bundling the weights into a single tar archive...
2026-08-22 13:16.42 [INFO ] [CLI] Bundled the weights into a tar archive (953.3 MiB).
2026-08-22 13:16.42 [INFO ] [CLI] Uploading the weights to the server...
2026-08-22 13:17.00 [INFO ] [CLI] Uploading weights... 0% (16.0 KiB / 953.3 MiB)
2026-08-22 13:17.20 [INFO ] [CLI] Uploading weights... 79% (754.7 MiB / 953.3 MiB)
2026-08-22 13:17.38 [INFO ] [CLI] Confirming the upload with the server...

============================================================
Summary
============================================================
2026-08-22 13:17.40 [DONE ] [CLI] Model registered and verified successfully.

[ Result Summary ]
 Model     Qwen/Qwen2.5-0.5B-Instruct
 Ref ID    6f708192-0314-4d5e-af60-7182934b5c6d
 Status    uploaded
 Revision  7ae557604adf67be50417f59c2c2f167def9a775
 Uploaded  953.3 MiB

[ Next Steps ]
Create a project with this model:
  $ np projects create --model Qwen/Qwen2.5-0.5B-Instruct

The full round trip (weight download, upload, and Graph Extractor verification) takes a few minutes for a model this size. Larger models take longer. Even after the command above finishes, give np models ls a little time before it shows Usable. Check back with np models ls --filter type=CUSTOM rather than assuming it is ready as soon as the command returns.

For a gated Hugging Face repository (most meta-llama or google cards), accept the license on the card's page, then provide HF_TOKEN:

export HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxx
np models register model_register.yaml --wait

Step 2. Register your dataset

Sampling happens outside Arm AI Model Optimizer: you prepare your calibration and evaluation data yourself. For a plain-text corpus like this one, that means the plain_text format, with non-empty .txt files laid flat at the top level, and 1 document per file:

my_corpus_calib_200/
├─ sample_00001.txt
├─ sample_00002.txt
├─ sample_00003.txt
└─ ... (200 files total)

sample_00001.txt

The city council approved the new transit funding plan on Tuesday after
months of public hearings. Officials estimate the expanded bus routes
will begin service early next year, pending final budget approval ...

Filenames are up to you; sample_0000N.txt is just convention, not a requirement. Subfolders are not scanned, so keep every file at the top level. Each one needs actual content (an empty .txt is rejected). my_corpus_eval_500 follows the same layout. Validation reads every file in the folder (not just a sample of them), and it also expects at least a minimum number of usable rows to come out the other end: a handful of tiny files can fail with something like "Filtered dataset rows are less than config num_samples" even though the format itself is correct, so keep folders in roughly the tens-of-files range rather than just two or three. Keep calibration and evaluation folders disjoint yourself; Arm AI Model Optimizer does not check for overlap between them.

Register calibration and evaluation as 2 separate calls.

np datasets register --source ./my_corpus_calib_200 --calibration \
  --task TEXT_GENERATION --raw-data plain_text --name my-corpus-calib

np datasets register --source ./my_corpus_eval_500 --evaluation \
  --task TEXT_GENERATION --raw-data plain_text --name my-corpus-eval

Registration is synchronous: each call uploads, validates, and reports success or failure immediately. There is no job to poll for.

Sample output

============================================================
Summary
============================================================
2026-08-22 13:42.15 [DONE ] [CLI] Dataset registered successfully.

[ Result Summary ]
  Sample    my-corpus-calib
  Ref ID    a1b2c3d4-5e6f-7890-abcd-ef1234567890
  Split     CALIBRATION
  Tasks     Text Generation
  Uploaded  4.8 MB
  Status    CREATED

[ Next Steps ]
Create a project with this dataset:
   np projects create --calibration-dataset my-corpus-calib ...

Field	Required	Description
--source	Yes	Path to your already-sampled data folder
--calibration / --evaluation	Yes	Purpose (mutually exclusive); determines which validation rules apply
--task	Yes	Task type, UPPER_SNAKE_CASE (for example TEXT_GENERATION)
--raw-data	Yes	Which folder format you used; checked against --task before anything uploads
--name	No	Auto-generated if omitted

When registered, your dataset shows up in np datasets ls alongside Preset datasets (as a CUSTOM row with Seed 0 / Strategy FULL / Shuffle False, since you did your own sampling and the server takes the folder as given), and can be passed to np projects create --calibration-dataset or --evaluation-dataset just like a preset.

--raw-data is checked twice: once before upload (does the format name exist, and can it be read as your --task? An instant HTTP 400 follows if not, before any data transfers), and once after upload (does the folder's actual structure match that format? This is a real dry-run through the loader). A folder that fails the second check comes back as a clear rejection rather than a silent failure:

np datasets register --source ./my_corpus_calib_200 --calibration \
  --task TEXT_GENERATION --raw-data plain_text --name my-corpus-calib

Sample output: rejected

Here the .txt files were left inside a subfolder instead of at the top level, the pitfall called out above:

2026-08-22 13:43.00 [INFO ] [CLI] Registering Dataset
2026-08-22 13:43.00 [INFO ] [CLI] Uploading the archive to the server...
2026-08-22 13:43.01 [INFO ] [CLI] Registering and validating on the server. Validation reads every sample once per task, so this can take several minutes - do not interrupt it.

============================================================
Registration rejected
============================================================

[ Task results ]
 Text Generation  REJECT   TEXT_GENERATION: LlmPreprocessConfig.data_path must point to a
                  HuggingFace Arrow dataset or a directory containing .txt files | hints:
                  missing entries against the standard structure of the declared raw-data
                  format 'plain_text' — Top-level *.txt files (or HF Arrow's
                  dataset_dict.json/state.json) (candidates: *.txt, dataset_dict.json,
                  state.json) | current top level: texts/

[ Validation details ]
 Samples counted by the server  0
 Engine code                    NP-SDV-422
 Sample ref ID                  <ref_id>
2026-08-22 13:43.01 [WARN ] [CLI] The archive was uploaded but the sample was NOT registered, so it cannot be used yet.

[ Next Steps ]
Fix the folder for the rejected tasks above, then register again:
  $ np datasets register --source ./my_corpus_calib_200 --calibration --task TEXT_GENERATION --raw-data plain_text --name my-corpus-calib

2026-08-22 13:43.01 [ERROR] [CLI] [NP-CAP-400] Backend error (HTTP 400, code=FRS_007): np-engine rejected the dataset validation request (422). REJECT
2026-08-22 13:43.01 [WARN ] [CLI] Hint: The sample stays unusable until every --task passes. Fix the folder for the rejected tasks and run the same command again - re-registering the same archive is idempotent, so an unchanged folder is not uploaded twice.

The hints: segment is the useful part: it names what the format expects (candidates:) next to what your folder actually has (current top level:). This means that the fix is usually visible without opening the format reference. Note that the command exits with an error and a rejected registration does not show up in np datasets ls.

Re-registering the exact same folder with the exact same options is detected by content fingerprint and just returns the existing dataset rather than creating a duplicate. Change any option (a different --task, a different folder, and so on) and it registers as a new one.

Step 3. Verify what you registered

Both registrations show up in the workspace listings. --filter type=CUSTOM narrows each one to what you registered yourself, leaving out the presets.

np models ls --filter type=CUSTOM

Sample output

2026-08-22 13:45.10 [INFO ] [CLI] Listing Models
                                                  Models (1)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┓
┃ Name                       ┃ Type   ┃ Domain ┃ Task            ┃ Source      ┃ Format ┃ Framework ┃ Status ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━┩
│ Qwen/Qwen2.5-0.5B-Instruct │ CUSTOM │ LLM    │ Text Generation │ HUGGINGFACE │ HF     │ PYTORCH   │ Usable │
└────────────────────────────┴────────┴────────┴─────────────────┴─────────────┴────────┴───────────┴────────┘

[ Next Steps ]
Create a project with a model:
  $ np projects create --model <name>

Status is the gate: only a Usable model can go into a project. Step 1 registered with --wait, so Graph Extractor verification had already finished by the time the command returned. Without --wait, np models register comes back as soon as the upload completes and verification keeps running in the background, so the row sits at a non-Usable status for a while. Creating a project against it then fails with a specific explanation rather than a generic error:

[ERROR] [CLI] [NP-CAP-400] Backend error (HTTP 400, code=BAD_REQUEST):
Model 'Qwen/Qwen2.5-0.5B-Instruct' cannot be used yet: its weight upload is
not complete (upload_status=UPLOADED). Finish uploading the model weights
and confirm the upload before creating a project with it.

np datasets ls --filter type=CUSTOM

Sample output

2026-08-22 13:45.22 [INFO ] [CLI] Listing Datasets
                                                      Datasets (2)
┏━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓
┃ Name            ┃ Type   ┃ Task            ┃ Split       ┃ Raw Data   ┃ Samples ┃ Seed ┃ Strategy ┃ Shuffle ┃ Status ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩
│ my-corpus-calib │ CUSTOM │ Text Generation │ CALIBRATION │ plain_text │     200 │    0 │ FULL     │ False   │ Usable │
│ my-corpus-eval  │ CUSTOM │ Text Generation │ EVALUATION  │ plain_text │     500 │    0 │ FULL     │ False   │ Usable │
└─────────────────┴────────┴─────────────────┴─────────────┴────────────┴─────────┴──────┴──────────┴─────────┴────────┘

[ Next Steps ]
Create a project with a dataset:
  $ np projects create --calibration-dataset <name> --evaluation-dataset <name>

CUSTOM rows read Seed 0 / Strategy FULL / Shuffle False: you sampled the data yourself, so the server takes the folder as given instead of drawing from it. Dataset registration is synchronous, so anything listed here already passed validation.

Step 4. Create and run a project

np projects create \
  --name my-llm-project \
  --model "Qwen/Qwen2.5-0.5B-Instruct" \
  --device raspberry_pi_5_16gb --runtime executorch --backend xnnpack \
  --calibration-dataset my-corpus-calib \
  --evaluation-dataset my-corpus-eval

cd my-llm-project
np experiments create --steps aq,go,gq
cd experiment_00
np run --wait
np report

Next steps

• Bring Your Own Model (BYOM) and Bring Your Own Data (BYOD) formats: how validation and the Model List work internally, plus the exact folder layout for every supported dataset task