Bring Your Own Model (BYOM) and Bring Your Own Data (BYOD) formats

Use model registration and dataset registration to bring your own models and datasets into Arm AI Model Optimizer. Registration runs once for each model or dataset and produces the metadata that subsequent commands such as np projects create and np run depend on.

Model registration (BYOM)

Registration replaces manual structure declaration with an automatic trace. np models register runs the Graph Extractor once, at registration time, against the Hugging Face repository ID that you provide. Arm AI Model Optimizer downloads the model weights itself as part of registration. The trace result is stored as a model bundle and reused by np experiments create to build run.yaml directly, without re-tracing.

Scope

BYOM registration is limited to large language model (LLM) text-generation models in this release.

Recipe fields

model_register.yaml takes 6 fields. See model_register.yaml reference for the full schema and a worked example.

Field	Required	Notes
model	Yes	Hugging Face repository ID (the source of the derived model name)
domain	Yes	LLM (fixed this release)
task	Yes	TEXT_GENERATION (fixed this release)
pipeline	Yes	Trace entry function name. Schema-optional (defaults to forward), but every model on the list needs generate
default_dtype	No	Default dtype for any inputs entry that omits one
inputs	Yes	Input tensors, keyed by name (see model_register.yaml reference)

Model name

There is no separate name field. The registered name is derived from model (used without modification, for example, meta-llama/Llama-3.2-1B), and a name collision within your workspace is an error.

Model list gate

You can register a Hugging Face model only if it appears on a server-managed model list. The list itself is not exposed in the CLI, and a model that is not on the list is rejected with a pointer to your account team's licensing process. From this release, the model list consists of LLM text-generation models only, all using the generate pipeline. See Models available for BYOM registration for more information. Local .pt2 files are not accepted for BYOM. Only the Hugging Face repository ID is accepted, because Graph Extractor tracing and license-provenance checks require the original model source.

2-phase model registration

np models register [path] [--wait] uploads the model and its recipe. Graph Extractor verification then runs asynchronously by default. Pass --wait to run verification synchronously. A model becomes usable only after verification passes.

np models register                       # looks for ./model_register.yaml
np models register ./recipes/qwen.yaml    # or an explicit path — any filename works
np models register ./recipes/qwen.yaml --wait   # block until verification finishes

Without --wait, the command returns as soon as the upload is complete. Verification continues in the background. np models ls shows the status of a model:

Status	Meaning	Usable in np projects create --model?
Uploading	Upload finished (or still in progress), verification pending or running	❌ Rejected
Usable	Verification passed	✅
Failed	Upload or verification failed (a missing pipeline field is a common cause)	❌ Reregister to fix

Trying to create a project against a model that is still Uploading fails with a specific, actionable error:

[ERROR] [CLI] [NP-CAP-400] Backend error (HTTP 400, code=BAD_REQUEST):
Model 'Qwen/Qwen2.5-0.5B-Instruct' cannot be used yet: its weight upload is
not complete (upload_status=UPLOADED). Finish uploading the model weights
and confirm the upload before creating a project with it.

If you registered with --wait, verification has already completed by the time the command returns, so you can go straight to np projects create. Without --wait, check np models ls for Usable first. Verification time varies with model size: a 0.5B model takes roughly 2 minutes, including a worker cold start.

Models available for BYOM registration

This section is grouped by model family.

Qwen2.5 (qwen2_5_0_5b_instruct, 13 variants)

Model ID	Size	Specialization	Tuning type	License
Qwen/Qwen2.5-0.5B	0.5B	General	Base	apache-2.0
Qwen/Qwen2.5-1.5B	1.5B	General	Base	apache-2.0
Qwen/Qwen2.5-1.5B-Instruct	1.5B	General	Instruct	apache-2.0
Qwen/Qwen2.5-3B	3B	General	Base	qwen-research
Qwen/Qwen2.5-3B-Instruct	3B	General	Instruct	qwen-research
Qwen/Qwen2.5-Coder-0.5B	0.5B	Coder	Base	apache-2.0
Qwen/Qwen2.5-Coder-0.5B-Instruct	0.5B	Coder	Instruct	apache-2.0
Qwen/Qwen2.5-Coder-1.5B	1.5B	Coder	Base	apache-2.0
Qwen/Qwen2.5-Coder-1.5B-Instruct	1.5B	Coder	Instruct	apache-2.0
Qwen/Qwen2.5-Coder-3B	3B	Coder	Base	qwen-research
Qwen/Qwen2.5-Coder-3B-Instruct	3B	Coder	Instruct	qwen-research
Qwen/Qwen2.5-Math-1.5B	1.5B	Math	Base	apache-2.0
Qwen/Qwen2.5-Math-1.5B-Instruct	1.5B	Math	Instruct	apache-2.0

Qwen3 (qwen3_1_7b, 7 variants)

Model ID	Size	Specialization	Tuning type	License
Qwen/Qwen3-0.6B	0.6B	General	Post-trained (thinking/non-thinking)	apache-2.0
Qwen/Qwen3-4B	4B	General	Post-trained (thinking/non-thinking)	apache-2.0
Qwen/Qwen3-0.6B-Base	0.6B	General	Base	apache-2.0
Qwen/Qwen3-1.7B-Base	1.7B	General	Base	apache-2.0
Qwen/Qwen3-4B-Base	4B	General	Base	apache-2.0
Qwen/Qwen3-4B-Instruct-2507	4B	General	Instruct (2507)	apache-2.0
Qwen/Qwen3-4B-Thinking-2507	4B	General	Thinking (2507)	apache-2.0

TinyLlama (tinyllama_1_1b_chat, 16 variants)

Model ID	Size	Specialization	Tuning type	License
TinyLlama/TinyLlama_v1.1	1.1B	General	Base (v1.1)	apache-2.0
TinyLlama/TinyLlama_v1.1_chinese	1.1B	Chinese	v1.1	apache-2.0
TinyLlama/TinyLlama_v1.1_math_code	1.1B	Math/Code	v1.1	apache-2.0
TinyLlama/TinyLlama-1.1B-Chat-v0.1	1.1B	General	Chat v0.1 (legacy)	apache-2.0
TinyLlama/TinyLlama-1.1B-Chat-v0.2	1.1B	General	Chat v0.2 (legacy)	apache-2.0
TinyLlama/TinyLlama-1.1B-Chat-v0.3	1.1B	General	Chat v0.3 (legacy)	apache-2.0
TinyLlama/TinyLlama-1.1B-Chat-v0.4	1.1B	General	Chat v0.4 (legacy)	apache-2.0
TinyLlama/TinyLlama-1.1B-Chat-v0.5	1.1B	General	Chat v0.5 (legacy)	apache-2.0
TinyLlama/TinyLlama-1.1B-Chat-v0.6	1.1B	General	Chat v0.6 (legacy)	apache-2.0
TinyLlama/TinyLlama-1.1B-step-50K-105b	1.1B	General	Pretraining checkpoint (105B tokens)	apache-2.0
TinyLlama/TinyLlama-1.1B-intermediate-step-240k-503b	1.1B	General	Pretraining checkpoint (503B tokens)	apache-2.0
TinyLlama/TinyLlama-1.1B-intermediate-step-480k-1T	1.1B	General	Pretraining checkpoint (1T tokens)	apache-2.0
TinyLlama/TinyLlama-1.1B-intermediate-step-715k-1.5T	1.1B	General	Pretraining checkpoint (1.5T tokens)	apache-2.0
TinyLlama/TinyLlama-1.1B-intermediate-step-955k-token-2T	1.1B	General	Pretraining checkpoint (2T tokens)	apache-2.0
TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T	1.1B	General	Pretraining checkpoint (2.5T tokens)	apache-2.0
TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T	1.1B	General	Pretraining checkpoint (3T tokens)	apache-2.0

SmolLM3 (smollm3, 1 variant)

Model ID	Size	Specialization	Tuning type	License
HuggingFaceTB/SmolLM3-3B-Base	3B	General	Base	apache-2.0

Gemma 2 (gemma2_2b, 2 variants)

Model ID	Size	Specialization	Tuning type	License
google/gemma-2-2b-it	2B	General	Instruction-tuned	gemma
google/gemma-2-2b-jpn-it	2B	Japanese	Instruction-tuned	gemma

Llama 3.2 (llama3_2_1b_instruct, 3 variants)

Model ID	Size	Specialization	Tuning type	License
meta-llama/Llama-3.2-1B	1B	General	Pretrained	llama3.2
meta-llama/Llama-3.2-3B	3B	General	Pretrained	llama3.2
meta-llama/Llama-3.2-3B-Instruct	3B	General	Instruct	llama3.2

Gemma 3 (gemma3_1b_it, 3 variants)

Model ID	Size	Specialization	Tuning type	License
google/gemma-3-270m	270M	General	Pretrained	gemma
google/gemma-3-270m-it	270M	General	Instruction-tuned	gemma
google/gemma-3-1b-pt	1B	General	Pretrained	gemma

Phi-4 (phi_4_mini, 2 variants)

Model ID	Size	Specialization	Tuning type	License
microsoft/Phi-4-mini-reasoning	3.8B	Mini	Reasoning	mit
microsoft/Phi-4-mini-flash-reasoning	3.8B	Mini	Flash Reasoning	mit

Phi-3 (phi3_mini_3_8b, 1 variant)

Model ID	Size	Specialization	Tuning type	License
microsoft/Phi-3-mini-128k-instruct	3.8B	Mini	Instruct (128K context)	mit

Cards under a llama3.2 or gemma license tag are gate-controlled on Hugging Face itself. You must accept the license on the card page and set HF_TOKEN before you register. apache-2.0 and mit cards need no gating. qwen-research tags carry Qwen's own license terms but are not Hugging Face-gated downloads.

Sample output:

$ np models register model_register.yaml --wait
2026-08-22 16:50.00 [INFO ] [CLI] Registering Model
2026-08-22 16:50.00 [INFO ] [CLI] Reading model metadata from the Hugging Face Hub...
2026-08-22 16:50.01 [INFO ] [CLI] Registering the model on the server...
2026-08-22 16:50.02 [INFO ] [CLI] Uploading model_register.yaml to the server...
2026-08-22 16:50.02 [INFO ] [CLI] Validating model_register.yaml on the server...
2026-08-22 16:50.03 [INFO ] [CLI] Downloading model weights from the Hugging Face Hub (several GB — this can take a while)...
2026-08-22 16:51.30 [INFO ] [CLI] Bundling the weights into a single tar archive...
2026-08-22 16:51.42 [INFO ] [CLI] Bundled the weights into a tar archive (953.3 MiB).
2026-08-22 16:51.42 [INFO ] [CLI] Uploading the weights to the server...
2026-08-22 16:52.20 [INFO ] [CLI] Uploading weights... 79% (754.7 MiB / 953.3 MiB)
2026-08-22 16:52.38 [INFO ] [CLI] Confirming the upload with the server...
2026-08-22 16:52.40 [DONE ] [CLI] Model registered and verified successfully.

$ np models ls --filter type=CUSTOM --sort name
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┓
┃ Name                                  ┃ Type   ┃ Domain ┃ Task            ┃ Source      ┃ Format ┃ Framework ┃ Status ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━┩
│ meta-llama/Llama-3.2-3B-Instruct      │ CUSTOM │ LLM    │ Text Generation │ HUGGINGFACE │ HF     │ PYTORCH   │ Usable │
└───────────────────────────────────────┴────────┴────────┴─────────────────┴─────────────┴────────┴───────────┴────────┘

Model bundle storage

Hugging Face–sourced models store their Graph Extractor trace result as a model bundle at registration time. Preset Nota-provided models and user-registered custom models share the same internal bundle format. You are never given back the original base model file, only the output artifacts that the pipeline produces. If a pipeline includes Advanced Quantizer, np run retraces the AQ output rather than reusing the registration-time bundle directly. Pipelines without AQ reuse the registration-time bundle.

Reregistering a model

Preset models are visible across the whole platform. A model you register yourself is visible only within the workspace you registered it in. Reregistering the same model name:

Situation	Result
Same model, same source revision	Rejected (already registered)
Same model, different source revision, no --force-update	Rejected (pass --force-update to replace it)
Same model, different source revision, with --force-update	Registers the new version and retires the old one from listings

Gated Hugging Face models

For a gated Hugging Face repository, provide a token, either inline for 1 call or exported for the session. The CLI reads the token directly without persisting it to disk. HUGGING_FACE_HUB_TOKEN is also recognized if you do not set HF_TOKEN.

# One-off
HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxx np models register --wait

# Or for the whole session
export HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxx

Get a token (read access is enough) from huggingface.co/settings/tokens.

note

The "no token set" warning appears for every registration without HF_TOKEN, whether or not the specific card is actually gated: the CLI checks whether the variable exists, not whether this particular model needs it:

HF_TOKEN is not set. Anonymous downloads from the Hugging Face Hub are rate
limited, so registering a model card can be slow. Set HF_TOKEN for relaxed
rate limits and access to gated models — it eases throttling but does not
raise your bandwidth cap. Example: HF_TOKEN=hf_xxx np models register

For a public card (for example, Qwen), this is just a slower, anonymous download. Registration still works without a token. A token is strictly required only for cards that Hugging Face itself has gate-controlled. You must accept the license on the card page first. The meta-llama and google entries in models available for BYOM registration are the ones that this applies to.

Registering a Hugging Face repository ID requires the model to be downloaded and loadable, because the Graph Extractor has to trace it. That download happens as part of registration for a repository ID, unlike a local HF-format folder, which is already on disk.

model_register.yaml

np models register model_register.yaml

See model_register.yaml reference for the full schema.

Dataset registration (BYOD)

Dataset registration is decoupled from the model list: its scope is defined by the tasks that the preset models cover. You sample calibration and evaluation data yourself outside Arm AI Model Optimizer. Use one of the following folder structures, and register each with its own call:

np datasets register --source ./my-data --task OBJECT_DETECTION --raw-data coco2017 --calibration

Field	Required	Description
--source	Yes	Path to your already-sampled data folder
--calibration, --evaluation	Yes	Purpose (mutually exclusive)
--task	Yes	Task type. Checked against --raw-data for compatibility, then against the folder itself. Repeat this field (--task A --task B) to register 1 folder for multiple tasks at once
--raw-data	Yes	The folder format you are using. See Supported formats.

Registration is synchronous: the command uploads, validates, and returns a pass or fail result in the same call. There is no job to poll for.

--task is required, even though --help does not mark it that way

np datasets register --help does not mark --task as a required field as it does for --source or --raw-data, because it is a repeatable option rather than a single value. Omitting it entirely is still rejected (NP-CDS-400).

Sampling-related flags (seed, strategy, shuffling) are not accepted, because Arm AI Model Optimizer is not sampling for you. It validates the structure and uploads what you provide. It also does not check label accuracy or overlap between your calibration and evaluation folders, so keep those separate yourself. For a task with no raw-data format (for example, text-embedding beyond what is listed), bring your data prepared as .pt or .npy tensors instead. This path skips structure validation and requires you to match the expected tensor keys.

Supported formats

--task	--raw-data value	Folder, in short
CLASSIFICATION	imagenet_folder	images/ + labels/labels.csv
OBJECT_DETECTION, INSTANCE_SEGMENTATION, MASK_GENERATION	coco2017	val2017/ + annotations/instances_*.json
SEGMENTATION	ade20k	images/ + annotations/ (matching-name PNG masks)
SEGMENTATION (portrait)	eg1800_portrait	images_data_crop/ + GT_png/
SUPER_RESOLUTION	div2k_paired	LR_bicubic/X4/ + HR/ (paired filenames)
TEXT_GENERATION	plain_text	flat top-level *.txt files
SPEECH_TO_TEXT	librispeech	audio + *.trans.txt transcripts
TEXT_EMBEDDING	beir	queries.jsonl + corpus.jsonl + qrels/*.tsv
TEXT_CLASSIFICATION	sst2_labeled	sentence + label tsv/csv
AUDIO_CLASSIFICATION, AUDIO_TOXICITY_DETECTION	audio_labeled	audio + label_map.json
IMAGE_TEXT_MATCHING	coco_captions_jsonl	images/ + captions.jsonl
FACE_DETECTION	wider_face, openimages_face	images/<event>/ + wider_face_split/*.txt
VISION_LANGUAGE_MODEL (calibration only)	vlm_captions_csv	images/ + annotations/*.csv

1 format can cover more than 1 task (coco2017 serves both OBJECT_DETECTION and INSTANCE_SEGMENTATION. audio_labeled serves both AUDIO_CLASSIFICATION and AUDIO_TOXICITY_DETECTION).

vlm_captions_csv is not in --help yet

np datasets register --help lists 13 formats. The server accepts a 14th, vlm_captions_csv, that the static help text in the CLI does not document. If you are checking what is supported, the rejection message from an invalid --raw-data value is the authoritative list. Vision-language model (VLM) evaluation itself is not registered through BYOD. Evaluation passes a benchmark name straight to VLMEvalKit, which loads its own data.

2-stage validation

--raw-data is checked twice, and the 2 stages catch different things:

Stage	Checks	When	On failure
1. Before upload	Does the format name exist, and can it be read as the --task you gave?	Immediately, before any data transfer starts	Instant HTTP 400. Nothing uploads, even for a multi-GB folder
2. After upload	Does the actual structure of the folder match what that format expects? (a real dry-run through the loader)	Right after upload, synchronously	Registration rejected with an engine code and a hint about what is different

# Stage 1 — unknown format name, rejected before any upload
$ np datasets register --source ./my-data --calibration --task CLASSIFICATION --raw-data imagenet_folderr
[ERROR] Backend error (HTTP 400): Unsupported raw-data format 'imagenet_folderr'.
        Supported formats: imagenet_folder, coco2017, ade20k, eg1800_portrait,
        div2k_paired, plain_text, librispeech, beir, sst2_labeled, audio_labeled,
        coco_captions_jsonl, vlm_captions_csv, wider_face, openimages_face

# Stage 1 — format exists, but can't read this --task
$ np datasets register --source ./my-data --calibration --task OBJECT_DETECTION --raw-data imagenet_folder
[ERROR] Backend error (HTTP 400): raw-data format 'imagenet_folder' cannot be
        read as OBJECT_DETECTION. Tasks supported by this format: CLASSIFICATION

# Stage 1 — --task omitted entirely
$ np datasets register --source ./my-data --calibration --raw-data imagenet_folder
[ERROR] [CLI] [NP-CDS-400] At least one --task is required.

# Stage 2 — format/task are valid, but the folder itself doesn't match
$ np datasets register --source ./my-data --evaluation --task OBJECT_DETECTION --raw-data coco2017
Object Detection  REJECT  does not match the standard structure of the
                  declared raw-data format 'coco2017' — expected val2017/ +
                  annotations/instances_*.json | current top level: images/, labels/

Does calibration need labels?

--evaluation always requires labels or ground truth because scoring compares predictions against them. Requirements for --calibration depend on the format. Some formats require only the model input. Other formats require label data because the calibration process uses it. Examples include the following:

• the low-resolution (LR) image for a super-resolution (SR) model
• the transcript for a speech-to-text (STT) model
• the relevance judgments (qrels) for a retrieval model
• caption text for a captioning model
• prompt text for a vision-language model

--raw-data	Calibration needs labels/annotations?
imagenet_folder	No
coco2017 (OBJECT_DETECTION, INSTANCE_SEGMENTATION)	No
coco2017 (MASK_GENERATION)	Yes — prompt points are derived from the COCO boxes
ade20k	No
eg1800_portrait	No
wider_face	No
sst2_labeled	No
div2k_paired	Yes — HR/ is the calibration target itself
librispeech	Yes — *.trans.txt is consumed directly
audio_labeled	Yes
beir	Yes — qrels/ is part of the standard layout either way
coco_captions_jsonl	Yes — the caption text is the input
vlm_captions_csv	Yes (calibration-only format)

Omitting the label or annotation folder does not always cause registration to fail. For a format in the "No" column, registration still succeeds. For a format in the "Yes" column, registration fails with HTTP 422.

Unsupported folder layout

After both previous stages complete, a folder that is simply missing an expected file still gets a specific rejection rather than a silent failure. For example, a WIDER Face folder missing its annotation file:

============================================================
Registration rejected
============================================================
  Face Detection    REJECT   Annotation file not found:
                     .../wider_face_split/wider_face_val_bbx_gt.txt

⚠ The archive was uploaded but the sample was NOT registered, so it
  cannot be used yet.

A rejected registration does not appear in np datasets ls. Fix the folder and register again.

Duplicate registration

Reregistering the same data with the same parameters is detected by content fingerprint and returns the existing dataset instead of creating a duplicate. Changing even one parameter (for example, a different sample count or a different seed) registers as a distinct dataset.

The fingerprint ignores --task

Content fingerprinting looks only at the data, not the task list, so reregistering the identical folder later with an additional --task produces 409 CONFLICT instead of adding that task. If a format serves more than one task and you want a single dataset usable for both (for example, coco2017 for OBJECT_DETECTION and INSTANCE_SEGMENTATION), pass every task you need in the same call: --task OBJECT_DETECTION --task INSTANCE_SEGMENTATION.

.pt and .npy key naming for LLMs

If you are bringing preprocessed tensors for a text-generation model, the tensor keys must match the pt2 input names for the model exactly. There is no automatic relabeling. Advanced Quantizer and Graph Quantizer consume different tensors (Hugging Face–style keys as opposed to post-export .pt2 keys), so a key that is correct for one stage will not automatically carry over to the other.

Retention

Uploaded datasets are retained for a limited period (a 3-month default, subject to legal review).

Format details

Each subsection shows the folder tree, what a key file actually contains, and the registration command. Calibration and evaluation use the same layout unless noted.

Classification (imagenet_folder)

imagenet/
├─ images/valid/
│  ├─ ILSVRC2012_val_00000007.JPEG
│  └─ ILSVRC2012_val_00000025.JPEG
└─ labels/
   └─ imagenet_valid.csv

labels/imagenet_valid.csv

image_id,class
ILSVRC2012_val_00000007.JPEG,334
ILSVRC2012_val_00000025.JPEG,23

np datasets register --source ./imagenet --calibration --task CLASSIFICATION --raw-data imagenet_folder

A per-class folder layout (images/<class>/*.jpg, no separate labels file) is also accepted. The class comes from the folder name instead of a CSV row.

Object detection, instance segmentation, and mask generation (coco2017)

coco2017/
├─ val2017/
│  └─ 000000000139.jpg
└─ annotations/
   └─ instances_val2017.json

annotations/instances_val2017.json

{
  "images": [{"id": 139, "file_name": "000000000139.jpg", "height": 426, "width": 640}],
  "annotations": [{"image_id": 139, "category_id": 62, "bbox": [237, 143, 25, 70], "iscrowd": 0}],
  "categories": [{"id": 1, "name": "person"}]
}

# Calibration folder is conventionally train2017/ instead of val2017/
np datasets register --source ./coco2017 --calibration \
   --task OBJECT_DETECTION --task INSTANCE_SEGMENTATION --raw-data coco2017

The image folder is train2017/ by convention for calibration data and val2017/ (or validation/data/) for evaluation data. However, the loader does not enforce the name itself. The loader picks its annotation file by content, not filename: it needs a categories key, so a captions_*.json in the same annotations/ folder (no categories key) is skipped rather than picked up by mistake.

MASK_GENERATION uses the same folder and command shape, but unlike OBJECT_DETECTION or INSTANCE_SEGMENTATION, it needs annotations/ even for calibration (see Does calibration need labels? above) because the segmentation prompt points are derived from the COCO boxes.

Segmentation (ade20k)

ade20k/
├─ images/
│  ├─ ADE_val_00000001.jpg
│  └─ ADE_val_00000002.jpg
└─ annotations/
   ├─ ADE_val_00000001.png
   └─ ADE_val_00000002.png

np datasets register --source ./ade20k --calibration --task SEGMENTATION --raw-data ade20k

Each mask is a PNG with the same name as its image (pixel value = class index). The mask folder might also be named masks/, SegmentationClass/, or GT_png/. The image folder might also be named images_data_crop/.

Portrait segmentation (eg1800_portrait)

portrait/
├─ images_data_crop/
│  ├─ 00001.jpg
│  └─ 00002.jpg
└─ GT_png/
   ├─ 00001.png
   └─ 00002.png

np datasets register --source ./portrait --calibration --task SEGMENTATION --raw-data eg1800_portrait

Super resolution (div2k_paired)

div2k/
├─ LR_bicubic/X4/
│  ├─ 0801x4.png
│  └─ 0802x4.png
└─ HR/
   ├─ 0801.png
   └─ 0802.png

np datasets register --source ./div2k --calibration --task SUPER_RESOLUTION --raw-data div2k_paired

LR filenames carry the scale suffix (x4), and high-resolution (HR) files keep the original name. The loader pairs them by that convention. Both folders are required, even for calibration.

Text generation (plain_text)

corpus/
├─ sample_00001.txt
├─ sample_00002.txt
└─ sample_00003.txt

sample_00001.txt

The transformer architecture has
become the foundation of modern ...

np datasets register --source ./corpus --calibration --task TEXT_GENERATION --raw-data plain_text

Non-empty *.txt files at the top level only (subfolders are not scanned). A Hugging Face Arrow-format folder is also recognized.

Speech to text (librispeech)

librispeech/
└─ LibriSpeech/
   └─ dev-clean/
      └─ 1272/128104/
         ├─ 1272-128104-0000.flac
         ├─ 1272-128104-0001.flac
         └─ 1272-128104.trans.txt

1272-128104.trans.txt (one line per audio file)

1272-128104-0000 HE PASSED THROUGH
1272-128104-0001 SAINT ALBANS AND CAME

np datasets register --source ./librispeech --calibration --task SPEECH_TO_TEXT --raw-data librispeech

Every audio file needs a transcript in the *.trans.txt alongside it, at any folder depth (a flat layout without the LibriSpeech/<split>/<speaker>/<chapter>/ nesting also works). The outer LibriSpeech/ folder follows the official LibriSpeech download layout. The split folder is dev-clean for calibration and test-clean for evaluation by convention.

Text embedding (beir)

beir/
├─ queries.jsonl
├─ corpus.jsonl
└─ qrels/
   └─ test.tsv

queries.jsonl

{"_id": "14044", "text": "who is ..."}

corpus.jsonl

{"_id": "14044_p0", "text": "Thus ..."}

qrels/test.tsv (tab-separated)

query-id	corpus-id	score
14044	14044_p0	1

np datasets register --source ./beir --calibration --task TEXT_EMBEDDING --raw-data beir

All 3 files are required in the same folder: this is the standard BEIR layout, used for both calibration and evaluation.

Text classification (sst2_labeled)

sst2/
├─ sample_00001.txt
├─ sample_00002.txt
└─ labels.csv

labels.csv

id,label
sample_00001,0
sample_00002,1

np datasets register --source ./sst2 --evaluation --task TEXT_CLASSIFICATION --raw-data sst2_labeled

Sentences without a label will not register for evaluation. A single combined .tsv/.csv also works, and the loader also looks for it directly under the source root (not just labels/).

Calibration does not need labels

--calibration only reads the input sentences, not labels.csv, so a folder with just sample_*.txt and no label file registers fine as calibration data. --evaluation needs the labels, because scoring compares predictions against them.

For sentence-pair datasets, for example, MNLI, put both sentences in the same .txt file separated by a single tab: premise<TAB>hypothesis.

Audio classification / audio toxicity detection (audio_labeled)

fsd50k/
├─ audio/
│  ├─ 00001.wav
│  └─ 00002.wav
└─ label_map.json

label_map.json

{
  "00001": [3, 7],
  "00002": [12]
}

np datasets register --source ./fsd50k --calibration --task AUDIO_CLASSIFICATION --raw-data audio_labeled
# Toxicity: same folder/format, --task AUDIO_TOXICITY_DETECTION

Keys are filenames without extension. Values are integer class-index arrays. Audio files can be .wav or .flac, at any folder depth. The loader looks for label_map.json at the source root, labels/, or annotations/. A label file is required even for calibration (see Does calibration need labels?).

label_map.json is not the only accepted shape: the FSD50K native layout (dev.csv/eval.csv plus vocabulary.csv, no JSON) registers and runs successfully without conversion. Use label_map.json when you are assembling label data yourself. Keep the native CSV layout for a dataset when it already has one, rather than converting unnecessarily.

Image-text matching (coco_captions_jsonl)

coco_itm/
├─ images/
│  └─ 000000000139.jpg
└─ captions.jsonl

captions.jsonl (one record per line, 4 keys)

{"image": "000000000139.jpg", "image_id": 139, "caption_id": 0, "caption": "A room with a chair"}

np datasets register --source ./coco_itm --calibration --task IMAGE_TEXT_MATCHING --raw-data coco_captions_jsonl

The file must use JSONL with 1 record on each line. A single JSON array on one line, for example a Karpathy-split export, is not accepted. The loader looks for it at captions.jsonl, any *.jsonl at the source root, annotations/*.jsonl, or captions/*.jsonl. The file is also required for calibration because the caption text is part of the model input (see Does calibration need labels?).

Face detection (wider_face / openimages_face)

The loader accepts 2 layouts. Use only 1 layout for each dataset.

Layout A: split subfolders

wider_face/
├─ images/train/
│  └─ 0--Parade/0013ea20.jpg
└─ annotations/
   └─ train_bbx_gt.txt

Evaluation data uses images/val/<event>/ and annotations/val_bbx_gt.txt instead of the train names above. Calibration data does not need annotations/ (see Does calibration need labels?).

Layout B: original WIDER convention

wider_face/
├─ images/0--Parade/
│  └─ 0_Parade_001.jpg
└─ wider_face_split/
   └─ wider_face_val_bbx_gt.txt

In Layout B, event folders go directly under images/ (no extra split folder), and the annotation path wider_face_split/wider_face_val_bbx_gt.txt is hardcoded with no alternate name, so match it exactly (calibration data uses the same val name here too). WIDER_val/images/<event>/ is also accepted as the top-level image path in this layout.

*_bbx_gt.txt

0--Parade/0013ea20.jpg
2
289 152 138 174 0 0 0 0 0 0
292 208 131 109 0 0 0 0 0 0

np datasets register --source ./wider_face --calibration --task FACE_DETECTION --raw-data wider_face

Face detection, Open Images source (openimages_face)

openimages_face uses the same split-subfolder layout as wider_face layout A above, except the annotation file is Open Images' own CSV instead of a converted WIDER .txt:

openimages_face/
├─ images/train/
└─ annotations/
   └─ train-face-bbox.csv

np datasets register --source ./openimages_face --calibration --task FACE_DETECTION --raw-data openimages_face

Registering is not the same as being runnable

Registration accepts the raw Open Images CSV without conversion. The CSV uses normalized XMin, XMax, YMin, and YMax columns. Before you run a pipeline, convert the normalized coordinates to absolute pixel x y w h values and save them as a WIDER-format *_bbx_gt.txt file. The loader reads this file at run time. The folder structure does not change.

Vision-language model, calibration only (vlm_captions_csv)

flickr30k/
├─ images/
│  └─ 1000092795.jpg
└─ annotations/
   └─ flickr_annotations_30k.csv

annotations/flickr_annotations_30k.csv (4 columns)

filename,img_id,split,raw
1000092795.jpg,0,train,"[""Two young guys with shaggy hair look at their hands.""]"

np datasets register --source ./flickr30k --calibration --task VISION_LANGUAGE_MODEL --raw-data vlm_captions_csv

raw is a JSON array of caption strings. Only the first caption is used (one row per image either way). split must include the literal value train. This format supports calibration only. VLM evaluation is not registered through BYOD. Instead, evaluation passes a benchmark name, for example --dataset MMVP, directly to VLMEvalKit. VLMEvalKit loads its own data instead of reading a registered dataset.

See Tutorial 4: Bring Your Own Model (BYOM) and Bring Your Own Data (BYOD) for an end-to-end walkthrough.