Adding a New Model Format to triton-inferencer

Recommended: declare the recipe in the model's config.json

A model whose config.json carries its own pre/postprocess recipe needs no change to this node, as long as it uses the primitives the backend already has (letterbox or stretch, pad, antialias, /255, mean/std; YOLO or DETR decode). rpdet writes the blocks from the plugin (e.g. RPOD.PREPROCESS / RPOD.POSTPROCESS):

json

"common": {
  "preprocess":  {"resize": "letterbox", "pad_value": 114, "antialias": true, "scale": 255, "mean": null, "std": null},
  "postprocess": {"format": "detr", "class_offset": 0}
}

generatePipelineConfig maps them 1:1 onto the backend parameters (letterbox, pad_value, resize_antialias, divide_255, mean, std, detection_format, detr_class_offset, end2end) and never looks at the model name. It also writes postprocess_format, which the editor uses as the format key (detection/detr, detection/yolo), so FORMAT_PARAMS needs no new line either. An unknown postprocess.format fails the conversion with a message instead of decoding as YOLO. RP-OD is the first family on this path; tests in test/pipeline-config.test.js (npm test).

The steps below are for a model that predates the blocks, or needs a new primitive in the C++ backend.

Quick reference

Adding a new model format (e.g. detection/efficientdet) requires two changes:

#	File	What to do
1	`triton-inferencer.html`	Add one line to `FORMAT_PARAMS`
2	Pipeline `config.pbtxt`	Set `model_format` parameter

If the new format needs a new postprocess path in the C++ backend, see Advanced below.

Step 1: Register the format in the UI

In triton-inferencer.html, find the FORMAT_PARAMS object (~line 785) and add an entry. The key is task/format, the value lists which UI parameters are visible for that combo:

const FORMAT_PARAMS = {
    'detection/yolo':      ['confidenceThreshold', 'iouThreshold', 'maxDetections'],
    'detection/rfdetr':    ['confidenceThreshold', 'maxDetections'],
    'detection/deimv2':    ['confidenceThreshold', 'maxDetections'],
    'segmentation/yolo':   ['confidenceThreshold', 'iouThreshold', 'maxDetections', 'maskThreshold', 'contourEpsilon'],
    'classification/yolo': [],
    // Add your new format here:
    'detection/efficientdet': ['confidenceThreshold', 'maxDetections'],
};

This single line automatically:

Adds the format to the Task Type grouped dropdown
Controls which parameter fields are shown/hidden
Updates the badge text on each parameter row

Available parameter keys: confidenceThreshold, iouThreshold, maxDetections, maskThreshold, contourEpsilon, the SAHI slicing set (SLICE_PARAMS), the Paddle DB knobs (dbThresh, …) and the det/seg+cls pipeline knobs (detConfThreshold, …) — see the registry itself for the current list.

The key is <task_type>/<model_format> as declared in the deployed config.pbtxt, with -e2e appended when end2end is "true".

Step 2: Set model_format in the pipeline config

In the deployed model's config.pbtxt, add a model_format parameter so the UI can auto-detect it:

parameters: { key: "task_type"     value: { string_value: "detection" } }
parameters: { key: "model_format"  value: { string_value: "efficientdet" } }

The backend JS endpoint (triton-inferencer.js) reads task_type and model_format from any config.pbtxt automatically — no code changes needed.

Step 3 (optional): Update pipeline templates

If this format will be reused, add a template to triton-server/pipeline_templates/:

triton-server/pipeline_templates/
    detection_pipeline.config.pbtxt        # YOLO detection (default)
    segmentation_pipeline.config.pbtxt     # YOLO segmentation
    classification_pipeline.config.pbtxt   # YOLO classification
    detection_detr_pipeline.config.pbtxt   # your new template

Include the model_format parameter in the template.

Advanced: New postprocess path

If the new format requires different postprocessing logic (not just different parameter visibility), you also need to modify the C++ backend:

Native postprocess — Add a new file in triton-server/backend/native/ (e.g. rp_post_efficientdet.cpp) implementing the decode function. Follow the pattern in rp_post_yolo.cpp and rp_post_detr.cpp.
Backend dispatch — In triton-server/backend/src/rp_pipeline.cc, the postprocess dispatch uses detection_format (0=yolo, 1=detr, 2=paddle_db) plus end2end. Add a new branch for your format. The model_format string is never read by the backend — it only drives the editor UI.
Config struct — If needed, add new fields to rp_pipeline_config.h and parse them in the config loading section of rp_pipeline.cc (search for pm.GetInt("detection_format").
Build — Add the new .cpp to triton-server/backend/CMakeLists.txt and triton-server/Dockerfile.

Any DETR-family head that exports as dets [N, Q, 4] (normalised cxcywh) + labels [N, Q, C] (raw logits) runs on the existing detection_format=1 decoder (sigmoid → global top-k over [query, class] → decode, no NMS). The only per-model knob is detr_class_offset: RF-DETR reserves class index 0 as a background slot (offset 1, the default); DEIMv2 is 0-indexed (offset 0). DEIMv2 was added this way — its rpdet exporter emits the raw head instead of the upstream orig_target_sizes deploy graph, so no backend decoder was needed.

Auto-generated configs (Firebase models)

When a model is downloaded from Firebase via firebase-model-manager.js, the pipeline and weights config.pbtxt files are generated automatically — no manual config authoring needed.

Two functions in modules/firebase-model-manager.js handle this:

Function	Generates	Source of truth
`generateWeightsConfig(modelId, onnxMeta)`	`{modelId}_weights/config.pbtxt` (tensorrt_plan)	ONNX input/output tensor metadata
`generatePipelineConfig(modelId, onnxMeta, inferenceConfig)`	`{modelId}/config.pbtxt` (rp_pipeline)	ONNX metadata + Firebase `config.json`

How it decides config values

Task type and model format come from the Firebase config.json downloaded alongside the ONNX file:

config.json → common.task         → "detection" / "segmentation" / "classification" / "anomaly"
config.json → common.model_name   → "yolo" / "rfdetr" / "deimv2" ("ANOMALY" → model_format "anomalib")
config.json → common.image_shape  → { height, width }
config.json → task_specific.*     → confidence, iou, classes

Tensor names and shapes are extracted from the ONNX model itself (via Python onnx library in an ephemeral container).

Format-specific differences

The generator applies the correct preprocessing parameters per model format:

Parameter	yolo (det/seg)	yolo (classification)	rfdetr	deimv2	anomalib (anomaly)
`letterbox`	`true`	`false` (+ `center_crop`)	`false`	`false`	`false`
`pad_value`	`114`	`114`	—	—	—
`mean`	`""`	`""`	`0.485,0.456,0.406`	`""`	`""` (inside the graph)
`std`	`""`	`""`	`0.229,0.224,0.225`	`""`	`""` (inside the graph)
`detection_format`	—	—	`1`	`1`	—
`detr_class_offset`	—	—	`1`	`0`	—
`resize_antialias`	—	`true` (PIL)	`true`	`true`	`true` (torchvision)
`apply_sigmoid`	`auto`	—	—	—	—
`iou_threshold`	yes	—	—	—	—
`conf_threshold`	`task_specific.confidence`	—	same	same	`task_specific.threshold` (OK/NOK, default 0.5)
`mask_threshold`	seg only: `0.5`	—	—	—	`0.5` (pixel threshold on the anomaly map)
`end2end`	`"true"` when `model_type` is `v26`	—	—	—	—

A model that declares common.preprocess / common.postprocess (RP-OD) is not in this table: its parameters are its declaration (letterbox 114 + antialias + /255, DETR with class offset 0 for RP-OD).

Anomaly (anomalib): rpdet's AnomalibExporter traces pre-processor + model + post-processor into one graph whose outputs pred_score [N,1] and anomaly_map[N,1,H,W] are min-max normalised around the learned thresholds, so 0.5 is the trained threshold on both. The backend (task_type: anomaly) tags the image OK/NOK (class_id 0/1) at conf_threshold, binarises the map at mask_threshold and turns each external contour into a polygon in original-image pixels — the same {"anomaly": {...}} contract rpcommons.AnomaliesResults emits.

Pipeline inputs per task

Task	Inputs
detection	RAW_IMAGE, CONF_THRESHOLD, IOU_THRESHOLD, MAX_DET, SWAP_RB
segmentation	detection + MASK_THRESHOLD, CONTOUR_EPSILON
classification	RAW_IMAGE, SWAP_RB
anomaly	RAW_IMAGE, CONF_THRESHOLD, MASK_THRESHOLD, CONTOUR_EPSILON, SWAP_RB
ocr_pipeline	RAW_IMAGE, SWAP_RB, CONF_THRESHOLD, MAX_DET, DB_THRESH, DB_UNCLIP_RATIO, DB_MAX_CANDIDATES, DB_MIN_BOX_SIZE, USE_DOC_ORIENTATION, USE_TEXTLINE_ORIENTATION

Weights config extras

The generated weights config includes a model_warmup section with zero_data: true, which pre-allocates GPU memory on model load and prevents first-inference latency spikes.

Adding support for a new model format in auto-generation

Prefer declaring the recipe in the model's config.json (see the top of this file): then nothing here changes. Only a format that cannot declare itself needs a branch in generatePipelineConfig() in modules/firebase-model-manager.js, following the existing isRtDetr / isDeim pattern.

PP-OCR presets: det resize and orientation stages

The PP-OCR det sub-config is generated with PaddleX's DetResizeForTest rule (paddle_det_limit_type=min, paddle_det_limit_side_len=64), which is what the rpdet / ocr-inferencer pipeline runs: images are never downscaled below their native resolution, only capped at paddle_det_max_side_limit, which the node sets to the converted engine's profile max (OCR_PROFILE_MAX.det, 1280). Keep those three in sync when changing the cap; the legacy paddle_det_resize_long / paddle_det_resize_max_only pair is still emitted for older backends.

An OCR preset whose catalog entry lists subs: ["det", "rec", "cls", "doc"] also installs two PP-LCNet classifiers as <id>_cls (textline orientation, 160x80, classes 0_degree/180_degree) and <id>_doc (document orientation, 224 center crop of a 256 short edge, classes 0/90/180/270). The top ocr_pipeline config references them as cls_model / doc_cls_model with both stages off; the node switches them per request with the USE_TEXTLINE_ORIENTATION / USE_DOC_ORIENTATION inputs (editor: Textline Orientation / Document Orientation). The backend rotates the image upright before det and maps every polygon back to the original frame; metadata.document_angle reports the rotation undone and metadata.orientation_angles the per-line 0/1 textline result. A preset installed before the orientation stages existed is no longer isModelReady (its top config lacks doc_cls_model), so the next deploy re-prepares it.

A recognizer alone is a catalog entry with task: "ocr_rec", subs: ["rec"] and its rec_format (e.g. svtrv2-b, exported by the openocr handler of scripts/export-triton-default-models). It installs only <id>_rec (+ weights), with the rec config its rec_format picks (svtrv2: ratio buckets, RGB, antialiased stretch). The ocr-triton node runs it for Recognition Only, and for OCR pairs it with a preset's det as the model <preset>--<rec id>: default-models.parseOcrCombo resolves the name, the node installs both entries if missing, then installOcrCombo writes only a top ocr_pipeline config whose det_model / cls_model / doc_cls_model are the preset's stages and rec_model is <rec id>_rec. Any preset's rec pairs the same way (<preset>--<other preset>).

Default (preset) models

triton_default_models.json lists the stock checkpoints offered in the editor; scripts/export-triton-default-models/ exports them (one handler per framework) and uploads to the public bucket. To validate a freshly exported model before uploading, point the node at a local copy with TRITON_DEFAULT_MODELS_BASE_URL=http://localhost:18080 (a directory per catalog id holding model.onnx + config.json).

Recognition width

The rec sub-config (generatePaddleRecConfig) resizes a line to height 48 and a width that follows the line, imgW = max(320, ceil(48 * w / h)), as PaddleX's OCRReisizeNormImg does. That holds for the pipeline's batched rec and for the standalone <preset>_rec model (the ocr-triton node's Recognition Only task). Both clamp the width to rec_max_w, the rec engine profile's maximum width (1280 for the shipped presets, taken from the ONNX profile at install time; the backend defaults to 1280 when the parameter is absent). A fixed target_w would squeeze any line wider than 320/48 of its height and read a different text from the ocr node.

File map

triton-inferencer/
    triton-inferencer.html          # UI: FORMAT_PARAMS registry, dropdown, param visibility
    triton-inferencer.js            # Backend: serves model list (auto-reads config.pbtxt)
    modules/
        firebase-model-manager.js   # Download + ONNX metadata + config.pbtxt generation + trtexec
        default-models.js           # Preset catalog lookup (TRITON_DEFAULT_MODELS_BASE_URL override)
    triton_default_models.json      # Preset catalog (shared with scripts/export-triton-default-models)
<repo>/triton-server/
    pipeline_templates/             # Template config.pbtxt files per task type
    backend/src/
        rp_pipeline.cc              # Main backend: config parsing + postprocess dispatch
        rp_pipeline_config.h        # Config struct (TaskType, detection_format, detr_class_offset, thresholds)
    backend/native/
        rp_post_yolo.cpp            # YOLO postprocess (NMS + decode)
        rp_post_detr.cpp            # DETR postprocess (no NMS; RF-DETR + DEIMv2 via detr_class_offset)
        rp_post_common.h            # Shared helpers (IoU, proto serialization)
    Dockerfile                      # Builds native .so files + backend

Adding a New Model Format to triton-inferencer ​

Recommended: declare the recipe in the model's config.json ​

Quick reference ​

Step 1: Register the format in the UI ​

Step 2: Set model_format in the pipeline config ​

Step 3 (optional): Update pipeline templates ​

Advanced: New postprocess path ​

Before writing a new decoder: can the model share the DETR path? ​

Auto-generated configs (Firebase models) ​

How it decides config values ​

Format-specific differences ​

Pipeline inputs per task ​

Weights config extras ​

Adding support for a new model format in auto-generation ​

PP-OCR presets: det resize and orientation stages ​

Default (preset) models ​

Recognition width ​

File map ​