Adding a New Model Format to triton-inferencer

Quick reference

Adding a new model format (e.g. detection/efficientdet) requires two changes:

#	File	What to do
1	triton-inferencer.html	Add one line to FORMAT_PARAMS
2	Pipeline config.pbtxt	Set model_format parameter

If the new format needs a new postprocess path in the C++ backend, see Advanced below.

---

Step 1: Register the format in the UI

In triton-inferencer.html, find the FORMAT_PARAMS object (~line 623) and add an entry. The key is task/format, the value lists which UI parameters are visible for that combo:

const FORMAT_PARAMS = {
    'detection/yolo':      ['confidenceThreshold', 'iouThreshold', 'maxDetections'],
    'detection/rt-detr':   ['confidenceThreshold', 'maxDetections'],
    'segmentation/yolo':   ['confidenceThreshold', 'iouThreshold', 'maxDetections', 'maskThreshold', 'contourEpsilon'],
    'classification/yolo': [],
    // Add your new format here:
    'detection/efficientdet': ['confidenceThreshold', 'maxDetections'],
};

This single line automatically:

• Adds the format to the Task Type grouped dropdown
• Controls which parameter fields are shown/hidden
• Updates the badge text on each parameter row

Available parameter keys: confidenceThreshold, iouThreshold, maxDetections, maskThreshold, contourEpsilon.

Step 2: Set model_format in the pipeline config

In the deployed model's config.pbtxt, add a model_format parameter so the UI can auto-detect it:

parameters: { key: "task_type"     value: { string_value: "detection" } }
parameters: { key: "model_format"  value: { string_value: "efficientdet" } }

The backend JS endpoint (triton-inferencer.js) reads task_type and model_format from any config.pbtxt automatically — no code changes needed.

Step 3 (optional): Update pipeline templates

If this format will be reused, add a template to engine/pipeline_templates/:

engine/pipeline_templates/
    detection_pipeline.config.pbtxt        # YOLO detection (default)
    segmentation_pipeline.config.pbtxt     # YOLO segmentation
    classification_pipeline.config.pbtxt   # YOLO classification
    detection_detr_pipeline.config.pbtxt   # your new template

Include the model_format parameter in the template.

---

Advanced: New postprocess path

If the new format requires different postprocessing logic (not just different parameter visibility), you also need to modify the C++ backend:

1. Native postprocess — Add a new file in engine/backend/native/ (e.g. rp_post_efficientdet.cpp) implementing the decode function. Follow the pattern in rp_post_yolo.cpp and rp_post_detr.cpp.

2. Backend dispatch — In engine/backend/src/rp_pipeline.cc, the postprocess dispatch currently uses detection_format (0=yolo, 1=detr). Add a new branch for your format.

3. Config struct — If needed, add new fields to rp_pipeline_config.h and parse them in the config loading section of rp_pipeline.cc (~line 430).

4. Build — Add the new .cpp to engine/backend/CMakeLists.txt and engine/Dockerfile.

Auto-generated configs (Firebase models)

When a model is downloaded from Firebase via firebase-model-manager.js, the pipeline and weights config.pbtxt files are generated automatically — no manual config authoring needed.

Two functions in modules/firebase-model-manager.js handle this:

Function	Generates	Source of truth
generateWeightsConfig(modelId, onnxMeta)	{modelId}_weights/config.pbtxt (tensorrt_plan)	ONNX input/output tensor metadata
generatePipelineConfig(modelId, onnxMeta, inferenceConfig)	{modelId}/config.pbtxt (rp_pipeline)	ONNX metadata + Firebase config.json

How it decides config values

Task type and model format come from the Firebase config.json downloaded alongside the ONNX file:

config.json → common.task         → "detection" / "segmentation" / "classification"
config.json → common.model_name   → "yolo" / "rt-detr"
config.json → common.image_shape  → { height, width }
config.json → task_specific.*     → confidence, iou, classes

Tensor names and shapes are extracted from the ONNX model itself (via Python onnx library in an ephemeral container).

Format-specific differences

The generator applies the correct preprocessing parameters per model format:

Parameter	yolo (det/seg)	yolo (classification)	rt-detr
letterbox	true	false	false
pad_value	114	114	—
mean	""	""	0.485,0.456,0.406
std	""	""	0.229,0.224,0.225
detection_format	—	—	1
apply_sigmoid	auto	—	—
iou_threshold	yes	—	—

Pipeline inputs per task

Task	Inputs
detection	RAW_IMAGE, CONF_THRESHOLD, IOU_THRESHOLD, MAX_DET, SWAP_RB
segmentation	detection + MASK_THRESHOLD, CONTOUR_EPSILON
classification	RAW_IMAGE, SWAP_RB

Weights config extras

The generated weights config includes a model_warmup section with zero_data: true, which pre-allocates GPU memory on model load and prevents first-inference latency spikes.

Adding support for a new model format in auto-generation

If a new format (e.g. efficientdet) needs to be auto-generated from Firebase, update generatePipelineConfig() in modules/firebase-model-manager.js to handle the new model_format value with its specific preprocessing parameters. The existing isRtDetr branching pattern shows how to add format-specific logic.

---

File map

triton-inferencer/
    triton-inferencer.html          # UI: FORMAT_PARAMS registry, dropdown, param visibility
    triton-inferencer.js            # Backend: serves model list (auto-reads config.pbtxt)
    engine/
        pipeline_templates/         # Template config.pbtxt files per task type
        backend/src/
            rp_pipeline.cc          # Main backend: config parsing + postprocess dispatch
            rp_pipeline_config.h    # Config struct (TaskType, detection_format, thresholds)
        native/
            rp_post_yolo.cpp        # YOLO postprocess (NMS + decode)
            rp_post_detr.cpp        # DETR postprocess (no NMS)
            rp_post_common.h        # Shared helpers (IoU, proto serialization)
        Dockerfile                  # Builds native .so files + backend