Appearance

Inferencer Node ​

Overview ​

The Inferencer node provides AI-powered computer vision inference by running machine learning models in Docker containers. It acts as a bridge between Node-RED flows and Python-based ML models, supporting detection, classification, segmentation, anomaly detection, and OCR tasks.

The node uses a modular 7-stage inference pipeline and communicates with the inference server via gRPC.

Key Features ​

  • Multiple ML frameworks: YOLO, PaddlePaddle, RF-DETR, Anomalib (TensorRT via the Triton Inferencer)
  • Task support: Object detection, image classification, instance/semantic segmentation, anomaly detection, OCR, recognition
  • Docker-based execution: Isolated Python containers with automatic lifecycle management
  • Dynamic worker scaling: Scale inference workers without restarting containers
  • Dynamic models: Load models at runtime or configure statically
  • Always-on warmup: Server-side synthetic image generation for zero cold-start latency
  • Firebase integration: Automatic model download from Firebase storage with unified model dropdown
  • Promise mode: Asynchronous processing for high-throughput batch workflows
  • Config prefabs: 13 pre-built model type configurations for quick setup
  • Output customization: Configure detection boxes, masks, and result formats
  • Class mapping: Rename or filter model output classes
  • Debug visualization: Display processed images on Node-RED canvas
  • Performance tracking: Built-in metrics at msg.performance.inference.{node_name}

Architecture ​

Inference Pipeline ​

The node processes each request through a 7-stage modular pipeline:

validate-request → resolve-model → ensure-model → process-images → execute-inference → convert-results → assemble-response
  1. validate-request: Validates input message structure and image format
  2. resolve-model: Determines target model (static config or dynamic from message)
  3. ensure-model: Downloads model from Firebase if needed, verifies model availability
  4. process-images: Reads and preprocesses images (format conversion, color space)
  5. execute-inference: Sends gRPC request to inference server, handles concurrency
  6. convert-results: Transforms raw protobuf results into output format
  7. assemble-response: Builds final message with results and performance metrics

Components ​

  1. Node-RED Node: Configuration, message routing, and pipeline orchestration
  2. Shared inference server: one Docker container for every inferencer node, holding many models at once (one worker process each)
  3. gRPC Protocol: Communication between Node-RED and containers
  4. Model Storage: /opt/storage/models/ directory
  5. Firebase: Optional model download source

Data Flow ​

Input Images → Pipeline → gRPC → Docker Container → ML Model → Predictions → Output
                                       ↓
                              Dynamic Worker Scaling

Supported Model Types ​

FrameworkTaskConfig Prefab
YOLODetectionyolo_detection
YOLOClassificationyolo_classification
YOLOSegmentationyolo_segmentation
PaddlePaddleDetectionpaddle_detection
PaddlePaddleOCRpaddle_ocr
PaddlePaddleRecognitionpaddle_recognition
PaddlePaddleDocument Rotationpaddle_document_rotation
PipelineOCRpipeline_ocr
RF-DETRDetectionrfdetr_detection
AnomalibAnomalyanomaly_anomaly

Configuration ​

Settings Tab ​

Name ​

  • Type: String
  • Optional: Yes
  • Description: Display name for the node

Input Field ​

  • Type: Message property path
  • Default: payload
  • Description: Message field containing input images (single image or array)

Output Field ​

  • Type: Message property path
  • Default: payload
  • Description: Where inference results will be stored
  • Note: Performance stats saved to msg.performance.inference.{node_name}

Model Source ​

  • Type: Select
  • Options:
    • Select here: Choose from available models in /opt/storage/models or from Firebase (unified dropdown)
    • Dynamic: Provide model name at runtime via message/flow/global property

Model Name (Static Mode) ​

  • Type: Dropdown (unified Firebase + local models)
  • Description: Select from available models — shows model name, task type, framework, and source (local/Firebase)

Model Field (Dynamic Mode) ​

  • Type: TypedInput (msg, flow, global)
  • Example: msg.modelName, flow.currentModel, global.activeModel
  • Description: Property path containing model name at runtime
  • Note: Enables automatic model download if Firebase is configured

Promise Mode ​

  • Type: Checkbox
  • Default: Disabled
  • Description: Return promises instead of waiting for results
  • Use Case: High-throughput batch processing — resolve promises in a downstream function node using Promise.all()

Promises Field ​

  • Type: Message property path
  • Default: promises
  • Visible: When Promise Mode enabled
  • Description: Array field to store pending promises

Maximum Concurrent Predictions ​

  • Type: Number
  • Range: 1-20
  • Default: 5
  • Description: Parallel requests per server
  • Behavior: Excess requests queue until slot available

Show Debug Image ​

  • Type: Checkbox
  • Default: Disabled
  • Description: Display processed images on Node-RED canvas

Debug Interval ​

  • Type: Number
  • Default: 1
  • Visible: When debug enabled
  • Description: Show every Nth image (1 = all images)

Debug Image Width ​

  • Type: Number (pixels)
  • Default: 160
  • Visible: When debug enabled
  • Description: Display width for debug images

Box Thickness / Font Size ​

  • Type: Number (1-10) / Number (0.3-2)
  • Default: 2 / 0.5
  • Visible: When debug enabled
  • Description: Stroke and caption size of the overlay, scaled with the image the way rp-add-bbs scales them

The debug image is drawn on the device, by the renderer shared with ocr-inferencer, triton-inferencer and vlm-online-inferencer, from the same result the message carries. What it shows depends on the task: box outlines with tag: confidence for detection, filled masks for segmentation and anomaly, the text of each box for OCR, and a corner tag for classification, recognition and document orientation.

JSON Config Tab ​

Advanced model configuration in JSON format. Structure varies by model type. Each model type has a corresponding config prefab that provides sensible defaults.

Common Fields (All Models) ​

json
{
  "common": {
    "model_name": "my-model",
    "config_type": "predict",
    "task": "Detection",
    "device": "auto",
    "image_shape": {
      "width": 640,
      "height": 640
    },
    "verbose": false,
    "max_batch": 100
  }
}

Fields:

  • model_name: Model identifier
  • config_type: Always "predict"
  • task: Detection, Classification, Segmentation, Anomaly, OCR, Recognition
  • device: "auto", "cpu", "cuda", "mps"
  • image_shape: Target dimensions for resizing
  • verbose: Enable detailed logging
  • max_batch: Maximum batch size

Detection Models (YOLO, RF-DETR, PaddlePaddle) ​

json
{
  "common": { },
  "task_specific": {
    "conf_threshold": 0.5,
    "nms_iou_threshold": 0.7,
    "max_det": 300
  }
}

Classification Models ​

json
{
  "common": { },
  "task_specific": {
    "top_k": 5
  }
}

Segmentation Models (YOLO) ​

json
{
  "common": { },
  "task_specific": {
    "conf_threshold": 0.5,
    "retina_masks": true
  }
}

Anomaly Detection ​

json
{
  "common": { },
  "task_specific": {
    "threshold": 0.5,
    "normalize": true
  },
  "model_specific": {
    "optional": {
      "image_threshold": 0.5,
      "pixel_threshold": 0.5
    }
  }
}

Constraint: min_batch <= opt_batch <= max_batch

Output formats ​

There is nothing to configure. Every result carries every coordinate format that can be derived from it — see Output.

The node used to have an output-format selector, which kept only the formats you ticked and deleted the rest, and among them a mask output: an RGBA bitmap the size of the image, rasterized from the contours, one per result item. Both are gone. The contours are always there, and a flow that needs pixels can rasterize them where it needs them instead of paying megabytes a message for a raster nothing downstream read.

Classes Tab ​

Map or filter model output classes.

Features:

  • Rename classes to custom labels
  • Filter results by remapping to empty string
  • Per-model configuration storage

Example:

Original: "person" → Custom: "human"
Original: "car" → Custom: "vehicle"
Original: "background" → Custom: "" (filtered out)

Image Format ​

The node accepts multiple image formats:

1. Rosepetal Bitmap Format ​

javascript
{
  width: 1920,
  height: 1080,
  data: Buffer,           // Raw pixel data
  colorSpace: "RGB",      // "GRAY", "RGB", "RGBA", "BGR", "BGRA"
  channels: 3,            // Auto-inferred if omitted
  dtype: "uint8"          // Currently only uint8 supported
}

Color space mapping:

  • GRAY: 1 channel
  • RGB / BGR: 3 channels
  • RGBA / BGRA: 4 channels

2. JPEG/PNG Buffers ​

javascript
msg.payload = fs.readFileSync('image.jpg');

The node automatically decodes standard image formats.

3. Array of Images ​

javascript
msg.payload = [image1, image2, image3];

Processes multiple images in batch for improved performance.

Input ​

Basic Input ​

javascript
msg.payload = imageBuffer;
return msg;

Dynamic Model Selection ​

javascript
// Configure node with Model Source: Dynamic, Model Field: msg.modelName
msg.modelName = "yolo-detection-v8";
msg.payload = imageBuffer;
return msg;

Warmup Request ​

javascript
msg.warmup = true;
return msg;

Triggers model warmup. The server generates a synthetic image internally — no need to provide one.

Inferencer Request (Control Messages) ​

javascript
// Load a specific model
msg.inferencerRequest = { type: "loadModel", modelName: "my-model" };

// Unload the current model
msg.inferencerRequest = { type: "unloadModel" };

// Trigger warmup
msg.inferencerRequest = { type: "warmup" };

Output ​

Results are written to msg.<outputField> (msg.inference by default), and the timings to msg.performance.inference.<nodeName>.

One image in, one result out; an array of images in, an array of results out, indexed by input order.

Within one image, detection and segmentation return a list — one entry per instance found, possibly empty. Classification, recognition, OCR, document orientation and anomaly return a single object, because they answer one question about the whole image.

Coordinates ​

Every coordinate is relative to the image, 0–1, rounded to 6 decimals, and area is the fraction of the image covered. This holds for every task, both inference nodes, and both backends, so a downstream rp-crop or rp-add-bbs works the same wherever the result came from and whatever the image size is.

Every result carries all the box formats, derived from the same geometry:

FormatShapeNotes
raw_boxes[[x0,y0], [x1,y1], [x2,y2], [x3,y3]]four corners, clockwise from top-left; keeps rotation
4points{x0, y0, x1, y1, x2, y2, x3, y3}the same corners as flat properties
cwh{x, y, w, h, r}centre, size, rotation in degrees
xywhr{x, y, w, h, r}top-left corner, size, rotation in degrees
2points{xmin, ymin, xmax, ymax}axis-aligned; rotation is lost

For a segmentation or anomaly result the box is the bounding box of the polygons. Picking a format is a downstream concern: nothing has to be configured here, and changing your mind costs no redeploy.

Detection ​

javascript
[
  {
    tag: "bottle",
    confidence: 0.95,
    classId: 1,
    color: { r: 255, g: 0, b: 0 },
    raw_boxes: [[0.1, 0.1], [0.9, 0.1], [0.9, 0.8], [0.1, 0.8]],
    "2points": { xmin: 0.1, ymin: 0.1, xmax: 0.9, ymax: 0.8 }
    // ...plus cwh, xywhr and 4points
  }
]

Segmentation ​

javascript
[
  {
    tag: "person",
    confidence: 0.92,
    classId: 0,
    area: 0.35,
    color: { r: 0, g: 255, b: 0 },
    polygons: [ [[0.1, 0.2], [0.5, 0.2], [0.5, 0.9], [0.1, 0.9]] ],
    "2points": { xmin: 0.1, ymin: 0.2, xmax: 0.5, ymax: 0.9 }
  }
]

polygons is a list of contours, so one instance can have several (a shape with holes). Instances with no contours are dropped.

Classification ​

javascript
{
  tag: "cat",
  confidence: 0.97,
  classId: 3,
  all_tags: [
    { tag: "cat", confidence: 0.97, classId: 3 },
    { tag: "dog", confidence: 0.02, classId: 1 }
  ]
}

Anomaly ​

javascript
{
  tag: "NOK",
  confidence: 0.88,
  classId: 1,
  area: 0.12,
  color: { r: 255, g: 0, b: 0 },
  polygons: [ [[0.3, 0.4], [0.6, 0.4], [0.6, 0.7], [0.3, 0.7]] ],
  "2points": { xmin: 0.3, ymin: 0.4, xmax: 0.6, ymax: 0.7 }
}

tag is OK or NOK, decided by confidence against the model's own threshold — 0.5 for an anomalib export, which is min-max normalized, so 0.5 is the learned threshold. The polygons and the area come from thresholding the anomaly map, not from the verdict, so an OK image can still carry a mask.

Changed. Anomaly used to return a list of one, with polygons in pixels of the model's mask (512×512 for a typical export) rather than relative to the image, and area as a count of those pixels. The same model therefore answered different numbers in this node and in triton-inferencer. Flows doing payload[0], or scaling those coordinates themselves, need updating.

Performance ​

javascript
{
  performance: {
    inference: {
      "my-inferencer": {
        inferenceTime: 45.2,
        preprocessTime: 5.1,
        postprocessTime: 3.8,
        totalTime: 54.1
      }
    }
  }
}

Promise Mode Output ​

javascript
{
  promises: [
    Promise { <pending> },
    Promise { <pending> }
  ]
}

Resolve in a function node:

javascript
msg.results = await Promise.all(msg.promises);
return msg;

Usage Examples ​

Example 1: Simple Object Detection ​

javascript
// Function node: Load image
msg.payload = {
  width: 640, height: 480,
  data: imageBuffer,
  colorSpace: "RGB"
};
return msg;

Inferencer configuration:

  • Model: yolo-v8-detection
  • Input: payload
  • Output: detections
javascript
// Function node: Filter high confidence
msg.payload = msg.detections.filter(det => det.box.confidence > 0.8);
return msg;

Example 2: Batch Processing ​

javascript
// Function node: Prepare batch
msg.payload = [
  { width: 640, height: 480, data: buffer1, colorSpace: "RGB" },
  { width: 640, height: 480, data: buffer2, colorSpace: "RGB" },
  { width: 640, height: 480, data: buffer3, colorSpace: "RGB" }
];
return msg;

Output:

javascript
msg.payload = [
  [detection1a, detection1b],  // Results from image 1
  [detection2a],               // Results from image 2
  []                           // No detections in image 3
];

Example 3: Dynamic Model Selection ​

javascript
// Function node: Choose model based on input
if (msg.topic === "quality-check") {
  msg.modelName = "defect-detection-model";
} else if (msg.topic === "classification") {
  msg.modelName = "product-classifier";
}
msg.payload = imageData;
return msg;

Example 4: Multi-Model Pipeline ​

[Camera] → [Inferencer: Detection] → [Function: Crop] → [Inferencer: Classification] → [Output]

Detection step:

javascript
msg.detections = msg.payload;
msg.payload = cropDetections(msg.payload); // Extract regions
return msg;

Classification step:

javascript
msg.classifications = msg.payload;
return msg;

Performance Optimization ​

Concurrency Settings ​

Low throughput (sporadic requests):

  • Workers: 1
  • Max concurrent: 5

High throughput (production):

  • Workers: 3-10
  • Max concurrent: 10-20

Batch Processing ​

Optimal batch size:

  • Small models (YOLO-nano): 10-20 images
  • Medium models (YOLO-v8): 5-10 images
  • Large models (Segmentation): 2-5 images

Model Warmup ​

Warmup is always enabled (since v1.2.1). The server generates a synthetic image internally on startup, eliminating cold-start latency on the first real request.

Docker Container Management ​

Container Lifecycle ​

  1. Startup: Node creates Docker container with inference server
  2. Health Check: Waits for gRPC handshake
  3. Warmup: Server-side synthetic image warmup
  4. Running: Process inference requests
  5. Shutdown: Clean container removal on deploy/close

Container Image ​

europe-southwest1-docker.pkg.dev/rosepetal-artifact/node-red-vp/rosepetal-detection:<tag>

Container Logs ​

bash
# View container logs
docker logs <container-id>

# Find inferencer containers
docker ps | grep rosepetal-serving

Memory Usage ​

Per container (approximate):

  • YOLO-nano: 500MB-1GB
  • YOLO-v8: 2GB-4GB
  • Segmentation: 4GB-8GB
  • PaddlePaddle OCR: 2GB-3GB

Worker scaling multiplies memory usage proportionally.

Model Management ​

Model Directory Structure ​

/opt/storage/models/
├── yolo-v8-detection/
│   ├── model.pt
│   └── config.json
├── product-classifier/
│   ├── model.onnx
│   └── config.json
└── segmentation-model/
    ├── model.pt
    └── config.json

Adding Models Manually ​

  1. Create directory: /opt/storage/models/<model-name>/
  2. Copy model file: model.pt, model.onnx, etc.
  3. Create config.json (optional, for auto-detection)
  4. Refresh Node-RED editor

Firebase Model Download ​

Requirements:

  • Firebase config node connected
  • Model exists in Firebase storage
  • Dynamic mode enabled or Firebase model selected in unified dropdown

Behavior:

  • Checks local storage first
  • Downloads if missing
  • Caches for future use
  • Shows download progress in status

Error Handling ​

Common Errors ​

"Docker image not available" ​

  • Cause: Inference Docker image not pulled
  • Solution: docker pull <image-name>

"Model not found" ​

  • Cause: Model doesn't exist in /opt/storage/models
  • Solution: Verify model name, check directory, download if needed

"Container failed to start" ​

  • Cause: Port conflict, insufficient memory, corrupted model
  • Solution: Check Docker logs, verify system resources

"gRPC connection failed" ​

  • Cause: Container not ready, network issue
  • Solution: Wait for container startup, check Docker status

"Invalid image format" ​

  • Cause: Missing required fields, wrong data type
  • Solution: Validate image object structure

"CUDA out of memory" ​

  • Cause: Batch too large, concurrent requests exceeded capacity
  • Solution: Reduce batch size, lower concurrency, use CPU

Debugging ​

Enable verbose logging:

json
{
  "common": {
    "verbose": true
  }
}

Check container logs:

bash
docker logs <container-id>

Enable debug visualization:

  • Check "Show debug image"
  • Set debug interval: 1
  • Verify images display correctly

Best Practices ​

Model Selection ​

  1. Match task to model: Detection vs Classification vs Segmentation
  2. Consider speed/accuracy tradeoff: nano (fast) vs large (accurate)
  3. Use the Triton Inferencer for TensorRT: this node serves PyTorch/Paddle models; TensorRT engines are the Triton node's job
  4. Test on representative data: Validate before production
  5. Version models: Track changes, enable rollback

Configuration ​

  1. Use config prefabs: Start from built-in configurations for each model type
  2. Document class mappings: Clear naming conventions
  3. Set appropriate thresholds: Balance false positives/negatives

Maintenance ​

  1. Clean up unused containers: docker system prune
  2. Monitor disk space: Models can be large
  3. Update Docker images: Stay current with improvements
  4. Log performance metrics: Track degradation over time

Troubleshooting ​

Slow Inference ​

Possible causes:

  • Model too large for hardware
  • CPU inference on GPU model
  • High concurrent load

Solutions:

  • Use a smaller model variant, or the Triton Inferencer for TensorRT
  • Enable GPU if available
  • Reduce concurrency
  • Cache models locally

High Memory Usage ​

Possible causes:

  • Too many models resident at once
  • Large batch sizes
  • Memory leak in model

Solutions:

  • Lower MAX_LOADED_MODELS, or set an idle timeout so unused models are evicted
  • Process smaller batches
  • Update to latest image

Inconsistent Results ​

Possible causes:

  • Wrong color space
  • Incorrect image dimensions
  • Threshold too sensitive

Solutions:

  • Validate input format
  • Check preprocessing
  • Adjust confidence thresholds
  • Enable debug visualization

See Also ​