Using Python Dependencies with node-red-contrib-rosepetal-python-executor

This guide shows how to use external Python libraries like OpenCV, TensorFlow, scikit-learn, etc. with the Python executor node.

Method 1: Global Installation (Simplest)

Install packages globally using pip:

# Install OpenCV
pip3 install opencv-python opencv-contrib-python

# Install other common libraries
pip3 install numpy pandas matplotlib scikit-learn tensorflow torch pillow requests

Pros: Simple, works immediately Cons: Affects system Python, potential version conflicts

Method 2: Virtual Environment (Recommended)

Create a dedicated virtual environment for your Node-RED Python dependencies:

# Create virtual environment
python3 -m venv /opt/node-red-python-env

# Activate it
source /opt/node-red-python-env/bin/activate

# Install packages
pip install opencv-python opencv-contrib-python numpy pandas matplotlib pillow

# Deactivate
deactivate

In Node-RED: Set Python Path to /opt/node-red-python-env/bin/python

Pros: Clean separation, no system conflicts, reproducible Cons: Requires more setup

Method 3: Conda Environment

Using conda for complex dependency management:

# Create conda environment
conda create -n node-red-python python=3.9

# Activate
conda activate node-red-python

# Install packages
conda install opencv numpy pandas matplotlib pillow scikit-learn

# Or mix conda and pip
pip install some-pip-only-package

In Node-RED: Set Python Path to /path/to/conda/envs/node-red-python/bin/python

Method 4: Docker Container

For complex setups or when you need specific system libraries:

FROM python:3.9-slim

# Install system dependencies
RUN apt-get update && apt-get install -y \
    libgl1-mesa-glx \
    libglib2.0-0 \
    libsm6 \
    libxext6 \
    libxrender-dev \
    libgomp1

# Install Python packages
RUN pip install opencv-python numpy pandas matplotlib pillow

# Set working directory
WORKDIR /app

# Copy your Node-RED flow runner
COPY lib/python-runner.py .

# Run as non-root user
USER 1000:1000

CMD ["python", "python-runner.py"]

Common Libraries and Installation

Computer Vision

# OpenCV
pip install opencv-python opencv-contrib-python

# Pillow (PIL)
pip install Pillow

# scikit-image
pip install scikit-image

Machine Learning

# TensorFlow
pip install tensorflow

# PyTorch
pip install torch torchvision

# scikit-learn
pip install scikit-learn

# XGBoost
pip install xgboost

Data Science

# NumPy, Pandas, Matplotlib
pip install numpy pandas matplotlib seaborn

# Jupyter (for development)
pip install jupyter

Web & APIs

# Requests
pip install requests

# Flask/FastAPI (for creating web endpoints)
pip install flask fastapi uvicorn

Example: Using OpenCV for Image Processing

Here's a complete example of using OpenCV in the Python executor:

1. Install OpenCV

pip3 install opencv-python numpy

2. Node-RED Flow Example

import cv2
import numpy as np
import base64

# Get image from message (automatically decoded from base64)
image_data = msg['payload']

# Convert to OpenCV format
if isinstance(image_data, str):
    # If it's base64 string, decode it
    image_bytes = base64.b64decode(image_data)
    nparr = np.frombuffer(image_bytes, np.uint8)
    img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
else:
    # If it's already numpy array
    img = image_data

# Apply OpenCV operations
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (15, 15), 0)
edges = cv2.Canny(blurred, 50, 150)

# Convert back to color for output
result = cv2.cvtColor(edges, cv2.COLOR_GRAY2BGR)

# Encode result back to base64 (automatic)
_, buffer = cv2.imencode('.jpg', result)
msg['payload'] = buffer.tobytes()

# Add metadata
msg['operation'] = 'edge_detection'
msg['original_shape'] = img.shape
msg['processed_shape'] = result.shape

return msg

3. Advanced OpenCV Example

import cv2
import numpy as np

# Load image
img = cv2.imdecode(np.frombuffer(msg['payload'], np.uint8), cv2.IMREAD_COLOR)

# Face detection
face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')
faces = face_cascade.detectMultiScale(img, 1.3, 5)

# Draw rectangles around faces
for (x, y, w, h) in faces:
    cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)

# Encode result
_, buffer = cv2.imencode('.jpg', img)
msg['payload'] = buffer.tobytes()
msg['faces_detected'] = len(faces)
msg['face_coordinates'] = faces.tolist()

return msg

Troubleshooting

Common Issues

1. ModuleNotFoundError: Library not installed in the Python environment

   # Check which Python the node is using
   which python3
   
   # Install in the correct environment
   pip3 install package-name

2. Permission Errors: Node-RED user can't access Python environment

   # Make sure Node-RED user has access to the Python environment
   sudo chown -R node-red:node-red /opt/node-red-python-env

3. Version Conflicts: Different libraries need different versions

   # Use virtual environment to isolate dependencies
   python3 -m venv myenv
   source myenv/bin/activate
   pip install specific-versions

Performance Tips

1. Use Persistent Mode: For libraries with slow import times (TensorFlow, OpenCV)
2. Preload Libraries: Import expensive libraries at the top of your code
3. Cache Models: Load ML models once and reuse them
4. Memory Management: Use del for large objects when done

Security Considerations

1. Virtual Environment: Use isolated environments for production
2. User Permissions: Run Node-RED with limited user permissions
3. Input Validation: Validate all inputs before processing
4. Timeout Settings: Set appropriate timeouts for long-running operations

Example Virtual Environment Setup Script

#!/bin/bash
# setup-python-env.sh

ENV_PATH="/opt/node-red-python"
NODE_RED_USER="node-red"

# Create virtual environment
python3 -m venv $ENV_PATH

# Activate environment
source $ENV_PATH/bin/activate

# Upgrade pip
pip install --upgrade pip

# Install common packages
pip install \
    opencv-python \
    opencv-contrib-python \
    numpy \
    pandas \
    matplotlib \
    pillow \
    scikit-learn \
    requests \
    flask

# Set ownership
sudo chown -R $NODE_RED_USER:$NODE_RED_USER $ENV_PATH

echo "Virtual environment created at: $ENV_PATH"
echo "Set Python Path in Node-RED to: $ENV_PATH/bin/python"

Run with:

chmod +x setup-python-env.sh
./setup-python-env.sh