Dynamic Shape with a Custom Node

Introduction

This guide shows how to configure a simple Directed Acyclic Graph (DAG) with a custom node that performs input resizing before passing input data to the model.

The node below is provided as a demonstration. See instructions for how to build and use the custom node: Image Transformation.

To run inference with this setup, we will use the following:

When using the face_detection_retail_0004 model with the face_detection.py script, images are loaded and resized to the desired width and height. Then the output from the server is processed and inference results are displayed with bounding boxes drawn around the detected faces.

Steps

Clone OpenVINO™ Model Server GitHub repository and enter model_server directory.

git clone https://github.com/openvinotoolkit/model_server.git
cd model_server

Download the Pretrained Model

Download the model files and store them in the models directory:

mkdir -p models/face_detection/1
curl https://storage.openvinotoolkit.org/repositories/open_model_zoo/2022.1/models_bin/3/face-detection-retail-0004/FP32/face-detection-retail-0004.bin https://storage.openvinotoolkit.org/repositories/open_model_zoo/2022.1/models_bin/3/face-detection-retail-0004/FP32/face-detection-retail-0004.xml -o models/face_detection/1/face-detection-retail-0004.bin -o models/face_detection/1/face-detection-retail-0004.xml

Pull the Latest Model Server Image

Pull the latest version of OpenVINO™ Model Server from Docker Hub :

docker pull openvino/model_server:latest

Build a Custom Node

  1. Go to the custom node examples directory:

    cd src/custom_nodes
    
  2. Build the custom node:

    # replace to 'redhat` if using UBI base image
    export BASE_OS=ubuntu
    
    make BASE_OS=${BASE_OS} NODES=image_transformation
    
  3. Copy the custom node to the models repository:

    cp lib/${BASE_OS}/libcustom_node_image_transformation.so ../../models/libcustom_node_image_transformation.so
    

Create Model Server Configuration File

Go to the models directory:

cd ../../models

Create a new file named config.json in the models directory:

echo '{
    "model_config_list": [
        {
            "config": {
                "name": "face_detection_retail",
                "base_path": "/models/face_detection"
            }
        }
    ],
    "custom_node_library_config_list": [
        {"name": "image_transformation",
            "base_path": "/models/libcustom_node_image_transformation.so"}
    ],
    "pipeline_config_list": [
        {
            "name": "face_detection",
            "inputs": ["data"],
            "nodes": [
                {
                    "name": "image_transformation_node",
                    "library_name": "image_transformation",
                    "type": "custom",
                    "params": {
                        "target_image_width": "300",
                        "target_image_height": "300",

                        "mean_values": "[123.675,116.28,103.53]",
                        "scale_values": "[58.395,57.12,57.375]",

                        "original_image_color_order": "BGR",
                        "target_image_color_order": "BGR",

                        "original_image_layout": "NCHW",
                        "target_image_layout": "NCHW"
                    },
                    "inputs": [
                        {"image": {
                                "node_name": "request",
                                "data_item": "data"}}],
                    "outputs": [
                        {"data_item": "image",
                            "alias": "transformed_image"}]
                },
                {
                    "name": "face_detection_node",
                    "model_name": "face_detection_retail",
                    "type": "DL model",
                    "inputs": [
                        {"data": 
                            {
                             "node_name": "image_transformation_node",
                             "data_item": "transformed_image"
                            }
                        }
                    ],
                    "outputs": [
                        {"data_item": "detection_out",
                         "alias": "face_detection_output"}
                    ]
                }
            ],
            "outputs": [
                {"detection_out": {
                        "node_name": "face_detection_node",
                        "data_item": "face_detection_output"}}
            ]
        }
    ]
}' >> config.json

Start Model Server Container with Downloaded Model

Start the container with the image pulled in the previous step and mount <models_dir> :

docker run --rm -d -v ${PWD}:/models -p 9000:9000 openvino/model_server:latest --config_path /models/config.json --port 9000

Run the Client

cd ../demos/face_detection/python
virtualenv .venv
. .venv/bin/activate
pip install -r ../../common/python/requirements.txt
mkdir results_500x500

python face_detection.py --grpc_port 9000 --width 500 --height 500 --input_images_dir ../../common/static/images/people --output_dir results_500x500 --model_name face_detection

mkdir results_600x400

python face_detection.py --grpc_port 9000 --width 600 --height 400 --input_images_dir ../../common/static/images/people --output_dir results_600x400 --model_name face_detection

Results of running the client will be available in directories specified in --output_dir