Converting YOLO* Models to the Intermediate Representation (IR)

This tutorial explains how to convert real-time object detection YOLOv1*, YOLOv2*, and YOLOv3* public models to the Intermediate Representation (IR). All YOLO* models are originally implemented in the DarkNet* framework and consist of two files:

Depending on a YOLO model version, the Model Optimizer converts it differently:

Convert YOLOv3 Model to IR

On GitHub*, you can find several public versions of TensorFlow YOLOv3 model implementation. This tutorial explains how to convert YOLOv3 model from the https://github.com/mystic123/tensorflow-yolo-v3 repository (commit ed60b90) to IR , but the process is similar for other versions of TensorFlow YOLOv3 model.

Overview of YOLOv3 Model Architecture

Originally, YOLOv3 model includes feature extractor called Darknet-53 with three branches at the end that make detections at three different scales. These branches must end with the YOLO Region layer.

Region layer was first introduced in the DarkNet framework. Other frameworks, including TensorFlow, do not have the Region implemented as a single layer, so every author of public YOLOv3 model creates it using simple layers. This badly affects performance. For this reason, the main idea of YOLOv3 model conversion to IR is to cut off these custom Region-like parts of the model and complete the model with the Region layers where required.

Dump YOLOv3 TensorFlow* Model

To dump TensorFlow model out of https://github.com/mystic123/tensorflow-yolo-v3 GitHub repository (commit ed60b90), follow the instructions below:

  1. Clone the repository:
    git clone https://github.com/mystic123/tensorflow-yolo-v3.git
    cd tensorflow-yolo-v3
  2. (Optional) Checkout to the commit that the conversion was tested on:
    git checkout ed60b90
  3. Download coco.names file from the DarkNet website OR use labels that fit your task.
  4. Download the yolov3.weights (for the YOLOv3 model) or yolov3-tiny.weights (for the YOLOv3-tiny model) file OR use your pretrained weights with the same structure
  5. Run a converter:

If you have YOLOv3 weights trained for an input image with the size different from 416 (320, 608 or your own), please provide the --size key with the size of your image specified while running the converter. For example, run the following command for an image with size 608:

python3 convert_weights_pb.py --class_names coco.names --data_format NHWC --weights_file yolov3_608.weights --size 608

Convert YOLOv3 TensorFlow Model to the IR

To solve the problems explained in the YOLOv3 architecture overview section, use the yolo_v3.json or yolo_v3_tiny.json (depending on a model) configuration file with custom operations located in the <OPENVINO_INSTALL_DIR>/deployment_tools/model_optimizer/extensions/front/tf repository.

It consists of several attributes:

[
{
"id": "TFYOLOV3",
"match_kind": "general",
"custom_attributes": {
"classes": 80,
"coords": 4,
"num": 9,
"mask": [0, 1, 2],
"entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]
}
}
]

where:

To generate the IR of the YOLOv3 TensorFlow model, run:

python3 mo_tf.py
--input_model /path/to/yolo_v3.pb
--tensorflow_use_custom_operations_config $MO_ROOT/extensions/front/tf/yolo_v3.json
--batch 1

To generate the IR of the YOLOv3-tiny TensorFlow model, run:

python3 mo_tf.py
--input_model /path/to/yolo_v3_tiny.pb
--tensorflow_use_custom_operations_config $MO_ROOT/extensions/front/tf/yolo_v3_tiny.json
--batch 1

where:

OpenVINO™ toolkit provides a demo that uses YOLOv3 model. For more information, refer to Object Detection YOLO* V3 Demo, Async API Performance Showcase.

Convert YOLOv1 and YOLOv2 Models to the IR

Before converting Choose a YOLOv1 or YOLOv2 model version that best suits your task. Download model configuration file and corresponding weight file:

To convert DarkNet YOLOv1 and YOLOv2 models to IR, follow the next stepsg:

  1. Install DarkFlow
  2. Convert DarkNet YOLOv1 or YOLOv2 model to TensorFlow using DarkFlow
  3. Convert TensorFlow YOLOv1 or YOLOv2 model to IR

Install DarkFlow*

You need DarkFlow to convert YOLOv1 and YOLOv2 models to TensorFlow. To install DarkFlow:

  1. Install DarkFlow required dependencies.
  2. Clone DarkFlow git repository:
    git clone https://github.com/thtrieu/darkflow.git
  3. Go to the root directory of the cloned repository:
    cd darkflow
  4. Install DarkFlow using the instructions from the README.md file in the DarkFlow repository.

Convert DarkNet* YOLOv1 or YOLOv2 Model to TensorFlow*

To convert YOLOv1 or YOLOv2 model to TensorFlow, go to the root directory of the cloned DarkFlow repository and run the following command:

python3 ./flow --model <path_to_model>/<model_name>.cfg --load <path_to_model>/<model_name>.weights --savepb

If the model was successfully converted, you can find the <model_name>.meta and <model_name>.pb files in built_graph subdirectory of the cloned DarkFlow repository.

File <model_name>.pb is a TensorFlow representation of the YOLO model.

Convert TensorFlow YOLOv1 or YOLOv2 Model to the IR

Converted TensorFlow YOLO model is missing Region layer and its parameters. Original YOLO Region layer parameters are stored in the configuration <path_to_model>/<model_name>.cfg file under the [region] title.

To recreate the original model structure, use the corresponding yolo .json configuration file with custom operations and Region layer parameters when converting the model to the IR. This file is located in the <OPENVINO_INSTALL_DIR>/deployment_tools/model_optimizer/extensions/front/tf directory.

If chosen model has specific values of this parameters, create another configuration file with custom operations and use it for conversion.

To generate the IR of the YOLOv1 model, provide TensorFlow YOLOv1 or YOLOv2 model to the Model Optimizer with the following parameters:

python3 ./mo_tf.py
--input_model <path_to_model>/<model_name>.pb \
--batch 1 \
--tensorflow_use_custom_operations_config <OPENVINO_INSTALL_DIR>/deployment_tools/model_optimizer/extensions/front/tf/<yolo_config>.json

where: