Post-Training Optimization Toolkit


Starting with the 2020.1 version, OpenVINO™ toolkit delivers the Post-Training Optimization Tool designed to accelerate the inference of DL models by converting them into a more hardware-friendly representation by applying specific methods that do not require re-training, for example, post-training quantization. For more details about the low-precision flow in OpenVINO™, refer to the Low Precision Optimization Guide.

Post-Training Optimization Toolkit includes standalone command-line tool and Python* API that provide the following key features:


Install Post-Training Optimization Toolkit

The toolkit is distributed as a part of the OpenVINO release package and available after installation from the <INSTALL_DIR>/deployment_tools/tools/post_training_optimization_toolkit. It's recommended to create a separate Python* environment before installing the OpenVINO components. To install the toolkit into your environment, follow the steps below:

  1. Before using optimization toolkit, install the AccuracyChecker component:
    1. Navigate to <INSTALL_DIR>/deployment_tools/open_model_zoo/tools/accuracy_checker.
    2. Run the script:
      python3 install
  2. Install the toolkit:
    1. Navigate to <INSTALL_DIR>/deployment_tools/tools/post_training_optimization_toolkit.
    2. Run the script:
      python3 install

Now the toolkit is available in the command line by the pot alias.

Use Post-Training Optimization Command-line Tool

The toolkit works with pre-trained models in the OpenVINO Intermediate Representation (IR) format, which means that you need to convert the model using Model Optimizer before running the optimization tool. In addition, it's highly recommended to make sure that the model can be successfully inferred and achieves the similar accuracy numbers as the reference model from the original framework. In order to do this, the AccuracyChecker tool can be used.

To run the command-line Post-Training Optimization Tool:

  1. Activate OpenVINO environment in the command-line shell running the following script:
    source /opt/intel/openvino/bin/
  2. Prepare a configuration file for the tool using the examples in the configs folder. To simplify this step, an AccuracyChecker configuration file for the full-precision model can be used and referred.
  3. Launch the command-line tool with the configuration file running the following command:
    pot -c <path_to_config_file>
    For all available usage options, use the -h, --help arguments or refer to the Command-Line Arguments below.
  4. By default, the results are dumped into the separate output subfolder inside the results folder that is created in the same directory where the tool is run from. Use the -e option to evaluate the accuracy directly from the tool.

Command-Line Arguments

The following command-line options are available to run the tool:

Argument Description
-h, --help Optional. Show help message and exit.
-c CONFIG, --config CONFIG Path to a config file with task/model-specific parameters.
-e, --evaluate Optional. Evaluate model on the whole dataset after optimization.
--output-dir OUTPUT_DIR Optional. A directory where results are saved. Default: ./results.
-sm, --save-model Optional. Save the original (FP32) model.
-d, --direct-dump Optional. Save results directly to output directory without additional subfolders.
--log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG} Optional. Log level to print. INFO is by default.