Deploy with OpenVINO

Once the OpenVINO application development has been finished, usually application developers need to deploy their applications to end users. There are several ways how to achieve that:

  • Set a dependency on existing prebuilt packages (so called centralized distribution):

    • Using Debian / RPM packages, a recommended way for a family of Linux operation systems

    • Using pip package manager on PyPi, default approach for Python-based applications

    • Using Docker images. If the application should be deployed as a Docker image, developer can use a pre-built runtime OpenVINO Docker image as a base image in the Dockerfile for the application container image. You can find more info about available OpenVINO Docker images in the Install Guides for Linux and Windows. Also, if you need to customize OpenVINO Docker image, you can use Docker CI Framework to generate a Dockerfile and built it.

  • Grab a necessary functionality of OpenVINO together with your application (so-called local distribution):

The table below shows which distribution type can be used depending on target operation system:

Distribution type

Operation systems

Debian packages

Ubuntu 18.04 long-term support (LTS), 64-bit; Ubuntu 20.04 long-term support (LTS), 64-bit

RMP packages

Red Hat Enterprise Linux 8, 64-bit

Docker images

Ubuntu 18.04 long-term support (LTS), 64-bit; Ubuntu 20.04 long-term support (LTS), 64-bit; Red Hat Enterprise Linux 8, 64-bit; Windows Server Core base LTSC 2019, 64-bit; Windows 10, version 20H2, 64-bit

PyPi (pip package manager)

See https://pypi.org/project/openvino/

OpenVINO Deployment Manager

All operation systems

Local distribution

All operation systems

Build OpenVINO statically and link into the final app

All operation systems

Depending on the distribution type, the granularity of OpenVINO packages may vary: PyPi distribution OpenVINO has a single package ‘openvino’ containing all the runtime libraries and plugins, while more configurable ways like Local distribution provide higher granularity, so it is important to now some details about the set of libraries which are part of OpenVINO Runtime package:

_images/deployment_simplified.png
  • The main library openvino is used by C++ user’s applications to link against with. The library provides all OpenVINO Runtime public API for both OpenVINO API 2.0 and Inference Engine, nGraph APIs. For C language applications openvino_c is additionally required for distribution.

  • The optional plugin libraries like openvino_intel_cpu_plugin (matching openvino_.+_plugin pattern) are used to provide inference capabilities on specific devices or additional capabitilies like Hetero execution or Multi-Device execution.

  • The optional plugin libraries like openvino_ir_frontnend (matching openvino_.+_frontend) are used to provide capabilities to read models of different file formats like OpenVINO IR, ONNX or Paddle.

The optional means that if the application does not use the capability enabled by the plugin, the plugin’s library or package with the plugin is not needed in the final distribution.

The information above covers granularity aspects of majority distribution types, more detailed information is only needed and provided in Local Distribution.

Note

Depending on target OpenVINO devices, you also have to use Configurations for GPU, Configurations for GNA, Configurations for NCS2 or Configurations for VPU for proper configuration of deployed machines.