Overview of OpenVINO Plugin Library#
The plugin architecture of OpenVINO allows to develop and plug independent inference
solutions dedicated to different devices. Physically, a plugin is represented as a dynamic library
exporting the single create_plugin_engine
function that allows to create a new plugin instance.
OpenVINO Plugin Library#
OpenVINO plugin dynamic library consists of several main components:
-
Provides information about devices of a specific type.
Can create an compiled model instance which represents a Neural Network backend specific graph structure for a particular device in opposite to the ov::Model which is backend-independent.
Can import an already compiled graph structure from an input stream to a compiled model object.
-
Is an execution configuration compiled for a particular device and takes into account its capabilities.
Holds a reference to a particular device and a task executor for this device.
Can create several instances of Inference Request.
Can export an internal backend specific graph structure to an output stream.
-
Runs an inference pipeline serially.
Can extract performance counters for an inference pipeline execution profiling.
Asynchronous Inference Request class:
Wraps the Inference Request class and runs pipeline stages in parallel on several task executors based on a device-specific pipeline structure.
-
Provides the plugin specific properties.
-
Provides the device specific remote context. Context allows to create remote tensors.
-
Provides the device specific remote tensor API and implementation.
Note
This documentation is written based on the Template
plugin, which demonstrates plugin development details. Find the complete code of the Template
, which is fully compilable and up-to-date, at <openvino source dir>/src/plugins/template
.