Class ov::Core#

class Core#

This class represents an OpenVINO runtime Core entity.

User applications can create several Core class instances, but in this case the underlying plugins are created multiple times and not shared between several Core instances. The recommended way is to have a single Core instance per application.

Unnamed Group

std::shared_ptr<ov::Model> read_model(const std::string &model_path, const std::string &bin_path = {}, const ov::AnyMap &properties = {}) const#

Reads models from IR / ONNX / PDPD / TF / TFLite file formats.

Parameters:

model_path – Path to a model.
bin_path – Path to a data file. For IR format (*.bin):
- if bin_path is empty, will try to read a bin file with the same name as xml and
- if the bin file with the same name is not found, will load IR without weights. For the following file formats the bin_path parameter is not used:
- ONNX format (*.onnx)
- PDPD (*.pdmodel)
- TF (*.pb, *.meta, SavedModel directory)
- TFLite (*.tflite)
properties – Optional map of pairs: (property name, property value) relevant only for this read operation.

Returns:

A model.

Unnamed Group

template<typename ...Properties> inline util::EnableIfAllStringAny<CompiledModel, Properties...> read_model(const std::string &model_path, const std::string &bin_path, Properties&&... properties) const#

Reads models from IR / ONNX / PDPD / TF / TFLite file formats.

Parameters:

model_path – Path to a model.
bin_path – Path to a data file. For IR format (*.bin):
- if bin_path is empty, will try to read a bin file with the same name as xml and
- if the bin file with the same name is not found, will load IR without weights. For the following file formats the bin_path parameter is not used:
- ONNX format (*.onnx)
- PDPD (*.pdmodel)
- TF (*.pb, *.meta, SavedModel directory)
- TFLite (*.tflite)
properties – Optional pack of pairs: (property name, property value) relevant only for this read operation.

Returns:

A model.

Unnamed Group

CompiledModel compile_model(const std::string &model_path, const AnyMap &properties = {})#

Reads and loads a compiled model from the IR/ONNX/PDPD file to the default OpenVINO device selected by the AUTO plugin.

This can be more efficient than using the Core::read_model + Core::compile_model(model_in_memory_object) flow, especially for cases when caching is enabled and a cached model is available.

Parameters:

model_path – Path to a model.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model.

Unnamed Group

template<typename ...Properties> inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::string &model_path, Properties&&... properties)#

Reads and loads a compiled model from IR / ONNX / PDPD file to the default OpenVINO device selected by AUTO plugin.

This can be more efficient than using read_model + compile_model(Model) flow especially for cases when caching is enabled and cached model is available

Template Parameters:

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters:

model_path – path to model with string or wstring
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation

Returns:

A compiled model

Unnamed Group

CompiledModel compile_model(const std::string &model_path, const std::string &device_name, const AnyMap &properties = {})#

Reads a model and creates a compiled model from the IR/ONNX/PDPD file.

This can be more efficient than using the Core::read_model + Core::compile_model(model_in_memory_object) flow, especially for cases when caching is enabled and a cached model is available.

Parameters:

model_path – Path to a model.
device_name – Name of a device to load a model to.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model.

Unnamed Group

template<typename ...Properties> inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::string &model_path, const std::string &device_name, Properties&&... properties)#

Reads a model and creates a compiled model from the IR/ONNX/PDPD file.

This can be more efficient than using read_model + compile_model(Model) flow especially for cases when caching is enabled and cached model is available.

Template Parameters:

Properties – Should be a pack of std::pair<std::string, ov::Any> types.

Parameters:

model_path – Path to a model.
device_name – Name of a device to load a model to.
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model.

Unnamed Group

void add_extension(const std::string &library_path)#

Registers an extension to a Core object.

Parameters:: library_path – Path to the library with ov::Extension.

Public Functions

explicit Core(const std::string &xml_config_file = {})#

Constructs an OpenVINO Core instance with devices and their plugins description.

There are two ways how to configure device plugins:

(default) Use XML configuration file in case of dynamic libraries build;
Use strictly defined configuration in case of static libraries build.

Parameters:

xml_config_file – Path to the .xml file with plugins to load from. If path contains only file name with extension, file will be searched in a folder with OpenVINO runtime shared library. If the XML configuration file is not specified, default OpenVINO Runtime plugins are loaded from:

(dynamic build) default plugins.xml file located in the same folder as OpenVINO runtime shared library;
(static build) statically defined configuration. In this case path to the .xml file is ignored.

std::map<std::string, Version> get_versions(const std::string &device_name) const#

Returns device plugins version information. Device name can be complex and identify multiple devices at once like HETERO:CPU,GPU; in this case, std::map contains multiple entries, each per device.

Parameters:: device_name – Device name to identify a plugin.
Returns:: A vector of versions.

std::shared_ptr<ov::Model> read_model(const std::string &model, const Tensor &weights) const#

Reads models from IR / ONNX / PDPD / TF / TFLite formats.

Note

Created model object shares the weights with the weights object. Thus, do not create weights on temporary data that can be freed later, since the model constant data will point to an invalid memory.

Parameters:

model – String with a model in IR / ONNX / PDPD / TF / TFLite format.
weights – Shared pointer to a constant tensor with weights. Reading ONNX / PDPD / TF / TFLite models does not support loading weights from the weights tensors.

Returns:

A model.

CompiledModel compile_model(const std::shared_ptr<const ov::Model> &model, const AnyMap &properties = {})#

Creates and loads a compiled model from a source model to the default OpenVINO device selected by the AUTO plugin.

Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources).

Parameters:

model – Model object acquired from Core::read_model.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model.

template<typename ...Properties> inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::shared_ptr<const ov::Model> &model, Properties&&... properties)#

Creates and loads a compiled model from a source model to the default OpenVINO device selected by AUTO plugin.

Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources)

Template Parameters:

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters:

model – Model object acquired from Core::read_model
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation

Returns:

A compiled model

CompiledModel compile_model(const std::shared_ptr<const ov::Model> &model, const std::string &device_name, const AnyMap &properties = {})#

Creates a compiled model from a source model object.

Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources).

Parameters:

model – Model object acquired from Core::read_model.
device_name – Name of a device to load a model to.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model.

template<typename ...Properties> inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::shared_ptr<const ov::Model> &model, const std::string &device_name, Properties&&... properties)#

Creates a compiled model from a source model object.

Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources)

Template Parameters:

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters:

model – Model object acquired from Core::read_model
device_name – Name of device to load model to
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation

Returns:

A compiled model

CompiledModel compile_model(const std::string &model, const ov::Tensor &weights, const std::string &device_name, const AnyMap &properties = {})#

Reads a model and creates a compiled model from the IR/ONNX/PDPD memory.

Note

Parameters:

model – String with a model in IR/ONNX/PDPD format.
weights – Shared pointer to a constant tensor with weights. Reading ONNX/PDPD models does not support loading weights from the weights tensors.
device_name – Name of a device to load a model to.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model.

template<typename ...Properties> inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::string &model, const ov::Tensor &weights, const std::string &device_name, Properties&&... properties)#

Reads a model and creates a compiled model from the IR/ONNX/PDPD memory.

Note

Parameters:

model – String with a model in IR/ONNX/PDPD format.
weights – Shared pointer to a constant tensor with weights. Reading ONNX/PDPD models does not support loading weights from the weights tensors.
device_name – Name of a device to load a model to.

Template Parameters:

Properties – Should be a pack of std::pair<std::string, ov::Any> types.

Returns:

A compiled model.

CompiledModel compile_model(const std::shared_ptr<const ov::Model> &model, const RemoteContext &context, const AnyMap &properties = {})#

Creates a compiled model from a source model within a specified remote context.

Parameters:

model – Model object acquired from Core::read_model.
context – A reference to a RemoteContext object.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model object.

template<typename ...Properties> inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::shared_ptr<const ov::Model> &model, const RemoteContext &context, Properties&&... properties)#

Creates a compiled model from a source model within a specified remote context.

Template Parameters:

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters:

model – Model object acquired from Core::read_model
context – Pointer to RemoteContext object
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation

Returns:

A compiled model object

void add_extension(const std::shared_ptr<ov::Extension> &extension)#

Registers an extension to a Core object.

Parameters:: extension – Pointer to the extension.

void add_extension(const std::vector<std::shared_ptr<ov::Extension>> &extensions)#

Registers extensions to a Core object.

Parameters:: extensions – Vector of loaded extensions.

template<class T, typename std::enable_if<std::is_base_of<ov::Extension, T>::value, bool>::type = true> inline void add_extension(const T &extension)#

Registers an extension to a Core object.

Parameters:: extension – Extension class that is inherited from the ov::Extension class.

template<class T, class ...Targs, typename std::enable_if<std::is_base_of<ov::Extension, T>::value, bool>::type = true> inline void add_extension(const T &extension, Targs... args)#

Registers extensions to a Core object.

Parameters:

extension – Extension class that is inherited from the ov::Extension class.
args – A list of extensions.

template<class T, typename std::enable_if<std::is_base_of<ov::op::Op, T>::value, bool>::type = true> inline void add_extension()#: Registers a custom operation inherited from ov::op::Op.

template<class T, class ...Targs, typename std::enable_if<std::is_base_of<ov::op::Op, T>::value && sizeof...(Targs), bool>::type = true> inline void add_extension()#: Registers custom operations inherited from ov::op::Op.

CompiledModel import_model(std::istream &model_stream, const std::string &device_name, const AnyMap &properties = {})#

Imports a compiled model from the previously exported one.

Parameters:

model_stream – std::istream input stream containing a model previously exported using the ov::CompiledModel::export_model method.
device_name – Name of a device to import a compiled model for. Note, if device_name device was not used to compile the original mode, an exception is thrown.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model.

template<typename ...Properties> inline util::EnableIfAllStringAny<CompiledModel, Properties...> import_model(std::istream &model_stream, const std::string &device_name, Properties&&... properties)#

Imports a compiled model from the previously exported one.

Template Parameters:

Properties – Should be the pack of std::pair<std::string, ov::Any> types.

Parameters:

model_stream – Model stream.
device_name – Name of a device to import a compiled model for. Note, if device_name device was not used to compile the original mode, an exception is thrown.
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model.

CompiledModel import_model(std::istream &model_stream, const RemoteContext &context, const AnyMap &properties = {})#

Imports a compiled model from the previously exported one with the specified remote context.

Parameters:

model_stream – std::istream input stream containing a model previously exported from ov::CompiledModel::export_model
context – A reference to a RemoteContext object. Note, if the device from context was not used to compile the original mode, an exception is thrown.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model.

template<typename ...Properties> inline util::EnableIfAllStringAny<CompiledModel, Properties...> import_model(std::istream &model_stream, const RemoteContext &context, Properties&&... properties)#

Imports a compiled model from the previously exported one with the specified remote context.

Template Parameters:

Properties – Should be the pack of std::pair<std::string, ov::Any> types.

Parameters:

model_stream – Model stream.
context – Pointer to a RemoteContext object.
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation.

Returns:

A compiled model.

SupportedOpsMap query_model(const std::shared_ptr<const ov::Model> &model, const std::string &device_name, const AnyMap &properties = {}) const#

Query device if it supports the specified model with specified properties.

Parameters:

device_name – Name of a device to query.
model – Model object to query.
properties – Optional map of pairs: (property name, property value).

Returns:

An object containing a map of pairs an operation name -> a device name supporting this operation.

template<typename ...Properties> inline util::EnableIfAllStringAny<SupportedOpsMap, Properties...> query_model(const std::shared_ptr<const ov::Model> &model, const std::string &device_name, Properties&&... properties) const#

Queries a device if it supports the specified model with specified properties.

Template Parameters:

Properties – Should be the pack of std::pair<std::string, ov::Any> types.

Parameters:

device_name – Name of a device to query.
model – Model object to query.
properties – Optional pack of pairs: (property name, property value) relevant only for this query operation.

Returns:

An object containing a map of pairs an operation name -> a device name supporting this operation.

void set_property(const AnyMap &properties)#

Sets properties for all the registered devices, acceptable keys can be found in openvino/runtime/properties.hpp.

Parameters:: properties – Map of pairs: (property name, property value).

template<typename ...Properties> inline util::EnableIfAllStringAny<void, Properties...> set_property(Properties&&... properties)#

Sets properties for all the registered devices, acceptable keys can be found in openvino/runtime/properties.hpp.

Template Parameters:: Properties – Should be a pack of std::pair<std::string, ov::Any> types.
Parameters:: properties – Optional pack of pairs: property name, property value.

void set_property(const std::string &device_name, const AnyMap &properties)#

Sets properties for a device, acceptable keys can be found in openvino/runtime/properties.hpp.

Parameters:

device_name – Name of a device.
properties – Map of pairs: (property name, property value).

template<typename ...Properties> inline util::EnableIfAllStringAny<void, Properties...> set_property(const std::string &device_name, Properties&&... properties)#

Sets properties for a device, acceptable keys can be found in openvino/runtime/properties.hpp.

Template Parameters:

Properties – Should be the pack of std::pair<std::string, ov::Any> types.

Parameters:

device_name – Name of a device.
properties – Optional pack of pairs: (property name, property value).

Any get_property(const std::string &device_name, const std::string &name) const#

Gets properties related to device behaviour.

The method extracts information that can be set via the set_property method.

Parameters:

device_name – Name of a device to get a property value.
name – Property name.

Returns:

Value of a property corresponding to the property name.

Any get_property(const std::string &device_name, const std::string &name, const AnyMap &arguments) const#

Gets properties related to device behaviour.

The method extracts information that can be set via the set_property method.

Parameters:

device_name – Name of a device to get a property value.
name – Property name.
arguments – Additional arguments to get a property.

Returns:

Value of a property corresponding to the property name.

inline Any get_property(const std::string &name) const#

Gets properties related to core behaviour.

The method extracts information that can be set via the set_property method.

Parameters:: name – Property name.
Returns:: Value of a property corresponding to the property name.

template<typename T, PropertyMutability M> inline T get_property(const std::string &device_name, const ov::Property<T, M> &property) const#

Gets properties related to device behaviour.

The method is needed to request common device or system properties. It can be device name, temperature, and other devices-specific values.

Template Parameters:

T – Type of a returned value.
M – Property mutability.

Parameters:

device_name – Name of a device to get a property value.
property – Property object.

Returns:

Property value.

template<typename T, PropertyMutability M> inline T get_property(const std::string &device_name, const ov::Property<T, M> &property, const AnyMap &arguments) const#

Gets properties related to device behaviour.

The method is needed to request common device or system properties. It can be device name, temperature, other devices-specific values.

Template Parameters:

T – Type of a returned value.
M – Property mutability.

Parameters:

device_name – Name of a device to get a property value.
property – Property object.
arguments – Additional arguments to get a property.

Returns:

Property value.

template<typename T, PropertyMutability M, typename ...Args> inline util::EnableIfAllStringAny<T, Args...> get_property(const std::string &device_name, const ov::Property<T, M> &property, Args&&... args) const#

Gets properties related to device behaviour.

The method is needed to request common device or system properties. It can be device name, temperature, other devices-specific values.

Template Parameters:

T – Type of a returned value.
M – Property mutability.
Args – Set of additional arguments ended with property object variable.

Parameters:

device_name – Name of a device to get a property value.
property – Property object.
args – Optional pack of pairs: (argument name, argument value) ended with property object.

Returns:

Property value.

std::vector<std::string> get_available_devices() const#

Returns devices available for inference. Core objects go over all registered plugins and ask about available devices.

Returns:: A vector of devices. The devices are returned as { CPU, GPU.0, GPU.1, NPU }. If there is more than one device of a specific type, they are enumerated with the .# suffix. Such enumerated device can later be used as a device name in all Core methods like Core::compile_model, Core::query_model, Core::set_property and so on.

void register_plugin(const std::string &plugin, const std::string &device_name, const ov::AnyMap &config = {})#

Note

For security purposes it suggested to specify absolute path to register plugin.

Parameters:

plugin – Path (absolute or relative) or name of a plugin. Depending on platform, plugin is wrapped with shared library suffix and prefix to identify library full name. For example, on Linux platform, plugin name specified as plugin_name will be wrapped as libplugin_name.so. Plugin search algorithm:
- If plugin points to an exact library path (absolute or relative), it will be used.
- If plugin specifies file name (libplugin_name.so) or plugin name (plugin_name), it will be searched by file name (libplugin_name.so) in CWD or in paths pointed by PATH/LD_LIBRARY_PATH/DYLD_LIBRARY_PATH environment variables depending on the platform.
device_name – Device name to register a plugin for.
config – Plugin configuration options

void unload_plugin(const std::string &device_name)#

Unloads the previously loaded plugin identified by device_name from OpenVINO Runtime. The method is needed to remove loaded plugin instance and free its resources. If plugin for a specified device has not been created before, the method throws an exception.

Note

This method does not remove plugin from the plugins known to OpenVINO Core object.

Parameters:: device_name – Device name identifying plugin to remove from OpenVINO Runtime.

void register_plugins(const std::string &xml_config_file)#

Registers a device plugin to the OpenVINO Runtime Core instance using an XML configuration file with plugins description.

The XML file has the following structure:

<ie>
    <plugins>
        <plugin name="" location="">
            <extensions>
                <extension location=""/>
            </extensions>
            <properties>
                <property key="" value=""/>
            </properties>
        </plugin>
    </plugins>
</ie>

name identifies name of a device enabled by a plugin.
location specifies absolute path to dynamic library with a plugin. The path can also be relative to XML file directory. It allows having common config for different systems with different configurations.
properties are set to a plugin via the ov::Core::set_property method.
extensions are set to a plugin via the ov::Core::add_extension method.

Note

For security purposes it suggested to specify absolute path to register plugin.

Parameters:: xml_config_file – A path to .xml file with plugins to register.

RemoteContext create_context(const std::string &device_name, const AnyMap &remote_properties)#

Creates a new remote shared context object on the specified accelerator device using specified plugin-specific low-level device API parameters (device handle, pointer, context, etc.).

Parameters:

device_name – Name of a device to create a new shared context on.
remote_properties – Map of device-specific shared context remote properties.

Returns:

Reference to a created remote context.

template<typename ...Properties> inline util::EnableIfAllStringAny<RemoteContext, Properties...> create_context(const std::string &device_name, Properties&&... remote_properties)#

Creates a new shared context object on specified accelerator device using specified plugin-specific low level device API properties (device handle, pointer, etc.)

Template Parameters:

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters:

device_name – Name of a device to create new shared context on.
remote_properties – Pack of device-specific shared context remote properties.

Returns:

A shared pointer to a created remote context.

RemoteContext get_default_context(const std::string &device_name)#

Gets a pointer to default (plugin-supplied) shared context object for the specified accelerator device.

Parameters:: device_name – Name of a device to get a default shared context from.
Returns:: Reference to a default remote context.