Class ov::CompiledModel#

class CompiledModel#

This class represents a compiled model.

A model is compiled by a specific device by applying multiple optimization transformations, then mapping to compute kernels.

Public Functions

CompiledModel() = default#

Default constructor.

~CompiledModel()#

Destructor that preserves unloading order of an implementation object and reference to library.

std::shared_ptr<const Model> get_runtime_model() const#

Gets runtime model information from a device. This object represents an internal device-specific model that is optimized for a particular accelerator. It contains device-specific nodes, runtime information and can be used only to understand how the source model is optimized and which kernels, element types, and layouts are selected for optimal inference.

Returns:

A model containing Executable Graph Info.

const std::vector<ov::Output<const ov::Node>> &inputs() const#

Gets all inputs of a compiled model. Inputs are represented as a vector of outputs of the ov::op::v0::Parameter operations. They contain information about input tensors such as tensor shape, names, and element type.

Returns:

std::vector of model inputs.

const ov::Output<const ov::Node> &input() const#

Gets a single input of a compiled model. The input is represented as an output of the ov::op::v0::Parameter operation. The input contains information about input tensor such as tensor shape, names, and element type.

Note

If a model has more than one input, this method throws ov::Exception.

Returns:

Compiled model input.

const ov::Output<const ov::Node> &input(size_t i) const#

Gets input of a compiled model identified by i. The input contains information about input tensor such as tensor shape, names, and element type.

Note

The method throws ov::Exception if input with the specified index i is not found.

Parameters:

i – Index of input.

Returns:

Compiled model input.

const ov::Output<const ov::Node> &input(const std::string &tensor_name) const#

Gets input of a compiled model identified by tensor_name. The input contains information about input tensor such as tensor shape, names, and element type.

Note

The method throws ov::Exception if input with the specified tensor name tensor_name is not found.

Parameters:

tensor_name – The input tensor name.

Returns:

Compiled model input.

const std::vector<ov::Output<const ov::Node>> &outputs() const#

Get all outputs of a compiled model. Outputs are represented as a vector of output from the ov::op::v0::Result operations. Outputs contain information about output tensors such as tensor shape, names, and element type.

Returns:

std::vector of model outputs.

const ov::Output<const ov::Node> &output() const#

Gets a single output of a compiled model. The output is represented as an output from the ov::op::v0::Result operation. The output contains information about output tensor such as tensor shape, names, and element type.

Note

If a model has more than one output, this method throws ov::Exception.

Returns:

Compiled model output.

const ov::Output<const ov::Node> &output(size_t i) const#

Gets output of a compiled model identified by index. The output contains information about output tensor such as tensor shape, names, and element type.

Note

The method throws ov::Exception if output with the specified index index is not found.

Parameters:

i – Index of input.

Returns:

Compiled model output.

const ov::Output<const ov::Node> &output(const std::string &tensor_name) const#

Gets output of a compiled model identified by tensor_name. The output contains information about output tensor such as tensor shape, names, and element type.

Note

The method throws ov::Exception if output with the specified tensor name tensor_name is not found.

Parameters:

tensor_nameOutput tensor name.

Returns:

Compiled model output.

InferRequest create_infer_request()#

Creates an inference request object used to infer the compiled model. The created request has allocated input and output tensors (which can be changed later).

Returns:

InferRequest object

void export_model(std::ostream &model_stream)#

Exports the current compiled model to an output stream std::ostream. The exported model can also be imported via the ov::Core::import_model method.

Parameters:

model_streamOutput stream to store the model to.

void set_property(const AnyMap &properties)#

Sets properties for the current compiled model.

Parameters:

properties – Map of pairs: (property name, property value).

template<typename ...Properties>
inline util::EnableIfAllStringAny<void, Properties...> set_property(Properties&&... properties)#

Sets properties for the current compiled model.

Template Parameters:

Properties – Should be the pack of std::pair<std::string, ov::Any> types.

Parameters:

properties – Optional pack of pairs: (property name, property value).

Any get_property(const std::string &name) const#

Gets properties for current compiled model.

The method is responsible for extracting information that affects compiled model inference. The list of supported configuration values can be extracted via CompiledModel::get_property with the ov::supported_properties key, but some of these keys cannot be changed dynamically, for example, ov::device::id cannot be changed if a compiled model has already been compiled for a particular device.

Parameters:

nameProperty key, can be found in openvino/runtime/properties.hpp.

Returns:

Property value.

template<typename T, PropertyMutability mutability>
inline T get_property(const ov::Property<T, mutability> &property) const#

Gets properties related to device behaviour.

The method extracts information that can be set via the set_property method.

Template Parameters:

T – Type of a returned value.

Parameters:

propertyProperty object.

Returns:

Value of property.

void release_memory()#

Release intermediate memory.

This method forces the Compiled model to release memory allocated for intermediate structures, e.g. caches, tensors, temporal buffers etc., when possible

RemoteContext get_context() const#

Returns pointer to device-specific shared context on a remote accelerator device that was used to create this CompiledModel.

Returns:

A context.

bool operator!() const noexcept#

Checks if the current CompiledModel object is not initialized.

Returns:

true if the current CompiledModel object is not initialized; false, otherwise.

explicit operator bool() const noexcept#

Checks if the current CompiledModel object is initialized.

Returns:

true if the current CompiledModel object is initialized; false, otherwise.