Class ov::CompiledModel#
-
class CompiledModel#
This class represents a compiled model.
A model is compiled by a specific device by applying multiple optimization transformations, then mapping to compute kernels.
Public Functions
-
CompiledModel() = default#
Default constructor.
-
~CompiledModel()#
Destructor that preserves unloading order of an implementation object and reference to library.
-
std::shared_ptr<const Model> get_runtime_model() const#
Gets runtime model information from a device. This object represents an internal device-specific model that is optimized for a particular accelerator. It contains device-specific nodes, runtime information and can be used only to understand how the source model is optimized and which kernels, element types, and layouts are selected for optimal inference.
- Returns:
A model containing Executable Graph Info.
-
const std::vector<ov::Output<const ov::Node>> &inputs() const#
Gets all inputs of a compiled model. Inputs are represented as a vector of outputs of the ov::op::v0::Parameter operations. They contain information about input tensors such as tensor shape, names, and element type.
- Returns:
std::vector of model inputs.
-
const ov::Output<const ov::Node> &input() const#
Gets a single input of a compiled model. The input is represented as an output of the ov::op::v0::Parameter operation. The input contains information about input tensor such as tensor shape, names, and element type.
Note
If a model has more than one input, this method throws ov::Exception.
- Returns:
Compiled model input.
-
const ov::Output<const ov::Node> &input(size_t i) const#
Gets input of a compiled model identified by
i
. The input contains information about input tensor such as tensor shape, names, and element type.Note
The method throws ov::Exception if input with the specified index
i
is not found.- Parameters:
i – Index of input.
- Returns:
Compiled model input.
-
const ov::Output<const ov::Node> &input(const std::string &tensor_name) const#
Gets input of a compiled model identified by
tensor_name
. The input contains information about input tensor such as tensor shape, names, and element type.Note
The method throws ov::Exception if input with the specified tensor name
tensor_name
is not found.- Parameters:
tensor_name – The input tensor name.
- Returns:
Compiled model input.
-
const std::vector<ov::Output<const ov::Node>> &outputs() const#
Get all outputs of a compiled model. Outputs are represented as a vector of output from the ov::op::v0::Result operations. Outputs contain information about output tensors such as tensor shape, names, and element type.
- Returns:
std::vector of model outputs.
-
const ov::Output<const ov::Node> &output() const#
Gets a single output of a compiled model. The output is represented as an output from the ov::op::v0::Result operation. The output contains information about output tensor such as tensor shape, names, and element type.
Note
If a model has more than one output, this method throws ov::Exception.
- Returns:
Compiled model output.
-
const ov::Output<const ov::Node> &output(size_t i) const#
Gets output of a compiled model identified by
index
. The output contains information about output tensor such as tensor shape, names, and element type.Note
The method throws ov::Exception if output with the specified index
index
is not found.- Parameters:
i – Index of input.
- Returns:
Compiled model output.
-
const ov::Output<const ov::Node> &output(const std::string &tensor_name) const#
Gets output of a compiled model identified by
tensor_name
. The output contains information about output tensor such as tensor shape, names, and element type.Note
The method throws ov::Exception if output with the specified tensor name
tensor_name
is not found.- Parameters:
tensor_name – Output tensor name.
- Returns:
Compiled model output.
-
InferRequest create_infer_request()#
Creates an inference request object used to infer the compiled model. The created request has allocated input and output tensors (which can be changed later).
- Returns:
InferRequest object
-
void export_model(std::ostream &model_stream)#
Exports the current compiled model to an output stream
std::ostream
. The exported model can also be imported via the ov::Core::import_model method.See also
- Parameters:
model_stream – Output stream to store the model to.
-
void set_property(const AnyMap &properties)#
Sets properties for the current compiled model.
- Parameters:
properties – Map of pairs: (property name, property value).
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<void, Properties...> set_property(Properties&&... properties)# Sets properties for the current compiled model.
- Template Parameters:
Properties – Should be the pack of
std::pair<std::string, ov::Any>
types.- Parameters:
properties – Optional pack of pairs: (property name, property value).
-
Any get_property(const std::string &name) const#
Gets properties for current compiled model.
The method is responsible for extracting information that affects compiled model inference. The list of supported configuration values can be extracted via CompiledModel::get_property with the ov::supported_properties key, but some of these keys cannot be changed dynamically, for example, ov::device::id cannot be changed if a compiled model has already been compiled for a particular device.
-
template<typename T, PropertyMutability mutability>
inline T get_property(const ov::Property<T, mutability> &property) const# Gets properties related to device behaviour.
The method extracts information that can be set via the set_property method.
- Template Parameters:
T – Type of a returned value.
- Parameters:
property – Property object.
- Returns:
Value of property.
-
void release_memory()#
Release intermediate memory.
This method forces the Compiled model to release memory allocated for intermediate structures, e.g. caches, tensors, temporal buffers etc., when possible
-
RemoteContext get_context() const#
Returns pointer to device-specific shared context on a remote accelerator device that was used to create this CompiledModel.
- Returns:
A context.
-
bool operator!() const noexcept#
Checks if the current CompiledModel object is not initialized.
- Returns:
true
if the current CompiledModel object is not initialized;false
, otherwise.
-
explicit operator bool() const noexcept#
Checks if the current CompiledModel object is initialized.
- Returns:
true
if the current CompiledModel object is initialized;false
, otherwise.
-
CompiledModel() = default#