Group Basics

group ov_model_cpp_api

OpenVINO Core C++ API to work with ov::Model, dynamic and static shapes, types

Functions

template<typename ForwardIt>
size_t shape_size(ForwardIt start_dim, const ForwardIt end_dim)

Number of elements in a subset of dimensions of a shape. Returns a product of dimensions in a range [start_dim;end_dim)

template<typename SHAPE_TYPE>
size_t shape_size(const SHAPE_TYPE &shape)

Number of elements in spanned by a shape.

class Dimension
#include <dimension.hpp>

Class representing a dimension, which may be dynamic (undetermined until runtime), in a shape or shape-like object.

Static dimensions may be implicitly converted from value_type. A dynamic dimension is constructed with Dimension() or Dimension::dynamic().

Public Functions

Dimension(value_type dimension)

Construct a static dimension.

Parameters

dimension – Value of the dimension.

Dimension(value_type min_dimension, value_type max_dimension)

Construct a dynamic dimension with bounded range.

Parameters
  • min_dimension – The lower inclusive limit for the dimension

  • max_dimension – The upper inclusive limit for the dimension

Dimension(const std::string &str)

Construct a dimension from string.

Parameters

str – String to parse to dimension.

Dimension() = default

Construct a dynamic dimension with range [0, …].

inline bool is_static() const

Check whether this dimension is static.

Returns

true if the dimension is static, else false.

inline bool is_dynamic() const

Check whether this dimension is dynamic.

Returns

false if the dimension is static, else true.

value_type get_length() const

Convert this dimension to value_type. This dimension must be static and non-negative.

Throws

std::invalid_argument – If this dimension is dynamic or negative.

inline const Interval &get_interval() const

Return the interval of valid lengths.

bool same_scheme(const Dimension &dim) const

Check whether this dimension represents the same scheme as the argument (both dynamic, or equal).

Parameters

dim – The other dimension to compare this dimension to.

Returns

true if this dimension and dim are both dynamic, or if they are both static and equal; otherwise, false.

bool compatible(const Dimension &d) const

Check whether this dimension is capable of being merged with the argument dimension.

Two dimensions are considered compatible if it is possible to merge them. (See Dimension::merge.)

Parameters

d – The dimension to compare this dimension with.

Returns

true if this dimension is compatible with d, else false.

bool relaxes(const Dimension &d) const

Check whether this dimension is a relaxation of the argument.

A dimension d1 relaxes (or is a relaxation of) d2 if d1 and d2 are static and equal, or d1 is dynamic.

d1.relaxes(d2) is equivalent to d2.refines(d1).

Parameters

d – The dimension to compare this dimension with.

Returns

true if this dimension relaxes d, else false.

bool refines(const Dimension &d) const

Check whether this dimension is a refinement of the argument.

A dimension d2 refines (or is a refinement of) d1 if d1 and d2 are static and equal, or d2 is dynamic.

d1.refines(d2) is equivalent to d2.relaxes(d1).

Parameters

d – The dimension to compare this dimension with.

Returns

true if this dimension relaxes d, else false.

Dimension operator+(const Dimension &dim) const

Addition operator for Dimension.

Parameters

dim – Right operand for addition.

Returns

Smallest interval dimension enclosing inputs

Dimension operator-(const Dimension &dim) const

Subtraction operator for Dimension.

Parameters

dim – Right operand for subtraction.

Returns

Smallest interval dimension enclosing inputs

Dimension operator/(const value_type divisor) const

Division operator for Dimension divided by a value_type parameter.

Parameters

divisor – Right operand for division.

Returns

Smallest interval dimension enclosing inputs

inline Dimension &operator/=(const value_type divisor)

Divided-into operator for Dimension.

Parameters

divisor – Right operand for multiplication.

Returns

A reference to *this, after updating *this to the value *this * dim.

Dimension operator*(const Dimension &dim) const

Multiplication operator for Dimension.

Parameters

dim – Right operand for multiplicaiton.

Returns

Smallest interval containing all “produces” which are 0 if either of this or dim has length 0, else unbounded if either is unbounded, else product of lengths.

inline Dimension &operator+=(const Dimension &dim)

Add-into operator for Dimension.

Parameters

dim – Right operand for addition.

Returns

A reference to *this, after updating *this to the value *this + dim.

inline Dimension &operator*=(const Dimension &dim)

Multiply-into operator for Dimension.

Parameters

dim – Right operand for multiplication.

Returns

A reference to *this, after updating *this to the value *this * dim.

Dimension operator&(const Dimension &dim) const

Intersection of dimensions.

Dimension &operator&=(const Dimension &dim)

Intersection of dimensions.

std::string to_string() const

String representation of Dimension.

Public Static Functions

static bool merge(Dimension &dst, const Dimension &d1, const Dimension &d2)

Try to merge two Dimension objects together.

  • If d1 is dynamic, writes d2 to dst and returns true.

  • If d2 is dynamic, writes d1 to dst and returns true.

  • If d1 and d2 are static and equal, writes d1 to dst and returns true.

  • If d1 and d2 are both static and unequal, leaves dst unchanged and returns false.

Parameters
  • dst[out] Reference to write the merged Dimension into.

  • d1 – First dimension to merge.

  • d2 – Second dimension to merge.

Returns

true if merging succeeds, else false.

static bool broadcast_merge(Dimension &dst, const Dimension &d1, const Dimension &d2)

Try to merge two Dimension objects together with implicit broadcasting of unit-sized dimension to non unit-sized dimension.

static inline Dimension dynamic()

Create a dynamic dimension.

Returns

A dynamic dimension.

Friends

inline friend void swap(Dimension &a, Dimension &b)

Swap of dimensions.

class Extension
#include <extension.hpp>

The class provides the base interface for OpenVINO extensions.

Subclassed by ov::BaseOpExtension, ov::frontend::ConversionExtensionBase, ov::frontend::DecoderTransformationExtension, ov::frontend::ProgressReporterExtension, ov::frontend::TelemetryExtension

class Model : public std::enable_shared_from_this<Model>
#include <model.hpp>

A user-defined model.

Public Functions

explicit Model(const ov::OutputVector &results, const std::string &name = "")

Constructs a Model. Lists of parameters and variables will be generated automatically based on traversing the graph from the results.

Model(const ov::OutputVector &results, const ov::SinkVector &sinks, const std::string &name = "")

Constructs a Model. Lists of parameters and variables will be generated automatically based on traversing the graph from the results and the sinks.

size_t get_output_size() const

Return the number of outputs for this Model.

std::shared_ptr<ov::Node> get_output_op(size_t i) const

Return the op that generates output i.

std::shared_ptr<ov::Model> clone() const

Clones the original model.

std::vector<ov::Output<ov::Node>> outputs()

Model outputs.

std::vector<ov::Output<ov::Node>> inputs()

Model inputs.

const ov::element::Type &get_output_element_type(size_t i) const

Return the element type of output i.

const Shape &get_output_shape(size_t i) const

Return the shape of element i.

const PartialShape &get_output_partial_shape(size_t i) const

Return the partial shape of element i.

std::shared_ptr<ov::Node> get_result() const

Check that there is a single result and return it.

const std::string &get_name() const

Get the unique name of the model.

Returns

A const reference to the model’s unique name.

void set_friendly_name(const std::string &name)

Sets a friendly name for a model. This does not overwrite the unique name of the model and is retrieved via get_friendly_name(). Used mainly for debugging.

Parameters

name – is the friendly name to set

const std::string &get_friendly_name() const

Gets the friendly name for a model. If no friendly name has been set via set_friendly_name then the model’s unique name is returned.

Returns

A const reference to the model’s friendly name.

size_t get_graph_size() const

Returns the sum of the size of all nodes in the graph plus the size of all constant data. This has little value beyond comparing the relative size of graphs and should not be considered the actual memory consumption of a graph.

bool is_dynamic() const

Returns true if any of the op’s defined in the model contains partial shape.

void replace_parameter(size_t parameter_index, const std::shared_ptr<ov::op::v0::Parameter> &parameter)

Replace the parameter_indexth parameter of the model with parameter.

All users of the parameter_indexth parameter are redirected to parameter, and the parameter_indexth entry in the model parameter list is replaced with parameter.

Parameters
  • parameter_index – The index of the parameter to replace.

  • parameter – The parameter to substitute for the parameter_indexth parameter.

inline const ov::ParameterVector &get_parameters() const

Return the model parameters.

inline const ov::ResultVector &get_results() const

Return a list of model’s outputs.

int64_t get_parameter_index(const std::shared_ptr<ov::op::v0::Parameter> &parameter) const

Index for parameter, or -1.

int64_t get_result_index(const ov::Output<ov::Node> &value) const

Return the index of this model’s Result represented by the “value” Output object. This method returns -1 if an the passed output is not related to the Results of a model.

Parameters

valueOutput containing Node

int64_t get_result_index(const ov::Output<const ov::Node> &value) const

Return the index of this model’s Result represented by the “value” Output object. This method returns -1 if an the passed output is not related to the Results of a model.

Parameters

valueOutput containing Node

bool evaluate(ov::TensorVector &output_tensors, const ov::TensorVector &input_tensors, ov::EvaluationContext &evaluation_context) const

Evaluate the model on inputs, putting results in outputs.

Parameters
  • output_tensors – Tensors for the outputs to compute. One for each result

  • input_tensors – Tensors for the inputs. One for each inputs.

  • evaluation_context – Storage of additional settings and attributes that can be used when evaluating the model. This additional information can be shared across nodes.

bool evaluate(ov::TensorVector &output_tensors, const ov::TensorVector &input_tensors) const

Evaluate the model on inputs, putting results in outputs.

Parameters
  • output_tensors – Tensors for the outputs to compute. One for each result

  • input_tensors – Tensors for the inputs. One for each inputs.

inline const ov::SinkVector &get_sinks() const

Return a list of model’s sinks.

void add_sinks(const ov::SinkVector &sinks)

Add new sink nodes to the list. Method doesn’t validate graph, it should be done manually after all changes.

Parameters

sinks – new sink nodes

void remove_sink(const std::shared_ptr<ov::op::Sink> &sink)

Delete sink node from the list of sinks. Method doesn’t delete node from graph.

Parameters

sink – Sink to delete

void add_results(const ov::ResultVector &results)

Add new Result nodes to the list. Method doesn’t validate graph, it should be done manually after all changes.

Parameters

results – new Result nodes

void remove_result(const std::shared_ptr<ov::op::v0::Result> &result)

Delete Result node from the list of results. Method will not delete node from graph.

Parameters

result – Result node to delete

void add_parameters(const ov::ParameterVector &params)

Add new Parameter nodes to the list.

Method doesn’t change or validate graph, it should be done manually. For example, if you want to replace ReadValue node by Parameter, you should do the following steps:

  • replace node ReadValue by Parameter in graph

  • call add_parameter() to add new input to the list

  • call graph validation to check correctness of changes

Parameters

params – new Parameter nodes

void remove_parameter(const std::shared_ptr<ov::op::v0::Parameter> &param)

Delete Parameter node from the list of parameters. Method will not delete node from graph. You need to replace Parameter with other operation manually. Attention: Indexing of parameters can be changed.

Possible use of method is to replace input by variable. For it the following steps should be done:

  • Parameter node should be replaced by ReadValue

  • call remove_parameter(param) to remove input from the list

  • check if any parameter indexes are saved/used somewhere, update it for all inputs because indexes can be changed

  • call graph validation to check all changes

Parameters

param – Parameter node to delete

void add_variables(const ov::op::util::VariableVector &variables)

Add new variables to the list. Method doesn’t validate graph, it should be done manually after all changes.

Parameters

variables – new variables to add

void remove_variable(const ov::op::util::Variable::Ptr &variable)

Delete variable from the list of variables. Method doesn’t delete nodes that used this variable from the graph.

Parameters

variable – Variable to delete

inline const ov::op::util::VariableVector &get_variables() const

Return a list of model’s variables.

ov::op::util::Variable::Ptr get_variable_by_id(const std::string &variable_id) const

Return a variable by specified variable_id.

inline RTMap &get_rt_info()

Returns a runtime info.

Returns

reference to ov::AnyMap with runtime info

inline const RTMap &get_rt_info() const

Returns a constant runtime info.

Returns

reference to const ov::AnyMap with runtime info

template<class T, class ...Args, typename std::enable_if<!std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(Args... args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters
  • T – the type of returned value

  • Args – types of variadic arguments

Parameters

args – path to the runtime attribute

Returns

constant reference to value from runtime info

template<class T, class ...Args, typename std::enable_if<std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(Args... args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters
  • T – the type of returned value

  • Args – types of variadic arguments

Parameters

args – path to the runtime attribute

Returns

constant reference to value from runtime info

template<class T, typename std::enable_if<!std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(const std::vector<std::string> &args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters

T – the type of returned value

Parameters

args – vector with path to the runtime attribute

Returns

constant reference to value from runtime info

template<class T, typename std::enable_if<std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(const std::vector<std::string> &args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters

T – the type of returned value

Parameters

args – vector with path to the runtime attribute

Returns

constant reference to value from runtime info

template<class ...Args>
inline bool has_rt_info(Args... args) const

Checks if given path exists in runtime info.

Template Parameters

Args – types of variadic arguments

Parameters

args – path to the runtime attribute

Returns

true if path exists, otherwise false

bool has_rt_info(const std::vector<std::string> &args) const

Checks if given path exists in runtime info.

Parameters

args – vector with path to the runtime attribute

Returns

true if path exists, otherwise false

template<class T, class ...Args>
inline void set_rt_info(const T &argument, Args... args)

Add value inside the runtime info.

Template Parameters
  • T – type of new value

  • Args – types of variadic arguments

Parameters
  • argument – value for the runtime info

  • args – path to the runtime attribute

template<class T>
inline void set_rt_info(const T &argument, const std::vector<std::string> &args)

Add value inside the runtime info.

Template Parameters

T – type of new value

Parameters
  • argument – value for the runtime info

  • args – vector with path to the runtime attribute

class Node : public std::enable_shared_from_this<Node>
#include <node.hpp>

Nodes are the backbone of the graph of Value dataflow. Every node has zero or more nodes as arguments and one value, which is either a tensor or a (possibly empty) tuple of values.

Subclassed by ov::op::Op, ov::pass::pattern::op::Pattern

Public Functions

virtual void validate_and_infer_types()

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual const ov::op::AutoBroadcastSpec &get_autob() const
Returns

the autobroadcasr spec

virtual bool has_evaluate() const

Allows to get information about availability of evaluate method for the current operation.

virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values) const

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values, const ov::EvaluationContext &evaluationContext) const

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

  • evaluation_context – Storage of additional settings and attributes that can be used when evaluating the op.

Returns

true if successful

inline virtual OutputVector decompose_op() const

Decomposes the FusedOp into a sub-graph consisting of core openvino ops.

Returns

A vector of nodes comprising the sub-graph. The order of output tensors must match the match output tensors of the FusedOp

virtual const type_info_t &get_type_info() const = 0

Returns the NodeTypeInfo for the node’s class. During transition to type_info, returns a dummy type_info for Node if the class has not been updated yet.

void set_arguments(const NodeVector &arguments)

Sets/replaces the arguments with new arguments.

void set_arguments(const OutputVector &arguments)

Sets/replaces the arguments with new arguments.

void set_argument(size_t position, const Output<Node> &argument)

Sets/replaces the arguments with new arguments.

void set_output_size(size_t output_size)

Sets the number of outputs.

virtual std::string description() const

Get the string name for the type of the node, such as Add or Multiply. The class name, must not contain spaces as it is used for codegen.

Returns

A const reference to the node’s type name

const std::string &get_name() const

Get the unique name of the node.

Returns

A const reference to the node’s unique name.

void set_friendly_name(const std::string &name)

Sets a friendly name for a node. This does not overwrite the unique name of the node and is retrieved via get_friendly_name(). Used mainly for debugging. The friendly name may be set exactly once.

Parameters

name – is the friendly name to set

const std::string &get_friendly_name() const

Gets the friendly name for a node. If no friendly name has been set via set_friendly_name then the node’s unique name is returned.

Returns

A const reference to the node’s friendly name.

virtual std::ostream &write_description(std::ostream &os, uint32_t depth = 0) const

Writes a description of a node to a stream.

Parameters
  • os – The stream; should be returned

  • depth – How many levels of inputs to describe

Returns

The stream os

const std::vector<std::shared_ptr<Node>> &get_control_dependencies() const

Get control dependencies registered on the node.

const std::vector<Node*> &get_control_dependents() const

Get nodes dependent on this node.

void add_control_dependency(std::shared_ptr<Node> node)

This node cannot execute until node executes.

void remove_control_dependency(std::shared_ptr<Node> node)

Remove the dependency of this node on node.

void clear_control_dependencies()

Remove all dependencies from this node.

void clear_control_dependents()

Remove this node as a dependency from all dependent nodes.

void add_node_control_dependencies(const std::shared_ptr<const Node> &source_node)

This node absorbs the control dependencies of source_node.

void add_node_control_dependents(const std::shared_ptr<const Node> &source_node)

This node becomes a dependent of every node dependent on source_node.

void transfer_control_dependents(std::shared_ptr<Node> replacement)

This node’s control dependencies are replaced by replacement.

size_t get_output_size() const

Returns the number of outputs from the node.

const element::Type &get_output_element_type(size_t i) const

Returns the element type for output i.

const element::Type &get_element_type() const

Checks that there is exactly one output and returns its element type.

const Shape &get_output_shape(size_t i) const

Returns the shape for output i.

const PartialShape &get_output_partial_shape(size_t i) const

Returns the partial shape for output i.

Output<const Node> get_default_output() const

Return the output to use when converting to an Output<Node> with no index specified. Throws when not supported.

virtual size_t get_default_output_index() const

Returns the output of the default output, or throws if there is none.

size_t no_default_index() const

Throws no default.

const Shape &get_shape() const

Checks that there is exactly one output and returns its shape.

descriptor::Tensor &get_output_tensor(size_t i) const

Returns the tensor for output or input i.

size_t get_input_size() const

Returns the number of inputs for the op.

const element::Type &get_input_element_type(size_t i) const

Returns the element type of input i.

const Shape &get_input_shape(size_t i) const

Returns the shape of input i.

const PartialShape &get_input_partial_shape(size_t i) const

Returns the partial shape of input i.

bool has_same_type(std::shared_ptr<const Node> node) const

True if this and node have one output with same element type and shape.

NodeVector get_users(bool check_is_used = false) const

Get all the nodes that uses the current node.

inline bool operator<(const Node &other) const

Use instance ids for comparison instead of memory addresses to improve determinism.

std::vector<Input<Node>> inputs()
Returns

A vector containing a handle for each of this node’s inputs, in order.

std::vector<Input<const Node>> inputs() const
Returns

A vector containing a handle for each of this node’s inputs, in order.

std::vector<Output<Node>> input_values() const
Returns

A vector containing the values for each input

std::vector<Output<Node>> outputs()
Returns

A vector containing a handle for each of this node’s outputs, in order.

std::vector<Output<const Node>> outputs() const
Returns

A vector containing a handle for each of this node’s outputs, in order.

Input<Node> input(size_t input_index)
Throws

std::out_of_range – if the node does not have at least input_index+1 inputs.

Returns

A handle to the input_indexth input of this node.

Input<const Node> input(size_t input_index) const
Throws

std::out_of_range – if the node does not have at least input_index+1 inputs.

Returns

A handle to the input_indexth input of this node.

Output<Node> output(size_t output_index)
Throws

std::out_of_range – if the node does not have at least output_index+1 outputs.

Returns

A handle to the output_indexth output of this node.

Output<const Node> output(size_t output_index) const
Throws

std::out_of_range – if the node does not have at least output_index+1 outputs.

Returns

A handle to the output_indexth output of this node.

template<>
class Input<Node>
#include <node_input.hpp>

A handle for one of a node’s inputs.

Public Functions

Input(Node *node, size_t index)

Constructs a Input.

Parameters
  • node – Pointer to the node for the input handle.

  • index – The index of the input.

Node *get_node() const
Returns

A pointer to the node referenced by this input handle.

size_t get_index() const
Returns

The index of the input referred to by this input handle.

const element::Type &get_element_type() const
Returns

The element type of the input referred to by this input handle.

const Shape &get_shape() const
Returns

The shape of the input referred to by this input handle.

const PartialShape &get_partial_shape() const
Returns

The partial shape of the input referred to by this input handle.

Output<Node> get_source_output() const
Returns

A handle to the output that is connected to this input.

descriptor::Tensor &get_tensor() const
Returns

A reference to the tensor descriptor for this input.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
Returns

A shared pointer to the tensor descriptor for this input.

bool get_is_relevant_to_shapes() const
Returns

true if this input is relevant to its node’s output shapes; else false.

bool get_is_relevant_to_values() const
Returns

true if this input is relevant to its node’s output values; else false.

void replace_source_output(const Output<Node> &new_source_output) const

Replaces the source output of this input.

Parameters

new_source_output – A handle for the output that will replace this input’s source.

RTMap &get_rt_info()
Returns

The reference to runtime info map

const RTMap &get_rt_info() const
Returns

The constant reference to runtime info map

template<>
class Input<const Node>
#include <node_input.hpp>

A handle for one of a node’s inputs.

Public Functions

Input(const Node *node, size_t index)

Constructs a Input.

Parameters
  • node – Pointer to the node for the input handle.

  • index – The index of the input.

const Node *get_node() const
Returns

A pointer to the node referenced by this input handle.

size_t get_index() const
Returns

The index of the input referred to by this input handle.

const element::Type &get_element_type() const
Returns

The element type of the input referred to by this input handle.

const Shape &get_shape() const
Returns

The shape of the input referred to by this input handle.

const PartialShape &get_partial_shape() const
Returns

The partial shape of the input referred to by this input handle.

Output<Node> get_source_output() const
Returns

A handle to the output that is connected to this input.

descriptor::Tensor &get_tensor() const
Returns

A reference to the tensor descriptor for this input.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
Returns

A shared pointer to the tensor descriptor for this input.

bool get_is_relevant_to_shapes() const
Returns

true if this input is relevant to its node’s output shapes; else false.

bool get_is_relevant_to_values() const
Returns

true if this input is relevant to its node’s output values; else false.

const RTMap &get_rt_info() const
Returns

The constant reference to runtime info map

template<>
class Output<Node>
#include <node_output.hpp>

A handle for one of a node’s outputs.

Public Functions

Output(Node *node, size_t index)

Constructs a Output.

Parameters
  • node – A pointer to the node for the output handle.

  • index – The index of the output.

Output(const std::shared_ptr<Node> &node, size_t index)

Constructs a Output.

Parameters
  • node – A shared_ptr to the node for the output handle.

  • index – The index of the output.

template<typename T>
inline Output(const std::shared_ptr<T> &node)

Constructs a Output, referencing the zeroth output of the node.

Parameters

node – A shared_ptr to the node for the output handle.

Output() = default

A null output.

Node *get_node() const
Returns

A pointer to the node referred to by this output handle.

std::shared_ptr<Node> get_node_shared_ptr() const
Returns

A shared_ptr to the node referred to by this output handle.

size_t get_index() const
Returns

The index of the output referred to by this output handle.

descriptor::Tensor &get_tensor() const
Returns

A reference to the tensor descriptor for this output.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
Returns

A shared point to the tensor ptr for this output.

void set_tensor_ptr(std::shared_ptr<descriptor::Tensor> tensor_ptr)
Returns

Set new tensor desc shared pointer to this output

const element::Type &get_element_type() const
Returns

The element type of the output referred to by this output handle.

const Shape &get_shape() const
Returns

The shape of the output referred to by this output handle.

const PartialShape &get_partial_shape() const
Returns

The partial shape of the output referred to by this output handle.

RTMap &get_rt_info()
Returns

The reference to runtime info map

const RTMap &get_rt_info() const
Returns

The constant reference to runtime info map

const std::unordered_set<std::string> &get_names() const
Returns

The tensor names associated with this output

std::string get_any_name() const
Returns

Any tensor names associated with this output

void set_names(const std::unordered_set<std::string> &names)
Returns

Set tensor names associated with this output

void add_names(const std::unordered_set<std::string> &names)
Returns

Add tensor names associated with this output

std::set<Input<Node>> get_target_inputs() const
Returns

A set containing handles for all inputs targeted by the output referenced by this output handle.

void remove_target_input(const Input<Node> &target_input) const

Removes a target input from the output referenced by this output handle.

Parameters

target_input – The target input to remove.

void replace(const Output<Node> &replacement)

Replace all users of this value with replacement.

template<>
class Output<const Node>
#include <node_output.hpp>

A handle for one of a node’s outputs.

Public Functions

Output(const Node *node, size_t index)

Constructs a Output.

Parameters
  • node – A pointer to the node for the output handle.

  • index – The index of the output.

Output(const std::shared_ptr<const Node> &node, size_t index)

Constructs a Output.

Parameters
  • node – A shared_ptr to the node for the output handle.

  • index – The index of the output.

template<typename T>
inline Output(const std::shared_ptr<const T> &node)

Constructs a Output, referencing the zeroth output of the node.

Parameters

node – A shared_ptr to the node for the output handle.

Output() = default

A null output.

const Node *get_node() const
Returns

A pointer to the node referred to by this output handle.

std::shared_ptr<const Node> get_node_shared_ptr() const
Returns

A shared_ptr to the node referred to by this output handle.

size_t get_index() const
Returns

The index of the output referred to by this output handle.

descriptor::Tensor &get_tensor() const
Returns

A reference to the tensor descriptor for this output.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
Returns

A shared point to the tensor ptr for this output.

const element::Type &get_element_type() const
Returns

The element type of the output referred to by this output handle.

const Shape &get_shape() const
Returns

The shape of the output referred to by this output handle.

const PartialShape &get_partial_shape() const
Returns

The partial shape of the output referred to by this output handle.

const RTMap &get_rt_info() const
Returns

The constant reference to runtime info map

const std::unordered_set<std::string> &get_names() const
Returns

The tensor names associated with this output

std::string get_any_name() const
Returns

Any tensor name associated with this output

std::set<Input<Node>> get_target_inputs() const
Returns

A set containing handles for all inputs targeted by the output referenced by this output handle.

class PartialShape
#include <partial_shape.hpp>

Class representing a shape that may be partially or totally dynamic.

A PartialShape may have:

  • Dynamic rank. (Informal notation: ?)

  • Static rank, but dynamic dimensions on some or all axes. (Informal notation examples: {1,2,?,4}, {?,?,?})

  • Static rank, and static dimensions on all axes. (Informal notation examples: {1,2,3,4}, {6}, {})

Public Functions

PartialShape(std::initializer_list<Dimension> init)

Constructs a shape with static rank from an initializer list of Dimension.

Examples:

PartialShape s{2,3,4};                     // rank=3, all dimensions static
PartialShape s{};                          // rank=0
PartialShape s{2,Dimension::dynamic(),3};  // rank=3, dimension 1 dynamic
Parameters

init – The Dimension values for the constructed shape.

PartialShape(std::vector<Dimension> dimensions)

Constructs a PartialShape with static rank from a vector of Dimension.

Parameters

dimensions – The Dimension values for the constructed shape.

PartialShape(const std::vector<Dimension::value_type> &dimensions)

Constructs a PartialShape with static rank from a vector of dimensions values.

Parameters

dimensions – The Dimension values for the constructed shape.

PartialShape()

Constructs a static PartialShape with zero rank (the shape of a scalar).

PartialShape(const Shape &shape)

Constructs a static PartialShape from a PartialShape.

Parameters

shape – The PartialShape to convert into PartialShape.

PartialShape(const std::string &shape)

Constructs a static PartialShape from a string.

Parameters

shape – The string to parse into PartialShape.

bool is_static() const

Check if this shape is static.

A shape is considered static if it has static rank, and all dimensions of the shape are static.

Returns

true if this shape is static, else false.

inline bool is_dynamic() const

Check if this shape is dynamic.

A shape is considered static if it has static rank, and all dimensions of the shape are static.

Returns

false if this shape is static, else true.

inline Rank rank() const

Get the rank of the shape.

Returns

The rank of the shape. This will be Rank::dynamic() if the rank of the shape is dynamic.

bool compatible(const PartialShape &s) const

Check whether this shape is compatible with the argument, i.e., whether it is possible to merge them.

Two shapes are compatible if

  • one or both of them has dynamic rank, or

  • both shapes have dynamic and equal rank, and their dimensions are elementwise compatible (see Dimension::compatible()).

Parameters

s – The shape to be checked for compatibility with this shape.

Returns

true if this shape is compatible with s, else false.

bool same_scheme(const PartialShape &s) const

Check whether this shape represents the same scheme as the argument.

Two shapes s1 and s2 represent the same scheme if

  • they both have dynamic rank, or

  • they both have static and equal rank r, and for every i from 0 to r-1, s1[i] represents the same scheme as s2[i] (see Dimension::same_scheme()).

Parameters

s – The shape whose scheme is being compared with this shape.

Returns

true if this shape represents the same scheme as s, else false.

bool relaxes(const PartialShape &s) const

Check whether this shape is a relaxation of the argument.

Intuitively, a PartialShape s1 is said to relax s2 (or is a relaxation of s2) if it is “more permissive” than s2. In other words, s1 is a relaxation of s2 if anything you can form by plugging things into the dynamic dimensions of s2 is also something you can form by plugging things into the dynamic dimensions of s1, but not necessarily the other way around.

s1.relaxes(s2) is equivalent to s2.refines(s1).

Formally, PartialShape s1 is said to relax PartialShape s2 if:

  • For every i from 0 to r-1, either s1[i] contains s2[i].

Parameters

s – The shape which is being compared against this shape.

Returns

true if this shape relaxes s, else false.

bool refines(const PartialShape &s) const

Check whether this shape is a refinement of the argument.

Intuitively, a PartialShape s1 is said to relax s2 (or is a relaxation of s2) if it is “less permissive” than s2. In other words, s1 is a relaxation of s2 if anything you can form by plugging things into the dynamic dimensions of s1 is also something you can form by plugging things into the dynamic dimensions of s2, but not necessarily the other way around.

s1.refines(s2) is equivalent to s2.relaxes(s1).

Formally, PartialShape s1 is said to refine PartialShape s2 if:

  • s2 has dynamic rank, or

  • s1 and s2 both have static rank r, and for every i from 0 to r-1, either s2[i] is dynamic, or s1[i] == s2[i].

Parameters

s – The shape which is being compared against this shape.

Returns

true if this shape refines s, else false.

bool merge_rank(const Rank &r)

Checks that this shape’s rank is compatible with r, and, if this shape’s rank is dynamic and r is static, updates this shape to have a rank of r with dimensions all dynamic.

Returns

true if this shape’s rank is compatible with r, else false.

Shape to_shape() const

Convert a static PartialShape to a PartialShape.

Throws

std::invalid_argument – If this PartialShape is dynamic.

Returns

A new PartialShape s where s[i] = size_t((*this)[i]).

bool all_non_negative() const

Returns true if all static dimensions of the tensor are non-negative, else false.

Dimension &operator[](std::ptrdiff_t i)

Index operator for PartialShape, with bound checking.

Parameters

i – The index of the dimension being selected in range [-rank, rank).

Returns

A reference to the ith Dimension of this shape.

const Dimension &operator[](std::ptrdiff_t i) const

Index operator for PartialShape, with bound checking.

Parameters

i – The index of the dimension being selected in range [-rank, rank).

Returns

A reference to the ith Dimension of this shape.

inline explicit operator std::vector<Dimension>() const

Returns a vector of the dimensions. This has no meaning if dynamic.

Shape get_max_shape() const

Get the max bounding shape.

Shape get_min_shape() const

Get the min bounding shape.

Shape get_shape() const

Get the unique shape.

inline iterator begin() noexcept

Returns a read/write iterator that points to the first element in the shape. Iteration is done in ordinary element order.

inline const_iterator begin() const noexcept

Returns a read-only (constant) iterator that points to the first element in the shape. Iteration is done in ordinary element order.

inline iterator end() noexcept

Returns a read/write iterator that points one past the last element in the shape. Iteration is done in ordinary element order.

inline const_iterator end() const noexcept

Returns a read-only (constant) iterator that points one past the last element in the shape. Iteration is done in ordinary element order.

inline reverse_iterator rbegin() noexcept

Returns a read/write reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.

inline const_reverse_iterator rbegin() const noexcept

Returns a read-only (constant) reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.

inline reverse_iterator rend() noexcept

Returns a read/write reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.

inline const_reverse_iterator rend() const noexcept

Returns a read-only (constant) reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.

inline const_iterator cbegin() const noexcept

Returns a read-only (constant) iterator that points to the first element in the shape. Iteration is done in ordinary element order.

inline const_iterator cend() const noexcept

Returns a read-only (constant) iterator that points one past the last element in the shape. Iteration is done in ordinary element order.

inline const_reverse_iterator crbegin() const noexcept

Returns a read-only (constant) reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.

inline const_reverse_iterator crend() const noexcept

Returns a read-only (constant) reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.

inline void resize(size_t count)

Resizes dimensions container to contain count elements.

inline size_t size() const

Returns size of dimension vector. Requires rank to be static.

inline iterator insert(iterator position, const Dimension &val)

Returns a read/write iterator that points to the inserted element in the shape.

inline void insert(iterator position, size_t n, const Dimension &val)

Inserts count copies of the value before position.

template<class InputIterator>
inline void insert(iterator position, InputIterator first, InputIterator last)

Inserts elements from range [first, last) before position.

inline void reserve(size_t n)

Requests that the dimensions vector capacity be enough to contain n elements.

inline void push_back(const Dimension &val)

push element to the end of partial shape

template<class ...Args>
inline void emplace_back(Args&&... args)

emplace element to the end of partial shape

std::string to_string() const

String representation of PartialShape.

Public Static Functions

static PartialShape dynamic(Rank r = Rank::dynamic())

Construct a PartialShape with the given rank and all dimensions (if any) dynamic.

Returns

A PartialShape with the given rank, and all dimensions (if any) dynamic.

static bool merge_into(PartialShape &dst, const PartialShape &src)

Try to merge one shape into another.

Merges src into dst, returning true on success and false on failure. If false is returned, the effect on dst is unspecified.

To merge two partial shapes s1 and s2 is to find the most permissive partial shape s that is no more permissive than s1 or s2, if s exists. For example:

merge(?,?) -> ?
merge(?,{?,?}) -> {?,?}
merge({?,?},{?,?}) -> {?,?}
merge({1,2,3,4},?) -> {1,2,3,4}
merge({1,2},{1,?}) -> {1,2}
merge({1,2,?,?},{1,?,3,?}) -> {1,2,3,?}
merge({1,2,3},{1,2,3}) -> {1,2,3}

merge({1,?},{2,?}) fails [dimension 0 constraints are inconsistent]
merge({?,?},{?,?,?}) fails [ranks are inconsistent]

This function (merge_into) performs the “merge” operation described above on dst and src, but overwrites dst with the result and returns true if merging is successful; if merging is unsuccessful, the function returns false and may make unspecified changes to dst.

Parameters
  • dst[inout] The shape that src will be merged into.

  • src – The shape that will be merged into dst.

Returns

true if merging succeeds, else false.

static bool broadcast_merge_into(PartialShape &dst, const PartialShape &src, const ov::op::AutoBroadcastSpec &autob)

Try to merge one shape into another along with implicit broadcasting.

Friends

friend OPENVINO_API std::ostream & operator<< (std::ostream &str, const PartialShape &shape)

Inserts a human-readable representation of a PartialShape into an output stream.

The output to the stream is in “informal” notation. In other words:

  • If shape has dynamic rank, inserts the string ?.

  • If shape has static rank, inserts the string {, then inserts each dimension of shape into the output stream separated by commas, then inserts }.

Example:

PartialShape s1{PartialShape::dynamic())};
PartialShape s2{};
PartialShape s3{1,Dimension::dynamic(),2,3};
PartialShape s4{2,3,4};
std::cout << s1 << std::endl
          << s2 << std::endl
          << s3 << std::endl
          << s4 << std::endl;

Output:

?
{}
{1,?,2,3}
{2,3,4}
Parameters
  • str – The output stream targeted for insertion.

  • shape – The shape to be inserted into str.

Returns

A reference to str after insertion.

friend OPENVINO_API PartialShape operator+ (const PartialShape &s1, const PartialShape &s2)

Elementwise addition of two PartialShape objects.

  • If s1 or s2 has dynamic rank, returns PartialShape::dynamic().

  • If s1 ands2` both have static rank, and their ranks are unequal, throws std::invalid_argument.

  • If s1 and s2 both have static rank, and their ranks are equal, returns a new shape whose ith dimension is s1[i] + s2[i].

Parameters
  • s1 – Left operand for addition.

  • s2 – Right operand for addition.

Throws

std::invalid_argument – If s1 and s2 have inconsistent ranks.

Returns

The result of elementwise adding s1 to s2 (see description).

class PrePostProcessor
#include <pre_post_process.hpp>

Main class for adding pre- and post- processing steps to existing ov::Model.

This is a helper class for writing easy pre- and post- processing operations on ov::Model object assuming that any preprocess operation takes one input and produces one output.

For advanced preprocessing scenarios, like combining several functions with multiple inputs/outputs into one, client’s code can use transformation passes over ov::Model

Public Functions

explicit PrePostProcessor(const std::shared_ptr<Model> &function)

Default constructor.

Parameters

function – Existing function representing loaded model

PrePostProcessor(PrePostProcessor&&) noexcept

Default move constructor.

PrePostProcessor &operator=(PrePostProcessor&&) noexcept

Default move assignment operator.

~PrePostProcessor()

Default destructor.

InputInfo &input()

Gets input pre-processing data structure. Should be used only if model/function has only one input Using returned structure application’s code is able to set user’s tensor data (e.g layout), preprocess steps, target model’s data.

Returns

Reference to model’s input information structure

InputInfo &input(const std::string &tensor_name)

Gets input pre-processing data structure for input identified by it’s tensor name.

Parameters

tensor_nameTensor name of specific input. Throws if tensor name is not associated with any input in a model

Returns

Reference to model’s input information structure

InputInfo &input(size_t input_index)

Gets input pre-processing data structure for input identified by it’s order in a model.

Parameters

input_indexInput index of specific input. Throws if input index is out of range for associated function

Returns

Reference to model’s input information structure

OutputInfo &output()

Gets output post-processing data structure. Should be used only if model/function has only one output Using returned structure application’s code is able to set model’s output data, post-process steps, user’s tensor data (e.g layout)

Returns

Reference to model’s output information structure

OutputInfo &output(const std::string &tensor_name)

Gets output post-processing data structure for output identified by it’s tensor name.

Parameters

tensor_nameTensor name of specific output. Throws if tensor name is not associated with any input in a model

Returns

Reference to model’s output information structure

OutputInfo &output(size_t output_index)

Gets output post-processing data structure for output identified by it’s order in a model.

Parameters

output_indexOutput index of specific output. Throws if output index is out of range for associated function

Returns

Reference to model’s output information structure

std::shared_ptr<Model> build()

Adds pre/post-processing operations to function passed in constructor.

Returns

Function with added pre/post-processing operations

class Shape :