Group Basics#

group ov_model_cpp_api

OpenVINO Core C++ API to work with ov::Model, dynamic and static shapes, types

Functions

template<typename ForwardIt> size_t shape_size(ForwardIt start_dim, const ForwardIt end_dim)#: Number of elements in a subset of dimensions of a shape. Returns a product of dimensions in a range [start_dim;end_dim)

template<typename SHAPE_TYPE> size_t shape_size(const SHAPE_TYPE &shape)#: Number of elements in spanned by a shape.

class Dimension

#include <dimension.hpp>

Class representing a dimension, which may be dynamic (undetermined until runtime), in a shape or shape-like object.

Static dimensions may be implicitly converted from value_type. A dynamic dimension is constructed with Dimension() or Dimension::dynamic().

Public Functions

Dimension(value_type dimension)

Construct a static dimension.

Parameters:: dimension – Value of the dimension.

Dimension(value_type min_dimension, value_type max_dimension)

Construct a dynamic dimension with bounded range.

Parameters:

min_dimension – The lower inclusive limit for the dimension
max_dimension – The upper inclusive limit for the dimension

Dimension(const std::string &str)

Construct a dimension from string.

Parameters:: str – String to parse to dimension.

Dimension() = default: Construct a dynamic dimension with range [0, …].

inline bool is_static() const

Check whether this dimension is static.

Returns:: true if the dimension is static, else false.

inline bool is_dynamic() const

Check whether this dimension is dynamic.

Returns:: false if the dimension is static, else true.

value_type get_length() const

Convert this dimension to value_type. This dimension must be static and non-negative.

Throws:: std::invalid_argument – If this dimension is dynamic or negative.

inline const Interval &get_interval() const: Return the interval of valid lengths.

bool same_scheme(const Dimension &dim) const

Check whether this dimension represents the same scheme as the argument (both dynamic, or equal).

Parameters:: dim – The other dimension to compare this dimension to.
Returns:: true if this dimension and dim are both dynamic, or if they are both static and equal; otherwise, false.

bool compatible(const Dimension &d) const

Check whether this dimension is capable of being merged with the argument dimension.

Two dimensions are considered compatible if it is possible to merge them. (See Dimension::merge.)

Parameters:: d – The dimension to compare this dimension with.
Returns:: true if this dimension is compatible with d, else false.

bool relaxes(const Dimension &d) const

Check whether this dimension is a relaxation of the argument.

A dimension d1 relaxes (or is a relaxation of) d2 if d1 and d2 are static and equal, or d1 is dynamic.

d1.relaxes(d2) is equivalent to d2.refines(d1).

Parameters:: d – The dimension to compare this dimension with.
Returns:: true if this dimension relaxes d, else false.

bool refines(const Dimension &d) const

Check whether this dimension is a refinement of the argument.

A dimension d2 refines (or is a refinement of) d1 if d1 and d2 are static and equal, or d2 is dynamic.

d1.refines(d2) is equivalent to d2.relaxes(d1).

Parameters:: d – The dimension to compare this dimension with.
Returns:: true if this dimension relaxes d, else false.

Dimension operator+(const Dimension &dim) const

Addition operator for Dimension.

Parameters:: dim – Right operand for addition.
Returns:: Smallest interval dimension enclosing inputs

Dimension operator-(const Dimension &dim) const

Subtraction operator for Dimension.

Parameters:: dim – Right operand for subtraction.
Returns:: Smallest interval dimension enclosing inputs

Dimension operator/(const value_type divisor) const

Division operator for Dimension divided by a value_type parameter.

Parameters:: divisor – Right operand for division.
Returns:: Smallest interval dimension enclosing inputs

inline Dimension &operator/=(const value_type divisor)

Divided-into operator for Dimension.

Parameters:: divisor – Right operand for multiplication.
Returns:: A reference to *this, after updating *this to the value *this * dim.

Dimension operator*(const Dimension &dim) const

Multiplication operator for Dimension.

Parameters:: dim – Right operand for multiplicaiton.
Returns:: Smallest interval containing all “produces” which are 0 if either of this or dim has length 0, else unbounded if either is unbounded, else product of lengths.

inline Dimension &operator+=(const Dimension &dim)

Add-into operator for Dimension.

Parameters:: dim – Right operand for addition.
Returns:: A reference to *this, after updating *this to the value *this + dim.

inline Dimension &operator*=(const Dimension &dim)

Multiply-into operator for Dimension.

Parameters:: dim – Right operand for multiplication.
Returns:: A reference to *this, after updating *this to the value *this * dim.

Dimension operator&(const Dimension &dim) const: Intersection of dimensions.

Dimension &operator&=(const Dimension &dim): Intersection of dimensions.

std::string to_string() const: String representation of Dimension.

bool has_symbol() const: Indicates if meaningful symbol was set to the Dimension.

std::shared_ptr<ov::Symbol> get_symbol() const: Returns symbol of the Dimension.

void set_symbol(const std::shared_ptr<ov::Symbol> &s): Sets symbol of the Dimension.

Public Static Functions

static bool merge(Dimension &dst, const Dimension &d1, const Dimension &d2)

Try to merge two Dimension objects together.

If d1 is dynamic, writes d2 to dst and returns true.
If d2 is dynamic, writes d1 to dst and returns true.
If d1 and d2 are static and equal, writes d1 to dst and returns true.
If d1 and d2 are both static and unequal, leaves dst unchanged and returns false.

Parameters:

dst – [out] Reference to write the merged Dimension into.
d1 – First dimension to merge.
d2 – Second dimension to merge.

Returns:

true if merging succeeds, else false.

static bool broadcast_merge(Dimension &dst, const Dimension &d1, const Dimension &d2): Try to merge two Dimension objects together with implicit broadcasting of unit-sized dimension to non unit-sized dimension.

static inline Dimension dynamic()

Create a dynamic dimension.

Returns:: A dynamic dimension.

Friends

inline friend void swap(Dimension &a, Dimension &b): Swap of dimensions.

class Extension

#include <extension.hpp>

The class provides the base interface for OpenVINO extensions.

Subclassed by ov::BaseOpExtension, ov::frontend::ConversionExtensionBase, ov::frontend::DecoderTransformationExtension, ov::frontend::ProgressReporterExtension, ov::frontend::TelemetryExtension

class Model : public std::enable_shared_from_this<Model>

#include <model.hpp>

A user-defined model.

Public Functions

explicit Model(const ov::OutputVector &results, const std::string &name = ""): Constructs a Model. Lists of parameters and variables will be generated automatically based on traversing the graph from the results.

Model(const ov::OutputVector &results, const ov::SinkVector &sinks, const std::string &name = ""): Constructs a Model. Lists of parameters and variables will be generated automatically based on traversing the graph from the results and the sinks.

size_t get_output_size() const: Return the number of outputs for this Model.

std::shared_ptr<ov::Node> get_output_op(size_t i) const: Return the op that generates output i.

std::shared_ptr<ov::Model> clone() const: Clones the original model.

std::vector<ov::Output<ov::Node>> outputs(): Model outputs.

std::vector<ov::Output<ov::Node>> inputs(): Model inputs.

const ov::element::Type &get_output_element_type(size_t i) const: Return the element type of output i.

const Shape &get_output_shape(size_t i) const: Return the shape of element i.

const PartialShape &get_output_partial_shape(size_t i) const: Return the partial shape of element i.

std::shared_ptr<ov::Node> get_result() const: Check that there is a single result and return it.

const std::string &get_name() const

Get the unique name of the model.

Returns:: A const reference to the model’s unique name.

void set_friendly_name(const std::string &name)

Sets a friendly name for a model. This does not overwrite the unique name of the model and is retrieved via get_friendly_name(). Used mainly for debugging.

Parameters:: name – is the friendly name to set

const std::string &get_friendly_name() const

Gets the friendly name for a model. If no friendly name has been set via set_friendly_name then the model’s unique name is returned.

Returns:: A const reference to the model’s friendly name.

size_t get_graph_size() const: Returns the sum of the size of all nodes in the graph plus the size of all constant data. This has little value beyond comparing the relative size of graphs and should not be considered the actual memory consumption of a graph.

bool is_dynamic() const: Returns true if any of the op’s defined in the model contains partial shape.

void replace_parameter(size_t parameter_index, const std::shared_ptr<ov::op::v0::Parameter> &parameter)

Replace the parameter_indexth parameter of the model with parameter.

All users of the parameter_indexth parameter are redirected to parameter, and the parameter_indexth entry in the model parameter list is replaced with parameter.

Parameters:

parameter_index – The index of the parameter to replace.
parameter – The parameter to substitute for the parameter_indexth parameter.

inline const ov::ParameterVector &get_parameters() const: Return the model parameters.

inline const ov::ResultVector &get_results() const: Return a list of model’s outputs.

int64_t get_parameter_index(const std::shared_ptr<ov::op::v0::Parameter> &parameter) const: Index for parameter, or -1.

int64_t get_result_index(const ov::Output<ov::Node> &value) const

Return the index of this model’s Result represented by the “value” Output object. This method returns -1 if an the passed output is not related to the Results of a model.

Parameters:: value – Output containing Node

int64_t get_result_index(const ov::Output<const ov::Node> &value) const

Return the index of this model’s Result represented by the “value” Output object. This method returns -1 if an the passed output is not related to the Results of a model.

Parameters:: value – Output containing Node

bool evaluate(ov::TensorVector &output_tensors, const ov::TensorVector &input_tensors, ov::EvaluationContext &evaluation_context) const

Evaluate the model on inputs, putting results in outputs.

Parameters:

output_tensors – Tensors for the outputs to compute. One for each result
input_tensors – Tensors for the inputs. One for each inputs.
evaluation_context – Storage of additional settings and attributes that can be used when evaluating the model. This additional information can be shared across nodes.

bool evaluate(ov::TensorVector &output_tensors, const ov::TensorVector &input_tensors) const

Evaluate the model on inputs, putting results in outputs.

Parameters:

output_tensors – Tensors for the outputs to compute. One for each result
input_tensors – Tensors for the inputs. One for each inputs.

inline const ov::SinkVector &get_sinks() const: Return a list of model’s sinks.

void add_sinks(const ov::SinkVector &sinks)

Add new sink nodes to the list. Method doesn’t validate graph, it should be done manually after all changes.

Parameters:: sinks – new sink nodes

void remove_sink(const std::shared_ptr<ov::op::Sink> &sink)

Delete sink node from the list of sinks. Method doesn’t delete node from graph.

Parameters:: sink – Sink to delete

void add_results(const ov::ResultVector &results)

Add new Result nodes to the list. Method doesn’t validate graph, it should be done manually after all changes.

Parameters:: results – new Result nodes

void remove_result(const std::shared_ptr<ov::op::v0::Result> &result)

Delete Result node from the list of results. Method will not delete node from graph.

Parameters:: result – Result node to delete

void add_parameters(const ov::ParameterVector &params)

Add new Parameter nodes to the list.

Method doesn’t change or validate graph, it should be done manually. For example, if you want to replace ReadValue node by Parameter, you should do the following steps:

replace node ReadValue by Parameter in graph
call add_parameter() to add new input to the list
call graph validation to check correctness of changes

Parameters:: params – new Parameter nodes

void remove_parameter(const std::shared_ptr<ov::op::v0::Parameter> &param)

Delete Parameter node from the list of parameters. Method will not delete node from graph. You need to replace Parameter with other operation manually. Attention: Indexing of parameters can be changed.

Possible use of method is to replace input by variable. For it the following steps should be done:

Parameter node should be replaced by ReadValue
call remove_parameter(param) to remove input from the list
check if any parameter indexes are saved/used somewhere, update it for all inputs because indexes can be changed
call graph validation to check all changes

Parameters:: param – Parameter node to delete

void add_variables(const ov::op::util::VariableVector &variables)

Add new variables to the list. Method doesn’t validate graph, it should be done manually after all changes.

Parameters:: variables – new variables to add

void remove_variable(const ov::op::util::Variable::Ptr &variable)

Delete variable from the list of variables. Method doesn’t delete nodes that used this variable from the graph.

Parameters:: variable – Variable to delete

inline const ov::op::util::VariableVector &get_variables() const: Return a list of model’s variables.

ov::op::util::Variable::Ptr get_variable_by_id(const std::string &variable_id) const: Return a variable by specified variable_id.

inline RTMap &get_rt_info()

Returns a runtime info.

Returns:: reference to ov::AnyMap with runtime info

inline const RTMap &get_rt_info() const

Returns a constant runtime info.

Returns:: reference to const ov::AnyMap with runtime info

template<class T, class ...Args, typename std::enable_if<!std::is_same<T, ov::Any>::value, bool>::type = true> inline const T &get_rt_info(Args... args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters:

T – the type of returned value
Args – types of variadic arguments

Parameters:

args – path to the runtime attribute

Returns:

constant reference to value from runtime info

template<class T, class ...Args, typename std::enable_if<std::is_same<T, ov::Any>::value, bool>::type = true> inline const T &get_rt_info(Args... args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters:

T – the type of returned value
Args – types of variadic arguments

Parameters:

args – path to the runtime attribute

Returns:

constant reference to value from runtime info

template<class T, typename std::enable_if<!std::is_same<T, ov::Any>::value, bool>::type = true> inline const T &get_rt_info(const std::vector<std::string> &args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters:: T – the type of returned value
Parameters:: args – vector with path to the runtime attribute
Returns:: constant reference to value from runtime info

template<class T, typename std::enable_if<std::is_same<T, ov::Any>::value, bool>::type = true> inline const T &get_rt_info(const std::vector<std::string> &args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters:: T – the type of returned value
Parameters:: args – vector with path to the runtime attribute
Returns:: constant reference to value from runtime info

template<class ...Args> inline bool has_rt_info(Args... args) const

Checks if given path exists in runtime info.

Template Parameters:: Args – types of variadic arguments
Parameters:: args – path to the runtime attribute
Returns:: true if path exists, otherwise false

bool has_rt_info(const std::vector<std::string> &args) const

Checks if given path exists in runtime info.

Parameters:: args – vector with path to the runtime attribute
Returns:: true if path exists, otherwise false

template<class T, class ...Args> inline void set_rt_info(const T &argument, Args... args)

Add value inside the runtime info.

Template Parameters:

T – type of new value
Args – types of variadic arguments

Parameters:

argument – value for the runtime info
args – path to the runtime attribute

template<class T> inline void set_rt_info(const T &argument, const std::vector<std::string> &args)

Add value inside the runtime info.

Template Parameters:

T – type of new value

Parameters:

argument – value for the runtime info
args – vector with path to the runtime attribute

class Node : public std::enable_shared_from_this<Node>

#include <node.hpp>

Nodes are the backbone of the graph of Value dataflow. Every node has zero or more nodes as arguments and one value, which is either a tensor or a (possibly empty) tuple of values.

Subclassed by ov::op::Op, ov::pass::pattern::op::Pattern

Public Functions

virtual void validate_and_infer_types()

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual const ov::op::AutoBroadcastSpec &get_autob() const

Returns:: the autobroadcasr spec

virtual bool has_evaluate() const: Allows to get information about availability of evaluate method for the current operation.

virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values) const

Evaluates the op on input_values putting results in output_values.

Parameters:

output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.

Returns:

true if successful

virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values, const ov::EvaluationContext &evaluationContext) const

Evaluates the op on input_values putting results in output_values.

Parameters:

output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
evaluation_context – Storage of additional settings and attributes that can be used when evaluating the op.

Returns:

true if successful

inline virtual OutputVector decompose_op() const

Decomposes the FusedOp into a sub-graph consisting of core openvino ops.

Returns:: A vector of nodes comprising the sub-graph. The order of output tensors must match the match output tensors of the FusedOp

virtual const type_info_t &get_type_info() const = 0: Returns the NodeTypeInfo for the node’s class. During transition to type_info, returns a dummy type_info for Node if the class has not been updated yet.

void set_arguments(const NodeVector &arguments): Sets/replaces the arguments with new arguments.

void set_arguments(const OutputVector &arguments): Sets/replaces the arguments with new arguments.

void set_argument(size_t position, const Output<Node> &argument): Sets/replaces the arguments with new arguments.

void set_output_size(size_t output_size): Sets the number of outputs.

virtual std::string description() const

Get the string name for the type of the node, such as Add or Multiply. The class name, must not contain spaces as it is used for codegen.

Returns:: A const reference to the node’s type name

const std::string &get_name() const

Get the unique name of the node.

Returns:: A const reference to the node’s unique name.

void set_friendly_name(const std::string &name)

Sets a friendly name for a node. This does not overwrite the unique name of the node and is retrieved via get_friendly_name(). Used mainly for debugging. The friendly name may be set exactly once.

Parameters:: name – is the friendly name to set

const std::string &get_friendly_name() const

Gets the friendly name for a node. If no friendly name has been set via set_friendly_name then the node’s unique name is returned.

Returns:: A const reference to the node’s friendly name.

virtual std::ostream &write_description(std::ostream &os, uint32_t depth = 0) const

Writes a description of a node to a stream.

Parameters:

os – The stream; should be returned
depth – How many levels of inputs to describe

Returns:

The stream os

const std::vector<std::shared_ptr<Node>> &get_control_dependencies() const: Get control dependencies registered on the node.

const std::vector<Node*> &get_control_dependents() const: Get nodes dependent on this node.

void add_control_dependency(std::shared_ptr<Node> node): This node cannot execute until node executes.

void remove_control_dependency(std::shared_ptr<Node> node): Remove the dependency of this node on node.

void clear_control_dependencies(): Remove all dependencies from this node.

void clear_control_dependents(): Remove this node as a dependency from all dependent nodes.

void add_node_control_dependencies(const std::shared_ptr<const Node> &source_node): This node absorbs the control dependencies of source_node.

void add_node_control_dependents(const std::shared_ptr<const Node> &source_node): This node becomes a dependent of every node dependent on source_node.

void transfer_control_dependents(std::shared_ptr<Node> replacement): This node’s control dependencies are replaced by replacement.

size_t get_output_size() const: Returns the number of outputs from the node.

const element::Type &get_output_element_type(size_t i) const: Returns the element type for output i.

const element::Type &get_element_type() const: Checks that there is exactly one output and returns its element type.

const Shape &get_output_shape(size_t i) const: Returns the shape for output i.

const PartialShape &get_output_partial_shape(size_t i) const: Returns the partial shape for output i.

Output<const Node> get_default_output() const: Return the output to use when converting to an Output<Node> with no index specified. Throws when not supported.

virtual size_t get_default_output_index() const: Returns the output of the default output, or throws if there is none.

size_t no_default_index() const: Throws no default.

const Shape &get_shape() const: Checks that there is exactly one output and returns its shape.

descriptor::Tensor &get_output_tensor(size_t i) const: Returns the tensor for output or input i.

size_t get_input_size() const: Returns the number of inputs for the op.

const element::Type &get_input_element_type(size_t i) const: Returns the element type of input i.

const Shape &get_input_shape(size_t i) const: Returns the shape of input i.

const PartialShape &get_input_partial_shape(size_t i) const: Returns the partial shape of input i.

bool has_same_type(std::shared_ptr<const Node> node) const: True if this and node have one output with same element type and shape.

NodeVector get_users(bool check_is_used = false) const: Get all the nodes that uses the current node.

inline bool operator<(const Node &other) const: Use instance ids for comparison instead of memory addresses to improve determinism.

std::vector<Input<Node>> inputs()

Returns:: A vector containing a handle for each of this node’s inputs, in order.

std::vector<Input<const Node>> inputs() const

Returns:: A vector containing a handle for each of this node’s inputs, in order.

std::vector<Output<Node>> input_values() const

Returns:: A vector containing the values for each input

std::vector<Output<Node>> outputs()

Returns:: A vector containing a handle for each of this node’s outputs, in order.

std::vector<Output<const Node>> outputs() const

Returns:: A vector containing a handle for each of this node’s outputs, in order.

Input<Node> input(size_t input_index)

Throws:: std::out_of_range – if the node does not have at least input_index+1 inputs.
Returns:: A handle to the input_indexth input of this node.

Input<const Node> input(size_t input_index) const

Throws:: std::out_of_range – if the node does not have at least input_index+1 inputs.
Returns:: A handle to the input_indexth input of this node.

Output<Node> output(size_t output_index)

Throws:: std::out_of_range – if the node does not have at least output_index+1 outputs.
Returns:: A handle to the output_indexth output of this node.

Output<const Node> output(size_t output_index) const

Throws:: std::out_of_range – if the node does not have at least output_index+1 outputs.
Returns:: A handle to the output_indexth output of this node.

template<> class Input<Node>

#include <node_input.hpp>

A handle for one of a node’s inputs.

Public Functions

Input(Node *node, size_t index)

Constructs a Input.

Parameters:

node – Pointer to the node for the input handle.
index – The index of the input.

Node *get_node() const

Returns:: A pointer to the node referenced by this input handle.

size_t get_index() const

Returns:: The index of the input referred to by this input handle.

OV_NO_DANGLING const element::Type & get_element_type () const

Returns:: The element type of the input referred to by this input handle.

OV_NO_DANGLING const Shape & get_shape () const

Returns:: The shape of the input referred to by this input handle.

OV_NO_DANGLING const PartialShape & get_partial_shape () const

Returns:: The partial shape of the input referred to by this input handle.

Output<Node> get_source_output() const

Returns:: A handle to the output that is connected to this input.

OV_NO_DANGLING descriptor::Tensor & get_tensor () const

Returns:: A reference to the tensor descriptor for this input.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const

Returns:: A shared pointer to the tensor descriptor for this input.

bool get_is_relevant_to_shapes() const

Returns:: true if this input is relevant to its node’s output shapes; else false.

bool get_is_relevant_to_values() const

Returns:: true if this input is relevant to its node’s output values; else false.

void replace_source_output(const Output<Node> &new_source_output) const

Replaces the source output of this input.

Parameters:: new_source_output – A handle for the output that will replace this input’s source.

RTMap &get_rt_info()

Returns:: The reference to runtime info map

OV_NO_DANGLING const RTMap & get_rt_info () const

Returns:: The constant reference to runtime info map

template<> class Input<const Node>

#include <node_input.hpp>

A handle for one of a node’s inputs.

Public Functions

Input(const Node *node, size_t index)

Constructs a Input.

Parameters:

node – Pointer to the node for the input handle.
index – The index of the input.

const Node *get_node() const

Returns:: A pointer to the node referenced by this input handle.

size_t get_index() const

Returns:: The index of the input referred to by this input handle.

OV_NO_DANGLING const element::Type & get_element_type () const

Returns:: The element type of the input referred to by this input handle.

OV_NO_DANGLING const Shape & get_shape () const

Returns:: The shape of the input referred to by this input handle.

OV_NO_DANGLING const PartialShape & get_partial_shape () const

Returns:: The partial shape of the input referred to by this input handle.

Output<Node> get_source_output() const

Returns:: A handle to the output that is connected to this input.

OV_NO_DANGLING descriptor::Tensor & get_tensor () const

Returns:: A reference to the tensor descriptor for this input.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const

Returns:: A shared pointer to the tensor descriptor for this input.

bool get_is_relevant_to_shapes() const

Returns:: true if this input is relevant to its node’s output shapes; else false.

bool get_is_relevant_to_values() const

Returns:: true if this input is relevant to its node’s output values; else false.

OV_NO_DANGLING const RTMap & get_rt_info () const

Returns:: The constant reference to runtime info map

template<> class Output<Node>

#include <node_output.hpp>

A handle for one of a node’s outputs.

Public Functions

Output(Node *node, size_t index)

Constructs a Output.

Parameters:

node – A pointer to the node for the output handle.
index – The index of the output.

Output(const std::shared_ptr<Node> &node, size_t index)

Constructs a Output.

Parameters:

node – A shared_ptr to the node for the output handle.
index – The index of the output.

template<typename T> inline Output(const std::shared_ptr<T> &node)

Constructs a Output, referencing the default output of the node. If the node doesn’t have a default output, an exception will be thrown.

Parameters:: node – A shared_ptr to the node for the output handle.

Output() = default: A null output.

Node *get_node() const

Returns:: A pointer to the node referred to by this output handle.

std::shared_ptr<Node> get_node_shared_ptr() const

Returns:: A shared_ptr to the node referred to by this output handle.

size_t get_index() const

Returns:: The index of the output referred to by this output handle.

OV_NO_DANGLING descriptor::Tensor & get_tensor () const

Returns:: A reference to the tensor descriptor for this output.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const

Returns:: A shared point to the tensor ptr for this output.

void set_tensor_ptr(std::shared_ptr<descriptor::Tensor> tensor_ptr)

Returns:: Set new tensor desc shared pointer to this output

OV_NO_DANGLING const element::Type & get_element_type () const

Returns:: The element type of the output referred to by this output handle.

OV_NO_DANGLING const Shape & get_shape () const

Returns:: The shape of the output referred to by this output handle.

OV_NO_DANGLING const PartialShape & get_partial_shape () const

Returns:: The partial shape of the output referred to by this output handle.

RTMap &get_rt_info()

Returns:: The reference to runtime info map

OV_NO_DANGLING const RTMap & get_rt_info () const

Returns:: The constant reference to runtime info map

OV_NO_DANGLING const std::unordered_set< std::string > & get_names () const

Returns:: The tensor names associated with this output

std::string get_any_name() const

Returns:: Any tensor names associated with this output

void set_names(const std::unordered_set<std::string> &names)

Returns:: Set tensor names associated with this output

void add_names(const std::unordered_set<std::string> &names)

Returns:: Add tensor names associated with this output

std::set<Input<Node>> get_target_inputs() const

Returns:: A set containing handles for all inputs targeted by the output referenced by this output handle.

void remove_target_input(const Input<Node> &target_input) const

Removes a target input from the output referenced by this output handle.

Parameters:: target_input – The target input to remove.

void replace(const Output<Node> &replacement): Replace all users of this value with replacement.

template<> class Output<const Node>

#include <node_output.hpp>

A handle for one of a node’s outputs.

Public Functions

Output(const Node *node, size_t index)

Constructs a Output.

Parameters:

node – A pointer to the node for the output handle.
index – The index of the output.

Output(const std::shared_ptr<const Node> &node, size_t index)

Constructs a Output.

Parameters:

node – A shared_ptr to the node for the output handle.
index – The index of the output.

template<typename T> inline Output(const std::shared_ptr<const T> &node)

Constructs a Output, referencing the zeroth output of the node.

Parameters:: node – A shared_ptr to the node for the output handle.

Output() = default: A null output.

const Node *get_node() const

Returns:: A pointer to the node referred to by this output handle.

std::shared_ptr<const Node> get_node_shared_ptr() const

Returns:: A shared_ptr to the node referred to by this output handle.

size_t get_index() const

Returns:: The index of the output referred to by this output handle.

OV_NO_DANGLING descriptor::Tensor & get_tensor () const

Returns:: A reference to the tensor descriptor for this output.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const

Returns:: A shared point to the tensor ptr for this output.

OV_NO_DANGLING const element::Type & get_element_type () const

Returns:: The element type of the output referred to by this output handle.

OV_NO_DANGLING const Shape & get_shape () const

Returns:: The shape of the output referred to by this output handle.

OV_NO_DANGLING const PartialShape & get_partial_shape () const

Returns:: The partial shape of the output referred to by this output handle.

OV_NO_DANGLING const RTMap & get_rt_info () const

Returns:: The constant reference to runtime info map

OV_NO_DANGLING const std::unordered_set< std::string > & get_names () const

Returns:: The tensor names associated with this output

std::string get_any_name() const

Returns:: Any tensor name associated with this output

std::set<Input<Node>> get_target_inputs() const

Returns:: A set containing handles for all inputs targeted by the output referenced by this output handle.

class PartialShape

#include <partial_shape.hpp>

Class representing a shape that may be partially or totally dynamic.

A PartialShape may have:

Dynamic rank. (Informal notation: ?)
Static rank, but dynamic dimensions on some or all axes. (Informal notation examples: {1,2,?,4}, {?,?,?})
Static rank, and static dimensions on all axes. (Informal notation examples: {1,2,3,4}, {6}, {})

Public Functions

PartialShape(std::initializer_list<Dimension> init)

Constructs a shape with static rank from an initializer list of Dimension.

Examples:

PartialShape s{2,3,4};                     // rank=3, all dimensions static
PartialShape s{};                          // rank=0
PartialShape s{2,Dimension::dynamic(),3};  // rank=3, dimension 1 dynamic

Parameters:: init – The Dimension values for the constructed shape.

PartialShape(std::vector<Dimension> dimensions)

Constructs a PartialShape with static rank from a vector of Dimension.

Parameters:: dimensions – The Dimension values for the constructed shape.

PartialShape(const std::vector<Dimension::value_type> &dimensions)

Constructs a PartialShape with static rank from a vector of dimensions values.

Parameters:: dimensions – The Dimension values for the constructed shape.

PartialShape(): Constructs a static PartialShape with zero rank (the shape of a scalar).

PartialShape(const Shape &shape)

Constructs a static PartialShape from a PartialShape.

Parameters:: shape – The PartialShape to convert into PartialShape.

PartialShape(const std::string &shape)

Constructs a static PartialShape from a string.

Parameters:: shape – The string to parse into PartialShape.

bool is_static() const

Check if this shape is static.

A shape is considered static if it has static rank, and all dimensions of the shape are static.

Returns:: true if this shape is static, else false.

inline bool is_dynamic() const

Check if this shape is dynamic.

A shape is considered static if it has static rank, and all dimensions of the shape are static.

Returns:: false if this shape is static, else true.

inline Rank rank() const

Get the rank of the shape.

Returns:: The rank of the shape. This will be Rank::dynamic() if the rank of the shape is dynamic.

bool compatible(const PartialShape &s) const

Check whether this shape is compatible with the argument, i.e., whether it is possible to merge them.

Two shapes are compatible if

one or both of them has dynamic rank, or
both shapes have dynamic and equal rank, and their dimensions are elementwise compatible (see Dimension::compatible()).

Parameters:: s – The shape to be checked for compatibility with this shape.
Returns:: true if this shape is compatible with s, else false.

bool same_scheme(const PartialShape &s) const

Check whether this shape represents the same scheme as the argument.

Two shapes s1 and s2 represent the same scheme if

they both have dynamic rank, or
they both have static and equal rank r, and for every i from 0 to r-1, s1[i] represents the same scheme as s2[i] (see Dimension::same_scheme()).

Parameters:: s – The shape whose scheme is being compared with this shape.
Returns:: true if this shape represents the same scheme as s, else false.

bool relaxes(const PartialShape &s) const

Check whether this shape is a relaxation of the argument.

Intuitively, a PartialShape s1 is said to relax s2 (or is a relaxation of s2) if it is “more permissive” than s2. In other words, s1 is a relaxation of s2 if anything you can form by plugging things into the dynamic dimensions of s2 is also something you can form by plugging things into the dynamic dimensions of s1, but not necessarily the other way around.

s1.relaxes(s2) is equivalent to s2.refines(s1).

Formally, PartialShape s1 is said to relax PartialShape s2 if:

For every i from 0 to r-1, either s1[i] contains s2[i].

Parameters:: s – The shape which is being compared against this shape.
Returns:: true if this shape relaxes s, else false.

bool refines(const PartialShape &s) const

Check whether this shape is a refinement of the argument.

Intuitively, a PartialShape s1 is said to relax s2 (or is a relaxation of s2) if it is “less permissive” than s2. In other words, s1 is a relaxation of s2 if anything you can form by plugging things into the dynamic dimensions of s1 is also something you can form by plugging things into the dynamic dimensions of s2, but not necessarily the other way around.

s1.refines(s2) is equivalent to s2.relaxes(s1).

Formally, PartialShape s1 is said to refine PartialShape s2 if:

s2 has dynamic rank, or
s1 and s2 both have static rank r, and for every i from 0 to r-1, either s2[i] is dynamic, or s1[i] == s2[i].

Parameters:: s – The shape which is being compared against this shape.
Returns:: true if this shape refines s, else false.

bool merge_rank(const Rank &r)

Checks that this shape’s rank is compatible with r, and, if this shape’s rank is dynamic and r is static, updates this shape to have a rank of r with dimensions all dynamic.

Returns:: true if this shape’s rank is compatible with r, else false.

Shape to_shape() const

Convert a static PartialShape to a PartialShape.

Throws:: std::invalid_argument – If this PartialShape is dynamic.
Returns:: A new PartialShape s where s[i] = size_t((*this)[i]).

bool all_non_negative() const: Returns true if all static dimensions of the tensor are non-negative, else false.

Dimension &operator[](std::ptrdiff_t i)

Index operator for PartialShape, with bound checking.

Parameters:: i – The index of the dimension being selected in range [-rank, rank).
Returns:: A reference to the ith Dimension of this shape.

const Dimension &operator[](std::ptrdiff_t i) const

Index operator for PartialShape, with bound checking.

Parameters:: i – The index of the dimension being selected in range [-rank, rank).
Returns:: A reference to the ith Dimension of this shape.

inline explicit operator std::vector<Dimension>() const: Returns a vector of the dimensions. This has no meaning if dynamic.

Shape get_max_shape() const: Get the max bounding shape.

Shape get_min_shape() const: Get the min bounding shape.

Shape get_shape() const: Get the unique shape.

inline iterator begin() noexcept: Returns a read/write iterator that points to the first element in the shape. Iteration is done in ordinary element order.

inline const_iterator begin() const noexcept: Returns a read-only (constant) iterator that points to the first element in the shape. Iteration is done in ordinary element order.

inline iterator end() noexcept: Returns a read/write iterator that points one past the last element in the shape. Iteration is done in ordinary element order.

inline const_iterator end() const noexcept: Returns a read-only (constant) iterator that points one past the last element in the shape. Iteration is done in ordinary element order.

inline reverse_iterator rbegin() noexcept: Returns a read/write reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.

inline const_reverse_iterator rbegin() const noexcept: Returns a read-only (constant) reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.

inline reverse_iterator rend() noexcept: Returns a read/write reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.

inline const_reverse_iterator rend() const noexcept: Returns a read-only (constant) reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.

inline const_iterator cbegin() const noexcept: Returns a read-only (constant) iterator that points to the first element in the shape. Iteration is done in ordinary element order.

inline const_iterator cend() const noexcept: Returns a read-only (constant) iterator that points one past the last element in the shape. Iteration is done in ordinary element order.

inline const_reverse_iterator crbegin() const noexcept: Returns a read-only (constant) reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.

inline const_reverse_iterator crend() const noexcept: Returns a read-only (constant) reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.

inline void resize(size_t count): Resizes dimensions container to contain count elements.

inline size_t size() const: Returns size of dimension vector. Requires rank to be static.

inline iterator insert(iterator position, const Dimension &val): Returns a read/write iterator that points to the inserted element in the shape.

inline void insert(iterator position, size_t n, const Dimension &val): Inserts count copies of the value before position.

template<class InputIterator> inline void insert(iterator position, InputIterator first, InputIterator last): Inserts elements from range [first, last) before position.

inline void reserve(size_t n): Requests that the dimensions vector capacity be enough to contain n elements.

inline void push_back(const Dimension &val): push element to the end of partial shape

template<class ...Args> inline void emplace_back(Args&&... args): emplace element to the end of partial shape

std::string to_string() const: String representation of PartialShape.

Public Static Functions

static PartialShape dynamic(Rank r = Rank::dynamic())

Construct a PartialShape with the given rank and all dimensions (if any) dynamic.

Returns:: A PartialShape with the given rank, and all dimensions (if any) dynamic.

static bool merge_into(PartialShape &dst, const PartialShape &src)

Try to merge one shape into another.

Merges src into dst, returning true on success and false on failure. If false is returned, the effect on dst is unspecified.

To merge two partial shapes s1 and s2 is to find the most permissive partial shape s that is no more permissive than s1 or s2, if s exists. For example:

merge(?,?) -> ?
merge(?,{?,?}) -> {?,?}
merge({?,?},{?,?}) -> {?,?}
merge({1,2,3,4},?) -> {1,2,3,4}
merge({1,2},{1,?}) -> {1,2}
merge({1,2,?,?},{1,?,3,?}) -> {1,2,3,?}
merge({1,2,3},{1,2,3}) -> {1,2,3}

merge({1,?},{2,?}) fails [dimension 0 constraints are inconsistent]
merge({?,?},{?,?,?}) fails [ranks are inconsistent]

This function (merge_into) performs the “merge” operation described above on dst and src, but overwrites dst with the result and returns true if merging is successful; if merging is unsuccessful, the function returns false and may make unspecified changes to dst.

Parameters:

dst – [inout] The shape that src will be merged into.
src – The shape that will be merged into dst.

Returns:

true if merging succeeds, else false.

static bool broadcast_merge_into(PartialShape &dst, const PartialShape &src, const ov::op::AutoBroadcastSpec &autob): Try to merge one shape into another along with implicit broadcasting.

Friends

friend OPENVINO_API std::ostream & operator<< (std::ostream &str, const PartialShape &shape)

Inserts a human-readable representation of a PartialShape into an output stream.

The output to the stream is in “informal” notation. In other words:

If shape has dynamic rank, inserts the string ?.
If shape has static rank, inserts the string {, then inserts each dimension of shape into the output stream separated by commas, then inserts }.

Example:

PartialShape s1{PartialShape::dynamic())};
PartialShape s2{};
PartialShape s3{1,Dimension::dynamic(),2,3};
PartialShape s4{2,3,4};
std::cout << s1 << std::endl
          << s2 << std::endl
          << s3 << std::endl
          << s4 << std::endl;

Output:

?
{}
{1,?,2,3}
{2,3,4}

Parameters:

str – The output stream targeted for insertion.
shape – The shape to be inserted into str.

Returns:

A reference to str after insertion.

friend OPENVINO_API PartialShape operator+ (const PartialShape &s1, const PartialShape &s2)

Elementwise addition of two PartialShape objects.

If s1 or s2 has dynamic rank, returns PartialShape::dynamic().
If s1 ands2` both have static rank, and their ranks are unequal, throws std::invalid_argument.
If s1 and s2 both have static rank, and their ranks are equal, returns a new shape whose ith dimension is s1[i] + s2[i].

Parameters:

s1 – Left operand for addition.
s2 – Right operand for addition.

Throws:

std::invalid_argument – If s1 and s2 have inconsistent ranks.

Returns:

The result of elementwise adding s1 to s2 (see description).

class PrePostProcessor

#include <pre_post_process.hpp>

Main class for adding pre- and post- processing steps to existing ov::Model.

This is a helper class for writing easy pre- and post- processing operations on ov::Model object assuming that any preprocess operation takes one input and produces one output.

For advanced preprocessing scenarios, like combining several functions with multiple inputs/outputs into one, client’s code can use transformation passes over ov::Model

Public Functions

explicit PrePostProcessor(const std::shared_ptr<Model> &function)

Default constructor.

Parameters:: function – Existing function representing loaded model

PrePostProcessor(PrePostProcessor&&) noexcept: Default move constructor.

PrePostProcessor &operator=(PrePostProcessor&&) noexcept: Default move assignment operator.

~PrePostProcessor(): Default destructor.

InputInfo &input()

Gets input pre-processing data structure. Should be used only if model/function has only one input Using returned structure application’s code is able to set user’s tensor data (e.g layout), preprocess steps, target model’s data.

Returns:: Reference to model’s input information structure

InputInfo &input(const std::string &tensor_name)

Gets input pre-processing data structure for input identified by it’s tensor name.

Parameters:: tensor_name – Tensor name of specific input. Throws if tensor name is not associated with any input in a model
Returns:: Reference to model’s input information structure

InputInfo &input(size_t input_index)

Gets input pre-processing data structure for input identified by it’s order in a model.

Parameters:: input_index – Input index of specific input. Throws if input index is out of range for associated function
Returns:: Reference to model’s input information structure

OutputInfo &output()

Gets output post-processing data structure. Should be used only if model/function has only one output Using returned structure application’s code is able to set model’s output data, post-process steps, user’s tensor data (e.g layout)

Returns:: Reference to model’s output information structure

OutputInfo &output(const std::string &tensor_name)

Gets output post-processing data structure for output identified by it’s tensor name.

Parameters:: tensor_name – Tensor name of specific output. Throws if tensor name is not associated with any input in a model
Returns:: Reference to model’s output information structure

OutputInfo &output(size_t output_index)

Gets output post-processing data structure for output identified by it’s order in a model.

Parameters:: output_index – Output index of specific output. Throws if output index is out of range for associated function
Returns:: Reference to model’s output information structure

std::shared_ptr<Model> build()

Adds pre/post-processing operations to function passed in constructor.

Returns:: Function with added pre/post-processing operations

class Shape : public std::vector<size_t>

#include <shape.hpp>

Shape for a tensor.

Public Functions

OPENVINO_API Shape::reference operator[] (std::ptrdiff_t i)

Gets dimension at index.

Parameters:: i – Index to shape dimension [-rank, rank).
Returns:: A reference to i-th dimension of this shape.

OPENVINO_API Shape::const_reference operator[] (std::ptrdiff_t i) const

Gets dimension at index.

Parameters:: i – Index to shape dimension [-rank, rank).
Returns:: A const reference to i-th dimension of this shape.

OPENVINO_API Shape::reference at (std::ptrdiff_t i)

Gets dimension at index, with bounds checking.

Parameters:: i – Index to shape dimension [-rank, rank).
Returns:: A reference to i-th dimension of this shape.

OPENVINO_API Shape::const_reference at (std::ptrdiff_t i) const

Gets dimension at index, with bounds checking.

Parameters:: i – Index to shape dimension [-rank, rank).
Returns:: A const reference to i-th dimension of this shape.

class Symbol

#include <symbol.hpp>

Class representing unique symbol for the purpose of symbolic shape inference. Equality of symbols is being tracked by Disjoint-set data structure.

Public Functions

Symbol() = default: Default constructs a unique symbol.

struct DiscreteTypeInfo

#include <type.hpp>

Type information for a type system without inheritance; instances have exactly one type not related to any other type.

Supports three functions, ov::is_type<Type>, ov::as_type<Type>, and ov::as_type_ptr<Type> for type-safe dynamic conversions via static_cast/static_ptr_cast without using C++ RTTI. Type must have a static type_info member and a virtual get_type_info() member that returns a reference to its type_info member.