Group Execution model utilities

group ov_dev_exec_model

Contains ExecutionNode and its properties.

Variables

static const char ORIGINAL_NAMES[] = "originalLayersNames"

Used to get a string of layer names separated by a comma from the original IR, which were fused/merged to the current executable primitive.

static const char IMPL_TYPE[] = "primitiveType"

Used to get a type of the executable primitive.

static const char OUTPUT_PRECISIONS[] = "outputPrecisions"

Used to get output precisions of the executable primitive.

static const char PERF_COUNTER[] = "execTimeMcs"

Used to get a value of execution time of the executable primitive.

static const char OUTPUT_LAYOUTS[] = "outputLayouts"

Used to get output layouts of primitive.

static const char EXECUTION_ORDER[] = "execOrder"

Used to get an execution order of primitive.

static const char LAYER_TYPE[] = "layerType"

Used to get a type of primitive.

static const char RUNTIME_PRECISION[] = "runtimePrecision"

Used to get runtime precision of the executable primitive.

class ExecutionNode : public ov::op::Op
#include <exec_model_info.hpp>

The Execution node which is used to represent node in execution graph.

It contains the following type of information in node runtime information:

  • ExecGraphInfoSerialization::ORIGINAL_NAMES

  • ExecGraphInfoSerialization::IMPL_TYPE

  • ExecGraphInfoSerialization::OUTPUT_PRECISIONS

  • ExecGraphInfoSerialization::PERF_COUNTER

  • ExecGraphInfoSerialization::OUTPUT_LAYOUTS

  • ExecGraphInfoSerialization::EXECUTION_ORDER

  • ExecGraphInfoSerialization::LAYER_TYPE

  • ExecGraphInfoSerialization::RUNTIME_PRECISION

Public Functions

ExecutionNode()

A default constructor with no node inputs and 0 output ports.

ExecutionNode(const ov::OutputVector &arguments, size_t output_size = 1)

Constructs a new execution node with a given parameters.

Parameters
  • arguments[in] Inputs nodes

  • output_size[in] A number of output ports

std::shared_ptr<ov::Node> clone_with_new_inputs(const ov::OutputVector &inputs) const override

Creates a new execution node with the same state, but different input nodes.

Parameters

inputs[in] The input nodes

Returns

A newly created execution node

virtual bool visit_attributes(ov::AttributeVisitor&) override

Visits attributes of the node.

Parameters

visitor[in] An attribute visitor

Returns

Returns true if an operation has completed successfully

namespace ov

transformation aligns elementwise constant inputs ranks with its output rank

A namespace with const values for Execution Graph parameters names.

Executable Model Info is represented in ov::Model format with general ExecutionNode nodes inside including connections between the nodes. Each node describes an executable hardware-specific primitive and stores its parameters within ExecutionNode::get_rt_info map. There is a list of general keys for the parameters map.

OpenVINO C++ API.

Resolves transpose_b key from MatMul operation if corresponding input is constant or FakeQuantize by inserting Transpose.

Functions

LP_TRANSFORMATIONS_API void mark_as_bias (const std::shared_ptr< Node > &node)
LP_TRANSFORMATIONS_API bool marked_as_bias (const std::shared_ptr< const Node > &node)
std::ostream &operator<<(std::ostream &out, const Mask &mask)
Mask::Ptr getMask(const Output<const Node> &output)
Mask::Ptr getMask(const Output<Node> &output)
void setMask(Output<Node> output, const Mask::Ptr &mask)
void setMask(Input<Node> node, const Mask::Ptr &mask)
void mark_as_decompression(const std::shared_ptr<Node> &node)
void unmark_as_decompression(const std::shared_ptr<Node> &node)
bool is_decompression(const std::shared_ptr<Node> &node)
void mark_as_dequantization_node(const std::shared_ptr<Node> &node)
bool is_dequantization_node(const std::shared_ptr<Node> &node)
void disable_fp16_compression(const std::shared_ptr<Node> &node)
void enable_fp16_compression(const std::shared_ptr<Node> &node)
bool fp16_compression_is_disabled(const std::shared_ptr<const Node> &node)
void postpone_fp16_compression(RTMap &rt_info)
bool is_fp16_compression_postponed(const RTMap &rt_info)
void do_not_postpone_fp16_compression(RTMap &rt_info)
std::string getFusedNames(const std::shared_ptr<ov::Node> &node)

getFusedNames return string with operation names separated by coma in alphabetical order

Parameters

node[in] The node will be used to get FusedNames attribute

std::vector<std::string> getFusedNamesVector(const std::shared_ptr<ov::Node> &node)

getFusedNamesVector return vector of fused names sorted in alphabetical order

Parameters

node[in] The node will be used to get FusedNames attribute

Returns

vector of strings

void mark_shape_subgraph(const std::shared_ptr<Node> &node)
void unmark_shape_subgraph(const std::shared_ptr<Node> &node)
bool is_shape_subgraph(const std::shared_ptr<const Node> &node)
void enable_keep_const_precision(const std::shared_ptr<Node> &node)
void disable_keep_const_precision(const std::shared_ptr<Node> &node)
bool is_keep_const_precision(const std::shared_ptr<const Node> &node)
bool has_nms_selected_indices(const Node *node)
void set_nms_selected_indices(Node *node)
void disable_divide_conversion(const std::shared_ptr<Node> &node)
void enable_divide_conversion(const std::shared_ptr<Node> &node)
bool divide_is_nonconvertible(const std::shared_ptr<Node> &node)
inline bool has_old_api_map_element_type(const std::shared_ptr<Node> &node)
inline OldApiMapElementType get_old_api_map_element_type(const std::shared_ptr<Node> &node)
inline void set_old_api_map_element_type(const std::shared_ptr<Node> &node, const OldApiMapElementType &old_api_map)
inline bool has_old_api_map_order(const std::shared_ptr<Node> &node)
inline OldApiMapOrder get_old_api_map_order(const std::shared_ptr<Node> &node)
inline void set_old_api_map_order(std::shared_ptr<Node> &node, const OldApiMapOrder &old_api_map)
void set_original_precision_attribute(const std::shared_ptr<Node> &node, const element::Type_t original_precision)
void reset_original_precision_attribute(const std::shared_ptr<Node> &node)
element::Type_t get_original_precision(const std::shared_ptr<Node> &node)
bool is_preprocesing_node(const std::shared_ptr<Node> &node)
void set_is_preprocessing_node(std::shared_ptr<Node> node)
std::string getPrimitivesPriority(const std::shared_ptr<Node> &node)

getPrimitivesPriority return string with primitive priorities value

Parameters

node[in] The node will be used to get PrimitivesPriority attribute

bool has_strides_prop(const Input<Node> &node)
ov::Strides get_strides_prop(const Input<Node> &node)
void insert_strides_prop(Input<Node> &node, const Strides &strides)
void remove_strides_prop(Input<Node> &node)
void mark_as_no_sinking_node(const std::shared_ptr<Node> &node)
void reset_no_sinking_attribute(const std::shared_ptr<Node> &node)
bool is_sinking_node(const std::shared_ptr<Node> &node)
bool is_sinking_node(const Node *node)
bool is_sinking_node(ov::Output<ov::Node> output)
std::shared_ptr<ov::MappedMemory> load_mmap_object(const std::string &path)

Returns mapped memory for a file from provided path. Instead of reading files, we can map the memory via mmap for Linux in order to avoid time-consuming reading and reduce memory consumption.

Parameters

path – Path to a file which memory will be mmaped.

Returns

MappedMemory shared ptr object which keep mmaped memory and control the lifetime.

template<typename A, typename B>
A copy_from(B &b)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const AxisSet &axis_set)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const AxisVector &axis_vector)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const Coordinate &coordinate)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const CoordinateDiff &coordinate_diff)
OPENVINO_API std::ostream & operator<< (std::ostream &str, const Dimension &dimension)

Insert a human-readable representation of a dimension into an output stream.

Inserts the string ? if dimension is dynamic; else inserts dimension.get_length().

Parameters
  • str – The output stream targeted for insertion.

  • dimension – The dimension to be inserted into str.

Returns

A reference to str after insertion.

template<typename Type, typename Value>
std::enable_if<std::is_convertible<Value, std::string>::value, Type>::type as_enum(const Value &value)

Returns the enum value matching the string.

template<typename Value>
const std::string &as_string(Value value)

Returns the string matching the enum value.

static inline std::ostream &write_all_to_stream(std::ostream &str)
template<typename T, typename ...TS>
std::ostream &write_all_to_stream(std::ostream &str, T &&arg, TS&&... args)
template<class T, typename std::enable_if<!std::is_same<typename std::decay<T>::type, std::string>::value>::type* = nullptr>
std::string stringify(T &&arg)
template<class T, typename std::enable_if<std::is_same<typename std::decay<T>::type, std::string>::value>::type* = nullptr>
T &stringify(T &&arg)
void create_extensions(std::vector<Extension::Ptr>&)

The entry point for library with OpenVINO extensions.

Parameters

vector – of extensions

OPENVINO_API void traverse_nodes (const std::shared_ptr< const Model > &p, const std::function< void(const std::shared_ptr< Node > &)> &f)
OPENVINO_API void traverse_nodes (const Model *p, const std::function< void(const std::shared_ptr< Node > &)> &f)
OPENVINO_API void traverse_nodes (const NodeVector &subgraph_results, const std::function< void(const std::shared_ptr< Node > &)> &f, const NodeVector &subgraph_params={})

Visit each node in a sub-graph of the entire graph.

Traverses a sub-graph starting from subgraph_results moving up towards parameter nodes. Traversal stops if it hits a node in subgraph_params.

Most useful for finding parameters of a graph directly from the result nodes and not from function parameters or extracting a subgraph relevant to the computation of certain outputs

Parameters
  • subgraph_results – The output nodes of the sub-graph

  • fModel to execute at each node in the traversal

  • subgraph_paramsInput nodes of the sub-graph (optional)

OPENVINO_API void replace_node (const std::shared_ptr< Node > &target, const std::shared_ptr< Node > &replacement, const std::vector< int64_t > &output_order)

Replace the node target with the node replacement, i.e., redirect all users and control dependencies of target to replacement.

This is primarily used in graph-rewriting passes. For example, we might “fuse” two Concat operations as follows:

(Step 0: Original graph)

A B | | v v N0[Concat, concatenation_axis=3] C | | v v N1[Concat, concatenation_axis=3] | | v v some_user another_user

(Step 1: Construct replacement)

shared_ptr<Node> new_N1 = make_shared<op::Concat>({A,B,C},3);

A————————————-&#8212;. | | | B————-&#8212;)&#8212;. | | | | v v | | N0[Concat, concatenation_axis=3] C–&#8212;)&#8212;)&#8212;. | | | | | v v v v v N1[Concat, concatenation_axis=3] new_N1[Concat, concatenation_axis=3] | | v v some_user another_user

(Step 2: Replace N1 with new_N1)

replace_node(N1, new_N1);

A————————————-&#8212;. | | | B————-&#8212;)&#8212;. | | | | v v | | N0[Concat, concatenation_axis=3] C–&#8212;)&#8212;)&#8212;. | | | | | v v v v v N1[Concat, concatenation_axis=3] new_N1[Concat, concatenation_axis=3] | | v v some_user another_user

(Step 3: N0 and N1 are now dead, nodes will be freed)

[happens automatically, once all shared_ptrs to N1 are released]

A————————————-&#8212;. | B————-&#8212;)&#8212;. | | | | C–&#8212;)&#8212;)&#8212;. | | | v v v new_N1[Concat, concatenation_axis=3] | | v v some_user another_user

NOTE 1: replace_node is not type-safe (the graph is not revalidated). For example, the following is allowed, even if node some_user requires an input of shape 2x2:

(Before) A(shape=2x2) B(shape=3x3) | v some_user(requires 2x2 input)

(After &#8212; graph is now invalid)

 replace_node(A, B);

 A(shape=2x2)  B(shape=3x3)
               |
               v
            some_user(requires 2x2 input)
NOTE 2: it is possible to insert a cycle into the graph with replace_node, resulting in an invalid graph. Care must be taken to avoid this. One common example is when you are attempting to insert a new node M “after”a nodeN`. For example, you might expect this to work:

shared_ptr<Node> M = make_shared<SomeUnaryOp>(N); replace_node(M, N);

The problem is that at replacement time, N itself is a user of M. So we end up introducing a cycle as follows:

  N
  |
  v
other users…
|||
vvv

 N------------>M
 |
 v
other users…
|||
vvv

         .----.
        |      |
        |      |
 N      `----->M
               |
               v
          other users...
To avoid the cycle, a valid way to perform the above desired insertion would be,
   auto new_N = N->clone_with_new_inputs(N->input_values());
   shared_ptr<Node> M = make_shared<SomeUnaryOp>(new_N);
   replace_node(N, M);

Parameters
  • targetNode to be replaced.

  • replacementNode to replace target with.

  • output_order – Vector determines order of replacement node’s outputs.

OPENVINO_API void replace_node (const std::shared_ptr< Node > &target, const OutputVector &replacement_values)

Replace target.outputs[i] with replacement_values[i] and transfer control dependents and.

OPENVINO_API void replace_node (const std::shared_ptr< Node > &target, const std::shared_ptr< Node > &replacement)
OPENVINO_API void replace_nodes (const std::shared_ptr< Model > &f, const std::unordered_map< std::shared_ptr< op::v0::Parameter >, std::shared_ptr< op::v0::Parameter > > &parameter_replacement_map, const std::unordered_map< std::shared_ptr< Node >, std::shared_ptr< Node > > &body_replacement_map)

Replace multiple nodes in a function.

Limitations:

  • No check is made that the replaced nodes in parameter_replacement_map are actually among the bound parameters of f. (If a parameter appears in the map that is not bound by f, it will be silently ignored.)

  • If a parameter node appears as a key in both parameter_replacement_map and in body_replacement_map, behavior is unspecified.

Parameters
  • fModel where replacement is taking place.

  • parameter_replacement_map – A mapping from parameter shared pointers to parameter shared pointers. For each pair (k,v) in the map, parameter k is replaced by parameter v, except if k==v or k is not a parameter bound by f, in which case the pair (k,v) is ignored.

  • body_replacement_map – A mapping from node shared pointers to node shared pointers. For each pair (k,v) in the map, node k is replaced by node v, except if k==v, the pair (k,v) is ignored. Note that if k is a parameter, its users will be redirected to v, but k will not be replaced in the function’s parameter list.

template<typename T>
std::vector<std::shared_ptr<Node>> topological_sort(T root_nodes)

Topological sort of nodes needed to compute root_nodes.

OPENVINO_API std::shared_ptr< ov::Model > clone_model (const ov::Model &model, std::unordered_map< Node *, std::shared_ptr< Node > > &node_map)
OPENVINO_API std::shared_ptr< ov::Model > clone_model (const ov::Model &model)

input model is cloned and returned

OPENVINO_API bool compare_constants (const std::shared_ptr< Node > &n1, const std::shared_ptr< Node > &n2)
OPENVINO_API bool replace_output_update_name (Output< Node > node, const Output< Node > &node_input)
OPENVINO_API bool replace_node_update_name (const std::shared_ptr< Node > &target, const std::shared_ptr< Node > &replacement)
OPENVINO_API void serialize (const std::shared_ptr< const ov::Model > &m, const std::string &xml_path, const std::string &bin_path="", ov::pass::Serialize::Version version=ov::pass::Serialize::Version::UNSPECIFIED)

Serialize given model into IR. The generated .xml and .bin files will be saved into provided paths. This method serializes model “as-is” that means no weights compression and other possible transformations are applied. It is recommended to use ov::save_model function instead of ov::serialize, because it is aligned with default model conversion flow.

Parameters
  • mModel which will be converted to IR representation.

  • xml_path – Path where .xml file will be saved.

  • bin_path – Path where .bin file will be saved (optional). The same name as for xml_path will be used by default.

  • versionVersion of the generated IR (optional).

OPENVINO_API void save_model (const std::shared_ptr< const ov::Model > &model, const std::string &output_model, bool compress_to_fp16=true)

Save given model into IR. Floating point weights are compressed to FP16 by default. This method saves a model to IR applying all necessary transformations that usually applied in model conversion flow provided by mo tool. Paricularly, floatting point weights are compressed to FP16.

Parameters
  • modelModel which will be converted to IR representation.

  • output_model – Path to the output model file, must have extension .xml

  • compress_to_fp16 – Whether to compress floatting point weights to FP16 (true by default)

OPENVINO_API std::ostream & operator<< (std::ostream &str, const Interval &interval)
std::shared_ptr<Model> clone_ov_model(const Model &func, std::unordered_map<Node*, std::shared_ptr<Node>> &node_map)
OPENVINO_API std::ostream & operator<< (std::ostream &, const Model &)
OPENVINO_API ov::Dimension get_batch (const std::shared_ptr< const ov::Model > &f)

Helper method to get associated batch size for a Model.

Checks layout of each parameter in a Model and extracts value for N (B) dimension. All values are then merged and returned

Throws

ov::AssertFailure – with details in case of error. Possible errors are:

  • There is no parameter with layout set. Model shall have at least one parameter with layout with ‘N’ dimension. Recommended fix is to use Parameter::set_layout API, e.g. model->get_parameters()[some_index]->set_layout("NCHW");

  • Several parameters have conflicting N dimension, e.g. param1 NCHW{1,3,224,224} and param2 NCHW{2,3,224,224}. This is ambiguous, most probably first dimension is incorrectly marked as ‘batch’ (N) in some layout. User shall fix it before using of ‘get_batch’ (in example above correct layout for param2 from ‘NCHW’ to ‘CHWN’)

Parameters

fModel where to look for a batch_size value

Returns

Dimension representing current batch size. Can represent a number or be a dynamic

OPENVINO_API void set_batch (const std::shared_ptr< ov::Model > &model, ov::Dimension batch_size)

Helper method to set batch size to a Model.

Checks layout of each parameter in a Model and sets value for N (B) dimension. Then performs validation and type propagation

Throws

ov::AssertFailure – with details in case of error. Possible errors are:

  • There is no parameter with N dimension in layout. Model shall have at least one parameter with layout with ‘N’ dimension. Recommended fix is to use Parameter::set_layout API, e.g. model->get_parameters()[some_index]->set_layout("NCHW");

  • Several parameters have conflicting N dimension, e.g. param1 NCHW{1,3,224,224} and param2 NCHW{3,224,224,1}. This is ambiguous (1 != 3), most probably some dimension is incorrectly marked as ‘batch’ (N) in some layout. User shall fix it before using of ‘set_batch’ (in example above correct layout for param2 from ‘NCHW’ to ‘CHWN’)

  • Validation fails after setting batch_size. Model becomes in inconsistent state after new batch size value is applied. Possible reason could be that layout was not set for some parameters, or batch size can’t be applied to model at all

Parameters
  • model – model where to set batch_size value

  • batch_size – Batch size value. For dynamic batch size, Dimension::dynamic() can be passed.

OPENVINO_API std::string node_validation_failure_loc_string (const Node *node)
OPENVINO_API std::ostream & operator<< (std::ostream &, const Node &)
OPENVINO_API std::ostream & operator<< (std::ostream &, const Node *)
void OPENVINO_API check_new_args_count (const Node *const node, const OutputVector &new_args)

Check new arguments size if match node inputs count.

This check is required in cloning ov::Node.

Parameters
  • node – Pointer to node.

  • new_args – Vector with new outputs to check.

OPENVINO_API std::ostream & operator<< (std::ostream &out, const Input< Node > &input)
OPENVINO_API std::ostream & operator<< (std::ostream &out, const Input< const Node > &input)
OPENVINO_API std::ostream & operator<< (std::ostream &out, const Output< Node > &output)
OPENVINO_API std::ostream & operator<< (std::ostream &out, const Output< const Node > &output)
OPENVINO_API OutputVector as_output_vector (const NodeVector &args)
OPENVINO_API NodeVector as_node_vector (const OutputVector &values)
OPENVINO_API ResultVector as_result_vector (const OutputVector &values)

Returns a ResultVector referencing values.

OPENVINO_API PartialShape operator+ (const PartialShape &s1, const PartialShape &s2)

Elementwise addition of two PartialShape objects.

  • If s1 or s2 has dynamic rank, returns PartialShape::dynamic().

  • If s1 ands2` both have static rank, and their ranks are unequal, throws std::invalid_argument.

  • If s1 and s2 both have static rank, and their ranks are equal, returns a new shape whose ith dimension is s1[i] + s2[i].

Parameters
  • s1 – Left operand for addition.

  • s2 – Right operand for addition.

Throws

std::invalid_argument – If s1 and s2 have inconsistent ranks.

Returns

The result of elementwise adding s1 to s2 (see description).

OPENVINO_API std::ostream & operator<< (std::ostream &str, const PartialShape &shape)

Inserts a human-readable representation of a PartialShape into an output stream.

The output to the stream is in “informal” notation. In other words:

  • If shape has dynamic rank, inserts the string ?.

  • If shape has static rank, inserts the string {, then inserts each dimension of shape into the output stream separated by commas, then inserts }.

Example:

PartialShape s1{PartialShape::dynamic())};
PartialShape s2{};
PartialShape s3{1,Dimension::dynamic(),2,3};
PartialShape s4{2,3,4};
std::cout << s1 << std::endl
          << s2 << std::endl
          << s3 << std::endl
          << s4 << std::endl;

Output:

?
{}
{1,?,2,3}
{2,3,4}
Parameters
  • str – The output stream targeted for insertion.

  • shape – The shape to be inserted into str.

Returns

A reference to str after insertion.

OPENVINO_API void copy_runtime_info (const std::shared_ptr< ov::Node > &from, const std::shared_ptr< ov::Node > &to)
OPENVINO_API void copy_runtime_info (const std::shared_ptr< ov::Node > &from, ov::NodeVector to)
OPENVINO_API void copy_runtime_info (const ov::NodeVector &from, const std::shared_ptr< ov::Node > &to)
OPENVINO_API void copy_runtime_info (const ov::NodeVector &from, ov::NodeVector to)
OPENVINO_API void copy_output_runtime_info (const ov::OutputVector &from, ov::OutputVector to)
OPENVINO_API std::ostream & operator<< (std::ostream &os, const RuntimeAttribute &attrubute)
template<typename ForwardIt>
size_t shape_size(ForwardIt start_dim, const ForwardIt end_dim)

Number of elements in a subset of dimensions of a shape. Returns a product of dimensions in a range [start_dim;end_dim)

template<typename SHAPE_TYPE>
size_t shape_size(const SHAPE_TYPE &shape)

Number of elements in spanned by a shape.

template<typename SHAPE_TYPE>
std::vector<size_t> row_major_strides(const SHAPE_TYPE &shape)

Row-major strides for a shape.

template<typename SHAPE_TYPE>
size_t row_major_stride(const SHAPE_TYPE &shape, size_t axis)
template<typename SHAPE_TYPE>
inline bool is_scalar(const SHAPE_TYPE &shape)
template<typename SHAPE_TYPE>
inline bool is_vector(const SHAPE_TYPE &shape)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const Shape &shape)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const Strides &strides)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const DiscreteTypeInfo &info)
bool ::type is_type (Value value)
Type *::type as_type (Value value)
template<typename T, typename U>
auto as_type_ptr(const U &value) -> decltype(::ov::util::AsTypePtr<U>::template call<T>(value))

Casts a std::shared_ptr<Value> to a std::shared_ptr<Type> if it is of type Type, nullptr otherwise

OPENVINO_API PartialShape infer_convolution_forward (const Node *node, const PartialShape &data_batch_shape, const Strides &data_dilation, const CoordinateDiff &data_padding_below, const CoordinateDiff &data_padding_above, const PartialShape &filters_shape, const Strides &filter_strides, const Strides &filter_dilation)
OPENVINO_API void infer_auto_padding (const Shape &image_shape, const Shape &filter_shape, const Strides &filter_strides, const Strides &filter_dilations, const op::PadType pad_type, CoordinateDiff &padding_above, CoordinateDiff &padding_below)
OPENVINO_API int64_t normalize_axis (const Node *node, std::int64_t axis, const Rank &tensor_rank)

Handle out of range axis.

Parameters
  • node[in] The node with requested axis.

  • axis[in] The requested axis value.

  • tensor_rank[in] The corresponding tensor rank.

Returns

Checking if axis is in range [-tensor_rank, tensor_rank-1], otherwise returns error. If negative axis, it counts from the last to the first axis, by adding tensor_rank to axis.

OPENVINO_API std::vector< size_t > normalize_axes (const std::string &node_description, const std::vector< int64_t > &axes, const Rank &tensor_rank)

Handle out of range axes in vector.

Parameters
  • node_description[in] The name of node with requested axes.

  • axes[in] The requested vector of axes.

  • tensor_rank[in] The corresponding tensor rank.

Returns

If any negative axis in vector, it counts from the last to the first axis, by adding tensor_rank to axis.

OPENVINO_API int64_t normalize_axis (const std::string &node_description, std::int64_t axis, const Rank &tensor_rank)

Handle out of range axis.

Parameters
  • node_description[in] The node with requested axis.

  • axis[in] The requested axis value.

  • tensor_rank[in] The corresponding tensor rank.

Returns

Checking if axis is in range [-tensor_rank, tensor_rank-1], otherwise returns error. If negative axis, it counts from the last to the first axis, by adding tensor_rank to axis.

OPENVINO_API int64_t normalize_axis (const Node *node, std::int64_t axis, std::uint64_t tensor_rank, std::int64_t axis_range_min, std::int64_t axis_range_max)

Handle out of range axis.

Parameters
  • node[in] The node with requested axis.

  • axis[in] The requested axis value.

  • tensor_rank[in] The corresponding tensor rank.

  • axis_range_min[in] The min value of accepted range for axis.

  • axis_range_max[in] The max value of accepted range for axis.

Returns

Checking if axis is in range [axis_range_min, axis_range_max], otherwise returns error. If negative axis, it counts from the last to the first axis, by adding tensor_rank to axis.

OPENVINO_API int64_t normalize_axis (const std::string &node_description, std::int64_t axis, std::uint64_t tensor_rank, std::int64_t axis_range_min, std::int64_t axis_range_max)

Handle out of range axis.

Parameters
  • node_description[in] The name of node with requested axis.

  • axis[in] The requested axis value.

  • tensor_rank[in] The corresponding tensor rank.

  • axis_range_min[in] The min value of accepted range for axis.

  • axis_range_max[in] The max value of accepted range for axis.

Returns

Checking if axis is in range [axis_range_min, axis_range_max], otherwise returns error. If negative axis, it counts from the last to the first axis, by adding tensor_rank to axis.

OPENVINO_API void normalize_axes (const Node *node, const int64_t &tensor_rank, std::vector< int64_t > &axes)

Handle out of range axes in vector. If any negative axis in vector, it counts from the last to the first axis, by adding tensor_rank to axis. Changes axes vector inplace.

Parameters
  • node[in] The node with requested axes.

  • tensor_rank[in] The corresponding tensor rank.

  • axes[inout] The requested vector of axes.

OPENVINO_API bool evaluate_as_partial_shape (const Output< Node > &output, PartialShape &pshape)

Evaluates lower and upper value estimations for the output tensor. Estimation would be represented as partial shape object using Dimension(min, max) for each element.

Parameters
  • outputNode output pointing to the tensor for estimation.

  • pshape – Resulting estimation would be stored in this PartialShape.

Returns

boolean status if value evaluation was successful.

OPENVINO_API std::shared_ptr< op::v0::Constant > get_constant_from_source (const Output< Node > &source)

Runs an estimation of source tensor. If it succeeded to calculate both bounds and they are the same returns Constant operation from the resulting bound, otherwise nullptr.

OPENVINO_API bool default_label_evaluator (const Node *node, TensorLabelVector &output_labels)

Propagates value label from 0 input to the only output through an operation. Not applicable for operations which require values interaction (example: mathematical operations). Could be used for movement operations (example: gathering, shape change)

Parameters
  • node – Operation to be performed

  • output_labels – Vector of TensorLabel objects representing resulting value labels

Returns

boolean status if label evaluation was successful.

OPENVINO_API void generate_transpose_default_order (std::vector< int64_t > &axes_order, const size_t length)

Generates transpose default axes order at end of input vector.

Default axes order is decreasing sequence numbers which start from length - 1.

Parameters
  • axes_order – Vector where default order will be generated.

  • length – Sequence length of axes order.

OPENVINO_API bool is_valid_axes_order (const std::vector< int64_t > &axes_order, const size_t size)

Check if vector of axes order has got valid values.

Axes order has to be unique numbers in range of [0, size).

Parameters
  • axes_order – Vector with axes order to check.

  • sizeInput for transpose rank size.

Returns

true if axes order is valid otherwise false.

OPENVINO_API bool has_no_labels (const TensorLabel &labels)

Checks label tensor if there is no label.

Parameters

labels – Label tensor for check.

Returns

True if there is no labels, otherwise false.

OPENVINO_API std::ostream & operator<< (std::ostream &s, const Version &version)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const std::map< std::string, Version > &versions)
OPENVINO_API_C (const Version) get_openvino_version() noexcept

Gets the current OpenVINO version.

Returns

The current OpenVINO version

OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v1::BinaryConvolution::BinaryConvolutionMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v0::DepthToSpace::DepthToSpaceMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v9::GridSample::InterpolationMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v9::GridSample::PaddingMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v0::Interpolate::InterpolateMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::LSTMWeightsFormat &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v8::MatrixNms::DecayFunction &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v8::MatrixNms::SortResultType &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v1::NonMaxSuppression::BoxEncodingType &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v3::NonMaxSuppression::BoxEncodingType &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v5::NonMaxSuppression::BoxEncodingType &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v9::NonMaxSuppression::BoxEncodingType &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v1::Reverse::Mode &type)
std::ostream &operator<<(std::ostream &s, const op::v3::ROIAlign::PoolingMode &mode)
std::ostream &operator<<(std::ostream &s, const op::v9::ROIAlign::PoolingMode &mode)
std::ostream &operator<<(std::ostream &s, const op::v9::ROIAlign::AlignedMode &mode)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v5::Round::RoundMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v0::SpaceToDepth::SpaceToDepthMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::util::InterpolateBase::InterpolateMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::util::InterpolateBase::CoordinateTransformMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::util::InterpolateBase::NearestMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::util::InterpolateBase::ShapeCalcMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::util::MulticlassNmsBase::SortResultType &type)
void OPENVINO_API mark_as_precision_sensitive (ov::Input< ov::Node > node_input)
void OPENVINO_API unmark_as_precision_sensitive (ov::Input< ov::Node > node_input)
bool OPENVINO_API is_precision_sensitive (const ov::Input< ov::Node > &node_input)
OPENVINO_API void set_up_symbolic_info (const std::shared_ptr< ov::Model > &model, const std::shared_ptr< ov::TableOfEquivalence > &table)
OPENVINO_API void set_up_symbolic_info (const ov::Output< ov::Node > &output, const std::shared_ptr< ov::TableOfEquivalence > &table)
OPENVINO_API void populate_tensor_with_missing_labels (ov::descriptor::Tensor &tensor)
OPENVINO_API bool skip_invalidation (const ov::descriptor::Tensor &tensor)
OPENVINO_API std::shared_ptr< ov::TableOfEquivalence > table_of_equivalence (const std::shared_ptr< ov::Model > &model)
OPENVINO_API std::shared_ptr< ov::TableOfEquivalence > table_of_equivalence (const ov::descriptor::Tensor &tensor)
OPENVINO_API void remove_symbolic_info (const std::shared_ptr< ov::Model > &model, bool outermost_model=true)
const OPENVINO_API OpSet & get_opset1 ()

Returns opset1.

const OPENVINO_API OpSet & get_opset2 ()

Returns opset2.

const OPENVINO_API OpSet & get_opset3 ()

Returns opset3.

const OPENVINO_API OpSet & get_opset4 ()

Returns opset4.

const OPENVINO_API OpSet & get_opset5 ()

Returns opset5.

const OPENVINO_API OpSet & get_opset6 ()

Returns opset6.

const OPENVINO_API OpSet & get_opset7 ()

Returns opset7.

const OPENVINO_API OpSet & get_opset8 ()

Returns opset8.

const OPENVINO_API OpSet & get_opset9 ()

Returns opset9.

const OPENVINO_API OpSet & get_opset10 ()

Returns opset10.

const OPENVINO_API OpSet & get_opset11 ()

Returns opset11.

const OPENVINO_API OpSet & get_opset12 ()

Returns opset12.

const OPENVINO_API OpSet & get_opset13 ()

Returns opset13.

const OPENVINO_API std::map< std::string, std::function< const ov::OpSet &()> > & get_available_opsets ()

Returns map of available opsets.

std::size_t coordinate_index(const Coordinate &c, const Shape &s)
size_t coordinate_offset(const std::vector<size_t> &coordinate, const std::vector<size_t> &strides)

Calculate offset from begin of buffer based on coordinate and strides.

If coordinates and strides have different sizes then result is undefined behaviour.

Parameters
  • coordinate – Vector with multi-dimension coordinates.

  • strides – Vector with multi-dimension strides

Returns

Offset of element from start of buffer.

template<class T>
constexpr bool is_floating_point()

Check if T is OpenVINO floating point precision.

Returns

True if OpenVino floating point precision.

OV_ITT_DOMAIN(OV_PP_CAT(TYPE_LIST_, ov_eval))
template<class TContainer>
constexpr auto make_tensor_accessor(const TContainer &c) -> TensorAccessor<TContainer>

Makes TensorAccessor for specific tensor container.

See also

TensorAccessor for supported types.

Template Parameters

TContainer – Type of tensor containers

Parameters

c – Container of tensors.

Returns

TensorContainer for specific type.

auto make_tensor_accessor() -> const TensorAccessor<void>&

Makes empty TensorAccessor which return empty tensor for any port number.

Returns

TensorAccessor to return empty tensor.

template<class T, class TResult = std::vector<T>, class UnaryOperation>
TResult get_raw_data_as(const element::Type_t et, const void *const ptr, const size_t size, UnaryOperation &&func)

Get the raw data as TResult object.

Template Parameters
  • T – TResult data type.

  • TResult – Type of return object, must support creation of std::inserter. Default std::vector<T>.

  • UnaryOperation – Unary function object applied on data with signature (T f(const U u)).

Parameters
  • et – Element type of input data.

  • ptr – Pointer to data of type et.

  • size – Data size as number of elements.

  • func – Unary operation function object.

Throws

ov::AssertionFailure – for not supported element type.

Returns

Object of TResult with data from input pointer and transformed by unary operation.

template<class T, class TResult = std::vector<T>, class UnaryOperation = ov::util::Cast<T>>
TResult get_tensor_data_as(const Tensor &t, UnaryOperation &&func = ov::util::Cast<T>())

Get data from ov:tensor as object TResult.

Template Parameters
  • T – TResult data type.

  • TResult – Type of return object, must support creation of std::inserter. Default std::vector<T>.

  • UnaryOperation – Unary function object applied on data with signature (T f(const U u)).

Parameters
  • tInput tensor.

  • func – Unary operation function object.

Returns

Object of TResult with data from tensor.

FRONTEND_API void shutdown ()

Shut down the OpenVINO by deleting all static-duration objects allocated by the library and releasing dependent resources.

You might want to use this function if you are developing a dynamically-loaded library which should clean up all resources after itself when the library is unloaded.

Note

This function should be used by advanced user to control unload the resources.

std::unordered_set<std::string> get_supported_nodes(const std::shared_ptr<const ov::Model> &model, std::function<void(std::shared_ptr<ov::Model>&)> transform, std::function<bool(const std::shared_ptr<ov::Node>)> is_node_supported)

Returns set of nodes from original model which are determined as supported after applied transformation pipeline.

Parameters
  • model – Original model

  • transform – Transformation pipeline function

  • is_node_supported – Function returning whether node is supported or not

Returns

Set of strings which contains supported node names

std::shared_ptr<ITensor> make_tensor(const element::Type type, const Shape &shape, const Allocator &allocator = {})

Constructs Tensor using element type and shape. Allocate internal host storage using default allocator.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • allocator – allocates memory for internal tensor storage

std::shared_ptr<ITensor> make_tensor(const element::Type type, const Shape &shape, void *host_ptr, const Strides &strides = {})

Constructs Tensor using element type and shape. Wraps allocated host memory.

Note

Does not perform memory allocation internally

Parameters
  • typeTensor element type

  • shapeTensor shape

  • host_ptr – Pointer to pre-allocated host memory

  • strides – Optional strides parameters in bytes. Strides are supposed to be computed automatically based on shape and element size

std::shared_ptr<ITensor> make_tensor(const std::shared_ptr<ITensor> &other, const Coordinate &begin, const Coordinate &end)

Constructs region of interest (ROI) tensor form another tensor.

Note

Does not perform memory allocation internally

Note

A Number of dimensions in begin and end must match number of dimensions in other.get_shape()

Parameters
  • other – original tensor

  • begin – start coordinate of ROI object inside of the original object.

  • end – end coordinate of ROI object inside of the original object.

ov::Tensor make_tensor(const ov::SoPtr<ITensor> &tensor)

Constructs public ov::Tensor class.

Parameters

tensorTensor implementation

Returns

OpenVINO Tensor

bool check_open_mp_env_vars(bool include_omp_num_threads = true)

Checks whether OpenMP environment variables are defined.

Parameters

include_omp_num_threads[in] Indicates if the omp number threads is included

Returns

True if any OpenMP environment variable is defined, false otherwise

std::vector<int> get_available_numa_nodes()

Returns available CPU NUMA nodes (on Linux, and Windows [only with TBB], single node is assumed on all other OSes)

Returns

NUMA nodes

std::vector<int> get_available_cores_types()

Returns available CPU cores types (on Linux, and Windows) and ONLY with TBB, single core type is assumed otherwise.

Returns

Vector of core types

int get_number_of_cpu_cores(bool big_cores_only = false)

Returns number of CPU physical cores on Linux/Windows (which is considered to be more performance friendly for servers) (on other OSes it simply relies on the original parallel API of choice, which usually uses the logical cores). call function with ‘false’ to get #phys cores of all types call function with ‘true’ to get #phys ‘Big’ cores number of ‘Little’ = ‘all’ - ‘Big’.

Parameters

big_cores_only[in] Additionally limits the number of reported cores to the ‘Big’ cores only.

Returns

Number of physical CPU cores.

int get_number_of_logical_cpu_cores(bool big_cores_only = false)

Returns number of CPU logical cores on Linux/Windows (on other OSes it simply relies on the original parallel API of choice, which uses the ‘all’ logical cores). call function with ‘false’ to get #logical cores of all types call function with ‘true’ to get #logical ‘Big’ cores number of ‘Little’ = ‘all’ - ‘Big’.

Parameters

big_cores_only[in] Additionally limits the number of reported cores to the ‘Big’ cores only.

Returns

Number of logical CPU cores.

int get_number_of_blocked_cores()

Returns number of blocked CPU cores. Please note that this is a temporary interface for performance optimization on a specific platform. May be removed in future release.

Returns

Number of blocked CPU cores.

bool with_cpu_x86_sse42()

Checks whether CPU supports SSE 4.2 capability.

Returns

True is SSE 4.2 instructions are available, false otherwise

bool with_cpu_x86_avx()

Checks whether CPU supports AVX capability.

Returns

True is AVX instructions are available, false otherwise

bool with_cpu_x86_avx2()

Checks whether CPU supports AVX2 capability.

Returns

True is AVX2 instructions are available, false otherwise

bool with_cpu_x86_avx2_vnni()

Checks whether CPU supports AVX2_VNNI capability.

Returns

True is AVX2_VNNI instructions are available, false otherwise

bool with_cpu_x86_avx512f()

Checks whether CPU supports AVX 512 capability.

Returns

True is AVX512F (foundation) instructions are available, false otherwise

bool with_cpu_x86_avx512_core()

Checks whether CPU supports AVX 512 capability.

Returns

True is AVX512F, AVX512BW, AVX512DQ instructions are available, false otherwise

bool with_cpu_x86_avx512_core_vnni()

Checks whether CPU supports AVX 512 VNNI capability.

Returns

True is AVX512F, AVX512BW, AVX512DQ, AVX512_VNNI instructions are available, false otherwise

bool with_cpu_x86_bfloat16()

Checks whether CPU supports BFloat16 capability.

Returns

True is tAVX512_BF16 instructions are available, false otherwise

bool with_cpu_x86_avx512_core_fp16()

Checks whether CPU supports fp16 capability.

Returns

True is tAVX512_FP16 instructions are available, false otherwise

bool with_cpu_x86_avx512_core_amx_int8()

Checks whether CPU supports AMX int8 capability.

Returns

True is tAMX_INT8 instructions are available, false otherwise

bool with_cpu_x86_avx512_core_amx_bf16()

Checks whether CPU supports AMX bf16 capability.

Returns

True is tAMX_BF16 instructions are available, false otherwise

bool with_cpu_x86_avx512_core_amx()

Checks whether CPU supports AMX capability.

Returns

True is tAMX_INT8 or tAMX_BF16 instructions are available, false otherwise

bool is_cpu_map_available()

Checks whether cpu_mapping Available.

Returns

True is CPU mapping is available, false otherwise

int get_num_numa_nodes()

Get number of numa nodes.

Returns

Number of numa nodes

int get_num_sockets()

Get number of sockets.

Returns

Number of sockets

std::vector<std::vector<int>> get_proc_type_table()

Returns a table of number of processor types on Linux/Windows.

  1. Processor table of one socket CPU desktop ALL_PROC | MAIN_CORE_PROC | EFFICIENT_CORE_PROC | HYPER_THREADING_PROC 32 8 16 8 // Total number of one socket

Returns

A table about number of CPU cores of different types defined with ColumnOfProcessorTypeTable The following are two example of processor type table.

  1. Processor table of two socket CPUs XEON server ALL_PROC | MAIN_CORE_PROC | EFFICIENT_CORE_PROC | HYPER_THREADING_PROC 96 48 0 48 // Total number of two sockets 48 24 0 24 // Number of socket one 48 24 0 24 // Number of socket two

std::vector<std::vector<int>> get_org_proc_type_table()

Returns a table of original number of processor types without filtering other plugins occupying CPU resources. The difference from get_proc_type_table: This is used to get the configuration of current machine. For example, GPU plugin occupies all Pcores, there is only one type core in proc_type_table from get_proc_type_table(). If user wants to get the real configuration of this machine which should be got from get_org_proc_type_table.

Returns

A table about number of CPU cores of different types defined with ColumnOfProcessorTypeTable

void reserve_available_cpus(const std::vector<std::vector<int>> streams_info_table, std::vector<std::vector<int>> &stream_processors, const int cpu_status = NOT_USED)

Get and reserve available cpu ids.

Parameters
  • streams_info_table[in] streams information table.

  • stream_processors[in] processors grouped in stream which is used in core binding in cpu streams executor

  • cpu_status[in] set cpu status

void set_cpu_used(const std::vector<int> &cpu_ids, const int used)

Set CPU_MAP_USED_FLAG of cpu_mapping.

Parameters
  • cpu_ids[in] cpus in cpu_mapping.

  • used[in] update CPU_MAP_USED_FLAG of cpu_mapping with this flag bit

int get_socket_by_numa_node(int numa_node_id)

Get socket id by current numa node id.

Parameters

numa_node_id[in] numa node id

Returns

socket id

int get_org_socket_id(int socket_id)

Get original socket id by current socket id, the input socket id is recalculated after filtering (like numactl), while the original socket id is the original id before filtering.

Parameters

socket_id[in] socket id

Returns

socket id

int get_org_numa_id(int numa_node_id)

Get original numa node id by current numa node id, the input numa node id is recalculated after filtering (like numactl), while the original numa node id is the original id before filtering.

Parameters

numa_node_id[in] numa node id

Returns

numa node id

static MemBandwidthPressure MemBandwidthPressureTolerance(const std::shared_ptr<ov::Model> model, const float cache_size, const float memThresholdAssumeLimited = MemBandwidthPressure::LIMITED)
class Allocator
#include <allocator.hpp>

Wraps allocator implementation to provide safe way to store allocater loaded from shared library And constructs default based on new delete c++ calls allocator if created without parameters Accepts any std::pmr::memory_resource like allocator.

Public Functions

~Allocator()

Destructor preserves unloading order of implementation object and reference to library.

Allocator()

Default constructor.

Allocator(const Allocator &other) = default

Default copy constructor.

Parameters

other – other Allocator object

Allocator &operator=(const Allocator &other) = default

Default copy assignment operator.

Parameters

other – other Allocator object

Returns

reference to the current object

Allocator(Allocator &&other) = default

Default move constructor.

Parameters

other – other Allocator object

Allocator &operator=(Allocator &&other) = default

Default move assignment operator.

Parameters

other – other Allocator object

Returns

reference to the current object

OPENVINO_SUPPRESS_DEPRECATED_START Allocator(const AllocatorImpl::Ptr &impl)

Constructs Allocator from the initialized std::shared_ptr.

Parameters

impl – Initialized shared pointer

template<typename A, typename std::enable_if<!std::is_convertible<A, AllocatorImpl::Ptr>::value && !std::is_same<typename std::decay<A>::type, Allocator>::value && !std::is_abstract<typename std::decay<A>::type>::value && !std::is_convertible<typename std::decay<A>::type, std::shared_ptr<Base>>::value, bool>::type = true>
inline Allocator(A &&a)

Initialize allocator using any allocator like object.

Template Parameters

A – Type of allocator

Parameters

a – allocator object

OPENVINO_SUPPRESS_DEPRECATED_END void * allocate (const size_t bytes, const size_t alignment=alignof(max_align_t))

Allocates memory.

Parameters
  • bytes – The size in bytes at least to allocate

  • alignment – The alignment of storage

Throws

Exception – if specified size and alignment is not supported

Returns

Handle to the allocated resource

void deallocate(void *ptr, const size_t bytes = 0, const size_t alignment = alignof(max_align_t))

Releases the handle and all associated memory resources which invalidates the handle.

Parameters
  • ptr – The handle to free

  • bytes – The size in bytes that was passed into allocate() method

  • alignment – The alignment of storage that was passed into allocate() method

bool operator==(const Allocator &other) const

Compares with other Allocator.

Parameters

other – Other instance of allocator

Returns

true if and only if memory allocated from one Allocator can be deallocated from the other and vice versa

bool operator!() const noexcept

Checks if current Allocator object is not initialized.

Returns

true if current Allocator object is not initialized, false - otherwise

explicit operator bool() const noexcept

Checks if current Allocator object is initialized.

Returns

true if current Allocator object is initialized, false - otherwise

interface AllocatorImpl : public std::enable_shared_from_this<AllocatorImpl>
#include <allocator.hpp>

Tries to act like std::pmr::memory_resource

Deprecated:

This class will be removed in 2024.0 release

Public Types

using Ptr = std::shared_ptr<AllocatorImpl>

A smart pointer containing AllocatorImpl object.

Public Functions

virtual void *allocate(const size_t bytes, const size_t alignment = alignof(max_align_t)) = 0

Allocates memory.

Parameters
  • bytes – The size in bytes at least to allocate

  • alignment – The alignment of storage

Throws

Exception – if specified size and alignment is not supported

Returns

Handle to the allocated resource

virtual void deallocate(void *handle, const size_t bytes, size_t alignment = alignof(max_align_t)) = 0

Releases the handle and all associated memory resources which invalidates the handle.

Parameters
  • handle – The handle to free

  • bytes – The size in bytes that was passed into allocate() method

  • alignment – The alignment of storage that was passed into allocate() method

virtual bool is_equal(const AllocatorImpl &other) const = 0

Compares with other AllocatorImpl.

Parameters

other – Other instance of allocator

Returns

true if and only if memory allocated from one AllocatorImpl can be deallocated from the other and vice versa

class Any
#include <any.hpp>

This class represents an object to work with different types.

Public Functions

Any() = default

Default constructor.

Any(const Any &other)

Сopy constructor.

Parameters

other – other Any object

Any &operator=(const Any &other)

Сopy assignment operator.

Parameters

other – other Any object

Returns

reference to the current object

Any(Any &&other) = default

Default move constructor.

Parameters

other – other Any object

Any &operator=(Any &&other) = default

Default move assignment operator.

Parameters

other – other Any object

Returns

reference to the current object

~Any()

Destructor preserves unloading order of implementation object and reference to library.

template<typename T, typename std::enable_if<!std::is_same<decay_t<T>, Any>::value && !std::is_abstract<decay_t<T>>::value && !std::is_convertible<decay_t<T>, Base::Ptr>::value, bool>::type = true>
inline Any(T &&value)

Constructor creates any with object.

Template Parameters

TAny type

Parameters

value – object

Any(const char *str)

Constructor creates string any from char *.

Parameters

str – char array

Any(const std::nullptr_t)

Empty constructor.

const std::type_info &type_info() const

Returns type info

Returns

type info

bool empty() const

Checks that any contains a value

Returns

false if any contains a value else false

template<class T>
inline bool is() const

Check that stored type can be casted to specified type. If internal type supports Base.

Template Parameters

T – Type of value

Returns

true if type of value is correct. Return false if any is empty

template<class T>
inline std::enable_if<std::is_convertible<T, std::shared_ptr<RuntimeAttribute>>::value, T>::type &as()

Dynamic cast to specified type

Template Parameters

T – type

Returns

casted object

template<class T>
inline std::enable_if<!std::is_convertible<T, std::shared_ptr<RuntimeAttribute>>::value && !std::is_same<T, std::string>::value && std::is_default_constructible<T>::value && (util::Istreamable<T>::value || util::Readable<T>::value), T>::type &as()

Dynamic cast to specified type

Template Parameters

T – type

Returns

casted object

template<class T>
inline std::enable_if<!std::is_convertible<T, std::shared_ptr<RuntimeAttribute>>::value && !std::is_same<T, std::string>::value && (!std::is_default_constructible<T>::value || (!util::Istreamable<T>::value && !util::Readable<T>::value)), T>::type &as()

Dynamic cast to specified type

Template Parameters

T – type

Returns

casted object

template<class T>
inline std::enable_if<std::is_same<T, std::string>::value, T>::type &as()

Dynamic cast to specified type

Template Parameters

T – type

Returns

casted object

template<class T>
inline const T &as() const

Dynamic cast to specified type

Template Parameters

T – type

Returns

const reference to caster object

bool operator==(const Any &other) const

The comparison operator for the Any.

Parameters

other – object to compare

Returns

true if objects are equal

bool operator==(const std::nullptr_t&) const

The comparison operator for the Any.

Parameters

other – object to compare

Returns

true if objects are equal

bool operator!=(const Any &other) const

The comparison operator for the Any.

Parameters

other – object to compare

Returns

true if objects aren’t equal

void print(std::ostream &stream) const

Prints underlying object to the given output stream. Uses operator<< if it is defined, leaves stream unchanged otherwise. In case of empty any or nullptr stream immediately returns.

Parameters

streamOutput stream object will be printed to.

void read(std::istream &stream)

Read into underlying object from the given input stream. Uses operator>> if it is defined, leaves stream unchanged otherwise. In case of empty any or nullptr stream immediately returns.

Parameters

streamOutput stream object will be printed to.

void *addressof()

Returns address to internal value if any is not empty and nullptr instead.

Returns

address to internal stored value

const void *addressof() const

Returns address to internal value if any is not empty and nullptr instead.

Returns

address to internal stored value

Public Static Functions

template<typename T, typename ...Args>
static inline Any make(Args&&... args)

Inplace value construction function.

Template Parameters
  • TAny type

  • Args – pack of paramter types passed to T constructor

Parameters

args – pack of paramters passed to T constructor

class AssertFailure : public ov::Exception
#include <except.hpp>

Base class for check failure exceptions.

Subclassed by ov::NodeValidationFailure, ov::NotImplemented, ov::frontend::GeneralFailure, ov::frontend::InitializationFailure, ov::frontend::NotImplementedFailure, ov::frontend::OpConversionFailure, ov::frontend::OpValidationFailure

template<typename AT>
class AttributeAdapter
#include <attribute_adapter.hpp>

An AttributeAdapter “captures” an attribute as an AT& and makes it available as a ValueAccessor<VAT>.

template<>
class AttributeAdapter<AxisVector> : public ov::IndirectVectorValueAccessor<AxisVector, std::vector<int64_t>>
#include <axis_vector.hpp>
template<>
class AttributeAdapter<bool> : public ov::DirectValueAccessor<bool>
#include <attribute_adapter.hpp>

Access a bool as a bool.

template<>
class AttributeAdapter<Coordinate> : public ov::IndirectVectorValueAccessor<Coordinate, std::vector<int64_t>>
#include <coordinate.hpp>
template<>
class AttributeAdapter<CoordinateDiff> : public ov::IndirectVectorValueAccessor<CoordinateDiff, std::vector<int64_t>>
#include <coordinate_diff.hpp>
template<>
class AttributeAdapter<double> : public ov::DirectValueAccessor<double>
#include <attribute_adapter.hpp>

Access a double as a double.

template<>
class AttributeAdapter<float> : public ov::IndirectScalarValueAccessor<float, double>
#include <attribute_adapter.hpp>
template<>
class AttributeAdapter<int16_t> : public ov::IndirectScalarValueAccessor<int16_t, int64_t>
#include <attribute_adapter.hpp>

Access an int16_t as an int64_t.

template<>
class AttributeAdapter<int32_t> : public ov::IndirectScalarValueAccessor<int32_t, int64_t>
#include <attribute_adapter.hpp>

Access an int32_t as an int64_t.

template<>
class AttributeAdapter<int64_t> : public ov::DirectValueAccessor<int64_t>
#include <attribute_adapter.hpp>

Access an int64_t as an int64_t.

template<>
class AttributeAdapter<int8_t> : public ov::IndirectScalarValueAccessor<int8_t, int64_t>
#include <attribute_adapter.hpp>

Access an int8_t and an int64_t.

template<>
class AttributeAdapter<Layout> : public ov::ValueAccessor<std::string>
#include <layout.hpp>

Public Functions

virtual const std::string &get() override

Returns the value.

virtual void set(const std::string &value) override

Sets the value.

template<> Type > : public ov::EnumAttributeAdapterBase< ngraph::reduction::Type >
#include <distributed.hpp>
template<> AutoBroadcastSpec > : public ov::VisitorAdapter
#include <attr_types.hpp>
template<> AutoBroadcastType > : public ov::EnumAttributeAdapterBase< op::AutoBroadcastType >
#include <attr_types.hpp>
template<> BroadcastModeSpec > : public ov::VisitorAdapter
#include <attr_types.hpp>
template<> BroadcastType > : public ov::EnumAttributeAdapterBase< op::BroadcastType >
#include <attr_types.hpp>
template<> EpsMode > : public ov::EnumAttributeAdapterBase< op::EpsMode >
#include <attr_types.hpp>
template<> GeluApproximationMode > : public ov::EnumAttributeAdapterBase< op::GeluApproximationMode >
#include <gelu.hpp>
template<> LSTMWeightsFormat > : public ov::EnumAttributeAdapterBase< op::LSTMWeightsFormat >
#include <lstm_cell.hpp>
template<> MVNEpsMode > : public ov::EnumAttributeAdapterBase< op::MVNEpsMode >
#include <mvn.hpp>
template<> PadMode > : public ov::EnumAttributeAdapterBase< op::PadMode >
#include <attr_types.hpp>
template<> PadType > : public ov::EnumAttributeAdapterBase< op::PadType >
#include <attr_types.hpp>
template<> RecurrentSequenceDirection > : public ov::EnumAttributeAdapterBase< op::RecurrentSequenceDirection >
#include <attr_types.hpp>
template<> RoundingType > : public ov::EnumAttributeAdapterBase< op::RoundingType >
#include <attr_types.hpp>
template<> TopKMode > : public ov::EnumAttributeAdapterBase< op::TopKMode >
#include <attr_types.hpp>
template<> TopKSortType > : public ov::EnumAttributeAdapterBase< op::TopKSortType >
#include <attr_types.hpp>
template<> CoordinateTransformMode > : public ov::EnumAttributeAdapterBase< op::util::InterpolateBase::CoordinateTransformMode >
#include <interpolate_base.hpp>
template<> InterpolateMode > : public ov::EnumAttributeAdapterBase< op::util::InterpolateBase::InterpolateMode >
#include <interpolate_base.hpp>
template<> NearestMode > : public ov::EnumAttributeAdapterBase< op::util::InterpolateBase::NearestMode >
#include <interpolate_base.hpp>
template<> ShapeCalcMode > : public ov::EnumAttributeAdapterBase< op::util::InterpolateBase::ShapeCalcMode >
#include <interpolate_base.hpp>
template<> SortResultType > : public ov::EnumAttributeAdapterBase< op::util::MulticlassNmsBase::SortResultType >
#include <multiclass_nms_base.hpp>
template<> DepthToSpaceMode > : public ov::EnumAttributeAdapterBase< op::v0::DepthToSpace::DepthToSpaceMode >
#include <depth_to_space.hpp>
template<> InterpolateMode > : public ov::EnumAttributeAdapterBase< op::v0::Interpolate::InterpolateMode >
#include <interpolate.hpp>
template<> SpaceToDepthMode > : public ov::EnumAttributeAdapterBase< op::v0::SpaceToDepth::SpaceToDepthMode >
#include <space_to_depth.hpp>
template<> Reduction > : public ov::EnumAttributeAdapterBase< op::v12::ScatterElementsUpdate::Reduction >
#include <scatter_elements_update.hpp>
template<> BinaryConvolutionMode > : public ov::EnumAttributeAdapterBase< op::v1::BinaryConvolution::BinaryConvolutionMode >
#include <binary_convolution.hpp>
template<> BoxEncodingType > : public ov::EnumAttributeAdapterBase< op::v1::NonMaxSuppression::BoxEncodingType >
#include <non_max_suppression.hpp>
template<> Mode > : public ov::EnumAttributeAdapterBase< op::v1::Reverse::Mode >
#include <reverse.hpp>
template<> BoxEncodingType > : public ov::EnumAttributeAdapterBase< op::v3::NonMaxSuppression::BoxEncodingType >
#include <non_max_suppression.hpp>
template<> PoolingMode > : public ov::EnumAttributeAdapterBase< op::v3::ROIAlign::PoolingMode >
#include <roi_align.hpp>
template<> SpecialBodyPorts > : public ov::DirectValueAccessor< op::v5::Loop::SpecialBodyPorts >
#include <loop.hpp>
template<> BoxEncodingType > : public ov::EnumAttributeAdapterBase< op::v5::NonMaxSuppression::BoxEncodingType >
#include <non_max_suppression.hpp>
template<> RoundMode > : public ov::EnumAttributeAdapterBase< op::v5::Round::RoundMode >
#include <round.hpp>
template<> DecayFunction > : public ov::EnumAttributeAdapterBase< op::v8::MatrixNms::DecayFunction >
#include <matrix_nms.hpp>
template<> SortResultType > : public ov::EnumAttributeAdapterBase< op::v8::MatrixNms::SortResultType >
#include <matrix_nms.hpp>
template<> InterpolationMode > : public ov::EnumAttributeAdapterBase< op::v9::GridSample::InterpolationMode >
#include <grid_sample.hpp>
template<> PaddingMode > : public ov::EnumAttributeAdapterBase< op::v9::GridSample::PaddingMode >
#include <grid_sample.hpp>
template<> BoxEncodingType > : public ov::EnumAttributeAdapterBase< op::v9::NonMaxSuppression::BoxEncodingType >
#include <non_max_suppression.hpp>
template<> AlignedMode > : public ov::EnumAttributeAdapterBase< op::v9::ROIAlign::AlignedMode >
#include <roi_align.hpp>
template<> PoolingMode > : public ov::EnumAttributeAdapterBase< op::v9::ROIAlign::PoolingMode >
#include <roi_align.hpp>
template<> AxisSet > : public ov::ValueAccessor< std::vector< int64_t > >
#include <axis_set.hpp>

Public Functions

virtual const std::vector<int64_t> &get() override

Returns the value.

virtual void set(const std::vector<int64_t> &value) override

Sets the value.

template<> Dimension > : public ov::DirectValueAccessor< ov::Dimension >
#include <dimension.hpp>
template<> Type > : public ov::ValueAccessor< std::string >
#include <element_type.hpp>

Public Functions

virtual const std::string &get() override

Returns the value.

virtual void set(const std::string &value) override

Sets the value.

template<> Type_t > : public ov::EnumAttributeAdapterBase< ov::element::Type_t >
#include <element_type.hpp>
template<> TypeVector > : public ov::DirectValueAccessor< ov::element::TypeVector >
#include <element_type.hpp>
template<> NodeVector > : public ov::VisitorAdapter
#include <node.hpp>
template<> FrameworkNodeAttrs > : public ov::DirectValueAccessor< ov::op::util::FrameworkNodeAttrs >
#include <framework_node.hpp>
template<> PartialShape > : public ov::DirectValueAccessor< ov::PartialShape >
#include <partial_shape.hpp>
template<> Shape > : public ov::IndirectVectorValueAccessor< ov::Shape, std::vector< int64_t > >
#include <shape.hpp>
template<>
class AttributeAdapter<ParameterVector> : public ov::VisitorAdapter
#include <parameter.hpp>
template<>
class AttributeAdapter<ResultVector> : public ov::VisitorAdapter
#include <result.hpp>
template<> string > > : public ov::DirectValueAccessor< std::set< std::string > >
#include <attribute_adapter.hpp>
template<> AlignedBuffer > > : public ov::DirectValueAccessor< std::shared_ptr< ngraph::runtime::AlignedBuffer > >
#include <aligned_buffer.hpp>
template<> Variable > > : public ov::DirectValueAccessor< std::shared_ptr< op::util::Variable > >
#include <variable.hpp>
template<> Model > > : public ov::DirectValueAccessor< std::shared_ptr< ov::Model > >
#include <model.hpp>
template<> Node > > : public ov::VisitorAdapter
#include <node.hpp>

Visits a reference to a node that has been registered with the visitor.

template<> string > : public ov::DirectValueAccessor< std::string >
#include <attribute_adapter.hpp>

Access a string as a string.

template<> vector< double > > : public ov::DirectValueAccessor< std::vector< double > >
#include <attribute_adapter.hpp>

Access a vector<double>

template<> vector< float > > : public ov::DirectValueAccessor< std::vector< float > >
#include <attribute_adapter.hpp>

Access a vector<float>

template<> vector< int16_t > > : public ov::DirectValueAccessor< std::vector< int16_t > >
#include <attribute_adapter.hpp>

Access a vector<int16_t>

template<> vector< int32_t > > : public ov::DirectValueAccessor< std::vector< int32_t > >
#include <attribute_adapter.hpp>

Access a vector<int32_t>

template<> vector< int64_t > > : public ov::DirectValueAccessor< std::vector< int64_t > >
#include <attribute_adapter.hpp>

Access a vector<int64_t>

template<> vector< int8_t > > : public ov::DirectValueAccessor< std::vector< int8_t > >
#include <attribute_adapter.hpp>

Access a vector<int8_t>

Note: These class bodies cannot be defined with templates because of interactions between dllexport and templates on Windows.

template<> InputDescription > > > : public ov::DirectValueAccessor< std::vector< std::shared_ptr< op::util::MultiSubGraphOp::InputDescription > > >
#include <multi_subgraph_base.hpp>
template<> OutputDescription > > > : public ov::DirectValueAccessor< std::vector< std::shared_ptr< op::util::MultiSubGraphOp::OutputDescription > > >
#include <multi_subgraph_base.hpp>
template<> string > > : public ov::DirectValueAccessor< std::vector< std::string > >
#include <attribute_adapter.hpp>

Access a vector<string>

template<> vector< uint16_t > > : public ov::DirectValueAccessor< std::vector< uint16_t > >
#include <attribute_adapter.hpp>

Access a vector<uint16_t>

template<> vector< uint32_t > > : public ov::DirectValueAccessor< std::vector< uint32_t > >
#include <attribute_adapter.hpp>

Access a vector<uint32_t>

template<> vector< uint64_t > > : public ov::DirectValueAccessor< std::vector< uint64_t > >
#include <attribute_adapter.hpp>

Access a vector<uint64_t>

template<> vector< uint8_t > > : public ov::DirectValueAccessor< std::vector< uint8_t > >
#include <attribute_adapter.hpp>

Access a vector<uint8_t>

template<>
class AttributeAdapter<Strides> : public ov::IndirectVectorValueAccessor<Strides, std::vector<int64_t>>
#include <strides.hpp>
template<>
class AttributeAdapter<uint16_t> : public ov::IndirectScalarValueAccessor<uint16_t, int64_t>
#include <attribute_adapter.hpp>

Access a uint16_t as an int64_t.

template<>
class AttributeAdapter<uint32_t> : public ov::IndirectScalarValueAccessor<uint32_t, int64_t>
#include <attribute_adapter.hpp>

Access a uint32_t as an int64_t.

template<>
class AttributeAdapter<uint64_t> : public ov::IndirectScalarValueAccessor<uint64_t, int64_t>
#include <attribute_adapter.hpp>

Access a uint64_t as an int64_t.

template<>
class AttributeAdapter<uint8_t> : public ov::IndirectScalarValueAccessor<uint8_t, int64_t>
#include <attribute_adapter.hpp>

Access a uint8_t as an int64_t.

class AttributeVisitor
#include <attribute_visitor.hpp>

Visits the attributes of a node, primarily for serialization-like tasks.

Attributes are the node parameters that are always compile-time constants. Values computed from the graph topology and attributes during compilation are not attributes.

Attributes have a wide variety of types, but serialization formats are more restricted. We assume serialization easily supports scalar types of bool 64-bit signed, string, and double, and has specialized ways to support numeric arrays and raw data+size. The visitor and adapter convert between the limited serialization types and the unlimited attribute types.

A visitor is passed to an op’s visit_attributes method. The visit_attributes method calls the template method visitor.on_attribute<AT>(const std::string& name, AT& value) on each attribute. The visitor can read or write the attribute’s value. The on_attribute method creates an AttributeAdapter<AT> for the value and passes it to one of the visitors on_adapter methods. The on_adapter methods expect a reference to a ValueAccessor<VAT> or a VisitorAdapter. A ValueAccessor<VAT> has get/set methods that can be used to read/write the attribute value as type VAT. These methods are triggered by deriving AttributeAdapter<AT> from ValueAccessor<VAT>. For more complex cases, such as structs, the on_adapter method for VisitorAdapter passes the name and visitor to the adapter, so that the adapter can perform additional work such as visiting struct members or sequence values.

When a node visits an attribute with structure, the node’s on_attribute passes a name for the entire attribute, but the struct will have its own methods to be visited. Similarly, a vector will have a sequence of members to be visited. The adapter may use the visitor methods start_struct/finish_struct and start_vector/next_vector/finish_vector to inidicate nexted members.

The visitor method get_name_with_context creates a generic nested version of the name. Visitors can override according to their serialization requirements.

Attributes that are shared_ptr<Node> are special. They must have been already been registered with the visitor using register_node, which needs a shared pointer to a node and a string ID. The ID string will be used to serialize the node or find the node during deserialization.

Subclassed by ov::frontend::FWVisitor, ov::frontend::FWVisitorInputAttributes

Public Functions

virtual void on_adapter(const std::string &name, ValueAccessor<void> &adapter) = 0

handles all specialized on_adapter methods implemented by the visitor.

The adapter implements get_type_info(), which can be used to determine the adapter directly or via is_type and as_type on any platform

virtual void on_adapter(const std::string &name, VisitorAdapter &adapter)

Hook for adapters that need visitor access.

virtual void on_adapter(const std::string &name, ValueAccessor<std::shared_ptr<ov::Model>> &adapter)

Provides API to handle openvino Function attribute type, accessed as ValueAccessor.

Parameters
  • name – attribute name

  • adapter – reference to a Function ValueAccessor<VAT>

template<typename AT>
inline void on_attribute(const std::string &name, AT &value)

The generic visitor. There must be a definition of AttributeAdapter<T> that can convert to a ValueAccessor

inline const std::vector<std::string> &get_context() const
Returns

The nested context of visits

virtual std::string get_name_with_context()
Returns

context prepended to names

virtual void start_structure(const std::string &name)

Start visiting a nested structure.

virtual std::string finish_structure()

Finish visiting a nested structure.

virtual void register_node(const std::shared_ptr<Node> &node, node_id_t id = invalid_node_id)

Associate a node with an id.

No node may be used as an attribute unless it has already been registered with an ID. References to nodes are visited with a ValueAccessor of their ID.

virtual std::shared_ptr<Node> get_registered_node(node_id_t id)

Returns the node with the given id, or nullptr if there is no registered node.

virtual node_id_t get_registered_node_id(const std::shared_ptr<Node> &node)

Returns the id for the node, or -1 if the node is not registered.

class AvgPoolPrecisionPreservedAttribute : public ov::PrecisionPreservedAttribute
#include <avg_pool_precision_preserved_attribute.hpp>

AvgPoolPrecisionPreservedAttribute is utility attribute which is used only during AvgPool operation precision preserved property definition.

For more details about the attribute, refer to AvgPoolPrecisionPreservedAttribute page in the Inference Engine Developer Guide.

class AxisSet : public std::set<size_t>
#include <axis_set.hpp>

A set of axes.

class AxisVector : public std::vector<size_t>
#include <axis_vector.hpp>

A vector of axes.

class BaseOpExtension : public ov::Extension
#include <op_extension.hpp>

The base interface for OpenVINO operation extensions.

Subclassed by ov::OpExtension< T >

Public Functions

virtual const ov::DiscreteTypeInfo &get_type_info() const = 0

Returns the type info of operation.

Returns

ov::DiscreteTypeInfo

virtual ov::OutputVector create(const ov::OutputVector &inputs, ov::AttributeVisitor &visitor) const = 0

Method creates an OpenVINO operation.

Parameters
  • inputs – vector of input ports

  • visitor – attribute visitor which allows to read necessaty arguments

Returns

vector of output ports

virtual std::vector<ov::Extension::Ptr> get_attached_extensions() const = 0

Returns extensions that should be registered together with this extension class object.

Attached extensions may include frontend extensions that OpenVINO op to framework ops or necessary transformations that should be applied to the network which consist of target op.

Returns

~BaseOpExtension() override

Destructor.

class bfloat16
#include <bfloat16.hpp>
class BiasAttribute : public ov::RuntimeAttribute
#include <bias_attribute.hpp>
class Busy : public ov::Exception
#include <exception.hpp>

Thrown in case of calling the InferRequest methods while the request is busy with compute operation.

class Cancelled : public ov::Exception
#include <exception.hpp>

Thrown in case of cancelled asynchronous operation.

struct CheckLocInfo
#include <except.hpp>
class CompiledModel
#include <compiled_model.hpp>

This class represents a compiled model.

A model is compiled by a specific device by applying multiple optimization transformations, then mapping to compute kernels.

Public Functions

CompiledModel() = default

Default constructor.

~CompiledModel()

Destructor that preserves unloading order of an implementation object and reference to library.

std::shared_ptr<const Model> get_runtime_model() const

Gets runtime model information from a device. This object represents an internal device-specific model that is optimized for a particular accelerator. It contains device-specific nodes, runtime information and can be used only to understand how the source model is optimized and which kernels, element types, and layouts are selected for optimal inference.

Returns

A model containing Executable Graph Info.

const std::vector<ov::Output<const ov::Node>> &inputs() const

Gets all inputs of a compiled model. Inputs are represented as a vector of outputs of the ov::op::v0::Parameter operations. They contain information about input tensors such as tensor shape, names, and element type.

Returns

std::vector of model inputs.

const ov::Output<const ov::Node> &input() const

Gets a single input of a compiled model. The input is represented as an output of the ov::op::v0::Parameter operation. The input contains information about input tensor such as tensor shape, names, and element type.

Note

If a model has more than one input, this method throws ov::Exception.

Returns

Compiled model input.

const ov::Output<const ov::Node> &input(size_t i) const

Gets input of a compiled model identified by i. The input contains information about input tensor such as tensor shape, names, and element type.

Note

The method throws ov::Exception if input with the specified index i is not found.

Parameters

i – Index of input.

Returns

Compiled model input.

const ov::Output<const ov::Node> &input(const std::string &tensor_name) const

Gets input of a compiled model identified by tensor_name. The input contains information about input tensor such as tensor shape, names, and element type.

Note

The method throws ov::Exception if input with the specified tensor name tensor_name is not found.

Parameters

tensor_name – The input tensor name.

Returns

Compiled model input.

const std::vector<ov::Output<const ov::Node>> &outputs() const

Get all outputs of a compiled model. Outputs are represented as a vector of output from the ov::op::v0::Result operations. Outputs contain information about output tensors such as tensor shape, names, and element type.

Returns

std::vector of model outputs.

const ov::Output<const ov::Node> &output() const

Gets a single output of a compiled model. The output is represented as an output from the ov::op::v0::Result operation. The output contains information about output tensor such as tensor shape, names, and element type.

Note

If a model has more than one output, this method throws ov::Exception.

Returns

Compiled model output.

const ov::Output<const ov::Node> &output(size_t i) const

Gets output of a compiled model identified by index. The output contains information about output tensor such as tensor shape, names, and element type.

Note

The method throws ov::Exception if output with the specified index index is not found.

Parameters

i – Index of input.

Returns

Compiled model output.

const ov::Output<const ov::Node> &output(const std::string &tensor_name) const

Gets output of a compiled model identified by tensor_name. The output contains information about output tensor such as tensor shape, names, and element type.

Note

The method throws ov::Exception if output with the specified tensor name tensor_name is not found.

Parameters

tensor_nameOutput tensor name.

Returns

Compiled model output.

InferRequest create_infer_request()

Creates an inference request object used to infer the compiled model. The created request has allocated input and output tensors (which can be changed later).

Returns

InferRequest object

void export_model(std::ostream &model_stream)

Exports the current compiled model to an output stream std::ostream. The exported model can also be imported via the ov::Core::import_model method.

Parameters

model_streamOutput stream to store the model to.

void set_property(const AnyMap &properties)

Sets properties for the current compiled model.

Parameters

properties – Map of pairs: (property name, property value).

template<typename ...Properties>
inline util::EnableIfAllStringAny<void, Properties...> set_property(Properties&&... properties)

Sets properties for the current compiled model.

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types.

Parameters

properties – Optional pack of pairs: (property name, property value).

Any get_property(const std::string &name) const

Gets properties for current compiled model.

The method is responsible for extracting information that affects compiled model inference. The list of supported configuration values can be extracted via CompiledModel::get_property with the ov::supported_properties key, but some of these keys cannot be changed dynamically, for example, ov::device::id cannot be changed if a compiled model has already been compiled for a particular device.

Parameters

nameProperty key, can be found in openvino/runtime/properties.hpp.

Returns

Property value.

template<typename T, PropertyMutability mutability>
inline T get_property(const ov::Property<T, mutability> &property) const

Gets properties related to device behaviour.

The method extracts information that can be set via the set_property method.

Template Parameters

T – Type of a returned value.

Parameters

propertyProperty object.

Returns

Value of property.

RemoteContext get_context() const

Returns pointer to device-specific shared context on a remote accelerator device that was used to create this CompiledModel.

Returns

A context.

bool operator!() const noexcept

Checks if the current CompiledModel object is not initialized.

Returns

true if the current CompiledModel object is not initialized; false, otherwise.

explicit operator bool() const noexcept

Checks if the current CompiledModel object is initialized.

Returns

true if the current CompiledModel object is initialized; false, otherwise.

class Coordinate : public std::vector<size_t>
#include <coordinate.hpp>

Coordinates for a tensor element.

class CoordinateDiff : public std::vector<std::ptrdiff_t>
#include <coordinate_diff.hpp>

A difference (signed) of tensor element coordinates.

class CoordinateIterator
#include <coordinate_transform.hpp>

A useful class that allows to iterate over the tensor coordinates. For example, for tensor with dimensions {2, 3} this iterator produces the following coordinates: {0,0}, {0,1}, {0,2}, {1,0}, {1,1}, {2,2}.

Deprecated:

Public Functions

inline CoordinateIterator(const Shape &target_shape)

Coordinates iterator constructor.

Parameters

target_shape – The target shape for coordinates iteration

void operator++()

The postfix operation increment the iterator by one.

CoordinateIterator operator++(int)

The prefix operation increment the iterator by one.

void operator+=(size_t n)

Increments iterator n times.

Parameters

n – number of elements it should be advanced

const Coordinate &operator*() const noexcept

Iterator dereferencing operator returns reference to current pointed coordinate.

bool operator!=(const CoordinateIterator &it) const noexcept

Checks for iterator inequality.

Parameters

it – second iterator to compare

bool operator==(const CoordinateIterator &it) const noexcept

Checks for iterator equality.

Parameters

it – second iterator to compare

size_t advance(size_t axis) noexcept

Increments iterator using specified axis of the shape n times.

Parameters

axis – index used for iteration

Public Static Functions

static const CoordinateIterator &end()

Useful function to build the last iterator. Returns a singleton that points to the last iterator.

class CoordinateTransform : protected ov::CoordinateTransformBasic
#include <coordinate_transform.hpp>

Class which allows to calculate item index with given coordinates in tensor and helps to iterate over the subset of coordinates. Tensor items should be placed in memory in row-major order.

Deprecated:

Public Functions

size_t index(const Coordinate &c) const

The tensor element index calculation by given coordinate.

Parameters

c – tensor element coordinate

bool has_source_coordinate(const Coordinate &c) const

Checks that coordinate belongs to given coordinates subset.

Parameters

c – tensor element coordinate

Coordinate to_source_coordinate(const Coordinate &c) const

Convert a target-space coordinate to a source-space coordinate.

Parameters

c – tensor element coordinate

CoordinateIterator begin() const noexcept

Returns an iterator to the first coordinate of the tensor.

const CoordinateIterator &end() const noexcept

Returns an iterator to the coordinate following the last element of the tensor.

class CoordinateTransformBasic
#include <coordinate_transform.hpp>

Class which allows to calculate item index with given coordinates in tensor and helps to iterate over all coordinates. Tensor items should be placed in memory in row-major order.

Subclassed by ov::CoordinateTransform

Public Functions

size_t index(const Coordinate &c) const

The tensor element index calculation by given coordinate.

Deprecated:

Parameters

c – tensor element coordinate

CoordinateIterator begin() const noexcept

Returns an iterator to the first coordinate of the tensor.

const CoordinateIterator &end() const noexcept

Returns an iterator to the coordinate following the last element of the tensor.

class Core
#include <core.hpp>

This class represents an OpenVINO runtime Core entity.

User applications can create several Core class instances, but in this case the underlying plugins are created multiple times and not shared between several Core instances. The recommended way is to have a single Core instance per application.

Unnamed Group

CompiledModel compile_model(const std::string &model_path, const AnyMap &properties = {})

Reads and loads a compiled model from the IR/ONNX/PDPD file to the default OpenVINO device selected by the AUTO plugin.

This can be more efficient than using the Core::read_model + Core::compile_model(model_in_memory_object) flow, especially for cases when caching is enabled and a cached model is available.

Parameters
  • model_path – Path to a model.

  • properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model.

Unnamed Group

template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::string &model_path, Properties&&... properties)

Reads and loads a compiled model from IR / ONNX / PDPD file to the default OpenVINO device selected by AUTO plugin.

This can be more efficient than using read_model + compile_model(Model) flow especially for cases when caching is enabled and cached model is available

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters
  • model_path – path to model with string or wstring

  • properties – Optional pack of pairs: (property name, property value) relevant only for this load operation

Returns

A compiled model

Unnamed Group

CompiledModel compile_model(const std::string &model_path, const std::string &device_name, const AnyMap &properties = {})

Reads a model and creates a compiled model from the IR/ONNX/PDPD file.

This can be more efficient than using the Core::read_model + Core::compile_model(model_in_memory_object) flow, especially for cases when caching is enabled and a cached model is available.

Parameters
  • model_path – Path to a model.

  • device_name – Name of a device to load a model to.

  • properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model.

Unnamed Group

template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::string &model_path, const std::string &device_name, Properties&&... properties)

Reads a model and creates a compiled model from the IR/ONNX/PDPD file.

This can be more efficient than using read_model + compile_model(Model) flow especially for cases when caching is enabled and cached model is available.

Template Parameters

Properties – Should be a pack of std::pair<std::string, ov::Any> types.

Parameters
  • model_path – Path to a model.

  • device_name – Name of a device to load a model to.

  • properties – Optional pack of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model.

Public Functions

explicit Core(const std::string &xml_config_file = {})

Constructs an OpenVINO Core instance with devices and their plugins description.

There are two ways how to configure device plugins:

  1. (default) Use XML configuration file in case of dynamic libraries build;

  2. Use strictly defined configuration in case of static libraries build.

Parameters

xml_config_file – Path to the .xml file with plugins to load from. If path contains only file name with extension, file will be searched in a folder with OpenVINO runtime shared library. If the XML configuration file is not specified, default OpenVINO Runtime plugins are loaded from:

  1. (dynamic build) default plugins.xml file located in the same folder as OpenVINO runtime shared library;

  2. (static build) statically defined configuration. In this case path to the .xml file is ignored.

std::map<std::string, Version> get_versions(const std::string &device_name) const

Returns device plugins version information. Device name can be complex and identify multiple devices at once like HETERO:CPU,GPU; in this case, std::map contains multiple entries, each per device.

Parameters

device_name – Device name to identify a plugin.

Returns

A vector of versions.

std::shared_ptr<ov::Model> read_model(const std::string &model_path, const std::string &bin_path = {}) const

Reads models from IR / ONNX / PDPD / TF / TFLite file formats.

Parameters
  • model_path – Path to a model.

  • bin_path – Path to a data file. For IR format (*.bin):

    • if bin_path is empty, will try to read a bin file with the same name as xml and

    • if the bin file with the same name is not found, will load IR without weights. For the following file formats the bin_path parameter is not used:

    • ONNX format (*.onnx)

    • PDPD (*.pdmodel)

    • TF (*.pb)

    • TFLite (*.tflite)

Returns

A model.

std::shared_ptr<ov::Model> read_model(const std::string &model, const Tensor &weights) const

Reads models from IR / ONNX / PDPD / TF / TFLite formats.

Note

Created model object shares the weights with the weights object. Thus, do not create weights on temporary data that can be freed later, since the model constant data will point to an invalid memory.

Parameters
  • model – String with a model in IR / ONNX / PDPD / TF / TFLite format.

  • weights – Shared pointer to a constant tensor with weights. Reading ONNX / PDPD / TF / TFLite models does not support loading weights from the weights tensors.

Returns

A model.

CompiledModel compile_model(const std::shared_ptr<const ov::Model> &model, const AnyMap &properties = {})

Creates and loads a compiled model from a source model to the default OpenVINO device selected by the AUTO plugin.

Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources).

Parameters
  • modelModel object acquired from Core::read_model.

  • properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model.

template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::shared_ptr<const ov::Model> &model, Properties&&... properties)

Creates and loads a compiled model from a source model to the default OpenVINO device selected by AUTO plugin.

Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources)

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters
  • modelModel object acquired from Core::read_model

  • properties – Optional pack of pairs: (property name, property value) relevant only for this load operation

Returns

A compiled model

CompiledModel compile_model(const std::shared_ptr<const ov::Model> &model, const std::string &device_name, const AnyMap &properties = {})

Creates a compiled model from a source model object.

Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources).

Parameters
  • modelModel object acquired from Core::read_model.

  • device_name – Name of a device to load a model to.

  • properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model.

template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::shared_ptr<const ov::Model> &model, const std::string &device_name, Properties&&... properties)

Creates a compiled model from a source model object.

Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources)

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters
  • modelModel object acquired from Core::read_model

  • device_name – Name of device to load model to

  • properties – Optional pack of pairs: (property name, property value) relevant only for this load operation

Returns

A compiled model

CompiledModel compile_model(const std::string &model, const ov::Tensor &weights, const std::string &device_name, const AnyMap &properties = {})

Reads a model and creates a compiled model from the IR/ONNX/PDPD memory.

Note

Created model object shares the weights with the weights object. Thus, do not create weights on temporary data that can be freed later, since the model constant data will point to an invalid memory.

Parameters
  • model – String with a model in IR/ONNX/PDPD format.

  • weights – Shared pointer to a constant tensor with weights. Reading ONNX/PDPD models does not support loading weights from the weights tensors.

  • device_name – Name of a device to load a model to.

  • properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model.

template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::string &model, const ov::Tensor &weights, const std::string &device_name, Properties&&... properties)

Reads a model and creates a compiled model from the IR/ONNX/PDPD memory.

Note

Created model object shares the weights with the weights object. Thus, do not create weights on temporary data that can be freed later, since the model constant data will point to an invalid memory.

Parameters
  • model – String with a model in IR/ONNX/PDPD format.

  • weights – Shared pointer to a constant tensor with weights. Reading ONNX/PDPD models does not support loading weights from the weights tensors.

  • device_name – Name of a device to load a model to.

Template Parameters

Properties – Should be a pack of std::pair<std::string, ov::Any> types.

Returns

A compiled model.

CompiledModel compile_model(const std::shared_ptr<const ov::Model> &model, const RemoteContext &context, const AnyMap &properties = {})

Creates a compiled model from a source model within a specified remote context.

Parameters
  • modelModel object acquired from Core::read_model.

  • context – A reference to a RemoteContext object.

  • properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model object.

template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::shared_ptr<const ov::Model> &model, const RemoteContext &context, Properties&&... properties)

Creates a compiled model from a source model within a specified remote context.

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters
  • modelModel object acquired from Core::read_model

  • context – Pointer to RemoteContext object

  • properties – Optional pack of pairs: (property name, property value) relevant only for this load operation

Returns

A compiled model object

OPENVINO_SUPPRESS_DEPRECATED_START void add_extension (const std::shared_ptr< InferenceEngine::IExtension > &extension)

Registers OpenVINO 1.0 extension to a Core object.

Deprecated:

This method is deprecated. Please use other Core::add_extension methods.

Parameters

extension – Pointer to the already loaded extension.

OPENVINO_SUPPRESS_DEPRECATED_END void add_extension (const std::string &library_path)

Registers an extension to a Core object.

Parameters

library_path – Path to the library with ov::Extension.

void add_extension(const std::shared_ptr<ov::Extension> &extension)

Registers an extension to a Core object.

Parameters

extension – Pointer to the extension.

void add_extension(const std::vector<std::shared_ptr<ov::Extension>> &extensions)

Registers extensions to a Core object.

Parameters

extensions – Vector of loaded extensions.

template<class T, typename std::enable_if<std::is_base_of<ov::Extension, T>::value, bool>::type = true>
inline void add_extension(const T &extension)

Registers an extension to a Core object.

Parameters

extensionExtension class that is inherited from the ov::Extension class.

template<class T, class ...Targs, typename std::enable_if<std::is_base_of<ov::Extension, T>::value, bool>::type = true>
inline void add_extension(const T &extension, Targs... args)

Registers extensions to a Core object.

Parameters
  • extensionExtension class that is inherited from the ov::Extension class.

  • args – A list of extensions.

template<class T, typename std::enable_if<std::is_base_of<ov::op::Op, T>::value, bool>::type = true>
inline void add_extension()

Registers a custom operation inherited from ov::op::Op.

template<class T, class ...Targs, typename std::enable_if<std::is_base_of<ov::op::Op, T>::value && sizeof...(Targs), bool>::type = true>
inline void add_extension()

Registers custom operations inherited from ov::op::Op.

CompiledModel import_model(std::istream &model_stream, const std::string &device_name, const AnyMap &properties = {})

Imports a compiled model from the previously exported one.

Parameters
  • model_stream – std::istream input stream containing a model previously exported using the ov::CompiledModel::export_model method.

  • device_name – Name of a device to import a compiled model for. Note, if device_name device was not used to compile the original mode, an exception is thrown.

  • properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model.

template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> import_model(std::istream &model_stream, const std::string &device_name, Properties&&... properties)

Imports a compiled model from the previously exported one.

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types.

Parameters
  • model_streamModel stream.

  • device_name – Name of a device to import a compiled model for. Note, if device_name device was not used to compile the original mode, an exception is thrown.

  • properties – Optional pack of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model.

CompiledModel import_model(std::istream &model_stream, const RemoteContext &context, const AnyMap &properties = {})

Imports a compiled model from the previously exported one with the specified remote context.

Parameters
  • model_stream – std::istream input stream containing a model previously exported from ov::CompiledModel::export_model

  • context – A reference to a RemoteContext object. Note, if the device from context was not used to compile the original mode, an exception is thrown.

  • properties – Optional map of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model.

template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> import_model(std::istream &model_stream, const RemoteContext &context, Properties&&... properties)

Imports a compiled model from the previously exported one with the specified remote context.

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types.

Parameters
  • model_streamModel stream.

  • context – Pointer to a RemoteContext object.

  • properties – Optional pack of pairs: (property name, property value) relevant only for this load operation.

Returns

A compiled model.

SupportedOpsMap query_model(const std::shared_ptr<const ov::Model> &model, const std::string &device_name, const AnyMap &properties = {}) const

Query device if it supports the specified model with specified properties.

Parameters
  • device_name – Name of a device to query.

  • modelModel object to query.

  • properties – Optional map of pairs: (property name, property value).

Returns

An object containing a map of pairs an operation name -> a device name supporting this operation.

template<typename ...Properties>
inline util::EnableIfAllStringAny<SupportedOpsMap, Properties...> query_model(const std::shared_ptr<const ov::Model> &model, const std::string &device_name, Properties&&... properties) const

Queries a device if it supports the specified model with specified properties.

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types.

Parameters
  • device_name – Name of a device to query.

  • modelModel object to query.

  • properties – Optional pack of pairs: (property name, property value) relevant only for this query operation.

Returns

An object containing a map of pairs an operation name -> a device name supporting this operation.

void set_property(const AnyMap &properties)

Sets properties for all the registered devices, acceptable keys can be found in openvino/runtime/properties.hpp.

Parameters

properties – Map of pairs: (property name, property value).

template<typename ...Properties>
inline util::EnableIfAllStringAny<void, Properties...> set_property(Properties&&... properties)

Sets properties for all the registered devices, acceptable keys can be found in openvino/runtime/properties.hpp.

Template Parameters

Properties – Should be a pack of std::pair<std::string, ov::Any> types.

Parameters

properties – Optional pack of pairs: property name, property value.

void set_property(const std::string &device_name, const AnyMap &properties)

Sets properties for a device, acceptable keys can be found in openvino/runtime/properties.hpp.

Parameters
  • device_name – Name of a device.

  • properties – Map of pairs: (property name, property value).

template<typename ...Properties>
inline util::EnableIfAllStringAny<void, Properties...> set_property(const std::string &device_name, Properties&&... properties)

Sets properties for a device, acceptable keys can be found in openvino/runtime/properties.hpp.

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types.

Parameters
  • device_name – Name of a device.

  • properties – Optional pack of pairs: (property name, property value).

Any get_property(const std::string &device_name, const std::string &name) const

Gets properties related to device behaviour.

The method extracts information that can be set via the set_property method.

Parameters
  • device_name – Name of a device to get a property value.

  • nameProperty name.

Returns

Value of a property corresponding to the property name.

Any get_property(const std::string &device_name, const std::string &name, const AnyMap &arguments) const

Gets properties related to device behaviour.

The method extracts information that can be set via the set_property method.

Parameters
  • device_name – Name of a device to get a property value.

  • nameProperty name.

  • arguments – Additional arguments to get a property.

Returns

Value of a property corresponding to the property name.

inline Any get_property(const std::string &name) const

Gets properties related to core behaviour.

The method extracts information that can be set via the set_property method.

Parameters

nameProperty name.

Returns

Value of a property corresponding to the property name.

template<typename T, PropertyMutability M>
inline T get_property(const std::string &device_name, const ov::Property<T, M> &property) const

Gets properties related to device behaviour.

The method is needed to request common device or system properties. It can be device name, temperature, and other devices-specific values.

Template Parameters
  • T – Type of a returned value.

  • MProperty mutability.

Parameters
  • device_name – Name of a device to get a property value.

  • propertyProperty object.

Returns

Property value.

template<typename T, PropertyMutability M>
inline T get_property(const std::string &device_name, const ov::Property<T, M> &property, const AnyMap &arguments) const

Gets properties related to device behaviour.

The method is needed to request common device or system properties. It can be device name, temperature, other devices-specific values.

Template Parameters
  • T – Type of a returned value.

  • MProperty mutability.

Parameters
  • device_name – Name of a device to get a property value.

  • propertyProperty object.

  • arguments – Additional arguments to get a property.

Returns

Property value.

template<typename T, PropertyMutability M, typename ...Args>
inline util::EnableIfAllStringAny<T, Args...> get_property(const std::string &device_name, const ov::Property<T, M> &property, Args&&... args) const

Gets properties related to device behaviour.

The method is needed to request common device or system properties. It can be device name, temperature, other devices-specific values.

Template Parameters
  • T – Type of a returned value.

  • MProperty mutability.

  • Args – Set of additional arguments ended with property object variable.

Parameters
  • device_name – Name of a device to get a property value.

  • propertyProperty object.

  • args – Optional pack of pairs: (argument name, argument value) ended with property object.

Returns

Property value.

std::vector<std::string> get_available_devices() const

Returns devices available for inference. Core objects go over all registered plugins and ask about available devices.

Returns

A vector of devices. The devices are returned as { CPU, GPU.0, GPU.1, GNA }. If there is more than one device of a specific type, they are enumerated with the .# suffix. Such enumerated device can later be used as a device name in all Core methods like Core::compile_model, Core::query_model, Core::set_property and so on.

void register_plugin(const std::string &plugin, const std::string &device_name, const ov::AnyMap &config = {})

Register a new device and plugin that enables this device inside OpenVINO Runtime.

Note

For security purposes it suggested to specify absolute path to register plugin.

Parameters
  • plugin – Path (absolute or relative) or name of a plugin. Depending on platform, plugin is wrapped with shared library suffix and prefix to identify library full name. For example, on Linux platform, plugin name specified as plugin_name will be wrapped as libplugin_name.so. Plugin search algorithm:

    • If plugin points to an exact library path (absolute or relative), it will be used.

    • If plugin specifies file name (libplugin_name.so) or plugin name (plugin_name), it will be searched by file name (libplugin_name.so) in CWD or in paths pointed by PATH/LD_LIBRARY_PATH/DYLD_LIBRARY_PATH environment variables depending on the platform.

  • device_name – Device name to register a plugin for.

  • config – Plugin configuration options

void unload_plugin(const std::string &device_name)

Unloads the previously loaded plugin identified by device_name from OpenVINO Runtime. The method is needed to remove loaded plugin instance and free its resources. If plugin for a specified device has not been created before, the method throws an exception.

Note

This method does not remove plugin from the plugins known to OpenVINO Core object.

Parameters

device_name – Device name identifying plugin to remove from OpenVINO Runtime.

void register_plugins(const std::string &xml_config_file)

Registers a device plugin to the OpenVINO Runtime Core instance using an XML configuration file with plugins description.

The XML file has the following structure:

<ie>
    <plugins>
        <plugin name="" location="">
            <extensions>
                <extension location=""/>
            </extensions>
            <properties>
                <property key="" value=""/>
            </properties>
        </plugin>
    </plugins>
</ie>

  • name identifies name of a device enabled by a plugin.

  • location specifies absolute path to dynamic library with a plugin. The path can also be relative to XML file directory. It allows having common config for different systems with different configurations.

  • properties are set to a plugin via the ov::Core::set_property method.

  • extensions are set to a plugin via the ov::Core::add_extension method.

Note

For security purposes it suggested to specify absolute path to register plugin.

Parameters

xml_config_file – A path to .xml file with plugins to register.

RemoteContext create_context(const std::string &device_name, const AnyMap &remote_properties)

Creates a new remote shared context object on the specified accelerator device using specified plugin-specific low-level device API parameters (device handle, pointer, context, etc.).

Parameters
  • device_name – Name of a device to create a new shared context on.

  • remote_properties – Map of device-specific shared context remote properties.

Returns

Reference to a created remote context.

template<typename ...Properties>
inline util::EnableIfAllStringAny<RemoteContext, Properties...> create_context(const std::string &device_name, Properties&&... remote_properties)

Creates a new shared context object on specified accelerator device using specified plugin-specific low level device API properties (device handle, pointer, etc.)

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters
  • device_name – Name of a device to create new shared context on.

  • remote_properties – Pack of device-specific shared context remote properties.

Returns

A shared pointer to a created remote context.

RemoteContext get_default_context(const std::string &device_name)

Gets a pointer to default (plugin-supplied) shared context object for the specified accelerator device.

Parameters

device_name – Name of a device to get a default shared context from.

Returns

Reference to a default remote context.

class Decompression : public ov::RuntimeAttribute
#include <decompression.hpp>

Decompression class represents runtime info attribute that marks operation as used as decompression for Compressed Only format.

class DequantizationNode : public ov::RuntimeAttribute
#include <dequantization_node.hpp>

DequantizationNode class represents runtime info attribute that marks operation that are part of dequantization subgraph.

class DeviceIDParser
#include <device_id_parser.hpp>

Class parses device name and id.

class Dimension
#include <dimension.hpp>

Class representing a dimension, which may be dynamic (undetermined until runtime), in a shape or shape-like object.

Static dimensions may be implicitly converted from value_type. A dynamic dimension is constructed with Dimension() or Dimension::dynamic().

Public Functions

Dimension(value_type dimension)

Construct a static dimension.

Parameters

dimension – Value of the dimension.

Dimension(value_type min_dimension, value_type max_dimension)

Construct a dynamic dimension with bounded range.

Parameters
  • min_dimension – The lower inclusive limit for the dimension

  • max_dimension – The upper inclusive limit for the dimension

Dimension(const std::string &str)

Construct a dimension from string.

Parameters

str – String to parse to dimension.

Dimension() = default

Construct a dynamic dimension with range [0, …].

inline bool is_static() const

Check whether this dimension is static.

Returns

true if the dimension is static, else false.

inline bool is_dynamic() const

Check whether this dimension is dynamic.

Returns

false if the dimension is static, else true.

value_type get_length() const

Convert this dimension to value_type. This dimension must be static and non-negative.

Throws

std::invalid_argument – If this dimension is dynamic or negative.

inline const Interval &get_interval() const

Return the interval of valid lengths.

bool same_scheme(const Dimension &dim) const

Check whether this dimension represents the same scheme as the argument (both dynamic, or equal).

Parameters

dim – The other dimension to compare this dimension to.

Returns

true if this dimension and dim are both dynamic, or if they are both static and equal; otherwise, false.

bool compatible(const Dimension &d) const

Check whether this dimension is capable of being merged with the argument dimension.

Two dimensions are considered compatible if it is possible to merge them. (See Dimension::merge.)

Parameters

d – The dimension to compare this dimension with.

Returns

true if this dimension is compatible with d, else false.

bool relaxes(const Dimension &d) const

Check whether this dimension is a relaxation of the argument.

A dimension d1 relaxes (or is a relaxation of) d2 if d1 and d2 are static and equal, or d1 is dynamic.

d1.relaxes(d2) is equivalent to d2.refines(d1).

Parameters

d – The dimension to compare this dimension with.

Returns

true if this dimension relaxes d, else false.

bool refines(const Dimension &d) const

Check whether this dimension is a refinement of the argument.

A dimension d2 refines (or is a refinement of) d1 if d1 and d2 are static and equal, or d2 is dynamic.

d1.refines(d2) is equivalent to d2.relaxes(d1).

Parameters

d – The dimension to compare this dimension with.

Returns

true if this dimension relaxes d, else false.

Dimension operator+(const Dimension &dim) const

Addition operator for Dimension.

Parameters

dim – Right operand for addition.

Returns

Smallest interval dimension enclosing inputs

Dimension operator-(const Dimension &dim) const

Subtraction operator for Dimension.

Parameters

dim – Right operand for subtraction.

Returns

Smallest interval dimension enclosing inputs

Dimension operator/(const value_type divisor) const

Division operator for Dimension divided by a value_type parameter.

Parameters

divisor – Right operand for division.

Returns

Smallest interval dimension enclosing inputs

inline Dimension &operator/=(const value_type divisor)

Divided-into operator for Dimension.

Parameters

divisor – Right operand for multiplication.

Returns

A reference to *this, after updating *this to the value *this * dim.

Dimension operator*(const Dimension &dim) const

Multiplication operator for Dimension.

Parameters

dim – Right operand for multiplicaiton.

Returns

Smallest interval containing all “produces” which are 0 if either of this or dim has length 0, else unbounded if either is unbounded, else product of lengths.

inline Dimension &operator+=(const Dimension &dim)

Add-into operator for Dimension.

Parameters

dim – Right operand for addition.

Returns

A reference to *this, after updating *this to the value *this + dim.

inline Dimension &operator*=(const Dimension &dim)

Multiply-into operator for Dimension.

Parameters

dim – Right operand for multiplication.

Returns

A reference to *this, after updating *this to the value *this * dim.

Dimension operator&(const Dimension &dim) const

Intersection of dimensions.

Dimension &operator&=(const Dimension &dim)

Intersection of dimensions.

std::string to_string() const

String representation of Dimension.

Public Static Functions

static bool merge(Dimension &dst, const Dimension &d1, const Dimension &d2)

Try to merge two Dimension objects together.

  • If d1 is dynamic, writes d2 to dst and returns true.

  • If d2 is dynamic, writes d1 to dst and returns true.

  • If d1 and d2 are static and equal, writes d1 to dst and returns true.

  • If d1 and d2 are both static and unequal, leaves dst unchanged and returns false.

Parameters
  • dst[out] Reference to write the merged Dimension into.

  • d1 – First dimension to merge.

  • d2 – Second dimension to merge.

Returns

true if merging succeeds, else false.

static bool broadcast_merge(Dimension &dst, const Dimension &d1, const Dimension &d2)

Try to merge two Dimension objects together with implicit broadcasting of unit-sized dimension to non unit-sized dimension.

static inline Dimension dynamic()

Create a dynamic dimension.

Returns

A dynamic dimension.

Friends

inline friend void swap(Dimension &a, Dimension &b)

Swap of dimensions.

template<typename AT>
class DirectValueAccessor : public ov::ValueAccessor<AT>
#include <attribute_adapter.hpp>

Public Functions

inline virtual const AT &get() override

Returns the value.

inline virtual void set(const AT &value) override

Sets the value.

class DisableCleanupAttribute : public ov::RuntimeAttribute
#include <disable_cleanup_attribute.hpp>
class DisableFP16Compression : public ov::RuntimeAttribute
#include <disable_fp16_compression.hpp>

DisableFP16Compression class represents runtime info attribute that marks operation as prohibited to convert to lower precision (e.g. to FP16) and they should be inferred precisely in the original precision.

struct DiscreteTypeInfo
#include <type.hpp>

Type information for a type system without inheritance; instances have exactly one type not related to any other type.

Supports three functions, ov::is_type<Type>, ov::as_type<Type>, and ov::as_type_ptr<Type> for type-safe dynamic conversions via static_cast/static_ptr_cast without using C++ RTTI. Type must have a static type_info member and a virtual get_type_info() member that returns a reference to its type_info member.

template<element::Type_t>
struct element_type_traits
#include <element_type_traits.hpp>
template<> bf16 >
#include <element_type_traits.hpp>
template<> boolean >
#include <element_type_traits.hpp>
template<> f16 >
#include <element_type_traits.hpp>
template<> f32 >
#include <element_type_traits.hpp>
template<> f64 >
#include <element_type_traits.hpp>
template<> i16 >
#include <element_type_traits.hpp>
template<> i32 >
#include <element_type_traits.hpp>
template<> i4 >
#include <element_type_traits.hpp>
template<> i64 >
#include <element_type_traits.hpp>
template<> i8 >
#include <element_type_traits.hpp>
template<> nf4 >
#include <element_type_traits.hpp>
template<> string >
#include <element_type_traits.hpp>
template<> u1 >
#include <element_type_traits.hpp>
template<> u16 >
#include <element_type_traits.hpp>
template<> u32 >
#include <element_type_traits.hpp>
template<> u4 >
#include <element_type_traits.hpp>
template<> u64 >
#include <element_type_traits.hpp>
template<> u8 >
#include <element_type_traits.hpp>
template<typename AT>
class EnumAttributeAdapterBase : public ov::ValueAccessor<std::string>
#include <attribute_adapter.hpp>

Access an enum via a string.

Template Parameters

AT – The attribute type enum class

Public Functions

inline virtual const std::string &get() override

Returns the value.

inline virtual void set(const std::string &value) override

Sets the value.

template<typename T>
class EnumMask
#include <enum_mask.hpp>

Public Types

using value_type = typename std::underlying_type<T>::type

Make sure the template type is an enum.

Extract the underlying type of the enum.

Public Functions

constexpr EnumMask() = default

Some bit operations are not safe for signed values, we require enum type to use unsigned underlying type.

inline bool is_any_set(const EnumMask &p) const

Check if any of the input parameter enum bit mask match.

inline bool is_set(const EnumMask &p) const

Check if all of the input parameter enum bit mask match.

inline bool is_any_clear(const EnumMask &p) const

Check if any of the input parameter enum bit mask does not match.

inline bool is_clear(const EnumMask &p) const

Check if all of the input parameter enum bit mask do not match.

template<typename EnumType>
class EnumNames
#include <enum_names.hpp>

Uses a pairings defined by EnumTypes::get() to convert between strings and enum values.

Public Static Functions

static inline EnumType as_enum(const std::string &name)

Converts strings to enum values.

static inline const std::string &as_string(EnumType e)

Converts enum values to strings.

class Exception : public std::runtime_error
#include <except.hpp>

Base error for ov runtime errors.

Subclassed by ngraph::builder::numpy_autobroadcast_incompatible_shapes, ov::AssertFailure, ov::Busy, ov::Cancelled, ov::op::util::error::UnknownActivationFunction

class Extension
#include <extension.hpp>

The class provides the base interface for OpenVINO extensions.

Subclassed by ov::BaseOpExtension, ov::frontend::ConversionExtensionBase, ov::frontend::DecoderTransformationExtension, ov::frontend::ProgressReporterExtension, ov::frontend::TelemetryExtension

class float16
#include <float16.hpp>
class FusedNames : public ov::RuntimeAttribute
#include <fused_names_attribute.hpp>

FusedName class represents runtime info attribute that stores all operation names that was fully or partially fused into node.

Public Functions

FusedNames() = default

A default constructor

inline explicit FusedNames(const std::string &name)

Constructs a new object consisting of a single name *.

Parameters

name[in] The name

void fuseWith(const FusedNames &names)

Unites current set of already fused names with another FusedNames object.

Parameters

names[in] Another object to fuse with

std::string getNames() const

return string with operation names separated by coma in alphabetical order

std::vector<std::string> getVectorNames() const

return vector of fused names sorted in alphabetical order

Returns

vector if strings

class IAsyncInferRequest : public ov::IInferRequest
#include <iasync_infer_request.hpp>

Base class with default implementation of asynchronous multi staged inference request. To customize pipeline stages derived class should change the content of IAsyncInferRequest::m_pipeline member container. It consists of pairs of tasks and executors which will run the task. The class is recommended to be used by plugins as a base class for asynchronous inference request implementation.

Example

Here is an example of asynchronous inference request implementation for some accelerator device. It uses 5 different executors to run different stages of a synchronous inference request.

Note

To synchronize derived context with stages derived class should call IAsyncInferRequest::stop_and_wait() function in destructor.

Public Functions

virtual void start_async()

Start inference of specified input(s) in asynchronous mode.

Note

The method returns immediately. Inference starts also immediately.

virtual void wait()

Waits for the result to become available.

virtual bool wait_for(const std::chrono::milliseconds &timeout)

Waits for the result to become available. Blocks until specified timeout has elapsed or the result becomes available, whichever comes first.

Parameters

timeout – - maximum duration in milliseconds to block for

Returns

A true if results are ready.

virtual void cancel()

Cancel current inference request execution.

virtual void set_callback(std::function<void(std::exception_ptr)> callback)

Set callback function which will be called on success or failure of asynchronous request.

Parameters

callback – - function to be called with the following description:

virtual void infer() override

Infers specified input(s) in synchronous mode.

Note

blocks all method of InferRequest while request is ongoing (running or waiting in queue)

virtual std::vector<ov::ProfilingInfo> get_profiling_info() const override

Queries performance measures per layer to identify the most time consuming operation.

Note

Not all plugins provide meaningful data.

Returns

Vector of profiling information for operations in a model.

virtual ov::SoPtr<ov::ITensor> get_tensor(const ov::Output<const ov::Node> &port) const override

Gets an input/output tensor for inference.

Note

If the tensor with the specified port is not found, an exception is thrown.

Parameters

port – Port of the tensor to get.

Returns

Tensor for the port port.

virtual void set_tensor(const ov::Output<const ov::Node> &port, const ov::SoPtr<ov::ITensor> &tensor) override

Sets an input/output tensor to infer.

Parameters
  • port – Port of the input or output tensor.

  • tensor – Reference to a tensor. The element_type and shape of a tensor must match the model’s input/output element_type and size.

virtual std::vector<ov::SoPtr<ov::ITensor>> get_tensors(const ov::Output<const ov::Node> &port) const override

Gets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of tensors must match the batch size. The current version supports setting tensors to model inputs only. If port is associated with output (or any other non-input node), an exception is thrown.

Parameters
  • port – Port of the input tensor.

  • tensorsInput tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.

Returns

vector of tensors

virtual void set_tensors(const ov::Output<const ov::Node> &port, const std::vector<ov::SoPtr<ov::ITensor>> &tensors) override

Sets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of tensors must match the batch size. The current version supports setting tensors to model inputs only. If port is associated with output (or any other non-input node), an exception is thrown.

Parameters
  • port – Port of the input tensor.

  • tensorsInput tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.

virtual std::vector<ov::SoPtr<ov::IVariableState>> query_state() const override

Gets state control interface for the given infer request.

State control essential for recurrent models.

Returns

Vector of Variable State objects.

virtual const std::shared_ptr<const ov::ICompiledModel> &get_compiled_model() const override

Gets pointer to compiled model (usually synchronous request holds the compiled model)

Returns

Pointer to the compiled model

virtual const std::vector<ov::Output<const ov::Node>> &get_inputs() const override

Gets inputs for infer request.

Returns

vector of input ports

virtual const std::vector<ov::Output<const ov::Node>> &get_outputs() const override

Gets outputs for infer request.

Returns

vector of output ports

class ICompiledModel : public std::enable_shared_from_this<ICompiledModel>
#include <icompiled_model.hpp>

OpenVINO ICompiledModel interface.

Public Functions

ICompiledModel(const std::shared_ptr<const ov::Model> &model, const std::shared_ptr<const ov::IPlugin> &plugin, const std::shared_ptr<ov::threading::ITaskExecutor> &task_executor = std::make_shared<ov::threading::CPUStreamsExecutor>(ov::threading::IStreamsExecutor::Config{"Default"}), const std::shared_ptr<ov::threading::ITaskExecutor> &callback_executor = std::make_shared<ov::threading::CPUStreamsExecutor>(ov::threading::IStreamsExecutor::Config{"Callback"}))

Constructor for ICompiledModel interface.

Parameters
  • model – OpenVINO model representation

  • plugin – Pointer to plugin

  • task_executor – Task executor (CPUStreamsExecutor by default)

  • callback_executor – Callback executor (CPUStreamsExecutor by default)

ICompiledModel(const std::shared_ptr<const ov::Model> &model, const std::shared_ptr<const ov::IPlugin> &plugin, const ov::SoPtr<ov::IRemoteContext> &context, const std::shared_ptr<ov::threading::ITaskExecutor> &task_executor = std::make_shared<ov::threading::CPUStreamsExecutor>(ov::threading::IStreamsExecutor::Config{"Default"}), const std::shared_ptr<ov::threading::ITaskExecutor> &callback_executor = std::make_shared<ov::threading::CPUStreamsExecutor>(ov::threading::IStreamsExecutor::Config{"Callback"}))

Constructor for ICompiledModel interface with remote context.

Parameters
  • model – OpenVINO model representation

  • plugin – Pointer to plugin

  • context – Remote context

  • task_executor – Task executor (CPUStreamsExecutor by default)

  • callback_executor – Callback executor (CPUStreamsExecutor by default)

virtual const std::vector<ov::Output<const ov::Node>> &outputs() const

Gets all outputs from compiled model.

Returns

model outputs

virtual const std::vector<ov::Output<const ov::Node>> &inputs() const

Gets all inputs from compiled model.

Returns

model inputs

virtual std::shared_ptr<ov::IAsyncInferRequest> create_infer_request() const

Create infer request.

Returns

Asynchronous infer request interface

virtual void export_model(std::ostream &model) const = 0

Export compiled model to stream.

Parameters

model – output stream

virtual std::shared_ptr<const ov::Model> get_runtime_model() const = 0

Returns runtime model.

Returns

OpenVINO Model which represents runtime graph

virtual void set_property(const ov::AnyMap &properties) = 0

Allows to set property.

Parameters

properties – new plugin properties

virtual ov::Any get_property(const std::string &name) const = 0

Returns property.

Parameters

nameProperty name

Returns

Property value

ov::SoPtr<ov::IRemoteContext> get_context() const

Creates device specific remote context.

Returns

OpenVINO RemoteContext

interface ICore
#include <icore.hpp>

Minimal ICore interface to allow plugin to get information from Core OpenVINO class.

Subclassed by InferenceEngine::ICore

Public Functions

virtual std::shared_ptr<ov::Model> read_model(const std::string &model, const ov::Tensor &weights, bool frontend_mode = false) const = 0

Reads IR xml and bin (with the same name) files.

Parameters
  • model – string with IR

  • weights – shared pointer to constant blob with weights

  • frontend_mode – read network without post-processing or other transformations

Returns

shared pointer to ov::Model

virtual std::shared_ptr<ov::Model> read_model(const std::string &model_path, const std::string &bin_path) const = 0

Reads IR xml and bin files.

Parameters
  • model_path – path to IR file

  • bin_path – path to bin file, if path is empty, will try to read bin file with the same name as xml and if bin file with the same name was not found, will load IR without weights.

Returns

shared pointer to ov::Model

virtual ov::SoPtr<ov::ICompiledModel> compile_model(const std::shared_ptr<const ov::Model> &model, const std::string &device_name, const ov::AnyMap &config = {}) const = 0

Creates a compiled mdel from a model object.

Users can create as many models as they need and use them simultaneously (up to the limitation of the hardware resources)

Parameters
  • model – OpenVINO Model

  • device_name – Name of device to load model to

  • config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation

Returns

A pointer to compiled model

virtual ov::SoPtr<ov::ICompiledModel> compile_model(const std::shared_ptr<const ov::Model> &model, const ov::SoPtr<ov::IRemoteContext> &context, const ov::AnyMap &config = {}) const = 0

Creates a compiled model from a model object.

Users can create as many models as they need and use them simultaneously (up to the limitation of the hardware resources)

Parameters
  • model – OpenVINO Model

  • context – “Remote” (non-CPU) accelerator device-specific execution context to use

  • config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation

Returns

A pointer to compiled model

virtual ov::SoPtr<ov::ICompiledModel> compile_model(const std::string &model_path, const std::string &device_name, const ov::AnyMap &config) const = 0

Creates a compiled model from a model file.

Users can create as many models as they need and use them simultaneously (up to the limitation of the hardware resources)

Parameters
  • model_path – Path to model

  • device_name – Name of device to load model to

  • config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation

Returns

A pointer to compiled model

virtual ov::SoPtr<ov::ICompiledModel> compile_model(const std::string &model_str, const ov::Tensor &weights, const std::string &device_name, const ov::AnyMap &config) const = 0

Creates a compiled model from a model memory.

Users can create as many models as they need and use them simultaneously (up to the limitation of the hardware resources)

Parameters
  • model_str – String data of model

  • weightsModel’s weights

  • device_name – Name of device to load model to

  • config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation

Returns

A pointer to compiled model

virtual ov::SoPtr<ov::ICompiledModel> import_model(std::istream &model, const std::string &device_name, const ov::AnyMap &config = {}) const = 0

Creates a compiled model from a previously exported model.

Parameters
  • model – model stream

  • device_name – Name of device load executable model on

  • config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation*

Returns

A pointer to compiled model

virtual ov::SoPtr<ov::ICompiledModel> import_model(std::istream &modelStream, const ov::SoPtr<ov::IRemoteContext> &context, const ov::AnyMap &config = {}) const = 0

Creates a compiled model from a previously exported model.

Parameters
  • model – model stream

  • context – Remote context

  • config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation*

Returns

A pointer to compiled model

virtual ov::SupportedOpsMap query_model(const std::shared_ptr<const ov::Model> &model, const std::string &device_name, const ov::AnyMap &config) const = 0

Query device if it supports specified network with specified configuration.

Parameters
  • model – OpenVINO Model

  • device_name – A name of a device to query

  • config – Optional map of pairs: (config parameter name, config parameter value)

Returns

An object containing a map of pairs a layer name -> a device name supporting this layer.

virtual std::vector<std::string> get_available_devices() const = 0

Returns devices available for neural networks inference.

Returns

A vector of devices. The devices are returned as { CPU, GPU.0, GPU.1, MYRIAD } If there more than one device of specific type, they are enumerated with .# suffix.

virtual ov::SoPtr<ov::IRemoteContext> create_context(const std::string &device_name, const AnyMap &args) const = 0

Create a new shared context object on specified accelerator device using specified plugin-specific low level device API parameters (device handle, pointer, etc.)

Parameters
  • device_name – Name of a device to create new shared context on.

  • params – Map of device-specific shared context parameters.

Returns

A shared pointer to a created remote context.

virtual ov::SoPtr<ov::IRemoteContext> get_default_context(const std::string &device_name) const = 0

Get a pointer to default shared context object for the specified device.

Parameters

device_name – - A name of a device to get create shared context from.

Returns

A shared pointer to a default remote context.

virtual Any get_property(const std::string &device_name, const std::string &name, const AnyMap &arguments) const = 0

Gets properties related to device behaviour.

Parameters
  • device_name – Name of a device to get a property value.

  • nameProperty name.

  • arguments – Additional arguments to get a property.

Returns

Value of a property corresponding to the property name.

template<typename T, PropertyMutability M>
inline T get_property(const std::string &device_name, const Property<T, M> &property) const

Gets properties related to device behaviour.

Template Parameters
  • T – Type of a returned value.

  • MProperty mutability.

Parameters
  • deviceName – Name of a device to get a property value.

  • propertyProperty object.

Returns

Property value.

template<typename T, PropertyMutability M>
inline T get_property(const std::string &device_name, const Property<T, M> &property, const AnyMap &arguments) const

Gets properties related to device behaviour.

Template Parameters
  • T – Type of a returned value.

  • MProperty mutability.

Parameters
  • deviceName – Name of a device to get a property value.

  • propertyProperty object.

  • arguments – Additional arguments to get a property.

Returns

Property value.

virtual AnyMap get_supported_property(const std::string &full_device_name, const AnyMap &properties) const = 0

Get only properties that are supported by specified device.

Parameters
  • full_device_name – Name of a device (can be either virtual or hardware)

  • properties – Properties that can contains configs that are not supported by device

Returns

map of properties that are supported by device

virtual ~ICore()

Default virtual destructor.

class IInferRequest
#include <iinfer_request.hpp>

An internal API of inference request to be implemented by plugin.

Subclassed by ov::IAsyncInferRequest, ov::ISyncInferRequest

Public Functions

virtual void infer() = 0

Infers specified input(s) in synchronous mode.

Note

blocks all method of InferRequest while request is ongoing (running or waiting in queue)

virtual std::vector<ov::ProfilingInfo> get_profiling_info() const = 0

Queries performance measures per layer to identify the most time consuming operation.

Note

Not all plugins provide meaningful data.

Returns

Vector of profiling information for operations in a model.

virtual ov::SoPtr<ov::ITensor> get_tensor(const ov::Output<const ov::Node> &port) const = 0

Gets an input/output tensor for inference.

Note

If the tensor with the specified port is not found, an exception is thrown.

Parameters

port – Port of the tensor to get.

Returns

Tensor for the port port.

virtual void set_tensor(const ov::Output<const ov::Node> &port, const ov::SoPtr<ov::ITensor> &tensor) = 0

Sets an input/output tensor to infer.

Parameters
  • port – Port of the input or output tensor.

  • tensor – Reference to a tensor. The element_type and shape of a tensor must match the model’s input/output element_type and size.

virtual std::vector<ov::SoPtr<ov::ITensor>> get_tensors(const ov::Output<const ov::Node> &port) const = 0

Gets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of tensors must match the batch size. The current version supports setting tensors to model inputs only. If port is associated with output (or any other non-input node), an exception is thrown.

Parameters
  • port – Port of the input tensor.

  • tensorsInput tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.

Returns

vector of tensors

virtual void set_tensors(const ov::Output<const ov::Node> &port, const std::vector<ov::SoPtr<ov::ITensor>> &tensors) = 0

Sets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of tensors must match the batch size. The current version supports setting tensors to model inputs only. If port is associated with output (or any other non-input node), an exception is thrown.

Parameters
  • port – Port of the input tensor.

  • tensorsInput tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.

virtual std::vector<ov::SoPtr<ov::IVariableState>> query_state() const = 0

Gets state control interface for the given infer request.

State control essential for recurrent models.

Returns

Vector of Variable State objects.

virtual const std::shared_ptr<const ov::ICompiledModel> &get_compiled_model() const = 0

Gets pointer to compiled model (usually synchronous request holds the compiled model)

Returns

Pointer to the compiled model

virtual const std::vector<ov::Output<const ov::Node>> &get_inputs() const = 0

Gets inputs for infer request.

Returns

vector of input ports

virtual const std::vector<ov::Output<const ov::Node>> &get_outputs() const = 0

Gets outputs for infer request.

Returns

vector of output ports

template<typename AT, typename VAT>
class IndirectScalarValueAccessor : public ov::ValueAccessor<VAT>
#include <attribute_adapter.hpp>

Public Functions

inline virtual const VAT &get() override

Returns the value.

inline virtual void set(const VAT &value) override

Sets the value.

template<typename AT, typename VAT>
class IndirectVectorValueAccessor : public ov::ValueAccessor<VAT>
#include <attribute_adapter.hpp>

Public Functions

inline virtual const VAT &get() override

Returns the value.

inline virtual void set(const VAT &value) override

Sets the value.

class InferRequest
#include <infer_request.hpp>

This is a class of infer request that can be run in asynchronous or synchronous manners.

Public Functions

InferRequest() = default

Default constructor.

InferRequest(const InferRequest &other) = default

Default copy constructor.

Parameters

other – Another InferRequest object.

InferRequest &operator=(const InferRequest &other) = default

Default copy assignment operator.

Parameters

other – Another InferRequest object.

Returns

Reference to the current object.

InferRequest(InferRequest &&other) = default

Default move constructor.

Parameters

other – Another InferRequest object.

InferRequest &operator=(InferRequest &&other) = default

Default move assignment operator.

Parameters

other – Another InferRequest object.

Returns

Reference to the current object.

~InferRequest()

Destructor that preserves unloading order of implementation object and reference to the library.

Note

To preserve destruction order inside the default generated assignment operator, _impl is stored before _so. Use the destructor to remove implementation object before referencing to the library explicitly.

void set_tensor(const std::string &tensor_name, const Tensor &tensor)

Sets an input/output tensor to infer on.

Parameters
  • tensor_name – Name of the input or output tensor.

  • tensor – Reference to the tensor. The element_type and shape of the tensor must match the model’s input/output element_type and size.

void set_tensor(const ov::Output<const ov::Node> &port, const Tensor &tensor)

Sets an input/output tensor to infer.

Parameters
void set_tensor(const ov::Output<ov::Node> &port, const Tensor &tensor)

Sets an input/output tensor to infer.

Parameters
void set_tensors(const std::string &tensor_name, const std::vector<Tensor> &tensors)

Sets a batch of tensors for input data to infer by tensor name. Model input must have batch dimension, and the number of tensors must match the batch size. The current version supports setting tensors to model inputs only. If tensor_name is associated with output (or any other non-input node), an exception is thrown.

Parameters
  • tensor_name – Name of the input tensor.

  • tensorsInput tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.

void set_tensors(const ov::Output<const ov::Node> &port, const std::vector<Tensor> &tensors)

Sets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of tensors must match the batch size. The current version supports setting tensors to model inputs only. If port is associated with output (or any other non-input node), an exception is thrown.

Parameters
  • port – Port of the input tensor.

  • tensorsInput tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.

void set_input_tensor(size_t idx, const Tensor &tensor)

Sets an input tensor to infer.

Parameters
  • idx – Index of the input tensor. If idx is greater than the number of model inputs, an exception is thrown.

  • tensor – Reference to the tensor. The element_type and shape of the tensor must match the model’s input/output element_type and size.

void set_input_tensor(const Tensor &tensor)

Sets an input tensor to infer models with single input.

Note

If model has several inputs, an exception is thrown.

Parameters

tensor – Reference to the input tensor.

void set_input_tensors(const std::vector<Tensor> &tensors)

Sets a batch of tensors for single input data. Model input must have batch dimension, and the number of tensors must match the batch size.

Parameters

tensorsInput tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.

void set_input_tensors(size_t idx, const std::vector<Tensor> &tensors)

Sets a batch of tensors for input data to infer by input name. Model input must have batch dimension, and number of tensors must match the batch size.

Parameters
  • idx – Name of the input tensor.

  • tensorsInput tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.

void set_output_tensor(size_t idx, const Tensor &tensor)

Sets an output tensor to infer.

Note

Index of the input preserved accross ov::Model, ov::CompiledModel, and ov::InferRequest.

Parameters
  • idx – Index of the output tensor.

  • tensor – Reference to the output tensor. The type of the tensor must match the model output element type and shape.

void set_output_tensor(const Tensor &tensor)

Sets an output tensor to infer models with single output.

Note

If model has several outputs, an exception is thrown.

Parameters

tensor – Reference to the output tensor.

Tensor get_tensor(const std::string &tensor_name)

Gets an input/output tensor for inference by tensor name.

Parameters

tensor_name – Name of a tensor to get.

Returns

The tensor with name tensor_name. If the tensor is not found, an exception is thrown.

Tensor get_tensor(const ov::Output<const ov::Node> &port)

Gets an input/output tensor for inference.

Note

If the tensor with the specified port is not found, an exception is thrown.

Parameters

port – Port of the tensor to get.

Returns

Tensor for the port port.

Tensor get_tensor(const ov::Output<ov::Node> &port)

Gets an input/output tensor for inference.

Note

If the tensor with the specified port is not found, an exception is thrown.

Parameters

port – Port of the tensor to get.

Returns

Tensor for the port port.

Tensor get_input_tensor(size_t idx)

Gets an input tensor for inference.

Parameters

idx – Index of the tensor to get.

Returns

Tensor with the input index idx. If the tensor with the specified idx is not found, an exception is thrown.

Tensor get_input_tensor()

Gets an input tensor for inference.

Returns

The input tensor for the model. If model has several inputs, an exception is thrown.

Tensor get_output_tensor(size_t idx)

Gets an output tensor for inference.

Parameters

idx – Index of the tensor to get.

Returns

Tensor with the output index idx. If the tensor with the specified idx is not found, an exception is thrown.

Tensor get_output_tensor()

Gets an output tensor for inference.

Returns

Output tensor for the model. If model has several outputs, an exception is thrown.

void infer()

Infers specified input(s) in synchronous mode.

Note

It blocks all methods of InferRequest while request is ongoing (running or waiting in a queue). Calling any method leads to throwning the ov::Busy exception.

void cancel()

Cancels inference request.

std::vector<ProfilingInfo> get_profiling_info() const

Queries performance measures per layer to identify the most time consuming operation.

Note

Not all plugins provide meaningful data.

Returns

Vector of profiling information for operations in a model.

void start_async()

Starts inference of specified input(s) in asynchronous mode.

Note

It returns immediately. Inference starts also immediately. Calling any method while the request in a running state leads to throwning the ov::Busy exception.

void wait()

Waits for the result to become available. Blocks until the result becomes available.

bool wait_for(const std::chrono::milliseconds timeout)

Waits for the result to become available. Blocks until the specified timeout has elapsed or the result becomes available, whichever comes first.

Parameters

timeout – Maximum duration, in milliseconds, to block for.

Returns

True if inference request is ready and false, otherwise.

void set_callback(std::function<void(std::exception_ptr)> callback)

Sets a callback std::function that is called on success or failure of an asynchronous request.

Warning

Do not capture strong references to OpenVINO runtime objects into callback. Following objects should not be captured like:

  • ov::InferRequest

  • ov::ExecutableNetwork

  • ov::Core As specified objects implement shared reference concept do not capture this objects by value. It can lead to memory leaks or undefined behaviour! Try to use weak references or pointers.

Parameters

callback – callback object which will be called on when inference finish.

std::vector<VariableState> query_state()

Gets state control interface for the given infer request.

State control essential for recurrent models.

Returns

Vector of Variable State objects.

void reset_state()

Resets all internal variable states for relevant infer request to a value specified as default for the corresponding ReadValue node.

CompiledModel get_compiled_model()

Returns a compiled model that creates this inference request.

Returns

Compiled model object.

bool operator!() const noexcept

Checks if the current InferRequest object is not initialized.

Returns

True if the current InferRequest object is not initialized; false, otherwise.

explicit operator bool() const noexcept

Checks if the current InferRequest object is initialized.

Returns

True if the current InferRequest object is initialized; false, otherwise.

bool operator!=(const InferRequest &other) const noexcept

Compares whether this request wraps the same impl underneath.

Parameters

other – Another inference request.

Returns

True if the current InferRequest object does not wrap the same impl as the operator’s arg.

bool operator==(const InferRequest &other) const noexcept

Compares whether this request wraps the same impl underneath.

Parameters

other – Another inference request.

Returns

True if the current InferRequest object wraps the same impl as the operator’s arg.

template<typename NodeType>
class Input
#include <node_input.hpp>
template<>
class Input<const Node>
#include <node_input.hpp>

A handle for one of a node’s inputs.

Public Functions

Input(const Node *node, size_t index)

Constructs a Input.

Parameters
  • node – Pointer to the node for the input handle.

  • index – The index of the input.

const Node *get_node() const
Returns

A pointer to the node referenced by this input handle.

size_t get_index() const
Returns

The index of the input referred to by this input handle.

const element::Type &get_element_type() const
Returns

The element type of the input referred to by this input handle.

const Shape &get_shape() const
Returns

The shape of the input referred to by this input handle.

const PartialShape &get_partial_shape() const
Returns

The partial shape of the input referred to by this input handle.

Output<Node> get_source_output() const
Returns

A handle to the output that is connected to this input.

descriptor::Tensor &get_tensor() const
Returns

A reference to the tensor descriptor for this input.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
Returns

A shared pointer to the tensor descriptor for this input.

bool get_is_relevant_to_shapes() const
Returns

true if this input is relevant to its node’s output shapes; else false.

bool get_is_relevant_to_values() const
Returns

true if this input is relevant to its node’s output values; else false.

const RTMap &get_rt_info() const
Returns

The constant reference to runtime info map

template<>
class Input<Node>
#include <node_input.hpp>

A handle for one of a node’s inputs.

Public Functions

Input(Node *node, size_t index)

Constructs a Input.

Parameters
  • node – Pointer to the node for the input handle.

  • index – The index of the input.

Node *get_node() const
Returns

A pointer to the node referenced by this input handle.

size_t get_index() const
Returns

The index of the input referred to by this input handle.

const element::Type &get_element_type() const
Returns

The element type of the input referred to by this input handle.

const Shape &get_shape() const
Returns

The shape of the input referred to by this input handle.

const PartialShape &get_partial_shape() const
Returns

The partial shape of the input referred to by this input handle.

Output<Node> get_source_output() const
Returns

A handle to the output that is connected to this input.

descriptor::Tensor &get_tensor() const
Returns

A reference to the tensor descriptor for this input.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
Returns

A shared pointer to the tensor descriptor for this input.

bool get_is_relevant_to_shapes() const
Returns

true if this input is relevant to its node’s output shapes; else false.

bool get_is_relevant_to_values() const
Returns

true if this input is relevant to its node’s output values; else false.

void replace_source_output(const Output<Node> &new_source_output) const

Replaces the source output of this input.

Parameters

new_source_output – A handle for the output that will replace this input’s source.

RTMap &get_rt_info()
Returns

The reference to runtime info map

const RTMap &get_rt_info() const
Returns

The constant reference to runtime info map

class Interval
#include <interval.hpp>

Interval arithmetic.

An interval is the set of integers from m_min_val through m_max_val. The value s_max acts like infinity. The addition, subtraction, or multiplication of intervals is the smallest interval containing the sums, differences, or products of elements of the two intervals. An empty interval is canonicalized to [s_max, s_max].

Public Functions

Interval() = default

Interval of everything.

Interval(const Interval &interval) = default

Copy constructor.

Interval(value_type min_val, value_type max_val)

Closed interval {x|min_val <= x <= max_val}.

Interval(value_type val)

Single-valued interval; just contains val.

inline size_type size() const

The number of elements in the interval. Zero if max < min.

inline bool empty() const

Returns true if the interval has no elements.

inline value_type get_min_val() const

the inclusive lower bound of the interval

inline void set_min_val(value_type val)

Set the inclusive lower bound of the interval.

inline value_type get_max_val() const

the inclusive upper bound of the interval

inline void set_max_val(value_type val)

Set the inclusive upper bound of the interval.

inline bool has_upper_bound() const

True if the upper bound is finite.

bool operator==(const Interval &interval) const

True if min and max bounds match.

Interval operator+(const Interval &interval) const

The interval whose elements are a sum of an element from each interval.

Interval &operator+=(const Interval &interval)

Extend this interval to sums of elements in this interval and interval.

Interval operator-(const Interval &interval) const

The interval whose elements are a difference of an element from each interval.

Interval &operator-=(const Interval &interval)

Extend this interval to differences of elements in this interval and interval.

Interval operator*(const Interval &interval) const

The smallest interval whose elements are a product of an element from each interval.

Interval &operator*=(const Interval &interval)

Extend this interval to products of elements in this interval and interval.

Interval operator&(const Interval &interval) const

The interval that is the intersection of this interval and interval.

Interval &operator&=(const Interval &interval)

Change this interval to only include elements also in interval.

inline bool contains(value_type value) const

True if this interval includes value.

bool contains(const Interval &interval) const

True if this interval includes all the values in interval.

Public Static Attributes

static constexpr value_type s_max = {std::numeric_limits<value_type>::max()}

The value used for no upper bound.

class IntervalsAlignmentAttribute : public SharedAttribute<IntervalsAlignmentSharedValue>
#include <intervals_alignment_attribute.hpp>

IntervalsAlignmentAttribute defines subgraph with the same quantization intervals alignment. FakeQuantize operations are included. The attribute is used by quantization operations.

For more details about the attribute, refer to IntervalsAlignmentAttribute page in the Inference Engine Developer Guide.

class IntervalsAlignmentSharedValue
#include <intervals_alignment_attribute.hpp>

IntervalsAlignmentSharedValue is used by IntervalsAlignmentAttribute as attribute shared value.

class Interval
#include <intervals_alignment_attribute.hpp>
class IPlugin : public std::enable_shared_from_this<IPlugin>
#include <iplugin.hpp>

OpenVINO Plugin Interface 2.0.

Public Functions

void set_version(const Version &version)

Sets a plugin version.

Parameters

version – A version to set

const Version &get_version() const

Returns a plugin version.

Returns

A constant ov::Version object

virtual std::shared_ptr<ov::ICompiledModel> compile_model(const std::shared_ptr<const ov::Model> &model, const ov::AnyMap &properties) const = 0

Compiles model from ov::Model object.

Parameters
  • model – A model object acquired from ov::Core::read_model or source construction

  • properties – A ov::AnyMap of properties relevant only for this load operation

Returns

Created Compiled Model object

virtual std::shared_ptr<ov::ICompiledModel> compile_model(const std::string &model_path, const ov::AnyMap &properties) const

Compiles model from ov::Model object.

Parameters
  • model_path – A path to model (path can be converted from unicode representation)

  • properties – A ov::AnyMap of properties relevant only for this load operation

Returns

Created Compiled Model object

virtual std::shared_ptr<ov::ICompiledModel> compile_model(const std::shared_ptr<const ov::Model> &model, const ov::AnyMap &properties, const ov::SoPtr<ov::IRemoteContext> &context) const = 0

Compiles model from ov::Model object, on specified remote context.

Parameters
  • model – A model object acquired from ov::Core::read_model or source construction

  • properties – A ov::AnyMap of properties relevant only for this load operation

  • context – A pointer to plugin context derived from RemoteContext class used to execute the model

Returns

Created Compiled Model object

virtual void set_property(const ov::AnyMap &properties) = 0

Sets properties for plugin, acceptable keys can be found in openvino/runtime/properties.hpp.

Parameters

properties – ov::AnyMap of properties

virtual ov::Any get_property(const std::string &name, const ov::AnyMap &arguments) const = 0

Gets properties related to plugin behaviour.

Parameters
  • nameProperty name.

  • arguments – Additional arguments to get a property.

Returns

Value of a property corresponding to the property name.

virtual ov::SoPtr<ov::IRemoteContext> create_context(const ov::AnyMap &remote_properties) const = 0

Creates a remote context instance based on a map of properties.

Parameters

remote_properties – Map of device-specific shared context remote properties.

Returns

A remote context object

virtual ov::SoPtr<ov::IRemoteContext> get_default_context(const ov::AnyMap &remote_properties) const = 0

Provides a default remote context instance if supported by a plugin.

Parameters

remote_properties – Map of device-specific shared context remote properties.

Returns

The default context.

virtual std::shared_ptr<ov::ICompiledModel> import_model(std::istream &model, const ov::AnyMap &properties) const = 0

Creates an compiled model from an previously exported model using plugin implementation and removes OpenVINO Runtime magic and plugin name.

Parameters
  • model – Reference to model output stream

  • properties – A ov::AnyMap of properties

Returns

An Compiled model

virtual std::shared_ptr<ov::ICompiledModel> import_model(std::istream &model, const ov::SoPtr<ov::IRemoteContext> &context, const ov::AnyMap &properties) const = 0

Creates an compiled model from an previously exported model using plugin implementation and removes OpenVINO Runtime magic and plugin name.

Parameters
  • model – Reference to model output stream

  • context – A pointer to plugin context derived from RemoteContext class used to execute the network

  • properties – A ov::AnyMap of properties

Returns

An Compiled model

virtual ov::SupportedOpsMap query_model(const std::shared_ptr<const ov::Model> &model, const ov::AnyMap &properties) const = 0

Queries a plugin about supported layers in model.

Parameters
  • modelModel object to query.

  • properties – Optional map of pairs: (property name, property value).

Returns

An object containing a map of pairs an operation name -> a device name supporting this operation.

virtual OPENVINO_SUPPRESS_DEPRECATED_START void add_extension (const std::shared_ptr< InferenceEngine::IExtension > &extension)

Registers legacy extension within plugin.

Deprecated:

This method allows to load legacy Inference Engine Extensions and will be removed in 2024.0 release

Parameters

extension – - pointer to already loaded legacy extension

OPENVINO_SUPPRESS_DEPRECATED_END void set_core (const std::weak_ptr< ov::ICore > &core)

Sets pointer to ICore interface.

Parameters

core – Pointer to Core interface

std::shared_ptr<ov::ICore> get_core() const

Gets reference to ICore interface.

Returns

Reference to ICore interface

bool is_new_api() const

Provides an information about used API.

Returns

true if new API is used

const std::shared_ptr<ov::threading::ExecutorManager> &get_executor_manager() const

Gets reference to tasks execution manager.

Returns

Reference to ExecutorManager interface

class IRemoteContext : public std::enable_shared_from_this<IRemoteContext>
#include <iremote_context.hpp>

Subclassed by InferenceEngine::IRemoteContextWrapper

Public Functions

virtual const ov::AnyMap &get_property() const = 0

Returns a map of device-specific parameters required for low-level operations with underlying object. Parameters include device/context handles, access flags, etc. Contents of the map returned depend on remote execution context that is currently set on the device (working scenario). Abstract method.

Returns

A map of name/Any elements.

virtual ov::SoPtr<ov::IRemoteTensor> create_tensor(const ov::element::Type &type, const ov::Shape &shape, const ov::AnyMap &params = {}) = 0

Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.

Parameters
  • type – Defines the element type of the tensor.

  • shape – Defines the shape of the tensor.

  • params – Map of the low-level tensor object parameters.

Returns

Pointer to a plugin object that implements the RemoteTensor interface.

virtual ov::SoPtr<ov::ITensor> create_host_tensor(const ov::element::Type type, const ov::Shape &shape)

This method is used to create a host tensor object friendly for the device in current context. For example, GPU context may allocate USM host memory (if corresponding extension is available), which could be more efficient than regular host memory.

Parameters
Returns

A tensor instance with device friendly memory.

class IRemoteTensor : public ITensor
#include <iremote_tensor.hpp>

Subclassed by ov::RemoteBlobTensor

Public Functions

virtual const AnyMap &get_properties() const = 0

Returns additional information associated with tensor.

Returns

Map of property names to properties

class ISyncInferRequest : public ov::IInferRequest
#include <isync_infer_request.hpp>

Interface for syncronous infer request.

Public Functions

ISyncInferRequest(const std::shared_ptr<const ov::ICompiledModel> &compiled_model)

Constructs syncronous inference request.

Parameters

compiled_model – pointer to compiled model

virtual ov::SoPtr<ov::ITensor> get_tensor(const ov::Output<const ov::Node> &port) const override

Gets an input/output tensor for inference.

Note

If the tensor with the specified port is not found, an exception is thrown.

Parameters

port – Port of the tensor to get.

Returns

Tensor for the port port.

virtual void set_tensor(const ov::Output<const ov::Node> &port, const ov::SoPtr<ov::ITensor> &tensor) override

Sets an input/output tensor to infer.

Parameters
  • port – Port of the input or output tensor.

  • tensor – Reference to a tensor. The element_type and shape of a tensor must match the model’s input/output element_type and size.

virtual std::vector<ov::SoPtr<ov::ITensor>> get_tensors(const ov::Output<const ov::Node> &port) const override

Gets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of tensors must match the batch size. The current version supports setting tensors to model inputs only. If port is associated with output (or any other non-input node), an exception is thrown.

Parameters
  • port – Port of the input tensor.

  • tensorsInput tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.

Returns

vector of tensors

virtual void set_tensors(const ov::Output<const ov::Node> &port, const std::vector<ov::SoPtr<ov::ITensor>> &tensors) override

Sets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of tensors must match the batch size. The current version supports setting tensors to model inputs only. If port is associated with output (or any other non-input node), an exception is thrown.

Parameters
  • port – Port of the input tensor.

  • tensorsInput tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.

virtual const std::vector<ov::Output<const ov::Node>> &get_inputs() const override

Gets inputs for infer request.

Returns

vector of input ports

virtual const std::vector<ov::Output<const ov::Node>> &get_outputs() const override

Gets outputs for infer request.

Returns

vector of output ports

virtual const std::shared_ptr<const ov::ICompiledModel> &get_compiled_model() const override

Gets pointer to compiled model (usually synchronous request holds the compiled model)

Returns

Pointer to the compiled model

class ITensorAccessor
#include <tensor_data_accessor.hpp>

Interface for data accessor.

Subclassed by ov::TensorAccessor< TContainer >

Public Functions

virtual Tensor operator()(size_t port) const = 0

Get tensor at port.

Parameters

port – Number of data port (operator input) to get tensor.

Returns

Tensor to data at port.

interface IVariableState : public std::enable_shared_from_this<IVariableState>
#include <ivariable_state.hpp>

Minimal interface for variable state implementation.

Public Functions

virtual const std::string &get_name() const

Gets a variable state name.

Returns

A string representing variable state name

virtual void reset()

Reset internal variable state for relevant infer request, to a value specified as default for according ReadValue node.

virtual void set_state(const ov::SoPtr<ov::ITensor> &state)

Sets the new state for the next inference.

Parameters

newState – A new state

virtual ov::SoPtr<ov::ITensor> get_state() const

Returns the value of the variable state.

Returns

The value of the variable state

class KeepConstPrecision : public ov::RuntimeAttribute
#include <keep_const_precision.hpp>

KeepConstPrecision class represents runtime info attribute that marks a Constant as prohibitted to fuse precision in ConvertPrecision.

class Layout
#include <layout.hpp>

ov::Layout represents the text information of tensor’s dimensions/axes. E.g. layout NCHW means that 4D tensor {-1, 3, 480, 640} will have:

  • 0: N = -1: batch dimension is dynamic

  • 1: C = 3: number of channels is ‘3’

  • 2: H = 480: image height is 480

  • 3: W = 640: image width is 640

Examples: ov::Layout can be specified for:

  • Preprocessing purposes. E.g.

    • To apply normalization (means/scales) it is usually required to set ‘C’ dimension in a layout.

    • To resize the image to specified width/height it is needed to set ‘H’ and ‘W’ dimensions in a layout

    • To transpose image - source and target layout can be set (see ov::preprocess::PreProcessSteps::convert_layout)

  • To set/get model’s batch (see ov::get_batch/ov::set_batch) it is required in general to specify ‘N’ dimension in layout for appropriate inputs

Refer also to ov::layout namespace for various additional helper functions of ov::Layout

Public Functions

Layout()

Constructs a dynamic Layout with no layout information.

inline Layout(const char *layoutStr)

Constructs a Layout with static or dynamic layout information based on string representation.

Parameters

layoutStr – The string used to construct Layout from. The string representation can be in the following form:

  • can define order and meaning for dimensions “NCHW”

  • partial layout specialization:

    • ”NC?” defines 3 dimensional layout, first two NC, 3rd one is not defined

    • ”N…C” defines layout with dynamic rank where 1st dimension is N, last one is C

    • ”NC…” defines layout with dynamic rank where first two are NC, others are not defined

  • only order of dimensions “adbc” (0312)

  • Advanced syntax can be used for multi-character names like “[N,C,H,W,…,CustomName]”

bool operator==(const Layout &rhs) const

Comparison operator (equal)

bool operator!=(const Layout &rhs) const

Comparison operator (not equal)

bool has_name(const std::string &dimensionName) const

Checks if dimension with specified name is in layout.

Returns

true if layout has information about dimension index with a given name

std::int64_t get_index_by_name(const std::string &dimensionName) const

Gets index of dimension with a specified name.

Throws

ov::AssertFailure – if dimension name is not found in a layout

Returns

Index of given dimension name

std::string to_string() const

String representation of Layout.

inline bool empty() const

Returns ‘true’ if layout has no information, i.e. equals to Layout()

Public Static Functions

static Layout scalar()

Constructs layout representing scalar.

class LayoutAttribute : public ov::RuntimeAttribute
#include <layout.hpp>
class MappedMemory
#include <mmap_object.hpp>

This class represents a mapped memory. Instead of reading files, we can map the memory via mmap for Linux or MapViewOfFile for Windows. The MappedMemory class is a abstraction to handle such memory with os-dependent details.

class Mask : public std::vector<std::set<uint64_t>>, public std::enable_shared_from_this<Mask>
#include <mask_attribute.hpp>

each element in vector represents dimension and each element in set is an id of dimensions which contains zeros.

struct MemBandwidthPressure
#include <performance_heuristics.hpp>
class MemorySolver
#include <memory_solver.hpp>

Helps to solve issue of optimal memory allocation only for particular execution order.

It works with abstract data description where

  • Node is index in execution order

  • Edge is Box object with size and start-finish indexes (live time)

Example:

Mem(offset) | |____| Box {4, 5} | |_____________| Box {2, 6} | |____| Box {3, 4} | |____| Box {2, 3} | |____| Box {6, 7} |_____________________________________ 1 2 3 4 5 6 7 8 9 ExecOrder

Boxes which has an ExecOrder-axis intersection should have no Mem-axis intersections. The goal is to define a minimal required memory blob to store all boxes with such constraints and specify all corresponding position on Mem axis(through offset field).

NOTE! Exec order is predefined.

Public Functions

inline int64_t solve()

Solve memory location with maximal reuse.

Returns

Size of common memory blob required for storing all

inline int64_t get_offset(int id) const

Provides calculated offset for specified box id

inline int64_t max_depth()

Additional info. Max sum of box sizes required for any time stamp.

inline int64_t max_top_depth()

Additional info. Max num of boxes required for any time stamp.

Public Static Functions

static inline int normalize_boxes(std::vector<Box> &boxes)

Performes inplace normalization of the input boxes.

Returns

lifespan of all boxes

struct Box
#include <memory_solver.hpp>

Representation of edge (size and live time)

Public Members

int start

Execution order index of first use. The data will be produced here.

int finish

The execution order index of last use. After that data will be released. -1 is a reserved value for “till to end”. The data will be alive to very end of execution.

int64_t size

Size of data. In abstract unit of measure (byte, simd, cache line, …)

int64_t id

Box identifier, unique for each box. Will be used to querying calculated offset.

class Model : public std::enable_shared_from_this<Model>
#include <model.hpp>

A user-defined model.

Public Functions

explicit Model(const ov::OutputVector &results, const std::string &name = "")

Constructs a Model. Lists of parameters and variables will be generated automatically based on traversing the graph from the results.

Model(const ov::OutputVector &results, const ov::SinkVector &sinks, const std::string &name = "")

Constructs a Model. Lists of parameters and variables will be generated automatically based on traversing the graph from the results and the sinks.

size_t get_output_size() const

Return the number of outputs for this Model.

std::shared_ptr<ov::Node> get_output_op(size_t i) const

Return the op that generates output i.

std::shared_ptr<ov::Model> clone() const

Clones the original model.

std::vector<ov::Output<ov::Node>> outputs()

Model outputs.

std::vector<ov::Output<ov::Node>> inputs()

Model inputs.

const ov::element::Type &get_output_element_type(size_t i) const

Return the element type of output i.

const Shape &get_output_shape(size_t i) const

Return the shape of element i.

const PartialShape &get_output_partial_shape(size_t i) const

Return the partial shape of element i.

std::shared_ptr<ov::Node> get_result() const

Check that there is a single result and return it.

const std::string &get_name() const

Get the unique name of the model.

Returns

A const reference to the model’s unique name.

void set_friendly_name(const std::string &name)

Sets a friendly name for a model. This does not overwrite the unique name of the model and is retrieved via get_friendly_name(). Used mainly for debugging.

Parameters

name – is the friendly name to set

const std::string &get_friendly_name() const

Gets the friendly name for a model. If no friendly name has been set via set_friendly_name then the model’s unique name is returned.

Returns

A const reference to the model’s friendly name.

size_t get_graph_size() const

Returns the sum of the size of all nodes in the graph plus the size of all constant data. This has little value beyond comparing the relative size of graphs and should not be considered the actual memory consumption of a graph.

bool is_dynamic() const

Returns true if any of the op’s defined in the model contains partial shape.

void replace_parameter(size_t parameter_index, const std::shared_ptr<ov::op::v0::Parameter> &parameter)

Replace the parameter_indexth parameter of the model with parameter.

All users of the parameter_indexth parameter are redirected to parameter, and the parameter_indexth entry in the model parameter list is replaced with parameter.

Parameters
  • parameter_index – The index of the parameter to replace.

  • parameter – The parameter to substitute for the parameter_indexth parameter.

inline const ov::ParameterVector &get_parameters() const

Return the model parameters.

inline const ov::ResultVector &get_results() const

Return a list of model’s outputs.

int64_t get_parameter_index(const std::shared_ptr<ov::op::v0::Parameter> &parameter) const

Index for parameter, or -1.

int64_t get_result_index(const ov::Output<ov::Node> &value) const

Return the index of this model’s Result represented by the “value” Output object. This method returns -1 if an the passed output is not related to the Results of a model.

Parameters

valueOutput containing Node

int64_t get_result_index(const ov::Output<const ov::Node> &value) const

Return the index of this model’s Result represented by the “value” Output object. This method returns -1 if an the passed output is not related to the Results of a model.

Parameters

valueOutput containing Node

bool evaluate(const ov::HostTensorVector &output_tensors, const ov::HostTensorVector &input_tensors, ov::EvaluationContext &evaluation_context) const

Evaluate the model on inputs, putting results in outputs.

Deprecated:

Use evaluate with ov::Tensor instead

Parameters
  • output_tensors – Tensors for the outputs to compute. One for each result

  • input_tensors – Tensors for the inputs. One for each inputs.

  • evaluation_context – Storage of additional settings and attributes that can be used when evaluating the model. This additional information can be shared across nodes.

bool evaluate(const ov::HostTensorVector &output_tensors, const ov::HostTensorVector &input_tensors) const

Evaluate the model on inputs, putting results in outputs.

Deprecated:

Use evaluate with ov::Tensor instead

Parameters
  • output_tensors – Tensors for the outputs to compute. One for each result

  • input_tensors – Tensors for the inputs. One for each inputs.

bool evaluate(ov::TensorVector &output_tensors, const ov::TensorVector &input_tensors, ov::EvaluationContext &evaluation_context) const

Evaluate the model on inputs, putting results in outputs.

Parameters
  • output_tensors – Tensors for the outputs to compute. One for each result

  • input_tensors – Tensors for the inputs. One for each inputs.

  • evaluation_context – Storage of additional settings and attributes that can be used when evaluating the model. This additional information can be shared across nodes.

bool evaluate(ov::TensorVector &output_tensors, const ov::TensorVector &input_tensors) const

Evaluate the model on inputs, putting results in outputs.

Parameters
  • output_tensors – Tensors for the outputs to compute. One for each result

  • input_tensors – Tensors for the inputs. One for each inputs.

inline const ov::SinkVector &get_sinks() const

Return a list of model’s sinks.

void add_sinks(const ov::SinkVector &sinks)

Add new sink nodes to the list. Method doesn’t validate graph, it should be done manually after all changes.

Parameters

sinks – new sink nodes

void remove_sink(const std::shared_ptr<ov::op::Sink> &sink)

Delete sink node from the list of sinks. Method doesn’t delete node from graph.

Parameters

sink – Sink to delete

void add_results(const ov::ResultVector &results)

Add new Result nodes to the list. Method doesn’t validate graph, it should be done manually after all changes.

Parameters

results – new Result nodes

void remove_result(const std::shared_ptr<ov::op::v0::Result> &result)

Delete Result node from the list of results. Method will not delete node from graph.

Parameters

result – Result node to delete

void add_parameters(const ov::ParameterVector &params)

Add new Parameter nodes to the list.

Method doesn’t change or validate graph, it should be done manually. For example, if you want to replace ReadValue node by Parameter, you should do the following steps:

  • replace node ReadValue by Parameter in graph

  • call add_parameter() to add new input to the list

  • call graph validation to check correctness of changes

Parameters

params – new Parameter nodes

void remove_parameter(const std::shared_ptr<ov::op::v0::Parameter> &param)

Delete Parameter node from the list of parameters. Method will not delete node from graph. You need to replace Parameter with other operation manually. Attention: Indexing of parameters can be changed.

Possible use of method is to replace input by variable. For it the following steps should be done:

  • Parameter node should be replaced by ReadValue

  • call remove_parameter(param) to remove input from the list

  • check if any parameter indexes are saved/used somewhere, update it for all inputs because indexes can be changed

  • call graph validation to check all changes

Parameters

param – Parameter node to delete

void add_variables(const ov::op::util::VariableVector &variables)

Add new variables to the list. Method doesn’t validate graph, it should be done manually after all changes.

Parameters

variables – new variables to add

void remove_variable(const ov::op::util::Variable::Ptr &variable)

Delete variable from the list of variables. Method doesn’t delete nodes that used this variable from the graph.

Parameters

variable – Variable to delete

inline const ov::op::util::VariableVector &get_variables() const

Return a list of model’s variables.

ov::op::util::Variable::Ptr get_variable_by_id(const std::string &variable_id) const

Return a variable by specified variable_id.

inline RTMap &get_rt_info()

Returns a runtime info.

Returns

reference to ov::AnyMap with runtime info

inline const RTMap &get_rt_info() const

Returns a constant runtime info.

Returns

reference to const ov::AnyMap with runtime info

template<class T, class ...Args, typename std::enable_if<!std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(Args... args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters
  • T – the type of returned value

  • Args – types of variadic arguments

Parameters

args – path to the runtime attribute

Returns

constant reference to value from runtime info

template<class T, class ...Args, typename std::enable_if<std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(Args... args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters
  • T – the type of returned value

  • Args – types of variadic arguments

Parameters

args – path to the runtime attribute

Returns

constant reference to value from runtime info

template<class T, typename std::enable_if<!std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(const std::vector<std::string> &args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters

T – the type of returned value

Parameters

args – vector with path to the runtime attribute

Returns

constant reference to value from runtime info

template<class T, typename std::enable_if<std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(const std::vector<std::string> &args) const

Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.

Template Parameters

T – the type of returned value

Parameters

args – vector with path to the runtime attribute

Returns

constant reference to value from runtime info

template<class ...Args>
inline bool has_rt_info(Args... args) const

Checks if given path exists in runtime info.

Template Parameters

Args – types of variadic arguments

Parameters

args – path to the runtime attribute

Returns

true if path exists, otherwise false

bool has_rt_info(const std::vector<std::string> &args) const

Checks if given path exists in runtime info.

Parameters

args – vector with path to the runtime attribute

Returns

true if path exists, otherwise false

template<class T, class ...Args>
inline void set_rt_info(const T &argument, Args... args)

Add value inside the runtime info.

Template Parameters
  • T – type of new value

  • Args – types of variadic arguments

Parameters
  • argument – value for the runtime info

  • args – path to the runtime attribute

template<class T>
inline void set_rt_info(const T &argument, const std::vector<std::string> &args)

Add value inside the runtime info.

Template Parameters

T – type of new value

Parameters
  • argument – value for the runtime info

  • args – vector with path to the runtime attribute

class NmsSelectedIndices : private ov::RuntimeAttribute
#include <nms_selected_indices.hpp>
class Node : public std::enable_shared_from_this<Node>
#include <node.hpp>

Nodes are the backbone of the graph of Value dataflow. Every node has zero or more nodes as arguments and one value, which is either a tensor or a (possibly empty) tuple of values.

Subclassed by ov::op::Op, ov::pass::pattern::op::Pattern

Public Functions

virtual void validate_and_infer_types()

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual const ov::op::AutoBroadcastSpec &get_autob() const
Returns

the autobroadcasr spec

virtual bool has_evaluate() const

Allows to get information about availability of evaluate method for the current operation.

virtual bool evaluate(const ov::HostTensorVector &output_values, const ov::HostTensorVector &input_values) const

Evaluates the op on input_values putting results in output_values.

Deprecated:

Use evaluate with ov::Tensor instead

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool evaluate(const ov::HostTensorVector &output_values, const ov::HostTensorVector &input_values, const EvaluationContext &evaluationContext) const

Evaluates the op on input_values putting results in output_values.

Deprecated:

Use evaluate with ov::Tensor instead

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

  • evaluation_context – Storage of additional settings and attributes that can be used when evaluating the op.

Returns

true if successful

virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values) const

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values, const ov::EvaluationContext &evaluationContext) const

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

  • evaluation_context – Storage of additional settings and attributes that can be used when evaluating the op.

Returns

true if successful

inline virtual OutputVector decompose_op() const

Decomposes the FusedOp into a sub-graph consisting of core openvino ops.

Returns

A vector of nodes comprising the sub-graph. The order of output tensors must match the match output tensors of the FusedOp

virtual const type_info_t &get_type_info() const = 0

Returns the NodeTypeInfo for the node’s class. During transition to type_info, returns a dummy type_info for Node if the class has not been updated yet.

void set_arguments(const NodeVector &arguments)

Sets/replaces the arguments with new arguments.

void set_arguments(const OutputVector &arguments)

Sets/replaces the arguments with new arguments.

void set_argument(size_t position, const Output<Node> &argument)

Sets/replaces the arguments with new arguments.

void set_output_size(size_t output_size)

Sets the number of outputs.

virtual std::string description() const

Get the string name for the type of the node, such as Add or Multiply. The class name, must not contain spaces as it is used for codegen.

Returns

A const reference to the node’s type name

const std::string &get_name() const

Get the unique name of the node.

Returns

A const reference to the node’s unique name.

void set_friendly_name(const std::string &name)

Sets a friendly name for a node. This does not overwrite the unique name of the node and is retrieved via get_friendly_name(). Used mainly for debugging. The friendly name may be set exactly once.

Parameters

name – is the friendly name to set

const std::string &get_friendly_name() const

Gets the friendly name for a node. If no friendly name has been set via set_friendly_name then the node’s unique name is returned.

Returns

A const reference to the node’s friendly name.

virtual std::ostream &write_description(std::ostream &os, uint32_t depth = 0) const

Writes a description of a node to a stream.

Parameters
  • os – The stream; should be returned

  • depth – How many levels of inputs to describe

Returns

The stream os

const std::vector<std::shared_ptr<Node>> &get_control_dependencies() const

Get control dependencies registered on the node.

const std::vector<Node*> &get_control_dependents() const

Get nodes dependent on this node.

void add_control_dependency(std::shared_ptr<Node> node)

This node cannot execute until node executes.

void remove_control_dependency(std::shared_ptr<Node> node)

Remove the dependency of this node on node.

void clear_control_dependencies()

Remove all dependencies from this node.

void clear_control_dependents()

Remove this node as a dependency from all dependent nodes.

void add_node_control_dependencies(const std::shared_ptr<const Node> &source_node)

This node absorbs the control dependencies of source_node.

void add_node_control_dependents(const std::shared_ptr<const Node> &source_node)

This node becomes a dependent of every node dependent on source_node.

void transfer_control_dependents(std::shared_ptr<Node> replacement)

This node’s control dependencies are replaced by replacement.

size_t get_output_size() const

Returns the number of outputs from the node.

const element::Type &get_output_element_type(size_t i) const

Returns the element type for output i.

const element::Type &get_element_type() const

Checks that there is exactly one output and returns its element type.

const Shape &get_output_shape(size_t i) const

Returns the shape for output i.

const PartialShape &get_output_partial_shape(size_t i) const

Returns the partial shape for output i.

Output<const Node> get_default_output() const

Return the output to use when converting to an Output<Node> with no index specified. Throws when not supported.

virtual size_t get_default_output_index() const

Returns the output of the default output, or throws if there is none.

size_t no_default_index() const

Throws no default.

const Shape &get_shape() const

Checks that there is exactly one output and returns its shape.

descriptor::Tensor &get_output_tensor(size_t i) const

Returns the tensor for output or input i.

size_t get_input_size() const

Returns the number of inputs for the op.

const element::Type &get_input_element_type(size_t i) const

Returns the element type of input i.

const Shape &get_input_shape(size_t i) const

Returns the shape of input i.

const PartialShape &get_input_partial_shape(size_t i) const

Returns the partial shape of input i.

bool has_same_type(std::shared_ptr<const Node> node) const

True if this and node have one output with same element type and shape.

NodeVector get_users(bool check_is_used = false) const

Get all the nodes that uses the current node.

inline bool operator<(const Node &other) const

Use instance ids for comparison instead of memory addresses to improve determinism.

std::vector<Input<Node>> inputs()
Returns

A vector containing a handle for each of this node’s inputs, in order.

std::vector<Input<const Node>> inputs() const
Returns

A vector containing a handle for each of this node’s inputs, in order.

std::vector<Output<Node>> input_values() const
Returns

A vector containing the values for each input

std::vector<Output<Node>> outputs()
Returns

A vector containing a handle for each of this node’s outputs, in order.

std::vector<Output<const Node>> outputs() const
Returns

A vector containing a handle for each of this node’s outputs, in order.

Input<Node> input(size_t input_index)
Throws

std::out_of_range – if the node does not have at least input_index+1 inputs.

Returns

A handle to the input_indexth input of this node.

Input<const Node> input(size_t input_index) const
Throws

std::out_of_range – if the node does not have at least input_index+1 inputs.

Returns

A handle to the input_indexth input of this node.

Output<Node> output(size_t output_index)
Throws

std::out_of_range – if the node does not have at least output_index+1 outputs.

Returns

A handle to the output_indexth output of this node.

Output<const Node> output(size_t output_index) const
Throws

std::out_of_range – if the node does not have at least output_index+1 outputs.

Returns

A handle to the output_indexth output of this node.

class NodeValidationFailure : public ov::AssertFailure
#include <node.hpp>

Public Functions

template<> OPENVINO_API void create (const char *file, int line, const char *check_string, std::pair< const Node *, const std::vector< PartialShape > * > &&ctx, const std::string &explanation)

Specialization to throw the NodeValidationFailure for shape inference using PartialShape

Parameters
  • check_loc_infoException location details to print.

  • ctxNodeValidationFailure context which got pointer to node and input shapes used for shape inference.

  • explanationException explanation string.

class NonconvertibleDivide : public ov::RuntimeAttribute
#include <nonconvertible_divide.hpp>

NonconvertibleDivide class represents runtime info attribute that marks a Divide as prohibitted to transform it to power.

class NotImplemented : public ov::AssertFailure
#include <except.hpp>

Exception class to be thrown on not implemented code.

class NoTransposeSinkingAttr : public ov::RuntimeAttribute
#include <transpose_sinking_attr.hpp>

NoTransposeSinkingAttr class represents runtime info attribute that marks transpose operation should not be moved be backward sinking propagation.

class OldApiMapElementType : public ov::RuntimeAttribute
#include <old_api_map_element_type_attribute.hpp>

OldApiMapElementType class represents runtime info attribute that stores legacy type that is required for obtaining IR in old API.

Public Functions

OldApiMapElementType() = default

A default constructor

inline OldApiMapElementType(const ov::element::Type &value)

Constructs a new OldApiMapElementType object.

Parameters

value[in] The object that stores values of OldApiMapElementType.

class OldApiMapOrder : public ov::RuntimeAttribute
#include <old_api_map_order_attribute.hpp>

OldApiMapOrder class represents runtime info attribute that stores order of the transpose that is required for obtaining IR in old API.

OldApiMapOrder stores the following information. Parameter: Order of the transpose which should be applied to Parameter with old API layout to obtain Parameter with new API layout.

Result: Order of the transpose which should be applied to Result with new API layout to obtain Result with old API layout.

Public Functions

OldApiMapOrder() = default

A default constructor

inline OldApiMapOrder(const std::vector<uint64_t> &value)

Constructs a new OldApiMapOrder object.

Parameters

value[in] The object that stores values of OldApiMapOrder.

template<class T>
class OpExtension : public ov::BaseOpExtension
#include <op_extension.hpp>

The default implementation of OpenVINO operation extensions.

Public Functions

inline OpExtension()

Default constructor.

inline virtual const ov::DiscreteTypeInfo &get_type_info() const override

Returns the type info of operation.

Returns

ov::DiscreteTypeInfo

inline virtual ov::OutputVector create(const ov::OutputVector &inputs, ov::AttributeVisitor &visitor) const override

Method creates an OpenVINO operation.

Parameters
  • inputs – vector of input ports

  • visitor – attribute visitor which allows to read necessaty arguments

Returns

vector of output ports

inline virtual std::vector<ov::Extension::Ptr> get_attached_extensions() const override

Returns extensions that should be registered together with this extension class object.

Attached extensions may include frontend extensions that OpenVINO op to framework ops or necessary transformations that should be applied to the network which consist of target op.

Returns

class OpSet
#include <opset.hpp>

Run-time opset information.

Subclassed by ngraph::OpSet

Public Functions

template<typename OP_TYPE>
inline void insert(const std::string &name)

Insert OP_TYPE into the opset with a special name and the default factory.

template<typename OP_TYPE>
inline void insert()

Insert OP_TYPE into the opset with the default name and factory.

ov::Node *create(const std::string &name) const

Create the op named name using it’s factory.

ov::Node *create_insensitive(const std::string &name) const

Create the op named name using it’s factory.

inline bool contains_type(const NodeTypeInfo &type_info) const

Return true if OP_TYPE is in the opset.

template<typename OP_TYPE>
inline bool contains_type() const

Return true if OP_TYPE is in the opset.

inline bool contains_type(const std::string &name) const

Return true if name is in the opset.

inline bool contains_type_insensitive(const std::string &name) const

Return true if name is in the opset.

inline bool contains_op_type(const Node *node) const

Return true if node’s type is in the opset.

template<class T>
class optional
#include <ov_optional.hpp>

Store optional object of type T (basic version of std::optional).

Note

If cpp17 used this class should be replaced by std::optional.

Template Parameters

T – Type of stored object.

class OriginalPrecisionAttribute : public ov::RuntimeAttribute
#include <original_precision_attribute.hpp>

OriginalPrecisionAttribute stores the original precision of the node to pass this information to plugins.

template<typename NodeType>
class Output
#include <node_output.hpp>
template<>
class Output<const Node>
#include <node_output.hpp>

A handle for one of a node’s outputs.

Public Functions

Output(const Node *node, size_t index)

Constructs a Output.

Parameters
  • node – A pointer to the node for the output handle.

  • index – The index of the output.

Output(const std::shared_ptr<const Node> &node, size_t index)

Constructs a Output.

Parameters
  • node – A shared_ptr to the node for the output handle.

  • index – The index of the output.

template<typename T>
inline Output(const std::shared_ptr<const T> &node)

Constructs a Output, referencing the zeroth output of the node.

Parameters

node – A shared_ptr to the node for the output handle.

Output() = default

A null output.

const Node *get_node() const
Returns

A pointer to the node referred to by this output handle.

std::shared_ptr<const Node> get_node_shared_ptr() const
Returns

A shared_ptr to the node referred to by this output handle.

size_t get_index() const
Returns

The index of the output referred to by this output handle.

descriptor::Tensor &get_tensor() const
Returns

A reference to the tensor descriptor for this output.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
Returns

A shared point to the tensor ptr for this output.

const element::Type &get_element_type() const
Returns

The element type of the output referred to by this output handle.

const Shape &get_shape() const
Returns

The shape of the output referred to by this output handle.

const PartialShape &get_partial_shape() const
Returns

The partial shape of the output referred to by this output handle.

const RTMap &get_rt_info() const
Returns

The constant reference to runtime info map

const std::unordered_set<std::string> &get_names() const
Returns

The tensor names associated with this output

std::string get_any_name() const
Returns

Any tensor name associated with this output

std::set<Input<Node>> get_target_inputs() const
Returns

A set containing handles for all inputs targeted by the output referenced by this output handle.

template<>
class Output<Node>
#include <node_output.hpp>

A handle for one of a node’s outputs.

Public Functions

Output(Node *node, size_t index)

Constructs a Output.

Parameters
  • node – A pointer to the node for the output handle.

  • index – The index of the output.

Output(const std::shared_ptr<Node> &node, size_t index)

Constructs a Output.

Parameters
  • node – A shared_ptr to the node for the output handle.

  • index – The index of the output.

template<typename T>
inline Output(const std::shared_ptr<T> &node)

Constructs a Output, referencing the zeroth output of the node.

Parameters

node – A shared_ptr to the node for the output handle.

Output() = default

A null output.

Node *get_node() const
Returns

A pointer to the node referred to by this output handle.

std::shared_ptr<Node> get_node_shared_ptr() const
Returns

A shared_ptr to the node referred to by this output handle.

size_t get_index() const
Returns

The index of the output referred to by this output handle.

descriptor::Tensor &get_tensor() const
Returns

A reference to the tensor descriptor for this output.

std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
Returns

A shared point to the tensor ptr for this output.

void set_tensor_ptr(std::shared_ptr<descriptor::Tensor> tensor_ptr)
Returns

Set new tensor desc shared pointer to this output

const element::Type &get_element_type() const
Returns

The element type of the output referred to by this output handle.

const Shape &get_shape() const
Returns

The shape of the output referred to by this output handle.

const PartialShape &get_partial_shape() const
Returns

The partial shape of the output referred to by this output handle.

RTMap &get_rt_info()
Returns

The reference to runtime info map

const RTMap &get_rt_info() const
Returns

The constant reference to runtime info map

const std::unordered_set<std::string> &get_names() const
Returns

The tensor names associated with this output

std::string get_any_name() const
Returns

Any tensor names associated with this output

void set_names(const std::unordered_set<std::string> &names)
Returns

Set tensor names associated with this output

void add_names(const std::unordered_set<std::string> &names)
Returns

Add tensor names associated with this output

std::set<Input<Node>> get_target_inputs() const
Returns

A set containing handles for all inputs targeted by the output referenced by this output handle.

void remove_target_input(const Input<Node> &target_input) const

Removes a target input from the output referenced by this output handle.

Parameters

target_input – The target input to remove.

void replace(const Output<Node> &replacement)

Replace all users of this value with replacement.

class PartialShape
#include <partial_shape.hpp>

Class representing a shape that may be partially or totally dynamic.

A PartialShape may have:

  • Dynamic rank. (Informal notation: ?)

  • Static rank, but dynamic dimensions on some or all axes. (Informal notation examples: {1,2,?,4}, {?,?,?})

  • Static rank, and static dimensions on all axes. (Informal notation examples: {1,2,3,4}, {6}, {})

Public Functions

PartialShape(std::initializer_list<Dimension> init)

Constructs a shape with static rank from an initializer list of Dimension.

Examples:

PartialShape s{2,3,4};                     // rank=3, all dimensions static
PartialShape s{};                          // rank=0
PartialShape s{2,Dimension::dynamic(),3};  // rank=3, dimension 1 dynamic
Parameters

init – The Dimension values for the constructed shape.

PartialShape(std::vector<Dimension> dimensions)

Constructs a PartialShape with static rank from a vector of Dimension.

Parameters

dimensions – The Dimension values for the constructed shape.

PartialShape(const std::vector<Dimension::value_type> &dimensions)

Constructs a PartialShape with static rank from a vector of dimensions values.

Parameters

dimensions – The Dimension values for the constructed shape.

PartialShape()

Constructs a static PartialShape with zero rank (the shape of a scalar).

PartialShape(const Shape &shape)

Constructs a static PartialShape from a PartialShape.

Parameters

shape – The PartialShape to convert into PartialShape.

PartialShape(const std::string &shape)

Constructs a static PartialShape from a string.

Parameters

shape – The string to parse into PartialShape.

bool is_static() const

Check if this shape is static.

A shape is considered static if it has static rank, and all dimensions of the shape are static.

Returns

true if this shape is static, else false.

inline bool is_dynamic() const

Check if this shape is dynamic.

A shape is considered static if it has static rank, and all dimensions of the shape are static.

Returns

false if this shape is static, else true.

inline Rank rank() const

Get the rank of the shape.

Returns

The rank of the shape. This will be Rank::dynamic() if the rank of the shape is dynamic.

bool compatible(const PartialShape &s) const

Check whether this shape is compatible with the argument, i.e., whether it is possible to merge them.

Two shapes are compatible if

  • one or both of them has dynamic rank, or

  • both shapes have dynamic and equal rank, and their dimensions are elementwise compatible (see Dimension::compatible()).

Parameters

s – The shape to be checked for compatibility with this shape.

Returns

true if this shape is compatible with s, else false.

bool same_scheme(const PartialShape &s) const

Check whether this shape represents the same scheme as the argument.

Two shapes s1 and s2 represent the same scheme if

  • they both have dynamic rank, or

  • they both have static and equal rank r, and for every i from 0 to r-1, s1[i] represents the same scheme as s2[i] (see Dimension::same_scheme()).

Parameters

s – The shape whose scheme is being compared with this shape.

Returns

true if this shape represents the same scheme as s, else false.

bool relaxes(const PartialShape &s) const

Check whether this shape is a relaxation of the argument.

Intuitively, a PartialShape s1 is said to relax s2 (or is a relaxation of s2) if it is “more permissive” than s2. In other words, s1 is a relaxation of s2 if anything you can form by plugging things into the dynamic dimensions of s2 is also something you can form by plugging things into the dynamic dimensions of s1, but not necessarily the other way around.

s1.relaxes(s2) is equivalent to s2.refines(s1).

Formally, PartialShape s1 is said to relax PartialShape s2 if:

  • For every i from 0 to r-1, either s1[i] contains s2[i].

Parameters

s – The shape which is being compared against this shape.

Returns

true if this shape relaxes s, else false.

bool refines(const PartialShape &s) const

Check whether this shape is a refinement of the argument.

Intuitively, a PartialShape s1 is said to relax s2 (or is a relaxation of s2) if it is “less permissive” than s2. In other words, s1 is a relaxation of s2 if anything you can form by plugging things into the dynamic dimensions of s1 is also something you can form by plugging things into the dynamic dimensions of s2, but not necessarily the other way around.

s1.refines(s2) is equivalent to s2.relaxes(s1).

Formally, PartialShape s1 is said to refine PartialShape s2 if:

  • s2 has dynamic rank, or

  • s1 and s2 both have static rank r, and for every i from 0 to r-1, either s2[i] is dynamic, or s1[i] == s2[i].

Parameters

s – The shape which is being compared against this shape.

Returns

true if this shape refines s, else false.

bool merge_rank(const Rank &r)

Checks that this shape’s rank is compatible with r, and, if this shape’s rank is dynamic and r is static, updates this shape to have a rank of r with dimensions all dynamic.

Returns

true if this shape’s rank is compatible with r, else false.

Shape to_shape() const

Convert a static PartialShape to a PartialShape.

Throws

std::invalid_argument – If this PartialShape is dynamic.

Returns

A new PartialShape s where s[i] = size_t((*this)[i]).

bool all_non_negative() const

Returns true if all static dimensions of the tensor are non-negative, else false.

const Dimension &operator[](size_t i) const

Index operator for PartialShape.

Parameters

i – The index of the dimension being selected.

Returns

A reference to the ith Dimension of this shape.

Dimension &operator[](size_t i)

Index operator for PartialShape.

Parameters

i – The index of the dimension being selected.

Returns

A reference to the ith Dimension of this shape.

inline explicit operator std::vector<Dimension>() const

Returns a vector of the dimensions. This has no meaning if dynamic.

Shape get_max_shape() const

Get the max bounding shape.

Shape get_min_shape() const

Get the min bounding shape.

Shape get_shape() const

Get the unique shape.

inline iterator begin() noexcept

Returns a read/write iterator that points to the first element in the shape. Iteration is done in ordinary element order.

inline const_iterator begin() const noexcept

Returns a read-only (constant) iterator that points to the first element in the shape. Iteration is done in ordinary element order.

inline iterator end() noexcept

Returns a read/write iterator that points one past the last element in the shape. Iteration is done in ordinary element order.

inline const_iterator end() const noexcept

Returns a read-only (constant) iterator that points one past the last element in the shape. Iteration is done in ordinary element order.

inline reverse_iterator rbegin() noexcept

Returns a read/write reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.

inline const_reverse_iterator rbegin() const noexcept

Returns a read-only (constant) reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.

inline reverse_iterator rend() noexcept

Returns a read/write reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.

inline const_reverse_iterator rend() const noexcept

Returns a read-only (constant) reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.

inline const_iterator cbegin() const noexcept

Returns a read-only (constant) iterator that points to the first element in the shape. Iteration is done in ordinary element order.

inline const_iterator cend() const noexcept

Returns a read-only (constant) iterator that points one past the last element in the shape. Iteration is done in ordinary element order.

inline const_reverse_iterator crbegin() const noexcept

Returns a read-only (constant) reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.

inline const_reverse_iterator crend() const noexcept

Returns a read-only (constant) reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.

inline void resize(size_t count)

Resizes dimensions container to contain count elements.

inline size_t size() const

Returns size of dimension vector. Requires rank to be static.

inline iterator insert(iterator position, const Dimension &val)

Returns a read/write iterator that points to the inserted element in the shape.

inline void insert(iterator position, size_t n, const Dimension &val)

Inserts count copies of the value before position.

template<class InputIterator>
inline void insert(iterator position, InputIterator first, InputIterator last)

Inserts elements from range [first, last) before position.

inline void reserve(size_t n)

Requests that the dimensions vector capacity be enough to contain n elements.

inline void push_back(const Dimension &val)

push element to the end of partial shape

template<class ...Args>
inline void emplace_back(Args&&... args)

emplace element to the end of partial shape

std::string to_string() const

String representation of PartialShape.

Public Static Functions

static PartialShape dynamic(Rank r = Rank::dynamic())

Construct a PartialShape with the given rank and all dimensions (if any) dynamic.

Returns

A PartialShape with the given rank, and all dimensions (if any) dynamic.

static bool merge_into(PartialShape &dst, const PartialShape &src)

Try to merge one shape into another.

Merges src into dst, returning true on success and false on failure. If false is returned, the effect on dst is unspecified.

To merge two partial shapes s1 and s2 is to find the most permissive partial shape s that is no more permissive than s1 or s2, if s exists. For example:

merge(?,?) -> ?
merge(?,{?,?}) -> {?,?}
merge({?,?},{?,?}) -> {?,?}
merge({1,2,3,4},?) -> {1,2,3,4}
merge({1,2},{1,?}) -> {1,2}
merge({1,2,?,?},{1,?,3,?}) -> {1,2,3,?}
merge({1,2,3},{1,2,3}) -> {1,2,3}

merge({1,?},{2,?}) fails [dimension 0 constraints are inconsistent]
merge({?,?},{?,?,?}) fails [ranks are inconsistent]

This function (merge_into) performs the “merge” operation described above on dst and src, but overwrites dst with the result and returns true if merging is successful; if merging is unsuccessful, the function returns false and may make unspecified changes to dst.

Parameters
  • dst[inout] The shape that src will be merged into.

  • src – The shape that will be merged into dst.

Returns

true if merging succeeds, else false.

static bool broadcast_merge_into(PartialShape &dst, const PartialShape &src, const ov::op::AutoBroadcastSpec &autob)

Try to merge one shape into another along with implicit broadcasting.

Friends

friend OPENVINO_API std::ostream & operator<< (std::ostream &str, const PartialShape &shape)

Inserts a human-readable representation of a PartialShape into an output stream.

The output to the stream is in “informal” notation. In other words:

  • If shape has dynamic rank, inserts the string ?.

  • If shape has static rank, inserts the string {, then inserts each dimension of shape into the output stream separated by commas, then inserts }.

Example:

PartialShape s1{PartialShape::dynamic())};
PartialShape s2{};
PartialShape s3{1,Dimension::dynamic(),2,3};
PartialShape s4{2,3,4};
std::cout << s1 << std::endl
          << s2 << std::endl
          << s3 << std::endl
          << s4 << std::endl;

Output:

?
{}
{1,?,2,3}
{2,3,4}
Parameters
  • str – The output stream targeted for insertion.

  • shape – The shape to be inserted into str.

Returns

A reference to str after insertion.

friend OPENVINO_API PartialShape operator+ (const PartialShape &s1, const PartialShape &s2)

Elementwise addition of two PartialShape objects.

  • If s1 or s2 has dynamic rank, returns PartialShape::dynamic().

  • If s1 ands2` both have static rank, and their ranks are unequal, throws std::invalid_argument.

  • If s1 and s2 both have static rank, and their ranks are equal, returns a new shape whose ith dimension is s1[i] + s2[i].

Parameters
  • s1 – Left operand for addition.

  • s2 – Right operand for addition.

Throws

std::invalid_argument – If s1 and s2 have inconsistent ranks.

Returns

The result of elementwise adding s1 to s2 (see description).

class PrecisionPreservedAttribute : public SharedAttribute<bool>
#include <precision_preserved_attribute.hpp>

PrecisionPreservedAttribute defines the precision preserved operation. If the attribute is absent, then an operation is not precision preserved.

For more details about the attribute, refer to PrecisionPreservedAttribute page in the Inference Engine Developer Guide.

Subclassed by ov::AvgPoolPrecisionPreservedAttribute

class PrecisionsAttribute : public SharedAttribute<std::vector<ov::element::Type>>
#include <precisions_attribute.hpp>

PrecisionsAttribute defines precision which is required for input/output port or an operation.

For more details about the attribute, refer to PrecisionsAttribute page in the Inference Engine Developer Guide.

class PrecisionSensitive : public ov::RuntimeAttribute
#include <precision_sensitive_attribute.hpp>

PrecisionSensitive class represents runtime info attribute that marks input to an operation as a precision sensitive and disables compression to FP16 of the subgraph before this input.

class PreprocessingAttribute : public ov::RuntimeAttribute
#include <preprocessing_attribute.hpp>
class PrimitivesPriority : public ov::RuntimeAttribute
#include <primitives_priority_attribute.hpp>
struct ProfilingInfo
#include <profiling_info.hpp>

Represents basic inference profiling information per operation.

If the operation is executed using tiling, the sum time per each tile is indicated as the total execution time. Due to parallel execution, the total execution time for all nodes might be greater than the total inference time.

Public Types

enum class Status

Defines the general status of a node.

Values:

enumerator NOT_RUN

A node is not executed.

enumerator OPTIMIZED_OUT

A node is optimized out during graph optimization phase.

enumerator EXECUTED

A node is executed.

Public Members

Status status

Defines the node status.

std::chrono::microseconds real_time

The absolute time, in microseconds, that the node ran (in total).

std::chrono::microseconds cpu_time

The net host CPU time that the node ran.

std::string node_name

Name of a node.

std::string exec_type

Execution type of a unit.

std::string node_type

Node type.

template<typename T, PropertyMutability mutability_ = PropertyMutability::RW>
class Property : public util::BaseProperty<T, PropertyMutability::RW>
#include <properties.hpp>

This class is used to bind property name with value type.

Template Parameters

T – type of value used to set or get property

Public Functions

template<typename ...Args>
inline std::pair<std::string, Any> operator()(Args&&... args) const

Constructs property.

Template Parameters

Args – property constructor arguments types

Parameters

args – property constructor arguments

Returns

Pair of name and type erased value.

template<typename T> RO > : public util::BaseProperty< T, PropertyMutability::RO >
#include <properties.hpp>

This class is used to bind read-only property name with value type.

Template Parameters

T – type of value used to pass or get property

struct PropertyName : public std::string
#include <properties.hpp>

This class is used to return property name and its mutability attribute.

Public Functions

inline PropertyName(const std::string &str, PropertyMutability mutability = PropertyMutability::RW)

Constructs property name object.

Parameters
  • str – property name

  • mutability – property mutability

inline bool is_mutable() const

check property mutability

Returns

true if property is mutable

class QuantizationAlignmentAttribute : public SharedAttribute<bool>
#include <quantization_alignment_attribute.hpp>

QuantizationAlignmentAttribute defines subgraph with the same quantization alignment. FakeQuantize operations are not included. The attribute is used by quantization operations.

For more details about the attribute, refer to QuantizationAlignmentAttribute page in the Inference Engine Developer Guide.

class QuantizationGranularityAttribute : public ov::RuntimeAttribute
#include <quantization_granularity_attribute.hpp>

QuantizationGranularityAttribute defines quantization granularity of operation inputs.

For more details about the attribute, refer to QuantizationGranularityAttribute page in the Inference Engine Developer Guide.

class QuantizationModeAttribute : public ov::RuntimeAttribute
#include <quantization_mode_attribute.hpp>
struct RawNodeOutput
#include <node.hpp>
class RemoteBlobTensor : public ov::IRemoteTensor
#include <remote_utils.hpp>

Tensor what contains InferenceEngine::RemoteBlob inside Blob owns the memory.

Public Functions

inline virtual const AnyMap &get_properties() const override

Returns additional information associated with tensor.

Returns

Map of property names to properties

class RemoteContext
#include <remote_context.hpp>

This class represents an abstraction

for remote (non-CPU) accelerator device-specific inference context. Such context represents a scope on the device within which compiled models and remote memory tensors can exist, function, and exchange data.

Subclassed by ov::intel_gpu::ocl::ClContext

Public Functions

RemoteContext() = default

Default constructor.

RemoteContext(const RemoteContext &other) = default

Default copy constructor.

Parameters

other – Another RemoteContext object.

RemoteContext &operator=(const RemoteContext &other) = default

Default copy assignment operator.

Parameters

other – Another RemoteContext object.

Returns

Reference to the current object.

RemoteContext(RemoteContext &&other) = default

Default move constructor.

Parameters

other – Another RemoteContext object.

RemoteContext &operator=(RemoteContext &&other) = default

Default move assignment operator.

Parameters

other – Another RemoteContext object.

Returns

Reference to the current object.

operator bool() const noexcept

Checks if current RemoteContext object is initialized.

Returns

true if current RemoteContext object is initialized, false - otherwise

~RemoteContext()

Destructor that preserves unloading order of implementation object and reference to the library.

template<typename T>
inline bool is() const noexcept

Checks if the RemoteContext object can be cast to the type T.

Template Parameters

T – Type to be checked. Must represent a class derived from RemoteContext.

Returns

True if this object can be dynamically cast to the type T*; false, otherwise.

template<typename T>
inline const T as() const

Casts this RemoteContext object to the type T.

Template Parameters

T – Type to cast to. Must represent a class derived from RemoteContext.

Returns

T Object.

RemoteTensor create_tensor(const element::Type &type, const Shape &shape, const AnyMap &params = {})

Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.

Parameters
  • type – Defines the element type of the tensor.

  • shape – Defines the shape of the tensor.

  • params – Map of the low-level tensor object parameters.

Returns

Pointer to a plugin object that implements the RemoteTensor interface.

AnyMap get_params() const

Returns a map of device-specific parameters required for low-level operations with the underlying object. Parameters include device/context handles, access flags, etc. Content of the returned map depends on a remote execution context that is currently set on the device (working scenario). Abstract method.

Returns

A map of name/parameter elements.

Tensor create_host_tensor(const element::Type type, const Shape &shape)

This method is used to create a host tensor object friendly for the device in current context. For example, GPU context may allocate USM host memory (if corresponding extension is available), which could be more efficient than regular host memory.

Parameters
Returns

A tensor instance with device friendly memory.

Public Static Functions

static void type_check(const RemoteContext &remote_context, const std::map<std::string, std::vector<std::string>> &type_info = {})

Internal method: checks remote type.

Parameters
  • remote_context – Remote context which type is checked.

  • type_info – Map with remote object runtime info.

Throws

Exception – if type check with the specified parameters failed.

class RemoteTensor : public ov::Tensor
#include <remote_tensor.hpp>

Remote memory access and interoperability API.

Subclassed by ov::intel_gpu::ocl::ClBufferTensor, ov::intel_gpu::ocl::ClImage2DTensor, ov::intel_gpu::ocl::USMTensor

Public Functions

void *data(const element::Type) = delete

Access to host memory is not available for RemoteTensor. To access a device-specific memory, cast to a specific RemoteTensor derived object and work with its properties or parse device memory properties via RemoteTensor::get_params.

Returns

Nothing, throws an exception.

ov::AnyMap get_params() const

Returns a map of device-specific parameters required for low-level operations with underlying object. Parameters include device/context/surface/buffer handles, access flags, etc. Content of the returned map depends on remote execution context that is currently set on the device (working scenario). Abstract method.

Returns

A map of name/parameter elements.

Public Static Functions

static void type_check(const Tensor &tensor, const std::map<std::string, std::vector<std::string>> &type_info = {})

Checks OpenVINO remote type.

Parameters
  • tensorTensor which type is checked.

  • type_info – Map with remote object runtime info.

Throws

Exception – if type check with specified parameters failed.

template<class TShape>
struct result_shape
#include <utils.hpp>

Get correct return type of input shape when call shape_infer.

The input shapes are vector like std::vector<TShape>, where TShape can be std::vector<const size_t> This will provide correct return especially for static shape which can work as reference to dimension or hold them.

Template Parameters

TShape – Type of input shape.

template<> Shape >
#include <utils.hpp>

Get correct result shape for ov::Shape which is same type.

template<>
struct result_shape<PartialShape>
#include <utils.hpp>

Get correct result shape for PartialShape which is same type.

class RoundingGuard
#include <rounding_guard.hpp>

Set current round direction for scoped block.

Round direction can be one of:

  • FE_DOWNWARD

  • FE_TONEAREST

  • FE_TOWARDZERO

  • FE_UPWARD see std <cfenv> header for details.

class RuntimeAttribute
#include <runtime_attribute.hpp>

Subclassed by SharedAttribute< IntervalsAlignmentSharedValue >, SharedAttribute< bool >, SharedAttribute< std::vector< ov::element::Type > >, SharedAttribute< T >, ov::BiasAttribute, ov::Decompression, ov::DequantizationNode, ov::DisableCleanupAttribute, ov::DisableFP16Compression, ov::FusedNames, ov::KeepConstPrecision, ov::LayoutAttribute, ov::NmsSelectedIndices, ov::NoTransposeSinkingAttr, ov::NonconvertibleDivide, ov::OldApiMapElementType, ov::OldApiMapOrder, ov::OriginalPrecisionAttribute, ov::PrecisionSensitive, ov::PreprocessingAttribute, ov::PrimitivesPriority, ov::QuantizationGranularityAttribute, ov::QuantizationModeAttribute, ov::ShapeSubgraph, ov::StridesPropagation, ov::SymbolicInfo, ov::frontend::tensorflow::GraphIterator, ov::pass::DisableConstantFolding, ov::pass::DisableFoldSubgraphEmptyInputs, ov::pass::DisableRemoveConcatZeroDimInput, ov::preprocess::TensorInfoMemoryType

template<class T, Direction D = Direction::FORWARD>
class SeqGen
#include <sequnce_generator.hpp>

Infinite generator of sequence increasing values.

Start value can be specified.

Template Parameters

T – Type of sequence values (must support ++ or ‘&#8212;’ operators).

class Shape : public std::vector<size_t>
#include <shape.hpp>

Shape for a tensor.

class ShapeSubgraph : public ov::RuntimeAttribute
#include <is_shape_subgraph.hpp>

ShapeSubgraph class represents runtime info attribute that marks shape subgraphs. Information whether the node belongs to the shape path or to the data path is needed during evaluate and CF.

template<class T>
struct SoPtr
#include <so_ptr.hpp>

This class instantiate object using shared library.

Template Parameters

T – An type of object SoPtr can hold

Public Functions

SoPtr() = default

Default constructor.

inline ~SoPtr()

Destructor preserves unloading order of implementation object and reference to library.

inline SoPtr(const std::shared_ptr<T> &ptr, const std::shared_ptr<void> &so)

Constructs an object with existing shared object reference and loaded pointer.

Parameters
  • ptr – pointer to the loaded object

  • so – Existing reference to library

inline SoPtr(const std::shared_ptr<T> &ptr)

Constructs an object with existing shared object reference.

Parameters

ptr – pointer to the loaded object

template<class U, typename std::enable_if<std::is_base_of<T, U>::value, bool>::type = true>
inline SoPtr(const std::shared_ptr<U> &ptr)

Constructs an object with existing shared object reference.

Parameters

ptr – pointer to the loaded object

template<typename U>
inline SoPtr(const SoPtr<U> &that)

The copy-like constructor, can create So Pointer that dereferenced into child type if T is derived of U.

Parameters

that – copied SoPtr object

inline T *operator->() const noexcept

Standard pointer operator.

Returns

underlined interface with disabled Release method

Public Members

std::shared_ptr<T> _ptr

Gets a smart pointer to the custom object.

std::shared_ptr<void> _so

The shared object or dynamic loaded library.

class Strides : public std::vector<size_t>
#include <strides.hpp>

Strides for a tensor.

class StridesPropagation : public ov::RuntimeAttribute
#include <strides_property.hpp>
class SymbolicInfo : public ov::RuntimeAttribute
#include <symbolic_info.hpp>

SymbolicInfo class represents runtime info attribute that instructs ov::Output objects to skip invalidation of partial values and labels during partial value propagation and keeps shared_ptr to TableOfEquivalence.

class Tensor
#include <tensor.hpp>

Tensor API holding host memory It can throw exceptions safely for the application, where it is properly handled.

Subclassed by ov::RemoteTensor

Public Functions

Tensor() = default

Default constructor.

Tensor(const Tensor &other, const std::shared_ptr<void> &so)

Copy constructor with adding new shared object.

Parameters
  • other – Original tensor

  • so – Shared object

Tensor(const Tensor &other) = default

Default copy constructor.

Parameters

other – other Tensor object

Tensor &operator=(const Tensor &other) = default

Default copy assignment operator.

Parameters

other – other Tensor object

Returns

reference to the current object

Tensor(Tensor &&other) = default

Default move constructor.

Parameters

other – other Tensor object

Tensor &operator=(Tensor &&other) = default

Default move assignment operator.

Parameters

other – other Tensor object

Returns

reference to the current object

~Tensor()

Destructor preserves unloading order of implementation object and reference to library.

Tensor(const element::Type &type, const Shape &shape, const Allocator &allocator = {})

Constructs Tensor using element type and shape. Allocate internal host storage using default allocator.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • allocator – allocates memory for internal tensor storage

Tensor(const element::Type &type, const Shape &shape, void *host_ptr, const Strides &strides = {})

Constructs Tensor using element type and shape. Wraps allocated host memory.

Note

Does not perform memory allocation internally

Parameters
  • typeTensor element type

  • shapeTensor shape

  • host_ptr – Pointer to pre-allocated host memory with initialized objects

  • strides – Optional strides parameters in bytes. Strides are supposed to be computed automatically based on shape and element size

Tensor(const ov::Output<const ov::Node> &port, const Allocator &allocator = {})

Constructs Tensor using port from node. Allocate internal host storage using default allocator.

Parameters
  • port – port from node

  • allocator – allocates memory for internal tensor storage

Tensor(const ov::Output<const ov::Node> &port, void *host_ptr, const Strides &strides = {})

Constructs Tensor using port from node. Wraps allocated host memory.

Note

Does not perform memory allocation internally

Parameters
  • port – port from node

  • host_ptr – Pointer to pre-allocated host memory with initialized objects

  • strides – Optional strides parameters in bytes. Strides are supposed to be computed automatically based on shape and element size

Tensor(const Tensor &other, const Coordinate &begin, const Coordinate &end)

Constructs region of interest (ROI) tensor form another tensor.

Note

Does not perform memory allocation internally

Note

A Number of dimensions in begin and end must match number of dimensions in other.get_shape()

Parameters
  • other – original tensor

  • begin – start coordinate of ROI object inside of the original object.

  • end – end coordinate of ROI object inside of the original object.

void set_shape(const ov::Shape &shape)

Set new shape for tensor, deallocate/allocate if new total size is bigger than previous one.

Note

Memory allocation may happen

Parameters

shape – A new shape

const element::Type &get_element_type() const
Returns

A tensor element type

const Shape &get_shape() const
Returns

A tensor shape

void copy_to(ov::Tensor dst) const

Copy tensor, destination tensor should have the same element type and shape.

Parameters

dst – destination tensor

bool is_continuous() const

Reports whether the tensor is continuous or not.

Returns

true if tensor is continuous

size_t get_size() const

Returns the total number of elements (a product of all the dims or 1 for scalar)

Returns

The total number of elements

size_t get_byte_size() const

Returns the size of the current Tensor in bytes.

Returns

Tensor’s size in bytes

Strides get_strides() const
Returns

Tensor’s strides in bytes

void *data(const element::Type &type = {}) const

Provides an access to the underlaying host memory.

Note

If type parameter is specified, the method throws an exception if specified type’s fundamental type does not match with tensor element type’s fundamental type

Parameters

type – Optional type parameter.

Returns

A host pointer to tensor memory

template<typename T, typename datatype = typename std::decay<T>::type>
inline T *data() const

Provides an access to the underlaying host memory casted to type T

Note

Throws exception if specified type does not match with tensor element type

Returns

A host pointer to tensor memory casted to specified type T.

bool operator!() const noexcept

Checks if current Tensor object is not initialized.

Returns

true if current Tensor object is not initialized, false - otherwise

explicit operator bool() const noexcept

Checks if current Tensor object is initialized.

Returns

true if current Tensor object is initialized, false - otherwise

template<typename T>
inline std::enable_if<std::is_base_of<Tensor, T>::value, bool>::type is() const noexcept

Checks if the Tensor object can be cast to the type T.

Template Parameters

T – Type to be checked. Must represent a class derived from the Tensor

Returns

true if this object can be dynamically cast to the type const T*. Otherwise, false

template<typename T>
inline const std::enable_if<std::is_base_of<Tensor, T>::value, T>::type as() const

Casts this Tensor object to the type T.

Template Parameters

T – Type to cast to. Must represent a class derived from the Tensor

Returns

T object

Public Static Functions

static void type_check(const Tensor &tensor)

Checks openvino tensor type.

Parameters

tensor – a tensor which type will be checked

Throws

Exception – if type check with specified tensor is not pass

template<class TContainer>
class TensorAccessor : public ov::ITensorAccessor
#include <tensor_data_accessor.hpp>

Tensor data accessor functor.

Creates the ov::Tensor found in tensors container. This accessor does not take ownership of tensors container. Supports following containers:

Template Parameters

TContainer – Type of tensor container.

Public Functions

inline constexpr TensorAccessor(const TContainer *tensors)

Construct a new Tensor Accessor object for tensors container.

Parameters

tensors – Pointer to container with tensors.

virtual Tensor operator()(size_t port) const override

Get tensor for given port number.

Parameters

port – Port number to get data.

Returns

Tensor to data or empty tensor if data not found.

virtual Tensor operator()(size_t port) const

Get tensor at port.

Parameters

port – Number of data port (operator input) to get tensor.

Returns

Tensor to data at port.

virtual Tensor operator()(size_t port) const

Get tensor at port.

Parameters

port – Number of data port (operator input) to get tensor.

Returns

Tensor to data at port.

virtual Tensor operator()(size_t port) const

Get tensor at port.

Parameters

port – Number of data port (operator input) to get tensor.

Returns

Tensor to data at port.

class TensorRemoteBlob : public InferenceEngine::RemoteBlob, public ov::legacy_convert::TensorHolder
#include <remote_utils.hpp>

Create InferenceEngine::RemoteBlob from the Tensor.

Public Functions

inline virtual AnyMap getParams() const override

Returns a map of device-specific parameters required for low-level operations with underlying object. Parameters include device/context/surface/buffer handles, access flags, etc. Contents of the map returned depend on remote execution context that is currently set on the device (working scenario). Abstract method.

Returns

A map of name/parameter elements.

inline virtual std::string getDeviceName() const noexcept override

Returns name of the device on which underlying object is allocated. Abstract method.

Returns

A device name string in the same format as that in plugin metric.

inline virtual std::shared_ptr<InferenceEngine::RemoteContext> getContext() const noexcept override

Returns device context which underlying object belongs to. Abstract method.

Returns

Pointer to plugin-specific context class object, which is derived from RemoteContext. Dynamic casting should be used if it is necessary to retrieve a pointer to original class.

inline virtual void allocate() noexcept override

Allocates memory to store the data.

Abstract method.

inline virtual bool deallocate() noexcept override

Releases previously allocated data.

Abstract method.

Returns

True if deallocation happens successfully, false otherwise.

inline virtual InferenceEngine::LockedMemory<void> rwmap() noexcept override

Gets read/write access to the memory in virtual space of the process. The function returns object which retains mapped memory. The memory been addressed in the MemoryBlob in general case can be allocated on remote device. This function maps remote memory to the memory in the virtual process space and after destruction of the LockedMemory will upload changed content to the accelerator.

To avoid extra copy of data, you can use rmap() and wmap() functions.

In case of memory originally allocated on the host, this function returns LockedMemory which will transparently refer to original memory address. No extra copy will happen

In general case, pointer received from that LockedMemory becomes invalid just after destruction of LockedMemory instance. Keep Locked memory alive while you need to address memory in the process on the host.

Abstract method.

Returns

A LockedMemory object

inline virtual InferenceEngine::LockedMemory<const void> rmap() const noexcept override

Gets read only access to the memory in virtual space of the process. The function returns object which retains mapped memory.

The memory been addressed in the MemoryBlob in general case can be allocated on remote device. This function copies remote memory to the memory in the virtual process space and after destruction of the LockedMemory it will not upload host memory back, because it is expected that content is not changed.

To have an ability change content, you can use rwmap() and wmap() functions.

In case of memory originally allocated on the host, this function returns LockedMemory which will transparently refer to original memory address. No extra copy will happen

In general case, pointer received from that LockedMemory becomes invalid just after destruction of LockedMemory instance. Keep Locked memory alive while you need to address memory in the process on the host.

Abstract method.

Returns

A LockedMemory object

inline virtual InferenceEngine::LockedMemory<void> wmap() noexcept override

Gets “write only direction” access to the memory in virtual space of the process. The function returns object which retains memory to be uploaded on device.

The memory been addressed in the MemoryBlob in general case can be allocated on remote device. This function does not copy of the content from the device to the memory in the virtual process space, the content of the memory just after calling of this function is not specified. After destruction of the LockedMemory, content will be upload host memory. In the same time there is no abilities to restrict reading from the memory, you need to care of reading from memory got by wmap(), it might have sense in some cases like filling of content and before uploading to device

To access data stored in the blob, you can use rwmap() and rmap() functions.

In case of memory originally allocated on the host, this function returns LockedMemory which will transparently refer to original memory address. No extra copy will happen

In general case, pointer received from that LockedMemory becomes invalid just after destruction of LockedMemory instance. Keep Locked memory alive while you need to address memory in the process on the host.

Abstract method.

Returns

A LockedMemory object

inline virtual const std::shared_ptr<InferenceEngine::IAllocator> &getAllocator() const noexcept override

Gets the allocator for allocator-based blobs.

Returns

The allocator for allocator-based blobs or if there is none then a nullptr.

inline virtual void *getHandle() const noexcept override

Gets the handle to allocated memory.

Returns

The handle to allocated memory for allocator-based blobs or if there is none then a nullptr.

struct TensorTransform : public ov::element::NotSupported<void>
#include <utils.hpp>
template<typename VAT>
class ValueAccessor
#include <attribute_adapter.hpp>

Provides access to an attribute of type AT as a value accessor type VAT.

Provides access to values via get/set methods from an m_value, typically from ValueReference.

The m_buffer holds a VAT, which may be wider than the attribute AT. For example, serializers that only support int64_t integers would use a ValueAccessor<vector<int64_t>> to reference a vector<int8_t> attribute. Destruction moves the value back to the attribute if it was changed.

Template Parameters

VAT – The adapter value type; may be wider than the value being accessed.

Subclassed by ov::DirectValueAccessor< bool >, ov::DirectValueAccessor< double >, ov::DirectValueAccessor< int64_t >, ov::DirectValueAccessor< op::v5::Loop::SpecialBodyPorts >, ov::DirectValueAccessor< ov::Dimension >, ov::DirectValueAccessor< ov::PartialShape >, ov::DirectValueAccessor< ov::element::TypeVector >, ov::DirectValueAccessor< ov::op::util::FrameworkNodeAttrs >, ov::DirectValueAccessor< std::set< std::string > >, ov::DirectValueAccessor< std::shared_ptr< ngraph::runtime::AlignedBuffer > >, ov::DirectValueAccessor< std::shared_ptr< op::util::Variable > >, ov::DirectValueAccessor< std::shared_ptr< ov::Model > >, ov::DirectValueAccessor< std::vector< double > >, ov::DirectValueAccessor< std::vector< float > >, ov::DirectValueAccessor< std::vector< int16_t > >, ov::DirectValueAccessor< std::vector< int32_t > >, ov::DirectValueAccessor< std::vector< int8_t > >, ov::DirectValueAccessor< std::vector< std::shared_ptr< op::util::MultiSubGraphOp::InputDescription > > >, ov::DirectValueAccessor< std::vector< std::shared_ptr< op::util::MultiSubGraphOp::OutputDescription > > >, ov::DirectValueAccessor< std::vector< std::string > >, ov::DirectValueAccessor< std::vector< uint16_t > >, ov::DirectValueAccessor< std::vector< uint32_t > >, ov::DirectValueAccessor< std::vector< uint64_t > >, ov::DirectValueAccessor< std::vector< uint8_t > >, ov::IndirectScalarValueAccessor< float, double >, ov::IndirectScalarValueAccessor< int16_t, int64_t >, ov::IndirectScalarValueAccessor< int32_t, int64_t >, ov::IndirectScalarValueAccessor< int8_t, int64_t >, ov::IndirectScalarValueAccessor< uint16_t, int64_t >, ov::IndirectScalarValueAccessor< uint32_t, int64_t >, ov::IndirectScalarValueAccessor< uint64_t, int64_t >, ov::IndirectScalarValueAccessor< uint8_t, int64_t >, ov::IndirectScalarValueAccessor< AT, VAT >, ov::IndirectVectorValueAccessor< AT, VAT >

Public Functions

virtual const VAT &get() = 0

Returns the value.

virtual void set(const VAT &value) = 0

Sets the value.

template<>
class ValueAccessor<void*> : public ov::ValueAccessor<void>
#include <attribute_adapter.hpp>
template<>
class ValueAccessor<void>
#include <attribute_adapter.hpp>

ValueAccessor<void> provides an accessor for values that do not have get/set methods via AttributeVisitor.on_adapter.

All ValueAccessors must be derived from ValueAccessor<void> so that an AttributeVisitor only needs to implement a subset of the on_adapter methods.

Subclassed by ov::ValueAccessor< void * >, ov::VisitorAdapter

Public Functions

virtual const DiscreteTypeInfo &get_type_info() const = 0

type info enables identification of the value accessor, as well as is_type and as_type.

class VariableState
#include <variable_state.hpp>

VariableState class.

Public Functions

VariableState() = default

Default constructor.

~VariableState()

Destructor that preserves unloading order of implementation object and reference to the library.

void reset()

Resets internal variable state for relevant infer request to a value specified as default for the corresponding ReadValue node.

std::string get_name() const

Gets the name of the current variable state. If length of an array is not enough, the name is truncated by len, null terminator is inserted as well. variable_id from the corresponding ReadValue is used as variable state name.

Returns

A string representing state name.

Tensor get_state() const

Returns the value of the variable state.

Returns

A tensor representing a state.

void set_state(const Tensor &state)

Sets the new state for the next inference.

Parameters

state – The current state to set.

struct Version
#include <version.hpp>

Represents version information that describes plugins and the OpemVINO library.

Public Members

const char *buildNumber

A null terminated string with build number.

const char *description

A null terminated description string.

class VisitorAdapter : public ov::ValueAccessor<void>
#include <attribute_adapter.hpp>

Adapters will see visitor.

Subclassed by ngraph::FactoryAttributeAdapter< BASE_TYPE >, ov::AttributeAdapter< ParameterVector >, ov::AttributeAdapter< ResultVector >, ov::AttributeAdapter< op::AutoBroadcastSpec >, ov::AttributeAdapter< op::BroadcastModeSpec >, ov::AttributeAdapter< ov::NodeVector >, ov::AttributeAdapter< std::shared_ptr< ov::Node > >

namespace batch_util

Functions

void mark_batch(const std::shared_ptr<ov::opset1::Parameter> &parameter, P2Btype &map, const std::unordered_set<label_t> &batches)
void mark_no_batch(const std::shared_ptr<ov::opset1::Parameter> &parameter, P2Btype &map)
void mark_layout_independent_batch(const std::shared_ptr<ov::opset1::Parameter> &parameter, const std::shared_ptr<ov::Node> &result, P2Btype &map)
void mark_with_unique_dimension_labels(const std::shared_ptr<Model> &m, const ov::DimensionTracker &dt)
void restore_original_dimensions(const std::shared_ptr<ov::Model> &model, const std::map<std::shared_ptr<ov::opset1::Parameter>, ov::PartialShape> &parameter_to_shape, bool leave_batch_dynamic = true, bool clear_labels = false)
bool check_batch_tracks_through_all_the_nodes(const std::shared_ptr<ov::Model> &m)
P2Btype find_batch(const std::shared_ptr<ov::Model> &m)
bool detach_detection_output(const std::shared_ptr<ov::Model> &f)
namespace cmp

Enums

enum Bound

Enumerate bounds to compare.

Values:

enumerator NONE
enumerator LOWER
enumerator UPPER
enumerator BOTH

Functions

template<class T, class U, typename std::enable_if<((std::is_signed<T>::value || std::is_same<T, float16>::value || std::is_same<T, bfloat16>::value) && (std::is_signed<U>::value || std::is_same<U, float16>::value || std::is_same<U, bfloat16>::value)) || (std::is_unsigned<T>::value && std::is_unsigned<U>::value)>::type* = nullptr>
constexpr bool lt(T a, U b) noexcept

Compare two values (a < b) in safe way against lossy integer conversion.

Template Parameters
  • T – Type of a value.

  • U – Type of b value.

Parameters
  • a – Value a.

  • b – Value b.

Returns

true if a less b otherwise false.

template<class T, class U>
constexpr bool gt(T a, U b) noexcept

Compare two values (a > b) in safe way against lossy integer conversion.

Template Parameters
  • T – Type of a value.

  • U – Type of b value.

Parameters
  • a – Value a.

  • b – Value b.

Returns

true if a > b otherwise false.

template<class T, class U>
constexpr bool le(T a, U b) noexcept

Compare two values (a <= b) in safe way against lossy integer conversion.

Template Parameters
  • T – Type of a value.

  • U – Type of b value.

Parameters
  • a – Value a.

  • b – Value b.

Returns

true if a <= b otherwise false.

template<class T, class U>
constexpr bool ge(T a, U b) noexcept

Compare two values (a >= b) in safe way against lossy integer conversion.

Template Parameters
  • T – Type of a value.

  • U – Type of b value.

Parameters
  • a – Value a.

  • b – Value b.

Returns

true if a >= b otherwise false.

template<class T, Bound BMode = Bound::NONE>
class Between
#include <compare.hpp>

Compare if value is between lower and upper bounds.

The Between comparator has four modes to check value:

  • Bound::None (lower, upper)

  • Bound::LOWER [lower, upper)

  • Bound::UPPER (lower, upper]

  • Bound::BOTH [lower, upper]

Template Parameters
  • T – Value type to compare.

  • BMode – Compare bounds mode.

template<class T>
class Equal
#include <compare.hpp>

Compare if value is equal to expected.

Template Parameters

T – Value type to compare.

template<class T>
class Less
#include <compare.hpp>

Compare if value is less to expected.

Template Parameters

T – Value type to compare.

namespace coordinates
namespace descriptor

Functions

OPENVINO_API std::string get_ov_tensor_legacy_name (const Tensor &tensor)
OPENVINO_API void set_ov_tensor_legacy_name (Tensor &tensor, const std::string &tensor_name)
OPENVINO_API std::ostream & operator<< (std::ostream &, const ov::descriptor::Tensor &)
class Input
#include <input.hpp>

Public Functions

Input(Node *node, size_t index, Output &output)
Parameters
  • node – The node that owns this input

  • index – The position of this tensor in all input tensors

  • output – The output that supplies a value for this input

Input(Node *node, size_t index)

Create an Input that is not connected to an output.

Parameters
  • node – The node that owns this input

  • index – The position of this tensor in all input tensors

std::shared_ptr<Node> get_node() const
Returns

the node that this is an input of

inline Node *get_raw_pointer_node() const
Returns

the raw pointer to the node that this is an input of

inline size_t get_index() const
Returns

the position within all supplied tensors of this input

inline const Output &get_output() const
Returns

the connected output

inline Output &get_output()
Returns

the connected output

inline bool has_output() const
Returns

true if an output is connected to the input.

const Tensor &get_tensor() const
Returns

the tensor of the connected output

Tensor &get_tensor()
Returns

the tensor of the connected output

void replace_output(const std::shared_ptr<Node> &node, size_t i)

Replace the current output that supplies a value for this input with output i of node.

void replace_output(Output &output)

Replace the current output that supplies a value for this input with output.

void remove_output()

Remove the output from this input. The node will not be valid until another output is supplied.

inline bool get_is_relevant_to_shape() const

See Node::set_input_is_relevant_to_shape for more details.

Returns

true if the value of this input is relevant to the output shapes of the corresponding node. (Usually this is false.)

inline bool get_is_relevant_to_value() const

See Node::set_input_is_relevant_to_value for more details.

Returns

true if the value of this input is relevant to the output value of the corresponding node. (Usually this is true.)

const Shape &get_shape() const
Returns

the shape of the connected output

const PartialShape &get_partial_shape() const
Returns

the partial shape of the connected output

const element::Type &get_element_type() const
Returns

the element type of the connected output

class Output
#include <output.hpp>

Public Functions

Output(Node *node, size_t index, const std::shared_ptr<Tensor> &tensor)
Parameters
  • nodeNode that owns this output.

  • index – Position of the output tensor in all output tensors

  • tensor – The tensor where the value will be written

const Shape &get_shape() const
Returns

the shape of the output

const PartialShape &get_partial_shape() const
Returns

the partial shape of the output

const element::Type &get_element_type() const
Returns

the element type of the output

class Tensor
#include <tensor.hpp>

Compile-time descriptor of a first-class value that is a tensor.

Public Functions

void set_lower_value(const ov::Tensor &value)

sets lower bound value description

void set_upper_value(const ov::Tensor &value)

sets upper bound value description

void set_value_label(const TensorLabel &value_label)

sets value label description

void invalidate_values()

unsets bound value descriptions

inline const ov::Tensor &get_lower_value() const

gets lower bound value description

inline const ov::Tensor &get_upper_value() const

gets upper bound value description

inline TensorLabel get_value_label() const

gets upper bound value description

inline bool has_and_set_bound() const

checks if lower and upper bound are set and point to the same HostTensor

namespace detail

Functions

template<class T> static auto collect_attached_extensions_onnx (std::vector< ov::Extension::Ptr > &res) -> decltype(typename T::template __openvino_framework_map_helper_onnx< T >().get(), void())
template<class>
static auto collect_attached_extensions_onnx(ov::Any) -> void
template<class T> static auto collect_attached_extensions_paddle (std::vector< ov::Extension::Ptr > &res) -> decltype(typename T::template __openvino_framework_map_helper_paddle< T >().get(), void())
template<class>
static auto collect_attached_extensions_paddle(ov::Any) -> void
template<class T> static auto collect_attached_extensions_tensorflow (std::vector< ov::Extension::Ptr > &res) -> decltype(typename T::template __openvino_framework_map_helper_tensorflow< T >().get(), void())
template<class>
static auto collect_attached_extensions_tensorflow(ov::Any) -> void
namespace device

Namespace with device properties.

Enums

enum class Type

Enum to define possible device types.

Values:

enumerator INTEGRATED

Device is integrated into host system.

enumerator DISCRETE

Device is not integrated into host system.

Variables

static constexpr Property<std::string> id = {"DEVICE_ID"}

the property for setting of required device to execute on values: device id starts from “0” - first device, “1” - second device, etc

static constexpr Priorities priorities = {"MULTI_DEVICE_PRIORITIES"}

Device Priorities config option, with comma-separated devices listed in the desired priority.

static constexpr Properties properties = {"DEVICE_PROPERTIES"}

Property to pass set of property values to specified device

Usage Example:

core.compile_model("HETERO"
    ov::device::priorities("GPU", "CPU"),
    ov::device::properties("CPU", ov::enable_profiling(true)),
    ov::device::properties("GPU", ov::enable_profiling(false)));
static constexpr Property<std::string, PropertyMutability::RO> full_name = {"FULL_DEVICE_NAME"}

Read-only property to get a std::string value representing a full device name.

static constexpr Property<std::string, PropertyMutability::RO> architecture = {"DEVICE_ARCHITECTURE"}

Read-only property which defines the device architecture.

static constexpr Property<UUID, PropertyMutability::RO> uuid = {"DEVICE_UUID"}

Read-only property which defines the UUID of the device.

static constexpr Property<LUID, PropertyMutability::RO> luid = {"DEVICE_LUID"}

Read-only property which defines the LUID of the device.

static constexpr Property<Type, PropertyMutability::RO> type = {"DEVICE_TYPE"}

Read-only property to get a type of device. See Type enum definition for possible return values.

static constexpr Property<std::map<element::Type, float>, PropertyMutability::RO> gops = {"DEVICE_GOPS"}

Read-only property which defines Giga OPS per second count (GFLOPS or GIOPS) for a set of precisions supported by specified device.

static constexpr Property<float, PropertyMutability::RO> thermal = {"DEVICE_THERMAL"}

Read-only property to get a float of device thermal.

static constexpr Property<std::vector<std::string>, PropertyMutability::RO> capabilities = {"OPTIMIZATION_CAPABILITIES"}

Read-only property to get a std::vector<std::string> of capabilities options per device.

struct LUID
#include <properties.hpp>

Structure which defines format of LUID.

Public Members

std::array<uint8_t, MAX_LUID_SIZE> luid

Array with luid for a device.

Public Static Attributes

static const uint64_t MAX_LUID_SIZE = 8

Max size of luid array (64 bits)

struct Priorities : public ov::Property<std::string>
#include <properties.hpp>

Type for device Priorities config option, with comma-separated devices listed in the desired priority.

Public Functions

template<typename ...Args>
inline std::pair<std::string, Any> operator()(Args&&... args) const

Constructs device priorities.

Template Parameters

Args – property constructor arguments types

Parameters

args – property constructor arguments

Returns

Pair of name and type erased value.

struct Properties : public ov::Property<std::map<std::string, std::map<std::string, Any>>>
#include <properties.hpp>

Type for property to pass set of properties to specified device.

Public Functions

inline std::pair<std::string, Any> operator()(const AnyMap &config) const

Constructs property.

Parameters

configs – set of property values with names

Returns

Pair of string key representation and type erased property value.

inline std::pair<std::string, Any> operator()(const std::string &device_name, const AnyMap &config) const

Constructs property.

Parameters
  • device_name – device plugin alias

  • config – set of property values with names

Returns

Pair of string key representation and type erased property value.

template<typename ...Properties>
inline util::EnableIfAllStringAny<std::pair<std::string, Any>, Properties...> operator()(const std::string &device_name, Properties&&... configs) const

Constructs property.

Template Parameters

Properties – Should be the pack of std::pair<std::string, ov::Any> types

Parameters
  • device_name – device plugin alias

  • configs – Optional pack of pairs: (config parameter name, config parameter value)

Returns

Pair of string key representation and type erased property value.

struct UUID
#include <properties.hpp>

Structure which defines format of UUID.

Public Members

std::array<uint8_t, MAX_UUID_SIZE> uuid

Array with uuid for a device.

Public Static Attributes

static const uint64_t MAX_UUID_SIZE = 16

Max size of uuid array (128 bits)

namespace capability

Namespace with possible values for ov::device::capabilities property.

Variables

static constexpr const auto FP32 = "FP32"

Device supports fp32 inference.

static constexpr const auto BF16 = "BF16"

Device supports bf16 inference.

static constexpr const auto FP16 = "FP16"

Device supports fp16 inference.

static constexpr const auto INT8 = "INT8"

Device supports int8 inference.

static constexpr const auto INT16 = "INT16"

Device supports int16 inference.

static constexpr const auto BIN = "BIN"

Device supports binary inference.

static constexpr const auto WINOGRAD = "WINOGRAD"

Device supports winograd optimization.

static constexpr const auto EXPORT_IMPORT = "EXPORT_IMPORT"

Device supports compiled model export and import.

namespace element

Typedefs

using TypeVector = std::vector<Type>

Enums

enum class Type_t

Enum to define possible element types.

Values:

enumerator undefined

Undefined element type.

enumerator dynamic

Dynamic element type.

enumerator boolean

boolean element type

enumerator bf16

bf16 element type

enumerator f16

f16 element type

enumerator f32

f32 element type

enumerator f64

f64 element type

enumerator i4

i4 element type

enumerator i8

i8 element type

enumerator i16

i16 element type

enumerator i32

i32 element type

enumerator i64

i64 element type

enumerator u1

binary element type

enumerator u4

u4 element type

enumerator u8

u8 element type

enumerator u16

u16 element type

enumerator u32

u32 element type

enumerator u64

u64 element type

enumerator nf4

nf4 element type

enumerator string

string element type

Functions

constexpr Type undefined(Type_t::undefined)

undefined element type

constexpr Type dynamic(Type_t::dynamic)

dynamic element type

constexpr Type boolean(Type_t::boolean)

boolean element type

constexpr Type bf16(Type_t::bf16)

bf16 element type

constexpr Type f16(Type_t::f16)

f16 element type

constexpr Type f32(Type_t::f32)

f32 element type

constexpr Type f64(Type_t::f64)

f64 element type

constexpr Type i4(Type_t::i4)

i4 element type

constexpr Type i8(Type_t::i8)

i8 element type

constexpr Type i16(Type_t::i16)

i16 element type

constexpr Type i32(Type_t::i32)

i32 element type

constexpr Type i64(Type_t::i64)

i64 element type

constexpr Type u1(Type_t::u1)

binary element type

constexpr Type u4(Type_t::u4)

u4 element type

constexpr Type u8(Type_t::u8)

u8 element type

constexpr Type u16(Type_t::u16)

u16 element type

constexpr Type u32(Type_t::u32)

u32 element type

constexpr Type u64(Type_t::u64)

u64 element type

constexpr Type nf4(Type_t::nf4)

nf4 element type

constexpr Type string(Type_t::string)

string element type

template<typename T>
Type from()
template<> OPENVINO_API Type from< char > ()
template<> OPENVINO_API Type from< bool > ()
template<> OPENVINO_API Type from< float > ()
template<> OPENVINO_API Type from< double > ()
template<> OPENVINO_API Type from< int8_t > ()
template<> OPENVINO_API Type from< int16_t > ()
template<> OPENVINO_API Type from< int32_t > ()
template<> OPENVINO_API Type from< int64_t > ()
template<> OPENVINO_API Type from< uint8_t > ()
template<> OPENVINO_API Type from< uint16_t > ()
template<> OPENVINO_API Type from< uint32_t > ()
template<> OPENVINO_API Type from< uint64_t > ()
template<> OPENVINO_API Type from< ov::bfloat16 > ()
template<> OPENVINO_API Type from< ov::float16 > ()
template<> OPENVINO_API Type from< std::string > ()
OPENVINO_API Type fundamental_type_for (const Type &type)
OPENVINO_API std::ostream & operator<< (std::ostream &out, const ov::element::Type &obj)
OPENVINO_API std::istream & operator>> (std::istream &out, ov::element::Type &obj)
template<class T>
bool is_max_of(const element::Type_t &type, const T &value)

Check if value has got maximum value of ov::element::Type_t.

Template Parameters

TInput value type.

Parameters
  • typeov::element type to get its maximum.

  • valueInput value for check.

Returns

True if input value has got maximum number specified by ov::element type otherwise false.

template<class T>
bool is_min_of(const element::Type_t type, const T &value)

Check if value has got minimum value of ov::element::Type_t.

Template Parameters

TInput value type.

Parameters
  • typeov::element type to get its minimum.

  • valueInput value for check.

Returns

True if input value has got minimum number specified by ov::element type otherwise false.

template<class T, class U = T>
U get_value_or_limit_of(const element::Type_t &type, const T &value)

Checks input value for element type maximum or minimum and return limit or value.

Template Parameters
  • TType of input value.

  • UType of return value. Default same as T.

Parameters
Returns

If value is maximum or minimum get limit of U otherwise value as U.

template<Type_t...>
struct IfTypeOf

Primary template defines suppoted element types.

The list of element types is used to check if runtime value of element type is one in the list. Base on this check the Visitor::visit function is called for specific element type.

Template Parameters

List – of supported ov::element types.

template<Type_t ET, Type_t... Others>
struct IfTypeOf<ET, Others...>
#include <element_visitor.hpp>

Applies visitor action for supported element type defined by template parameters.

Template Parameters

Public Static Functions

template<class Visitor, class ...Args>
static inline auto apply(Type_t et, Args&&... args) -> typename Visitor::result_type

Applies visitor action if input element type is same as ET.

Uses Visitor::visit<ET> function if et == ET, otherwise check input element type against Others.

Template Parameters
  • Visitor – Visitor class implementing visit function.

  • Args – Types of visit parameters.

Parameters
  • etInput element type.

  • args – Visitor arguments.

Returns

Value of result type returned by Visitor.

template<>
struct IfTypeOf<>
#include <element_visitor.hpp>

Applies visitor action for not supported ov::element type.

Public Static Functions

template<class Visitor, class ...Args>
static inline auto apply(Type_t et, Args&&... args) -> typename Visitor::result_type

Applies visitor default action if input element type is not not supported by IfTypeOf.

Uses Visitor::visit non-template function.

Template Parameters
  • Visitor – Visitor class implementing visit function.

  • Args – Types of visit parameters.

Parameters
  • etInput element type.

  • args – Visitor arguments.

Returns

Value of result type returned by Visitor.

template<class R, R... value>
struct NoAction
#include <element_visitor.hpp>

Helper visitor which defines no action for not supported type.

Template Parameters
  • RType of return value.

  • value – Default value returned.

template<>
struct NoAction<void>
#include <element_visitor.hpp>

Helper visitor which defines no action for not supported type if result is void type.

template<class R>
struct NotSupported
#include <element_visitor.hpp>

Helper visitor which throws ov::Exception for not supported element type.

Template Parameters

RType of return value.

class Type
#include <element_type.hpp>

Base class to define element type.

Public Functions

bool compatible(const element::Type &t) const

Checks whether this element type is merge-compatible with t.

Parameters

t – The element type to compare this element type to.

Returns

true if this element type is compatible with t, else false.

Public Static Functions

static bool merge(element::Type &dst, const element::Type &t1, const element::Type &t2)

Merges two element types t1 and t2, writing the result into dst and returning true if successful, else returning false.

To “merge” two element types t1 and t2 is to find the least restrictive element type t that is no more restrictive than t1 and t2, if t exists. More simply:

merge(dst,element::Type::dynamic,t) writes t to dst and returns true

merge(dst,t,element::Type::dynamic) writes t to dst and returns true

merge(dst,t1,t2) where t1, t2 both static and equal writes t1 to dst and returns true

merge(dst,t1,t2) where t1, t2 both static and unequal does nothing to dst, and returns false

namespace exec_model_info

Variables

static const char ORIGINAL_NAMES[] = "originalLayersNames"

Used to get a string of layer names separated by a comma from the original IR, which were fused/merged to the current executable primitive.

static const char IMPL_TYPE[] = "primitiveType"

Used to get a type of the executable primitive.

static const char OUTPUT_PRECISIONS[] = "outputPrecisions"

Used to get output precisions of the executable primitive.

static const char PERF_COUNTER[] = "execTimeMcs"

Used to get a value of execution time of the executable primitive.

static const char OUTPUT_LAYOUTS[] = "outputLayouts"

Used to get output layouts of primitive.

static const char EXECUTION_ORDER[] = "execOrder"

Used to get an execution order of primitive.

static const char LAYER_TYPE[] = "layerType"

Used to get a type of primitive.

static const char RUNTIME_PRECISION[] = "runtimePrecision"

Used to get runtime precision of the executable primitive.

class ExecutionNode : public ov::op::Op
#include <exec_model_info.hpp>

The Execution node which is used to represent node in execution graph.

It contains the following type of information in node runtime information:

  • ExecGraphInfoSerialization::ORIGINAL_NAMES

  • ExecGraphInfoSerialization::IMPL_TYPE

  • ExecGraphInfoSerialization::OUTPUT_PRECISIONS

  • ExecGraphInfoSerialization::PERF_COUNTER

  • ExecGraphInfoSerialization::OUTPUT_LAYOUTS

  • ExecGraphInfoSerialization::EXECUTION_ORDER

  • ExecGraphInfoSerialization::LAYER_TYPE

  • ExecGraphInfoSerialization::RUNTIME_PRECISION

Public Functions

ExecutionNode()

A default constructor with no node inputs and 0 output ports.

ExecutionNode(const ov::OutputVector &arguments, size_t output_size = 1)

Constructs a new execution node with a given parameters.

Parameters
  • arguments[in] Inputs nodes

  • output_size[in] A number of output ports

std::shared_ptr<ov::Node> clone_with_new_inputs(const ov::OutputVector &inputs) const override

Creates a new execution node with the same state, but different input nodes.

Parameters

inputs[in] The input nodes

Returns

A newly created execution node

virtual bool visit_attributes(ov::AttributeVisitor&) override

Visits attributes of the node.

Parameters

visitor[in] An attribute visitor

Returns

Returns true if an operation has completed successfully

namespace frontend

Typedefs

template<typename OVOpType = void>
using OpExtension = ov::frontend::OpExtensionBase<ov::frontend::ConversionExtension, OVOpType>
using FrontEndFactory = std::function<FrontEnd::Ptr()>
using FrontEndVersion = uint64_t

Each frontend plugin is responsible to export get_api_version function returning version of frontend API used for this plugin If version is not matched with OV_FRONTEND_API_VERSION - plugin will not be loaded by FrontEndManager.

using NamedOutputVector = std::vector<NamedOutput>
using CreatorFunction = std::function<OutputVector(const NodeContext&)>
using CreatorFunctionNamed = std::function<std::map<std::string, OutputVector>(const NodeContext&)>
using CreatorFunctionNamedAndIndexed = std::function<NamedOutputVector(const NodeContext&)>

Functions

inline const ov::OpSet &get_opset_by_name(const std::string &opset_name)

The helper function to return an instance of OpSet class initialized with operations from provided opset by name.

Parameters

opset_name – Opset name (opsetN) to initialize OpSet class.

inline std::shared_ptr<ov::Node> create_ov_node_by_name(const std::string &ov_type_name)

The helper function to create an instance of ov::Node class initialized by provided type name. Expected formats:

  • opsetN::OpName

  • opsetN.OpName

  • OpName

Parameters

ov_type_name – Type name of created ov::Node.

inline OutputVector indexed_from_named(const NamedOutputVector &outputs)
inline NamedOutputVector named_from_indexed(const OutputVector &outputs)
class ComplexTypeMark : public ov::op::util::FrameworkNode
#include <complex_type_mark.hpp>

Public Functions

inline virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ConversionExtension : public ov::frontend::ConversionExtensionBase
#include <conversion.hpp>
class ConversionExtensionBase : public ov::Extension
#include <conversion.hpp>

Subclassed by ov::frontend::ConversionExtension

class DecoderBase : public ov::frontend::IDecoder
#include <decoder.hpp>

Public Functions

virtual ov::Any get_attribute(const std::string &name) const = 0

Get attribute value by name.

Parameters

name – Attribute name

Returns

Shared pointer to appropriate value converted to openvino data type if it exists, ‘nullptr’ otherwise

virtual size_t get_input_size() const = 0

Get a number of inputs.

virtual void get_input_node(size_t input_port_idx, std::string &producer_name, std::string &producer_output_port_name, size_t &producer_output_port_index) const = 0

Get a producer name and its output port index.

Parameters
  • input_port_idxInput port index by which data is consumed

  • producer_name – A producer name

  • producer_output_port_nameOutput port name if exists

  • producer_output_port_indexOutput port index from which data is generated

virtual const std::string &get_op_type() const = 0

Get operation type.

virtual const std::string &get_op_name() const = 0

Get node name.

virtual ~DecoderBase()

Destructor.

class DecoderTransformationExtension : public ov::Extension
#include <decoder_transformation.hpp>

Holds a transformation that is applied just after the original model graph is decoded. This class is a holder for transformation. The transformation can be specified as FunctionPass or MathcerPass derivatives or as a function that can be used to build corresponding FunctionPass or MatcherPass object. The type of the extension is determined in the moment of creation by calling corresponding ctor.

Public Functions

explicit DecoderTransformationExtension(const std::function<bool(std::shared_ptr<ov::Model>)> &function_pass)

Create a custom functional pass where code of the pass is implemented as a function.

explicit DecoderTransformationExtension(const std::function<void(ov::pass::MatcherPass*)> &matcher_pass_initializer)

Create a custom matcher pass where the code of matcher pass initialization is a given function.

template<typename Transformation, typename std::enable_if<std::is_base_of<ov::pass::PassBase, Transformation>::value, bool>::type = true>
inline explicit DecoderTransformationExtension(const Transformation &transformation)

Register existing transformation object which will be copied and kept for further registration.

void register_pass(ov::pass::Manager &manager) const

Register pass from this object in a given pass manager object.

struct ExtensionHolder
#include <holder.hpp>
class FrontEnd
#include <frontend.hpp>

An interface for identifying a frontend for a particular framework. Provides an ability to load and convert of input model.

Public Functions

FrontEnd()

Default constructor.

template<typename ...Types>
inline bool supported(const Types&... vars) const

Validates if FrontEnd can recognize model with parameters specified. Same parameters should be used to load model.

Parameters

varsAny number of parameters of any type. What kind of parameters are accepted is determined by each FrontEnd individually, typically it is std::string containing path to the model file. For more information please refer to specific FrontEnd documentation.

Returns

true if model recognized, false - otherwise.

template<typename ...Types>
inline InputModel::Ptr load(const Types&... vars) const

Loads an input model by any specified arguments. Each FrontEnd separately defines what arguments it can accept.

Parameters

varsAny number of parameters of any type. What kind of parameters are accepted is determined by each FrontEnd individually, typically it is std::string containing path to the model file. For more information please refer to specific FrontEnd documentation.

Returns

Loaded input model.

virtual std::shared_ptr<ov::Model> convert(const InputModel::Ptr &model) const

Completely convert and normalize entire Model, throws if it is not possible.

Parameters

modelInput model

Returns

fully converted OV Model

virtual void convert(const std::shared_ptr<ov::Model> &partially_converted) const

Completely convert the remaining, not converted part of a Model.

Parameters

partiallyConverted – partially converted OV Model

virtual std::shared_ptr<ov::Model> convert_partially(const InputModel::Ptr &model) const

Convert only those parts of the model that can be converted leaving others as-is wrapped by FrameworkNode. Converted parts are normalized by additional transformations like it is done in convert method. If part of the graph cannot be converted, it is not guaranteed that the converted regions are completely normalized. Normalize should be called for each completely converted parts individually in this case.

Parameters

modelInput model

Returns

partially converted OV Model

virtual std::shared_ptr<ov::Model> decode(const InputModel::Ptr &model) const

Convert operations with one-to-one mapping with decoding nodes. Each decoding node is an OV node representing a single FW operation node with all attributes represented in FW-independent way.

Parameters

modelInput model

Returns

OV Model after decoding

virtual void normalize(const std::shared_ptr<ov::Model> &model) const

Runs normalization passes on Model that was loaded with partial conversion.

Parameters

Model – partially converted OV Model

virtual std::string get_name() const

Gets name of this FrontEnd. Can be used by clients if frontend is selected automatically by FrontEndManager::load_by_model.

Returns

Current frontend name. Empty string if not implemented

virtual void add_extension(const std::shared_ptr<ov::Extension> &extension)

Register base extension in the FrontEnd.

Parameters

extension – base extension

void add_extension(const std::vector<std::shared_ptr<ov::Extension>> &extensions)

Register base extensions in the FrontEnd.

Parameters

extensions – vector of extensions

void add_extension(const std::string &library_path)

Registers extension.

Parameters

library_path – path to library with ov::Extension

template<class T, typename std::enable_if<std::is_base_of<ov::Extension, T>::value, bool>::type = true>
inline void add_extension(const T &extension)

Registers extension.

Parameters

extensionExtension class which is inherited from ov::BaseOpExtension class

template<class T, class ...Targs, typename std::enable_if<std::is_base_of<ov::Extension, T>::value, bool>::type = true>
inline void add_extension(const T &extension, Targs... args)

Registers extensions.

Parameters

extensionExtension class which is inherited from ov::Extension class

class FrontEndManager
#include <manager.hpp>

Frontend management class, loads available frontend plugins on construction Allows load of frontends for particular framework, register new and list available frontends This is a main frontend entry point for client applications.

Public Functions

FrontEndManager()

Default constructor. Searches and loads of available frontends.

FrontEndManager(FrontEndManager&&) noexcept

Default move constructor.

FrontEndManager &operator=(FrontEndManager&&) noexcept

Default move assignment operator.

~FrontEndManager()

Default destructor.

FrontEnd::Ptr load_by_framework(const std::string &framework)

Loads frontend by name of framework and capabilities.

Parameters

framework – Framework name. Throws exception if name is not in list of available frontends

Returns

Frontend interface for further loading of models

template<typename ...Types>
inline FrontEnd::Ptr load_by_model(const Types&... vars)

Loads frontend by model fragments described by each FrontEnd documentation. Selects and loads appropriate frontend depending on model file extension and other file info (header)

Parameters

varsAny number of parameters of any type. What kind of parameters are accepted is determined by each FrontEnd individually, typically it is std::string containing path to the model file. For more information please refer to specific FrontEnd documentation.

Returns

Frontend interface for further loading of model. Returns ‘nullptr’ if no suitable frontend is found

std::vector<std::string> get_available_front_ends()

Gets list of registered frontends. Any not loaded frontends will be loaded by this call.

void register_front_end(const std::string &name, FrontEndFactory creator)

Register frontend with name and factory creation method.

Parameters
  • name – Name of front end

  • creator – Creation factory callback. Will be called when frontend is about to be created

void register_front_end(const std::string &name, const std::string &library_path)

Register frontend with name and factory loaded from provided library.

Parameters
  • name – Name of front end

  • library_path – Path (absolute or relative) or name of a frontend library. If name is provided, depending on platform, it will be wrapped with shared library suffix and prefix to identify library full name

struct FrontEndPluginInfo
#include <manager.hpp>

Each frontend plugin is responsible to export get_front_end_data function returning heap-allocated pointer to this structure. Will be used by FrontEndManager during loading of plugins.

class FWVisitor : public ov::AttributeVisitor
#include <op.hpp>

Public Functions

inline virtual void on_adapter(const std::string &name, ValueAccessor<void> &adapter) override

handles all specialized on_adapter methods implemented by the visitor.

The adapter implements get_type_info(), which can be used to determine the adapter directly or via is_type and as_type on any platform

class FWVisitorInputAttributes : public ov::AttributeVisitor
#include <op.hpp>

Public Functions

inline virtual void on_adapter(const std::string &name, ValueAccessor<void> &adapter) override

handles all specialized on_adapter methods implemented by the visitor.

The adapter implements get_type_info(), which can be used to determine the adapter directly or via is_type and as_type on any platform

class GeneralFailure : public ov::AssertFailure
#include <exception.hpp>
class IDecoder
#include <decoder.hpp>

Plays a role of node, block and module decoder.

Subclassed by ov::frontend::DecoderBase

class InitializationFailure : public ov::AssertFailure
#include <exception.hpp>
class InputModel
#include <input_model.hpp>

InputModel class represents an original, not yet converted model graph in a framework format given services to find places of interest in a graph or specialize/edit the model before conversion.

Editing requests may affect ability to convert the original model to OV Model. Aim to provide these editing capabilities is to unlock conversion for models that are not natively supported “as-is” because of undefined shapes, types or operations.

Specific front-end implementation is supposed to have a lazy implementation for all methods, not doing a complete load of a model without an explicit method call. For example, the list of all inputs are not pre-fetched by InputModel derived class instance creation, but only when get_inputs method is called. But it is not an obligation, the most convenient way should be chosen depending on the framework model representation.

All editing requests affect the model representation that is held behind the scene successive method calls observe a new graph structure.

Note

Class methods are divided into several groups: searching for places, naming and annotation, topology editing, setting tensor properties.

Public Functions

virtual std::vector<Place::Ptr> get_inputs() const

Returns all inputs for a model An input is a place in a graph where data is supposed to flow inside graph from outside. It can be a tensor, port, operation; which kind of place can be an output is FW dependent. Usually framework models have a dedicated artifact to code model input, it can be a tensor without producer, that writes to it in ONNX, or a special operation like Placeholder in TensorFlow.

Returns

A vector of input place references

virtual std::vector<Place::Ptr> get_outputs() const

Returns all output for a model An output is a terminal place in a graph where data escapes the flow. It can be a tensor, port, operation; which kind of place can be an output is FW dependent. In comparison to a graph input, the output is less formally defined thing and determination of initial list of outputs may include some conventions defined by a frontend itself, not a framework. For example, all output ports without consumers may be considered as outputs.

Returns

A vector of output place references

virtual Place::Ptr get_place_by_tensor_name(const std::string &tensor_name) const

Returns a tensor place by a tensor name following framework conventions, or nullptr if a tensor with this name doesn’t exist.

Parameters

tensor_name – Name of tensor

Returns

Tensor place corresponding to specified tensor name or nullptr if not exists

virtual Place::Ptr get_place_by_operation_name(const std::string &operation_name) const

Returns an operation place by an operation name following framework conventions, or nullptr if an operation with this name doesn’t exist.

Parameters

operation_name – Name of operation

Returns

Place representing operation or nullptr if not exists

virtual Place::Ptr get_place_by_operation_name_and_input_port(const std::string &operation_name, int input_port_index)

Returns an input port place by operation name and appropriate port index.

Parameters
  • operation_name – Name of operation

  • input_port_index – Index of input port for this operation

Returns

Place representing input port of operation or nullptr if not exists

virtual Place::Ptr get_place_by_operation_name_and_output_port(const std::string &operation_name, int output_port_index)

Returns an output port place by operation name and appropriate port index.

Parameters
  • operation_name – Name of operation

  • output_port_index – Index of output port for this operation

Returns

Place representing output port of operation or nullptr if not exists

virtual void set_name_for_tensor(const Place::Ptr &tensor, const std::string &new_name)

Sets name for tensor. Overwrites existing names of this place.

Parameters
  • tensorTensor place

  • new_name – New name for this tensor

virtual void add_name_for_tensor(const Place::Ptr &tensor, const std::string &new_name)

Adds new name for tensor.

Parameters
  • tensorTensor place

  • new_name – New name to be added to this place

virtual void set_name_for_operation(const Place::Ptr &operation, const std::string &new_name)

Sets name for operation. Overwrites existing names of this place.

Parameters
  • operation – Operation place

  • new_name – New name for this operation

virtual void free_name_for_tensor(const std::string &name)

Unassign specified name from tensor place(s)

Parameters

name – Name of tensor

virtual void free_name_for_operation(const std::string &name)

Unassign specified name from operation place(s)

Parameters

name – Name of operation

virtual void set_name_for_dimension(const Place::Ptr &place, size_t shape_dim_index, const std::string &dim_name)

Set name for a particular dimension of a place (e.g. batch dimension)

Parameters
  • placeModel’s place

  • shape_dim_indexDimension index

  • dim_name – Name to assign on this dimension

virtual void cut_and_add_new_input(const Place::Ptr &place, const std::string &new_name_optional = "")

Cut immediately before this place and assign this place as new input; prune all nodes that don’t contribute to any output.

Parameters
  • place – New place to be assigned as input

  • new_name_optional – Optional new name assigned to this input place

virtual void cut_and_add_new_output(const Place::Ptr &place, const std::string &new_name_optional = "")

Cut immediately after this place and assign this place as new output; prune all nodes that don’t contribute to any output.

Parameters
  • place – New place to be assigned as output

  • new_name_optional – Optional new name assigned to this output place

virtual Place::Ptr add_output(const Place::Ptr &place)

Assign this place as new output or add necessary nodes to represent a new output.

Parameters

place – Anchor point to add an output

Returns

new output place, may be the same as a given place

virtual void remove_output(const Place::Ptr &place)

Removes any sinks directly attached to this place with all inbound data flow if it is not required by any other output.

Parameters

placeModel place

virtual void override_all_outputs(const std::vector<Place::Ptr> &outputs)

Replaces all existing outputs with new ones removing all data flow that is not required for new outputs.

Parameters
  • outputs – Vector with places that will become new outputs; may intersect existing outputs.

  • outputs – Array of new output places

virtual void override_all_inputs(const std::vector<Place::Ptr> &inputs)

Modifies the graph to use new inputs instead of existing ones. New inputs should completely satisfy all existing outputs.

Parameters

inputs – Array of new input places

virtual void extract_subgraph(const std::vector<Place::Ptr> &inputs, const std::vector<Place::Ptr> &outputs)

Leaves only subgraph that are defined by new inputs and new outputs.

Parameters
  • inputs – Array of new input places

  • outputs – Array of new output places

virtual void set_partial_shape(const Place::Ptr &place, const ov::PartialShape &shape)

Defines all possible shape that may be used for this place; place should be uniquely refer to some data. This partial shape will be converted to corresponding shape of results OV nodes and will define shape inference when the model is converted to OV.

Parameters
  • placeModel place

  • shape – Partial shape for this place

virtual ov::PartialShape get_partial_shape(const Place::Ptr &place) const

Returns current partial shape used for this place.

Parameters

placeModel place

Returns

Partial shape for this place

virtual void set_element_type(const Place::Ptr &place, const ov::element::Type &type)

Sets new element type for a place.

Parameters
  • placeModel place

  • type – New element type

virtual ov::element::Type get_element_type(const Place::Ptr &place) const

Returns current element type used for this place.

Parameters

placeModel place

Returns

Element type for this place

virtual void set_tensor_value(const Place::Ptr &place, const void *value)

Freezes a tensor with statically defined value or replace existing value for already constant node or tensor.

Parameters
  • placeTensor place

  • value – Value for tensor place representing a memory buffer

virtual void set_tensor_partial_value(const Place::Ptr &place, const void *min_value, const void *max_value)

Defines partial value (lower bound and upper bound) for a tensor place TODO: more details for min_value and max_value format; who defines shape?

Parameters
  • placeTensor place

  • min_value – Lower bound of partial value for tensor place

  • max_value – Upper bound of partial value for tensor place

struct NamedOutput
#include <node_context.hpp>
class NodeContext
#include <node_context.hpp>

Public Functions

inline virtual size_t get_input_size() const

Returns a number of inputs.

inline virtual size_t get_input_size(const std::string &port_name) const

Returns a number of inputs.

inline virtual Output<Node> get_input(int idx) const

Returns exactly one input with a given idx; throws if there is no inputs or there are more than one input.

inline virtual Output<Node> get_input(const std::string &name, int idx) const

Returns exactly one input with a given name and idx; throws if there is no inputs or there are more than one input.

inline virtual Output<Node> get_input(const std::string &name) const

Returns exactly one input with a given name; throws if there is no inputs or there are more than one input.

inline virtual Any get_values_from_const_input(int idx) const

Returns values from Constant input with the given index as ov::Any. Throws an exception if the input cannot be represented as Constant.

template<class T>
inline T get_attribute(const std::string &name) const

Returns node attribute by name.

template<class T>
inline T get_attribute(const std::string &name, const T &def) const

Returns node attribute by name. Returns ‘def’ value if attribute does not exist.

inline bool has_attribute(const std::string &name) const

Check if an attribute of a given name exist.

virtual ov::Any get_attribute_as_any(const std::string &name) const = 0

Returns node attribute by name as ov::Any.

inline virtual size_t get_subgraph_size() const

Returns the number of sub-graphs that can be enumerated with get_subgraph.

inline virtual std::shared_ptr<Model> get_subgraph(int idx) const

Returns subgraph converted on demand by the first access If there is no query for specific sub-graph it shouldn’t be converted idx should be in range 0..get_subgraph_size()-1.

class NotImplementedFailure : public ov::AssertFailure
#include <exception.hpp>
class OpConversionFailure : public ov::AssertFailure
#include <exception.hpp>
class OpConversionFunction
#include <op.hpp>
class OpConversionFunctionInputAttributes
#include <op.hpp>
class OpConversionFunctionNamed
#include <op.hpp>
template<typename BaseConversionType, typename OVOpType = void>
class OpExtensionBase : public BaseConversionType
#include <op.hpp>
template<typename BaseConversionType>
class OpExtensionBase<BaseConversionType, void> : public BaseConversionType
#include <op.hpp>
class OpValidationFailure : public ov::AssertFailure
#include <exception.hpp>
class Place
#include <place.hpp>

An interface for identifying a place in a graph and iterate over it; can refer to an operation node, tensor, port etc.

Place can refer to Tensor, Input Edge, Input Port, Operation, Output Port, Output Edge

           [Tensor A]
               |
               | [Input Edge]
               |
               V
      -------------------
      [  [Input Port 0] ]
      [                 ]
      [   Operation A   ]
      [                 ]
      [ [Output Port 0] ]
      -------------------
               |
               | [Output Edge]
               |
               V
           [Tensor B]
               |
               | [Input Edge]
               |
               V
      -------------------
      [  [Input Port 0] ]
      [                 ]
      [   Operation B   ]
      [                 ]
      [ [Output Port 0] ]
      -------------------
               |
               | [Output Edge]
               |
               V
           [Tensor C]

Note

Each front end implementation provides specialization of this interface to represent a place in a model graph. Various methods in the front end classes accept and retrieve instances of Place to point to particular node part which should be modified or satisfies some criteria. For example, this class is used to report model inputs and outputs, for searching operations and tensors by name, for setting shape etc.

Public Functions

virtual std::vector<std::string> get_names() const

All associated names (synonyms) that identify this place in the graph in a framework specific way.

Returns

A vector of strings each representing a name that identifies this place in the graph. Can be empty if there are no names associated with this place or name cannot be attached.

virtual std::vector<Ptr> get_consuming_operations() const

Returns references to all operation nodes that consume data from this place.

Note

It can be called for any kind of graph place searching for the first consuming operations. It is optional if place has only one output port

Returns

A vector with all operation node references that consumes data from this place

virtual std::vector<Ptr> get_consuming_operations(int output_port_index) const

Returns references to all operation nodes that consume data from this place for specified output port.

Note

It can be called for any kind of graph place searching for the first consuming operations.

Parameters

output_port_index – If place is an operational node it specifies which output port should be considered.

Returns

A vector with all operation node references that consumes data from this place

virtual std::vector<Ptr> get_consuming_operations(const std::string &outputName) const

Returns references to all operation nodes that consume data from this place for specified output port.

Note

It can be called for any kind of graph place searching for the first consuming operations.

Parameters

outputName – If a given place is itself an operation node, this specifies name of output port group

Returns

A vector with all operation node references that consumes data from this place

virtual std::vector<Ptr> get_consuming_operations(const std::string &outputName, int outputPortIndex) const

Returns references to all operation nodes that consume data from this place for specified output port.

Note

It can be called for any kind of graph place searching for the first consuming operations.

Parameters
  • outputName – If a given place is itself an operation node, this specifies name of output port group, each group can have multiple ports

  • outputPortIndex – If place is an operational node it specifies which output port should be considered.

Returns

A vector with all operation node references that consumes data from this place

virtual Ptr get_target_tensor() const

Returns a tensor place that gets data from this place; applicable for operations, output ports and output edges which have only one output port.

Returns

A tensor place which hold the resulting value for this place

virtual Ptr get_target_tensor(const std::string &outputName) const

Returns a tensor place that gets data from this place; applicable for operations.

Parameters

outputName – Name of output port group

Returns

A tensor place which hold the resulting value for this place

virtual Ptr get_target_tensor(const std::string &outputName, int outputPortIndex) const

Returns a tensor place that gets data from this place; applicable for operations.

Parameters
  • outputName – Name of output port group, each group can have multiple ports

  • outputPortIndexOutput port index if the current place is an operation node and has multiple output ports

Returns

A tensor place which hold the resulting value for this place

virtual Ptr get_target_tensor(int output_port_index) const

Returns a tensor place that gets data from this place; applicable for operations.

Parameters

output_port_indexOutput port index if the current place is an operation node and has multiple output ports

Returns

A tensor place which hold the resulting value for this place

virtual Ptr get_source_tensor() const

Returns a tensor place that supplies data for this place; applicable for operations, input ports and input edges which have only one input port.

Returns

A tensor place which supplies data for this place

virtual Ptr get_source_tensor(int input_port_index) const

Returns a tensor place that supplies data for this place; applicable for operations.

Parameters

input_port_indexInput port index for operational nodes.

Returns

A tensor place which supplies data for this place

virtual Ptr get_source_tensor(const std::string &inputName) const

Returns a tensor place that supplies data for this place; applicable for operations.

Parameters

inputName – Name of input port group

Returns

A tensor place which supplies data for this place

virtual Ptr get_source_tensor(const std::string &inputName, int inputPortIndex) const

Returns a tensor place that supplies data for this place; applicable for operations.

Parameters
  • inputName – If a given place is itself an operation node, this specifies name of output port group, each group can have multiple ports

  • inputPortIndexInput port index for operational nodes.

Returns

A tensor place which supplies data for this place

virtual Ptr get_producing_operation() const

Get an operation node place that immediately produces data for this place; applicable if place has only one input port.

Returns

An operation place that produces data for this place

virtual Ptr get_producing_operation(int input_port_index) const

Get an operation node place that immediately produces data for this place.

Parameters

input_port_index – If a given place is itself an operation node, this specifies a port index

Returns

An operation place that produces data for this place

virtual Ptr get_producing_operation(const std::string &inputName) const

Get an operation node place that immediately produces data for this place.

Parameters

inputName – If a given place is itself an operation node, this specifies name of output port group

Returns

An operation place that produces data for this place

virtual Ptr get_producing_operation(const std::string &inputName, int inputPortIndex) const

Get an operation node place that immediately produces data for this place.

Parameters
  • inputName – If a given place is itself an operation node, this specifies name of output port group, each group can have multiple ports

  • inputPortIndex – If a given place is itself an operation node, this specifies a port index

Returns

An operation place that produces data for this place

virtual Ptr get_producing_port() const

Returns a port that produces data for this place.

virtual Ptr get_input_port() const

For operation node returns reference to an input port; applicable if operation node has only one input port.

Returns

Input port place or nullptr if not exists

virtual Ptr get_input_port(int input_port_index) const

For operation node returns reference to an input port with specified index.

Parameters

input_port_indexInput port index

Returns

Appropriate input port place or nullptr if not exists

virtual Ptr get_input_port(const std::string &input_name) const

For operation node returns reference to an input port with specified name; applicable if port group has only one input port.

Parameters

input_name – Name of port group

Returns

Appropriate input port place or nullptr if not exists

virtual Ptr get_input_port(const std::string &input_name, int input_port_index) const

For operation node returns reference to an input port with specified name and index.

Parameters
  • input_name – Name of port group, each group can have multiple ports

  • input_port_indexInput port index in a group

Returns

Appropriate input port place or nullptr if not exists

virtual Ptr get_output_port() const

For operation node returns reference to an output port; applicable for operations with only one output port.

Returns

Appropriate output port place or nullptr if not exists

virtual Ptr get_output_port(int output_port_index) const

For operation node returns reference to an output port with specified index.

Parameters

output_port_indexOutput port index

Returns

Appropriate output port place or nullptr if not exists

virtual Ptr get_output_port(const std::string &output_name) const

For operation node returns reference to an output port with specified name; applicable if port group has only one output port.

Parameters

output_name – Name of output port group

Returns

Appropriate output port place or nullptr if not exists

virtual Ptr get_output_port(const std::string &output_name, int output_port_index) const

For operation node returns reference to an output port with specified name and index.

Parameters
  • output_name – Name of output port group, each group can have multiple ports

  • output_port_indexOutput port index

Returns

Appropriate output port place or nullptr if not exists

virtual std::vector<Place::Ptr> get_consuming_ports() const

Returns all input ports that consume data flows through this place.

virtual bool is_input() const

Returns true if this place is input for a model.

virtual bool is_output() const

Returns true if this place is output for a model.

virtual bool is_equal(const Ptr &another) const

Returns true if another place is the same as this place.

Parameters

another – Another place object

virtual bool is_equal_data(const Ptr &another) const

Returns true if another place points to the same data.

Note

The same data means all places on path: output port -> output edge -> tensor -> input edge -> input port.

Parameters

another – Another place object

class ProgressReporterExtension : public ov::Extension
#include <progress_reporter.hpp>

Public Types

using progress_notifier_callback = std::function<void(float, unsigned int, unsigned int)>

A progress reporting callback signature. A FunctionObject that matches this signature should be passed to the constructor of this extension. The extension will then invoke this as a callback each time the progress needs to be reported. The callback itself is responsible for consuming the reported values.

Param progress

A float value in the range [0.0, 1.0] indicating the total progress of an operation.

Param total_steps

The total number of steps that a given instance of this extension is tracking

Param completed_completed

The current number of completed steps (out of the total number of steps to take)

Public Functions

inline ProgressReporterExtension()

The default constructor which creates a reporter that doesn’t report progress.

void report_progress(float progress, unsigned int total_steps, unsigned int completed_steps) const

The main method of this extension used to report the progress. This method forwards its arguments to the callback stored in this class.

Parameters
  • progress – A float value in the range [0.0, 1.0] indicating the total progress of an operation.

  • total_steps – The total number of steps that a given instance of this extension is tracking

  • completed_steps – The current number of completed steps (out of the total number of steps to take)

class TelemetryExtension : public ov::Extension
#include <telemetry.hpp>

Provides callback to report telemetry information back to Python code.

namespace tensorflow
class GraphIterator : private ov::RuntimeAttribute
#include <graph_iterator.hpp>

Abstract representation for an input model graph that gives nodes in topologically sorted order.

Public Functions

virtual size_t size() const = 0

Get a number of operation nodes in the graph.

virtual void reset() = 0

Set iterator to the start position.

virtual void next() = 0

Move to the next node in the graph.

virtual bool is_end() const = 0

Returns true if iterator goes out of the range of available nodes.

virtual std::shared_ptr<DecoderBase> get_decoder() const = 0

Return a pointer to a decoder of the current node.

virtual ~GraphIterator() = default

Destructor.

virtual std::shared_ptr<GraphIterator> get_body_graph_iterator(const std::string &func_name) const = 0

Checks if the main model graph contains a function of the requested name in the library Returns GraphIterator to this function and nullptr, if it does not exist.

virtual std::vector<std::string> get_input_names() const = 0

Returns a vector of input names in the original order.

virtual std::vector<std::string> get_output_names() const = 0

Returns a vector of output names in the original order.

inline virtual std::map<std::string, std::string> get_input_names_map() const

Returns a map from internal tensor name to (user-defined) external name for inputs.

inline virtual std::map<std::string, std::string> get_output_names_map() const

Returns a map from internal tensor name to (user-defined) external name for outputs.

namespace type
struct List
#include <decoder.hpp>
struct PyNone
#include <decoder.hpp>
struct Str
#include <decoder.hpp>
struct Tensor
#include <decoder.hpp>
namespace helpers

Functions

template<typename ACT, typename ...T, size_t N_ARGS = NumOfLambdaArgs<ACT>::value>
std::enable_if<N_ARGS == sizeof...(T) + 2, void>::type call_with_args(const ACT &body, size_t g_id, size_t iwork, T... arg)
template<typename ACT, typename ...T, size_t N_ARGS = NumOfLambdaArgs<ACT>::value>
std::enable_if<N_ARGS == sizeof...(T) + 1, void>::type call_with_args(const ACT &body, size_t g_id, size_t iwork, T... arg)
template<typename ACT, typename ...T, size_t N_ARGS = NumOfLambdaArgs<ACT>::value>
std::enable_if<N_ARGS == sizeof...(T), void>::type call_with_args(const ACT &body, size_t g_id, size_t iwork, T... arg)
template<typename T>
struct NumOfLambdaArgs
#include <parallel.hpp>
template<typename C, typename R, typename... Args> *)(Args...) const >
#include <parallel.hpp>
namespace hint

Namespace with hint properties.

Enums

enum class Priority

Enum to define possible priorities hints.

Values:

enumerator LOW

Low priority.

enumerator MEDIUM

Medium priority.

enumerator HIGH

High priority.

enumerator DEFAULT

Default priority is MEDIUM.

enum class PerformanceMode

Enum to define possible performance mode hints.

Values:

enumerator OPENVINO_ENUM_DEPRECATED

Undefined value, performance setting may vary from device to device.

enumerator LATENCY

Optimize for latency.

enumerator THROUGHPUT

Optimize for throughput.

enumerator CUMULATIVE_THROUGHPUT

Optimize for cumulative throughput.

enum class SchedulingCoreType

This enum contains definition of core type can be used for CPU tasks on different devices.

Values:

enumerator ANY_CORE

Any processors can be used.

enumerator PCORE_ONLY

Only processors of performance-cores can be used.

enumerator ECORE_ONLY

Only processors of efficient-cores can be used.

enum class ExecutionMode

Enum to define possible execution mode hints.

Values:

enumerator PERFORMANCE

Optimize for max performance, may apply properties which slightly affect accuracy.

enumerator ACCURACY

Optimize for max accuracy.

Variables

static constexpr Property<element::Type, PropertyMutability::RW> inference_precision = {"INFERENCE_PRECISION_HINT"}

Hint for device to use specified precision for inference.

static constexpr Property<Priority> model_priority = {"MODEL_PRIORITY"}

High-level OpenVINO model priority hint Defines what model should be provided with more performant bounded resource first.

static constexpr Property<PerformanceMode> performance_mode = {"PERFORMANCE_HINT"}

High-level OpenVINO Performance Hints unlike low-level properties that are individual (per-device), the hints are something that every device accepts and turns into device-specific settings.

static constexpr Property<SchedulingCoreType> scheduling_core_type = {"SCHEDULING_CORE_TYPE"}

This property defines CPU core type which can be used during inference.

Developer can use this property to select specific CPU cores for inference. Please refer SchedulingCoreType for all definition of core type.

The following code is an example to only use efficient-cores for inference on hybrid CPU. If user sets this configuration on a platform with only performance-cores, CPU inference will still run on the performance-cores.

ie.set_property(ov::hint::scheduling_core_type(ov::hint::SchedulingCoreType::ECORE_ONLY));
static constexpr Property<bool> enable_cpu_pinning = {"ENABLE_CPU_PINNING"}

This property allows CPU threads pinning during inference.

Developer can use this property to use or not use CPU threads pinning during inference. If user does not explicitly set value for this property, OpenVINO may choose any desired value based on internal logic.

The following code is example to use this property.

ie.set_property(ov::hint::enable_cpu_pinning(true));
ie.set_property(ov::hint::enable_cpu_pinning(false));
static constexpr Property<bool> enable_hyper_threading = {"ENABLE_HYPER_THREADING"}

This property define if using hyper threading during inference.

Developer can use this property to use or not use CPU pinning during inference. If user does not explicitly set value for this property, OpenVINO may choose any desired value based on internal logic.

The following code is example to use this property.

ie.set_property(ov::hint::enable_hyper_threading(true));
ie.set_property(ov::hint::enable_hyper_threading(false));
static constexpr Property<uint32_t> num_requests = {"PERFORMANCE_HINT_NUM_REQUESTS"}

(Optional) property that backs the (above) Performance Hints by giving additional information on how many inference requests the application will be keeping in flight usually this value comes from the actual use-case (e.g. number of video-cameras, or other sources of inputs)

static constexpr Property<std::shared_ptr<ov::Model>> model = {"MODEL_PTR"}

This key identifies shared pointer to the ov::Model, required for some properties (ov::max_batch_size and ov::optimal_batch_size)

static constexpr Property<bool, PropertyMutability::RW> allow_auto_batching = {"ALLOW_AUTO_BATCHING"}

Special key for auto batching feature configuration. Enabled by default.

static constexpr Property<ExecutionMode> execution_mode = {"EXECUTION_MODE_HINT"}

High-level OpenVINO Execution hint unlike low-level properties that are individual (per-device), the hints are something that every device accepts and turns into device-specific settings Execution mode hint controls preferred optimization targets (performance or accuracy) for given model.

namespace intel_auto

Namespace with Intel AUTO specific properties.

Enums

enum class SchedulePolicy

Enum to define the policy of scheduling inference request to target device in cumulative throughput mode on AUTO.

Values:

enumerator ROUND_ROBIN
enumerator DEVICE_PRIORITY
enumerator DEFAULT

Default schedule policy is DEVICE_PRIORITY.

Variables

static constexpr Property<bool> enable_startup_fallback = {"ENABLE_STARTUP_FALLBACK"}

auto device setting that enable/disable CPU as acceleration (or helper device) at the beginning

static constexpr Property<bool> enable_runtime_fallback = {"ENABLE_RUNTIME_FALLBACK"}

auto device setting that enable/disable runtime fallback to other devices when infer fails on current selected device

static constexpr Property<SchedulePolicy> schedule_policy = {"SCHEDULE_POLICY"}

High-level OpenVINO model policy hint Defines what scheduling policy should be used in AUTO CUMULATIVE_THROUGHPUT or MULTI case.

namespace intel_cpu

Namespace with Intel CPU specific properties.

Variables

static constexpr Property<bool> denormals_optimization = {"CPU_DENORMALS_OPTIMIZATION"}

This property define whether to perform denormals optimization.

Computation with denormals is very time consuming. FTZ(Flushing denormals to zero) and DAZ(Denormals as zero) could significantly improve the performance, but it does not comply with IEEE standard. In most cases, this behavior has little impact on model accuracy. Users could enable this optimization if no or acceptable accuracy drop is seen. The following code enables denormals optimization

ie.set_property(ov::denormals_optimization(true)); // enable denormals optimization

The following code disables denormals optimization

ie.set_property(ov::denormals_optimization(false)); // disable denormals optimization
static constexpr Property<float> sparse_weights_decompression_rate = {"CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE"}

This property defines threshold for sparse weights decompression feature activation.

Sparse weights decompression feature allows to pack weights for Matrix Multiplication operations directly in the CPU plugin at the model compilation stage and store non-zero values in a special packed format. Then, during the execution of the model, the weights are unpacked and used in the computational kernel. Since the weights are loaded from DDR/L3 cache in the packed format this significantly decreases memory consumption and as a consequence improve inference performance. The following code allows to set the sparse rate value.

core.set_property(ov::intel_cpu::sparse_weights_decompression_rate(0.8));
namespace intel_gna

Namespace with Intel GNA specific properties.

Enums

enum class ExecutionMode

Enum to define software acceleration mode.

Values:

enumerator AUTO

Uses Intel GNA if available, otherwise uses software execution mode on CPU.

enumerator HW

Uses Intel GNA if available, otherwise raises an error.

enumerator HW_WITH_SW_FBACK

Uses Intel GNA if available, otherwise raises an error. If the hardware queue is not empty, automatically falls back to CPU in the bit-exact mode.

enumerator SW_EXACT

Executes the GNA-compiled graph on CPU performing calculations in the same precision as the Intel GNA in the bit-exact mode.

enumerator SW_FP32

Executes the GNA-compiled graph on CPU but substitutes parameters and calculations from low precision to floating point

enum class HWGeneration

Enum to define HW compile and execution targets.

Values:

enumerator UNDEFINED

GNA HW generation is undefined.

enumerator GNA_1_0

GNA HW generation 1.0.

enumerator GNA_1_0_E

GNA HW generation 1.0 embedded.

enumerator GNA_2_0

GNA HW generation 2.0.

enumerator GNA_3_0

GNA HW generation 3.0.

enumerator GNA_3_1

GNA HW generation 3.1.

enumerator GNA_3_5

GNA HW generation 3.5.

enumerator GNA_3_5_E

GNA HW generation 3.5 embedded.

enumerator GNA_3_6

GNA HW generation 3.6.

enumerator GNA_4_0

GNA HW generation 4.0.

enum class PWLDesignAlgorithm

Enum to define PWL design algorithm.

Values:

enumerator UNDEFINED

PWL approximation algorithm is undefined.

enumerator RECURSIVE_DESCENT

Recursive Descent Algorithm.

enumerator UNIFORM_DISTRIBUTION

Uniform distribution algorithm.

Variables

static constexpr Property<std::string, PropertyMutability::RO> library_full_version = {"GNA_LIBRARY_FULL_VERSION"}

Property to get an std::string of GNA Library version, usually in the form <API_REVISION>.<RELEASE_LINE>.<RELEASE>.<BUILD>

static constexpr Property<std::map<std::string, float>> scale_factors_per_input = {"GNA_SCALE_FACTOR_PER_INPUT"}

Scale factor provided by the user to use static quantization. This option should be used with floating point value serialized to string with . (dot) as a decimal separator.

In the case of multiple inputs, individual scale factors can be provided using the map where key is layer name and value is scale factor. The input name shall not contain symbol “:”. Example:

ov::Core core;
auto model = core.read_model(model_path);
std::map<std::string, float> scale_factors;
for (auto& input : model->inputs()) {
    scale_factors[input.get_any_name()] = 1.0f;
}
core.set_property("GNA", ov::intel_gna::scale_factors_per_input(scale_factors));

static constexpr Property<std::string> firmware_model_image_path = {"GNA_FIRMWARE_MODEL_IMAGE"}

if turned on, dump GNA firmware model into specified file

static constexpr Property<ExecutionMode> execution_mode = {"GNA_DEVICE_MODE"}

GNA proc_type setting that should be one of AUTO, HW, GNA_HW_WITH_SW_FBACK, GNA_SW_EXACT or SW_FP32.

static constexpr Property<HWGeneration> execution_target = {"GNA_HW_EXECUTION_TARGET"}

The option to override the GNA HW execution target. May be one of GNA_2_0, GNA_3_0, GNA_3_5. By default (in case of no value set) the behavior depends on GNA HW availability: If GNA HW is present, use the option corresponding to this HW. If HW is not present, use the option corresponding to the latest fully supported GNA HW generation. A fully supported GNA HW generation means it must be supported by both the OV GNA Plugin and the core GNA Library. Currently, the latest supported GNA HW generation corresponds to GNA_3_5.

static constexpr Property<HWGeneration> compile_target = {"GNA_HW_COMPILE_TARGET"}

The option to override the GNA HW compile target. May be one of GNA_2_0, GNA_3_0, GNA_3_5. By default the same as execution_target.

static constexpr Property<bool> memory_reuse = {"GNA_COMPACT_MODE"}

if enabled produced minimum memory footprint for compiled model in GNA memory, default value is true

static constexpr Property<PWLDesignAlgorithm> pwl_design_algorithm = {"GNA_PWL_DESIGN_ALGORITHM"}

The option to set PWL design algorithm. By default the optimized algorithm called “Recursive Descent Algorithm for Finding

the Optimal Minimax Piecewise Linear Approximation of Convex Functions” is used. If value is UNIFORM_DISTRIBUTION then simple uniform distribution is used to create PWL approximation of activation functions. Uniform distribution usually gives poor approximation with the same number of segments.

static constexpr Property<float> pwl_max_error_percent = {"GNA_PWL_MAX_ERROR_PERCENT"}

The option to allow to specify the maximum error percent that the optimized algorithm finding will be used to find PWL functions. By default (in case of NO value set), 1.0 value is used.

namespace intel_gpu

Namespace with Intel GPU specific properties.

Typedefs

using gpu_handle_param = void*

Enums

enum class ContextType

Enum to define the type of the shared context.

Values:

enumerator OCL

Pure OpenCL context.

enumerator VA_SHARED

Context shared with a video decoding device.

enum class SharedMemType

Enum to define the type of the shared memory buffer.

Values:

enumerator OCL_BUFFER

Shared OpenCL buffer blob.

enumerator OCL_IMAGE2D

Shared OpenCL 2D image blob.

enumerator USM_USER_BUFFER

Shared USM pointer allocated by user.

enumerator USM_HOST_BUFFER

Shared USM pointer type with host allocation type allocated by plugin.

enumerator USM_DEVICE_BUFFER

Shared USM pointer type with device allocation type allocated by plugin.

enumerator VA_SURFACE

Shared video decoder surface or D3D 2D texture blob.

enumerator DX_BUFFER

Shared D3D buffer blob.

Variables

static constexpr Property<uint64_t, PropertyMutability::RO> device_total_mem_size = {"GPU_DEVICE_TOTAL_MEM_SIZE"}

Read-only property which defines size of memory in bytes available for the device. For iGPU it returns host memory size, for dGPU - dedicated gpu memory size.

static constexpr Property<std::string, PropertyMutability::RO> uarch_version = {"GPU_UARCH_VERSION"}

Read-only property to get microarchitecture identifier in major.minor.revision format.

static constexpr Property<int32_t, PropertyMutability::RO> execution_units_count = {"GPU_EXECUTION_UNITS_COUNT"}

Read-only property to get count of execution units for current GPU.

static constexpr Property<std::map<std::string, uint64_t>, PropertyMutability::RO> memory_statistics{"GPU_MEMORY_STATISTICS"}

Read-only property to get statistics of GPU memory allocated by engine for each allocation type It contains information about current memory usage.

static constexpr Property<bool> enable_loop_unrolling = {"GPU_ENABLE_LOOP_UNROLLING"}

Turning on this key enables to unroll recurrent layers such as TensorIterator or Loop with fixed iteration count. This key is turned on by default. Turning this key on will achieve better inference performance for loops with not too many iteration counts (less than 16, as a rule of thumb). Turning this key off will achieve better performance for both graph loading time and inference time with many iteration counts (greater than 16). Note that turning this key on will increase the graph loading time in proportion to the iteration counts. Thus, this key should be turned off if graph loading time is considered to be most important target to optimize.

static constexpr Property<bool> disable_winograd_convolution = {"GPU_DISABLE_WINOGRAD_CONVOLUTION"}

Turning on this key disables winograd convolution. Winograd convolution has different characteristics for accuracy and performance compared to other convolution implementations.

static constexpr Property<ContextType> context_type = {"CONTEXT_TYPE"}

Shared device context type: can be either pure OpenCL (OCL) or shared video decoder (VA_SHARED) context.

static constexpr Property<gpu_handle_param> ocl_context = {"OCL_CONTEXT"}

This key identifies OpenCL context handle in a shared context or shared memory blob parameter map.

static constexpr Property<int> ocl_context_device_id = {"OCL_CONTEXT_DEVICE_ID"}

This key identifies ID of device in OpenCL context if multiple devices are present in the context.

static constexpr Property<int> tile_id = {"TILE_ID"}

In case of multi-tile system, this key identifies tile within given context.

static constexpr Property<gpu_handle_param> ocl_queue = {"OCL_QUEUE"}

This key identifies OpenCL queue handle in a shared context.

static constexpr Property<gpu_handle_param> va_device = {"VA_DEVICE"}

This key identifies video acceleration device/display handle in a shared context or shared memory blob parameter map.

static constexpr Property<SharedMemType> shared_mem_type = {"SHARED_MEM_TYPE"}

This key identifies type of internal shared memory in a shared memory blob parameter map.

static constexpr Property<gpu_handle_param> mem_handle = {"MEM_HANDLE"}

This key identifies OpenCL memory handle in a shared memory blob parameter map.

static constexpr Property<gpu_handle_param> dev_object_handle = {"DEV_OBJECT_HANDLE"}

This key identifies video decoder surface handle in a shared memory blob parameter map.

static constexpr Property<uint32_t> va_plane = {"VA_PLANE"}

This key identifies video decoder surface plane in a shared memory blob parameter map.

namespace capability

Possible return value for ov::device::capabilities property.

Variables

static constexpr const auto HW_MATMUL = "GPU_HW_MATMUL"

Device has hardware block for matrix multiplication.

namespace hint

Typedefs

using ThrottleLevel = ov::hint::Priority

This enum represents the possible value of ov::intel_gpu::hint::queue_throttle property:

  • LOW is used for CL_QUEUE_THROTTLE_LOW_KHR OpenCL throttle hint

  • MEDIUM (DEFAULT) is used for CL_QUEUE_THROTTLE_MED_KHR OpenCL throttle hint

  • HIGH is used for CL_QUEUE_THROTTLE_HIGH_KHR OpenCL throttle hint

Variables

static constexpr Property<ThrottleLevel> queue_throttle = {"GPU_QUEUE_THROTTLE"}

This key instructs the GPU plugin to use OpenCL queue throttle hints as defined in https://www.khronos.org/registry/OpenCL/specs/opencl-2.1-extensions.pdf, chapter 9.19. This option should be used with ov::intel_gpu::hint::ThrottleLevel values.

static constexpr Property<ov::hint::Priority> queue_priority = {"GPU_QUEUE_PRIORITY"}

This key instructs the GPU plugin to use the OpenCL queue priority hint as defined in https://www.khronos.org/registry/OpenCL/specs/opencl-2.1-extensions.pdf. This option should be used with ov::hint::Priority:

  • LOW is used for CL_QUEUE_PRIORITY_LOW_KHR OpenCL priority hint

  • MEDIUM (DEFAULT) is used for CL_QUEUE_PRIORITY_MED_KHR OpenCL priority hint

  • HIGH is used for CL_QUEUE_PRIORITY_HIGH_KHR OpenCL priority hint

static constexpr Property<ov::hint::Priority> host_task_priority = {"GPU_HOST_TASK_PRIORITY"}

This key instructs the GPU plugin which cpu core type of TBB affinity used in load network. This option has 3 types of levels: HIGH, LOW, and ANY. It is only affected on Hybrid CPUs.

  • LOW - instructs the GPU Plugin to use LITTLE cores if they are available

  • MEDIUM (DEFAULT) - instructs the GPU Plugin to use any available cores (BIG or LITTLE cores)

  • HIGH - instructs the GPU Plugin to use BIG cores if they are available

static constexpr Property<int64_t> available_device_mem = {"AVAILABLE_DEVICE_MEM_SIZE"}

This key identifies available device memory size in bytes.

namespace memory_type

These keys instruct the GPU plugin to use surface/buffer memory type.

Variables

static constexpr auto surface = "GPU_SURFACE"

Native video decoder surface.

namespace ocl

Namespace with Intel GPU OpenCL specific remote objects.

Typedefs

using gpu_handle_param = void*

Shortcut for defining a handle parameter.

class ClBufferTensor : public ov::RemoteTensor
#include <ocl.hpp>

This class represents an abstraction for GPU plugin remote tensor which can be shared with user-supplied OpenCL buffer. The plugin object derived from this class can be obtained with ClContext::create_tensor() call.

Note

User can obtain OpenCL buffer handle from this class.

Subclassed by ov::intel_gpu::ocl::D3DBufferTensor

Public Functions

inline cl_mem get()

Returns the underlying OpenCL memory object handle.

Returns

underlying OpenCL memory object handle

inline operator cl_mem()

OpenCL memory handle conversion operator.

Returns

cl_mem

inline operator cl::Buffer()

Standard Khronos cl::Buffer wrapper conversion operator.

Returns

cl::Buffer object

Public Static Functions

static inline void type_check(const Tensor &tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters

tensor – a tensor to check

class ClContext : public ov::RemoteContext
#include <ocl.hpp>

This class represents an abstraction for GPU plugin remote context which is shared with OpenCL context object. The plugin object derived from this class can be obtained either with CompiledModel::get_context() or Core::create_context() calls.

Subclassed by ov::intel_gpu::ocl::D3DContext, ov::intel_gpu::ocl::VAContext

Public Functions

inline ClContext(Core &core, cl_context ctx, int ctx_device_id = 0)

Constructs context object from user-supplied OpenCL context handle.

Parameters
  • core – A reference to OpenVINO Runtime Core object

  • ctx – A OpenCL context to be used to create shared remote context

  • ctx_device_id – An ID of device to be used from ctx

inline ClContext(Core &core, cl_command_queue queue)

Constructs context object from user-supplied OpenCL context handle.

Note

Only latency mode is supported for such context sharing case.

Parameters
  • core – A reference to OpenVINO Runtime Core object

  • queue – An OpenCL queue to be used to create shared remote context. Queue will be reused inside the plugin.

inline cl_context get()

Returns the underlying OpenCL context handle.

Returns

cl_context

inline operator cl_context()

OpenCL context handle conversion operator for the ClContext object.

Returns

cl_context

inline operator cl::Context()

Standard Khronos cl::Context wrapper conversion operator for the ClContext object.

Returns

cl::Context object

inline std::pair<ClImage2DTensor, ClImage2DTensor> create_tensor_nv12(const cl::Image2D &nv12_image_plane_y, const cl::Image2D &nv12_image_plane_uv)

This function is used to construct a NV12 compound tensor object from two cl::Image2D wrapper objects. The resulting compound contains two remote tensors for Y and UV planes of the surface.

Parameters
  • nv12_image_plane_y – cl::Image2D object containing Y plane data.

  • nv12_image_plane_uv – cl::Image2D object containing UV plane data.

Returns

A pair of remote tensors for each plane

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl_mem buffer)

This function is used to obtain remote tensor object from user-supplied cl_mem object.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • buffer – A cl_mem object that should be wrapped by a remote tensor

Returns

A remote tensor instance

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl::Buffer &buffer)

This function is used to obtain remote tensor object from user-supplied cl::Buffer object.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • buffer – A cl::Buffer object that should be wrapped by a remote tensor

Returns

A remote tensor instance

inline ClImage2DTensor create_tensor(const element::Type type, const Shape &shape, const cl::Image2D &image)

This function is used to obtain remote tensor object from user-supplied cl::Image2D object.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • image – A cl::Image2D object that should be wrapped by a remote tensor

Returns

A remote tensor instance

inline USMTensor create_tensor(const element::Type type, const Shape &shape, void *usm_ptr)

This function is used to obtain remote tensor object from user-supplied USM pointer.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • usm_ptr – A USM pointer that should be wrapped by a remote tensor

Returns

A remote tensor instance

inline USMTensor create_usm_host_tensor(const element::Type type, const Shape &shape)

This function is used to allocate USM tensor with host allocation type.

Parameters
Returns

A remote tensor instance

inline USMTensor create_usm_device_tensor(const element::Type type, const Shape &shape)

This function is used to allocate USM tensor with device allocation type.

Parameters
Returns

A remote tensor instance

RemoteTensor create_tensor(const element::Type &type, const Shape &shape, const AnyMap &params = {})

Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.

Parameters
  • type – Defines the element type of the tensor.

  • shape – Defines the shape of the tensor.

  • params – Map of the low-level tensor object parameters.

Returns

Pointer to a plugin object that implements the RemoteTensor interface.

Public Static Functions

static inline void type_check(const RemoteContext &remote_context)

Checks that type defined runtime parameters are presented in remote object.

Parameters

remote_context – A remote context to check

class ClImage2DTensor : public ov::RemoteTensor
#include <ocl.hpp>

This class represents an abstraction for GPU plugin remote tensor which can be shared with user-supplied OpenCL 2D Image. The plugin object derived from this class can be obtained with ClContext::create_tensor() call.

Note

User can obtain OpenCL image handle from this class.

Subclassed by ov::intel_gpu::ocl::D3DSurface2DTensor, ov::intel_gpu::ocl::VASurfaceTensor

Public Functions

inline cl_mem get()

Returns the underlying OpenCL memory object handle.

Returns

underlying OpenCL memory object handle

inline operator cl_mem()

OpenCL memory handle conversion operator.

Returns

cl_mem

inline operator cl::Image2D()

Standard Khronos cl::Image2D wrapper conversion operator for the ClContext object.

Returns

cl::Image2D object

Public Static Functions

static inline void type_check(const Tensor &tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters

tensor – a tensor to check

class D3DBufferTensor : public ov::intel_gpu::ocl::ClBufferTensor
#include <dx.hpp>

This class represents an abstraction for GPU plugin remote tensor which is shared with Direct3D 11 buffer. The plugin object derived from this class can be obtained with D3DContext::create_tensor() call.

Note

User can also obtain OpenCL buffer handle from this class.

Public Functions

inline operator ID3D11Buffer*()

ID3D11Buffer conversion operator for the D3DContext object.

Returns

Pointer to underlying ID3D11Buffer interface

Public Static Functions

static inline void type_check(const Tensor &tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters

tensor – a tensor to check

class D3DContext : public ov::intel_gpu::ocl::ClContext
#include <dx.hpp>

This class represents an abstraction for GPU plugin remote context which is shared with Direct3D 11 device. The plugin object derived from this class can be obtained either with CompiledModel::get_context() or Core::create_context() calls.

Note

User can also obtain OpenCL context handle from this class.

Public Functions

inline operator ID3D11Device*()

ID3D11Device conversion operator for the D3DContext object.

Returns

Pointer to underlying ID3D11Device interface

inline D3DContext(Core &core, ID3D11Device *device, int target_tile_id = -1)

Constructs D3DContext remote context object from ID3D11Device.

Parameters
  • core – OpenVINO Runtime Core object instance

  • device – A pointer to ID3D11Device to be used to create a remote context

  • target_tile_id – Desired tile id within given context for multi-tile system. Default value (-1) means that root device should be used

inline std::pair<D3DSurface2DTensor, D3DSurface2DTensor> create_tensor_nv12(const size_t height, const size_t width, ID3D11Texture2D *nv12_surf)

This function is used to obtain a NV12 tensor from NV12 DXGI video decoder output. The resulting tensor contains two remote tensors for Y and UV planes of the surface.

Parameters
  • height – Height of Y plane

  • width – Width of Y plane

  • nv12_surf – A ID3D11Texture2D instance to create NV12 tensor from

Returns

A pair of remote tensors for each plane

inline D3DBufferTensor create_tensor(const element::Type type, const Shape &shape, ID3D11Buffer *buffer)

This function is used to obtain remote tensor object from ID3D11Buffer.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • buffer – A pointer to ID3D11Buffer instance to create remote tensor based on

Returns

A remote tensor instance

inline D3DSurface2DTensor create_tensor(const element::Type type, const Shape &shape, ID3D11Texture2D *surface, uint32_t plane = 0)

This function is used to obtain remote tensor object from ID3D11Texture2D.

Note

The underlying ID3D11Texture2D can also be a plane of output surface of DXGI video decoder

Parameters
  • typeTensor element type

  • shapeTensor shape

  • surface – Pointer to ID3D11Texture2D interface of the objects that owns NV12 texture

  • plane – ID of the plane to be shared (0 or 1)

Returns

D3DSurface2DTensor tensor

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl_mem buffer)

This function is used to obtain remote tensor object from user-supplied cl_mem object.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • buffer – A cl_mem object that should be wrapped by a remote tensor

Returns

A remote tensor instance

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl::Buffer &buffer)

This function is used to obtain remote tensor object from user-supplied cl::Buffer object.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • buffer – A cl::Buffer object that should be wrapped by a remote tensor

Returns

A remote tensor instance

inline ClImage2DTensor create_tensor(const element::Type type, const Shape &shape, const cl::Image2D &image)

This function is used to obtain remote tensor object from user-supplied cl::Image2D object.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • image – A cl::Image2D object that should be wrapped by a remote tensor

Returns

A remote tensor instance

inline USMTensor create_tensor(const element::Type type, const Shape &shape, void *usm_ptr)

This function is used to obtain remote tensor object from user-supplied USM pointer.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • usm_ptr – A USM pointer that should be wrapped by a remote tensor

Returns

A remote tensor instance

Public Static Functions

static inline void type_check(const RemoteContext &remote_context)

Checks that type defined runtime parameters are presented in remote object.

Parameters

remote_context – A remote context to check

class D3DSurface2DTensor : public ov::intel_gpu::ocl::ClImage2DTensor
#include <dx.hpp>

This class represents an abstraction for GPU plugin remote tensor which is shared with Direct3D 11 2D texture. The plugin object derived from this class can be obtained with D3DContext::create_tensor() call.

Note

User can also obtain OpenCL 2D image handle from this class.

Public Functions

inline operator ID3D11Texture2D*()

ID3D11Texture2D conversion operator for the D3DContext object.

Returns

Pointer to underlying ID3D11Texture2D interface

inline uint32_t plane()

Returns plane ID of underlying video decoder surface, or 0 if no video surface was shared.

Returns

Plane ID

Public Static Functions

static inline void type_check(const Tensor &remote_tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters

remote_tensor – remote tensor to check

class USMTensor : public ov::RemoteTensor
#include <ocl.hpp>

This class represents an abstraction for GPU plugin remote tensor which can be shared with user-supplied USM device pointer. The plugin object derived from this class can be obtained with ClContext::create_tensor() call.

Note

User can obtain USM pointer from this class.

Public Functions

inline void *get()

Returns the underlying USM pointer.

Returns

underlying USM pointer

Public Static Functions

static inline void type_check(const Tensor &tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters

tensor – a tensor to check

class VAContext : public ov::intel_gpu::ocl::ClContext
#include <va.hpp>

This class represents an abstraction for GPU plugin remote context which is shared with VA display object. The plugin object derived from this class can be obtained either with CompiledModel::get_context() or Core::create_context() calls.

Note

User can also obtain OpenCL context handle from this class.

Public Functions

inline operator VADisplay()

VADisplay conversion operator for the VAContext object.

Returns

Underlying VADisplay object handle

inline VAContext(Core &core, VADisplay device, int target_tile_id = -1)

Constructs remote context object from VA display handle.

Parameters
  • core – OpenVINO Runtime Core object

  • device – A VADisplay to create remote context from

  • target_tile_id – Desired tile id within given context for multi-tile system. Default value (-1) means that root device should be used

inline std::pair<VASurfaceTensor, VASurfaceTensor> create_tensor_nv12(const size_t height, const size_t width, const VASurfaceID nv12_surf)

This function is used to obtain a NV12 tensor from NV12 VA decoder output. The resulting tensor contains two remote tensors for Y and UV planes of the surface.

Parameters
  • height – A height of Y plane

  • width – A width of Y plane

  • nv12_surf – NV12 VASurfaceID to create NV12 from

Returns

A pair of remote tensors for each plane

inline VASurfaceTensor create_tensor(const element::Type type, const Shape &shape, const VASurfaceID surface, const uint32_t plane = 0)

This function is used to create remote tensor from VA surface handle.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • surface – A VASurfaceID to create remote tensor from

  • plane – An index of a plane inside VASurfaceID to create tensor from

Returns

A remote tensor wrapping VASurfaceID

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl_mem buffer)

This function is used to obtain remote tensor object from user-supplied cl_mem object.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • buffer – A cl_mem object that should be wrapped by a remote tensor

Returns

A remote tensor instance

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl::Buffer &buffer)

This function is used to obtain remote tensor object from user-supplied cl::Buffer object.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • buffer – A cl::Buffer object that should be wrapped by a remote tensor

Returns

A remote tensor instance

inline ClImage2DTensor create_tensor(const element::Type type, const Shape &shape, const cl::Image2D &image)

This function is used to obtain remote tensor object from user-supplied cl::Image2D object.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • image – A cl::Image2D object that should be wrapped by a remote tensor

Returns

A remote tensor instance

inline USMTensor create_tensor(const element::Type type, const Shape &shape, void *usm_ptr)

This function is used to obtain remote tensor object from user-supplied USM pointer.

Parameters
  • typeTensor element type

  • shapeTensor shape

  • usm_ptr – A USM pointer that should be wrapped by a remote tensor

Returns

A remote tensor instance

RemoteTensor create_tensor(const element::Type &type, const Shape &shape, const AnyMap &params = {})

Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.

Parameters
  • type – Defines the element type of the tensor.

  • shape – Defines the shape of the tensor.

  • params – Map of the low-level tensor object parameters.

Returns

Pointer to a plugin object that implements the RemoteTensor interface.

Public Static Functions

static inline void type_check(const RemoteContext &remote_context)

Checks that type defined runtime parameters are presented in remote object.

Parameters

remote_context – A remote context to check

class VASurfaceTensor : public ov::intel_gpu::ocl::ClImage2DTensor
#include <va.hpp>

This class represents an abstraction for GPU plugin remote tensor which is shared with VA output surface. The plugin object derived from this class can be obtained with VAContext::create_tensor() call.

Note

User can also obtain OpenCL 2D image handle from this class.

Public Functions

inline operator VASurfaceID()

VASurfaceID conversion operator for the VASurfaceTensor object.

Returns

VASurfaceID handle

inline uint32_t plane()

Returns plane ID of underlying video decoder surface.

Returns

Plane ID

Public Static Functions

static inline void type_check(const Tensor &tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters

tensor – a tensor to check

namespace internal

Variables

static constexpr Property<std::vector<PropertyName>, PropertyMutability::RO> supported_properties{"INTERNAL_SUPPORTED_PROPERTIES"}

Read-only property to get a std::vector<PropertyName> of supported internal properties.

static constexpr Property<std::vector<PropertyName>, PropertyMutability::RO> caching_properties = {"CACHING_PROPERTIES"}

Read-only property to get a std::vector<PropertyName> of properties which should affect the hash calculation for model cache.

static constexpr Property<bool, PropertyMutability::RW> exclusive_async_requests = {"EXCLUSIVE_ASYNC_REQUESTS"}

Allow to create exclusive_async_requests with one executor.

static constexpr Property<std::string, PropertyMutability::WO> config_device_id = {"CONFIG_DEVICE_ID"}

the property for setting of required device for which config to be updated values: device id starts from “0” - first device, “1” - second device, etc note: plugin may have different devices naming convention

static constexpr Property<ov::threading::IStreamsExecutor::ThreadBindingType, PropertyMutability::RW> cpu_bind_thread{"CPU_BIND_THREAD"}

The name for setting CPU affinity per thread option.

It is passed to Core::get_property()

The following options are implemented only for the TBB as a threading option ov::threading::IStreamsExecutor::ThreadBindingType::NUMA (pinning threads to NUMA nodes, best for real-life, contented cases) on the Windows and MacOS* this option behaves as YES ov::threading::IStreamsExecutor::ThreadBindingType::HYBRID_AWARE (let the runtime to do pinning to the cores types, e.g. prefer the “big” cores for latency tasks) on the hybrid CPUs this option is default

Also, the settings are ignored, if the OpenVINO compiled with OpenMP and any affinity-related OpenMP’s environment variable is set (as affinity is configured explicitly)

static constexpr Property<size_t, PropertyMutability::RW> threads_per_stream = {"THREADS_PER_STREAM"}

Limit #threads that are used by IStreamsExecutor to execute parallel_for calls.

static constexpr Property<std::string, PropertyMutability::RO> compiled_model_runtime_properties{"COMPILED_MODEL_RUNTIME_PROPERTIES"}

It contains compiled_model_runtime_properties information to make plugin runtime can check whether it is compatible with the cached compiled model, the result is returned by get_property() calling.

The information details are defined by plugin itself, each plugin may require different runtime contents. For example, CPU plugin will contain OV version, while GPU plugin will contain OV and GPU driver version, etc. Core doesn’t understand its content and only read it from plugin and write it into blob header.

static constexpr Property<bool, PropertyMutability::RO> compiled_model_runtime_properties_supported{"COMPILED_MODEL_RUNTIME_PROPERTIES_SUPPORTED"}

Check whether the attached compiled_model_runtime_properties is supported by this device runtime.

namespace itt
namespace domains

Functions

OV_ITT_DOMAIN(ov_eval)
namespace layout

Functions

OPENVINO_API bool has_batch (const Layout &layout)

Checks if layout has ‘batch’ dimension.

OPENVINO_API std::int64_t batch_idx (const Layout &layout)

Returns ‘batch’ dimension index.

Throws

ov::AssertFailure – if dimension doesn’t exist.

OPENVINO_API bool has_channels (const Layout &layout)

Checks if layout has ‘channels’ dimension.

Throws

ov::AssertFailure – if dimension doesn’t exist.

OPENVINO_API std::int64_t channels_idx (const Layout &layout)

Returns ‘channels’ dimension index.

Throws

ov::AssertFailure – if dimension doesn’t exist.

OPENVINO_API bool has_depth (const Layout &layout)

Checks if layout has ‘depth’ dimension.

OPENVINO_API std::int64_t depth_idx (const Layout &layout)

Returns ‘depth’ dimension index.

Throws

ov::AssertFailure – if dimension doesn’t exist.

OPENVINO_API bool has_height (const Layout &layout)

Checks if layout has ‘height’ dimension.

OPENVINO_API std::int64_t height_idx (const Layout &layout)

Returns ‘height’ dimension index.

Throws

ov::AssertFailure – if dimension doesn’t exist.

OPENVINO_API bool has_width (const Layout &layout)

Checks if layout has ‘width’ dimension.

OPENVINO_API std::int64_t width_idx (const Layout &layout)

Returns ‘width’ dimension index.

Throws

ov::AssertFailure – if dimension doesn’t exist.

OPENVINO_API void set_layout (ov::Output< ov::Node > output, const ov::Layout &layout)

Sets Layout of port.

Throws

ov::Exception – if port is not connected with Result or Parameter

OPENVINO_API ov::Layout get_layout (const ov::Output< ov::Node > &output)

Gets Layout of port.

Returns

layout from port and empty layout in other case

OPENVINO_API ov::Layout get_layout (const ov::Output< const ov::Node > &output)

Gets Layout of port.

Returns

layout from port and empty layout in other case

namespace legacy_convert

Functions

ov::SoPtr<ov::IRemoteContext> convert_remote_context(const std::shared_ptr<InferenceEngine::RemoteContext> &context)
InferenceEngine::Blob *get_hardware_blob(InferenceEngine::Blob *blob)
class TensorHolder
#include <remote_utils.hpp>

Subclassed by ov::TensorRemoteBlob

namespace log

Namespace with log level property and its possible values.

Enums

enum class Level

Enum to define possible log levels.

Values:

enumerator NO

disable any logging

enumerator ERR

error events that might still allow the application to continue running

enumerator WARNING

potentially harmful situations which may further lead to ERROR

enumerator INFO

informational messages that display the progress of the application at coarse-grained level

enumerator DEBUG

fine-grained events that are most useful to debug an application.

enumerator TRACE

finer-grained informational events than the DEBUG

Variables

static constexpr Property<Level> level = {"LOG_LEVEL"}

the property for setting desirable log level.

namespace op

Enums

enum class GeluApproximationMode

Specifies the approximation to calculate Gelu.

Values:

enumerator TANH
enumerator ERF
enum class LSTMWeightsFormat

Values:

enumerator FICO
enumerator ICOF
enumerator IFCO
enumerator IFOC
enumerator IOFC
enum class MVNEpsMode

Specifies how eps is applied in MVN.

Values:

enumerator INSIDE_SQRT
enumerator OUTSIDE_SQRT
enum class PadMode

Modes for the Pad operator.

Values:

enumerator CONSTANT
enumerator EDGE
enumerator REFLECT
enumerator SYMMETRIC
enum class PadType

Padding Type used for Convolution and Pooling

Follows ONNX padding type definitions EXPLICIT - Pad dimensions are explicity specified SAME_LOWER - Pad dimensions computed to match input shape Ceil(num_dims/2) at the beginning and Floor(num_dims/2) at the end SAME_UPPER - Pad dimensions computed to match input shape Floor(num_dims/2) at the beginning and Ceil(num_dims/2) at the end VALID - No padding AUTO - Deprecated. User should not use it in the future NOTSET - Deprecated. User should not use it in the future

Values:

enumerator EXPLICIT
enumerator SAME_LOWER
enumerator SAME_UPPER
enumerator VALID
enumerator AUTO
enumerator NOTSET
enum class RoundingType

Rounding Type used for Pooling operators.

Values:

enumerator FLOOR
enumerator CEIL
enum class AutoBroadcastType

Specifies the algorithm to use for implicit broadcasting of a tensor to align with another tensor.

NONE - No implicit broadcasting of tensor NUMPY - Numpy-style implicit broadcasting (https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) Right-align dimensions of the two tensors, with missing dimensions treated as size 1 dimensions. After alignment, for each dimension, their sizes should either match or one of them should be of size 1. Size 1 dimension will be implicitly broadcast to match the other size.

E.g., A: Shape(2, 1, 6) B: Shape( 3, 1) Result: Shape(2, 3, 6)

 A: Shape(2, 1, 6)
 B: Shape(   3, 1)
Result: Shape(2, 3, 6) PDPD - PaddlePaddle-style implicit broadcasting (https://github.com/PaddlePaddle/Paddle/blob/release/1.5/paddle/ fluid/operators/elementwise/elementwise_op.h#L126) Broadcast B to match the shape of A, where axis is the start dimension index to align B with A. If axis is -1 (default), i axis = rank(A) - rank(B). The trailing dimensions of size 1 for B will be ignored.

E.g., A: Shape(2, 3, 4, 5) B: Shape( 3, 4 ) with axis =1 Result: Shape(2, 3, 4, 5)

 A: Shape(2, 3, 4, 5)
 B: Shape(   3, 1   ) with axis = 1
Result: Shape(2, 3, 4, 5)

Values:

enumerator NONE
enumerator EXPLICIT
enumerator NUMPY
enumerator PDPD
enum class BroadcastType

BroadcastType specifies rules used for mapping of input tensor axes to output shape axes.

EXPLICIT - Mapping of the input data shape to output shape based on axes_mapping input. NUMPY - Numpy broadcasting rules, aligned with ONNX Broadcasting. (https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md) PDPD - PaddlePaddle-style implicit broadcasting. For more informaction see AutoBroadcastType documentation. BIDIRECTIONAL - The broadcast rule is similar to numpy.array(input) * numpy.ones(target_shape). Dimensions are right alignment.

Note

Broadcasting rules are different for Broadcast op and for element-wise ops. AutoBroadcastType::NUMPY is equivalent of BroadcastType::BIDIRECTIONAL according to spec.

Values:

enumerator NONE
enumerator EXPLICIT
enumerator NUMPY
enumerator PDPD
enumerator BIDIRECTIONAL
enum class EpsMode

Specifies how eps is combined with L2 value.

Values:

enumerator ADD
enumerator MAX
enum class TopKSortType

Values:

enumerator NONE
enumerator SORT_INDICES
enumerator SORT_VALUES
enum class TopKMode

Values:

enumerator MAX
enumerator MIN
enum class RecurrentSequenceDirection

This class defines possible recurrent sequence directions.

Values:

enumerator FORWARD
enumerator REVERSE
enumerator BIDIRECTIONAL

Functions

std::unordered_map< size_t, std::pair< ov::Tensor, ov::Tensor > > OPENVINO_API convert_input_types (OutputVector &inputs, const element::TypeVector &types)
ov::TensorVector OPENVINO_API get_output_tensors_of_original_type (const ov::TensorVector &fake_output_tensors, const element::TypeVector &types)
void OPENVINO_API reset_input_types (const std::unordered_map< size_t, std::pair< ov::Tensor, ov::Tensor > > &original_input_vals, OutputVector &inputs)
bool OPENVINO_API convert_outputs_to_fake_type (ov::TensorVector &outputs, ov::TensorVector &original_outputs, bool is_upper)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const GeluApproximationMode &type)
ov::op::util::LSTMWeightsFormat convert_lstm_weights_enums(LSTMWeightsFormat format)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const MVNEpsMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const v12::ScatterElementsUpdate::Reduction &reduction)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const PadMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const PadType &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const RoundingType &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const AutoBroadcastType &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const BroadcastType &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const EpsMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const TopKSortType &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const TopKMode &type)
OPENVINO_API std::ostream & operator<< (std::ostream &s, const RecurrentSequenceDirection &direction)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> copy_shape_infer(const Node *op, const std::vector<TShape> &input_shapes)
template<class OpType, class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> eltwise_shape_infer(const OpType *op, const std::vector<T> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const util::FFTBase *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::GatherBase *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::ConvertColorI420Base *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::MulticlassNmsBase *op, const std::vector<TShape> &input_shapes, const bool static_output = !std::is_same<PartialShape, TShape>::value, const bool ignore_bg_class = false)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::ConvertColorNV12Base *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::PadBase *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> reduce_shape_infer(const util::ReductionBase *op, bool keep_dims, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::ArithmeticReductionKeepDims *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::LogicalReductionKeepDims *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::ScatterElementsUpdateBase *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::ScatterNDBase *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::TopKBase *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())

TopK shape inference.

Template Parameters

TShape – Type of shape.

Parameters
  • op – Pointer to TopK operator.

  • input_shapesInput shapes of TopK.

  • constant_data – Map of constant data. DEfault empty.

Returns

Vector of output shapes for

template<class TShape, class TData, class TRes = std::vector<TData>, class UnaryOperation = ov::util::Cast<TData>, typename std::enable_if<!std::is_same<TShape, ov::PartialShape>::value>::type* = nullptr>
std::unique_ptr<TRes> get_input_const_data_as(const ov::Node *op, size_t idx, const ITensorAccessor &tensor_accessor, UnaryOperation &&func = ov::util::Cast<TData>())

Get the operator’s input const as pointer to vector of specified type.

The behaviour depends on shape type. The default output type is std::vector<TData> can be replace by other type which if is possible to construct it from constant data vector.

The behaviour depends on shape type. The default output type is std::vector<TData> can be replace by other type which if is possible to construct it from constant data vector.

Template Parameters
  • TShapeShape type which enabled this version (not ov::PartialShape)

  • TData – Type use to cast input’s data.

  • TRes – Result type which has got default type as std::vector<TData>.

  • UnaryOperation – Unary function object applied on data with signature (Ret f(const TData &a)).

  • TShapeShape type which enabled this version (ov::PartialShape)

  • TData – Type use to cast input’s data.

  • TRes – Result type which has got default type as std::vector<TData>.

  • UnaryOperation – Unary function object applied on data with signature (Ret f(const TData &a)).

Parameters
  • op – Pointer to operator.

  • idx – Operator’s input number.

  • tensor_accessorTensor accessor object.

  • func – Unary operation function object.

  • op – Pointer to operator.

  • idx – Operator’s input number.

  • tensor_accessorTensor accessor object.

  • func – Unary operation function object.

Returns

Pointer to constant data or nullptr if input has no constant data.

Returns

Pointer to constant data or nullptr if input has no constant data.

template<class TShape, class TDimValue = typename TShape::value_type::value_type, class UnaryOperation = ov::util::InTypeRange<TDimValue>, typename std::enable_if<!std::is_same<TShape, ov::PartialShape>::value>::type* = nullptr>
ov::optional<TShape> get_input_const_data_as_shape(const ov::Node *op, size_t port, const ITensorAccessor &tensor_accessor, UnaryOperation &&func = ov::util::InTypeRange<TDimValue>())

Get the input const data as shape object.

The input data can be processed by unary operation. By default is validated and casted to shape’s dimension type.

Template Parameters
  • TShapeShape type.

  • TDimValueDimension value type.

  • UnaryOperation – Unary function object applied on data with signature (Ret f(const TDimValue &a)).

Parameters
  • op – Pointer to operator.

  • portInput port number.

  • tensor_accessorTensor accessor object.

  • func – Unary operation function object to apply in input data. Default ov::utils::InTypeRange<TDimValue>.

Returns

Unique pointer to shape created from input data.

inline element::Type get_input_const_element_type(const ov::Node *const op, size_t port, const ITensorAccessor &ta)
template<class TShape, class TData, class TResult = std::vector<std::pair<TData, TData>>>
ov::optional<TResult> get_input_bounds(const ov::Node *op, size_t port, const ITensorAccessor &ta)

Get the input bounds from constant input or try evaluate bunds and return them as vector of pairs (lower, upper).

Template Parameters
  • TShapeShape type.

  • TData – Bound value type.

Parameters
  • op – Operator pointer.

  • portInput port number.

  • taTensor accessor to constant data.

Returns

Return optional vector of bounds as pair lower, upper when evaluated successful.

ov::Shape infer_broadcast_shape(const ov::Node *const op, const ov::Shape &first, const ov::Shape &second)

Inference broadcast shape for element wise operator according to broadcast specification stored in operator.

Parameters
  • op – Pointer to operator.

  • first – First input shape.

  • second – Second input shape.

Returns

Result shape from inputs with applied broadcast specification.

ov::Shape infer_broadcast_shape(const ov::Node *const op, const ov::TensorVector &inputs)

Inference broadcast shape from input tensor shapes for element wise operator according to broadcast specification stored in operator.

Parameters
  • op – Pointer to operator.

  • inputs – Tensors vector to get theirs shapes.

Returns

Result shape from input tensors shape with applied broadcast specification.

struct AutoBroadcastSpec
#include <attr_types.hpp>

Implicit broadcast specification.

struct BroadcastModeSpec
#include <attr_types.hpp>

Implicit broadcast specification.

class Op : public ov::Node
#include <op.hpp>

Root of all actual ops.

Subclassed by ov::exec_model_info::ExecutionNode, ov::op::Sink, ov::op::internal::NonMaxSuppressionIEInternal, ov::op::internal::RPE, ov::op::util::BinaryElementwiseArithmetic, ov::op::util::BinaryElementwiseBitwise, ov::op::util::BinaryElementwiseComparison, ov::op::util::BinaryElementwiseLogical, ov::op::util::BroadcastBase, ov::op::util::ConvertColorI420Base, ov::op::util::ConvertColorNV12Base, ov::op::util::ConvolutionBase, ov::op::util::DetectionOutputBase, ov::op::util::EmbeddingBagOffsetsBase, ov::op::util::EmbeddingBagPackedBase, ov::op::util::FFTBase, ov::op::util::GatherBase, ov::op::util::GatherNDBase, ov::op::util::IndexReduction, ov::op::util::InterpolateBase, ov::op::util::MaxPoolBase, ov::op::util::MultiSubGraphOp, ov::op::util::MulticlassNmsBase, ov::op::util::PadBase, ov::op::util::RNNCellBase, ov::op::util::ReadValueBase, ov::op::util::ReductionBase, ov::op::util::ScatterBase, ov::op::util::ScatterElementsUpdateBase, ov::op::util::ScatterNDBase, ov::op::util::ShapeOfBase, ov::op::util::TopKBase, ov::op::util::UnaryElementwiseArithmetic, ov::op::v0::BatchNormInference, ov::op::v0::CTCGreedyDecoder, ov::op::v0::Concat, ov::op::v0::Constant, ov::op::v0::Convert, ov::op::v0::CumSum, ov::op::v0::DepthToSpace, ov::op::v0::FakeQuantize, ov::op::v0::HardSigmoid, ov::op::v0::Interpolate, ov::op::v0::LRN, ov::op::v0::MVN, ov::op::v0::MatMul, ov::op::v0::NormalizeL2, ov::op::v0::PRelu, ov::op::v0::PSROIPooling, ov::op::v0::Parameter, ov::op::v0::PriorBox, ov::op::v0::PriorBoxClustered, ov::op::v0::Proposal, ov::op::v0::ROIPooling, ov::op::v0::Range, ov::op::v0::RegionYolo, ov::op::v0::ReorgYolo, ov::op::v0::Result, ov::op::v0::ReverseSequence, ov::op::v0::Selu, ov::op::v0::ShuffleChannels, ov::op::v0::SpaceToDepth, ov::op::v0::Squeeze, ov::op::v0::Tile, ov::op::v0::Unsqueeze, ov::op::v10::IsFinite, ov::op::v10::IsInf, ov::op::v10::IsNaN, ov::op::v10::Unique, ov::op::v12::GroupNormalization, ov::op::v13::BitwiseNot, ov::op::v13::FakeConvert, ov::op::v13::Multinomial, ov::op::v13::NMSRotated, ov::op::v13::ScaledDotProductAttention, ov::op::v1::AvgPool, ov::op::v1::BatchToSpace, ov::op::v1::ConvertLike, ov::op::v1::DeformablePSROIPooling, ov::op::v1::GatherTree, ov::op::v1::LogicalNot, ov::op::v1::NonMaxSuppression, ov::op::v1::OneHot, ov::op::v1::Reshape, ov::op::v1::Reverse, ov::op::v1::Select, ov::op::v1::Softmax, ov::op::v1::SpaceToBatch, ov::op::v1::Split, ov::op::v1::StridedSlice, ov::op::v1::Transpose, ov::op::v1::VariadicSplit, ov::op::v3::Bucketize, ov::op::v3::EmbeddingSegmentsSum, ov::op::v3::ExtractImagePatches, ov::op::v3::NonMaxSuppression, ov::op::v3::NonZero, ov::op::v3::ROIAlign, ov::op::v4::CTCLoss, ov::op::v4::Range, ov::op::v4::Swish, ov::op::v5::BatchNormInference, ov::op::v5::LogSoftmax, ov::op::v5::NonMaxSuppression, ov::op::v6::CTCGreedyDecoderSeqLen, ov::op::v6::ExperimentalDetectronDetectionOutput, ov::op::v6::ExperimentalDetectronGenerateProposalsSingleImage, ov::op::v6::ExperimentalDetectronPriorGridGenerator, ov::op::v6::ExperimentalDetectronROIFeatureExtractor, ov::op::v6::ExperimentalDetectronTopKROIs, ov::op::v6::GatherElements, ov::op::v6::MVN, ov::op::v7::Einsum, ov::op::v7::Roll, ov::op::v8::AdaptiveAvgPool, ov::op::v8::AdaptiveMaxPool, ov::op::v8::MatrixNms, ov::op::v8::PriorBox, ov::op::v8::RandomUniform, ov::op::v8::Slice, ov::op::v8::Softmax, ov::op::v9::Eye, ov::op::v9::GenerateProposals, ov::op::v9::GridSample, ov::op::v9::NonMaxSuppression, ov::op::v9::ROIAlign

Public Functions

inline virtual const ::ov::Node::type_info_t &get_type_info() const override

Returns the NodeTypeInfo for the node’s class. During transition to type_info, returns a dummy type_info for Node if the class has not been updated yet.

class Sink : public ov::op::Op
#include <sink.hpp>

Root of nodes that can be sink nodes.

Subclassed by ov::op::util::AssignBase

class TemporaryReplaceOutputType
#include <type_relaxed.hpp>

Set another type for a specified output for the period of time when an instance of the class exists. When the execution leaves the scope where an onject of TemporaryReplaceOutputType is defined, the type of the output is set to its original value. Used when initialized TypeRelaxed<BaseOp> operation in case when inputs have types that are not compatible with BaseOp infer function. In this case before TypeRelaxed is constructed the BaseOp contructor requires modified data types. So it should be

Public Functions

inline TemporaryReplaceOutputType(Output<Node> output, element::Type tmp_type)

Replace element type for a given output port by tmp_type.

inline Output<Node> get() const

Return the output port that was used in the constructor.

inline ~TemporaryReplaceOutputType()

Restores the original element type for the output.

template<typename BaseOp>
class TypeRelaxed : public BaseOp, public ov::op::TypeRelaxedBase
#include <type_relaxed.hpp>

Relaxes tensor element type requirements for BaseOp inputs and outputs This class template should be used with Node descendant class. Defines a new operation by extending the original BaseOp operation with ability to accept inputs and provide outputs with element type that is unusual for BaseOp. For example, TypeRelaxed<opset1::Add> can accept mixed-precision inputs and provide another type of output. New types are provided as inputs attributes for TypeRelaxed template and fixed. There is no any deduction logic for types are provided as a part of this class and it should be implemented outside if required.

Public Functions

template<typename ...Args>
inline TypeRelaxed(const element::TypeVector &_input_data_types, const element::TypeVector &_output_data_types, Args&&... args)

Creating a new TypeRelaxed operation by calling one of the original op ctors forwarding arguments directly.

class TypeRelaxedBase
#include <type_relaxed.hpp>

A base class for templated TypeRelaxed that maintains overridden input types and output types for an operation.

Subclassed by ov::op::TypeRelaxed< BaseOp >

Public Functions

inline const element::Type &get_overridden_output_type(size_t outputIndex = 0) const

This method may look similar to Node::get_output_element_type, but it is not the same thing, because get_output_element_type returns the result of type inference, so it is completely deduced from an operation inputs and attributes, and get_overridden_output_type returns value of the attribute that is used to deduce output type. In some cases they don’t match: get_overridden_output_type may return element::undefined for some index i, and get_output_element_type will return some real type for the same index i.

Returns

Data type that will be set for output with a given index outputIndex. If output with a specified index outputIndex hasn’t been set before, element::undefined will returned. Undefined means no type override happens for a given outputIndex and it will deduced as original operation defineds in its infer function.

inline void set_overridden_output_type(const element::Type &element_type, size_t outputIndex = 0)

Set data type that overrides the original data type for output port with outputIndex index In case if outputIndex is out of range of known outputs (and this class cannot detect the real number of outputs for original operation), the number of overridden outputs is changed according to a given outputIndex value.

inline const element::Type &get_origin_input_type(size_t inputIndex = 0) const
Returns

Data type that will be set for input when original shape/type inference function is called. If index inputIndex hasn’t been set before, element::undefined will returned. Undefined means that the type from input tensor descriptor is used for a given index.

inline void set_origin_input_type(const element::Type &element_type, size_t inputIndex = 0)

Set data type that overrides the original data type for input port with inputIndex index. In case if inputIndex is out of range of known inputs (and this class cannot detect the real number of inputs for original operation), the number of overridden inputs is changed according to a given inputIndex value. All new entries except one added at inputIndex position are undefined.

namespace convolution

Functions

template<class TOp, class TShape, typename std::enable_if<std::is_base_of<util::ConvolutionBackPropBase, TOp>::value>::type* = nullptr>
size_t calculate_num_spatial(const TOp *op, const std::vector<TShape> &input_shapes, const result_shape_t<TShape> &out_spatial_shape)
template<class TOp, class TShape, class TIter>
void apply_auto_pad(const TOp *op, const TShape &data_shape, const TShape &filters_shape, const result_shape_t<TShape> &out_spatial_shape, TIter pads_begin, TIter pads_end)

Apply auto padding for backward convolution.

The auto padding can be applied only if inputs and attributes of operator are validated. The input shapes must have got static ranks.

Parameters
  • op – Pointer to convolution operator.

  • data_shapeInput data shape (must be static rank).

  • filters_shapeInput filter shape (must be static rank).

  • out_spatial_shape – Reference to input with out spatial shape.

  • pads_begin – Iterator to begin of pads begin.

  • pads_end – Iterator to begin of pads end.

template<class TShape>
void apply_padding(const util::ConvolutionBackPropBase *op, const std::vector<TShape> &input_shapes, const result_shape_t<TShape> &out_spatial_shape, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)

Apply auto padding for back propagation convolutions.

Template Parameters

TShapeShape type.

Parameters
  • op – Pointer to back propagation convolution operator.

  • data_shapeInput data shape.

  • filters_shapeInput filter shape.

  • out_spatial_shapeInput output spatial shape.

template<class TOp, class TShape, class TContainer, typename std::enable_if<std::is_base_of<ov::op::util::ConvolutionBackPropBase, TOp>::value>::type* = nullptr>
void append_spatial_shape(const TOp *op, const TShape &data_shape, const TShape &filters_shape, const TContainer &pads_begin, const TContainer &pads_end, result_shape_t<TShape> &out_shape)

Append spatial dimension at end of output shape of back propagation convolution.

Template Parameters
  • TOp – Back propagation convolution operator type.

  • TShape – Type of shape.

Parameters
  • op – Pointer to operator.

  • data_shapeInput data shape.

  • filters_shapeInput filter shape.

  • out_shapeOutput shape to append spatial dimensions.

template<class TConv>
constexpr size_t filter_non_spatial_dims_count()

Provides convolution filter non spatial dimension count.

Note

If specific convolution operator requires different value provide specialization for this operator.

Template Parameters

TConv – Type of convolution operator.

Returns

Default value for convolution operators (2).

template<class TOp>
bool is_auto_pad(const TOp *op)

Checks if Op property auto_pad is set to same lower or upper.

Template Parameters

TOp – Type of operator (must have get_auto_pad member function).

Parameters

op – Pointer to operator.

Returns

True if auto pad enabled.

inline void resize_empty_padding(const size_t num_spatial, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)

Resize paddings if empty to number of spatial dimensions.

Parameters
  • num_spatial – Number of spatial dimensions.

  • pads_begin – Begin padding to resize.

  • pads_end – End padding to resize.

inline size_t num_spatial_from_attr(const util::ConvolutionBase *op)
template<class TOp, class TShape, typename std::enable_if<std::is_base_of<util::ConvolutionFwdPropBase, TOp>::value>::type* = nullptr>
size_t calculate_num_spatial(const TOp *op, const std::vector<TShape> &input_shapes)
template<class TOp, class TShape, class TIter, typename std::enable_if<std::is_base_of<util::ConvolutionFwdPropBase, TOp>::value || std::is_base_of<util::DeformableConvolutionBase, TOp>::value>::type* = nullptr>
void apply_auto_pad(const TOp *op, const TShape &data_shape, const TShape &filters_shape, TIter pads_begin, TIter pads_end)

Apply auto padding for forward convolution.

The auto padding can be applied only if inputs and attributes of operator are validated. The input shapes must have got static ranks.

Parameters
  • op – Pointer to convolution operator.

  • data_shapeInput data shape (must be static rank).

  • filters_shapeInput filter shape (must be static rank).

  • pads_begin – Iterator to begin of pads begin.

  • pads_end – Iterator to begin of pads end.

template<class TOp, class TShape, typename std::enable_if<std::is_base_of<util::ConvolutionFwdPropBase, TOp>::value || std::is_base_of<util::DeformableConvolutionBase, TOp>::value>::type* = nullptr>
void apply_padding(const TOp *op, const TShape &data_shape, const TShape &filters_shape, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)

Apply padding to forward propagation convolution besed on padding.

Template Parameters

TShape

Parameters
  • op – Pointer to coevolution operator.

  • data_shapeInput data shapes for shape inference.

  • filters_shapeInput filters shape for shape inference.

  • pads_begin – Begin padding to updated.

  • pads_end – End padding to update.

template<class TOp, class TShape, class TRShape = result_shape_t<TShape>, typename std::enable_if<std::is_base_of<util::ConvolutionFwdPropBase, TOp>::value || std::is_base_of<util::DeformableConvolutionBase, TOp>::value>::type* = nullptr>
void append_spatial_shape(const TOp *op, const TShape &data_shape, const TShape &filters_shape, CoordinateDiff &pads_begin, CoordinateDiff &pads_end, TRShape &out_shape)

Append spatial dimension at end of output shape of forward propagation convolution.

Template Parameters
  • TOp – Forward propagation convolution operator type.

  • TShape – Type of shape.

Parameters
  • op – Pointer to operator.

  • data_shapeInput data shape.

  • filters_shapeInput filter shape.

  • out_shapeOutput shape to append spatial dimensions.

template<>
constexpr size_t filter_non_spatial_dims_count<v1::GroupConvolutionBackpropData>()

Defines non-spatial dimension for filters for group convolution back propagation operator.

Returns

Value of non-spatial filter dimensions (3).

template<>
constexpr size_t filter_non_spatial_dims_count<v1::GroupConvolution>()

Defines non-spatial dimension for filters for group convolution operator.

Returns

Value of non-spatial filter dimensions (3).

Variables

constexpr auto num_spatial_undefined = util::num_spatial_undefined
constexpr size_t spatial_dim_offset = 2
namespace validate

Functions

template<class TShape>
void data_shape(const v1::BinaryConvolution *op, const TShape &data_shape)

Specific check of data shape for binary convolution data shape must be rank 4.

The shape_infer is same as for Convolution operator except this check.

See also

convolution_shape_inference.hpp

template<class TShape>
void filter_shape(const ov::op::util::ConvolutionBackPropBase *op, const TShape &filters_shape, const TShape &data_shape)
template<class TShape>
void data_shape(const ov::op::util::ConvolutionBase *op, const TShape &data_shape)
template<class TShape>
void filter_shape(const ov::op::util::ConvolutionBase *op, const TShape &filters_shape, const TShape &data_shape)
inline void common_attributes(const util::ConvolutionBase *op, const size_t num_spatial, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end)
inline void common_attributes(const util::ConvolutionBackPropBase *op, const size_t num_spatial, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end)
namespace deformable_conv

Functions

template<class TShape>
size_t calculate_num_spatial(const util::DeformableConvolutionBase *op, const std::vector<TShape> &input_shapes)
namespace validate

Functions

template<class TDeformableConv>
void group_attribute(const TDeformableConv *op, int64_t group, const std::string &name)
template<class TDeformableConv, class TDim>
void group_divisible_dimension(const TDeformableConv *op, const TDim &dim, const std::string name)
template<class TDeformableConv, class TDim>
void deformable_group_divisible_dimension(const TDeformableConv *op, const TDim &dim, const std::string name)
namespace detectron
namespace validate

Functions

std::pair<std::vector<PartialShape>, element::Type> all_inputs_same_floating_type(const Node *const op)

Validates if all op’s inputs have got same floating type and return inputs shapes and element type.

Parameters

op – Pointer to detector operator.

Returns

Input shapes and element type as pair.

namespace eye

Variables

constexpr std::array<char const*, 4> shape_names = {"'num_rows'", "'num_columns'", "'diagonal_index'", "'batch_shape'"}
namespace fft

Functions

template<class TRShape, typename std::enable_if<std::is_same<TRShape, PartialShape>::value>::type* = nullptr>
void apply_dims_from_sizes(const util::FFTBase *op, TRShape &output_shape, const std::vector<int64_t> &axes, const ITensorAccessor &ta)
namespace gather_nd

Functions

template<class TOp, class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> gather_nd_base_shape_infer(const TOp *op, const std::vector<TShape> &input_shapes)
namespace internal

Functions

template<class ShapeType, class TRShape = result_shape_t<ShapeType>>
std::vector<TRShape> shape_infer(const ov::op::internal::AUGRUCell *op, const std::vector<ShapeType> &input_shapes)
template<class ShapeType, class TRShape = result_shape_t<ShapeType>>
std::vector<TRShape> shape_infer(const ov::op::internal::AUGRUSequence *op, const std::vector<ShapeType> &input_shapes)
class AUGRUCell : public ov::op::util::RNNCellBase
#include <augru_cell.hpp>

AUGRUCell operation.

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class AUGRUSequence : public ov::op::util::RNNCellBase
#include <augru_sequence.hpp>

AUGRUSequence operation.

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class GenerateProposalsIEInternal : public ov::op::v9::GenerateProposals
#include <generate_proposals_ie_internal.hpp>

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class MulticlassNmsIEInternal : public opset9::MulticlassNms
#include <multiclass_nms_ie_internal.hpp>
template<typename BaseNmsOp>
class NmsStaticShapeIE : public BaseNmsOp
#include <nms_static_shape_ie.hpp>

Public Functions

inline NmsStaticShapeIE(const Output<Node> &boxes, const Output<Node> &scores, const Attributes &attrs)

Constructs a NmsStaticShapeIE operation.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • attrs – Attributes of the operation

class NonMaxSuppressionIEInternal : public ov::op::Op
#include <nms_ie_internal.hpp>

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class RPE : public ov::op::Op
#include <rotary_positional_embeddings.hpp>

Rotary Positional Embeddings operation Internal operation which may change in the future.

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

namespace interpolate

Functions

template<class T, class U, typename std::enable_if<std::is_same<T, U>::value>::type* = nullptr>
constexpr bool is_same_instance(const T &lhs, const U &rhs)
template<class TContainer>
void resize_padding(const ov::op::util::InterpolateBase *op, size_t input_rank, TContainer &pads_begin, TContainer &pads_end)

Resize padding to input rank.

Template Parameters

TContainer – Pads container type.

Parameters
  • op – Pointer to base of interpolate.

  • input_rank – Expected padding size.

  • pads_begin – Begin padding container.

  • pads_end – End padding container.

template<class TShape, class TInputIter, class TRShape = result_shape_t<TShape>>
TRShape make_padded_shape(const TShape &input, TInputIter pads_begin, TInputIter pads_end)

Makes padded shape from input shapes and padding values.

Note

The input shape must be static rank and padding count must match input shape rank.

Parameters
  • inputInput shape used as source for output result.

  • pads_begin – Dimensions begin padding values.

  • pads_end – Dimensions end padding values.

Returns

TShape Shape with dimensions of input plus paddings.

template<class TShape, class TRes = std::vector<int64_t>>
std::unique_ptr<TRes> get_axes(const Node *const op, size_t port, bool has_axes, size_t rank, const ITensorAccessor &ta)

Get the axes for interpolate from constant input or default value if op has no axes input.

Parameters
  • op – Pointer to operator.

  • port – Axes input port number.

  • has_axes – Flag if op has input with axes.

  • rank – input shape used for axes values validation.

  • taTensor accessor for input data.

Returns

Not null pointer with axes values or null pointer if can’t get axes from input.

template<class TShape, class TContainer>
void set_undefined_dim_on_axes(TShape &out, TContainer &axes)

Set the undefined dimensions on specified axes.

Parameters
  • outOutput shape to update.

  • axes – List of axes for update.

template<class TShape, class TContainer>
void update_dims_with_sizes_on_axes(TShape &out_shape, const TContainer &axes, const Node *const op, const size_t port, const ITensorAccessor &ta)

Update output shape with dimension size from input on specified axes.

Parameters
  • out_shapeOutput shape to be updated.

  • axes – List of axes for dimension update.

  • op – Pointer to operator.

  • port – Sizes/output target shape input with values

  • taTensor accessor.

template<class TShape>
void update_dims_with_scales_on_axes(TShape &out_shape, const std::vector<int64_t> &axes, const Node *const op, const size_t port, const ITensorAccessor &ta)

Update output shape by scaling dimensions on axes.

Parameters
  • out_shapeOutput shape to update.

  • axes – List of axes to scale dimension.

  • op – Pointer to operator.

  • port – Scales input port number with values.

  • taTensor accessor.

namespace validate

Functions

template<class TShape>
void input_rank_1d(const Node *const op, const std::vector<TShape> &shapes, size_t port)

Validates that input at port number from is 1-D rank.

Template Parameters

TShape

Parameters
  • op – Pointer to operator.

  • shapes – Vector of op’s input shapes.

  • port – Port number.

template<class TShape>
void are_inputs_except_first_1d(const Node *const op, const std::vector<TShape> &shapes)

Validates that inputs from 2nd to last are compatible 1-D rank.

Template Parameters

TShape

Parameters
  • op – Pointer to operator.

  • shapes – Vector of op’s input shapes.

template<class TContainer>
void axes_values(const Node *const op, const TContainer &axes, size_t rank)

Check if axes values in range [0,rank].

Template Parameters

TContainer – Type of axes container.

Parameters
  • op – Pointer to operator.

  • axes – Container with axes to check.

  • rank – Maximum value for axes values.

inline void input_elements_num(const Node *const op, const std::string &input_name, size_t element_count, size_t exp_count)

Check if number of elements in input is same as expected.

Parameters
  • op – Pointer to operator.

  • input_nameInput name.

  • element_count – Element count in tested input.

  • exp_count – Expected element count on tested input.

namespace multiclass_nms
namespace validate

Functions

template<class TShape>
void scores_shape(const Node *const op, const std::vector<TShape> &input_shapes)
template<class TShape>
void rois_num_shape(const Node *const op, const std::vector<TShape> &input_shapes)
template<class TShape>
void num_boxes(const Node *const op, const std::vector<TShape> &input_shapes)
namespace multinomial
namespace validate

Functions

void input_types(const Node *op)
namespace nms

Functions

template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Node *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Node *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta, const bool static_output)
namespace validate

Functions

template<class TShape>
bool scalar(const TShape &shape)
template<class TShape>
bool scalar_or_1d_tensor_with_1_element(const TShape &shape)
template<class TShape>
void boxes_shape(const Node *const op, const std::vector<TShape> &input_shapes)
template<class TShape>
void scores_shape(const Node *const op, const std::vector<TShape> &input_shapes)
template<class TShape>
void num_batches(const Node *const op, const std::vector<TShape> &input_shapes)
template<class TShape>
void num_boxes(const Node *const op, const std::vector<TShape> &input_shapes)
template<class TShape>
void boxes_last_dim(const Node *const op, const std::vector<TShape> &input_shapes)
template<class T>
void shapes(const Node *op, const std::vector<T> &input_shapes)
namespace pad

Functions

constexpr bool is_inf_padding(const std::pair<int64_t, int64_t> &pad_bounds)
namespace pooling

Functions

template<>
inline void valid_dilated_kernel_with_padding(const v1::AvgPool *op, const size_t kernel, const size_t pad_begin, const size_t pad_end, const size_t axis)
template<class TContainer>
void resize_empty_padding(const size_t num_spatial, TContainer &pads_begin, TContainer &pads_end)

Resize paddings if empty to number of spatial dimensions.

Parameters
  • num_spatial – Number of spatial dimensions.

  • pads_begin – Begin padding to resize.

  • pads_end – End padding to resize.

template<class TOp, class TShape, class TContainer>
void apply_padding(const TOp *op, const TShape &data_shape, const Strides &dilations, TContainer &pads_begin, TContainer &pads_end)

Apply pooling operator padding depends on auto pad value.

Parameters
  • op – Pointer to Pooling operator to apply padding.

  • data_shapeShape infer data input shape.

  • dilations – Kernel dilations.

  • pads_begin – Padding begin to update.

  • pads_end – Padding end to update.

template<class TOp, class TDim>
void valid_dilated_kernel_with_dim(const TOp *op, const size_t kernel, const TDim &dim, const size_t axis)
template<class TOp>
void valid_dilated_kernel_with_padding(const TOp *op, const size_t kernel, const size_t pad_begin, const size_t pad_end, const size_t axis)
template<class TOp, class TShape, class TContainer, class TRShape>
void append_spatial_shape(const TOp *op, const TShape &data_shape, const TContainer &pads_begin, const TContainer &pads_end, const Strides &dilations, TRShape &out_shape)

Append spatial shape to the end of output shape for pooling operator shape inference result.

Parameters
  • op – Pointer to pooling operator.

  • data_shapeShape inference input pooling data shape.

  • pads_begin – Pooling pads begin.

  • pads_end – Pooling pads end.

  • dilations – Kernel dilations.

  • out_shapeOutput shape for appending the spatial shape of pooling

template<class TOp, class TShape, class TContainer, class TRShape = result_shape_t<TShape>>
TRShape out_shape_infer(const TOp *op, const TShape &data_shape, const TContainer &pads_begin, const TContainer &pads_end, const Strides &dilations)

Shape inference helper used for pooling operators such Max Pool, Avg Pool.

template<class TShape, class TOp, class TRShape = result_shape_t<TShape>, typename std::enable_if<std::is_same<TOp, v8::AdaptiveAvgPool>::value || std::is_same<TOp, v8::AdaptiveMaxPool>::value>::type* = nullptr>
TRShape out_shape_infer(const TOp *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta)

Shape inference helper used for adaptive pooling operators.

Variables

constexpr size_t spatial_dim_offset = 2
namespace validate

Functions

template<class TOp, class TContainer>
void padding(const TOp *op, const TContainer &pads_begin, const TContainer &pads_end)
template<class TOp, class TShape>
void attributes(const TOp *op, const TShape &data_shape, const Strides &dilations)
namespace prior_box

Functions

template<class TDim, class TOp, typename std::enable_if<std::is_same<v0::PriorBox, TOp>::value || std::is_same<v8::PriorBox, TOp>::value>::type* = nullptr>
TDim number_of_priors(const TOp *const op)
template<class TDim>
TDim number_of_priors(const v0::PriorBoxClustered *const op)
template<class TOp, class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const TOp *const op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())

Variables

constexpr std::array<char const*, 2> input_names = {"output size", "image"}
namespace validate

Functions

inline std::vector<PartialShape> inputs_et(const Node *const op)
namespace proposal

Functions

template<class TOp, class TShape, class TRShape = result_shape_t<TShape>>
TRShape shape_infer_boxes(const TOp *op, const std::vector<TShape> &input_shapes)
namespace psroi_pooling
namespace validate

Functions

template<class TROIPooling, class TShape>
void feat_input_shape(const TROIPooling *op, const TShape feat_shape)
template<class TROIPooling>
void output_group_attr(const TROIPooling *op)
template<class TROIPooling>
void bins_attr(const TROIPooling *op)
template<class TROIPooling>
void mode_attr(const TROIPooling *op)
namespace reshape

Functions

template<class TDim, typename std::enable_if<std::is_same<typename std::decay<TDim>::type, Dimension>::value>::type* = nullptr>
TDim resolve_minus_one_dim(const Product<TDim> &product)
template<class TShape>
std::pair<TShape, int64_t> get_pattern_and_minus_one_idx(const Node *const op, const std::vector<std::pair<int64_t, int64_t>> &bounds)

Get the pattern and minus one idx from input bounds.

Parameters
  • op – Pointer to reshape node.

  • bounds – Vector of reshape pattern bounds.

Returns

Pair which got bounds converted to shape and minus_one index in pattern (-1 if not found).

template<class TShape, typename std::enable_if<std::is_same<TShape, PartialShape>::value>::type* = nullptr>
void set_pattern_labels(const Node *const op, TShape &shape)

Set the pattern labels on pattern shape if this input is labeled.

Shapes other than PartialShape have no labels.

Parameters
  • op – Pointer to reshape node.

  • shape – Pointer to shape for labels set.

template<class T, class U = void>
struct Product
#include <reshape_shape_inference.hpp>
template<class T> type >
#include <reshape_shape_inference.hpp>

Helper to resolve the input and output product for ov::Dimension (dynamic) dimensions.

template<class T> type >
#include <reshape_shape_inference.hpp>

Helper to resolve the input and output product for static dimensions.

namespace rnn

Functions

template<class TShape>
void validate_inputs_rank(const op::util::RNNCellBase *op, const std::vector<TShape> &input_shapes, const std::vector<Rank> &expected_ranks)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> cell_base_shape_infer(const op::util::RNNCellBase *op, const std::vector<TShape> &input_shapes, size_t num_gates, size_t num_state_nodes, bool linear_before_reset = false)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> seq_base_shape_infer(const op::util::RNNCellBase *op, const std::vector<TShape> &input_shapes, size_t num_gates, size_t num_state_nodes, op::RecurrentSequenceDirection direction, bool linear_before_reset = false)
namespace roi_align

Functions

template<class OpType, class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const OpType *op, const std::vector<TShape> &input_shapes)
namespace validate

Functions

inline element::Type data_and_roi_et(const Node *const op)

Validates ROIs align input data and ROIs element type.

Parameters

op – Pointer to ROIs align node.

Returns

Valid ROIs align output element type.

inline void batch_indicies_et(const Node *const op)

Check ROIs align batch indicies input element type.

Parameters

op – Pointer to ROIs align node.

namespace roi_pooling
namespace validate

Functions

template<class TROIPooling, class TShape>
void feat_intput_shape(const TROIPooling *op, const TShape &feat_shape)
template<class TROIPooling, class TShape>
void rois_input_shape(const TROIPooling *op, const TShape rois_shape)
template<class TROIPooling>
void output_roi_attr(const TROIPooling *op)
template<class TROIPooling>
void scale_attr(const TROIPooling *op)
template<class TROIPooling>
void method_attr(const TROIPooling *op)
namespace shape_of

Functions

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Node *op, std::vector<TShape> input_shapes)
namespace ShapeInferRange

Functions

template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> range_shape_infer(const Node *op, const std::vector<T> &input_shapes, bool output_is_integral, bool step_allows_zero, const ITensorAccessor &tensor_accessor)
namespace slice

Typedefs

using Bounds = std::pair<int64_t, int64_t>

Alias to dimension bounds for slice.

Functions

inline int64_t get_sliced_value(const int64_t dim, const int64_t start, const int64_t stop, const int64_t step)

Get sliced value in step for given dimension value and start, stop, step.

Note

This function cannot be use for step 0 (division by 0)

Parameters
  • dimDimension value.

  • start – Start of slice.

  • stop – Stop of slice.

  • step – Step of slice.

Returns

-1 for infinite number otherwise [0..int64_max] for finit step.

constexpr bool is_bounds_zero_crossing(const Bounds b)

Check if bounds can cross 0 value (rising edge).

Parameters

bInput interval bounds for check.

Returns

True if lower bound is negative and upper is not negative, otherwise false.

template<class TDim>
constexpr bool is_lb_within_dim(const int64_t lb, const TDim &dim)

Check if lower bound is within dimension.

Check valid only if bounds can cross zero value (lb is negative).

Parameters
  • lb – Lower bound for check.

  • dimDimension used to check lower bound.

Returns

True if lower bound is within dimension length, otherwise false.

template<class TDim>
constexpr bool is_ub_within_dim(const int64_t ub, const TDim &dim)

Check if upper bound is within dimension.

Check valid only if bounds can cross zero value (up is not negative).

Parameters
  • ub – Upper bound for check.

  • dimDimension used to check upper bound.

Returns

True if upper bound is within dimension length, otherwise false.

template<class TDim>
TDim make_dim(const TDim &dim, const Bounds &start, const Bounds &stop, int64_t step)

Make sliced dimension for input dimension by step from start to stop bounds.

Template Parameters

TDim – Type of in/out dimension.

Parameters
  • dimInput Dimension to slice.

  • start – Slice start bounds.

  • stop – Slice stop bounds.

  • step – Slice step.

Returns

Dimension with upper/lower values set according slice inputs.

Variables

constexpr std::array<char const*, 4> shape_names = {"start", "stop", "step", "axes"}
struct AxesMap
#include <slice_shape_inference.hpp>

Public Members

bool is_valid = {}

Flag indicates current axes map has valid data (unique).

std::map<size_t, size_t> m = {}

Map axis value to index of start, stop order.

namespace util

Typedefs

using ActivationFunctionType = std::shared_ptr<Node> (*)(const std::shared_ptr<Node>&, float, float)
using InputDescriptionVector = std::vector<util::SubGraphOp::InputDescription::Ptr>
using OutputDescriptionVector = std::vector<util::SubGraphOp::OutputDescription::Ptr>
using VariableVector = std::vector<Variable::Ptr>
using VariableMap = std::unordered_map<Variable::Ptr, VariableValue::Ptr>

Enums

enum class LSTMWeightsFormat

Values:

enumerator FICO
enumerator ICOF
enumerator IFCO
enumerator IFOC
enumerator IOFC
enum class LSTMPeepholesFormat

Values:

enumerator FIO
enumerator IOF
enumerator IFO

Functions

template<class T>
bool normalize_single_value(std::vector<T> vec, float &value, bool check_value_range = true)
template<class T>
bool has_op_with_type(const std::shared_ptr<const ov::Model> &function)
inline bool has_decompression_converts(const std::shared_ptr<const ov::Model> &function)
inline std::string create_ie_output_name(const Output<const Node> &output)
inline std::string create_ie_output_name(const Output<Node> &output)
inline std::string get_ie_output_name(const Output<const Node> &output)
inline std::string get_ie_output_name(const Output<Node> &output)
float cast_eps_to_float(double eps_d)

Convert epsilon value from double to float type.

If the value is too large, the epsilon is converted to std::numeric_limits<float>::min() or std::numeric_limits<float>::min(), otherwise static cast to float is called. The adjustment is made for positive values only, for negative it works as static cast.

Parameters

eps – Original value of the epsilon (double).

Returns

Epsilon value as float.

template<typename T>
bool has_constant_value(const std::shared_ptr<Node> &node, const T value, T epsilon = std::numeric_limits<T>::epsilon())
template<typename T>
bool has_constant_value(const std::shared_ptr<Node> &node, const std::vector<T> values, T epsilon = std::numeric_limits<T>::epsilon())
bool get_single_value(const std::shared_ptr<opset4::Constant> &const_node, float &value, bool check_value_range = true)
std::shared_ptr<Node> normalize_constant(const std::shared_ptr<opset4::Constant> &constant, const PartialShape &shape)
std::shared_ptr<Node> broadcastTo(const Output<Node> &input, const Shape &shape)
std::shared_ptr<Node> reshapeTo(const Output<Node> &input, const Shape &shape)
bool constantIsEqualTo(const std::shared_ptr<opset4::Constant> &const_node, float value, float eps = 1e-5)
bool has_f16_constants(const std::shared_ptr<const ov::Model> &function)
bool check_for_broadcast(const PartialShape &ref_shape, const PartialShape &other_shape)
std::shared_ptr<Node> activation(const std::string &activation_name, const Output<Node> &apply_to)
bool is_seq_len_provided(const std::shared_ptr<Node> &X, const std::shared_ptr<Node> &seq_len_input)
std::shared_ptr<Node> try_fold_unary_output(const std::shared_ptr<Node> &node)
std::shared_ptr<Node> clone_try_fold(const std::shared_ptr<Node> &node, const OutputVector &inputs)
bool shapes_equal_except_dynamic_expected_batch(const PartialShape &expected, const PartialShape &actual)
void visit_shape_path(ov::Node *node, std::unordered_set<ov::Node*> &visited, std::function<void(ov::Node*)> func)

Traverses a shapeOf subgraph starting from the node and not including the ShapeOf nodes, and calls “func” for each ov::Node.

Parameters
  • node – The node from which constant path is started.

  • visited – Set of nodes which were visited.

  • func – The function which is called for each visited node.

void visit_constant_path(ov::Node *node, std::unordered_set<ov::Node*> &visited, std::function<void(ov::Node*)> func)

Traverses a constant path starting from “node”, and calls “func” for each ov::Node. If the function was called for non-constant subgraph, exception is thrown.

Parameters
  • node – The node from which constant path is started.

  • visited – Set of nodes which were visited.

  • func – The function which is called for each visited node.

template<typename T, typename ...Args>
std::shared_ptr<Node> make_try_fold(Args&&... args)
template<class T>
Output<Node> eltwise_fold(const Output<Node> &input0, const Output<Node> &input1)
std::vector<Input<Node>> get_node_target_inputs(const std::shared_ptr<Node> &node)
std::shared_ptr<Node> node_to_get_shape_value_of_indices_from_shape_node(const std::shared_ptr<Node> &shape_node, const std::vector<size_t> &indices, const std::vector<std::shared_ptr<Node>> &copy_rt_info_from = {})
std::shared_ptr<Node> node_to_get_shape_value_of_indices_from_shape_source(const Output<Node> &shape_source, const std::vector<size_t> &indices, const std::vector<std::shared_ptr<Node>> &copy_rt_info_from = {})
bool is_dequantization_subgraph(const Output<Node> &node)
bool can_eliminate_eltwise_node(const std::shared_ptr<Node> &eltwise, const Output<Node> &constant, const Output<Node> &non_constant_input)
bool is_constant_and_all_values_equal_int(const Output<Node> &output, const int64_t &v)
bool is_on_constant_path(const ov::Output<ov::Node> &output)
template<typename T>
ov::pass::pattern::op::ValuePredicate constant_predicate(std::function<bool(const std::vector<T>&)> predicate)
ActivationFunction get_activation_func_by_name(const std::string &func_name)

Gets the activation function by name.

Parameters

func_name[in] The function name

Throws

UnknownActivationFunction – When provided func_name is unknown.

Returns

The activation function object.

AxisSet get_normalized_axes_from_tensor(const Node *const node, const Tensor &tensor, const Rank &rank)

Get the normalized axes as ov::AxisSet from raw tensor data.

Parameters
  • node – A node pointer used for detailed description if normalization fails.

  • tensorTensor with axes for normalization.

  • rank – Rank value to normalize axes.

Returns

Normalized axes as set.

std::tuple<element::Type, PartialShape> validate_and_infer_elementwise_args(Node *node)
OPENVINO_API bool is_unary_elementwise_arithmetic (const Node *node)
OPENVINO_API bool is_binary_elementwise_arithmetic (const Node *node)
OPENVINO_API bool is_binary_elementwise_comparison (const Node *node)
OPENVINO_API bool is_binary_elementwise_logical (const Node *node)
OPENVINO_API bool supports_auto_broadcast (const Node *node)
OPENVINO_API bool is_op (const Node *node)
OPENVINO_API bool is_parameter (const Node *node)
OPENVINO_API bool is_output (const Node *node)
OPENVINO_API bool is_sink (const Node *node)
OPENVINO_API bool is_constant (const Node *node)
OPENVINO_API bool is_commutative (const Node *node)
OPENVINO_API bool is_unary_elementwise_arithmetic (const std::shared_ptr< Node > &node)
OPENVINO_API bool is_binary_elementwise_arithmetic (const std::shared_ptr< Node > &node)
OPENVINO_API bool is_binary_elementwise_comparison (const std::shared_ptr< Node > &node)
OPENVINO_API bool is_binary_elementwise_logical (const std::shared_ptr< Node > &node)
OPENVINO_API bool supports_auto_broadcast (const std::shared_ptr< Node > &node)
OPENVINO_API bool is_op (const std::shared_ptr< Node > &node)
OPENVINO_API bool is_parameter (const std::shared_ptr< Node > &node)
OPENVINO_API bool is_output (const std::shared_ptr< Node > &node)
OPENVINO_API bool is_sink (const std::shared_ptr< Node > &node)
OPENVINO_API bool is_constant (const std::shared_ptr< Node > &node)
OPENVINO_API bool is_commutative (const std::shared_ptr< Node > &node)
OPENVINO_API void validate_seq_input_rank_dimension (const std::vector< ov::PartialShape > &input)

Validates static rank and dimension for provided input parameters. Additionally input_size dimension is checked for X and W inputs. Applies to LSTM, GRU and RNN Sequences.

Parameters

input[in] Vector with RNNSequence-like op inputs in following order: X, initial_hidden_state, sequence_lengths, W, R and B.

std::shared_ptr< Node > OPENVINO_API convert_lstm_node_format (const Output< Node > &node, LSTMWeightsFormat from_format, LSTMWeightsFormat to_format=LSTMWeightsFormat::FICO, int64_t axis=0)

Change data format of provided node.

Parameters
  • node[in] The input node to be permuted.

  • from_format[in] Original node weights format.

  • to_format[in] Weights format to convert to.

Returns

Node representing reshaped tensor according to to_format weights format.

std::shared_ptr< Node > OPENVINO_API convert_lstm_peepholes_format (const Output< Node > &node, LSTMPeepholesFormat from_format, LSTMPeepholesFormat to_format=LSTMPeepholesFormat::FIO, int64_t axis=0)
template<typename T, class TRShape = result_shape_t<T>>
void validate_target_shape_none(const ov::Node *op, const T &arg_shape, const AxisVector &axes_mapping_val, const TRShape &target_input_shape)
template<typename T, class TRShape = result_shape_t<T>>
void validate_target_shape_numpy(const ov::Node *op, const T &arg_shape, const TRShape &target_input_shape)
template<typename T, class TRShape = result_shape_t<T>>
void set_result_shape_pdpd(const ov::Node *op, const T &arg0_shape, const TRShape &target_input_shape, TRShape &result_shape, const ov::op::BroadcastModeSpec &broadcast_spec)
template<typename T, class TRShape = result_shape_t<T>>
void set_result_shape_bidirectional(const ov::Node *op, const T &arg_shape, TRShape &target_input_shape, TRShape &result_shape)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> broadcast_base_shape_infer(const ov::op::util::BroadcastBase *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape>
size_t num_spatial_from_shapes(const TShape &data_shape, const TShape &filter_shape, const size_t filter_non_spatial_dims_count)

Get num of spatial form convolution operator.

Tries get value from operator member if is not deduced (has -1 value) then tries evaluate it from input shapes.

Template Parameters
  • TConv – Convolution type (this function must be a friend of TConv to access private member).

  • TShapeShape type.

Parameters
  • op – Pointer to convolution operator.

  • data_shapeInput data shape.

  • flter_shapeInput filter shape.

Returns

Value of spatial dimension number or infinite bound (-1) if cannot evaluate.

inline bool is_attr_validation_required(const ConvolutionBase *op)

Checks if validation attributes is required.

Parameters

op – Pointer to convolution base operator.

Returns

True if internal number of spatial dimension not defined otherwise false.

inline size_t get_num_spatial(const ConvolutionBase *op)

Get the num spatil object.

Parameters

op

Returns

size_t

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DeformableConvolutionBase *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)
template<typename T, typename V = typename std::iterator_traits<typename T::iterator>::value_type::value_type>
void compute_num_classes(const DetectionOutputBase *op, const DetectionOutputBase::AttributesBase &attrs, const std::vector<T> &input_shapes, V &num_classes, V &num_prior_boxes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer_base(const DetectionOutputBase *op, const DetectionOutputBase::AttributesBase &attrs, const std::vector<T> &input_shapes, int64_t attribute_num_classes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ov::op::util::EmbeddingBagOffsetsBase *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ov::op::util::EmbeddingBagPackedBase *op, const std::vector<TShape> &input_shapes)
template<class T>
void check_1D_or_scalar_shape(const ov::op::v9::Eye *op, const T &input_shape, const std::string &name)
template<class T>
result_shape_t<T> reduce_shape(const T &input_shape, std::vector<int64_t> &axes, const bool keep_dims)

Variables

constexpr size_t num_spatial_undefined = std::numeric_limits<size_t>::max()
constexpr size_t spatial_dim_offset = 2
class ActivationFunction
#include <activation_functions.hpp>

Class representing activation function used in RNN cells.

Public Functions

std::shared_ptr<Node> operator()(const std::shared_ptr<Node> &arg) const

Calls stored activation function with provided node argument.

class ArithmeticReduction : public ov::op::util::ReductionBase
#include <arithmetic_reduction.hpp>

Abstract base class for arithmetic reduction operations, i.e., operations where chosen axes of the input tensors are eliminated (reduced out) by repeated application of a particular binary arithmetic operation.

Subclassed by ov::op::util::ArithmeticReductionKeepDims

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ArithmeticReductionKeepDims : public ov::op::util::ArithmeticReduction
#include <arithmetic_reductions_keep_dims.hpp>

Subclassed by ov::op::v1::ReduceMax, ov::op::v1::ReduceMean, ov::op::v1::ReduceMin, ov::op::v1::ReduceProd, ov::op::v1::ReduceSum, ov::op::v4::ReduceL1, ov::op::v4::ReduceL2

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual bool get_keep_dims() const override
Returns

If set to 1 it holds axes that are used for reduction. For each such axis, output dimension is equal to 1.

class AssignBase : public ov::op::Sink, public ov::op::util::VariableExtension
#include <assign_base.hpp>

Subclassed by ov::op::v3::Assign, ov::op::v6::Assign

Public Functions

inline explicit AssignBase(const OutputVector &arguments)

Constructs an AssignBase operation.

class BinaryElementwiseArithmetic : public ov::op::Op
#include <binary_elementwise_arithmetic.hpp>

Abstract base class for elementwise binary arithmetic operations, i.e., operations where the same scalar binary arithmetic operation is applied to each corresponding pair of elements in the two input tensors. Implicit broadcast of input tensors is supported through one of the AutoBroadcast modes.

For example, if the underlying arithmetic operation (determined by the subclass) is \(\mathit{op}(x,y)\), the input tensors \([[x_0,y_0],[z_0,w_0]]\) and \([[x_1,y_1],[z_1,w_1]]\) will be mapped to \([[\mathit{op}(x_0,x_1),\mathit{op}(y_0,y_1)],[\mathit{op}(z_0,z_1),\mathit{op}(w_0,w_1)]]\).

Inputs

Type

Description

arg0

\(N[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape. The element type \(N\) may be any numeric type.

arg1

\(N[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of the same element type as arg0.

autob

AutoBroadcastSpec

Auto broadcast specification.

Type

Description

\(N[d_1,\dots,d_n]\)

The tensor \(T\), where \(T[i_1,\dots,i_n] = \mathit{op}(\texttt{arg0}[i_1,\dots,i_n],\texttt{arg1}[i_1,\dots,i_n])\). This will always have the same shape and element type as the input tensors (after auto broadcasting).

Subclassed by ov::op::v0::SquaredDifference, ov::op::v1::Add, ov::op::v1::Divide, ov::op::v1::FloorMod, ov::op::v1::Maximum, ov::op::v1::Minimum, ov::op::v1::Mod, ov::op::v1::Multiply, ov::op::v1::Power, ov::op::v1::Subtract

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual const AutoBroadcastSpec &get_autob() const override
Returns

the autobroadcasr spec

class BinaryElementwiseBitwise : public ov::op::Op
#include <binary_elementwise_bitwise.hpp>

Subclassed by ov::op::v13::BitwiseAnd, ov::op::v13::BitwiseOr, ov::op::v13::BitwiseXor

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual const AutoBroadcastSpec &get_autob() const override
Returns

the autobroadcasr spec

class BinaryElementwiseComparison : public ov::op::Op
#include <binary_elementwise_comparison.hpp>

Abstract base class for elementwise binary comparison operations, i.e., operations where the same scalar binary comparison operation is applied to each corresponding pair of elements in two input tensors. Implicit broadcast of input tensors is supported through one of the AutoBroadcast modes.

For example, if the underlying comparison operation (determined by the subclass) is \(\mathit{op}(x,y)\), the input tensors \([[x_0,y_0],[z_0,w_0]]\) and \([[x_1,y_1],[z_1,w_1]]\) will be mapped to \([[\mathit{op}(x_0,x_1),\mathit{op}(y_0,y_1)],[\mathit{op}(z_0,z_1),\mathit{op}(w_0,w_1)]]\).

Inputs

Type

Description

arg0

\(E[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape and element type.

arg1

\(E[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of the same shape and element type as arg0.

autob

AutoBroadcastSpec

Auto broadcast specification.

Type

Description

\(\texttt{bool}[d_1,\dots,d_n]\)

The tensor \(T\), where \(T[i_1,\dots,i_n] = \mathit{op}(\texttt{arg0}[i_1,\dots,i_n],\texttt{arg1}[i_1,\dots,i_n])\). This will always have the same shape as the input tensors, and the element type bool.

Subclassed by ov::op::v1::Equal, ov::op::v1::Greater, ov::op::v1::GreaterEqual, ov::op::v1::Less, ov::op::v1::LessEqual, ov::op::v1::NotEqual

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual const AutoBroadcastSpec &get_autob() const override
Returns

the autobroadcasr spec

class BinaryElementwiseLogical : public ov::op::Op
#include <binary_elementwise_logical.hpp>

Abstract base class for elementwise binary logical operations, i.e., operations where the same scalar binary logical operation is applied to each corresponding pair of elements in two boolean input tensors. Implicit broadcast of input tensors is supported through one of the AutoBroadcast modes.

For example, if the underlying operation (determined by the subclass) is \(\mathit{op}(x,y)\), the input tensors \([[x_0,y_0],[z_0,w_0]]\) and \([[x_1,y_1],[z_1,w_1]]\) will be mapped to \([[\mathit{op}(x_0,x_1),\mathit{op}(y_0,y_1)],[\mathit{op}(z_0,z_1),\mathit{op}(w_0,w_1)]]\).

Inputs

Type

Description

arg0

\(\texttt{bool}[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape, with element type bool.

arg1

\(\texttt{bool}[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of the same shape and element type as arg0.

autob

AutoBroadcastSpec

Auto broadcast specification.

Type

Description

\(\texttt{bool}[d_1,\dots,d_n]\)

The tensor \(T\), where \(T[i_1,\dots,i_n] = \mathit{op}(\texttt{arg0}[i_1,\dots,i_n],\texttt{arg1}[i_1,\dots,i_n])\). This will always have the same shape as the input tensors, and the element type bool.

Subclassed by ov::op::v0::Xor, ov::op::v1::LogicalAnd, ov::op::v1::LogicalOr, ov::op::v1::LogicalXor

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual const AutoBroadcastSpec &get_autob() const override
Returns

the autobroadcasr spec

class BroadcastBase : public ov::op::Op
#include <broadcast_base.hpp>

Subclassed by ov::op::v1::Broadcast, ov::op::v3::Broadcast

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual std::pair<bool, AxisSet> get_broadcast_axes() const
Returns

true and the AxisSet if broadcast axes can be fully determined.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

struct ClipNegative
#include <reverse_shape_inference.hpp>

Clip if value type T is less than 0, otherwise cast to AxisSet::value_type.

class ConvertColorI420Base : public ov::op::Op
#include <convert_color_i420_base.hpp>

Base class for color conversion operation from I420 to RGB/BGR format. Input:

  • Operation expects input shape in NHWC layout.

  • Input NV12 image can be represented in a two ways: a) Single plane: NV12 height dimension is 1.5x bigger than image height. ‘C’ dimension shall be 1 b) Three separate planes: Y, U and V. In this case b1) Y plane has height same as image height. ‘C’ dimension equals to 1 b2) U plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 1. b3) V plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 1.

  • Supported element types: u8 or any supported floating-point type. Output:

  • Output node will have NHWC layout and shape HxW same as image spatial dimensions.

  • Number of output channels ‘C’ will be 3

    Conversion of each pixel from I420 (YUV) to RGB space is represented by following formulas: R = 1.164 * (Y - 16) + 1.596 * (V - 128) G = 1.164 * (Y - 16) - 0.813 * (V - 128) - 0.391 * (U - 128) B = 1.164 * (Y - 16) + 2.018 * (U - 128) Then R, G, B values are clipped to range (0, 255)

Subclassed by ov::op::v8::I420toBGR, ov::op::v8::I420toRGB

Public Types

enum class ColorConversion : int

Exact conversion format details Currently supports conversion from I420 to RGB or BGR.

Values:

enumerator I420_TO_RGB
enumerator I420_TO_BGR

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ConvertColorNV12Base : public ov::op::Op
#include <convert_color_nv12_base.hpp>

Base class for color conversion operation from NV12 to RGB/BGR format. Input:

  • Operation expects input shape in NHWC layout.

  • Input NV12 image can be represented in a two ways: a) Single plane: NV12 height dimension is 1.5x bigger than image height. ‘C’ dimension shall be 1 b) Two separate planes: Y and UV. In this case b1) Y plane has height same as image height. ‘C’ dimension equals to 1 b2) UV plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 2.

  • Supported element types: u8 or any supported floating-point type. Output:

  • Output node will have NHWC layout and shape HxW same as image spatial dimensions.

  • Number of output channels ‘C’ will be 3

    Conversion of each pixel from NV12 (YUV) to RGB space is represented by following formulas: R = 1.164 * (Y - 16) + 1.596 * (V - 128) G = 1.164 * (Y - 16) - 0.813 * (V - 128) - 0.391 * (U - 128) B = 1.164 * (Y - 16) + 2.018 * (U - 128) Then R, G, B values are clipped to range (0, 255)

Subclassed by ov::op::v8::NV12toBGR, ov::op::v8::NV12toRGB

Public Types

enum class ColorConversion : int

Exact conversion format details Currently supports conversion from NV12 to RGB or BGR, in future can be extended with NV21_to_RGBA/BGRA, etc.

Values:

enumerator NV12_TO_RGB
enumerator NV12_TO_BGR

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ConvolutionBackPropBase : public ov::op::util::ConvolutionBase
#include <convolution_backprop_base.hpp>

Base class for operations like back propagation convolution.

Subclassed by ov::op::v1::ConvolutionBackpropData, ov::op::v1::GroupConvolutionBackpropData

Public Functions

ConvolutionBackPropBase() = default

Constructs a conversion operation.

inline ConvolutionBackPropBase(const OutputVector &arguments, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT, const CoordinateDiff &output_padding = {})

Constructs a conversion operation.

Parameters
  • strides – Convolution strides.

  • pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.

  • pads_end – Amount of padding to be added to the end along each axis.

  • dilations – The distance in width and height between the weights in the filters tensor.

  • auto_pad – Specifies how the automatic calculation of padding should be done.

  • output_padding – The output padding adds additional amount of paddings per each spatial axis in the output tensor. clang-format on

class ConvolutionBase : public ov::op::Op
#include <convolution_base.hpp>

Base class for operations like convolutions.

Subclassed by ov::op::util::ConvolutionBackPropBase, ov::op::util::ConvolutionFwdPropBase, ov::op::util::DeformableConvolutionBase

Public Functions

ConvolutionBase() = default

Constructs a conversion operation.

inline ConvolutionBase(const OutputVector &arguments, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT)

Constructs a conversion operation.

Parameters
  • strides – Convolution strides.

  • pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.

  • pads_end – Amount of padding to be added to the end along each axis.

  • dilations – The distance in width and height between the weights in the filters tensor.

  • auto_pad – Specifies how the automatic calculation of padding should be done.

class ConvolutionFwdPropBase : public ov::op::util::ConvolutionBase
#include <convolution_base.hpp>

Base class for operations like back propagation convolution.

Subclassed by ov::op::v1::BinaryConvolution, ov::op::v1::Convolution, ov::op::v1::GroupConvolution

Public Functions

ConvolutionFwdPropBase() = default

Constructs a conversion operation.

inline ConvolutionFwdPropBase(const OutputVector &arguments, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT)

Constructs a conversion operation.

Parameters
  • strides – Convolution strides.

  • pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.

  • pads_end – Amount of padding to be added to the end along each axis.

  • dilations – The distance in width and height between the weights in the filters tensor.

  • auto_pad – Specifies how the automatic calculation of padding should be done.

class DeformableConvolutionBase : public ov::op::util::ConvolutionBase
#include <deformable_convolution_base.hpp>

Base class for operations DeformableConvolution v1 and DeformableConvolution v8.

Subclassed by ov::op::v1::DeformableConvolution, ov::op::v8::DeformableConvolution

Public Functions

DeformableConvolutionBase() = default

Constructs a conversion operation.

DeformableConvolutionBase(const OutputVector &arguments, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT, int64_t group = 1, int64_t deformable_group = 1)

Constructs a conversion operation.

Parameters
  • strides – Convolution strides.

  • pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.

  • pads_end – Amount of padding to be added to the end along each axis.

  • dilations – The distance in width and height between the weights in the filters tensor.

  • auto_pad – Specifies how the automatic calculation of padding should be done.

  • group – The number of groups which both output and input should be split into.

  • deformable_group – The number of groups which deformable values and output should be split into along the channel axis.

class DetectionOutputBase : public ov::op::Op
#include <detection_output_base.hpp>

DetectionOutputBase basic class for DetectionOutput v0 and v8.

Subclassed by ov::op::v0::DetectionOutput, ov::op::v8::DetectionOutput

struct AttributesBase
#include <detection_output_base.hpp>

Subclassed by ov::op::v0::DetectionOutput::Attributes

class EmbeddingBagOffsetsBase : public ov::op::Op
#include <embeddingbag_offsets_base.hpp>

Returns embeddings for given indices.

Subclassed by ov::op::v3::EmbeddingBagOffsetsSum

Public Functions

EmbeddingBagOffsetsBase() = default

Constructs a EmbeddingBagOffsetsBase operation.

EmbeddingBagOffsetsBase(const Output<Node> &emb_table, const Output<Node> &indices, const Output<Node> &offsets, const Output<Node> &default_index, const Output<Node> &per_sample_weights)

Constructs a EmbeddingBagOffsetsBase operation.

EmbeddingBagOffsetsBase constructs an output tensor by replacing every index in a given input tensor with a row (from the weights matrix) at that index

Parameters
  • emb_table – tensor containing the embedding lookup table of the module of shape [num_emb, emb_dim1, emb_dim2, …] and of type T

  • indices – tensor of shape [num_indices] and of type T_IND. Required

  • offsets – tensor of shape [batch] and of type T_IND containing the starting index positions of each “bag” in indices. Required.

  • per_sample_weigths – tensor of the same shape as indices and of type T. Each value in this tensor are multiplied with each value pooled from embedding table for each index. Optional.

  • default_index – scalar of type T_IND containing default index in embedding table to fill empty “bags”. If not provided empty “bags” are filled with zeros. Optional.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class EmbeddingBagPackedBase : public ov::op::Op
#include <embeddingbag_packed_base.hpp>

Returns embeddings for given indices.

Subclassed by ov::op::v3::EmbeddingBagPackedSum

Public Functions

EmbeddingBagPackedBase() = default

Constructs a EmbeddingBagPackedBase operation.

EmbeddingBagPackedBase(const Output<Node> &emb_table, const Output<Node> &indices, const Output<Node> &per_sample_weights)

Constructs a EmbeddingBagPackedBase operation.

EmbeddingBagPackedBase constructs an output tensor by replacing every index in a given input tensor with a row (from the weights matrix) at that index

Parameters
  • emb_tableTensor containing the embedding lookup table of the module of shape [num_emb, emb_dim1, emb_dim2, …] and of type T

  • indicesTensor of shape [batch, indices_per_bag] and of type T_IND. Required.

  • per_sample_weigths – tensor of the same shape as indices and of type T. Each value in this tensor are multiplied with each value pooled from embedding table for each index. Optional.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class FFTBase : public ov::op::Op
#include <fft_base.hpp>

Base class for operations DFT and DFT.

Subclassed by ov::op::v7::DFT, ov::op::v7::IDFT, ov::op::v9::IRDFT, ov::op::v9::RDFT

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class FrameworkNode : public ov::op::util::MultiSubGraphOp
#include <framework_node.hpp>

Subclassed by ov::frontend::ComplexTypeMark

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class FrameworkNodeAttrs
#include <framework_node.hpp>
class GatherBase : public ov::op::Op
#include <gather_base.hpp>

GatherBase basic class for Gather v1 and v7.

Subclassed by ov::op::v1::Gather, ov::op::v7::Gather, ov::op::v8::Gather

Public Functions

GatherBase(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &axis, const int64_t batch_dims = 0)
Parameters
  • data – The tensor from which slices are gathered

  • indicesTensor with indexes to gather

  • axis – The tensor is a dimension index to gather data from

  • batch_dims – The number of batch dimension in data and indices tensors

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class GatherNDBase : public ov::op::Op
#include <gather_nd_base.hpp>

GatherNDBase basic class for GatherND v5 and v8.

Subclassed by ov::op::v5::GatherND, ov::op::v8::GatherND

Public Functions

GatherNDBase(const Output<Node> &data, const Output<Node> &indices, const size_t batch_dims = 0)

Constructs a GatherND operation.

Parameters
  • dataNode producing data that are gathered

  • indicesNode producing indices by which the operation gathers elements or slices from data

  • batch_dims – Specifies a leading number of dimensions representing the batches

template<class T>
struct GetK
#include <topk_shape_inference.hpp>
template<class T>
struct GetNotNegative
#include <one_hot_shape_inference.hpp>
class IndexReduction : public ov::op::Op
#include <index_reduction.hpp>

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class InterpolateBase : public ov::op::Op
#include <interpolate_base.hpp>

Subclassed by ov::op::v11::Interpolate, ov::op::v4::Interpolate

Public Types

enum class ShapeCalcMode

PartialShape calculation mode.

SIZES - output shape for interpolated axes is calculated using input sizes SCALES - output shape for interpolated axes is calculated using input scales

Values:

enumerator SIZES
enumerator SCALES
enum class InterpolateMode

Interpolation mode.

NEAREST - nearest interpolation LINEAR - linear interpolation as in TensorFlow LINEAR_ONNX - linear interpolation as in ONNX CUBIC - cubic interpolation BILINEAR_PILLOW - bilinear interpolation as in Pillow BICUBIC_PILLOW - bicubic interpolation as in Pillow

Values:

enumerator NEAREST
enumerator LINEAR
enumerator LINEAR_ONNX
enumerator CUBIC
enumerator BILINEAR_PILLOW
enumerator BICUBIC_PILLOW
enum class CoordinateTransformMode

Mode of the calculation of the source coordinate from resized one.

These modes are modes from ONNXRuntime.

Values:

enumerator HALF_PIXEL
enumerator PYTORCH_HALF_PIXEL
enumerator ASYMMETRIC
enumerator TF_HALF_PIXEL_FOR_NN
enumerator ALIGN_CORNERS
enum class NearestMode

Rounding modes for the NEAREST interpolation.

Values:

enumerator ROUND_PREFER_FLOOR
enumerator ROUND_PREFER_CEIL
enumerator FLOOR
enumerator CEIL
enumerator SIMPLE

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

struct InterpolateAttrs
#include <interpolate_base.hpp>
class LogicalReduction : public ov::op::util::ReductionBase
#include <logical_reduction.hpp>

Abstract base class for logical reduction operations, i.e., operations where chosen axes of the input tensors are eliminated (reduced out) by repeated application of a particular binary logical operation.

Subclassed by ov::op::util::LogicalReductionKeepDims

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class LogicalReductionKeepDims : public ov::op::util::LogicalReduction
#include <logical_reduction_keep_dims.hpp>

Subclassed by ov::op::v1::ReduceLogicalAnd, ov::op::v1::ReduceLogicalOr

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual bool get_keep_dims() const override
Returns

If set to 1 it holds axes that are used for reduction. For each such axis, output dimension is equal to 1.

class MaxPoolBase : public ov::op::Op
#include <max_pool_base.hpp>

Subclassed by ov::op::v1::MaxPool, ov::op::v8::MaxPool

Public Functions

MaxPoolBase(const Output<Node> &arg, const Strides &strides, const Shape &pads_begin, const Shape &pads_end, const Shape &kernel, const op::RoundingType rounding_mode = op::RoundingType::FLOOR, const PadType auto_pad = op::PadType::EXPLICIT)
Parameters
  • arg – The node producing the input data batch tensor.

  • strides – The strides.

  • pads_begin – The beginning of padding shape.

  • pads_end – The end of padding shape.

  • kernel – The kernel shape.

  • rounding_mode – Whether to use ceiling or floor rounding type while computing output shape.

  • auto_pad – The pad type for automatically computing padding sizes.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline const Shape &get_kernel() const
Returns

The kernel shape.

inline const Strides &get_strides() const
Returns

The strides.

inline const Shape &get_pads_begin() const
Returns

The beginning of padding shape.

inline const Shape &get_pads_end() const
Returns

The end of padding shape.

inline PadType get_auto_pad() const
Returns

The pad type for pooling.

inline op::RoundingType get_rounding_type() const
Returns

The ceiling mode being used for output shape computations

class MulticlassNmsBase : public ov::op::Op
#include <multiclass_nms_base.hpp>

Base class for operations MulticlassNMS v8 and MulticlassNMS v9.

Subclassed by ov::op::v8::MulticlassNms, ov::op::v9::MulticlassNms

Public Functions

MulticlassNmsBase() = default

Constructs a conversion operation.

MulticlassNmsBase(const OutputVector &arguments, const Attributes &attrs)

Constructs a MulticlassNmsBase operation.

Parameters
  • argumentsNode list producing the box coordinates, scores, etc.

  • attrsAttributes of the operation

inline const Attributes &get_attrs() const

Returns attributes of the operation MulticlassNmsBase.

struct Attributes
#include <multiclass_nms_base.hpp>

Structure that specifies attributes of the operation.

class MultiSubGraphOp : public ov::op::Op
#include <multi_subgraph_base.hpp>

Abstract base class for sub-graph based ops, i.e ops that have some sub-graphs.

Subclassed by ov::op::util::FrameworkNode, ov::op::util::SubGraphOp, ov::op::v8::If

Public Functions

inline virtual const std::shared_ptr<Model> &get_function(size_t index) const

Gets internal sub-graph by index in MultiSubGraphOp.

Parameters

index – sub-graph’s index in op

Returns

pointer to Model with sub-graph

inline virtual const std::vector<std::shared_ptr<Model>> &get_functions() const

Gets internal sub-graphs.

Returns

a vector of pointers to sub-graph Models

inline virtual void set_function(int index, const std::shared_ptr<Model> &func)

Adds sub-graph to MultiSubGraphOp.

Parameters
  • index – index of new sub-graph

  • func – func new sub_graph as Model

inline const MultiSubgraphInputDescriptionVector &get_input_descriptions(int index) const

Gets vector with connections between operation inputs and internal sub-graph parameters.

Parameters

index – index of internal sub-graph

Returns

vector of input descriptions

inline MultiSubgraphInputDescriptionVector &get_input_descriptions(int index)

Gets vector with connections between operation inputs and internal sub-graph parameters.

Parameters

index – index of internal sub-graph

Returns

vector of input descriptions

inline const MultiSubgraphOutputDescriptionVector &get_output_descriptions(int index) const

Gets vector with connections between operation outputs and internal sub-graph results.

Parameters

index – index of internal sub-graph

Returns

vector of output descriptions

inline MultiSubgraphOutputDescriptionVector &get_output_descriptions(int index)

Gets vector with connections between operation outputs and internal sub-graph results.

Parameters

index – index of internal sub-graph

Returns

vector of output descriptions

inline void set_input_descriptions(int index, const MultiSubgraphInputDescriptionVector &inputs)

Sets vector with connections between operation inputs and internal sub-graph parameters.

Parameters
  • index – index of internal sub-graph

  • inputs – vector of input descriptions

inline void set_output_descriptions(int index, const MultiSubgraphOutputDescriptionVector &outputs)

Sets vector with connections between operation outputs and internal sub-graph results.

Parameters
  • index – index of internal sub-graph

  • outputs – vector of input descriptions

virtual void set_invariant_inputs(const Output<Node> &value, const ov::ParameterVector &bodies_parameters)

Set input decriptions for MultiSubGraphOp input.

Parameters
  • value – The value supplied as an input to the block.

  • bodies_parameters – vector of bodies parameters.

virtual Output<Node> set_body_outputs(const ResultVector &bodies_results)

Set output decriptions for MultiSubGraphOp output.

Parameters

bodies_results – vector of bodies results for one output.

Returns

value Output node for bodies_results.

inline virtual size_t get_internal_subgraphs_size() const

Get number of internal sub-graphs.

Returns

Number of sub-graphs.

inline virtual size_t get_input_descriptions_size() const

Get number of input descriptions.

Returns

Number of input descriptions

inline virtual size_t get_output_descriptions_size() const

Get number of output descriptions.

Returns

Number of output descriptions

class BodyOutputDescription : public ov::op::util::MultiSubGraphOp::OutputDescription
#include <multi_subgraph_base.hpp>

Produces an output from a specific iteration.

Public Functions

BodyOutputDescription(uint64_t body_value_index, uint64_t output_index, int64_t iteration = -1)

Constructs a new instance.

Parameters
  • body_value_index – A body value that produces the output

  • output_index – The SubGraphOp output index

  • iteration – which iteration (typically -1, final) will supply the value

class ConcatOutputDescription : public ov::op::util::MultiSubGraphOp::OutputDescription
#include <multi_subgraph_base.hpp>

Produces an output by concatenating an output from each iteration.

Public Functions

ConcatOutputDescription(uint64_t body_value_index, uint64_t output_index, int64_t start, int64_t stride, int64_t part_size, int64_t end, int64_t axis)

Constructs a new instance.

Parameters
  • body_value_index – A body value that produces the output

  • output_index – The MultiSubGraphOp output index

  • start – First index for slices

  • stride – Step amount for slices

  • part_size – Width of slices

  • end – Last index for slices

  • axis – Axis being sliced

class InputDescription
#include <multi_subgraph_base.hpp>

Abstract class describes a connection between a MultiSubGraphOp input and the body.

Subclassed by ov::op::util::MultiSubGraphOp::InvariantInputDescription, ov::op::util::MultiSubGraphOp::MergedInputDescription, ov::op::util::MultiSubGraphOp::SliceInputDescription

class InvariantInputDescription : public ov::op::util::MultiSubGraphOp::InputDescription
#include <multi_subgraph_base.hpp>

Produces an input.

Public Functions

InvariantInputDescription(uint64_t input_index, uint64_t body_parameter_index)

Constructs a new instance.

Parameters
  • input_index – Position of the MultiSubGraphOp input

  • body_parameter_index – Body parameter to receive input

class MergedInputDescription : public ov::op::util::MultiSubGraphOp::InputDescription
#include <multi_subgraph_base.hpp>

Describes a body input initialized from a MultiSubGraphOp input on the first iteration, and then a body output thereafter.

Public Functions

MergedInputDescription(uint64_t input_index, uint64_t body_parameter_index, uint64_t body_value_index)

Constructs a new instance.

Parameters
  • input_index – Position of the MultiSubGraphOp input supplying a value to body_parameter for the initial iteration.

  • body_parameter_index – Body parameter position to receive input.

  • body_value_index – Body value to supply body_parameter for successive iterations.

class OutputDescription
#include <multi_subgraph_base.hpp>

Abstract class describes how a MultiSubGraphOp output is produced from the body.

Subclassed by ov::op::util::MultiSubGraphOp::BodyOutputDescription, ov::op::util::MultiSubGraphOp::ConcatOutputDescription

class SliceInputDescription : public ov::op::util::MultiSubGraphOp::InputDescription
#include <multi_subgraph_base.hpp>

Describes a body input formed from slices of an input to MultiSubGraphOp.

Public Functions

SliceInputDescription(uint64_t input_index, uint64_t body_parameter_index, int64_t start, int64_t stride, int64_t part_size, int64_t end, int64_t axis)

Constructs a new instance.

Parameters
  • input_index – Position of the MultiSubGraphOp input

  • body_parameter_index – Body parameter position to receive input

  • start – First index for slices

  • stride – Step amount for slices

  • part_size – Width of slices

  • end – Last index for slices

  • axis – Axis being sliced

class PadBase : public ov::op::Op
#include <pad_base.hpp>

Subclassed by ov::op::v12::Pad, ov::op::v1::Pad

Public Functions

PadBase(const Output<Node> &arg, const Output<Node> &pads_begin, const Output<Node> &pads_end, const Output<Node> &arg_pad_value, PadMode pad_mode)

Constructs a generic padding operation.

Parameters
  • arg – The output producing input tensor to be padded.

  • pads_begin – The output which specifies the number of padding elements added before position 0 on each axis of arg.

  • pads_end – The output which specifies the number of padding elements after the last element on each axis.

  • arg_pad_value – The scalar output with the value used for padding if pad_mode is CONSTANT

  • pad_mode – The padding mode

PadBase(const Output<Node> &arg, const Output<Node> &pads_begin, const Output<Node> &pads_end, PadMode pad_mode)

Constructs a generic padding operation.

Parameters
  • arg – The output producing input tensor to be padded.

  • pads_begin – The output which specifies the number of padding elements added

  • pads_end – The output which specifies the number of padding elements after the last element on each axis.

  • pad_mode – The padding mode

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

CoordinateDiff get_pads_begin() const

return The node which specifies the number of padding elements added at the beginning of each axis

CoordinateDiff get_pads_end() const

return The node which specifies the number of padding elements added at the end of each axis

inline PadMode get_pad_mode() const
Returns

The padding mode.

class ReadValueBase : public ov::op::Op, public ov::op::util::VariableExtension
#include <read_value_base.hpp>

Subclassed by ov::op::v3::ReadValue, ov::op::v6::ReadValue

Public Functions

inline explicit ReadValueBase(const OutputVector &arguments)

Constructs an AssignBase operation.

class ReductionBase : public ov::op::Op
#include <reduction_base.hpp>

Subclassed by ov::op::util::ArithmeticReduction, ov::op::util::LogicalReduction

Public Functions

bool reduction_axes_constant() const
Returns

true if reduction axes are constant else false.

const AxisSet get_reduction_axes() const
Throws

CheckFailure – if the reduction axes are not constant. (Use reduction_axes_constant to check.)

Returns

The axis positions (0-based) to be eliminated through reduction.

void set_reduction_axes(const AxisSet &reduction_axes)

Change the reduction axes.

class RNNCellBase : public ov::op::Op
#include <rnn_cell_base.hpp>

Base class for all recurrent network cells.

Note

It holds all common attributes.

Subclassed by ov::op::internal::AUGRUCell, ov::op::internal::AUGRUSequence, ov::op::v0::LSTMCell, ov::op::v0::LSTMSequence, ov::op::v0::RNNCell, ov::op::v3::GRUCell, ov::op::v4::LSTMCell, ov::op::v5::GRUSequence, ov::op::v5::LSTMSequence, ov::op::v5::RNNSequence

Public Functions

RNNCellBase(const OutputVector &args, std::size_t hidden_size, float clip, const std::vector<std::string> &activations, const std::vector<float> &activations_alpha, const std::vector<float> &activations_beta)

Constructs a RNNCellBase class.

Parameters
  • hidden_size[in] The number of hidden units for recurrent cell.

  • clip[in] The value defining clipping range [-clip, clip] on input of activation functions.

  • activations[in] The vector of activation functions used inside recurrent cell.

  • activations_alpha[in] The vector of alpha parameters for activation functions in order respective to activation list.

  • activations_beta[in] The vector of beta parameters for activation functions in order respective to activation list.

void validate_input_rank_dimension(const std::vector<PartialShape> &input)

Validates static rank and dimension for provided input parameters. Additionally input_size dimension is checked for X and W inputs.

Parameters

input[in] Vector with RNN-Cell op inputs in following order: X, initial_hidden_state, W, R and B.

class ScatterBase : public ov::op::Op
#include <scatter_base.hpp>

Base class for ScatterXXX operators.

Subclassed by ov::op::v3::ScatterUpdate

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ScatterElementsUpdateBase : public ov::op::Op
#include <scatter_elements_update_base.hpp>

Subclassed by ov::op::v12::ScatterElementsUpdate, ov::op::v3::ScatterElementsUpdate

Public Functions

ScatterElementsUpdateBase(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &updates, const Output<Node> &axis)

The common base class for all ScatterElementsUpdate operator versions.

Parameters
  • dataInput data

  • indices – Data entry index that will be updated

  • updates – Update values

  • axis – Axis to scatter on

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ScatterNDBase : public ov::op::Op
#include <scatter_nd_base.hpp>

Base class for ScatterNDXXX operators.

Subclassed by ov::op::v3::ScatterNDUpdate

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ShapeOfBase : public ov::op::Op
#include <shape_of_base.hpp>

Subclassed by ov::op::v0::ShapeOf, ov::op::v3::ShapeOf

Public Functions

inline explicit ShapeOfBase(const OutputVector &arguments)

Constructs an ShapeOfBase operation.

class SubGraphOp : public ov::op::util::MultiSubGraphOp
#include <sub_graph_base.hpp>

Abstract base class for sub-graph based ops, i.e ops that have only one sub-graph.

Subclassed by ov::op::v0::TensorIterator, ov::op::v5::Loop

Public Functions

inline const std::vector<std::shared_ptr<InputDescription>> &get_input_descriptions() const
Returns

a reference to the input descriptions.

inline std::vector<std::shared_ptr<InputDescription>> &get_input_descriptions()
Returns

a reference to the input descriptions. Can add input descriptions before validation.

inline const std::vector<std::shared_ptr<OutputDescription>> &get_output_descriptions() const
Returns

a reference to the output descriptions.

inline std::vector<std::shared_ptr<OutputDescription>> &get_output_descriptions()
Returns

a reference to the output descriptions. Can add output descriptions before validation.

virtual void set_sliced_input(const std::shared_ptr<ov::op::v0::Parameter> &parameter, const Output<Node> &value, int64_t start, int64_t stride, int64_t part_size, int64_t end, int64_t axis)

Indicate that a body parameter comes from slices of a value.

Parameters
  • parameter – The parameter to receive the slices

  • value – The value to be sliced. This will be added as an input to SubGraphOp.

  • start – First index on axis of the slicing

  • stride – Stepping of the slice

  • part_size – Size of the slice on axis

  • end – The last index on axis of the slicing

  • axis – The axis to slice along

virtual void set_merged_input(const std::shared_ptr<ov::op::v0::Parameter> &body_parameter, const Output<Node> &initial_value, const Output<Node> &successive_value)

Indicates that a body parameter has an initial value in the first iteration and computed value thereafter.

Parameters
  • body_parameter[in] The body parameter

  • initial_value – Value for the parameter in first iteration. This will be added as an input to Loop.

  • successive_value – Value for the parameter in successive iterations. The value is what is active in the most recent completed iteration.

virtual void set_invariant_input(const std::shared_ptr<ov::op::v0::Parameter> &body_parameter, const Output<Node> &value)

Indicates that a body parameter has an invariant value during iteration that may depend on values computed outside of the iteration.

Parameters
  • body_parameter – The body parameter

  • value – The value supplied as an input to the block

virtual Output<Node> get_iter_value(const Output<Node> &body_value, int64_t iteration = -1)

Gets a value for a particular iteration point.

Parameters
  • body_value – The value

  • iteration – The iteration that supplies the value. Negative values are from the last iteration. Default value -1 (the last iteration).

Returns

The iterator value.

virtual Output<Node> get_concatenated_slices(const Output<Node> &value, int64_t start, int64_t stride, int64_t part_size, int64_t end, int64_t axis)

Concatenates slices from all iterations.

Parameters
  • value – The value supplying slice values from each iteration.

  • start – First index on axis of the slicing

  • stride – Stepping of the slice

  • part_size – Size of the slice on axis

  • end – The last index on axis of the slicing

  • axis – The axis to slice along

Returns

The concatenated slices.

class TopKBase : public ov::op::Op
#include <topk_base.hpp>

Subclassed by ov::op::v11::TopK, ov::op::v1::TopK, ov::op::v3::TopK

Public Functions

TopKBase(const Output<Node> &data, const Output<Node> &k, const int64_t axis, const std::string &mode, const std::string &sort, const element::Type &index_element_type = element::i32)

The common base class for all TopK operator versions.

Parameters
  • data – The input tensor

  • k – Specifies how many maximum/minimum elements should be computed

  • axis – The axis along which TopK should be computed

  • mode – Specifies whether the maximum or minimum elements are selected

  • sort – Specifies the order of output elements and/or indices Accepted values: none, index, value

  • index_element_type – Specifies the type of produced indices

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

uint64_t get_axis() const

Returns axis value after normalization.

Note

If input rank required to normalization is dynamic, the exception is thrown

inline int64_t get_provided_axis() const

Returns axis value before normalization.

size_t get_k() const

Returns the value of K, if available.

Note

If the second input to this op is a constant, the value is retrieved and returned. If the input is not constant(dynamic) this method returns 0

inline virtual size_t get_default_output_index() const override

Returns the output of the default output, or throws if there is none.

class UnaryElementwiseArithmetic : public ov::op::Op
#include <unary_elementwise_arithmetic.hpp>

Abstract base class for elementwise unary arithmetic operations, i.e., operations where the same scalar arithmetic operation is applied to each element.

For example, if the underlying operation (determined by the subclass) is \(\mathit{op}(x)\), the input tensor \([[x,y],[z,w]]\) will be mapped to \([[\mathit{op}(x),\mathit{op}(y)],[\mathit{op}(z),\mathit{op}(w)]]\).

Inputs

Type

Description

arg

\(N[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape. The element type \(N\) may be any numeric type.

Type

Description

\(N[d_1,\dots,d_n]\)

The tensor \(T\), where \(T[i_1,\dots,i_n] = \mathit{op}(\texttt{arg}[i_1,\dots,i_n])\). This will always have the same shape and element type as the input tensor.

Subclassed by ov::op::v0::Abs, ov::op::v0::Acos, ov::op::v0::Asin, ov::op::v0::Atan, ov::op::v0::Ceiling, ov::op::v0::Clamp, ov::op::v0::Cos, ov::op::v0::Cosh, ov::op::v0::Elu, ov::op::v0::Erf, ov::op::v0::Exp, ov::op::v0::Floor, ov::op::v0::GRN, ov::op::v0::Gelu, ov::op::v0::Log, ov::op::v0::Negative, ov::op::v0::Relu, ov::op::v0::Sigmoid, ov::op::v0::Sign, ov::op::v0::Sin, ov::op::v0::Sinh, ov::op::v0::Sqrt, ov::op::v0::Tan, ov::op::v0::Tanh, ov::op::v3::Acosh, ov::op::v3::Asinh, ov::op::v3::Atanh, ov::op::v4::HSwish, ov::op::v4::Mish, ov::op::v4::SoftPlus, ov::op::v5::HSigmoid, ov::op::v5::Round, ov::op::v7::Gelu, ov::op::v9::SoftSign

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Variable
#include <variable.hpp>
class VariableContext
#include <variable_context.hpp>

VariableContext stores and manages a evaluation context for Variables.

Public Functions

VariableContext() = default

Constructs an uninitialized VariableContext.

inline explicit VariableContext(const VariableMap &variable_values)

Constructor for VariableContext.

Parameters

variable_values – The values associated with a particular Variables.

inline void reset_variable_context() const

Sets the reset flags for all stored Variables to true.

inline void set_variable_values(const VariableMap &variable_values)

Sets the new values for Variables.

Parameters

variable_values – The new values associated with a particular Variable.

inline void set_variable_value(const Variable::Ptr &variable, const VariableValue::Ptr &variable_value)

Changes/sets the values for Variable.

Parameters
  • variable – New or stored Variable.

  • variable_value – The values associated with the variable.

inline void remove_variable_value(const Variable::Ptr &variable)

Removes context for a particular Variable.

Parameters

variable – The variable for which the context will be cleared.

inline const VariableMap &get_variable_values() const

Returns the current values for Variables.

inline VariableValue::Ptr get_variable_value(const Variable::Ptr &variable) const

Returns the value for specified Variable.

class VariableExtension
#include <variable_extension.hpp>

Subclassed by ov::op::util::AssignBase, ov::op::util::ReadValueBase

Public Functions

inline virtual std::shared_ptr<Variable> get_variable() const

Returns variable connected to this node.

inline virtual void set_variable(const std::shared_ptr<Variable> &variable)

Sets a new variable to be connected to this node.

Parameters

variable – New variable to be connected to this node.

inline virtual void set_variable_id(const std::string &variable_id)

Sets the identifier to a variable.

Parameters

variable_id – New identifier of the variable.

virtual std::string get_variable_id() const = 0

Returns the identifier of corresponding variable.

struct VariableInfo
#include <variable.hpp>
class VariableValue
#include <variable_value.hpp>

VariableValue stores data and state (reset flag) for a Variable, and provides an interface for changing them.

Public Functions

VariableValue()

Constructs an uninitialized VariableValue.

explicit VariableValue(ngraph::HostTensorPtr value)

Constructor for VariableValue.

Deprecated:

This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead

Parameters

value – The data for Variable.

VariableValue(ngraph::HostTensorPtr value, bool reset)

Constructor for VariableValue.

Deprecated:

This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead

Parameters
  • value – Data for Variable.

  • reset – The current state of the reset flag.

ngraph::HostTensorPtr get_value() const

Returns the current stored data.

Deprecated:

This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead

void set_value(const ngraph::HostTensorPtr &value)

Sets new values for Variable.

Deprecated:

This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead

Parameters

value – New data for Variable.

void set_reset(bool reset)

Sets the reset flag to a new state.

Parameters

reset – The new state of the reset flag.

bool get_reset() const

Returns the current reset flag state.

VariableValue(const ov::Tensor &value, bool reset)

Constructor for VariableValue.

Deprecated:

This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead

Parameters
  • value – Data for Variable.

  • reset – The current state of the reset flag.

const ov::Tensor &get_state() const

Returns the current stored data.

Deprecated:

This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead

void set_state(const ov::Tensor &value)

Sets new values for Variable.

Deprecated:

This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead

Parameters

value – New data for Variable.

namespace detail

Functions

std::shared_ptr<Node> sigmoid(const std::shared_ptr<Node> &arg, float alpha, float beta)
std::shared_ptr<Node> tanh(const std::shared_ptr<Node> &arg, float alpha, float beta)
std::shared_ptr<Node> relu(const std::shared_ptr<Node> &arg, float alpha, float beta)
std::shared_ptr<Node> hardsigmoid(const std::shared_ptr<Node> &arg, float alpha, float beta)
namespace embedding

Functions

template<class TShape, class TRShape = result_shape_t<TShape>>
TRShape out_shape_infer(const ov::Node *op, const TShape &emb_table_shape, const TShape &dim_shape_src)

Return a copy of the emb_table_shape with the first dimension replaced by the first dimension from the dim_shape_src

Template Parameters

TShapeShape type

Parameters
  • op – Pointer to operator.

  • emb_table_shape – The shape to be copied

  • dim_shape_src – The shape to copy the first dimension from, with dynamic or static rank > 1

Returns

The copy of the emb_table_shape with the first dimsnsion overwritten by dim_shape_src[0] if the rank is static, otherwise fully dynamic shape with dynamic rank.

namespace error
struct UnknownActivationFunction : public ov::Exception
#include <activation_functions.hpp>
namespace fft_common_validation

Enums

enum FFTKind

Values:

enumerator RealInput
enumerator ComplexInput

Functions

template<class T>
void validate_input_rank(const ov::op::util::FFTBase *op, const std::vector<T> &input_shapes, const T &input_shape, const T &axes_shape, int64_t input_rank, FFTKind fft_kind)
template<class T>
void validate_axes(const ov::op::util::FFTBase *op, const std::vector<T> &input_shapes, const T &axes_shape, std::vector<int64_t> &axes, int64_t input_rank, FFTKind fft_kind)
template<class T>
void validate_signal_size(const ov::op::util::FFTBase *op, const std::vector<T> &input_shapes, const T &axes_shape, const T &signal_size_shape)
template<class T>
void shape_validation(const ov::op::util::FFTBase *op, const std::vector<T> &input_shapes, std::vector<int64_t> *axes, FFTKind fft_kind)
namespace v0

Functions

template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Concat *op, const std::vector<T> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const CTCGreedyDecoder *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DepthToSpace *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DetectionOutput *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const FakeQuantize *op, const std::vector<T> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Interpolate *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor)
template<class T>
std::vector<result_shape_t<T>> shape_infer(const LSTMCell *op, const std::vector<T> &input_shapes)
template<class TShape>
std::vector<result_shape_t<TShape>> shape_infer(const LSTMSequence *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const MatMul *op, const std::vector<T> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const PriorBoxClustered *const op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const PriorBox *const op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Proposal *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const PSROIPooling *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Range *op, const std::vector<T> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const RegionYolo *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const ReorgYolo *op, const std::vector<T> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ReverseSequence *op, const std::vector<TShape> &input_shapes)
template<class TShape>
std::vector<result_shape_t<TShape>> shape_infer(const RNNCell *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ROIPooling *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ShapeOf *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ShuffleChannels *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ov::op::v0::SpaceToDepth *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Squeeze *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())

Do Squeeze shape inference.

Template Parameters

T – Type of input/output shapes.

Parameters
  • opSqueeze operator pointer.

  • input_shapesSqueeze input shapes.

  • taTensor accessor to constant data.

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Tile *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
template<class TOp>
void check_unsqueeze_axes_rank(const TOp *op, const Rank &rank)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Unsqueeze *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
class Abs : public ov::op::util::UnaryElementwiseArithmetic
#include <abs.hpp>

Elementwise absolute value operation.

Public Functions

Abs() = default

Constructs an absolute value operation.

Abs(const Output<Node> &arg)

Constructs an absolute value operation.

Output [d1, ...]

Parameters

argOutput that produces the input tensor.[d1, ...]

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Acos : public ov::op::util::UnaryElementwiseArithmetic
#include <acos.hpp>

Elementwise inverse cosine (arccos) operation.

Public Functions

Acos() = default

Constructs an arccos operation.

Acos(const Output<Node> &arg)

Constructs an arccos operation.

Output [d1, ...]

Parameters

argOutput that produces the input tensor.[d1, ...]

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Asin : public ov::op::util::UnaryElementwiseArithmetic
#include <asin.hpp>

Elementwise inverse sine (arcsin) operation.

Public Functions

Asin() = default

Constructs an arcsin operation.

Asin(const Output<Node> &arg)

Constructs an arcsin operation.

Output [d1, ...]

Parameters

argOutput that produces the input tensor.[d1, ...]

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Atan : public ov::op::util::UnaryElementwiseArithmetic
#include <atan.hpp>

Elementwise inverse tangent (arctan) operation.

Public Functions

Atan() = default

Constructs an arctan operation.

Atan(const Output<Node> &arg)

Constructs an arctan operation.

Output [d1, ...]

Parameters

argOutput that produces the input tensor.[d1, ...]

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class BatchNormInference : public ov::op::Op
#include <batch_norm.hpp>

BatchNormInference operation.

Public Functions

BatchNormInference(const Output<Node> &input, const Output<Node> &gamma, const Output<Node> &beta, const Output<Node> &mean, const Output<Node> &variance, double epsilon)
Parameters
  • input – [., C, …]

  • gamma – gamma scaling for normalized value. [C]

  • beta – bias added to the scaled normalized value [C]

  • mean – value for mean normalization [C]

  • variance – value for variance normalization [C]

  • epsilon – Avoids divsion by 0 if input has 0 variance

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Ceiling : public ov::op::util::UnaryElementwiseArithmetic
#include <ceiling.hpp>

Elementwise ceiling operation.

Public Functions

Ceiling() = default

Constructs a ceiling operation.

Ceiling(const Output<Node> &arg)

Constructs a ceiling operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Clamp : public ov::op::util::UnaryElementwiseArithmetic
#include <clamp.hpp>

Performs a clipping operation on all elements of the input node.

All input values that are outside of the <min;max> range are set to ‘min’ or ‘max’ depending on which side of the <min;max> range they are. The values that fall into this range remain unchanged.

Public Functions

Clamp(const Output<Node> &data, const double min, const double max)

Constructs a Clamp node.

Parameters
  • data – - Node producing the input tensor

  • min – - the lower bound of the <min;max> range

  • max – - the upper bound of the <min;max> range

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Concat : public ov::op::Op
#include <concat.hpp>

Concatenation operation.

Public Functions

Concat() = default

Constructs a concatenation operation.

Concat(const OutputVector &args, int64_t axis)

Constructs a concatenation operation.

Parameters
  • args – The outputs producing the input tensors.

  • axis – The axis along which to concatenate the input tensors.

Concat(const NodeVector &args, int64_t axis)

Constructs a concatenation operation.

Parameters
  • args – The nodes producing the input tensors.

  • axis – The axis along which to concatenate the input tensors.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline int64_t get_concatenation_axis() const
Returns

The concatenation axis.

inline int64_t get_axis() const
Returns

The concatenation axis.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Constant : public ov::op::Op
#include <constant.hpp>

Class for constants.

Public Functions

OPENVINO_SUPPRESS_DEPRECATED_START Constant(const std::shared_ptr<ngraph::runtime::Tensor> &tensor)

Initialize a constant from tensor.

Parameters

tensor – The tensor with data

template<typename T>
inline Constant(const element::Type &type, const Shape &shape, std::shared_ptr<ngraph::runtime::SharedBuffer<T>> data)

Constructs a tensor constant with the supplied data.

Parameters
  • type – The element type of the tensor constant.

  • shape – The shape of the tensor constant.

  • data – A pointer to pre-allocated shared data.

OPENVINO_SUPPRESS_DEPRECATED_END Constant(const ov::Tensor &tensor)

Initialize a constant from ov::Tensor.

Parameters

tensor – The ov::Tensor with data

template<typename T>
inline Constant(const element::Type &type, const Shape &shape, const std::vector<T> &values)

Constructs a tensor constant.

Parameters
  • type – The element type of the tensor constant.

  • shape – The shape of the tensor constant.

  • values – A vector of literals for initializing the tensor constant. The size of values must match the size of the shape.

Constant(const element::Type &type, const Shape &shape)

Create uninitialized constant.

template<class T, class = typename std::enable_if<std::is_fundamental<T>::value>::type>
inline Constant(const element::Type &type, const Shape &shape, T value)

Constructs a uniform tensor constant.

Parameters
  • type – The element type of the tensor constant.

  • shape – The shape of the tensor constant.

  • value – A scalar for initializing the uniform tensor constant. The value is broadcast to the specified shape.

Constant(const element::Type &type, const Shape &shape, const std::vector<std::string> &values)

Constructs a tensor constant This constructor is mainly to support deserialization of constants.

Parameters
  • type – The element type of the tensor constant.

  • shape – The shape of the tensor constant.

  • values – A list of string values to use as the constant data.

Constant(const element::Type &type, const Shape &shape, const void *data)

Constructs a tensor constant with the supplied data.

Parameters
  • type – The element type of the tensor constant.

  • shape – The shape of the tensor constant.

  • data – A void* to constant data.

inline virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

Shape get_shape_val() const

Returns the value of the constant node as a Shape object Can only be used on element::i64 nodes and interprets negative values as zeros.

Strides get_strides_val() const

Returns the value of the constant node as a Strides object Can only be used on element::i64 nodes and interprets negative values as zeros.

Coordinate get_coordinate_val() const

Returns the value of the constant node as a Coordinate object Can only be used on element::i64 nodes and interprets negative values as zeros.

CoordinateDiff get_coordinate_diff_val() const

Returns the value of the constant node as a CoordinateDiff object Can only be used on element::i64 nodes.

AxisVector get_axis_vector_val() const

Returns the value of the constant node as an AxisVector object Can only be used on element::i64 nodes and interprets negative values as zeros.

AxisSet get_axis_set_val() const

Returns the value of the constant node as an AxisSet object Can only be used on element::i64 nodes and interprets negative values as zeros. Repeated values are allowed.

size_t get_byte_size() const

Return data size in bytes.

std::vector<std::string> get_value_strings() const
Returns

The initialization literals for the tensor constant.

template<typename T>
inline std::vector<T> cast_vector(int64_t num_elements = -1) const

Return the Constant’s value as a vector cast to type T.

Template Parameters

T – Type to which data vector’s entries will be cast.

Parameters

num_elements – (Optional) Number of elements to cast. In default case returns all elements

Returns

Constant’s data vector.

Public Static Functions

template<typename T>
static inline std::shared_ptr<Constant> create(const element::Type &type, const Shape &shape, const std::vector<T> &values)

Wrapper around constructing a shared_ptr of a Constant.

Parameters
  • type – The element type of the tensor constant.

  • shape – The shape of the tensor constant.

  • values – A vector of values to use as the constant data.

template<typename T>
static inline std::shared_ptr<Constant> create(const element::Type &type, const Shape &shape, std::initializer_list<T> values)

Wrapper around constructing a shared_ptr of a Constant.

Parameters
  • type – The element type of the tensor constant.

  • shape – The shape of the tensor constant.

  • values – An initializer_list of values to use as the constant data.

static inline std::shared_ptr<Constant> create(const element::Type &type, const Shape &shape, const void *memory)

Wrapper around constructing a shared_ptr of a Constant.

Parameters
  • type – The element type of the tensor constant.

  • shape – The shape of the tensor constant.

  • memory – An continues memory chunk which contains the constant data.

class Convert : public ov::op::Op
#include <convert.hpp>

Elementwise type conversion operation.

Public Functions

Convert() = default

Constructs a conversion operation.

Convert(const Output<Node> &arg, const ov::element::Type &destination_type)

Constructs a conversion operation.

Parameters
  • argNode that produces the input tensor.

  • destination_type – Element type for the output tensor.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Cos : public ov::op::util::UnaryElementwiseArithmetic
#include <cos.hpp>

Elementwise cosine operation.

Public Functions

Cos() = default

Constructs a cosine operation.

Cos(const Output<Node> &arg)

Constructs a cosine operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Cosh : public ov::op::util::UnaryElementwiseArithmetic
#include <cosh.hpp>

Elementwise hyperbolic cosine (cosh) operation.

Public Functions

Cosh() = default

Constructs a hyperbolic cosine operation.

Cosh(const Output<Node> &arg)

Constructs a hyperbolic cosine operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class CTCGreedyDecoder : public ov::op::Op
#include <ctc_greedy_decoder.hpp>

CTCGreedyDecoder operation.

Public Functions

CTCGreedyDecoder(const Output<Node> &input, const Output<Node> &seq_len, const bool ctc_merge_repeated)

Constructs a CTCGreedyDecoder operation.

Parameters
  • input – Logits on which greedy decoding is performed

  • seq_len – Sequence lengths

  • ctc_merge_repeated – Whether to merge repeated labels

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class CumSum : public ov::op::Op
#include <cum_sum.hpp>

Tensor cumulative sum operation.

Compute the cumulative sum of the input tensor along the axis specified.

Public Functions

CumSum() = default

Constructs a cumulative summation operation.

CumSum(const Output<Node> &arg, const Output<Node> &axis, const bool exclusive = false, const bool reverse = false)

Constructs a cumulative summation operation.

Parameters
  • arg – The tensor to be summed.

  • axis – zero dimension tensor specifying axis position along which cumulative sum must be performed

  • exclusive – if set to true, the top element is not included

  • reverse – if set to true, will perform the sums in reverse direction

CumSum(const Output<Node> &arg, const bool exclusive = false, const bool reverse = false)

Constructs a cumulative summation operation with axis = 0.

Parameters

arg – The tensor to be summed

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class DepthToSpace : public ov::op::Op
#include <depth_to_space.hpp>

DepthToSpace permutes data from the depth dimension of the input blob into spatial dimensions.

Output node produces a tensor with shape: [N, C/(blocksize * blocksize), H * blocksize, W * blocksize]

Note

Values from the depth dimension (assuming NCHW layout) are moved in spatial blocks to the height and width dimensions.

Public Functions

DepthToSpace(const Output<Node> &data, const DepthToSpaceMode &mode, std::size_t block_size = 1)

Constructs a DepthToSpace operation.

Parameters
  • dataNode producing the input tensor

  • mode – Specifies how the input depth dimension is split to block coordinates

  • block_size – The size of the block of values to be moved

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class DetectionOutput : public ov::op::util::DetectionOutputBase
#include <detection_output.hpp>

Layer which performs non-max suppression to generate detection output using location and confidence predictions.

Public Functions

DetectionOutput(const Output<Node> &box_logits, const Output<Node> &class_preds, const Output<Node> &proposals, const Output<Node> &aux_class_preds, const Output<Node> &aux_box_preds, const Attributes &attrs)

Constructs a DetectionOutput operation.

Parameters
  • box_logits – Box logits

  • class_preds – Class predictions

  • proposals – Proposals

  • aux_class_preds – Auxilary class predictions

  • aux_box_preds – Auxilary box predictions

  • attrs – Detection Output attributes

DetectionOutput(const Output<Node> &box_logits, const Output<Node> &class_preds, const Output<Node> &proposals, const Attributes &attrs)

Constructs a DetectionOutput operation.

Parameters
  • box_logits – Box logits

  • class_preds – Class predictions

  • proposals – Proposals

  • attrs – Detection Output attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

struct Attributes : public ov::op::util::DetectionOutputBase::AttributesBase
#include <detection_output.hpp>
class Elu : public ov::op::util::UnaryElementwiseArithmetic
#include <elu.hpp>

Exponential Linear Unit x < 0 => f(x) = alpha * (exp(x) - 1.) x >= 0 => f(x) = x.

Public Functions

Elu(const Output<Node> &data, const double alpha)

Constructs an Elu operation.

Parameters
  • dataInput tensor

  • alpha – Multiplier for negative values

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Erf : public ov::op::util::UnaryElementwiseArithmetic
#include <erf.hpp>

Elementwise erf operation.

Public Functions

Erf() = default

Constructs a floor operation.

Erf(const Output<Node> &arg)

Constructs a floor operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Exp : public ov::op::util::UnaryElementwiseArithmetic
#include <exp.hpp>

Elementwise natural exponential (exp) operation.

Public Functions

Exp() = default

Constructs an exponential operation.

Exp(const Output<Node> &arg)

Constructs an exponential operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class FakeQuantize : public ov::op::Op
#include <fake_quantize.hpp>

Class performing element-wise linear quantization.

Note

Input floating point values are quantized into a discrete set of floating point values.

Public Functions

FakeQuantize(const Output<Node> &data, const Output<Node> &input_low, const Output<Node> &input_high, const Output<Node> &output_low, const Output<Node> &output_high, std::size_t levels, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a FakeQuantize operation node.

Parameters
  • data[in] The input data tensor.

  • input_low[in] The minimum limit for input values.

  • input_high[in] The maximum limit for input values.

  • output_low[in] The minimum quantized value.

  • output_high[in] The maximum quantized value.

  • levels[in] The number of quantization levels.

  • auto_broadcast[in] AutoBroadcast mode to be used for broadcasting limit values

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Floor : public ov::op::util::UnaryElementwiseArithmetic
#include <floor.hpp>

Elementwise floor operation.

Public Functions

Floor() = default

Constructs a floor operation.

Floor(const Output<Node> &arg)

Constructs a floor operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Gelu : public ov::op::util::UnaryElementwiseArithmetic
#include <gelu.hpp>

Gaussian Error Linear Unit f(x) = 0.5 * x * (1 + erf( x / sqrt(2) )

Public Functions

Gelu(const Output<Node> &data)

Constructs a Gelu operation.

Parameters

dataInput tensor

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class GRN : public ov::op::util::UnaryElementwiseArithmetic
#include <grn.hpp>

Global Response Normalization with L2 norm (across channels only).

Public Functions

GRN(const Output<Node> &data, float bias)

Constructs a GRN operation.

Parameters
  • data – - Node producing the input tensor

  • bias – - The bias added to the variance.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class HardSigmoid : public ov::op::Op
#include <hard_sigmoid.hpp>

Parameterized, bounded sigmoid-like, piecewise linear function. min(max(alpha*x + beta, 0), 1)

Public Functions

HardSigmoid(const Output<Node> &data, const Output<Node> &alpha, const Output<Node> &beta)

Constructs a HardSigmoid operation.

Parameters
  • dataInput tensor.

  • alpha[in] A scalar value representing the alpha parameter.

  • beta[in] A scalar value representing the beta parameter.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Interpolate : public ov::op::Op
#include <interpolate.hpp>

Layer which performs bilinear interpolation.

Public Functions

Interpolate(const Output<Node> &image, const Output<Node> &output_shape, const Attributes &attrs)

Constructs a Interpolate operation.

Parameters
  • imageInput image

  • output_shapeOutput shape of spatial axes

  • attrs – Interpolation attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

struct Attributes
#include <interpolate.hpp>

Structure that specifies attributes for interpolation.

class Log : public ov::op::util::UnaryElementwiseArithmetic
#include <log.hpp>

Elementwise natural log operation.

Public Functions

Log() = default

Constructs a natural log operation.

Log(const Output<Node> &arg)

Constructs a natural log operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class LRN : public ov::op::Op
#include <lrn.hpp>

Elementwise Local Response Normalization (LRN) operation.

Inputs

Type

Description

arg

\(N[n, c, d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape and numeric element type.

Type

Description

\(N[n, c, d_1,\dots,d_n]\)

The tensor \(T\), where \(T[n, c, d_1,\dots,d_n] = \frac{N[n,i,d_1,\dots,d_n]}{ (bias + alpha * (\sum_{i=max(0,(nsize-1)/2)}^{min(C, (nsize-1)/2)+1} N[n,i,d_1,\dots,d_n]^{2}) ^ {2})}\)

Public Functions

LRN() = default

Constructs a LRN operation.

LRN(const Output<Node> &arg, double alpha, double beta, double bias, size_t size)

Constructs a LRN operation.

Parameters

argNode that produces the input tensor.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class LSTMCell : public ov::op::util::RNNCellBase
#include <lstm_cell.hpp>

Class for single lstm cell node.

See also

LSTMSequence, RNNCell, GRUCell

Note

Following implementation supports:

Note

It calculates following equations:

        it = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Pi (.) Ct-1 + Wbi + Rbi)
        ft = f(Xt*(Wf^T) + Ht-1*(Rf^T) + Pf (.) Ct-1 + Wbf + Rbf)
        ct = g(Xt*(Wc^T) + Ht-1*(Rc^T) + Wbc + Rbc)
        Ct = ft (.) Ct-1 + it (.) ct
        ot = f(Xt*(Wo^T) + Ht-1*(Ro^T) + Po (.) Ct + Wbo + Rbo)
        Ht = ot (.) h(Ct)

        *       - Is a dot product,
        (.)     - is a Hadamard product (element-wise),
        f, g, h - are activation functions.

Note

This class represents only single cell (for current time step) and not the whole LSTM Sequence layer

Public Functions

LSTMCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &initial_cell_state, const Output<Node> &W, const Output<Node> &R, std::size_t hidden_size, LSTMWeightsFormat weights_format = LSTMWeightsFormat::IFCO, const std::vector<std::string> &activations = std::vector<std::string>{"sigmoid", "tanh", "tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f, bool input_forget = false)

Constructs LSTMCell node.

Parameters
  • X[in] The input tensor with shape: [batch_size, input_size].

  • initial_hidden_state[in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].

  • initial_cell_state[in] The cell state tensor at current time step with shape: [batch_size, hidden_size].

  • W[in] The gate weights tensor with shape: [4*hidden_size, input_size].

  • R[in] The recurrence weights tensor with shape: [4*hidden_size, hidden_size].

  • hidden_size[in] The number of hidden units for recurrent cell.

  • weights_format[in] The order of gates in weights tensors. The default format is IFCO since it is used by DNNL.

  • activations[in] The vector of activation functions used inside recurrent cell.

  • activations_alpha[in] The vector of alpha parameters for activation functions in order respective to activation list.

  • activations_beta[in] The vector of beta parameters for activation functions in order respective to activation list.

  • clip[in] The value defining clipping range [-clip, clip] on input of activation functions.

  • input_forget[in] Controls coupling input and forget gates.

LSTMCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &initial_cell_state, const Output<Node> &W, const Output<Node> &R, const Output<Node> &B, std::size_t hidden_size, LSTMWeightsFormat weights_format = LSTMWeightsFormat::IFCO, const std::vector<std::string> &activations = std::vector<std::string>{"sigmoid", "tanh", "tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f, bool input_forget = false)

Constructs LSTMCell node.

Parameters
  • X[in] The input tensor with shape: [batch_size, input_size].

  • initial_hidden_state[in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].

  • initial_cell_state[in] The cell state tensor at current time step with shape: [batch_size, hidden_size].

  • W[in] The weight tensor with shape: [4*hidden_size, input_size].

  • R[in] The recurrence weight tensor with shape: [4*hidden_size, hidden_size].

  • B[in] The bias tensor for gates with shape: [4*hidden_size].

  • hidden_size[in] The number of hidden units for recurrent cell.

  • weights_format[in] The order of gates in weights tensors. The default format is IFCO since it is used by DNNL.

  • activations[in] The vector of activation functions used inside recurrent cell.

  • activations_alpha[in] The vector of alpha parameters for activation functions in order respective to activation list.

  • activations_beta[in] The vector of beta parameters for activation functions in order respective to activation list.

  • clip[in] The value defining clipping range [-clip, clip] on input of activation functions.

  • input_forget[in] Controls coupling input and forget gates.

LSTMCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &initial_cell_state, const Output<Node> &W, const Output<Node> &R, const Output<Node> &B, const Output<Node> &P, std::size_t hidden_size, LSTMWeightsFormat weights_format = LSTMWeightsFormat::IFCO, const std::vector<std::string> &activations = std::vector<std::string>{"sigmoid", "tanh", "tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f, bool input_forget = false)

Constructs LSTMCell node.

Parameters
  • X[in] The input tensor with shape: [batch_size, input_size].

  • initial_hidden_state[in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].

  • initial_cell_state[in] The cell state tensor at current time step with shape: [batch_size, hidden_size].

  • W[in] The weight tensor with shape: [4*hidden_size, input_size].

  • R[in] The recurrence weight tensor with shape: [4*hidden_size, hidden_size].

  • B[in] The bias tensor for gates with shape: [4*hidden_size].

  • P[in] The weight tensor for peepholes with shape: [3*hidden_size] - 3 equals to only iof gates. The order is: input, output, forget gates.

  • hidden_size[in] The number of hidden units for recurrent cell.

  • weights_format[in] The order of gates in weights tensors. The default format is IFCO since it is used by DNNL.

  • activations[in] The vector of activation functions used inside recurrent cell.

  • activations_alpha[in] The vector of alpha parameters for activation functions in order respective to activation list.

  • activations_beta[in] The vector of beta parameters for activation functions in order respective to activation list.

  • clip[in] The value defining clipping range [-clip, clip] on input of activation functions.

  • input_forget[in] Controls coupling input and forget gates.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class LSTMSequence : public ov::op::util::RNNCellBase
#include <lstm_sequence.hpp>

Class for lstm sequence node.

See also

LSTMCell, RNNCell, GRUCell

Note

It follows notation and equations defined as in ONNX standard: https://github.com/onnx/onnx/blob/master/docs/Operators.md#LSTM

Public Functions

inline virtual size_t get_default_output_index() const override

Returns the output of the default output, or throws if there is none.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class MatMul : public ov::op::Op
#include <matmul.hpp>

Operator performing Matrix Multiplication.

Public Functions

MatMul(const Output<Node> &A, const Output<Node> &B, const bool &transpose_a = false, const bool &transpose_b = false)

Constructs an Matrix Multiplication operation.

Parameters
  • A – Matrix A

  • B – Matrix B

  • transpose_a – If matrix A should be transposed.

  • transpose_b – If matrix B should be transposed.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class MVN : public ov::op::Op
#include <mvn.hpp>

Operator performing Mean Variance Normalization.

Public Functions

MVN(const Output<Node> &data, bool across_channels = true, bool normalize_variance = true, double eps = 1e-9)

Constructs an MVN operation.

Parameters
  • dataInput tensor with data

  • normalize_variance – flag that denotes whether to perform variance normalization.

  • across_channels – flag that denotes if mean values are shared across channels.

  • eps – the number to be added to the variance to avoid division by zero when normalizing the value

MVN(const Output<Node> &data, AxisSet reduction_axes, bool normalize_variance = true, double eps = 1e-9)

Constructs an MVN operation.

Parameters
  • dataInput tensor with data

  • reduction_axes – A list of axes, along which to reduce.

  • normalize_variance – flag that denotes whether to perform variance normalization.

  • eps – the number to be added to the variance to avoid division by zero when normalizing the value

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Negative : public ov::op::util::UnaryElementwiseArithmetic
#include <negative.hpp>

Elementwise negative operation.

Public Functions

Negative() = default

Constructs a negative operation.

Negative(const Output<Node> &arg)

Constructs a negative operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

template<class TShape>
struct NegativeToZero
#include <tile_shape_inference.hpp>
class NormalizeL2 : public ov::op::Op
#include <normalize_l2.hpp>

Normalization with L2 norm.

Public Functions

NormalizeL2(const Output<Node> &data, const Output<Node> &axes, float eps, EpsMode eps_mode)

Constructs a NormalizeL2 operation.

Parameters
  • data – - Node producing the input tensor

  • axes – - Node indicating axes along which reduction is calculated

  • eps – - The epsilon added to L2 norm.

  • eps_mode – - Specifies how eps is combined with L2 value calculated before division

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Parameter : public ov::op::Op
#include <parameter.hpp>

A model parameter.

Parameters are nodes that represent the arguments that will be passed to user-defined models. Model creation requires a sequence of parameters. Basic graph operations do not need parameters attached to a model.

Public Functions

Parameter() = default

Constructions a tensor-typed parameter node.

Parameter(const ov::element::Type &element_type, const PartialShape &pshape)

Constructions a tensor-typed parameter node.

Parameters
  • element_type – The element type of the parameter.

  • pshape – The partial shape of the parameter.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

Layout get_layout() const

Returns current layout, or empty Layout if it is not set.

void set_layout(const Layout &layout)

Sets layout runtime information to tensor.

Parameters

layoutLayout to set. If empty (default constructed), layout runtime information is erased.

class PRelu : public ov::op::Op
#include <prelu.hpp>

Parametrized Relu x < 0 => f(x) = x * slope x >= 0 => f(x) = x.

Public Functions

PRelu(const Output<Node> &data, const Output<Node> &slope)

Constructs a PRelu operation.

Parameters
  • dataInput tensor

  • slope – Multipliers for negative values

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class PriorBox : public ov::op::Op
#include <prior_box.hpp>

Layer which generates prior boxes of specified sizes normalized to input image size.

Public Functions

PriorBox(const Output<Node> &layer_shape, const Output<Node> &image_shape, const Attributes &attrs)

Constructs a PriorBox operation.

Parameters
  • layer_shapeShape of layer for which prior boxes are computed

  • image_shapeShape of image to which prior boxes are scaled

  • attrsPriorBox attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

struct Attributes
#include <prior_box.hpp>
class PriorBoxClustered : public ov::op::Op
#include <prior_box_clustered.hpp>

Layer which generates prior boxes of specified sizes normalized to input image size.

Public Functions

PriorBoxClustered(const Output<Node> &layer_shape, const Output<Node> &image_shape, const Attributes &attrs)

Constructs a PriorBoxClustered operation.

Parameters
  • layer_shapeShape of layer for which prior boxes are computed

  • image_shapeShape of image to which prior boxes are scaled

  • attrsPriorBoxClustered attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

struct Attributes
#include <prior_box_clustered.hpp>
class Proposal : public ov::op::Op
#include <proposal.hpp>

Proposal operation.

Subclassed by ov::op::v4::Proposal

Public Functions

Proposal(const Output<Node> &class_probs, const Output<Node> &bbox_deltas, const Output<Node> &image_shape, const Attributes &attrs)

Constructs a Proposal operation.

Parameters
  • class_probs – Class probability scores

  • bbox_deltas – Prediction of bounding box deltas

  • image_shapeShape of image

  • attrsProposal op attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

void set_attrs(Attributes attrs)

Set the Proposal operator attributes.

Parameters

attrsAttributes to be set.

struct Attributes
#include <proposal.hpp>
class PSROIPooling : public ov::op::Op
#include <psroi_pooling.hpp>

PSROIPooling operation.

Public Functions

PSROIPooling(const Output<Node> &input, const Output<Node> &coords, const size_t output_dim, const size_t group_size, const float spatial_scale, int spatial_bins_x, int spatial_bins_y, const std::string &mode)

Constructs a PSROIPooling operation.

Parameters
  • inputInput feature map {N, C, …}

  • coords – Coordinates of bounding boxes

  • output_dimOutput channel number

  • group_size – Number of groups to encode position-sensitive scores

  • spatial_scale – Ratio of input feature map over input image size

  • spatial_bins_x – Numbers of bins to divide the input feature maps over width

  • spatial_bins_y – Numbers of bins to divide the input feature maps over height

  • mode – Mode of pooling - Avg or Bilinear

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

void set_output_dim(size_t output_dim)

Set the output channel dimension size.

Parameters

output_dim – Channel dimension size.

void set_group_size(size_t group_size)

Set the output groups number.

Parameters

group_size – Number of groups.

void set_spatial_scale(float scale)

Set the spatial scale.

Parameters

scale – Spatial scale value.

void set_spatial_bins_x(int x)

Set the number of bins over image width.

Parameters

x – Number of bins over width (x) axis.

void set_spatial_bins_y(int y)

Set the number of bins over image height.

Parameters

y – Number of bins over height (y) axis.

void set_mode(std::string mode)

Set the pooling mode.

Parameters

mode – Pooling mode name.

class Range : public ov::op::Op
#include <range.hpp>

Range operation, analogous to range() in Python.

Public Functions

Range() = default

Constructs an unitialized range operation.

Range(const Output<Node> &start, const Output<Node> &stop, const Output<Node> &step)

Constructs a range operation.

Parameters
  • start – The tensor producing the start value. Must be a scalar of integer element type, and same element type as stop and step.

  • stop – The tensor producing the stop value. Must be a scalar of integer element type, and same element type as start and step.

  • step – The tensor producing the step value. Must be a scalar of integer element type, and same element type as start and stop.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class RegionYolo : public ov::op::Op
#include <region_yolo.hpp>

RegionYolo operation.

Public Functions

RegionYolo(const Output<Node> &input, const size_t coords, const size_t classes, const size_t regions, const bool do_softmax, const std::vector<int64_t> &mask, const int axis, const int end_axis, const std::vector<float> &anchors = std::vector<float>{})

Constructs a RegionYolo operation.

Parameters
  • input[in] Input

  • coords[in] Number of coordinates for each region

  • classes[in] Number of classes for each region

  • regions[in] Number of regions

  • do_softmax[in] Compute softmax

  • mask[in] Mask

  • axis[in] Axis to begin softmax on

  • end_axis[in] Axis to end softmax on

  • anchors[in] A flattened list of pairs [width, height] that describes prior box sizes.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Relu : public ov::op::util::UnaryElementwiseArithmetic
#include <relu.hpp>

Elementwise Relu operation.

Public Functions

Relu(const Output<ov::Node> &arg)

Constructs a Relu operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReorgYolo : public ov::op::Op
#include <reorg_yolo.hpp>

ReorgYolo operation.

Public Functions

ReorgYolo(const Output<Node> &input, const size_t stride)

Constructs a ReorgYolo operation.

Parameters
  • inputInput

  • stride – Stride to reorganize input by

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Result : public ov::op::Op
#include <result.hpp>

Result operation.

Public Functions

Result() = default

Allows a value to be used as a function result.

Result(const Output<Node> &arg)

Allows a value to be used as a function result.

Parameters

argNode that produces the input tensor.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

Layout get_layout() const

Returns current layout, or empty Layout if it is not set.

void set_layout(const Layout &layout)

Sets layout runtime information to tensor.

Parameters

layoutLayout to set. If empty (default constructed), layout runtime information is erased.

class ReverseSequence : public ov::op::Op
#include <reverse_sequence.hpp>

ReverseSequence operation.

Public Functions

ReverseSequence(const Output<Node> &arg, const Output<Node> &seq_lengths, int64_t batch_axis = 0, int64_t seq_axis = 1)

Constructs a ReverseSequence operation.

Parameters
  • arg – tensor with input data to reverse

  • seq_lengths – 1D tensor of integers with sequence lengths in the input tensor.

  • batch_axis – index of the batch dimension.

  • seq_axis – index of the sequence dimension.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class RNNCell : public ov::op::util::RNNCellBase
#include <rnn_cell.hpp>

Class for single RNN cell node.

See also

LSTMSequence, LSTMCell, GRUCell

Note

It follows notation and equations defined as in ONNX standard: https://github.com/onnx/onnx/blob/master/docs/Operators.md#RNN

Note

It calculates following equations:

        Ht = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Wbi + Rbi)

        *       - Is a dot product,
        f       - is activation functions.

Note

This class represents only single cell (for current time step) and not the whole RNN Sequence layer

Public Functions

RNNCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &W, const Output<Node> &R, std::size_t hidden_size, const std::vector<std::string> &activations = std::vector<std::string>{"tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f)

Constructs RNNCell node.

Parameters
  • X[in] The input tensor with shape: [batch_size, input_size].

  • initial_hidden_state[in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].

  • W[in] The weight tensor with shape: [hidden_size, input_size].

  • R[in] The recurrence weight tensor with shape: [hidden_size, hidden_size].

  • hidden_size[in] The number of hidden units for recurrent cell.

  • activations[in] The vector of activation functions used inside recurrent cell.

  • activations_alpha[in] The vector of alpha parameters for activation functions in order respective to activation list.

  • activations_beta[in] The vector of beta parameters for activation functions in order respective to activation list.

  • clip[in] The value defining clipping range [-clip, clip] on input of activation functions.

RNNCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &W, const Output<Node> &R, const Output<Node> &B, std::size_t hidden_size, const std::vector<std::string> &activations = std::vector<std::string>{"tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f)

Constructs RNNCell node.

Parameters
  • X[in] The input tensor with shape: [batch_size, input_size].

  • initial_hidden_state[in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].

  • W[in] The weight tensor with shape: [hidden_size, input_size].

  • R[in] The recurrence weight tensor with shape: [hidden_size, hidden_size].

  • B[in] The bias tensor for input gate with shape: [hidden_size].

  • hidden_size[in] The number of hidden units for recurrent cell.

  • activations[in] The vector of activation functions used inside recurrent cell.

  • activations_alpha[in] The vector of alpha parameters for activation functions in order respective to activation list.

  • activations_beta[in] The vector of beta parameters for activation functions in order respective to activation list.

  • clip[in] The value defining clipping range [-clip, clip] on input of activation functions.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ROIPooling : public ov::op::Op
#include <roi_pooling.hpp>

ROIPooling operation.

Public Functions

ROIPooling(const Output<Node> &input, const Output<Node> &coords, const Shape &output_size, const float spatial_scale, const std::string &method = "max")

Constructs a ROIPooling operation.

Parameters
  • inputInput feature map {N, C, H, W}

  • coords – Coordinates of bounding boxes

  • output_size – Height/Width of ROI output features

  • spatial_scale – Ratio of input feature map over input image size

  • method – Method of pooling - Max or Bilinear

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

void set_output_roi(Shape output_size)

Set the output ROI feature map (pooled_h, pooled_w).

Parameters

output_sizeShape with pooling attributes pooled_h and pooled_w sizes.

const Shape &get_output_roi() const

Get the output ROI feature map shape (H x W)

Returns

Shape with pooled_h and pooled_w attributes.

void set_spatial_scale(float scale)

Set the spatial scale value.

Parameters

scale – Scale value to set.

void set_method(std::string method_name)

Set the method of pooling.

Parameters

method_name – Pooling method name.

class Selu : public ov::op::Op
#include <selu.hpp>

Performs a SELU activation function on all elements of the input node.

Public Functions

Selu(const Output<Node> &data, const Output<Node> &alpha, const Output<Node> &lambda)

Constructs a Selu node.

Parameters
  • data – - Node producing the input tensor

  • alpha – - Alpha coefficient of SELU operation

  • lambda – - Lambda coefficient of SELU operation

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ShapeOf : public ov::op::util::ShapeOfBase
#include <shape_of.hpp>

Operation that returns the shape of its input argument as a tensor.

Public Functions

ShapeOf(const Output<Node> &arg)

Constructs a shape-of operation.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ShuffleChannels : public ov::op::Op
#include <shuffle_channels.hpp>

Permutes data in the channel dimension of the input.

Public Functions

ShuffleChannels(const Output<Node> &data, const int64_t axis = 1, const int64_t group = 1)

Constructs a ShuffleChannels node.

Parameters
  • dataNode producing the input tensor.

  • axis – Channel dimension index in the data tensor. A negative value means that the index should be calculated from the back of the input data shape.

  • group – Number of group the channel dimension should be split into.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Sigmoid : public ov::op::util::UnaryElementwiseArithmetic
#include <sigmoid.hpp>

Sigmoid operation.

Public Functions

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Sign : public ov::op::util::UnaryElementwiseArithmetic
#include <sign.hpp>

Elementwise sign operation.

Public Functions

Sign(const Output<Node> &arg)

Constructs an elementwise sign operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Sin : public ov::op::util::UnaryElementwiseArithmetic
#include <sin.hpp>

Elementwise sine operation.

Inputs

Type

Description

arg

\(N[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape and numeric element type.

Type

Description

\(N[d_1,\dots,d_n]\)

The tensor \(T\), where \(T[i_1,\dots,i_n] = \sin(\texttt{arg}[i_1,\dots,i_n])\)

Public Functions

Sin(const Output<Node> &arg)

Constructs a sine operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Sinh : public ov::op::util::UnaryElementwiseArithmetic
#include <sinh.hpp>

Elementwise hyperbolic sine (sinh) operation.

Public Functions

Sinh(const Output<Node> &arg)

Constructs a hyperbolic sine operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class SpaceToDepth : public ov::op::Op
#include <space_to_depth.hpp>

SpaceToDepth permutes input tensor blocks of spatial data into depth dimension.

Note

Values from the height and width dimensions are moved to the depth dimension.

   Output node produces a tensor with shape:
   [N, C * blocksize * blocksize, H / blocksize, W / blocksize]

Public Functions

SpaceToDepth(const Output<Node> &data, const SpaceToDepthMode &mode, std::size_t block_size = 1)

Constructs a SpaceToDepth operation.

Parameters
  • data – - Node producing the input tensor

  • mode – Specifies how the output depth dimension is gathered from block coordinates and the old depth dimension.

  • block_size – - the size of the block of values to be moved

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Sqrt : public ov::op::util::UnaryElementwiseArithmetic
#include <sqrt.hpp>

Elementwise square root operation.

Inputs

Type

Description

arg

\(N[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape and numeric element type.

Type

Description

\(N[d_1,\dots,d_n]\)

The tensor \(T\), where \(T[i_1,\dots,i_n] = \sqrt{\texttt{arg}[i_1,\dots,i_n]}\)

Public Functions

Sqrt(const Output<Node> &arg)

Constructs a square operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class SquaredDifference : public ov::op::util::BinaryElementwiseArithmetic
#include <squared_difference.hpp>

Calculates an element-wise squared difference between two tensors.

y[i] = (x1[i] - x2[i])^2

Public Functions

inline SquaredDifference()

Constrcuts an uninitialized squared difference operation.

SquaredDifference(const Output<Node> &x1, const Output<Node> &x2, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs the squared difference operation.

Parameters
  • x1 – First input tensor

  • x2 – Second input tensor

  • auto_broadcast – Auto broadcast specification

class Squeeze : public ov::op::Op
#include <squeeze.hpp>

Squeeze operation.

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Tan : public ov::op::util::UnaryElementwiseArithmetic
#include <tan.hpp>

Elementwise tangent operation.

Inputs

Type

Description

arg

\(N[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape and numeric element type.

Type

Description

\(N[d_1,\dots,d_n]\)

The tensor \(T\), where \(T[i_1,\dots,i_n] = \tan(\texttt{arg}[i_1,\dots,i_n])\)

Public Functions

Tan(const Output<Node> &arg)

Constructs a tangent operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Tanh : public ov::op::util::UnaryElementwiseArithmetic
#include <tanh.hpp>

Elementwise hyperbolic tangent operation.

Public Functions

Tanh(const Output<Node> &arg)

Constructs a hyperbolic tangent operation.

Parameters

argNode that produces the input tensor.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class TensorIterator : public ov::op::util::SubGraphOp
#include <tensor_iterator.hpp>

Iterate a body over tensors, accumulating into tensors.

Public Functions

inline std::shared_ptr<Model> get_body() const
Returns

the body of the iteration

inline void set_body(const std::shared_ptr<Model> &body)
Parameters

body – set the body of the iteration

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Tile : public ov::op::Op
#include <tile.hpp>

Dynamic Tiling operation which repeats a tensor multiple times along each dimension.

Public Functions

Tile(const Output<Node> &data, const Output<Node> &repeats)

Perform dynamic padding of a tensor.

Parameters
  • data – The node producing input tensor to be padded.

  • repeats – The node producing the per-dimension replication factor

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

class Unsqueeze : public ov::op::Op
#include <unsqueeze.hpp>

Unsqueeze operation.

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Xor : public ov::op::util::BinaryElementwiseLogical
#include <xor.hpp>

Elementwise logical-xor operation.

Public Functions

Xor(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec())

Constructs a logical-xor operation.

Output [d0, ...]

Parameters
  • arg0Node that produces the first input tensor.[d0, ...]

  • arg1Node that produces the second input tensor.[d0, ...]

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

namespace lstm_cell

Variables

constexpr size_t gates_count = 4
constexpr size_t num_state_nodes = 2
constexpr size_t peepholes_count = 3
namespace v1

Functions

template<class TShape, class TContainer, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const AvgPool *op, const std::vector<TShape> &input_shapes, TContainer &pads_begin, TContainer &pads_end)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const BatchToSpace *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const ov::op::v1::Broadcast *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ConvolutionBackpropData *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end, const ITensorAccessor &ta = make_tensor_accessor())
template<class TOp, class TShape, class TRShape = result_shape_t<TShape>, typename std::enable_if<std::is_same<TOp, Convolution>::value || std::is_same<TOp, BinaryConvolution>::value>::type* = nullptr>
std::vector<TRShape> shape_infer(const TOp *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DeformableConvolution *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DeformablePSROIPooling *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const GatherTree *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const GroupConvolutionBackpropData *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const GroupConvolution *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)
template<class TShape, class TContainer, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const MaxPool *op, const std::vector<TShape> &input_shapes, TContainer &pads_begin, TContainer &pads_end)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const NonMaxSuppression *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
inline void resolve_axis(OneHot *op)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const OneHot *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Reshape *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Reverse *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())

Reverse shape inference.

Template Parameters

TShape – Type of shape.

Parameters
  • op – Pointer to Reverse operator.

  • input_shapesInput shapes of Reverse.

  • constant_data – Map of constant data. Default empty.

Returns

Vector of output shapes with one shape.

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Select *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const SpaceToBatch *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<typename T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Split *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())

Shape inference for Split V1 operator.

Note

The split operation cause label lost on splitted dimension even if number of splits is one, because in this case split will be removed by transformation (as NOP) and in fact label will be propagated.

Template Parameters

T – Type of shape.

Parameters
  • opSplit operator pointer.

  • input_shapesSplit input shapes.

  • taTensor accessor to constant data.

template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const StridedSlice *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class T, class TRShape = result_shape_t<T>>
TRShape calc_output_shape(const Transpose *const op, const T &input_shape, std::vector<int64_t> &axes_order)

Calculate transpose output shape.

Template Parameters

T – Type of shape

Parameters
Returns

Output shape

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Transpose *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())

Do transpose shape inference on input and output shapes.

Template Parameters
  • TShape – Type of input shapes.

  • TRShape – Type of return shapes.

Parameters
  • opTranspose operator pointer.

  • input_shapesInput shapes of transpose.

  • tensor_accessor – Accessor to constant data.

template<typename T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const VariadicSplit *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
class Add : public ov::op::util::BinaryElementwiseArithmetic
#include <add.hpp>

Elementwise addition operation.

Public Functions

inline Add()

Constructs an uninitialized addition operation.

Add(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs an addition operation.

Output [d0, ...]

Parameters
  • arg0Output that produces the first input tensor.[d0, ...]

  • arg1Output that produces the second input tensor.[d0, ...]

  • auto_broadcast – Auto broadcast specification. Default is Numpy-style implicit broadcasting.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class AvgPool : public ov::op::Op
#include <avg_pool.hpp>

Batched average pooling operation.

Public Functions

AvgPool() = default

Constructs a batched average pooling operation.

AvgPool(const Output<Node> &arg, const Strides &strides, const Shape &pads_begin, const Shape &pads_end, const Shape &kernel, bool exclude_pad, op::RoundingType rounding_type = op::RoundingType::FLOOR, const PadType &auto_pad = op::PadType::EXPLICIT)

Constructs a batched average pooling operation.

Parameters
  • arg – The output producing the input data batch tensor.[d1, dn]

  • strides – The strides.[n]

  • pads_begin – The beginning of padding shape.[n]

  • pads_end – The end of padding shape.[n]

  • kernel – The kernel shape.[n]

  • exclude_pad – If false then averages include padding elements, each treated as the number zero. If true, padding elements are entirely ignored when computing averages.

  • rounding_type – Whether to use ceiling or floor rounding type while computing output shape.

  • auto_pad – Padding type to use for additional padded dimensions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

const Shape &get_kernel() const
Returns

The kernel shape.

const Strides &get_strides() const
Returns

The strides.

const Shape &get_pads_begin() const
Returns

The beginning of padding shape.

const Shape &get_pads_end() const
Returns

The end of padding shape.

const PadType &get_auto_pad() const
Returns

The pad type for pooling.

class BatchToSpace : public ov::op::Op
#include <batch_to_space.hpp>

BatchToSpace permutes data from the batch dimension of the data tensor into spatial dimensions.

Note

Values from the batch dimension are moved in spatial blocks dimensions.

   Output node produces a tensor with shape:
   `[batch / (block_shape[0] * block_shape[1] * ... * block_shape[N - 1]),
    D_1 * block_shape[1] - crops_begin[1] - crops_end[1],
    D_2 * block_shape[2] - crops_begin[2] - crops_end[2], ...,
    D_{N - 1} * block_shape[N - 1] - crops_begin[N - 1] - crops_end[N - 1]`
    of the same type as `data` input.

Public Functions

BatchToSpace(const Output<Node> &data, const Output<Node> &block_shape, const Output<Node> &crops_begin, const Output<Node> &crops_end)

Constructs a BatchToSpace operation.

Parameters
  • dataNode producing the data tensor

  • block_shape – The sizes of the block of values to be moved

  • crops_begin – Specifies the amount to crop from the beginning along each axis of data input

  • crops_end – Specifies the amount to crop from the ending along each axis of data input.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class BinaryConvolution : public ov::op::util::ConvolutionFwdPropBase
#include <binary_convolution.hpp>

BinaryConvolution operation.

Public Functions

BinaryConvolution() = default

Constructs a binary convolution operation.

BinaryConvolution(const Output<Node> &data, const Output<Node> &kernel, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, BinaryConvolutionMode mode, float pad_value, const PadType &auto_pad = PadType::EXPLICIT)

Constructs a binary convolution operation.

Output [N, C_OUT, R1, ... Rf]

Parameters
  • data – The node producing the input data batch tensor.

  • kernel – The node producing the filters tensor.

  • strides – The strides.

  • pads_begin – The beginning of padding shape.

  • pads_end – The end of padding shape.

  • dilations – The dilations.

  • mode – Defines how input tensor 0/1 values and weights 0/1 are interpreted.

  • pad_value – Floating-point value used to fill pad area.

  • auto_pad – The pad type for automatically computing padding sizes.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline const BinaryConvolutionMode &get_mode() const
Returns

The mode of convolution.

inline float get_pad_value() const
Returns

The pad value.

class Broadcast : public ov::op::util::BroadcastBase
#include <broadcast.hpp>

Operation which “adds” axes to an input tensor, replicating elements from the input as needed along the new axes.

Public Functions

Broadcast() = default

Constructs a broadcast operation.

Broadcast(const Output<Node> &arg, const Output<Node> &target_shape, const Output<Node> &axes_mapping, const AutoBroadcastSpec &broadcast_spec = AutoBroadcastSpec())

Constructs a broadcast operation.

Parameters
  • arg – The input tensor to be broadcast.

  • target_shape – The shape of the output tensor.

  • axes_mapping – The axis positions (0-based) in the result that correspond to input axes. ‘Arg’ tensor is broadcast along the remaining axes. E.g., Input Shape - [3, 4], Target Shape - [3, 5, 4, 4] axes_mapping - [0, 2] => Broadcast along axes 1 and 3. axes_mapping - [0, 3] => Broadcast along axes 1 and 2.

  • broadcast_specBroadcast specification to use for determining broadcast axes. ‘axes_mapping’ is ignored if broadcast_spec is not NONE

Broadcast(const Output<Node> &arg, const Output<Node> &target_shape, const AutoBroadcastSpec &broadcast_spec = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a broadcast operation.

Parameters
  • arg – The input tensor to be broadcast.

  • target_shape – The shape of the output tensor.

  • broadcast_specBroadcast specification to use for determining broadcast axes

inline const AutoBroadcastSpec &get_broadcast_spec() const
Returns

Broadcast Specification.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ConvertLike : public ov::op::Op
#include <convert_like.hpp>

Elementwise type conversion operation.

Public Functions

ConvertLike() = default

Constructs a conversion operation.

ConvertLike(const Output<Node> &data, const Output<Node> &like)

Constructs a conversion operation.

Parameters
  • dataNode that produces the input tensor.

  • likeNode which provides the target type information for the conversion.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Convolution : public ov::op::util::ConvolutionFwdPropBase
#include <convolution.hpp>

Batched convolution operation, with optional window dilation and stride.

Public Functions

Convolution() = default

Constructs a batched convolution operation.

Convolution(const Output<Node> &data_batch, const Output<Node> &filters, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT)

Constructs a batched convolution operation.

Output [N, C_OUT, R1, ... Rf]

Parameters
  • data_batch – The node producing the input data batch tensor.[N, C_IN, D1, ... Df]

  • filters – The node producing the filters tensor.[C_OUT, C_IN, F1, ... Ff]

  • strides – The strides.[f]

  • dilations – The dilations.[f]

  • pads_begin – The beginning of padding shape.[f]

  • pads_end – The end of padding shape.[f]

  • auto_pad – The pad type for automatically computing padding sizes.[f]

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ConvolutionBackpropData : public ov::op::util::ConvolutionBackPropBase
#include <convolution.hpp>

Data batch backprop for batched convolution operation.

Public Functions

ConvolutionBackpropData() = default

Constructs a batched-convolution data batch-backprop operation.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

const PartialShape get_output_shape() const
Returns

The output spatial dimensions shape.

void infer_conv_backprop_output_spatial_shape(const std::vector<Dimension> &input_data_shape, const std::vector<Dimension> &filters_shape, const Strides &strides, const Strides &dilations, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const CoordinateDiff &output_padding, std::vector<Dimension> &output_spatial_shape)

Calculates output spatial features size.

Parameters
  • input_data_shape[in] The input data partial shape

  • filters_shape[in] The filters partial shape

  • strides[in] The strides values.

  • dilations[in] The dilations values.

  • pads_begin[in] The paddings at the beginning of axis.

  • pads_end[in] The paddings at the end of axis.

  • output_padding[in] The output padding values.

  • output_spatial_shape – The placeholder for computed output spatial partial shape.

class DeformableConvolution : public ov::op::util::DeformableConvolutionBase
#include <deformable_convolution.hpp>

DeformableConvolution operation.

Public Functions

DeformableConvolution() = default

Constructs a conversion operation.

DeformableConvolution(const Output<Node> &arg, const Output<Node> &offsets, const Output<Node> &filters, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT, const int64_t group = 1, const int64_t deformable_group = 1)

Constructs a conversion operation.

Parameters
  • argNode that produces the input tensor.

  • offsetsNode producing the deformable values tensor.

  • filtersNode producing the filters(kernels) tensor with OIZYX layout.

  • stridesConvolution strides.

  • pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.

  • pads_end – Amount of padding to be added to the end along each axis.

  • dilations – The distance in width and height between the weights in the filters tensor.

  • auto_pad – Specifies how the automatic calculation of padding should be done.

  • group – The number of groups which both output and input should be split into.

  • deformable_group – The number of groups which deformable values and output should be split into along the channel axis.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class DeformablePSROIPooling : public ov::op::Op
#include <deformable_psroi_pooling.hpp>

DeformablePSROIPooling operation.

Public Functions

DeformablePSROIPooling(const Output<Node> &input, const Output<Node> &coords, const Output<Node> &offsets, const int64_t output_dim, const float spatial_scale, const int64_t group_size = 1, const std::string mode = "bilinear_deformable", int64_t spatial_bins_x = 1, int64_t spatial_bins_y = 1, float trans_std = 1, int64_t part_size = 1)

Constructs a DeformablePSROIPooling operation.

Parameters
  • inputInput tensor with position sensitive score maps

  • coordsInput tensor with list of five element tuples describing ROI coordinates

  • offsetsInput tensor with transformation values

  • output_dim – Pooled output channel number

  • group_size – Number of horizontal bins per row to divide ROI area, it defines output width and height

  • spatial_scale – Multiplicative spatial scale factor to translate ROI coordinates from their input scale to the scale used when pooling

  • mode – Specifies mode for pooling.

  • spatial_bins_x – Specifies numbers of bins to divide ROI single bin over width

  • spatial_bins_y – Specifies numbers of bins to divide ROI single bin over height

  • no_trans – The flag that specifies whenever third input exists and contains transformation (offset) values

  • trans_std – The value that all transformation (offset) values are multiplied with

  • part_size – The number of parts the output tensor spatial dimensions are divided into. Basically it is the height and width of the third input

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Divide : public ov::op::util::BinaryElementwiseArithmetic
#include <divide.hpp>

Elementwise division operation.

Public Functions

inline Divide()

Constructs a division operation.

Divide(const Output<Node> &arg0, const Output<Node> &arg1, bool pythondiv, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a division operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • pythondiv – Use Python style rounding for integral type

  • auto_broadcast – Auto broadcast specification

Divide(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a division operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Equal : public ov::op::util::BinaryElementwiseComparison
#include <equal.hpp>

Elementwise is-equal operation.

Inputs

Type

Description

arg0

\(E[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape and element type.

arg1

\(E[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of the same shape and element type as arg0.

autob

AutoBroadcastSpec

Auto broadcast specification.

Type

Description

\(\texttt{bool}[d_1,\dots,d_n]\)

The tensor \(T\), where \(T[i_1,\dots,i_n] = 1\text{ if }\texttt{arg0}[i_1,\dots,i_n] = \texttt{arg1}[i_1,\dots,i_n]\text{, else } 0\)

Public Functions

inline Equal()

Constructs an equal operation.

Equal(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs an equal operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class FloorMod : public ov::op::util::BinaryElementwiseArithmetic
#include <floor_mod.hpp>

Elementwise FloorMod operation.

Public Functions

inline FloorMod()

Constructs an uninitialized addition operation.

FloorMod(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastType::NUMPY)

Constructs an Floor Mod operation.

Output [d0, ...]

Parameters
  • arg0Output that produces the first input tensor.[d0, ...]

  • arg1Output that produces the second input tensor.[d0, ...]

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Gather : public ov::op::util::GatherBase
#include <gather.hpp>

Gather slices from axis of data according to indices.

Public Functions

Gather(const Output<Node> &params, const Output<Node> &indices, const Output<Node> &axis)
Parameters
  • data – The tensor from which slices are gathered

  • indicesTensor with indexes to gather

  • axis – The tensor is a dimension index to gather data from

class GatherTree : public ov::op::Op
#include <gather_tree.hpp>

Generates the complete beams from the ids per each step and the parent beam ids.

Public Functions

GatherTree(const Output<Node> &step_ids, const Output<Node> &parent_idx, const Output<Node> &max_seq_len, const Output<Node> &end_token)
Parameters
  • step_idsTensor of shape [MAX_TIME, BATCH_SIZE, BEAM_WIDTH] with indices from per each step

  • parent_idxTensor of shape [MAX_TIME, BATCH_SIZE, BEAM_WIDTH] with parent beam indices

  • max_seq_lenTensor of shape [BATCH_SIZE] with maximum lengths for each sequence in the batch

  • end_tokenTensor of shape [MAX_TIME, BATCH_SIZE, BEAM_WIDTH]

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Greater : public ov::op::util::BinaryElementwiseComparison
#include <greater.hpp>

Elementwise greater-than operation.

Public Functions

inline Greater()

Constructs a greater-than operation.

Greater(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a greater-than operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class GreaterEqual : public ov::op::util::BinaryElementwiseComparison
#include <greater_eq.hpp>

Elementwise greater-than-or-equal operation.

Public Functions

inline GreaterEqual()

Constructs a greater-than-or-equal operation.

GreaterEqual(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a greater-than-or-equal operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class GroupConvolution : public ov::op::util::ConvolutionFwdPropBase
#include <group_conv.hpp>

Batched convolution operation, with optional window dilation and stride.

Public Functions

GroupConvolution() = default

Constructs a batched convolution operation.

GroupConvolution(const Output<Node> &data_batch, const Output<Node> &filters, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT)

Constructs a batched convolution operation.

Output [N, FC_OUT * GROUPS, R1, ... Rf]

Parameters
  • data_batch – The node producing the input data batch tensor.[N, C_IN, D1, ... Df]

  • filters – The node producing the filters tensor.[GROUPS, FC_OUT, FC_IN, F1, ... Ff]

  • strides – The strides.[f]

  • dilations – The dilations.[f]

  • pads_begin – The beginning of padding shape.[f]

  • pads_end – The end of padding shape.[f]

  • auto_pad – The pad type for automatically computing padding sizes.[f]

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class GroupConvolutionBackpropData : public ov::op::util::ConvolutionBackPropBase
#include <group_conv.hpp>

Data batch backprop for batched convolution operation.

Public Functions

GroupConvolutionBackpropData()

Constructs a batched-convolution data batch-backprop operation.

void infer_conv_backprop_output_spatial_shape(const std::vector<Dimension> &input_data_shape, const std::vector<Dimension> &filters_shape, const Strides &strides, const Strides &dilations, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const CoordinateDiff &output_padding, std::vector<Dimension> &output_spatial_shape)

Calculates output spatial features size.

Parameters
  • input_data_shape[in] The input data partial shape

  • filters_shape[in] The filters partial shape

  • strides[in] The strides values.

  • dilations[in] The dilations values.

  • pads_begin[in] The paddings at the beginning of axis.

  • pads_end[in] The paddings at the end of axis.

  • output_padding[in] The output padding values.

  • output_spatial_shape – The placeholder for computed output spatial partial shape.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

const PartialShape get_convolution_output_shape() const
Returns

The spatial shape of the output.

class Less : public ov::op::util::BinaryElementwiseComparison
#include <less.hpp>

Elementwise less-than operation.

Public Functions

inline Less()

Constructs a less-than operation.

Less(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a less-than operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class LessEqual : public ov::op::util::BinaryElementwiseComparison
#include <less_eq.hpp>

Elementwise less-than-or-equal operation.

Public Functions

inline LessEqual()

Constructs a less-than-or-equal operation.

LessEqual(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a less-than-or-equal operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class LogicalAnd : public ov::op::util::BinaryElementwiseLogical
#include <logical_and.hpp>

Elementwise logical-and operation.

Public Functions

LogicalAnd() = default

Constructs a logical-and operation.

LogicalAnd(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a logical-and operation.

Output [d0, ...]

Parameters
  • arg0Output that produces the first input tensor.[d0, ...]

  • arg1Output that produces the second input tensor.[d0, ...]

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class LogicalNot : public ov::op::Op
#include <logical_not.hpp>

Elementwise logical negation operation.

Public Functions

LogicalNot() = default

Constructs a logical negation operation.

LogicalNot(const Output<Node> &arg)

Constructs a logical negation operation.

Parameters

argNode that produces the input tensor.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class LogicalOr : public ov::op::util::BinaryElementwiseLogical
#include <logical_or.hpp>

Elementwise logical-or operation.

Public Functions

LogicalOr(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a logical-or operation.

Output [d0, ...]

Parameters
  • arg0Node that produces the first input tensor.[d0, ...]

  • arg1Node that produces the second input tensor.[d0, ...]

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class LogicalXor : public ov::op::util::BinaryElementwiseLogical
#include <logical_xor.hpp>

Elementwise logical-xor operation.

Public Functions

LogicalXor(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a logical-xor operation.

Output [d0, ...]

Parameters
  • arg0Node that produces the first input tensor.[d0, ...]

  • arg1Node that produces the second input tensor.[d0, ...]

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Maximum : public ov::op::util::BinaryElementwiseArithmetic
#include <maximum.hpp>

Elementwise maximum operation.

Public Functions

inline Maximum()

Constructs a maximum operation.

Maximum(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a maximum operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class MaxPool : public ov::op::util::MaxPoolBase
#include <max_pool.hpp>

Batched max pooling operation.

Public Functions

MaxPool() = default

Constructs a batched max pooling operation.

MaxPool(const Output<Node> &arg, const Strides &strides, const Shape &pads_begin, const Shape &pads_end, const Shape &kernel, const op::RoundingType rounding_type = op::RoundingType::FLOOR, const PadType auto_pad = op::PadType::EXPLICIT)

Constructs a batched max pooling operation.

Parameters
  • arg – The node producing the input data batch tensor.

  • strides – The strides.

  • pads_begin – The beginning of padding shape.

  • pads_end – The end of padding shape.

  • kernel – The kernel shape.

  • rounding_type – Whether to use ceiling or floor rounding type while computing output shape.

  • auto_pad – The pad type for automatically computing padding sizes.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Minimum : public ov::op::util::BinaryElementwiseArithmetic
#include <minimum.hpp>

Elementwise minimum operation.

Public Functions

inline Minimum()

Constructs a minimum operation.

Minimum(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a minimum operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Mod : public ov::op::util::BinaryElementwiseArithmetic
#include <mod.hpp>

Mod returns an element-wise division reminder with two given tensors applying multi-directional broadcast rules.

Public Functions

inline Mod()

Constructs a Mod node.

Mod(const Output<Node> &A, const Output<Node> &B, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))
Parameters
  • A – - Dividend tensor

  • B – - Divisor tensor

  • auto_broadcast – Auto broadcast specification

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Multiply : public ov::op::util::BinaryElementwiseArithmetic
#include <multiply.hpp>

Elementwise multiplication operation.

Public Functions

inline Multiply()

Constructs a multiplication operation.

Multiply(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a multiplication operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class NonMaxSuppression : public ov::op::Op
#include <non_max_suppression.hpp>

Elementwise addition operation.

Public Functions

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const Output<Node> &iou_threshold, const Output<Node> &score_threshold, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true)

Constructs a NonMaxSuppression operation.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • iou_thresholdNode producing intersection over union threshold

  • score_thresholdNode producing minimum score threshold

  • box_encoding – Specifies the format of boxes data encoding

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true)

Constructs a NonMaxSuppression operation with default values for the last 3 inputs.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box coordinates

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class NotEqual : public ov::op::util::BinaryElementwiseComparison
#include <not_equal.hpp>

Elementwise not-equal operation.

Public Functions

inline NotEqual()

Constructs a not-equal operation.

NotEqual(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a not-equal operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class OneHot : public ov::op::Op
#include <one_hot.hpp>

OneHot operation.

Public Functions

OneHot() = default

Constructs a one-hot operation.

OneHot(const Output<Node> &indices, const Output<Node> &depth, const Output<Node> &on_value, const Output<Node> &off_value, int64_t axis)

Constructs a one-hot operation.

Parameters
  • indicesInput tensor containing indices.

  • depth – Specifies number of classes and the size of one-hot dimension.

  • on_value – Specifies value that the locations in output tensor represented by indices in input take.

  • off_value – Specifies value that the locations in output tensor not represented by indices in input take.

  • axis – Axis along which one-hot representation in added.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

inline const int64_t &get_axis() const
Returns

The index of the one-hot axis.

class Pad : public ov::op::util::PadBase
#include <pad.hpp>

Generic padding operation.

Public Functions

Pad() = default

Constructs a Pad-1 operation.

Pad(const Output<Node> &arg, const Output<Node> &pads_begin, const Output<Node> &pads_end, const Output<Node> &arg_pad_value, PadMode pad_mode)

Constructs a Pad-1 operation.

Parameters
  • arg – The output producing input tensor to be padded.

  • pads_begin – The output which specifies the number of padding elements added before position 0 on each axis of arg.

  • pads_end – The output which specifies the number of padding elements after the last element on each axis.

  • arg_pad_value – The scalar output with the value used for padding if pad_mode is CONSTANT

  • pad_mode – The padding mode: CONSTANT, EDGE, REFLECT or SYMMETRIC. CONSTANT initializes new elements with arg_pad_value, EDGE uses the nearest value from arg. REFLECT and SYMMETRIC tile the background by flipping arg at the edge (SYMMETRIC) or on the last row/column/etc. (REFLECT).

Pad(const Output<Node> &arg, const Output<Node> &pads_begin, const Output<Node> &pads_end, PadMode pad_mode)

Constructs a Pad-1 operation.

Parameters
  • arg – The output producing input tensor to be padded.

  • pads_begin – The output which specifies the number of padding elements added

  • pads_end – The output which specifies the number of padding elements after the last element on each axis.

  • pad_mode – The padding mode: CONSTANT, EDGE, REFLECT or SYMMETRIC.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

class Power : public ov::op::util::BinaryElementwiseArithmetic
#include <power.hpp>

Elementwise exponentiation operation.

Inputs

Type

Description

arg0

\(N[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape and numeric element type.

arg1

\(N[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of the same shape and element type as arg0.

Type

Description

\(N[d_1,\dots,d_n]\)

The tensor \(T\), where \(T[i_1,\dots,i_n] = \texttt{arg0}[i_1,\dots,i_n]^{\texttt{arg1}[i_1,\dots,i_n]}\)

Public Functions

Power(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs an exponentiation operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReduceLogicalAnd : public ov::op::util::LogicalReductionKeepDims
#include <reduce_logical_and.hpp>

Performs a reduction using “logical and”.

The reduction is performed over slices of the first input. The slices shape depends on the values passed to the second input - the axes.

Public Functions

ReduceLogicalAnd(const Output<Node> &data, const Output<Node> &reduction_axes, const bool keep_dims = false)

Constructs a ReduceLogicalAnd node.

Parameters
  • data – - The input tensor with data to be reduced

  • reduction_axes – - The input tensor with information about axes over which the first tensor should be sliced prior to the reduction operation

  • keep_dims – - Indicates if the axes used for reduction should be held/kept

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReduceLogicalOr : public ov::op::util::LogicalReductionKeepDims
#include <reduce_logical_or.hpp>

Performs a reduction using “logical or”.

The reduction is performed over slices of the first input. The slices shape depends on the values passed to the second input - the axes.

Public Functions

ReduceLogicalOr(const Output<Node> &data, const Output<Node> &reduction_axes, const bool keep_dims = false)

Constructs a ReduceLogicalOr node.

Parameters
  • data – - The input tensor with data to be reduced

  • reduction_axes – - The input tensor with information about axes over which the first tensor should be sliced prior to the reduction operation

  • keep_dims – - Indicates if the axes used for reduction should be held/kept

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReduceMax : public ov::op::util::ArithmeticReductionKeepDims
#include <reduce_max.hpp>

ReduceMax operation.

Public Functions

ReduceMax() = default

Constructs a summation operation.

ReduceMax(const Output<Node> &arg, const Output<Node> &reduction_axes, bool keep_dims = false)

Constructs a summation operation.

Parameters
  • arg – The tensor to be summed.

  • reduction_axes – The axis positions (0-based) to be eliminated.

  • keep_dims – If set to 1 it holds axes that are used for reduction.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReduceMean : public ov::op::util::ArithmeticReductionKeepDims
#include <reduce_mean.hpp>

ReduceMean operation.

Public Functions

ReduceMean(const Output<Node> &arg, const Output<Node> &reduction_axes, bool keep_dims = false)
Parameters
  • arg – The tensor to be summed.

  • reduction_axes – The axis positions (0-based) to be eliminated.

  • keep_dims – If set to 1 it holds axes that are used for reduction.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReduceMin : public ov::op::util::ArithmeticReductionKeepDims
#include <reduce_min.hpp>

ReduceMin operation.

Public Functions

ReduceMin() = default

Constructs a summation operation.

ReduceMin(const Output<Node> &arg, const Output<Node> &reduction_axes, bool keep_dims = false)

Constructs a summation operation.

Parameters
  • arg – The tensor to be summed.

  • reduction_axes – The axis positions (0-based) to be eliminated.

  • keep_dims – If set to 1 it holds axes that are used for reduction.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReduceProd : public ov::op::util::ArithmeticReductionKeepDims
#include <reduce_prod.hpp>

Product reduction operation.

Reduces the tensor, eliminating the specified reduction axes by taking the product.

Public Functions

ReduceProd() = default

Constructs a product reduction operation.

ReduceProd(const Output<Node> &arg, const Output<Node> &reduction_axes, bool keep_dims = false)

Constructs a product reduction operation.

Parameters
  • arg – The tensor to be reduced.

  • reduction_axes – The axis positions (0-based) to be eliminated.

  • keep_dims – If set to true it holds axes that are used for reduction.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReduceSum : public ov::op::util::ArithmeticReductionKeepDims
#include <reduce_sum.hpp>

Tensor sum operation.

Element-wise sums the input tensor, eliminating the specified reduction axes. For example:

\[\begin{split} \mathit{sum}\left(\{0\}, \left[ \begin{array}{ccc} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{array} \right]\right) = \left[ (1 + 3 + 5), (2 + 4 + 6) \right] = \left[ 9, 12 \right]~~~\text{(dimension 0 (rows) is eliminated)} \end{split}\]

\[\begin{split} \mathit{sum}\left(\{1\}, \left[ \begin{array}{ccc} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{array} \right]\right) = \left[ (1 + 2), (3 + 4), (5 + 6) \right] = \left[ 3, 7, 11 \right]~~~\text{(dimension 1 (columns) is eliminated)} \end{split}\]

\[\begin{split} \mathit{sum}\left(\{0,1\}, \left[ \begin{array}{ccc} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{array} \right]\right) = (1 + 2) + (3 + 4) + (5 + 6) = 21~~~\text{(both dimensions (rows and columns) are eliminated)} \end{split}\]

Parameters

Description

reduction_axes

The axes to eliminate through summation.

keep_dims

If set to 1 it holds axes that are used for reduction.

Inputs

Type

Description

arg

\(N[d_1,\dots,d_n]~(n \geq 0)\)

An input tensor of any shape and numeric element type.

Type

Description

\(N[\textit{delete}(A,d_1,\dots,d_n)]\)

The tensor \(T\), where \(T\) is the input tensor with the reduction_axes \(A\) eliminated by summation.

Public Functions

ReduceSum() = default

Constructs a summation operation.

ReduceSum(const Output<Node> &arg, const Output<Node> &reduction_axes, bool keep_dims = false)

Constructs a summation operation.

Parameters
  • arg – The tensor to be summed.

  • reduction_axes – The axis positions (0-based) to be eliminated.

  • keep_dims – If set to 1 it holds axes that are used for reduction.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Reshape : public ov::op::Op
#include <reshape.hpp>

Tensor dynamic reshape operation.

“Converts” an input tensor into a new shape with the same number of elements. This op does not touch the actual data. If needed, use Transpose for that purpose.

Public Functions

Reshape(const Output<Node> &arg, const Output<Node> &shape_pattern, bool special_zero)

Constructs a dynamic reshape operation. This operation does not perform transpose.

Parameters
  • arg – The tensor to be reshaped.

  • shape_pattern – The node that defines output shape shape_pattern. If the input shape is \((a_0,\dots,a_{k-1})\) then the output shape must be of the form \((b_0,\dots,b_{j-1})\) where \(\Pi(a_i) = \Pi(b_i)\). A value of -1 is allowed for at most one dimension, in which case the dimension size is inferred based on element count of input tensor.

  • special_zero – Treats zeros in shape_pattern as wildcard flags indicating a copy from input shape at the same index.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Reverse : public ov::op::Op
#include <reverse.hpp>

Reverse operation.

Public Functions

Reverse(const Output<Node> &data, const Output<Node> &reversed_axes, const std::string &mode)

Constructs a reverse operation.

Parameters
  • data – The input tensor, some of whose axes are to be reversed.

  • reversed_axes – The axes to reverse in a form of a set of indices or boolean mask.

  • mode – The way reversed_axes should be interpreted - a set or a mask.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline Mode get_mode() const
Returns

The second input data interpretation mode.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Select : public ov::op::Op
#include <select.hpp>

Elementwise selection operation.

Inputs

Type

Description

arg0

\(\texttt{bool}[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of any shape, with element bool.

arg1

\(E[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of a shape that is broadcast-compatible with arg0, with any element type.

arg2

\(E[d_1,\dots,d_n]~(n \geq 0)\)

A tensor of a shape that is broadcast-compatible with arg0, and same element type as arg1.

auto_broadcast

AutoBroadcastSpec

Auto broadcast specification.

Type

Description

\(E[d_1,\dots,d_n]\)

The tensor \(T\), where \(T[i_1,\dots,i_n] = \texttt{arg1}[i_1,\dots,i_n]\text{ if }\texttt{arg0}[i_1,\dots,i_n] \neq 0\text{, else }\texttt{arg2}[i_1,\dots,i_n]\)

Public Functions

inline Select()

Constructs a selection operation.

Select(const Output<Node> &arg0, const Output<Node> &arg1, const Output<Node> &arg2, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a selection operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • arg2Node that produces the third input tensor.

  • auto_broadcast – Auto broadcast specification. Default is Numpy-style implicit broadcasting.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual const AutoBroadcastSpec &get_autob() const override
Returns

the autobroadcasr spec

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Softmax : public ov::op::Op
#include <softmax.hpp>

Softmax operation.

Public Functions

Softmax(const Output<Node> &arg, const size_t axis = 1)

Constructs a softmax operation.

Output [d0, ...]

Parameters
  • argNode that produces the first input tensor.[d0, ...]

  • axis – The axis position (0-based) on which to calculate the softmax.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class SpaceToBatch : public ov::op::Op
#include <space_to_batch.hpp>

SpaceToBatch permutes data tensor blocks of spatial data into batch dimension.

Note

Values from spatial blocks dimensions are moved in the batch dimension.

   Output node produces a tensor with shape: tensor with shape
   `[batch * block_shape[0] * block_shape[1] * ... * block_shape[N - 1],
    (pads_begin[1] + D_1 + pads_end[1]) / block_shape[1],
    (pads_begin[2] + D_2 + pads_end[2]) / block_shape[2], ...,
    (pads_begin[N - 1] + D_{N - 1} + pads_end[N - 1]) / block_shape[N - 1]`
    of the same type as `data` input.

Public Functions

SpaceToBatch(const Output<Node> &data, const Output<Node> &block_shape, const Output<ov::Node> &pads_begin, const Output<ov::Node> &pads_end)

Constructs a SpaceToBatch operation.

Parameters
  • dataNode producing the data tensor

  • block_shape – The sizes of the block of values to be moved

  • pads_begin – Specifies the padding for the beginning along each axis of data input

  • pads_end – Specifies the padding for the ending along each axis of data input.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Split : public ov::op::Op
#include <split.hpp>

Splits the input tensor into a list of equal sized tensors.

Public Functions

Split() = default

Constructs a split operation.

Split(const Output<Node> &data, const Output<Node> &axis, const size_t num_splits)

Constructs a split operation.

Parameters
  • data – The tensor to be split.

  • axis – The index of an axis in “data” along which to perform the split.

  • num_splits – The number of pieces that the data tensor should be split into.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class StridedSlice : public ov::op::Op
#include <strided_slice.hpp>

Takes a slice of an input tensor, i.e., the sub-tensor that resides within a bounding box, optionally with stride.

Public Functions

StridedSlice(const Output<Node> &data, const Output<Node> &begin, const Output<Node> &end, const Output<Node> &strides, const std::vector<int64_t> &begin_mask, const std::vector<int64_t> &end_mask, const std::vector<int64_t> &new_axis_mask = std::vector<int64_t>{}, const std::vector<int64_t> &shrink_axis_mask = std::vector<int64_t>{}, const std::vector<int64_t> &ellipsis_mask = std::vector<int64_t>{})

Constructs a dynamic tensor strided slice operation.

Parameters
  • data – The tensor to be sliced.

  • begin – 1D tensor with begin indexes for input blob slicing.

  • end – 1D tensor with end indexes for input blob slicing.

  • strides – The slicing strides; for example, strides of {n,m} means to take every nth row and every mth column of the input matrix.

  • begin_mask – When begin_mask[i] equal to 1 means that the corresponding dimension of the begin input is ignored.

  • end_mask – When end_mask[i] is 1, the corresponding dimension of the end input is ignored.

  • new_axis_mask – If new_axis_mask[i] is 1, a length 1 dimension is inserted on the i-th position.

  • shrink_axis_mask – If shrink_axis_mask[i] is 1, the dimension on the i-th position is deleted.

  • ellipsis_mask – It inserts missing dimensions on a position of a non-zero bit.

StridedSlice(const Output<Node> &data, const Output<Node> &begin, const Output<Node> &end, const std::vector<int64_t> &begin_mask, const std::vector<int64_t> &end_mask, const std::vector<int64_t> &new_axis_mask = std::vector<int64_t>{}, const std::vector<int64_t> &shrink_axis_mask = std::vector<int64_t>{}, const std::vector<int64_t> &ellipsis_mask = std::vector<int64_t>{})

Constructs a dynamic tensor strided slice operation.

Parameters
  • data – The tensor to be sliced.

  • begin – 1D tensor with begin indexes for input blob slicing.

  • end – 1D tensor with end indexes for input blob slicing.

  • begin_mask – When begin_mask[i] equal to 1 means that the corresponding dimension of the begin input is ignored.

  • end_mask – When end_mask[i] is 1, the corresponding dimension of the end input is ignored.

  • new_axis_mask – If new_axis_mask[i] is 1, a length 1 dimension is inserted on the i-th position.

  • shrink_axis_mask – If shrink_axis_mask[i] is 1, the dimension on the i-th position is deleted.

  • ellipsis_mask – It inserts missing dimensions on a position of a non-zero bit.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Subtract : public ov::op::util::BinaryElementwiseArithmetic
#include <subtract.hpp>

Elementwise subtraction operation.

Public Functions

Subtract(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a subtraction operation.

Parameters
  • arg0Node that produces the first input tensor.

  • arg1Node that produces the second input tensor.

  • auto_broadcast – Auto broadcast specification

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class TopK : public ov::op::util::TopKBase
#include <topk.hpp>

Computes indices and values of the k maximum/minimum values for each slice along specified axis.

Public Functions

TopK() = default

Constructs a TopK operation.

TopK(const Output<Node> &data, const Output<Node> &k, const int64_t axis, const std::string &mode, const std::string &sort, const element::Type &index_element_type = element::i32)

Constructs a TopK operation with two outputs: values and indices. By default the indices output is described by i32 data type.

Parameters
  • data – The input tensor

  • k – Specifies how many maximum/minimum elements should be computed (note: scalar input tensor)

  • axis – The axis along which to compute top k indices

  • mode – Specifies which operation (min or max) is used to select the biggest element of two.

  • sort – Specifies order of output elements and/or indices Accepted values: none, index, value

  • index_element_type – Specifies type of produced indices

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Transpose : public ov::op::Op
#include <transpose.hpp>

Tensor transpose operation.

Public Types

enum Ins

Inputs indexes and count.

Values:

enumerator ARG
enumerator ORDER
enumerator IN_COUNT
enum Outs

Outputs indexes and count.

Values:

enumerator ARG_T
enumerator OUT_COUNT

Public Functions

Transpose(const Output<Node> &arg, const Output<Node> &input_order)

Constructs a transpose operation.

Parameters
  • argNode producing the tensor to be transposed.

  • input_orderNode producing the permutation to apply to the axes of the input shape. Must be a vector with shape [n], where n is the rank of arg. The tensor’s value must contain every integer in the range [0, n-1].

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class VariadicSplit : public ov::op::Op
#include <variadic_split.hpp>

VariadicSplit operation splits an input tensor into pieces along some axis. The pieces may have variadic lengths depending on “split_lengths” attribute.

Public Functions

VariadicSplit() = default

Constructs a variadic split operation.

VariadicSplit(const Output<Node> &data, const Output<Node> &axis, const Output<Node> &split_lengths)

Constructs a variadic split operation.

outputs. The sum of split_lengths must match data.shape[axis]

Parameters
  • data – The tensor to be split.

  • axis – The index of an axis in “data” along which to perform the split.

  • split_lengths – A list containing the sizes of each output tensor along the split “axis”. Size of “split_lengths” should be equal to the number of

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual size_t get_default_output_index() const override

Returns the output of the default output, or throws if there is none.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

namespace v10
class IsFinite : public ov::op::Op
#include <is_finite.hpp>

Boolean mask that maps NaN and Infinity values to false and other values to true.

Public Functions

IsFinite(const Output<Node> &data)

Constructs a IsFinite operation.

Parameters

dataInput data tensor

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class IsInf : public ov::op::Op
#include <is_inf.hpp>

Boolean mask that maps infinite values to true.

Public Functions

IsInf(const Output<Node> &data)

Constructs a IsInf operation.

Parameters

dataInput data tensor

IsInf(const Output<Node> &data, const Attributes &attributes)

Constructs a IsInf operation.

Parameters
  • dataInput data tensor

  • attrsIsInf attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

struct Attributes
#include <is_inf.hpp>

A Structure which contains all IsInf attributes.

class IsNaN : public ov::op::Op
#include <is_nan.hpp>

Boolean mask that maps NaN values to true and other values to false.

Public Functions

IsNaN(const Output<Node> &data)

Constructs a IsNaN operation.

Parameters

dataInput data tensor

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Unique : public ov::op::Op
#include <unique.hpp>

Operator which selects and returns unique elements or unique slices of the input tensor.

Public Functions

Unique(const Output<Node> &data, const bool sorted = true, const element::Type &index_element_type = element::i64, const element::Type &count_element_type = element::i64)

Constructs a Unique operation.

Parameters
  • dataInput data tensor

  • sorted – Controls the order of the returned unique values (sorts ascendingly when true)

  • index_element_type – The data type for outputs containing indices

  • count_element_type – The data type for output containing repetition count

Unique(const Output<Node> &data, const Output<Node> &axis, const bool sorted = true, const element::Type &index_element_type = element::i64, const element::Type &count_element_type = element::i64)

Constructs a Unique operation.

Parameters
  • dataInput data tensor

  • axis – An input tensor containing the axis value

  • sorted – Controls the order of the returned unique values (sorts ascendingly when true)

  • index_element_type – The data type for outputs containing indices

  • count_element_type – The data type for output containing repetition count

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

namespace v11

Functions

template<class TShape, class TContainer, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Interpolate *op, const std::vector<TShape> &input_shapes, TContainer &pads_begin, TContainer &pads_end, const ITensorAccessor &tensor_accessor)
class Interpolate : public ov::op::util::InterpolateBase
#include <interpolate.hpp>

Interpolate operation.

Public Functions

Interpolate(const Output<Node> &image, const Output<Node> &scales_or_sizes, const InterpolateAttrs &attrs)

Constructs a Interpolate operation without ‘axes’ input.

Parameters
  • imageInput image

  • scales_or_sizes – Scales of spatial axes, i.e. output_shape / input_shape

  • attrs – Interpolation attributes

Interpolate(const Output<Node> &image, const Output<Node> &scales_or_sizes, const Output<Node> &axes, const InterpolateAttrs &attrs)

Constructs a Interpolate operation with ‘axes’ input.

Parameters
  • imageInput image

  • scales_or_sizes – Scales of spatial axes, i.e. output_shape / input_shape

  • axes – Interpolation axes

  • attrs – Interpolation attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class TopK : public ov::op::util::TopKBase
#include <topk.hpp>

Computes the top K elements of a given tensor along the specified axis.

Public Functions

TopK() = default

Constructs a TopK operation.

TopK(const Output<Node> &data, const Output<Node> &k, const int64_t axis, const std::string &mode, const std::string &sort, const element::Type &index_element_type = element::i32, const bool stable = false)

Constructs a TopK operation with two outputs: values and indices.

Parameters
  • data – The input tensor

  • k – Specifies how many maximum/minimum elements should be computed

  • axis – The axis along which the TopK operation should be executed

  • mode – Specifies whether TopK selects the largest or the smallest elements from each slice

  • sort – Specifies the order of corresponding elements of the output tensor

  • index_element_type – Specifies the data type of the elements in the ‘indices’ output tensor.

  • stable – Specifies whether the equivalent elements should maintain their relative order from the input tensor during sorting.

TopK(const Output<Node> &data, const Output<Node> &k, const int64_t axis, const TopKMode mode, const TopKSortType sort, const element::Type &index_element_type = element::i32, const bool stable = false)

Constructs a TopK operation with two outputs: values and indices.

Parameters
  • data – The input tensor

  • k – Specifies how many maximum/minimum elements should be computed

  • axis – The axis along which the TopK operation should be executed

  • mode – Specifies whether TopK selects the largest or the smallest elements from each slice

  • sort – Specifies the order of corresponding elements of the output tensor

  • index_element_type – Specifies the data type of the elements in the ‘indices’ output tensor.

  • stable – Specifies whether the equivalent elements should maintain their relative order from the input tensor during sorting.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

namespace v12

Functions

template<class TShape>
std::vector<TShape> shape_infer(const GroupNormalization *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ScatterElementsUpdate *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
class GroupNormalization : public ov::op::Op
#include <group_normalization.hpp>

GroupNormalization operation over the input tensor.

Public Functions

GroupNormalization(const Output<Node> &data, const Output<Node> &scale, const Output<Node> &bias, int64_t num_groups, double epsilon)
Parameters
  • data – The input tensor to be normalized

  • scale – The tensor containing scale values for each channel

  • bias – The tensor containing bias values for each channel

  • num_groups – The number of groups that the channel dimension will be divided into

  • epsilon – The value that prevents divisions by zero in GroupNormalization formula

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Pad : public ov::op::util::PadBase
#include <pad.hpp>

Generic padding operation.

Public Functions

Pad() = default

Constructs a Pad-12 operation.

Pad(const Output<Node> &arg, const Output<Node> &pads_begin, const Output<Node> &pads_end, PadMode pad_mode)

Constructs a Pad-12 operation.

Parameters
  • arg – The output producing input tensor to be padded.

  • pads_begin – The output which specifies the number of padding elements to add (or remove) before position 0 on each axis of arg.

  • pads_end – The output which specifies the number of padding elements to add (or remove) after the last element on each axis.

  • pad_mode – The padding mode: CONSTANT, EDGE, REFLECT or SYMMETRIC.

Pad(const Output<Node> &arg, const Output<Node> &pads_begin, const Output<Node> &pads_end, const Output<Node> &arg_pad_value, PadMode pad_mode)

Constructs a Pad-12 operation.

Parameters
  • arg – The output producing input tensor to be padded.

  • pads_begin – The output which specifies the number of padding elements to add (or remove) before position 0 on each axis of arg.

  • pads_end – The output which specifies the number of padding elements to add (or remove) after the last element on each axis.

  • arg_pad_value – The scalar output with the value used for padding if pad_mode is CONSTANT

  • pad_mode – The padding mode: CONSTANT, EDGE, REFLECT or SYMMETRIC.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

class ScatterElementsUpdate : public ov::op::util::ScatterElementsUpdateBase
#include <scatter_elements_update.hpp>

Public Types

enum class Reduction

Lists the supported reduction types for this version of the operator. See the specification for the description of how reduction works with ScatterElementsUpdate.

Values:

enumerator NONE
enumerator SUM
enumerator PROD
enumerator MIN
enumerator MAX
enumerator MEAN

Public Functions

ScatterElementsUpdate(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &updates, const Output<Node> &axis, const Reduction reduction = Reduction::NONE, const bool use_init_val = true)

Constructs a ScatterElementsUpdate node.

Parameters
  • dataInput data

  • indices – Data entry index that will be updated

  • updates – Update values

  • axis – Axis to scatter on

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

namespace v13

Functions

template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const FakeConvert *op, const std::vector<T> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Multinomial *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const NMSRotated *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const ScaledDotProductAttention *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
class BitwiseAnd : public ov::op::util::BinaryElementwiseBitwise
#include <bitwise_and.hpp>

Elementwise bitwise AND operation.

Public Functions

BitwiseAnd() = default

Constructs a bitwise AND operation.

BitwiseAnd(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a bitwise AND operation.

Output [d0, ...]

Parameters
  • arg0Output that produces the first input tensor.[d0, ...]

  • arg1Output that produces the second input tensor.[d0, ...]

  • auto_broadcast – Auto broadcast specification. Default is Numpy-style implicit broadcasting.

class BitwiseNot : public ov::op::Op
#include <bitwise_not.hpp>

Elementwise bitwise negation operation.

Public Functions

BitwiseNot() = default

Constructs a bitwise negation operation.

BitwiseNot(const Output<Node> &arg)

Constructs a bitwise negation operation.

Parameters

argNode that produces the input tensor.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class BitwiseOr : public ov::op::util::BinaryElementwiseBitwise
#include <bitwise_or.hpp>

Elementwise bitwise OR operation.

Public Functions

BitwiseOr() = default

Constructs a bitwise OR operation.

BitwiseOr(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a bitwise OR operation.

Output [d0, ...]

Parameters
  • arg0Output that produces the first input tensor.[d0, ...]

  • arg1Output that produces the second input tensor.[d0, ...]

  • auto_broadcast – Auto broadcast specification. Default is Numpy-style implicit broadcasting.

class BitwiseXor : public ov::op::util::BinaryElementwiseBitwise
#include <bitwise_xor.hpp>

Elementwise bitwise XOR operation.

Public Functions

BitwiseXor() = default

Constructs a bitwise XOR operation.

BitwiseXor(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))

Constructs a bitwise XOR operation.

Output [d0, ...]

Parameters
  • arg0Output that produces the first input tensor.[d0, ...]

  • arg1Output that produces the second input tensor.[d0, ...]

  • auto_broadcast – Auto broadcast specification. Default is Numpy-style implicit broadcasting.

class FakeConvert : public ov::op::Op
#include <fake_convert.hpp>

FakeConvert performs element-wise quantization of input values into a set of values corresponding to a target low-precision type.

Note

FakeConvert is an experimental operation and subject to change.

Public Functions

FakeConvert(const ov::Output<ov::Node> &data, const ov::Output<ov::Node> &scale, std::string destination_type = "f8e4m3")

Constructs FakeConvert operation (default shift).

Parameters
  • data – The input data tensor.

  • scaleTensor with a scale factor for the data input.

  • destination_type – The low precision type to be emulated.

FakeConvert(const ov::Output<ov::Node> &data, const ov::Output<ov::Node> &scale, const ov::Output<ov::Node> &shift, std::string destination_type = "f8e4m3")

Constructs FakeConvert operation.

Parameters
  • data – The input data tensor.

  • scaleTensor with a scale factor for the data input.

  • shiftTensor with a shift factor for the data input.

  • destination_type – The low precision type to be emulated.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Multinomial : public ov::op::Op
#include <multinomial.hpp>

Multinomial operation creates a sequence of indices of classes sampled from the multinomial distribution.

Public Functions

Multinomial(const Output<Node> &input, const Output<Node> &num_samples, const ov::element::Type_t convert_type, const bool with_replacement, const bool log_probs, const uint64_t global_seed = 0, const uint64_t op_seed = 0)

Multinomial operation creates a sequence of indices of classes sampled from the multinomial distribution.

Parameters
  • probsInput tensor containing at each index poisition probability/log probability of sampling a given class. Any floating-point precision values are allowed.

  • num_samples – Scalar or 1D tensor with a single value that determines the number of samples to generate per batch. Values should be of an integer type.

  • convert_type – Data type to which to convert the output class indices. Allowed values: i32/i64

  • with_replacement – Boolean that determines whether a sampled class can appear more than once in the output.

  • log_probs – Boolean that determines whether to treat input probabilities as log probabilities.

  • global_seed – First seed value (key) of Phillox random number generation algorithm. (See RandomUniform for details)

  • op_seed – Second seed value (counter) of Phillox random number generation algorithm. (See RandomUniform for details)

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class NMSRotated : public ov::op::Op
#include <nms_rotated.hpp>

NMSRotated operation.

Public Functions

NMSRotated(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const Output<Node> &iou_threshold, const Output<Node> &score_threshold, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64, const bool clockwise = true)

Constructs a NMSRotated operation.

Parameters
  • boxesNode containing the coordinates of the bounding boxes

  • scoresNode containing the scores of the bounding boxes

  • max_output_boxes_per_classNode containing maximum number of boxes to be selected per class

  • iou_thresholdNode containing intersection over union threshold

  • score_thresholdNode containing minimum score threshold

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output type of the first and third output

  • clockwise – Specifies the direction of the rotation

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ScaledDotProductAttention : public ov::op::Op
#include <scaled_dot_product_attention.hpp>

Scaled dot product attention operation from PyTorch.

Public Functions

ScaledDotProductAttention() = default

Constructs a ScaledDotProductAttention operation.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

namespace v3

Functions

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Assign *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const ov::op::v3::Broadcast *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Bucketize *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const EmbeddingSegmentsSum *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const ExtractImagePatches *op, const std::vector<T> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const GRUCell *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const NonMaxSuppression *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ROIAlign *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ShapeOf *op, const std::vector<TShape> &input_shapes)
class Acosh : public ov::op::util::UnaryElementwiseArithmetic
#include <acosh.hpp>

Elementwise inverse hyperbolic cos operation.

Public Functions

Acosh() = default

Constructs an Acosh operation.

Acosh(const Output<Node> &arg)

Constructs an Acosh operation.

Output [d1, ...]

Parameters

argOutput that produces the input tensor.[d1, ...]

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Asinh : public ov::op::util::UnaryElementwiseArithmetic
#include <asinh.hpp>

Elementwise inverse hyperbolic sin operation.

Public Functions

Asinh() = default

Constructs an Asinh operation.

Asinh(const Output<Node> &arg)

Constructs an Asinh operation.

Output [d1, ...]

Parameters

argOutput that produces the input tensor.[d1, ...]

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Assign : public ov::op::util::AssignBase
#include <assign.hpp>

Assign operation sets an input value to the variable with variable_id

Public Functions

Assign(const Output<Node> &new_value, const std::string &variable_id)

Constructs an Assign operation.

Parameters
  • new_valueNode that produces the input tensor.

  • variable_id – identifier of the variable to be updated.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual std::string get_variable_id() const override

Returns the identifier of corresponding variable.

class Atanh : public ov::op::util::UnaryElementwiseArithmetic
#include <atanh.hpp>

Elementwise inverse hyperbolic tangent operation.

Public Functions

Atanh() = default

Constructs an Atanh operation.

Atanh(const Output<Node> &arg)

Constructs an Atanh operation.

Output [d1, ...]

Parameters

argOutput that produces the input tensor.[d1, ...]

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Broadcast : public ov::op::util::BroadcastBase
#include <broadcast.hpp>

Operation which “adds” axes to an input tensor, replicating elements from the input as needed along the new axes.

Public Functions

Broadcast() = default

Constructs a broadcast operation.

Broadcast(const Output<Node> &arg, const Output<Node> &target_shape, const Output<Node> &axes_mapping, const BroadcastModeSpec &broadcast_spec = BroadcastType::EXPLICIT)

Constructs a broadcast operation.

Parameters
  • arg – The input tensor to be broadcast.

  • target_shape – The shape of the output tensor.

  • axes_mapping – The axis positions (0-based) in the result that correspond to input axes. ‘Arg’ tensor is broadcast along the remaining axes. E.g., Input Shape - [3, 4], Target Shape - [3, 5, 4, 4] axes_mapping - [0, 2] => Broadcast along axes 1 and 3. axes_mapping - [0, 3] => Broadcast along axes 1 and 2.

  • broadcast_specBroadcast specification to use for determining broadcast axes. ‘axes_mapping’ should not be provided if mode other than explicit (none) is used.

Broadcast(const Output<Node> &arg, const Output<Node> &target_shape, const BroadcastModeSpec &broadcast_spec = BroadcastType::NUMPY)

Constructs a broadcast operation.

Parameters
  • arg – The input tensor to be broadcast.

  • target_shape – The shape of the output tensor.

  • broadcast_specBroadcast specification to use for determining broadcast axes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual std::pair<bool, AxisSet> get_broadcast_axes() const override
Returns

true and the AxisSet if broadcast axes can be fully determined.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Bucketize : public ov::op::Op
#include <bucketize.hpp>

Operation that bucketizes the input based on boundaries.

Public Functions

Bucketize(const Output<Node> &data, const Output<Node> &buckets, const element::Type output_type = element::i64, const bool with_right_bound = true)

Constructs a Bucketize node.

Parameters
  • dataInput data to bucketize

  • buckets – 1-D of sorted unique boundaries for buckets

  • output_typeOutput tensor type, “i64” or “i32”, defaults to i64

  • with_right_bound – indicates whether bucket includes the right or left edge of interval. default true = includes right edge

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class EmbeddingBagOffsetsSum : public ov::op::util::EmbeddingBagOffsetsBase
#include <embeddingbag_offsets_sum.hpp>

Returns embeddings for given indices.

Public Functions

EmbeddingBagOffsetsSum() = default

Constructs a EmbeddingBagOffsetsSum operation.

EmbeddingBagOffsetsSum(const Output<Node> &emb_table, const Output<Node> &indices, const Output<Node> &offsets, const Output<Node> &default_index, const Output<Node> &per_sample_weights)

Constructs a EmbeddingBagOffsetsSum operation.

EmbeddingBagOffsetsSum constructs an output tensor by replacing every index in a given input tensor with a row (from the weights matrix) at that index

Parameters
  • emb_table – tensor containing the embedding lookup table of the module of shape [num_emb, emb_dim1, emb_dim2, …] and of type T

  • indices – tensor of shape [num_indices] and of type T_IND. Required

  • offsets – tensor of shape [batch] and of type T_IND containing the starting index positions of each “bag” in indices. Required.

  • default_index – scalar of type T_IND containing default index in embedding table to fill empty “bags”. If not provided empty “bags” are filled with zeros. Optional.

  • per_sample_weigths – tensor of the same shape as indices and of type T. Each value in this tensor are multiplied with each value pooled from embedding table for each index. Optional.

class EmbeddingBagPackedSum : public ov::op::util::EmbeddingBagPackedBase
#include <embeddingbag_packedsum.hpp>

Returns embeddings for given indices.

Public Functions

EmbeddingBagPackedSum() = default

Constructs a EmbeddingBagPackedSum operation.

EmbeddingBagPackedSum(const Output<Node> &emb_table, const Output<Node> &indices, const Output<Node> &per_sample_weights)

Constructs a EmbeddingBagPackedSum operation.

EmbeddingBagPackedSum constructs an output tensor by replacing every index in a given input tensor with a row (from the weights matrix) at that index

Parameters
  • emb_tableTensor containing the embedding lookup table of the module of shape [num_emb, emb_dim1, emb_dim2, …] and of type T

  • indicesTensor of shape [batch, indices_per_bag] and of type T_IND. Required.

  • per_sample_weigths – tensor of the same shape as indices and of type T. Each value in this tensor are multiplied with each value pooled from embedding table for each index. Optional.

class EmbeddingSegmentsSum : public ov::op::Op
#include <embedding_segments_sum.hpp>

Returns embeddings for given indices.

Public Functions

EmbeddingSegmentsSum() = default

Constructs a EmbeddingSegmentsSum operation.

EmbeddingSegmentsSum(const Output<Node> &emb_table, const Output<Node> &indices, const Output<Node> &segment_ids, const Output<Node> &num_segments, const Output<Node> &default_index, const Output<Node> &per_sample_weights)

Constructs a EmbeddingSegmentsSum operation.

EmbeddingSegmentsSum constructs an output tensor by replacing every index in a given input tensor with a row (from the weights matrix) at that index

Parameters
  • 'emb_table' – tensor containing the embedding lookup table of the module of shape [num_emb, emb_dim1, emb_dim2, …] and of type T

  • 'indices' – tensor of shape [num_indices] and of type T_IND. Required

  • <tt>segment_ids</tt> – tensor of shape [num_indices] and of type T_IND with indices into the output Tensor. Values should be sorted and can be repeated. Required.

  • <tt>num_segments</tt> – scalar of type T_IND indicating the number of segments. Required.

  • 'default_index' – scalar of type T_IND containing default index in embedding table to fill empty “bags”. If not provided empty “bags” are filled with zeros. Optional.

  • 'per_sample_weights' – tensor of the same shape as indices and of type T. Each value in this tensor are multiplied with each value pooled from embedding table for each index. Optional.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ExtractImagePatches : public ov::op::Op
#include <extractimagepatches.hpp>

ExtractImagePatches operation.

Public Functions

ExtractImagePatches(const Output<Node> &image, const Shape &sizes, const Strides &strides, const Shape &rates, const PadType &auto_pad)

Constructs a ExtractImagePatches operation.

Parameters
  • data – 4-D Input data to extract image patches

  • sizes – Patch size in the format of [size_rows, size_cols]

  • strides – Patch movement stride in the format of [stride_rows, stride_cols]

  • rates – Element seleciton rate for creating a patch. in the format of [rate_rows, rate_cols]

  • auto_pad – Padding type. it can be any value from valid, same_lower, same_upper

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class GRUCell : public ov::op::util::RNNCellBase
#include <gru_cell.hpp>

Class for GRU cell node.

Note

Note this class represents only single cell and not whole GRU layer.

Public Functions

GRUCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &W, const Output<Node> &R, std::size_t hidden_size)

Constructs GRUCell node.

Parameters
  • X[in] The input tensor with shape: [batch_size, input_size].

  • initial_hidden_state[in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].

  • W[in] The weight tensor with shape: [gates_count * hidden_size, input_size].

  • R[in] The recurrence weight tensor with shape: [gates_count * hidden_size, hidden_size].

  • hidden_size[in] The number of hidden units for recurrent cell.

GRUCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &W, const Output<Node> &R, std::size_t hidden_size, const std::vector<std::string> &activations, const std::vector<float> &activations_alpha, const std::vector<float> &activations_beta, float clip, bool linear_before_reset)

Constructs GRUCell node.

Parameters
  • X[in] The input tensor with shape: [batch_size, input_size].

  • initial_hidden_state[in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].

  • W[in] The weight tensor with shape: [gates_count * hidden_size, input_size].

  • R[in] The recurrence weight tensor with shape: [gates_count * hidden_size, hidden_size].

  • hidden_size[in] The number of hidden units for recurrent cell.

  • activations[in] The vector of activation functions used inside recurrent cell.

  • activations_alpha[in] The vector of alpha parameters for activation functions in order respective to activation list.

  • activations_beta[in] The vector of beta parameters for activation functions in order respective to activation list.

  • clip[in] The value defining clipping range [-clip, clip] on input of activation functions.

GRUCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &W, const Output<Node> &R, const Output<Node> &B, std::size_t hidden_size, const std::vector<std::string> &activations = std::vector<std::string>{"sigmoid", "tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f, bool linear_before_reset = false)

Constructs GRUCell node.

Parameters
  • X[in] The input tensor with shape: [batch_size, input_size].

  • initial_hidden_state[in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].

  • W[in] The weight tensor with shape: [gates_count * hidden_size, input_size].

  • R[in] The recurrence weight tensor with shape: [gates_count * hidden_size, hidden_size].

  • hidden_size[in] The number of hidden units for recurrent cell.

  • B[in] The sum of biases (weight and recurrence) for update, reset and hidden gates. If linear_before_reset := true then biases for hidden gates are placed separately (weight and recurrence). Shape: [gates_count * hidden_size] if linear_before_reset := false Shape: [(gates_count + 1) * hidden_size] if linear_before_reset := true

  • activations[in] The vector of activation functions used inside recurrent cell.

  • activations_alpha[in] The vector of alpha parameters for activation functions in order respective to activation list.

  • activations_beta[in] The vector of beta parameters for activation functions in order respective to activation list.

  • clip[in] The value defining clipping range [-clip, clip] on input of activation functions.

  • linear_before_reset[in] Whether or not to apply the linear transformation before multiplying by the output of the reset gate.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class NonMaxSuppression : public ov::op::Op
#include <non_max_suppression.hpp>

NonMaxSuppression operation.

Subclassed by ov::op::v4::NonMaxSuppression

Public Functions

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const Output<Node> &iou_threshold, const Output<Node> &score_threshold, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • iou_thresholdNode producing intersection over union threshold

  • score_thresholdNode producing minimum score threshold

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation with default values for the last 3 inputs.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box coordinates

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class NonZero : public ov::op::Op
#include <non_zero.hpp>

NonZero operation returning indices of non-zero elements in the input tensor.

Note

The indices are returned by-dimension in row-major order. For example the following output contains 3 indices of a 3D input tensor elements: [[0, 0, 2], [0, 1, 1], [0, 1, 2]] The values point to input elements at [0,0,0], [0,1,1] and [2,1,2]

Public Functions

NonZero() = default

Constructs a NonZero operation.

NonZero(const Output<Node> &arg)

Constructs a NonZero operation.

Note

The output type is int64.

Parameters

argNode that produces the input tensor.

NonZero(const Output<Node> &arg, const std::string &output_type)

Constructs a NonZero operation.

Parameters
  • argNode that produces the input tensor.

  • output_type – produce indices. Currently, only ‘int64’ or ‘int32’ are supported

NonZero(const Output<Node> &arg, const element::Type &output_type)

Constructs a NonZero operation.

Parameters
  • argNode that produces the input tensor.

  • output_type – produce indices. Currently, only int64 or int32 are supported

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReadValue : public ov::op::util::ReadValueBase
#include <read_value.hpp>

ReadValue operation creates the variable with variable_id and returns value of this variable.

Public Functions

ReadValue(const Output<Node> &init_value, const std::string &variable_id)

Constructs a ReadValue operation.

Parameters
  • init_valueNode that produces the input tensor.

  • variable_id – identificator of the variable to create.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual std::string get_variable_id() const override

Returns the identifier of corresponding variable.

class ROIAlign : public ov::op::Op
#include <roi_align.hpp>

ROIAlign operation.

Public Functions

ROIAlign(const Output<Node> &input, const Output<Node> &rois, const Output<Node> &batch_indices, const int pooled_h, const int pooled_w, const int sampling_ratio, const float spatial_scale, const std::string &mode)

Constructs a ROIAlign node matching the ONNX ROIAlign specification.

Parameters
  • inputInput feature map {N, C, H, W}

  • rois – Regions of interest to pool over

  • batch_indices – Indices of images in the batch matching the number or ROIs

  • pooled_h – Height of the ROI output features

  • pooled_w – Width of the ROI output features

  • sampling_ratio – Number of sampling points used to compute an output element

  • spatial_scale – Spatial scale factor used to translate ROI coordinates

  • mode – Method of pooling - ‘avg’ or ‘max’

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ScatterElementsUpdate : public ov::op::util::ScatterElementsUpdateBase
#include <scatter_elements_update.hpp>

ScatterElementsUpdate operation.

Public Functions

ScatterElementsUpdate(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &updates, const Output<Node> &axis)

Constructs a ScatterElementsUpdate node.

Parameters
  • dataInput data

  • indices – Data entry index that will be updated

  • updates – Update values

  • axis – Axis to scatter on

class ScatterNDUpdate : public ov::op::util::ScatterNDBase
#include <scatter_nd_update.hpp>

Add updates to slices from inputs addressed by indices.

Public Functions

inline ScatterNDUpdate(const Output<Node> &inputs, const Output<Node> &indices, const Output<Node> &updates)
Parameters
virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ScatterUpdate : public ov::op::util::ScatterBase
#include <scatter_update.hpp>

Set new values to slices from data addressed by indices.

Public Functions

ScatterUpdate(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &updates, const Output<Node> &axis)

Constructs ScatterUpdate operator object.

Parameters
  • data – The input tensor to be updated.

  • indices – The tensor with indexes which will be updated.

  • updates – The tensor with update values.

  • axis[in] The axis at which elements will be updated.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ShapeOf : public ov::op::util::ShapeOfBase
#include <shape_of.hpp>

Operation that returns the shape of its input argument as a tensor.

Public Functions

ShapeOf(const Output<Node> &arg, const element::Type output_type = element::i64)

Constructs a shape-of operation.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class TopK : public ov::op::util::TopKBase
#include <topk.hpp>

Computes indices and values of the k maximum/minimum values for each slice along specified axis.

Public Functions

TopK() = default

Constructs a TopK operation.

TopK(const Output<Node> &data, const Output<Node> &k, const int64_t axis, const std::string &mode, const std::string &sort, const element::Type &index_element_type = element::i32)

Constructs a TopK operation with two outputs: values and indices. By default the indices output is described by i32 data type.

Parameters
  • data – The input tensor

  • k – Specifies how many maximum/minimum elements should be computed (note: scalar input tensor)

  • axis – The axis along which to compute top k indices

  • mode – Specifies which operation (min or max) is used to select the biggest element of two.

  • sort – Specifies order of output elements and/or indices Accepted values: none, index, value

  • index_element_type – Specifies type of produced indices

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

namespace v4

Functions

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const CTCLoss *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TContainer, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Interpolate *op, const std::vector<TShape> &input_shapes, TContainer &pads_begin, TContainer &pads_end, const ITensorAccessor &tensor_accessor)
template<class TShape>
std::vector<result_shape_t<TShape>> shape_infer(const LSTMCell *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const NonMaxSuppression *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Proposal *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Range *op, const std::vector<T> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
class CTCLoss : public ov::op::Op
#include <ctc_loss.hpp>

CTCLoss operation.

Public Functions

CTCLoss(const Output<Node> &logits, const Output<Node> &logit_length, const Output<Node> &labels, const Output<Node> &label_length, const bool preprocess_collapse_repeated = false, const bool ctc_merge_repeated = true, const bool unique = false)

Constructs a CTCLoss operation.

Parameters
  • logits – 3-D tensor of logits

  • logit_length – 1-D tensor of length for each object from a batch

  • labels – 2-D tensor of labels for which likelyhood is estimated using logist

  • label_length – 1-D tensor of length for each label sequence

  • blank_index – Scalar used to mark a blank index

  • preprocess_collapse_repeated – Flag for preprocessing labels before loss calculation

  • ctc_merge_repeated – Flag for merging repeated characters in a potential alignment

  • unique – Flag to find unique elements in a target before matching with alignment

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class HSwish : public ov::op::util::UnaryElementwiseArithmetic
#include <hswish.hpp>

A HSwish Activation Function f(x) = x * min(max(x + 3, 0), 6) / 6 or f(x) = x * min(ReLU(x + 3), 6) / 6.

Public Functions

HSwish(const Output<Node> &arg)

Constructs a HSwish (hard version of Swish) operation.

Parameters

dataInput tensor

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Interpolate : public ov::op::util::InterpolateBase
#include <interpolate.hpp>

Interpolate operation.

Public Functions

Interpolate(const Output<Node> &image, const Output<Node> &output_shape, const Output<Node> &scales, const InterpolateAttrs &attrs)

Constructs a Interpolate operation without ‘axes’ input.

Parameters
  • imageInput image

  • output_shapeOutput shape of spatial axes

  • scales – Scales of spatial axes, i.e. output_shape / input_shape

  • attrs – Interpolation attributes

Interpolate(const Output<Node> &image, const Output<Node> &output_shape, const Output<Node> &scales, const Output<Node> &axes, const InterpolateAttrs &attrs)

Constructs a Interpolate operation with ‘axes’ input.

Parameters
  • imageInput image

  • output_shapeOutput shape of spatial axes

  • scales – Scales of spatial axes, i.e. output_shape / input_shape

  • axes – Interpolation axes

  • attrs – Interpolation attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class LSTMCell : public ov::op::util::RNNCellBase
#include <lstm_cell.hpp>

Class for single lstm cell node.

See also

LSTMSequence, RNNCell, GRUCell

Note

Following implementation supports:

Note

It calculates following equations:

        it = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Wbi + Rbi)
        ft = f(Xt*(Wf^T) + Ht-1*(Rf^T)  + Wbf + Rbf)
        ct = g(Xt*(Wc^T) + Ht-1*(Rc^T) + Wbc + Rbc)
        Ct = ft (.) Ct-1 + it (.) ct
        ot = f(Xt*(Wo^T) + Ht-1*(Ro^T) + Wbo + Rbo)
        Ht = ot (.) h(Ct)

        *       - Is a dot product,
        (.)     - is a Hadamard product (element-wise),
        f, g, h - are activation functions.

Note

This class represents only single cell (for current time step) and not the whole LSTM Sequence layer

Public Functions

LSTMCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &initial_cell_state, const Output<Node> &W, const Output<Node> &R, std::size_t hidden_size, const std::vector<std::string> &activations = std::vector<std::string>{"sigmoid", "tanh", "tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f)

Constructs LSTMCell node.

Parameters
  • X[in] The input tensor with shape: [batch_size, input_size].

  • initial_hidden_state[in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].

  • initial_cell_state[in] The cell state tensor at current time step with shape: [batch_size, hidden_size].

  • W[in] The gate weights tensor with shape: [4*hidden_size, input_size].

  • R[in] The recurrence weights tensor with shape: [4*hidden_size, hidden_size].

  • hidden_size[in] The number of hidden units for recurrent cell.

  • activations[in] The vector of activation functions used inside recurrent cell.

  • activations_alpha[in] The vector of alpha parameters for activation functions in order respective to activation list.

  • activations_beta[in] The vector of beta parameters for activation functions in order respective to activation list.

  • clip[in] The value defining clipping range [-clip, clip] on input of activation functions.

LSTMCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &initial_cell_state, const Output<Node> &W, const Output<Node> &R, const Output<Node> &B, std::size_t hidden_size, const std::vector<std::string> &activations = std::vector<std::string>{"sigmoid", "tanh", "tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f)

Constructs LSTMCell node.

Parameters
  • X[in] The input tensor with shape: [batch_size, input_size].

  • initial_hidden_state[in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].

  • initial_cell_state[in] The cell state tensor at current time step with shape: [batch_size, hidden_size].

  • W[in] The weight tensor with shape: [4*hidden_size, input_size].

  • R[in] The recurrence weight tensor with shape: [4*hidden_size, hidden_size].

  • B[in] The bias tensor for gates with shape: [4*hidden_size].

  • hidden_size[in] The number of hidden units for recurrent cell.

  • activations[in] The vector of activation functions used inside recurrent cell.

  • activations_alpha[in] The vector of alpha parameters for activation functions in order respective to activation list.

  • activations_beta[in] The vector of beta parameters for activation functions in order respective to activation list.

  • clip[in] The value defining clipping range [-clip, clip] on input of activation functions.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Mish : public ov::op::util::UnaryElementwiseArithmetic
#include <mish.hpp>

A Self Regularized Non-Monotonic Neural Activation Function f(x) = x * tanh(log(exp(x) + 1.))

Public Functions

Mish(const Output<Node> &arg)

Constructs an Mish operation.

Parameters

dataInput tensor

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class NonMaxSuppression : public ov::op::v3::NonMaxSuppression
#include <non_max_suppression.hpp>

NonMaxSuppression operation.

Public Functions

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const Output<Node> &iou_threshold, const Output<Node> &score_threshold, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • iou_thresholdNode producing intersection over union threshold

  • score_thresholdNode producing minimum score threshold

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation with default values for the last 3 inputs.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box coordinates

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Proposal : public ov::op::v0::Proposal
#include <proposal.hpp>

Proposal operation.

Public Functions

Proposal(const Output<Node> &class_probs, const Output<Node> &bbox_deltas, const Output<Node> &image_shape, const Attributes &attrs)

Constructs a Proposal operation.

Parameters
  • class_probs – Class probability scores

  • bbox_deltas – Prediction of bounding box deltas

  • image_shapeShape of image

  • attrsProposal op attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Range : public ov::op::Op
#include <range.hpp>

Range operation, analogous to arange() in Numpy.

Public Functions

Range() = default

Constructs an unitialized range operation.

Range(const Output<Node> &start, const Output<Node> &stop, const Output<Node> &step, element::Type output_type)

Constructs a range operation.

Parameters
  • start – The tensor producing the start value. Must be a scalar of numeric element type.

  • stop – The tensor producing the stop value. Must be a scalar of numeric element type.

  • step – The tensor producing the step value. Must be a scalar of numeric element type.

  • output_type – The type of the output.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReduceL1 : public ov::op::util::ArithmeticReductionKeepDims
#include <reduce_l1.hpp>

Reduction operation using L1 norm: L1(x) = sum(abs(x)) if all dimensions are specified for the normalisation.

Reduces the tensor, eliminating the specified reduction axes by taking the L1-norm.

Public Functions

ReduceL1() = default

Constructs a reducet L1-norm operation.

ReduceL1(const Output<Node> &arg, const Output<Node> &reduction_axes, bool keep_dims = false)

Constructs a reduce L1-norm operation.

Parameters
  • arg – The tensor to be reduced.

  • reduction_axes – The axis positions (0-based) to be eliminated.

  • keep_dims – If set to true it holds axes that are used for reduction.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReduceL2 : public ov::op::util::ArithmeticReductionKeepDims
#include <reduce_l2.hpp>

Reduction operation using L2 norm:

Reduces the tensor, eliminating the specified reduction axes by taking the L2-norm.

Public Functions

ReduceL2() = default

Constructs a reducet L2-norm operation.

ReduceL2(const Output<Node> &arg, const Output<Node> &reduction_axes, bool keep_dims = false)

Constructs a reduce L2-norm operation.

Parameters
  • arg – The tensor to be reduced.

  • reduction_axes – The axis positions (0-based) to be eliminated.

  • keep_dims – If set to true it holds axes that are used for reduction.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class SoftPlus : public ov::op::util::UnaryElementwiseArithmetic
#include <softplus.hpp>

A Self Regularized Non-Monotonic Neural Activation Function f(x) = ln(exp(x) + 1.)

Public Functions

SoftPlus(const Output<Node> &arg)

Constructs an SoftPlus operation.

Parameters

dataInput tensor

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Swish : public ov::op::Op
#include <swish.hpp>

A Swish Activation Function f(x) = x / (1.0 + exp(-beta * x)) or f(x) = x * sigmoid(beta * x)

Public Functions

Swish(const Output<Node> &arg, const Output<Node> &beta)

Constructs an Swish operation.

Parameters
  • dataInput tensor

  • beta – Scalar with beta value. If the argument is not specified then use the default value 1.0

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

namespace ctc_loss

Variables

constexpr auto shape_names = std::array<const char*, 5>{"logits", "logit length", "labels", "label length", "blank index"}
constexpr auto shape_ranks = std::array<int64_t, 4>{3, 1, 2, 1}
namespace lstm_cell

Variables

constexpr size_t gates_count = 4
namespace v5

Functions

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const GatherND *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ov::op::v5::GRUSequence *op, const std::vector<TShape> &input_shapes)
template<class TShape>
std::vector<result_shape_t<TShape>> shape_infer(const LSTMSequence *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const NonMaxSuppression *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor(), const bool static_output = !std::is_same<T, PartialShape>::value)
template<class TShape>
std::vector<result_shape_t<TShape>> shape_infer(const RNNSequence *op, const std::vector<TShape> &input_shapes)
class BatchNormInference : public ov::op::Op
#include <batch_norm.hpp>

BatchNormInference operation.

Public Functions

BatchNormInference(const Output<Node> &input, const Output<Node> &gamma, const Output<Node> &beta, const Output<Node> &mean, const Output<Node> &variance, double epsilon)
Parameters
  • input – [., C, …]

  • gamma – gamma scaling for normalized value. [C]

  • beta – bias added to the scaled normalized value [C]

  • mean – value for mean normalization [C]

  • variance – value for variance normalization [C]

  • epsilon – Avoids divsion by 0 if input has 0 variance

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class GatherND : public ov::op::util::GatherNDBase
#include <gather_nd.hpp>

GatherND operation.

Public Functions

GatherND(const Output<Node> &data, const Output<Node> &indices, const size_t batch_dims = 0)

Constructs a GatherND operation.

Parameters
  • dataNode producing data that are gathered

  • indicesNode producing indices by which the operation gathers elements or slices from data

  • batch_dims – Specifies a number of batch dimensions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class GRUSequence : public ov::op::util::RNNCellBase
#include <gru_sequence.hpp>

GRUSequence operation.

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class HSigmoid : public ov::op::util::UnaryElementwiseArithmetic
#include <hsigmoid.hpp>

A HSigmoid Activation Function f(x) = min(max(x + 3, 0), 6) / 6 or f(x) = min(ReLU(x + 3), 6) / 6.

Public Functions

HSigmoid(const Output<Node> &arg)

Constructs a HSigmoid operation.

Parameters

dataInput tensor

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class LogSoftmax : public ov::op::Op
#include <log_softmax.hpp>

LogSoftmax operation.

Public Functions

LogSoftmax(const Output<Node> &arg, const int64_t axis)

Constructs a LogSoftmax operation.

Output [d0, ...]

Parameters
  • argNode that produces the first input tensor.[d0, ...]

  • axis – The axis position (0-based) on which to calculate the LogSoftmax.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Loop : public ov::op::util::SubGraphOp
#include <loop.hpp>

Iterate a body over tensors, accumulating into tensors.

Public Functions

Loop() = default

Constructs a Loop operation.

Loop(const Output<Node> &trip_count, const Output<Node> &execution_condition)

Constructs a Loop operation.

Parameters
  • trip_countNode specifies the maximum number of iterations.

  • execution_conditionNode determines whether to execute the first iteration or not.

virtual Output<Node> get_concatenated_slices(const Output<Node> &value, int64_t start, int64_t stride, int64_t part_size, int64_t end, int64_t axis) override

Concatenates slices from all iterations.

Parameters
  • value – The value supplying slice values from each iteration.

  • start – First index on axis of the slicing

  • stride – Stepping of the slice

  • part_size – Size of the slice on axis

  • end – The last index on axis of the slicing

  • axis – The axis to slice along

Returns

The concatenated slices.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

struct SpecialBodyPorts
#include <loop.hpp>

Allows to define the purpose of inputs/outputs in the body.

class LSTMSequence : public ov::op::util::RNNCellBase
#include <lstm_sequence.hpp>

Class for lstm sequence node.

See also

LSTMCell, RNNCell, GRUCell

Note

It follows notation and equations defined as in ONNX standard: https://github.com/onnx/onnx/blob/master/docs/Operators.md#LSTM

Public Functions

inline virtual size_t get_default_output_index() const override

Returns the output of the default output, or throws if there is none.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class NonMaxSuppression : public ov::op::Op
#include <non_max_suppression.hpp>

NonMaxSuppression operation.

Public Functions

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation with default values in the last 4 inputs.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation with default values in the last. 3 inputs.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const Output<Node> &iou_threshold, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation with default values in the last. 2 inputs.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • iou_thresholdNode producing intersection over union threshold

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const Output<Node> &iou_threshold, const Output<Node> &score_threshold, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation with default value in the last. input.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • iou_thresholdNode producing intersection over union threshold

  • score_thresholdNode producing minimum score threshold

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const Output<Node> &iou_threshold, const Output<Node> &score_threshold, const Output<Node> &soft_nms_sigma, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • iou_thresholdNode producing intersection over union threshold

  • score_thresholdNode producing minimum score threshold

  • soft_nms_sigmaNode specifying the sigma parameter for Soft-NMS

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class RNNSequence : public ov::op::util::RNNCellBase
#include <rnn_sequence.hpp>

RNNSequence operation.

Public Functions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Round : public ov::op::util::UnaryElementwiseArithmetic
#include <round.hpp>

Elementwise round operation. The output is round to the nearest integer for each value. In case of halfs, the rule is defined in attribute ‘mode’: ‘HALF_TO_EVEN’ - round halfs to the nearest even integer. ‘HALF_AWAY_FROM_ZERO’: - round in such a way that the result heads away from zero.

Public Functions

Round() = default

Constructs a round operation.

Round(const Output<Node> &arg, const RoundMode mode)

Constructs a round operation.

Parameters
  • argNode that produces the input tensor.

  • mode – Rule to resolve halfs

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

namespace v6

Functions

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const CTCGreedyDecoderSeqLen *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ExperimentalDetectronDetectionOutput *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ExperimentalDetectronGenerateProposalsSingleImage *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ExperimentalDetectronPriorGridGenerator *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ExperimentalDetectronROIFeatureExtractor *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(ExperimentalDetectronTopKROIs *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const GatherElements *op, const std::vector<T> &input_shapes)
class Assign : public ov::op::util::AssignBase
#include <assign.hpp>

Assign operation sets an input value to the variable with variable_id

Public Functions

Assign(const Output<Node> &new_value, const std::shared_ptr<util::Variable> &variable)

Constructs an Assign operation.

Parameters
  • new_valueNode that produces the input tensor.

  • variable – Class for storing and synchronizing element types, shapes and identifiers between pairs of Assign/ReadValue nodes.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual std::string get_variable_id() const override

Returns the identifier of corresponding variable.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class CTCGreedyDecoderSeqLen : public ov::op::Op
#include <ctc_greedy_decoder_seq_len.hpp>

Operator performing CTCGreedyDecoder.

Public Functions

CTCGreedyDecoderSeqLen(const Output<Node> &input, const Output<Node> &seq_len, const bool merge_repeated = true, const element::Type &classes_index_type = element::i32, const element::Type &sequence_length_type = element::i32)

Constructs a CTCGreedyDecoderSeqLen operation.

Parameters
  • input – 3-D tensor of logits on which greedy decoding is performed

  • seq_len – 1-D tensor of sequence lengths

  • merge_repeated – Whether to merge repeated labels

  • classes_index_type – Specifies the output classes_index tensor type

  • sequence_length_type – Specifies the output sequence_length tensor type

CTCGreedyDecoderSeqLen(const Output<Node> &input, const Output<Node> &seq_len, const Output<Node> &blank_index, const bool merge_repeated = true, const element::Type &classes_index_type = element::i32, const element::Type &sequence_length_type = element::i32)

Constructs a CTCGreedyDecoderSeqLen operation.

Parameters
  • input – 3-D tensor of logits on which greedy decoding is performed

  • seq_len – 1-D tensor of sequence lengths

  • blank_index – Scalar or 1-D tensor with 1 element used to mark a blank index

  • merge_repeated – Whether to merge repeated labels

  • classes_index_type – Specifies the output classes_index tensor type

  • sequence_length_type – Specifies the output sequence_length tensor type

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline bool get_merge_repeated() const

Get merge_repeated attribute.

Returns

Current value of merge_repeated attribute

inline void set_merge_repeated(bool merge_repeated)

Set merge_repeated attribute.

Parameters

merge_repeated – A new value for the attribute

inline const element::Type &get_classes_index_type() const

Get classes_index_type attribute.

Returns

Current value of classes_index_type attribute

inline void set_classes_index_type(const element::Type &classes_index_type)

Set classes_index_type attribute.

Parameters

classes_index_type – Type of classes_index

inline const element::Type &get_sequence_length_type() const

Get sequence_length_type attribute.

Returns

Current value of sequence_length_type attribute

inline void set_sequence_length_type(const element::Type &sequence_length_type)

Set sequence_length_type attribute.

Parameters

sequence_length_type – Type of sequence length

class ExperimentalDetectronDetectionOutput : public ov::op::Op
#include <experimental_detectron_detection_output.hpp>

An operation ExperimentalDetectronDetectionOutput performs non-maximum suppression to generate the detection output using information on location and score predictions.

Public Functions

ExperimentalDetectronDetectionOutput(const Output<Node> &input_rois, const Output<Node> &input_deltas, const Output<Node> &input_scores, const Output<Node> &input_im_info, const Attributes &attrs)

Constructs a ExperimentalDetectronDetectionOutput operation.

Parameters
  • input_roisInput rois

  • input_deltasInput deltas

  • input_scoresInput scores

  • input_im_infoInput image info

  • attrsAttributes attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline const Attributes &get_attrs() const

Returns attributes of the operation ExperimentalDetectronDetectionOutput.

void set_attrs(Attributes attrs)

Set the attributes of the operation ExperimentalDetectronDetectionOutput.

Parameters

attrsAttributes to set.

struct Attributes
#include <experimental_detectron_detection_output.hpp>

Structure that specifies attributes of the operation.

class ExperimentalDetectronGenerateProposalsSingleImage : public ov::op::Op
#include <experimental_detectron_generate_proposals.hpp>

An operation ExperimentalDetectronGenerateProposalsSingleImage computes ROIs and their scores based on input data.

Public Functions

ExperimentalDetectronGenerateProposalsSingleImage(const Output<Node> &im_info, const Output<Node> &anchors, const Output<Node> &deltas, const Output<Node> &scores, const Attributes &attrs)

Constructs a ExperimentalDetectronGenerateProposalsSingleImage operation.

Parameters
  • im_infoInput image info

  • anchorsInput anchors

  • deltasInput deltas

  • scoresInput scores

  • attrs – Operation attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

struct Attributes
#include <experimental_detectron_generate_proposals.hpp>

Structure that specifies attributes of the operation.

class ExperimentalDetectronPriorGridGenerator : public ov::op::Op
#include <experimental_detectron_prior_grid_generator.hpp>

An operation ExperimentalDetectronPriorGridGenerator generates prior grids of specified sizes.

Public Functions

ExperimentalDetectronPriorGridGenerator(const Output<Node> &priors, const Output<Node> &feature_map, const Output<Node> &im_data, const Attributes &attrs)

Constructs a ExperimentalDetectronDetectionOutput operation.

Parameters
  • priorsInput priors

  • feature_mapInput feature map

  • im_data – Image data

  • attrs – attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline const Attributes &get_attrs() const

Returns attributes of this operation.

void set_attrs(Attributes attrs)

Set the attributes of the operation ExperimentalDetectronPriorGridGenerator.

Parameters

attrsAttributes to set.

struct Attributes
#include <experimental_detectron_prior_grid_generator.hpp>

Structure that specifies attributes of the operation.

class ExperimentalDetectronROIFeatureExtractor : public ov::op::Op
#include <experimental_detectron_roi_feature.hpp>

An operation ExperimentalDetectronROIFeatureExtractor is the ROIAlign operation applied over a feature pyramid.

Public Functions

ExperimentalDetectronROIFeatureExtractor(const OutputVector &args, const Attributes &attrs)

Constructs a ExperimentalDetectronROIFeatureExtractor operation.

Parameters
ExperimentalDetectronROIFeatureExtractor(const NodeVector &args, const Attributes &attrs)

Constructs a ExperimentalDetectronROIFeatureExtractor operation.

Parameters
virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline const Attributes &get_attrs() const

Returns attributes of the operation.

void set_attrs(Attributes attrs)

Set the ExperimentalDetectronROIFeatureExtractor’s attributes.

Parameters

attrsAttributes to set.

struct Attributes
#include <experimental_detectron_roi_feature.hpp>

Structure that specifies attributes of the operation.

class ExperimentalDetectronTopKROIs : public ov::op::Op
#include <experimental_detectron_topkrois.hpp>

An operation ExperimentalDetectronTopKROIs, according to the repository is TopK operation applied to probabilities of input ROIs.

Public Functions

ExperimentalDetectronTopKROIs(const Output<Node> &input_rois, const Output<Node> &rois_probs, size_t max_rois = 0)

Constructs a ExperimentalDetectronTopKROIs operation.

Parameters
  • input_roisInput rois

  • rois_probs – Probabilities for input rois

  • max_rois – Maximal numbers of output rois

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class GatherElements : public ov::op::Op
#include <gather_elements.hpp>

GatherElements operation.

Public Functions

GatherElements(const Output<Node> &data, const Output<Node> &indices, const int64_t axis)

Constructs a GatherElements operation.

Parameters
  • dataNode producing data that are gathered

  • indicesNode producing indices by which the operation gathers elements

  • axis – specifies axis along which indices are specified

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class MVN : public ov::op::Op
#include <mvn.hpp>

Operator performing Mean Variance Normalization.

Public Functions

MVN(const Output<Node> &data, const Output<Node> &reduction_axes, bool normalize_variance, float eps, MVNEpsMode eps_mode)

Constructs an MVN operation.

Parameters
  • dataInput tensor with data

  • reduction_axes – A list of axes, along which to reduce.

  • normalize_variance – flag that denotes whether to perform variance normalization.

  • eps – the number to be added to the variance to avoid division by zero when normalizing the value

  • eps_mode – the mode of applying epsilon

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values) const override

Evaluates the op on input_values putting results in output_values.

Parameters
  • output_values – Tensors for the outputs to compute. One for each result

  • input_values – Tensors for the inputs. One for each inputs.

Returns

true if successful

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class ReadValue : public ov::op::util::ReadValueBase
#include <read_value.hpp>

ReadValue operation gets an input value from the variable with variable_id and returns it as an output.

Public Functions

explicit ReadValue(const std::shared_ptr<util::Variable> &variable)

Constructs a ReadValue operation.

Parameters

variable – Class for storing and synchronizing element types, shapes and identifiers between pairs of Assign/ReadValue nodes.

ReadValue(const Output<Node> &init_value, const std::shared_ptr<util::Variable> &variable)

Constructs a ReadValue operation.

Parameters
  • init_valueNode that produces the input tensor.

  • variable – Class for storing and synchronizing element types, shapes and identifiers between pairs of Assign/ReadValue nodes.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual std::string get_variable_id() const override

Returns the identifier of corresponding variable.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

namespace v7

Functions

template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Einsum *op, const std::vector<T> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Roll *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
class DFT : public ov::op::util::FFTBase
#include <dft.hpp>

An operation DFT that computes the discrete Fourier transformation.

Public Functions

DFT(const Output<Node> &data, const Output<Node> &axes)

Constructs a DFT operation. DFT is performed for full size axes.

Parameters
  • dataInput data

  • axes – Axes to perform DFT

DFT(const Output<Node> &data, const Output<Node> &axes, const Output<Node> &signal_size)

Constructs a DFT operation.

Parameters
  • dataInput data

  • axes – Axes to perform DFT

  • signal_size – Signal sizes for ‘axes’

class Einsum : public ov::op::Op
#include <einsum.hpp>

Einsum operation.

Public Functions

Einsum(const OutputVector &inputs, const std::string &equation)

Constructs Einsum operation.

Parameters
  • inputsInput nodes on which Einsum operation performs contraction

  • equation – Einstein summation convention

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

void set_equation(std::string equation)

Set Einsum equation.

Parameters

equation – Equation string to be set.

inline const std::string &get_equation() const

Get an equation of Einsum operation.

Returns

Einsum equation

Public Static Functions

static void parse_equation(const std::string &equation, std::vector<std::string> &input_subscripts, std::string &output_subscript)

Check correctness of equation format and extract input subscripts and output subscript.

Parameters
  • equation – Equation to be parsed and checked

  • input_subscripts – A vector of extracted input subscripts

  • output_subscript – An output subscript

static std::vector<std::string> extract_labels(const std::string &subscript)

Extract labels (from subscript) that can be alphabetic letters or ellipsis.

Parameters

subscript – Subscript

Returns

A vector of extracted labels from the input subscript in the order of appearence

class Gather : public ov::op::util::GatherBase
#include <gather.hpp>

Gather slices from axis of data according to indices.

Public Functions

Gather(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &axis, const int64_t batch_dims = 0)
Parameters
  • data – The tensor from which slices are gathered

  • indicesTensor with indexes to gather

  • axis – The tensor is a dimension index to gather data from

  • batch_dims – The number of batch dimension in data and indices tensors. If batch_dims = 0 Gather v7 is identical to Gather v1.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Gelu : public ov::op::util::UnaryElementwiseArithmetic
#include <gelu.hpp>

Gaussian Error Linear Unit f(x) = 0.5 * x * (1 + erf( x / sqrt(2) ) for “approximation” = “erf” f(x) = 0.5 * x * (1 + tanh([sqrt(2 / pi)] * [x + 0.044715^3]) for “approximation” = “tanh”.

Public Functions

Gelu(const Output<Node> &data, GeluApproximationMode mode = GeluApproximationMode::ERF)

Constructs a Gelu operation.

Parameters
  • dataInput tensor

  • mode – Approximation mode

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class IDFT : public ov::op::util::FFTBase
#include <idft.hpp>

An operation IDFT that computes the inverse discrete Fourier transformation.

Public Functions

IDFT(const Output<Node> &data, const Output<Node> &axes)

Constructs a IDFT operation. IDFT is performed for full size axes.

Parameters
  • dataInput data

  • axes – Axes to perform IDFT

IDFT(const Output<Node> &data, const Output<Node> &axes, const Output<Node> &signal_size)

Constructs a IDFT operation.

Parameters
  • dataInput data

  • axes – Axes to perform IDFT

  • signal_size – Signal sizes for ‘axes’

class Roll : public ov::op::Op
#include <roll.hpp>

Tensor roll operation.

Public Functions

Roll(const Output<Node> &data, const Output<Node> &shift, const Output<Node> &axes)

Constructs a roll operation.

Parameters
  • dataNode producing the tensor to be shifted.

  • shiftNode producing the 0D or 1D tensor which specifies the number of places by which the elements are shifted.

  • axesNode producing the 0D or 1D tensor which specifies axes along which elements are shifted.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

namespace v8

Functions

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const AdaptiveAvgPool *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const AdaptiveMaxPool *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DeformableConvolution *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DetectionOutput *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const GatherND *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const MatrixNms *op, const std::vector<TShape> &input_shapes)
template<class TShape, class TContainer, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const MaxPool *op, const std::vector<TShape> &input_shapes, TContainer &pads_begin, TContainer &pads_end)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const PriorBox *const op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const RandomUniform *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Slice *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
class AdaptiveAvgPool : public ov::op::Op
#include <adaptive_avg_pool.hpp>

Adaptive average pooling operation.

Public Functions

AdaptiveAvgPool(const Output<Node> &data, const Output<Node> &output_shape)

Constructs adaptive average pooling operation.

Parameters
  • dataInput data

  • output_shape – 1D tensor describing output shape for spatial dimensions.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class AdaptiveMaxPool : public ov::op::Op
#include <adaptive_max_pool.hpp>

Adaptive max pooling operation.

Public Functions

AdaptiveMaxPool(const Output<Node> &data, const Output<Node> &output_shape, const ov::element::Type &index_element_type = ov::element::i64)

Constructs adaptive max pooling operation.

Parameters
  • dataInput data

  • output_shape – 1D tensor describing output shape for spatial dimensions.

  • index_element_type – Specifies the output tensor type for indices output

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class DeformableConvolution : public ov::op::util::DeformableConvolutionBase
#include <deformable_convolution.hpp>

DeformableConvolution operation.

Public Functions

DeformableConvolution() = default

Constructs a conversion operation.

DeformableConvolution(const Output<Node> &arg, const Output<Node> &offsets, const Output<Node> &filters, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT, const int64_t group = 1, const int64_t deformable_group = 1, const bool bilinear_interpolation_pad = false)

Constructs a conversion operation.

Parameters
  • argNode that produces the input tensor.

  • offsetsNode producing the deformable values tensor.

  • filtersNode producing the filters(kernels) tensor with OIZYX layout.

  • strides – Convolution strides.

  • pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.

  • pads_end – Amount of padding to be added to the end along each axis.

  • dilations – The distance in width and height between the weights in the filters tensor.

  • auto_pad – Specifies how the automatic calculation of padding should be done.

  • group – The number of groups which both output and input should be split into.

  • deformable_group – The number of groups which deformable values and output should be split into along the channel axis.

  • bilinear_interpolation_pad – The flag that determines the mode of bilinear interpolation execution. If the flag is true and the sampling location is within one pixel outside of the feature map boundary, then bilinear interpolation is performed on the zero padded feature map. If the flag is false and the sampling location is within one pixel outside of the feature map boundary, then the sampling location shifts to the inner boundary of the feature map.`

DeformableConvolution(const Output<Node> &arg, const Output<Node> &offsets, const Output<Node> &filters, const Output<Node> &mask, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT, const int64_t group = 1, const int64_t deformable_group = 1, const bool bilinear_interpolation_pad = false)

Constructs a conversion operation.

Parameters
  • argNode that produces the input tensor.

  • offsetsNode producing the deformable values tensor.

  • filtersNode producing the filters(kernels) tensor with OIZYX layout.

  • maskNode producing the mask(mask) tensor.

  • strides – Convolution strides.

  • pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.

  • pads_end – Amount of padding to be added to the end along each axis.

  • dilations – The distance in width and height between the weights in the filters tensor.

  • auto_pad – Specifies how the automatic calculation of padding should be done.

  • group – The number of groups which both output and input should be split into.

  • deformable_group – The number of groups which deformable values and output should be split into along the channel axis.

  • bilinear_interpolation_pad – The flag that determines the mode of bilinear interpolation execution. If the flag is true and the sampling location is within one pixel outside of the feature map boundary, then bilinear interpolation is performed on the zero padded feature map. If the flag is false and the sampling location is within one pixel outside of the feature map boundary, then the sampling location shifts to the inner boundary of the feature map.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class DetectionOutput : public ov::op::util::DetectionOutputBase
#include <detection_output.hpp>

Layer which performs non-max suppression to generate detection output using location and confidence predictions.

Public Functions

DetectionOutput(const Output<Node> &box_logits, const Output<Node> &class_preds, const Output<Node> &proposals, const Output<Node> &aux_class_preds, const Output<Node> &aux_box_preds, const Attributes &attrs)

Constructs a DetectionOutput operation.

Parameters
  • box_logits – Box logits

  • class_preds – Class predictions

  • proposals – Proposals

  • aux_class_preds – Auxilary class predictions

  • aux_box_preds – Auxilary box predictions

  • attrs – Detection Output attributes

DetectionOutput(const Output<Node> &box_logits, const Output<Node> &class_preds, const Output<Node> &proposals, const Attributes &attrs)

Constructs a DetectionOutput operation.

Parameters
  • box_logits – Box logits

  • class_preds – Class predictions

  • proposals – Proposals

  • attrs – Detection Output attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class Gather : public ov::op::util::GatherBase
#include <gather.hpp>

Gather slices from axis of data according to indices. Negative indices are supported and indicate reverse indexing from the end.

Public Functions

Gather(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &axis, const int64_t batch_dims = 0)
Parameters
  • data – The tensor from which slices are gathered

  • indicesTensor with indexes to gather

  • axis – The tensor is a dimension index to gather data from

  • batch_dims – The number of batch dimension in data and indices tensors.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class GatherND : public ov::op::util::GatherNDBase
#include <gather_nd.hpp>

GatherND operation.

Public Functions

GatherND(const Output<Node> &data, const Output<Node> &indices, const size_t batch_dims = 0)

Constructs a GatherND operation.

Parameters
  • dataNode producing data that are gathered

  • indicesNode producing indices by which the operation gathers elements or slices from data

  • batch_dims – Specifies a number of batch dimensions

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class I420toBGR : public ov::op::util::ConvertColorI420Base
#include <i420_to_bgr.hpp>

Color conversion operation from I420 to BGR format. Input:

  • Input NV12 image can be represented in two ways: a) Single plane (as it is in the file): I420 height dimension is 1.5x bigger than image height. ‘C’ dimension shall be 1. b) Three separate planes (used this way in many physical video sources): Y, U and V. In this case b1) Y plane has height same as image height. ‘C’ dimension equals to 1 b2) U plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 1. b3) V plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 1.

  • Supported element types: u8 or any supported floating-point type. Output:

  • Output node will have NHWC layout and shape HxW same as image spatial dimensions.

  • Number of output channels ‘C’ will be 3, as per interleaved BGR format, first channel is B, last is R

    Conversion of each pixel from I420 (YUV) to RGB space is represented by following formulas: R = 1.164 * (Y - 16) + 1.596 * (V - 128) G = 1.164 * (Y - 16) - 0.813 * (V - 128) - 0.391 * (U - 128) B = 1.164 * (Y - 16) + 2.018 * (U - 128) Then R, G, B values are clipped to range (0, 255)

Public Functions

explicit I420toBGR(const Output<Node> &arg)

Constructs a conversion operation from input image in I420 format As per I420 format definition, node height dimension shall be 1.5 times bigger than image height so that image (w=640, h=480) is represented by NHWC shape {N,720,640,1} (height*1.5 x width)

Parameters

argNode that produces the input tensor. Input tensor represents image in NV12 format (YUV).

explicit I420toBGR(const Output<Node> &arg_y, const Output<Node> &arg_u, const Output<Node> &arg_v)

Constructs a conversion operation from 2-plane input image in NV12 format In general case Y channel of image can be separated from UV channel which means that operation needs two nodes for Y and UV planes respectively. Y plane has one channel, and UV has 2 channels, both expect ‘NHWC’ layout.

Parameters
  • arg_yNode that produces the input tensor for Y plane (NHWC layout). Shall have WxH dimensions equal to image dimensions. ‘C’ dimension equals to 1.

  • arg_uNode that produces the input tensor for U plane (NHWC layout). ‘H’ is half of image height, ‘W’ is half of image width, ‘C’ dimension equals to 1.

  • arg_vNode that produces the input tensor for V plane (NHWC layout). ‘H’ is half of image height, ‘W’ is half of image width, ‘C’ dimension equals to 1.

class I420toRGB : public ov::op::util::ConvertColorI420Base
#include <i420_to_rgb.hpp>

Color conversion operation from I420 to RGB format. Input:

  • Input NV12 image can be represented in two ways: a) Single plane (as it is in the file): I420 height dimension is 1.5x bigger than image height. ‘C’ dimension shall be 1. b) Three separate planes (used this way in many physical video sources): Y, U and V. In this case b1) Y plane has height same as image height. ‘C’ dimension equals to 1 b2) U plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 1. b3) V plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 1.

  • Supported element types: u8 or any supported floating-point type. Output:

  • Output node will have NHWC layout and shape HxW same as image spatial dimensions.

  • Number of output channels ‘C’ will be 3, as per interleaved RGB format, first channel is R, last is B

    Conversion of each pixel from I420 (YUV) to RGB space is represented by following formulas: R = 1.164 * (Y - 16) + 1.596 * (V - 128) G = 1.164 * (Y - 16) - 0.813 * (V - 128) - 0.391 * (U - 128) B = 1.164 * (Y - 16) + 2.018 * (U - 128) Then R, G, B values are clipped to range (0, 255)

Public Functions

explicit I420toRGB(const Output<Node> &arg)

Constructs a conversion operation from input image in I420 format As per I420 format definition, node height dimension shall be 1.5 times bigger than image height so that image (w=640, h=480) is represented by NHWC shape {N,720,640,1} (height*1.5 x width)

Parameters

argNode that produces the input tensor. Input tensor represents image in NV12 format (YUV).

explicit I420toRGB(const Output<Node> &arg_y, const Output<Node> &arg_u, const Output<Node> &arg_v)

Constructs a conversion operation from 2-plane input image in NV12 format In general case Y channel of image can be separated from UV channel which means that operation needs two nodes for Y and UV planes respectively. Y plane has one channel, and UV has 2 channels, both expect ‘NHWC’ layout.

Parameters
  • arg_yNode that produces the input tensor for Y plane (NHWC layout). Shall have WxH dimensions equal to image dimensions. ‘C’ dimension equals to 1.

  • arg_uNode that produces the input tensor for U plane (NHWC layout). ‘H’ is half of image height, ‘W’ is half of image width, ‘C’ dimension equals to 1.

  • arg_vNode that produces the input tensor for V plane (NHWC layout). ‘H’ is half of image height, ‘W’ is half of image width, ‘C’ dimension equals to 1.

class If : public ov::op::util::MultiSubGraphOp
#include <if.hpp>

If operation.

Public Functions

If(const Output<Node> &execution_condition)

Constructs If with condition.

Parameters

execution_condition – condition node.

inline const std::shared_ptr<Model> &get_then_body() const

gets then_body as ov::Model.

Returns

then_body as ov::Model.

inline const std::shared_ptr<Model> &get_else_body() const

gets else_body as ov::Model.

Returns

else_body as ov::Model.

inline void set_then_body(const std::shared_ptr<Model> &body)

sets new ov::Model as new then_body.

Parameters

body – new body for ‘then’ branch.

inline void set_else_body(const std::shared_ptr<Model> &body)

sets new ov::Model as new else_body.

Parameters

body – new body for ‘else’ branch.

void set_input(const Output<Node> &value, const std::shared_ptr<v0::Parameter> &then_parameter, const std::shared_ptr<v0::Parameter> &else_parameter)

sets new input to the operation associated with parameters of each sub-graphs

Parameters
  • value – input to operation

  • then_parameter – parameter for then_body or nullptr

  • else_parameter – parameter for else_body or nullpt

Output<Node> set_output(const std::shared_ptr<v0::Result> &then_result, const std::shared_ptr<v0::Result> &else_result)

sets new output from the operation associated with results of each sub-graphs

Parameters
  • then_result – result from then_body

  • else_parameter – result from else_body

Returns

output from operation

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class MatrixNms : public ov::op::Op
#include <matrix_nms.hpp>

MatrixNms operation.

Public Functions

MatrixNms() = default

Constructs a conversion operation.

MatrixNms(const Output<Node> &boxes, const Output<Node> &scores, const Attributes &attrs)

Constructs a MatrixNms operation.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • attrsAttributes of the operation

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline const Attributes &get_attrs() const

Returns attributes of the operation MatrixNms.

struct Attributes
#include <matrix_nms.hpp>

Structure that specifies attributes of the operation.

class MaxPool : public ov::op::util::MaxPoolBase
#include <max_pool.hpp>

MaxPooling operation with values and indices calculated as individual outputs.

Public Functions

MaxPool() = default

Constructs an empty MaxPool operation.

MaxPool(const Output<Node> &arg, const Strides &strides, const Strides &dilations, const Shape &pads_begin, const Shape &pads_end, const Shape &kernel, const op::RoundingType rounding_type = op::RoundingType::FLOOR, const PadType auto_pad = op::PadType::EXPLICIT, const element::Type index_element_type = element::i64, const int64_t axis = 0)

Constructs a parametrized MaxPool operation.

Parameters
  • argOutput of a node producing the feature tensor to be pooled.

  • strides – The strides of the pooling filter.

  • dilations – The dilations of the pooling filter.

  • pads_begin – Paddings at the beginning of each spatial axis.

  • pads_end – Paddings at the end of each spatial axis.

  • kernel – The kernel shape.

  • rounding_type – Whether to use ceiling or floor rounding type while computing the output shape.

  • auto_pad – The pad type for automatic calculation of the padding sizes.

  • index_element_type – The data type used by the second output tensor containing the selected indices.

  • axis – Indicates a dimension in the input data shape which should be used as a starting point for calculation of the upper bound of allowed values of the indices output.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline const Strides &get_dilations() const noexcept
Returns

The pooling filter’s dilations.

inline element::Type get_index_element_type() const noexcept
Returns

The data type of the second output tensor (indices).

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class MulticlassNms : public ov::op::util::MulticlassNmsBase
#include <multiclass_nms.hpp>

MulticlassNms operation.

Public Functions

MulticlassNms() = default

Constructs a conversion operation.

MulticlassNms(const Output<Node> &boxes, const Output<Node> &scores, const Attributes &attrs)

Constructs a MulticlassNms operation.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • attrs – Attributes of the operation

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class NV12toBGR : public ov::op::util::ConvertColorNV12Base
#include <nv12_to_bgr.hpp>

Color conversion operation from NV12 to RGB format. Input:

  • Input NV12 image can be represented in two ways: a) Single plane (as it is in the file): NV12 height dimension is 1.5x bigger than image height. ‘C’ dimension shall be 1. b) Two separate planes (used this way in many physical video sources): Y and UV. In this case b1) Y plane has height same as image height. ‘C’ dimension equals to 1 b2) UV plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 2.

  • Supported element types: u8 or any supported floating-point type. Output:

  • Output node will have NHWC layout and shape HxW same as image spatial dimensions.

  • Number of output channels ‘C’ will be 3, as per interleaved RGB format, first channel is B, last is R

    Conversion of each pixel from NV12 (YUV) to RGB space is represented by following formulas: R = 1.164 * (Y - 16) + 1.596 * (V - 128) G = 1.164 * (Y - 16) - 0.813 * (V - 128) - 0.391 * (U - 128) B = 1.164 * (Y - 16) + 2.018 * (U - 128) Then R, G, B values are clipped to range (0, 255)

Public Functions

explicit NV12toBGR(const Output<Node> &arg)

Constructs a conversion operation from input image in NV12 format As per NV12 format definition, node height dimension shall be 1.5 times bigger than image height so that image (w=640, h=480) is represented by NHWC shape {N,720,640,1} (height*1.5 x width)

Parameters

argNode that produces the input tensor. Input tensor represents image in NV12 format (YUV).

explicit NV12toBGR(const Output<Node> &arg_y, const Output<Node> &arg_uv)

Constructs a conversion operation from 2-plane input image in NV12 format In general case Y channel of image can be separated from UV channel which means that operation needs two nodes for Y and UV planes respectively. Y plane has one channel, and UV has 2 channels, both expect ‘NHWC’ layout.

Parameters
  • arg_yNode that produces the input tensor for Y plane (NHWC layout). Shall have WxH dimensions equal to image dimensions. ‘C’ dimension equals to 1.

  • arg_uvNode that produces the input tensor for UV plane (NHWC layout). ‘H’ is half of image height, ‘W’ is half of image width, ‘C’ dimension equals to 2. Channel 0 represents ‘U’, channel 1 represents ‘V’ channel

class NV12toRGB : public ov::op::util::ConvertColorNV12Base
#include <nv12_to_rgb.hpp>

Color conversion operation from NV12 to RGB format. Input:

  • Input NV12 image can be represented in two ways: a) Single plane (as it is in the file): NV12 height dimension is 1.5x bigger than image height. ‘C’ dimension shall be 1. b) Two separate planes (used this way in many physical video sources): Y and UV. In this case b1) Y plane has height same as image height. ‘C’ dimension equals to 1 b2) UV plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 2.

  • Supported element types: u8 or any supported floating-point type. Output:

  • Output node will have NHWC layout and shape HxW same as image spatial dimensions.

  • Number of output channels ‘C’ will be 3, as per interleaved RGB format, first channel is R, last is B

    Conversion of each pixel from NV12 (YUV) to RGB space is represented by following formulas: R = 1.164 * (Y - 16) + 1.596 * (V - 128) G = 1.164 * (Y - 16) - 0.813 * (V - 128) - 0.391 * (U - 128) B = 1.164 * (Y - 16) + 2.018 * (U - 128) Then R, G, B values are clipped to range (0, 255)

Public Functions

explicit NV12toRGB(const Output<Node> &arg)

Constructs a conversion operation from input image in NV12 format As per NV12 format definition, node height dimension shall be 1.5 times bigger than image height so that image (w=640, h=480) is represented by NHWC shape {N,720,640,1} (height*1.5 x width)

Parameters

argNode that produces the input tensor. Input tensor represents image in NV12 format (YUV).

NV12toRGB(const Output<Node> &arg_y, const Output<Node> &arg_uv)

Constructs a conversion operation from 2-plane input image in NV12 format In general case Y channel of image can be separated from UV channel which means that operation needs two nodes for Y and UV planes respectively. Y plane has one channel, and UV has 2 channels, both expect ‘NHWC’ layout.

Parameters
  • arg_yNode that produces the input tensor for Y plane (NHWC layout). Shall have WxH dimensions equal to image dimensions. ‘C’ dimension equals to 1.

  • arg_uvNode that produces the input tensor for UV plane (NHWC layout). ‘H’ is half of image height, ‘W’ is half of image width, ‘C’ dimension equals to 2. Channel 0 represents ‘U’, channel 1 represents ‘V’ channel

class PriorBox : public ov::op::Op
#include <prior_box.hpp>

Layer which generates prior boxes of specified sizes normalized to input image size.

Public Functions

PriorBox(const Output<Node> &layer_shape, const Output<Node> &image_shape, const Attributes &attrs)

Constructs a PriorBox operation.

Parameters
  • layer_shapeShape of layer for which prior boxes are computed

  • image_shapeShape of image to which prior boxes are scaled

  • attrsPriorBox attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

struct Attributes
#include <prior_box.hpp>
class RandomUniform : public ov::op::Op
#include <random_uniform.hpp>

Tensor RandomUniform operation.

Public Functions

RandomUniform(const Output<Node> &out_shape, const Output<Node> &min_val, const Output<Node> &max_val, const ov::element::Type &out_type, uint64_t global_seed = 0, uint64_t op_seed = 0)

Constructs a RandomUniform operation.

Parameters
  • out_shapeNode producing the tensor with output shape.

  • min_valNode producing the tensor with minimum value.

  • max_valNode producing the tensor with maximum value.

  • out_typeOutput type of the tensor.

  • global_seed – Global seed value.

  • op_seed – Operational seed value.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline virtual bool constant_fold(OutputVector &output_values, const OutputVector &inputs_values) override
Returns

Turns off constant folding for RandomUniform operation.

inline const ov::element::Type &get_out_type() const
Returns

The output tensor type.

inline uint64_t get_global_seed() const
Returns

The global seed value.

inline uint64_t get_op_seed() const
Returns

The operational seed value.

inline std::pair<uint64_t, uint64_t> get_state() const
Returns

The state value.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Slice : public ov::op::Op
#include <slice.hpp>

Slice operation.

Public Functions

Slice(const Output<Node> &data, const Output<Node> &start, const Output<Node> &stop, const Output<Node> &step)

Constructs Slice operation (default axes).

Parameters
  • data – The tensor to be sliced.

  • start – 1D tensor with start indices of the slice.

  • stop – 1D tensor with end indices of the slice.

  • step – 1D tensor specifies the increment to use in slicing along corresponding axes.

Slice(const Output<Node> &data, const Output<Node> &start, const Output<Node> &stop, const Output<Node> &step, const Output<Node> &axes)

Constructs Slice operation.

Parameters
  • data – The tensor to be sliced.

  • start – 1D tensor with start indices of the slice.

  • stop – 1D tensor with end indices of the slice.

  • step – 1D tensor specifies the increment to use in slicing along corresponding axes.

  • axes – 1D tensor indicating which dimensions the values in the start and stop apply to.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class Softmax : public ov::op::Op
#include <softmax.hpp>

Softmax operation with negative axis values.

Public Functions

Softmax(const Output<Node> &arg, const int64_t axis = 1)

Constructs a softmax operation.

Output [d0, ...]

Parameters
  • argNode that produces the first input tensor.[d0, ...]

  • axis – The axis position (0-based) in range [-rank(arg), rank(arg) - 1] on which to calculate the softmax.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

namespace v9

Functions

template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Eye *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())

Eye v9 shape inference compute output shapes.

Template Parameters

TShape – Type of shape.

Parameters
  • op – Pointer to Eye operator.

  • input_shapesInput shapes of Eye.

  • taTensor accessor to constant data.

Returns

Vector with output shapes.

template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const GenerateProposals *op, const std::vector<T> &input_shapes)
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const GridSample *op, const std::vector<TShape> &input_shapes)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const IRDFT *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const NonMaxSuppression *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor(), const bool static_output = !std::is_same<T, PartialShape>::value)
template<typename B>
B get_ouput_dimension_bound(B b)
template<class DimType>
DimType get_rdft_output_dimension(DimType d)
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const RDFT *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ROIAlign *op, const std::vector<TShape> &input_shapes)
class Eye : public ov::op::Op
#include <eye.hpp>

Tensor Eye operation.

Public Functions

Eye(const Output<Node> &num_rows, const Output<Node> &num_columns, const Output<Node> &diagonal_index, const Output<Node> &batch_shape, const ov::element::Type &out_type)

Constructs a Eye operation.

Parameters
  • num_rowsNode producing the tensor with row number.

  • num_columnsNode producing the tensor with column number.

  • diagonal_indexNode producing the tensor with the index of diagonal with ones.

  • batch_shapeNode producing the tensor with batch shape.

  • out_typeOutput type of the tensor.

Eye(const Output<Node> &num_rows, const Output<Node> &num_columns, const Output<Node> &diagonal_index, const ov::element::Type &out_type)

Constructs a Eye operation without batch_shape.

Parameters
  • num_rowsNode producing the tensor with row number.

  • num_columnsNode producing the tensor with column number.

  • diagonal_indexNode producing the tensor with the index of diagonal with ones.

  • out_typeOutput type of the tensor.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

inline const ov::element::Type &get_out_type() const
Returns

The output tensor type.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

class GenerateProposals : public ov::op::Op
#include <generate_proposals.hpp>

An operation GenerateProposals computes ROIs and their scores based on input data.

Subclassed by ov::op::internal::GenerateProposalsIEInternal

Public Functions

GenerateProposals(const Output<Node> &im_info, const Output<Node> &anchors, const Output<Node> &deltas, const Output<Node> &scores, const Attributes &attrs, const element::Type &roi_num_type = element::i64)

Constructs a GenerateProposals operation.

Parameters
  • im_infoInput image info

  • anchorsInput anchors

  • deltasInput deltas

  • scoresInput scores

  • attrs – Operation attributes

  • roi_num_type – roi_num type

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

struct Attributes
#include <generate_proposals.hpp>

Structure that specifies attributes of the operation.

class GridSample : public ov::op::Op
#include <grid_sample.hpp>

Operator performing interpolated sampling of the input tensor.

Public Functions

GridSample(const Output<Node> &data, const Output<Node> &grid, const Attributes &attributes)

Constructs a GridSample operation.

Parameters
  • dataInput data tensor (input image)

  • grid – Normalized interpolation coordinates

  • attrsGridSample attributes

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

struct Attributes
#include <grid_sample.hpp>

A Structure which contains all GridSample attributes.

class IRDFT : public ov::op::util::FFTBase
#include <irdft.hpp>

An operation IRDFT that computes the discrete inverse complex-to-real Fourier transformation.

Public Functions

IRDFT(const Output<Node> &data, const Output<Node> &axes)

Constructs a IRDFT operation. IRDFT is performed for full size axes.

Parameters
  • dataInput data

  • axes – Axes to perform IRDFT

IRDFT(const Output<Node> &data, const Output<Node> &axes, const Output<Node> &signal_size)

Constructs a IRDFT operation.

Parameters
  • dataInput data

  • axes – Axes to perform IRDFT

  • signal_size – Signal sizes for ‘axes’

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class MulticlassNms : public ov::op::util::MulticlassNmsBase
#include <multiclass_nms.hpp>

MulticlassNms operation.

Public Functions

MulticlassNms() = default

Constructs a conversion operation.

MulticlassNms(const Output<Node> &boxes, const Output<Node> &scores, const Attributes &attrs)

Constructs a MulticlassNms operation.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • attrs – Attributes of the operation

MulticlassNms(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &roisnum, const Attributes &attrs)

Constructs a MulticlassNms operation.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • roisnumNode producing the number of rois

  • attrs – Attributes of the operation

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class NonMaxSuppression : public ov::op::Op
#include <non_max_suppression.hpp>

NonMaxSuppression operation.

Public Functions

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation with default values in the last 4 inputs.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation with default values in the last 3 inputs.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const Output<Node> &iou_threshold, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation with default values in the last. 2 inputs.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • iou_thresholdNode producing intersection over union threshold

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const Output<Node> &iou_threshold, const Output<Node> &score_threshold, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation with default value in the last. input.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • iou_thresholdNode producing intersection over union threshold

  • score_thresholdNode producing minimum score threshold

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

NonMaxSuppression(const Output<Node> &boxes, const Output<Node> &scores, const Output<Node> &max_output_boxes_per_class, const Output<Node> &iou_threshold, const Output<Node> &score_threshold, const Output<Node> &soft_nms_sigma, const BoxEncodingType box_encoding = BoxEncodingType::CORNER, const bool sort_result_descending = true, const ov::element::Type &output_type = ov::element::i64)

Constructs a NonMaxSuppression operation.

Parameters
  • boxesNode producing the box coordinates

  • scoresNode producing the box scores

  • max_output_boxes_per_classNode producing maximum number of boxes to be selected per class

  • iou_thresholdNode producing intersection over union threshold

  • score_thresholdNode producing minimum score threshold

  • soft_nms_sigmaNode specifying the sigma parameter for Soft-NMS

  • box_encoding – Specifies the format of boxes data encoding

  • sort_result_descending – Specifies whether it is necessary to sort selected boxes across batches

  • output_type – Specifies the output tensor type

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class RDFT : public ov::op::util::FFTBase
#include <rdft.hpp>

An operation RDFT that computes the discrete real-to-complex Fourier transformation.

Public Functions

RDFT(const Output<Node> &data, const Output<Node> &axes)

Constructs a RDFT operation. RDFT is performed for full size axes.

Parameters
  • dataInput data

  • axes – Axes to perform RDFT

RDFT(const Output<Node> &data, const Output<Node> &axes, const Output<Node> &signal_size)

Constructs a RDFT operation.

Parameters
  • dataInput data

  • axes – Axes to perform RDFT

  • signal_size – Signal sizes for ‘axes’

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class ROIAlign : public ov::op::Op
#include <roi_align.hpp>

Public Functions

ROIAlign(const Output<Node> &input, const Output<Node> &rois, const Output<Node> &batch_indices, const int pooled_h, const int pooled_w, const int sampling_ratio, const float spatial_scale, const PoolingMode mode, const AlignedMode aligned_mode = AlignedMode::ASYMMETRIC)

Constructs a ROIAlign operation.

Parameters
  • inputInput feature map {N, C, H, W}

  • rois – Regions of interest to pool over

  • batch_indices – Indices of images in the batch matching the number or ROIs

  • pooled_h – Height of the ROI output features

  • pooled_w – Width of the ROI output features

  • sampling_ratio – Number of sampling points used to compute an output element

  • spatial_scale – Spatial scale factor used to translate ROI coordinates

  • mode – Method of pooling - ‘avg’ or ‘max’

  • aligned_mode – Method of coordinates alignment - ‘asymmetric’, ‘half_pixel_for_nn’ or ‘half_pixel’

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

class SoftSign : public ov::op::util::UnaryElementwiseArithmetic
#include <softsign.hpp>

Public Functions

SoftSign(const Output<Node> &arg)

Constructs a SoftSign operation.

Parameters

argNode that produces the input tensor.[d0, ...] Output [d0, ...]

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.

virtual bool has_evaluate() const override

Allows to get information about availability of evaluate method for the current operation.

namespace validate

Functions

template<class TShape>
std::string shape_infer_explanation_str(const std::vector<TShape> &shapes, const std::string &explanation)

Provides NodeValidationFailure exception explanation string.

Parameters
  • shapes – Vector of shapes used for inference to be printed before explanation.

  • explanation – String with exception explanation.

Returns

Explanation string.

namespace opset1
namespace opset10
namespace opset11
namespace opset12
namespace opset13
namespace opset2
namespace opset3
namespace opset4
namespace opset5
namespace opset6
namespace opset7
namespace opset8
namespace opset9
namespace pass

Typedefs

using TransposeSinkingGeneral = ov::pass::transpose_sinking::TSGeneral
using PassPropertyMask = ov::EnumMask<PassProperty>
using param_callback = std::function<bool(const std::shared_ptr<const ::ov::Node>)>
using param_callback_map = std::map<ov::DiscreteTypeInfo, param_callback>

Enums

enum class PassProperty : uint32_t

Values:

enumerator REQUIRE_STATIC_SHAPE
enumerator CHANGE_DYNAMIC_STATE

Functions

void disable_fold_subgraph_empty_inputs(const std::shared_ptr<Node> &node)
void enable_fold_subgraph_empty_inputs(const std::shared_ptr<Node> &node)
bool fold_subgraph_empty_inputs_is_disabled(const std::shared_ptr<Node> &node)
OPENVINO_API void disable_remove_concat_zerodim_input (const std::shared_ptr< Node > &node)
OPENVINO_API void enable_remove_concat_zerodim_input (const std::shared_ptr< Node > &node)
OPENVINO_API bool remove_concat_zerodim_input_is_disabled (const std::shared_ptr< Node > &node)
OPENVINO_API void disable_constant_folding (const std::shared_ptr< Node > &node)

this method disables constant folding for given node. Under constant folding we consider ConstantFolding transformation, so other type of constant folding like get_constant_from_source doesn’t work with this attribute. Also before using this attribute please consider two corner cases:

  1. If for sub-graph like ShapeOf->ShapeOf we disable cf for first ShapeOf node, it doesn’t spread to the second ShapeOf, so the entire sub-graph will be folded. (In case if first ShapeOf has exactly one consumer)

  2. If node with disable_constant_folding was replaced with another node, the attribute will be lost because it is not copyable.

OPENVINO_API void enable_constant_folding (const std::shared_ptr< Node > &node)
OPENVINO_API bool constant_folding_is_disabled (const std::shared_ptr< Node > &node)

Check if constant folding is disabled on Node.

Parameters

node – Smart pointer to the node.

Returns

true if attribute constant folding set otherwise false.

OPENVINO_API bool constant_folding_is_disabled (const Node *const node)

Check if constant folding is disabled on Node.

Parameters

node – Pointer to the node.

Returns

true if attribute constant folding set otherwise false.

Variables

class OPENVINO_API RemoveConcatZeroDimInput
class OPENVINO_API DisableRemoveConcatZeroDimInput
constexpr auto float16_min_normalized = float16::from_bits(0x0400)
class AdaptivePoolToReduce : public ov::pass::MatcherPass
#include <adaptive_pool_to_reduce.hpp>

AdaptivePoolToReduce transformation replaces AdaptiveXXXPool with ReduceXXX when possible.

class AddAddFusion : public ov::pass::MatcherPass
#include <lin_op_sequence_fusion.hpp>
class AddFakeQuantizeFusion : public ov::pass::MatcherPass
#include <add_fake_quantize_fusion.hpp>

AddFakeQuantizeFusion transformation replaces following graph: Add->FakeQuantize to a single FakeQuantize Restrictions:

  • second input to Add is a Constant

class AddMultiplyFusion : public ov::pass::MatcherPass
#include <lin_op_sequence_fusion.hpp>
class AddOldApiMapToParameters : public ov::pass::MatcherPass
#include <compress_float_constants.hpp>

AddOldApiMapToParameters transformation adds OldApiMap to each float input to the model.

class AlignEltwiseInputRanks : public ov::pass::MatcherPass
#include <align_eltwise_input_ranks.hpp>
class AlignMixedFP32FP16Types : public ov::pass::ModelPass
#include <align_mixed_fp32_fp16_types.hpp>

AlignMixedFP32FP16Types adds Converts to keep mixed FP16/FP32 graph type consistent.

class ApplyTableOfEquivalence : public ov::pass::ModelPass
#include <label_optimization.hpp>

Resets symbols / labels on output shapes and values according to table of symbol / label equivalence. It allows to reduce number of labels used in the model and to disambiguate label values.

class Attributes
#include <attributes.hpp>
class AUGRUCellFusion : public ov::pass::MatcherPass
#include <augru_cell_fusion.hpp>

AUGRUCellFusion transformation replaces a sequence of operations with AUGRUCell op.

Supported activations: 1st is Sigmoid, 2nd is Tanh Clip attribute is not supported. Linear_before_reset attribute is not supported. Supported weights format: ‘rzh’

class BackwardGraphRewrite : public ov::pass::GraphRewrite
#include <graph_rewrite.hpp>

Subclassed by ov::pass::StridesOptimization

class BatchNormDecomposition : public ov::pass::MatcherPass
#include <batch_norm_decomposition.hpp>
class BatchToSpaceFusion : public ov::pass::MatcherPass
#include <batch_to_space_fusion.hpp>

BatchToSpaceFusion transformation replaces following graph: Transpose (or Reshape) -> DepthToSpace -> StridedSlice -> Transpose (or Reshape) to BatchToSpace Restrictions:

  • input rank must be 4

  • Transpose permutation must be [1, 0, 2, 3]

  • DepthToSpaceMode must be BLOCKS_FIRST

class BidirectionalGRUSequenceDecomposition : public ov::pass::MatcherPass
#include <bidirectional_sequences_decomposition.hpp>

Decompose GRUSequence to forward and reverse GRUSequence.

class BidirectionalLSTMSequenceDecomposition : public ov::pass::MatcherPass
#include <bidirectional_sequences_decomposition.hpp>

Decompose LSTMSequence to forward and reverse LSTMSequence.

class BidirectionalRNNSequenceDecomposition : public ov::pass::MatcherPass
#include <bidirectional_sequences_decomposition.hpp>

Decompose RNNSequence to forward and reverse RNNSequence.

class BidirectionalSequenceDecomposition : public ov::pass::GraphRewrite
#include <bidirectional_sequences_decomposition.hpp>

Container for all types of sequences decomposition.

class BinarizeWeights : public ov::pass::MatcherPass
#include <binarize_weights.hpp>

This transformation converts weights to -1/+1 form and applies normalization factors to output low/high and after Convolution. For example, following graph.

    .... ....  out_low  out_high           weights ..    ..  out_low out_high
      |    |      |        |                  |     |    |      |     |
     +--------------------------+           +--------------------------+
     | FakeQuantize (levels==2) |           | FakeQuantize (levels==2) |
     |     (on activations)     |           |       (on weights)       |
     +--------------------------+           +--------------------------+
                   |                                      |
                   |                                      |
                   -----------------    -------------------
                                   |    |
                                   v    v
                              +-------------+
                              | Convolution |
                              +-------------+
                                     |
                                     v
is transformed to:
             normalized normalized
    .... ....  out_low   out_high
      |    |      |         |
     +--------------------------+           +--------------------------+
     | FakeQuantize (levels==2) |           |         Constant         |
     |     (on activations)     |           | (with converted weights) |
     +--------------------------+           +--------------------------+
                   |                                      |
                   |                                      |
                   -----------------    -------------------
                                   |    |
                                   v    v
                              +-------------+
                              | Convolution |
                              +-------------+
                                     |
                                     v
                              +------------+     +---------------------------------------------------------------+
                              |  Multiply  | <---| Constant (normalization factor coming from FQ on activations) |
                              +------------+     +---------------------------------------------------------------+
                                     |
                                     v
                              +------------+     +-----------------------------------------------------------+
                              |  Multiply  | <---| Constant (normalization factor coming from FQ on weights) |
                              +------------+     +------------------------------------------------------------
                                     |
                                     v
Normalization factors are chosen based output_high value. If it’s zero - norm factor is equal to output_low and output_high otherwise

class BroadcastConstRangeReplacement : public ov::pass::MatcherPass
#include <broadcast_const_range_replacement.hpp>

BroadcastConstRangeReplacement replaces Constant filled with range values starting from 0 and replaces it with Range op.

class BroadcastElementwiseFusion : public ov::pass::MatcherPass
#include <broadcast_elementwise_fusion.hpp>

Removing Broadcast OP before ElementWise if output shape of Broadcast are equal neighboring input shape of ElementWise.

class BroadcastTransition : public ov::pass::MatcherPass
#include <broadcast_transition.hpp>

BroadcastTransition transformation moves broadcast through binary eltwise operation.

class ChainedMaximumOptimization : public ov::pass::MatcherPass
#include <chained_maximum.hpp>

Optimizes graphs based on value labels / symbols Maximum(Maximum(A, B), B) -> Maximum(A, B) Maximum(Maximum(A, B), A) -> Maximum(A, B)

class ChangePlaceholderTypes : public ov::pass::ModelPass
#include <change_placeholder_types.hpp>

Add OldApiMap with legacy type for Parameter node.

class ClampFusion : public ov::pass::MatcherPass
#include <clamp_fusion.hpp>

ClampFusion transformation replaces following graph: Maximum->Minimum to Clamp Restrictions:

  • one of the parameters to Maximum is a scalar constant

  • one of the parameters to Minimum is a scalar constant

class CommonOptimizations : public ov::pass::ModelPass
#include <common_optimizations.hpp>
class CompressFloatConstants : public ov::pass::GraphRewrite
#include <compress_float_constants.hpp>

CompressFloatConstants transformation replaces FP32/FP64 Constants with FP16 ones.

Public Functions

inline CompressFloatConstants(bool postponed = false)

Transformation constructor.

Parameters

postponed – Postponed compression, see ov::pass::CompressFloatConstantsImpl for details.

class CompressFloatConstantsImpl : public ov::pass::MatcherPass
#include <compress_float_constants.hpp>

CompressFloatConstantsImpl transformation replaces FP32/FP64 Constants with FP16 ones.

Public Functions

CompressFloatConstantsImpl(bool postponed = false)

Transformation constructor.

Parameters

postponed – If true then the transformation won’t compress the constants keeping them in the original type but still will insert Converts. This is a special mode of operation that requires another transformation to apply a real compression on constants. Constants eligible for postponed compression are marked with a special rt_info tag.

class CompressQuantizeWeights : public ov::pass::MatcherPass
#include <compress_quantize_weights.hpp>
class ConcatFusion : public ov::pass::MatcherPass
#include <concat_fusion.hpp>

ConcatFusion transformation fuses sequence of Concats.

class ConcatReduceFusion : public ov::pass::GraphRewrite
#include <concat_reduce_fusion.hpp>

ConcatReduceFusion pass replaces the following graph:

          +---------------+            +---------------+
          │               │            |               |
          │     input     │            |     input     |
          │               │            |               |
          +---------------+            +----------------
                  |                            |
                  |                            |
                  \                            /
                   \                          /
                    \                        /
                     \                      /
                      \                    /
                       \                  /
                        \                /
                         \              /
                          \            /
                         +---------------+
                         |               |
                         |     Concat    |
                         |               |
                         +----------------
                                 |
                                 v
                         +---------------+
                         |               |
                         |   ReduceMin/  |
                         |   ReduceMax   |
                         +----------------
by a single Minimum/Maximum with 2 inputs and tries to eliminate Squeeze/Unsqueeze layers before and after Min/Max.

class ConstantFolding : public ov::pass::ModelPass
#include <constant_folding.hpp>

Constant folding iterates over the function and tries to evaluate nodes with constant inputs. Such nodes are then replaced with new Constants containing the result of a folded operation.

class ConvertBatchToSpace : public ov::pass::MatcherPass
#include <convert_batch_to_space.hpp>

ConvertBatchToSpace transformation decomposes BatchToSpace layer to Reshape->Transpose->Reshape->Crop.

Param convert_by_elements

- reduces the maximum number of dimensions that arise during the transformation if enabled. Default value: true. false - BatchToSpace decomposes to Reshape->Transpose->Reshape->Crop. During transformation, the number of tensor dimensions can be increased by length of block_shape input of BatchToSpace layer. true - BatchToSpace decomposes to N x (Reshape->Transpose->Reshape)->Crop, where N = length of block_shape input of BatchToSpace layer. During transformation, the number of tensor dimensions can be increased by 1.

class ConvertBitwiseAndToLogicalAnd : public ov::pass::MatcherPass
#include <convert_bitwise_to_logical_bool.hpp>
class ConvertBitwiseNotToLogicalNot : public ov::pass::MatcherPass
#include <convert_bitwise_to_logical_bool.hpp>
class ConvertBitwiseOrToLogicalOr : public ov::pass::MatcherPass
#include <convert_bitwise_to_logical_bool.hpp>
class ConvertBitwiseXorToLogicalXor : public ov::pass::MatcherPass
#include <convert_bitwise_to_logical_bool.hpp>
class ConvertBroadcast3 : public ov::pass::MatcherPass
#include <convert_broadcast3.hpp>
class ConvertBroadcastToTiles : public ov::pass::MatcherPass
#include <convert_broadcast_to_tiles.hpp>
class ConvertCompressedOnlyToLegacy : public ov::pass::ModelPass
#include <convert_compression_only_to_legacy.hpp>

ConvertCompressedOnlyToLegacy transformation converts compression only FP16 format to legacy FP16 format.

class ConvertConvertLike : public ov::pass::MatcherPass
#include <convert_convertlike.hpp>
class ConvertDeformableConv8To1 : public ov::pass::MatcherPass
#include <convert_deformable_conv_v8_to_v1.hpp>

ConvertDeformableConv8To1 converts v8::DeformableConvolution into v1::DeformableConvolution.

class ConvertDepthToSpace : public ov::pass::MatcherPass
#include <convert_depth_to_space.hpp>
class ConvertDetectionOutput1ToDetectionOutput8 : public ov::pass::MatcherPass
#include <detection_output_upgrade.hpp>

ConvertDetectionOutput1ToDetectionOutput8 converts v0::DetectionOutput into v8::DetectionOutput.

class ConvertDetectionOutput8ToDetectionOutput1 : public ov::pass::MatcherPass
#include <detection_output_downgrade.hpp>

ConvertDetectionOutput8ToDetectionOutput1 converts v8::DetectionOutput into v0::DetectionOutput.

class ConvertDivide : public ov::pass::MatcherPass
#include <convert_divide.hpp>
class ConvertDivideWithConstant : public ov::pass::MatcherPass
#include <convert_divide.hpp>
class ConvertFP32ToFP16 : public ov::pass::ModelPass
#include <convert_fp32_to_fp16.hpp>

ConvertFP32ToFP16 transformation.

class ConvertGather0D : public ov::pass::MatcherPass
#include <convert_gather_0d.hpp>

ConvertGather0D decomposes v1::Gather operation into v0::Unsqueeze + v1::Gather + v0::Squeeze pattern when gather indices is scalar.

class ConvertGather1ToGather7 : public ov::pass::MatcherPass
#include <convert_gather_upgrade.hpp>

ConvertGather1ToGather7 converts v1::Gather into v7::Gather.

class ConvertGather7ToGather1 : public ov::pass::MatcherPass
#include <convert_gather_downgrade.hpp>

ConvertGather7ToGather1 converts v7::Gather into v1::Gather.

class ConvertGather7ToGather8 : public ov::pass::MatcherPass
#include <convert_gather_upgrade.hpp>

ConvertGather7ToGather8 converts v7::Gather into v8::Gather.

class ConvertGather8ToGather7 : public ov::pass::MatcherPass
#include <convert_gather_downgrade.hpp>

ConvertGather8ToGather7 converts v8::Gather into v7::Gather.

class ConvertGELU : public ov::pass::MatcherPass
#include <convert_gelu.hpp>
class ConvertGP9ToGPIEInternal : public ov::pass::MatcherPass
#include <convert_gp9_to_gp_ie_internal.hpp>
class ConvertGRUSequenceToTensorIterator : public ov::pass::MatcherPass
#include <convert_sequences_to_tensor_iterator.hpp>

ConvertGRUSequenceToTensorIterator transformation converts GRUSequence layer to TensorIterator *.

class ConvertInterpolate11ToInterpolate4 : public ov::pass::MatcherPass
#include <convert_interpolate11_downgrade.hpp>

Converts Interpolate version 11 to Interpolate version 4 if the new op uses any of the v4 allowed interpolation modes.

class ConvertInterpolate1ToInterpolate4 : public ov::pass::MatcherPass
#include <convert_interpolate1_to_interpolate4.hpp>

ConvertInterpolate1ToInterpolate4 covert v0:interpolate into v4::Interpolate.

class ConvertLSTMSequenceToTensorIterator : public ov::pass::MatcherPass
#include <convert_sequences_to_tensor_iterator.hpp>

ConvertLSTMSequenceToTensorIterator transformation converts LSTMSequence layer to TensorIterator *.

class ConvertMatrixNmsToMatrixNmsIE : public ov::pass::MatcherPass
#include <convert_matrix_nms_to_matrix_nms_ie.hpp>
class ConvertMaxPool1ToMaxPool8 : public ov::pass::MatcherPass
#include <convert_maxpool_upgrade.hpp>

ConvertMaxPool1ToMaxPool8 converts v1::MaxPool into v8::MaxPool.

class ConvertMaxPool8ToMaxPool1 : public ov::pass::MatcherPass
#include <convert_maxpool_downgrade.hpp>

ConvertMaxPool8ToMaxPool1 converts v8::MaxPool into v1::MaxPool.

class ConvertMinimum : public ov::pass::MatcherPass
#include <convert_minimum_to_power_and_max.hpp>
class ConvertMod : public ov::pass::MatcherPass
#include <convert_mod.hpp>
class ConvertMulticlassNms8ToMulticlassNms9 : public ov::pass::MatcherPass
#include <convert_multiclass_nms_upgrade.hpp>
class ConvertMulticlassNmsToMulticlassNmsIE : public ov::pass::MatcherPass
#include <convert_multiclass_nms_to_multiclass_nms_ie.hpp>
class ConvertMVN1ToMVN6 : public ov::pass::MatcherPass
#include <convert_mvn1_to_mvn6.hpp>

ConvertMVN1ToMVN6 covert v0:MVN into v6::MVN.

class ConvertNegative : public ov::pass::MatcherPass
#include <convert_negative.hpp>
class ConvertNMS1ToNMS5 : public ov::pass::MatcherPass
#include <convert_previous_nms_to_nms_5.hpp>
class ConvertNMS1ToNMS9 : public ov::pass::MatcherPass
#include <convert_previous_nms_to_nms_9.hpp>
class ConvertNMS3ToNMS5 : public ov::pass::MatcherPass
#include <convert_previous_nms_to_nms_5.hpp>
class ConvertNMS3ToNMS9 : public ov::pass::MatcherPass
#include <convert_previous_nms_to_nms_9.hpp>
class ConvertNMS4ToNMS5 : public ov::pass::MatcherPass
#include <convert_previous_nms_to_nms_5.hpp>
class ConvertNMS4ToNMS9 : public ov::pass::MatcherPass
#include <convert_previous_nms_to_nms_9.hpp>
class ConvertNMS5ToNMS9 : public ov::pass::MatcherPass
#include <convert_previous_nms_to_nms_9.hpp>
class ConvertNMS9ToNMSIEInternal : public ov::pass::MatcherPass
#include <convert_nms9_to_nms_ie_internal.hpp>
class ConvertNmsGatherPathToUnsigned : public ov::pass::GraphRewrite
#include <convert_nms_gather_path_to_unsigned.hpp>

Converts Gather indices to unsigned if indices are from NMS selected indices output. NMS returns -1 for not selected boxes, old version of Gather fill corresponding output for such indices with zero. But new Gather-8 has support of negative indices indicating counting from the end. In order to keep such behaviour (until dynamism is not supported) instead of -1 new Gather-8 will accept UINT32_MAX which is always outside of the bounds and corresponding output for such indices in gather always will be filled with zeros.

class ConvertNMSRotatedToNMSIEInternal : public ov::pass::MatcherPass
#include <convert_nms_rotated_to_nms_ie_internal.hpp>
class ConvertNMSToNMSIEInternal : public ov::pass::MatcherPass
#include <convert_nms_to_nms_ie_internal.hpp>
class ConvertOpSet2ToOpSet1 : public ov::pass::ModelPass
#include <convert_opset2_to_opset1.hpp>
class ConvertOpSet3ToOpSet2 : public ov::pass::ModelPass
#include <convert_opset3_to_opset2.hpp>
class ConvertPad12ToPad1 : public ov::pass::MatcherPass
#include <convert_pad12_downgrade.hpp>

Converts Pad v12 to Pad v1.

class ConvertPadToGroupConvolution : public ov::pass::MatcherPass
#include <convert_pad_to_group_conv.hpp>

ConvertPadToGroupConvolution transformation replaces Pad operation with GroupConvolution but has some restrictions on Pad parameters:

  1. PadMode must be Constant and value is equal to 0

  2. Padding must be applied only for spatial dimensions

  3. Input shape rank must be static and greater than 3

  4. Padding values must be non-negative

class ConvertPrecision : public ov::pass::ModelPass
#include <convert_precision.hpp>
class ConvertPriorBox8To0 : public ov::pass::MatcherPass
#include <convert_prior_box_v8_to_v0.hpp>

ConvertPriorBox8To1 converts v8::PriorBox into v0::PriorBox.

class ConvertQuantizeDequantize : public ov::pass::MatcherPass
#include <convert_quantize_dequantize.hpp>

ConvertQuantizeDequantize transformation replaces following graph: FakeQuantize->Convert->Convert->Subtract->Multiply with a single FakeQuantize. Restrictions:

  • quantized data type must be i8 or u8

  • ’levels’ attribute to FakeQuantize must be equal to 256

  • (output_low, output_high) must be (-128, 127) or (0, 256) (depends on sign of quantized data type)

  • ’zero_point’ and ‘scale’ must be broadcastable to FakeQuantize’s output

class ConvertReduceLogicalAndToReshape : public CvtReduceBase
#include <convert_reduce_to_reshape.hpp>
class ConvertReduceLogicalOrToReshape : public CvtReduceBase
#include <convert_reduce_to_reshape.hpp>
class ConvertReduceMaxToPooling : public ConvertReduceBase
#include <convert_reduce_to_pooling.hpp>
class ConvertReduceMaxToReshape : public CvtReduceBase
#include <convert_reduce_to_reshape.hpp>
class ConvertReduceMeanToPooling : public ConvertReduceBase
#include <convert_reduce_to_pooling.hpp>
class ConvertReduceMeanToReshape : public CvtReduceBase
#include <convert_reduce_to_reshape.hpp>
class ConvertReduceMinToReshape : public CvtReduceBase
#include <convert_reduce_to_reshape.hpp>
class ConvertReduceProdToReshape : public CvtReduceBase
#include <convert_reduce_to_reshape.hpp>
class ConvertReduceSumToPooling : public ConvertReduceBase
#include <convert_reduce_to_pooling.hpp>
class ConvertReduceSumToReshape : public CvtReduceBase
#include <convert_reduce_to_reshape.hpp>
class ConvertReduceToPooling : public ov::pass::GraphRewrite
#include <convert_reduce_to_pooling.hpp>
class ConvertReduceToReshape : public ov::pass::GraphRewrite
#include <convert_reduce_to_reshape.hpp>
class ConvertRNNSequenceToTensorIterator : public ov::pass::MatcherPass
#include <convert_sequences_to_tensor_iterator.hpp>

ConvertRNNSequenceToTensorIterator transformation converts RNNSequence layer to TensorIterator *.

class ConvertROIAlign3To9 : public ov::pass::MatcherPass
#include <convert_roi_align_v3_to_v9.hpp>

ConvertROIAlign3To9 converts v3::ROIAlign into v9::ROIAlign.

class ConvertROIAlign9To3 : public ov::pass::MatcherPass
#include <convert_roi_align_v9_to_v3.hpp>

ConvertROIAlign9To3 converts v9::ROIAlign into v3::ROIAlign.

class ConvertScatterElementsToScatter : public ov::pass::MatcherPass
#include <convert_scatter_elements_to_scatter.hpp>

ConvertScatterElementsToScatter convert opset3::ScatterElementsUpdate to opset3::ScatterUpdate.

class ConvertScatterElementsUpdate12ToScatterElementsUpdate3 : public ov::pass::MatcherPass
#include <convert_scatter_elements_update12_downgrade.hpp>

Converts Pad v12 to Pad v1.

class ConvertSequenceToTensorIterator : public ov::pass::GraphRewrite
#include <convert_sequences_to_tensor_iterator.hpp>
class ConvertShapeOf3 : public ov::pass::MatcherPass
#include <convert_shapeof3.hpp>
class ConvertShuffleChannels3 : public ov::pass::MatcherPass
#include <convert_shuffle_channels3.hpp>
class ConvertSoftMax1ToSoftMax8 : public ov::pass::MatcherPass
#include <convert_softmax_upgrade.hpp>

ConvertSoftMax1ToSoftMax8 converts v1::SoftMax into v8::SoftMax.

class ConvertSoftMax8ToSoftMax1 : public ov::pass::MatcherPass
#include <convert_softmax_downgrade.hpp>

ConvertSoftMax8ToSoftMax1 converts v8::SoftMax into v1::SoftMax.

class ConvertSpaceToBatch : public ov::pass::MatcherPass
#include <convert_space_to_batch.hpp>

ConvertSpaceToBatch transformation decomposes SpaceToBatch layer to Pad->Reshape->Transpose->Reshape.

Param convert_by_elements

- reduces the maximum number of dimensions that arise during the transformation if enabled. Default value: true. false - SpaceToBatch decomposes to Pad->Reshape->Transpose->Reshape. During transformation, the number of tensor dimensions can be increased by length of block_shape input of SpaceToBatch layer. true - SpaceToBatch decomposes to Pad-> N x (Reshape->Transpose->Reshape), where N = length of block_shape input of SpaceToBatch layer. During transformation, the number of tensor dimensions can be increased by 1.

class ConvertSpaceToDepth : public ov::pass::MatcherPass
#include <convert_space_to_depth.hpp>
class ConvertSubtract : public ov::pass::MatcherPass
#include <convert_subtract.hpp>
class ConvertSubtractWithConstant : public ov::pass::MatcherPass
#include <convert_subtract.hpp>
class ConvertTensorIteratorToGRUSequence : public ov::pass::MatcherPass
#include <convert_ti_to_sequences.hpp>

Finds all TensorIterator layers, detects the pattern Squeeze->GRUCell->Unsqueeze in the TensorIterator body, converts this pattern to GRUSequence layer and replaces them TensorIterator.

class ConvertTensorIteratorToLSTMSequence : public ov::pass::MatcherPass
#include <convert_ti_to_sequences.hpp>

Finds all TensorIterator layers, detects the pattern Squeeze->LSTMCell->Unsqueeze in the TensorIterator body, converts this pattern to LSTMSequence layer and replaces them TensorIterator.

class ConvertTensorIteratorToRNNSequence : public ov::pass::MatcherPass
#include <convert_ti_to_sequences.hpp>

Finds all TensorIterator layers, detects the pattern Squeeze->RNNCell->Unsqueeze in the TensorIterator body, converts this pattern to RNNSequence layer and replaces them TensorIterator.

class ConvertTensorIteratorToSequence : public ov::pass::GraphRewrite
#include <convert_ti_to_sequences.hpp>
class ConvertTopK11ToTopK3 : public ov::pass::MatcherPass
#include <convert_topk11_downgrade.hpp>

Converts TopK version 11 to TopK version 3 if TopK 11 stable attribute is set to false.

class ConvertTopK3 : public ov::pass::MatcherPass
#include <convert_topk3.hpp>
class ConvertU4WeightsZeroPointToScalar : public ov::pass::MatcherPass
#include <convert_u4_weights_zero_point_to_scalar.hpp>

Converts U4 weights zero point to scalar if all values are equal.

class ConvertXorToLogicalXor : public ov::pass::MatcherPass
#include <convert_xor_to_logical_xor.hpp>

ConvertXorToLogicalXor converts v0::Xor to v1::LogicalXor.

class ConvolutionBackpropDataMultiplyFusion : public ov::pass::MatcherPass
#include <conv_mul_fusion.hpp>
class ConvolutionMultiplyFusion : public ov::pass::MatcherPass
#include <conv_mul_fusion.hpp>
class ConvolutionToGroupConvolutionFusion : public ov::pass::MatcherPass
#include <convolution_to_group_convolution_fusion.hpp>

ConvolutionToGroupConvolutionFusion transformation replaces following graph: Split (or VariadicSplit) / \ Conv … Conv \ / \ / Concat.

to GroupConvolution

class ConvStridesPropagation : public ov::pass::MatcherPass
#include <strides_optimization.hpp>

ConvStridesPropagation either propagates stride (greater than 1) from Convolution up through the graph or inserts pooling between current node and its consumers if the consumers have different StridesProp attributes. Strides can be propagated if Convolution kernel is {1, 1, …}.

class ConvToBinaryConv : public ov::pass::MatcherPass
#include <conv_to_binary_conv.hpp>

This transformation converts Convolution to BinaryConvolution under following conditions:

  • first input to Convolution is a FakeQuantize with levels==2 with output low,high being either (0, 1) or (-1, 1)

  • second input (weights) is a constant with values -1 or 1 The transformation also converts weights to binary Constant (with ‘u1’ type) For example, when output_low is equal to 0 and output_high is equal to 1, following graph

     .... ....  out_low   out_high
       |    |      |         |
      +--------------------------+           +-------------------------------------+
      | FakeQuantize (levels==2) |           |               Constant              |
      |     (on activations)     |           | (weights containing -1 or 1 values) |
      +--------------------------+           +-------------------------------------+
                    |                                      |
                    |                                      |
                    -----------------    -------------------
                                    |    |
                                    v    v
                               +-------------+
                               | Convolution |
                               +-------------+
                                      |
                                      v
    
    is transformed to:
     .... ....  out_low   out_high
       |    |      |         |
      +--------------------------+           +---------------------------------+
      | FakeQuantize (levels==2) |           |     Constant (with u1 type)     |
      |     (on activations)     |           | (with u1 type - binary weights) |
      +--------------------------+           +---------------------------------+
                    |                                      |
                    |                                      |
                    -----------------    -------------------
                                    |    |
                                    v    v
                            +-------------------+
                            | BinaryConvolution |
                            +-------------------+
                                      |
                                      v
                               +------------+     +----------------------------------------------------+
                               |            |     |                   Constant                         |
                               |     Add    | <---|          (weights from original graph,             |
                               |            |     |  sum-reduced over [1,..., len(weights.shape)] axes |
                               +------------+     +----------------------------------------------------+
                                      |
                                      v
                               +------------+     +-----+
                               |  Multiply  | <---| 0.5 |
                               +------------+     +-----+
                                      |
                                      v
    

class DepthToSpaceFusion : public ov::pass::MatcherPass
#include <depth_to_space_fusion.hpp>

DepthToSpaceFusion transformation detects Reshape-Transpose-Reshape pattern and tries to fuse it into a single DepthToSpace layer.

DepthToSpaceFusion transformation is optional and disabled by default. The transformation can be enabled with callback using setCallback method. See the example below.

Callback example:

// This callback enables DepthToSpaceFusion transformation
auto callback = [](const std::shared_ptr<const ov::Node> & node) -> bool {
    return std::dynamic_pointer_cast<const ov::opset3::DepthToSpace>(node) != nullptr;
};

auto p = ov::pass::DepthToSpaceFusion();
p.setCallback(callback);
p.run_on_function(f);

class DeReshapeFullyConnected : public ov::pass::MatcherPass
#include <dereshape_matmul.hpp>

Transformation uses symbol / label information to optimize out Reshape operations surrounding special cases of MatMul. It checks that surrounding Reshapes are only manipulating with batch dimensions of tensor in a do-undo kind of way. The difference with previous optimization is that this case has Reshape only on one input of MatMul and the other input is strictly 2D. Such MatMuls are also called FullyConnected.

Example: Before: [A,B,4096] -> Reshape -> [A*B,4096] MatMul [A*B,4608] -> Reshape -> [A,B,4608] [4096,4608]

After: [A,B,4096] -> MatMul -> [A,B,4608] [4096,4608] ->

class DeReshapeMatMul : public ov::pass::MatcherPass
#include <dereshape_matmul.hpp>

Transformation uses symbol / label information to optimize out Reshape operations surrounding MatMul. It checks that surrounding Reshapes are only manipulating with batch dimensions of tensor in a do-undo kind of way.

Example: Before: [A,B,C,D] -> Reshape -> [A*B,C,D] MatMul [A*B,C,E] -> Reshape -> [A,B,C,E] [A,B,D,E] -> Reshape -> [A*B,D,E]

After: [A,B,C,D] -> MatMul -> [A,B,C,E] [A,B,D,E] ->

Transformation allows slightly different variations of the pattern on inputs of MatMul.

  • Simplest pattern contains only Reshape operation on MatMul input: Reshape -> MatMul

  • The next acceptable variation is Concat of two inputs on MatMul input: Reshape -[-> Concat -]-> MatMul This variation would be transformed with realignment of the other input of Concat and the other outputs of Concat with the help of Reshape operations

  • The most complex variation on the MatMul input pattern is with Binary Elementwise Operation with scalar second input: Reshape -[-> Concat -]-[-> BEA (scalar) -]-> MatMul

Additionally, transformation supports variation of the pattern on output of MatMul. It allows for Binary Elementwise Arithmetic operation without second input scalar restriction. MatMul -[-> BEA -]-> Reshape this pattern variation is only applicable for the case when input reshapes are 4D -> 3D and output reshape is 3D -> 4D. Additionally, shape labels on output of MatMul should be equal to the input shape labels of the last Reshape, meaning that this Binary Elementwise Arithmetic doesn’t perform any broadcasting of input coming from MatMul &#8212; only other input may be broadcasted to the MatMul input of this BEA. This effect (equality of MatMul output shape labels and output shape of BEA) is being handled by LabelResolvingThroughSelect transformation in the particular models that this variation targets.

Full pattern this transformation searches for: -> Reshape -[-> Concat -]-[-> BEA (scalar) -]-> MatMul -[-> BEA -]-> Reshape -> -> Reshape -[-> Concat -]-[-> BEA (scalar) -]->

NOTE: input branches could be (and in observed model cases are) asymmetrical, meaning that the presence of Concat on one input of MatMul doesn’t require the other input to also have Concat

class DilatedConvolutionConverter : public ov::pass::MatcherPass
#include <dilated_convolution_converter.hpp>

DilatedConvolutionConverter transformation replaces following graph: SpaceToBatch -> Convolution(GroupConvolution) -> BatchToSpace to a single Convolution(GroupConvolution) node with updated pads and dilations Restrictions:

  • pads in SpaceToBatch must have 0 on first and second position

class DisableConstantFolding : public ov::RuntimeAttribute
#include <constant_folding.hpp>
class DisableDecompressionConvertConstantFolding : public ov::pass::MatcherPass
#include <mark_decompression_convert_constant_folding.hpp>

Disables ConstantFolding for Convert operation in compressed function.

class DisableFoldSubgraphEmptyInputs : public ov::RuntimeAttribute
#include <fold_subgraph_empty_inputs.hpp>
class DisableRandomUniformConstantFolding : public ov::pass::MatcherPass
#include <disable_random_uniform_constant_folding.hpp>

Disables ConstantFolding for RandomUniform operation. It is required as RandomUniform should generate new sequence each run.

class DisableRemoveConcatZeroDimInput : public ov::RuntimeAttribute
#include <remove_concat_zero_dim_input.hpp>
class DisableShapeOfConstantFolding : public ov::pass::MatcherPass
#include <disable_shapeof_constant_folding.hpp>
class DivideFusion : public ov::pass::MatcherPass
#include <divide_fusion.hpp>

DivideFusion transformation replaces a sub-graph Pow(y, -1) * x or x * Pow(y, -1) with Divide(x,y)

class DropoutWithRandomUniformReplacer : public ov::pass::MatcherPass
#include <dropout_with_random_uniform_replacer.hpp>

This transformation replaces possible Dropout block (in inference mode) with RandomUniform to Broadcast of half-ones in a sub-graph.

Dropout block: RandomUniform ——-&#8212;> Add &#8212;> Floor /\ /\ /\ | | | Const(0) Const(1) Const(1) min_val max_val

Resulted block: Broadcast —-&#8212;> Add &#8212;> Floor /\ /\ | | Const(0.5) Const(1)

class EinsumDecomposition : public ov::pass::MatcherPass
#include <einsum_decomposition.hpp>

EinsumDecomposition transformation decomposes Einsum-7 operation into a sub-graph with more simple operations: Transpose, Reshape, MatMul, ReduceSum, Unsqueeze, ShapeOf, ReduceProd, StridedSlice, and Concat.

class EliminateConcat : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminateConcat eliminates concat that does nothing.

class EliminateConvert : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminateConvert eliminates convert that does nothing.

class EliminateConvertNonZero : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminateConvertNonZero eliminates convert before NonZero.

class EliminateDuplicateTIInputs : public ov::pass::MatcherPass
#include <eliminate_duplicate_ti_inputs.hpp>
class EliminateEltwise : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminateEltwise eliminates eltwise ops that do nothing.

class EliminateGatherUnsqueeze : public ov::pass::MatcherPass
#include <eliminate_unsqueeze_gather.hpp>

Matches Gather ->[Binary Operation]-> Unsqueeze If axis for Gather and Unsqueeze is the same and Gather indices are scalar Unsqueeze is being removed and indices become 1D. Must be executed after SharedOpOptimization &#8212; It is possible to have multiple similar Unsqueeze operations after Gather, so they must be optimized beforehand.

class EliminateNopBroadcast : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminateNopBroadcast eliminates broadcast or tile with all ones on the second input.

class EliminatePad : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminatePad eliminates pad that does nothing.

class EliminateScatterUpdate : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminateScatterUpdate eliminates scatter ops that do nothing (updates/indices are empty)

class EliminateSplit : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminateSplit eliminates split that does nothing.

class EliminateSplitConcat : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminateSplit eliminates split+concat pairs which do nothing.

class EliminateSqueeze : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminateSqueeze eliminates squeeze that does nothing.

class EliminateTranspose : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

EliminateTranspose eliminates transpose that does nothing.

class EliminateUnsqueezeGather : public ov::pass::MatcherPass
#include <eliminate_unsqueeze_gather.hpp>

Remove Unsqueeze + Gather pair, if Gather gathers data by dimension that was previously added by Unsqueeze.

class EnableDecompressionConvertConstantFolding : public ov::pass::MatcherPass
#include <mark_decompression_convert_constant_folding.hpp>

Enables ConstantFolding for Convert operation in compressed function.

class EnableShapeOfConstantFolding : public ov::pass::MatcherPass
#include <enable_shapeof_constant_folding.hpp>

This transformation enables constfoldability for ShapeOf nodes that was disabled by DisableShapeOfConstantFolding.

class EyeDecomposition : public ov::pass::MatcherPass
#include <eye_decomposition.hpp>

Do eye decomposition to sub-graph (model).

class FakeQuantizeDecomposition : public ov::pass::MatcherPass
#include <fq_decomposition.hpp>

FakeQuantizeDecomposition transformation decomposes FakeQuantize layer.

Expression from specification: if x <= min(input_low, input_high): output = output_low elif x > max(input_low, input_high): output = output_high else: output = round((x - input_low) / (input_high - input_low) * (levels-1)) / (levels-1) * (output_high - output_low) + output_low

expand brackets into round: round(x * (levels-1) / (input_high - input_low) - input_low * (levels-1) / (input_high - input_low)) div on (levels-1) and mult on (output_high - output_low) => mult on (output_high - output_low) / (levels-1)

=> round(x * (levels-1) / (input_high - input_low) - input_low * (levels-1) / (input_high - input_low)) * (output_high - output_low) / (levels-1) + output_low

This transformation doesn’t support following cases:

  1. At least one ‘range’ input is not Constant

  2. At least one ‘input_low’ input value greater or equal than ‘input_high’ input value

class FakeQuantizeMulFusion : public ov::pass::MatcherPass
#include <fq_mul_fusion.hpp>

This transformation looks for a FQ + Mul pair in the graph and moves the Mul operation above the FQ node. The last two inputs of FQ are multiplied by the value that was originally below the FQ node.

class FakeQuantizeReshapeFusion : public ov::pass::MatcherPass
#include <fq_reshape_fusion.hpp>

This transformation looks for a FQ + Reshape pair in the graph and moves the Reshape operation above the FQ node. Shapes of limit inputs are updated following FQ broadcasting semantics.

class FindBatch : public ov::pass::ModelPass
#include <dimension_tracking.hpp>
class FlushFP32SubnormalsToZero : public ov::pass::MatcherPass
#include <flush_fp32_subnormals_to_zero.hpp>
class FoldSubgraphEmptyInputs : public ov::pass::MatcherPass
#include <fold_subgraph_empty_inputs.hpp>
class FusedNamesCleanup : public ov::pass::ModelPass
#include <fused_names_cleanup.hpp>

FusedNamesCleanup removes fused_names attribute.

class FuseFilteringBoxesBySize : public ov::pass::GraphRewrite
#include <remove_filtering_boxes_by_size.hpp>
class GatherNopElimination : public ov::pass::MatcherPass
#include <simplify_shape_of_sub_graph.hpp>

GatherNopElimination transformation optimizes out useless Gather operations.

class Gelu7Downgrade : public ov::pass::MatcherPass
#include <gelu7_downgrade.hpp>

Gelu7Downgrade converts v7::Gelu operation to v2::Gelu unconditionally. This is done because only limited set of plugins support v7::Gelu which has an attribute specifying approximation mode. For other plugins the behaviour is to use v2 version of the operation which does not support the approximation mode.

class GeluFusion : public ov::pass::GraphRewrite
#include <gelu_fusion.hpp>

GeluFusion transformation replaces various sub-graphs with a Gelu op.

class GeluFusionWithErfFour : public ov::pass::MatcherPass
#include <gelu_fusion.hpp>

GeluFusion transformation replaces a sub-graph x * (0.5 + 0.5 * erf(x * (1 / sqrt(2)))) with a Gelu op.

class GeluFusionWithErfOne : public ov::pass::MatcherPass
#include <gelu_fusion.hpp>

GeluFusion transformation replaces a sub-graph (0.5 * x) * (1 + erf(x / sqrt(2))) with a Gelu op.

class GeluFusionWithErfThree : public ov::pass::MatcherPass
#include <gelu_fusion.hpp>

GeluFusion transformation replaces a sub-graph x * (0.5 * (1 + erf(x / sqrt(2)))) with a Gelu op.

class GeluFusionWithErfTwo : public ov::pass::MatcherPass
#include <gelu_fusion.hpp>

GeluFusion transformation replaces a sub-graph 0.5 * (x * (1 + erf(x / sqrt(2)))) with a Gelu op.

class GeluFusionWithTanh : public ov::pass::MatcherPass
#include <gelu_fusion.hpp>

GeluFusion transformation replaces a sub-graph x * (0.5 * (1 + tanh([sqrt(2 / pi)] * [x + 0.044715^3]))) with a Gelu (Tanh) op.

class GeluFusionWithTanhNoPower : public ov::pass::MatcherPass
#include <gelu_fusion.hpp>

GeluFusion transformation replaces a sub-graph x * 0.5 * (1 + tanh((x * 0.044715 * x + 1) * x * sqrt(2 / pi))) with a Gelu (Tanh) op.

class GraphRewrite : public ov::pass::ModelPass
#include <graph_rewrite.hpp>

GraphRewrite is a container for MatcherPasses that allows to run them on Function in efficient way.

Graph rewrite pass is used for matcher passes execution on Function. To register MatcherPass use

See also

add_matcher<T>(args) method where T is a MatcherPass class. As a default algorithm graph rewrite pass traverse Function in topological order and applies registered matcher passes for each node. But if all registered matcher passes have type based root node in Matcher pattern then efficient mechanism is used to execute them. Matcher pattern root is type based if it’s operation from opset or pattern::op::WrapType. Note: when implementing pattern for Matcher make sure that root node is an operation from opset or has ov::pattern::op::WrapType. That will help GraphRewrite to execute matcher passes more efficient.

Subclassed by ConvertBitwiseToLogical, ov::pass::BackwardGraphRewrite, ov::pass::BidirectionalSequenceDecomposition, ov::pass::CompressFloatConstants, ov::pass::ConcatReduceFusion, ov::pass::ConvertNmsGatherPathToUnsigned, ov::pass::ConvertReduceToPooling, ov::pass::ConvertReduceToReshape, ov::pass::ConvertSequenceToTensorIterator, ov::pass::ConvertTensorIteratorToSequence, ov::pass::FuseFilteringBoxesBySize, ov::pass::GeluFusion, ov::pass::HSigmoidFusion, ov::pass::HSwishFusion, ov::pass::InitMasks, ov::pass::LinOpSequenceFusion, ov::pass::MVNFusion, ov::pass::NopElimination, ov::pass::PReluFusion, ov::pass::PadFusion, ov::pass::PropagateMasks, ov::pass::PullThroughReduce, ov::pass::SwishFusion, ov::pass::TransposeSinking, ov::pass::low_precision::TypeRelaxedReplacer, ov::pass::transpose_sinking::TSGeneralBackward, ov::pass::transpose_sinking::TSGeneralForward

Public Functions

template<typename T, bool Enabled = true, class ...Args, typename std::enable_if<std::is_base_of<pass::MatcherPass, T>::value, bool>::type = true>
inline std::shared_ptr<T> add_matcher(Args&&... args)

Register given transformation class type to GraphRewrite execution list All registered transformations will be executed in a single graph traversal. Example below show the basic usage of pass::GraphRewrite.

pass::Manager manager;
auto anchor = manager.register_pass<GraphRewrite>();
anchor->add_matcher<MatcherPassA>();
anchor->add_matcher<MatcherPassB>();
anchor->set_name("CommonMatchers");
manager.run_passes(f);
For some purposes transformation can be registered and disabled by default.
anchor->add_matcher<MatcherPassB, false>();

Returns

shared_ptr to the transformation instance

template<typename T, class ...Args, typename std::enable_if<std::is_base_of<pass::GraphRewrite, T>::value, bool>::type = true>
inline void add_matcher(Args&&... args)

Register passes from GraphRewrite class that contains sequence of matcher passes registered in its ctor. For example:

class ov::pass::LinFusions: public ov::pass::GraphRewrite { public: OPENVINO_RTTI(“LinFusion”); Fusions() { add_matcher<ov::pass::AddFusion>(); add_matcher<ov::pass::MulFusion>(); } };

pass::Manager manager; auto anchor = manager.register_pass<GraphRewrite>(); anchor->add_matcher<LinFusions>(); anchor->add_matcher<OtherFusions>(); anchor->set_name(“CommonFusions”); manager.run_passes(f);

In this case all matcher passes from LinFusions pass will be united with other registered matchers.

virtual void set_pass_config(const std::shared_ptr<PassConfig> &pass_config) override

Set PassConfig for particular transformation instance.

Parameters

pass_config – is a PassConfig shared_ptr

class GroupConvolutionBackpropDataMultiplyFusion : public ov::pass::MatcherPass
#include <conv_mul_fusion.hpp>
class GroupConvolutionMultiplyFusion : public ov::pass::MatcherPass
#include <conv_mul_fusion.hpp>
class GroupedGatherElimination : public ov::pass::MatcherPass
#include <simplify_shape_of_sub_graph.hpp>

GroupedGatherElimination transformation replaces group of Gather operations with the first Gather in this group and updated indices input in case all Gathers in the group are consumed by the same Concat in incremental order.

class GroupedSliceToVSplitOptimization : public ov::pass::ModelPass
#include <optimize_strided_slice.hpp>

GroupedSliceToVSplitOptimization transformation replaces group of Slice operations with VariadicSplit. All Slice operations must slice data with the same axis and step = 1.

class GroupedStridedSliceOptimizer : public ov::pass::ModelPass
#include <optimize_strided_slice.hpp>

GroupedStridedSliceOptimizer transformation replaces group of StridedSlice operations with VariadicSplit. All StridedSlice operations must slice data with the same axis and stride = 1.

class GroupNormalizationDecomposition : public ov::pass::MatcherPass
#include <group_normalization_decomposition.hpp>
class GRUCellDecomposition : public ov::pass::MatcherPass
#include <gru_cell_decomposition.hpp>

GRUCellDecomposition transformation decomposes GRUCell layer with inputs X, H, W, R, B to Add, Split, MatMul, Multiply and Subtract ops according to the formula: (.) - Denotes element-wise multiplication.

  • Denotes dot product. f, g - are activation functions

zt = f(Xt*(Wz^T) + Ht-1*(Rz^T) + Wbz + Rbz) rt = f(Xt*(Wr^T) + Ht-1*(Rr^T) + Wbr + Rbr) ht = g(Xt*(Wh^T) + (rt (.) Ht-1)*(Rh^T) + Rbh + Wbh) # when linear_before_reset := false # (default) ht = g(Xt*(Wh^T) + (rt (.) (Ht-1*(Rh^T) + Rbh)) + Wbh) # when linear_before_reset:= true Ht = (1 - zt) (.) ht + zt (.) Ht-1

class GRUCellFusion : public ov::pass::MatcherPass
#include <gru_cell_fusion.hpp>

GRUCellFusion transformation replaces a sequence of operations with GRUCell op.

If BiasAdds are not present in the pattern, then Constants with zero values will be created to match the specification.

Supported activations: Relu, Sigmoid, Tanh Clip attribute is not supported. Linear_before_reset attribute is not supported. Supported weights formats: zr, rz

class Hash : public ov::pass::ModelPass
#include <hash.hpp>

Hash transformation calculates hash value for ov::Model.

Public Functions

Hash(uint64_t &output_hash_value)

Hash pass constructor.

Parameters

output_hash_value – Reference to output value. By applying hash pass on function, resulting hash value will be set to this variable

class HSigmoidDecomposition : public ov::pass::MatcherPass
#include <hsigmoid_decomposition.hpp>

HSigmoidDecomposition transformation into sub-graph (min(Relu(x + 3), 6) * const(1/6).

class HSigmoidFusion : public ov::pass::GraphRewrite
#include <hsigmoid_fusion.hpp>

HSigmoidFusion transformation replaces various sub-graphs with a HSigmoid op.

class HSigmoidFusionWithClampDiv : public ov::pass::MatcherPass
#include <hsigmoid_fusion.hpp>

HSigmoidFusion transformation replaces a sub-graph (Clamp(x + 3, 0, 6) * / 6) with a HSigmoid op.

class HSigmoidFusionWithClampMul : public ov::pass::MatcherPass
#include <hsigmoid_fusion.hpp>

HSigmoidFusion transformation replaces a sub-graph (Clamp(x + 3, 0, 6) * const(1/6)) with a HSigmoid op.

class HSigmoidFusionWithoutRelu : public ov::pass::MatcherPass
#include <hsigmoid_fusion.hpp>

HSigmoidFusion transformation replaces a sub-graph (min(max(x + 3, 0), 6) / 6) with a HSigmoid op.

class HSigmoidFusionWithReluDiv : public ov::pass::MatcherPass
#include <hsigmoid_fusion.hpp>

HSigmoidFusion transformation replaces a sub-graph ((min(Relu(x + 3), 6)) / 6) with a HSigmoid op.

class HSigmoidFusionWithReluMul : public ov::pass::MatcherPass
#include <hsigmoid_fusion.hpp>

HSigmoidFusion transformation replaces a sub-graph ((min(Relu(x + 3), 6)) * const(1/6)) with a HSigmoid op.

class HSwishDecomposition : public ov::pass::MatcherPass
#include <hswish_decomposition.hpp>

HSwishDecomposition transformation into sub-graph x * (min(Relu(x + 3), 6) * const(1/6).

class HSwishFusion : public ov::pass::GraphRewrite
#include <hswish_fusion.hpp>

HSwishFusion transformation replaces various sub-graphs with a HSwish op.

class HSwishFusionWithClamp : public ov::pass::MatcherPass
#include <hswish_fusion.hpp>

HSwishFusion transformation replaces a sub-graph (Clamp(x + 3, 0, 6) * x) with a HSwish * 6.

class HSwishFusionWithHSigmoid : public ov::pass::MatcherPass
#include <hswish_fusion.hpp>

HSwishFusion transformation replaces a sub-graph x * HSigmoid(x) with a HSwish op.

class HSwishFusionWithReluDiv : public ov::pass::MatcherPass
#include <hswish_fusion.hpp>

HSwishFusion transformation replaces a sub-graph (x * (min(Relu(x + 3), 6))) / 6 with a HSwish op.

class HSwishFusionWithReluMul : public ov::pass::MatcherPass
#include <hswish_fusion.hpp>

HSwishFusion transformation replaces a sub-graph (x * (min(Relu(x + 3), 6)) * const(1/6) with a HSwish op.

class InitConstMask : public ov::pass::MatcherPass
#include <pruning.hpp>

Check Constant operation values by given dimensions and set masks according to results that are bases on condition lambda function. Works for Constant with floating point type (f16, f32, f64).

class InitMasks : public ov::pass::GraphRewrite
#include <pruning.hpp>

Initialising masks for pruned operations.

class InitNodeInfo : public ov::pass::ModelPass
#include <init_node_info.hpp>

InitNodeInfo transformation helps to set runtime info attributes in a single place.

Every runtime info attribute that needs to be initialized should be registered in run_on_function method. Also do not forget to override init methods for registered attribute. This transformations should be called first in transformation pipeline. If attribute was already set initialization will be skipped for this node.

class InterpolateSequenceFusion : public ov::pass::MatcherPass
#include <interpolate_sequence_fusion.hpp>

InterpolateSequenceFusion transformation replaces a sequence of operations to Interpolate op.

class KeepConstAndDecompression : public ov::pass::MatcherPass
#include <mark_decompression_convert_constant_folding.hpp>

Disables ConstantFolding for Convert operation and prevents conversion of f16 Consts to f32.

class KeepConstantsPrecisionAndAddConverts : public ov::pass::MatcherPass
#include <mark_decompression_convert_constant_folding.hpp>

Prevents Consts precision conversion and adds Convert with disabled ConstantFolding.

class LabelResolvingThroughSelect : public ov::pass::MatcherPass
#include <symbolic_optimizations.hpp>

Transformation requires equal labels on one input of Add and output of last Reshape in the pattern: -> Add -> Reshape -[then or else input]-> Select -> Softmax -> Reshape ->

If shape labels onn mentioned tensors are equal we proved that no broadcasting of this input was done for Add and for Select. Therefore, we can put the same labels on the output of Add and Select. This transformation helps propagate labels and will not be needed if we would use information on equality of products of input and output dimensions of Reshape operations

class LeakyReluFusion : public ov::pass::MatcherPass
#include <leaky_relu_fusion.hpp>

LeakyReluFusion transformation replaces following graph: Multiply->Maximum to LeakyRelu.

class LinOpSequenceFusion : public ov::pass::GraphRewrite
#include <lin_op_sequence_fusion.hpp>

LinOpSequenceFusion transformation fuses linear operation sequence.

class LogSoftmaxDecomposition : public ov::pass::MatcherPass
#include <log_softmax_decomposition.hpp>

LogSoftmaxDecomposition transformation into sub-graph x - log(reduce_sum(exp(x), axis)).

class LowLatency2 : public ov::pass::ModelPass
#include <low_latency.hpp>

The transformation finds all TensorIterator/Loop layers in the network, processes all back edges that describe a connection between Result and Parameter of the TensorIterator/Loop bodies,and inserts ReadValue and Assign layers at the input and output corresponding to this back edge. Supported platforms: CPU, GNA.

The example below describes the changes made by the transformation [] - TensorIterator body () - new layer BE - back-edge

before applying the transformation: -> input1[BE_1 -> Parameter -> Layers … -> Result -> BE_1 ]output1->

after applying the transformation: ->(ReadValue)-> input1[BE_1 ->Parameter->Layers …->Result->BE_1]output1 ->(Assign) \ ->… After applying the transformation, the resulting network can be inferred step by step, the states will store between inferences.

class LSTMCellDecomposition : public ov::pass::MatcherPass
#include <lstm_cell_decomposition.hpp>

LSTMCellDecomposition transformation decomposes LSTMCell layer with inputs X, H, C, W, R, B to Add, Split, MatMul, Multiply ops according to the formula: (.) - Denotes element-wise multiplication.

  • Denotes dot product. f, g, h - are activation functions.

it = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Wbi + Rbi) ft = f(Xt*(Wf^T) + Ht-1*(Rf^T) + Wbf + Rbf) ct = g(Xt*(Wc^T) + Ht-1*(Rc^T) + Wbc + Rbc) ot = f(Xt*(Wo^T) + Ht-1*(Ro^T) + Wbo + Rbo) Ct = ft (.) Ct-1 + it (.) ct Ht = ot (.) h(Ct)

class LSTMCellFusion : public ov::pass::MatcherPass
#include <lstm_cell_fusion.hpp>

LSTMCellFusion transformation replaces a sequence of operations with LSTMCell op.

class LSTMStatesBroadcast : public ov::pass::ModelPass
#include <lstm_states_broadcast.hpp>

In case LSTMCell has constant initial hidden and cell state with single batch size we make them broadcast-able by batch.

class MakeStateful : public ov::pass::ModelPass
#include <make_stateful.hpp>

The transformation replaces the provided pairs Parameter and Result with Memory layers ReadValue and Assign.

class Manager
#include <manager.hpp>

Manager class allows to manage transformation passes.

Public Functions

template<typename T, bool Enable = true, class ...Args>
inline std::shared_ptr<T> register_pass(Args&&... args)

Register given transformation class type to execution list Example below show the basic usage of pass::Manager.

pass::Manager manager;
manager.register_pass<MyTransformation>(/*transformation constructor ars*&zwj;/);
manager.run_passes(f);
For some purposes transformation can be registered and disabled by default.
manager.register_pass<MyTransformation, false>();

Returns

shared_ptr to the transformation instance

bool run_passes(std::shared_ptr<Model> model)

Runs registered transformations on a given model.

Parameters

modelInput model

Returns

Returns true if the model was changed by transformations, false otherwise.

void set_per_pass_validation(bool new_state)

Set flag to enable/disable running Validate pass after executing each registered pass.

Parameters

new_state – Value “true” enables Validate pass run; “false”, otherwise

inline std::shared_ptr<PassConfig> get_pass_config()
Returns

PassConfig shared object. This object is used for transformations pipeline configuration. This object allows to disable/enable transformations execution, set callback to particular transformation. For mo details see PassConfig class.

class MarkCompressedFloatConstants : public ov::pass::MatcherPass
#include <mark_decompression_convert_constant_folding.hpp>

Prevents ConstantFolding for f16/bf16 Const + Convert_To_FP32 to keep original FW float Constants. Original precision should be kept as long as possible, this prevents redundant conversions and saves memory. E.g. if original FW model was already compressed no need to upcast during CF, store intermediate f32 consts and then again compress them to f16 during save_model.

class MarkDequantizationSubgraph : public ov::pass::MatcherPass
#include <mark_dequantization_subgraph.hpp>

MarkDequantizationSubgraph marks dequantization subgraph, that is: Convert->Subtract(optional)->Multiply in two ways:

class MarkDividesInShapeSubgraphs : public ov::pass::MarkPrecisionSensitiveShapeOfSubgraphs
#include <mark_precision_sensitive_shapeof_subgraphs.hpp>

MarkDividesInShapeSubgraphs marks the Divide layers inside of all shape subgraphs starting from precision-sensitive input and ending at the ShapeOf node as disabled for ConvertDivide transformation.

class MarkPrecisionSensitiveConstants : public ov::pass::MarkPrecisionSensitiveShapeOfSubgraphs
#include <mark_precision_sensitive_shapeof_subgraphs.hpp>

MarkPrecisionSensitiveConstants marks the constants inside of all shape subgraphs starting from precision-sensitive inputs and ending at the ShapeOf node as disabled for FP16 compression.

class MarkPrecisionSensitiveShapeOfSubgraphs : public ov::pass::ModelPass
#include <mark_precision_sensitive_shapeof_subgraphs.hpp>

MarkPrecisionSensitiveShapeOfSubgraphs marks entirely all shape subgraphs starting from precision-sensitive inputs and ending at the ShapeOf node as disabled for FP16 compression.

Subclassed by ov::pass::MarkDividesInShapeSubgraphs, ov::pass::MarkPrecisionSensitiveConstants, ov::pass::MarkShapeOfSubgraphs

class MarkShapeOfSubgraphs : public ov::pass::MarkPrecisionSensitiveShapeOfSubgraphs
#include <mark_precision_sensitive_shapeof_subgraphs.hpp>

MarkShapeOfSubgraphs marks shape subgraphs. Information whether the node belongs to the shape path or to the data path is needed during evaluate and CF.

class MarkSugraphsToKeepInMixedPrecision : public ov::pass::ModelPass
#include <mark_subgraphs_to_keep_in_mixed_precision.hpp>
class MatcherPass : public ov::pass::PassBase
#include <graph_rewrite.hpp>

MatcherPass is a basic block for pattern based transformations. It describes pattern and action that is applied if pattern is matched.

MatcherPass consists of Matcher and matcher_pass_callback that needs to be implemented and finally registered by using

See also

register_matcher. MatcherPass can be executed on node within

See also

apply method. To run matcher pass on Function use GraphRewrite. In addition MatcherPass provides a way for adding new operations into GraphRewrite execution queue. That means that operations that were created inside transformation callback can be added for matching. To register node use

See also

register_new_node method. GraphRewrite automatically takes registered nodes and put them to execution queue. If multiple nodes were register make sure that they were registered in topological order. Note: when implementing pattern for Matcher make sure that root node is an operation from opset or has ov::pass::pattern::op::WrapType. That will help GraphRewrite to execute matcher passes more efficient.

Subclassed by ConvertReduceBase, CvtReduceBase, ov::pass::AUGRUCellFusion, ov::pass::AdaptivePoolToReduce, ov::pass::AddAddFusion, ov::pass::AddFakeQuantizeFusion, ov::pass::AddMultiplyFusion, ov::pass::AddOldApiMapToParameters, ov::pass::AlignEltwiseInputRanks, ov::pass::BatchNormDecomposition, ov::pass::BatchToSpaceFusion, ov::pass::BidirectionalGRUSequenceDecomposition, ov::pass::BidirectionalLSTMSequenceDecomposition, ov::pass::BidirectionalRNNSequenceDecomposition, ov::pass::BinarizeWeights, ov::pass::BroadcastConstRangeReplacement, ov::pass::BroadcastElementwiseFusion, ov::pass::BroadcastTransition, ov::pass::ChainedMaximumOptimization, ov::pass::ClampFusion, ov::pass::CompressFloatConstantsImpl, ov::pass::CompressQuantizeWeights, ov::pass::ConcatFusion, ov::pass::ConvStridesPropagation, ov::pass::ConvToBinaryConv, ov::pass::ConvertBatchToSpace, ov::pass::ConvertBitwiseAndToLogicalAnd, ov::pass::ConvertBitwiseNotToLogicalNot, ov::pass::ConvertBitwiseOrToLogicalOr, ov::pass::ConvertBitwiseXorToLogicalXor, ov::pass::ConvertBroadcast3, ov::pass::ConvertBroadcastToTiles, ov::pass::ConvertConvertLike, ov::pass::ConvertDeformableConv8To1, ov::pass::ConvertDepthToSpace, ov::pass::ConvertDetectionOutput1ToDetectionOutput8, ov::pass::ConvertDetectionOutput8ToDetectionOutput1, ov::pass::ConvertDivide, ov::pass::ConvertDivideWithConstant, ov::pass::ConvertGELU, ov::pass::ConvertGP9ToGPIEInternal, ov::pass::ConvertGRUSequenceToTensorIterator, ov::pass::ConvertGather0D, ov::pass::ConvertGather1ToGather7, ov::pass::ConvertGather7ToGather1, ov::pass::ConvertGather7ToGather8, ov::pass::ConvertGather8ToGather7, ov::pass::ConvertInterpolate11ToInterpolate4, ov::pass::ConvertInterpolate1ToInterpolate4, ov::pass::ConvertLSTMSequenceToTensorIterator, ov::pass::ConvertMVN1ToMVN6, ov::pass::ConvertMatrixNmsToMatrixNmsIE, ov::pass::ConvertMaxPool1ToMaxPool8, ov::pass::ConvertMaxPool8ToMaxPool1, ov::pass::ConvertMinimum, ov::pass::ConvertMod, ov::pass::ConvertMulticlassNms8ToMulticlassNms9, ov::pass::ConvertMulticlassNmsToMulticlassNmsIE, ov::pass::ConvertNMS1ToNMS5, ov::pass::ConvertNMS1ToNMS9, ov::pass::ConvertNMS3ToNMS5, ov::pass::ConvertNMS3ToNMS9, ov::pass::ConvertNMS4ToNMS5, ov::pass::ConvertNMS4ToNMS9, ov::pass::ConvertNMS5ToNMS9, ov::pass::ConvertNMS9ToNMSIEInternal, ov::pass::ConvertNMSRotatedToNMSIEInternal, ov::pass::ConvertNMSToNMSIEInternal, ov::pass::ConvertNegative, ov::pass::ConvertPad12ToPad1, ov::pass::ConvertPadToGroupConvolution, ov::pass::ConvertPriorBox8To0, ov::pass::ConvertQuantizeDequantize, ov::pass::ConvertRNNSequenceToTensorIterator, ov::pass::ConvertROIAlign3To9, ov::pass::ConvertROIAlign9To3, ov::pass::ConvertScatterElementsToScatter, ov::pass::ConvertScatterElementsUpdate12ToScatterElementsUpdate3, ov::pass::ConvertShapeOf3, ov::pass::ConvertShuffleChannels3, ov::pass::ConvertSoftMax1ToSoftMax8, ov::pass::ConvertSoftMax8ToSoftMax1, ov::pass::ConvertSpaceToBatch, ov::pass::ConvertSpaceToDepth, ov::pass::ConvertSubtract, ov::pass::ConvertSubtractWithConstant, ov::pass::ConvertTensorIteratorToGRUSequence, ov::pass::ConvertTensorIteratorToLSTMSequence, ov::pass::ConvertTensorIteratorToRNNSequence, ov::pass::ConvertTopK11ToTopK3, ov::pass::ConvertTopK3, ov::pass::ConvertU4WeightsZeroPointToScalar, ov::pass::ConvertXorToLogicalXor, ov::pass::ConvolutionBackpropDataMultiplyFusion, ov::pass::ConvolutionMultiplyFusion, ov::pass::ConvolutionToGroupConvolutionFusion, ov::pass::DeReshapeFullyConnected, ov::pass::DeReshapeMatMul, ov::pass::DepthToSpaceFusion, ov::pass::DilatedConvolutionConverter, ov::pass::DisableDecompressionConvertConstantFolding, ov::pass::DisableRandomUniformConstantFolding, ov::pass::DisableShapeOfConstantFolding, ov::pass::DivideFusion, ov::pass::DropoutWithRandomUniformReplacer, ov::pass::EinsumDecomposition, ov::pass::EliminateConcat, ov::pass::EliminateConvert, ov::pass::EliminateConvertNonZero, ov::pass::EliminateDuplicateTIInputs, ov::pass::EliminateEltwise, ov::pass::EliminateGatherUnsqueeze, ov::pass::EliminateNopBroadcast, ov::pass::EliminatePad, ov::pass::EliminateScatterUpdate, ov::pass::EliminateSplit, ov::pass::EliminateSplitConcat, ov::pass::EliminateSqueeze, ov::pass::EliminateTranspose, ov::pass::EliminateUnsqueezeGather, ov::pass::EnableDecompressionConvertConstantFolding, ov::pass::EnableShapeOfConstantFolding, ov::pass::EyeDecomposition, ov::pass::FakeQuantizeDecomposition, ov::pass::FakeQuantizeMulFusion, ov::pass::FakeQuantizeReshapeFusion, ov::pass::FlushFP32SubnormalsToZero, ov::pass::FoldSubgraphEmptyInputs, ov::pass::GRUCellDecomposition, ov::pass::GRUCellFusion, ov::pass::GatherNopElimination, ov::pass::Gelu7Downgrade, ov::pass::GeluFusionWithErfFour, ov::pass::GeluFusionWithErfOne, ov::pass::GeluFusionWithErfThree, ov::pass::GeluFusionWithErfTwo, ov::pass::GeluFusionWithTanh, ov::pass::GeluFusionWithTanhNoPower, ov::pass::GroupConvolutionBackpropDataMultiplyFusion, ov::pass::GroupConvolutionMultiplyFusion, ov::pass::GroupNormalizationDecomposition, ov::pass::GroupedGatherElimination, ov::pass::HSigmoidDecomposition, ov::pass::HSigmoidFusionWithClampDiv, ov::pass::HSigmoidFusionWithClampMul, ov::pass::HSigmoidFusionWithReluDiv, ov::pass::HSigmoidFusionWithReluMul, ov::pass::HSigmoidFusionWithoutRelu, ov::pass::HSwishDecomposition, ov::pass::HSwishFusionWithClamp, ov::pass::HSwishFusionWithHSigmoid, ov::pass::HSwishFusionWithReluDiv, ov::pass::HSwishFusionWithReluMul, ov::pass::InitConstMask, ov::pass::InterpolateSequenceFusion, ov::pass::KeepConstAndDecompression, ov::pass::KeepConstantsPrecisionAndAddConverts, ov::pass::LSTMCellDecomposition, ov::pass::LSTMCellFusion, ov::pass::LabelResolvingThroughSelect, ov::pass::LeakyReluFusion, ov::pass::LogSoftmaxDecomposition, ov::pass::MVN6Decomposition, ov::pass::MVNFusionWithConstantsInside, ov::pass::MVNFusionWithoutConstants, ov::pass::MarkCompressedFloatConstants, ov::pass::MarkDequantizationSubgraph, ov::pass::MatMulConstTransposesExtraction, ov::pass::MatMulMultiplyFusion, ov::pass::MishFusion, ov::pass::MulFakeQuantizeFusion, ov::pass::MultiplyConvolutionBackpropDataFusion, ov::pass::MultiplyConvolutionFusion, ov::pass::MultiplyGroupConvolutionBackpropDataFusion, ov::pass::MultiplyGroupConvolutionFusion, ov::pass::MultiplyMultiplyFusion, ov::pass::NearestNeighborUpsamplingFusion, ov::pass::NonZeroHorizontalFusion, ov::pass::NopBroadcast, ov::pass::NopSliceBeforeGatherElements, ov::pass::NormalizeL2Decomposition, ov::pass::NormalizeL2Fusion, ov::pass::PReluFusionAbsSubMulMulAdd, ov::pass::PReluFusionMultiplyAdd, ov::pass::PReluFusionMultiplySub, ov::pass::PReluFusionNegReluMulAdd, ov::pass::PReluFusionNegativeAdd, ov::pass::PReluFusionNegativeSub, ov::pass::PadFusionAvgPool, ov::pass::PadFusionConvolution, ov::pass::PadFusionConvolutionBackpropData, ov::pass::PadFusionGroupConvolution, ov::pass::PadFusionGroupConvolutionBackpropData, ov::pass::PrepareShapeOpsForEliminationAroundBE, ov::pass::Proposal1Scales, ov::pass::Proposal4Scales, ov::pass::PullReshapeThroughReduce, ov::pass::PullSqueezeThroughEltwise, ov::pass::PullTransposeThroughFQUp, ov::pass::PullUnsqueezeThroughReduce, ov::pass::RNNCellDecomposition, ov::pass::RPE_Fusion, ov::pass::RandomUniformFusion, ov::pass::ReduceL1Decomposition, ov::pass::ReduceL2Decomposition, ov::pass::ReduceMerge, ov::pass::ReduceReshapeFusion, ov::pass::ReluFakeQuantizeFusion, ov::pass::RemoveConcatZeroDimInput, ov::pass::RemoveFilteringBoxesBySize, ov::pass::ReplaceConcatReduceByMinOrMax, ov::pass::ReshapeAMatMul, ov::pass::ReshapeBMatMul, ov::pass::ReshapeOptimizations, ov::pass::ReshapePRelu, ov::pass::ReshapeSequenceFusion, ov::pass::ReshapeSinkingMatMul, ov::pass::ReshapeTo1D, ov::pass::ScaledDotProductAttentionDecomposition, ov::pass::SelectWithOneValueCondition, ov::pass::SequenceFusion, ov::pass::ShapeOfConstFolding, ov::pass::ShuffleChannelsFusion, ov::pass::SimplifyCTCGreedyDecoderSeqLen, ov::pass::SimplifyGatherShapeOf, ov::pass::SimplifySecondInputOfReshape, ov::pass::SkipGatherBeforeTransposeAndReshape, ov::pass::SliceToStridedSlice, ov::pass::SoftPlusDecomposition, ov::pass::SoftPlusFusion, ov::pass::SoftPlusToMishFusion, ov::pass::SoftSignDecomposition, ov::pass::SoftmaxDecomposition, ov::pass::SoftmaxFusion, ov::pass::SpaceToBatchFusion, ov::pass::SplitConcatPairToInterpolateFusion, ov::pass::SplitSqueezeConcatFusion, ov::pass::SqueezeStridedSlice, ov::pass::StridedSliceSqueeze, ov::pass::SubtractFusion, ov::pass::SupportedNodesStridesPropagation, ov::pass::SwishFusionWithBeta, ov::pass::SwishFusionWithSigmoid, ov::pass::SwishFusionWithSigmoidWithBeta, ov::pass::SwishFusionWithoutBeta, ov::pass::TransposeConvert, ov::pass::TransposeEltwise, ov::pass::TransposeFQReduction, ov::pass::TransposeFuse, ov::pass::TransposeMatMul, ov::pass::TransposeReduction, ov::pass::TransposeReshapeEliminationForMatmul, ov::pass::TransposeToReshape, ov::pass::UniqueDecomposition, ov::pass::UnsupportedNodesStridesPropagation, ov::pass::WeightsDequantizeToFakeQuantize, ov::pass::WrapInterpolateIntoTransposes, ov::pass::low_precision::ConvertSubtractConstant, ov::pass::low_precision::CreatePrecisionsDependentAttribute< AttributeType, OperationType >, ov::pass::low_precision::LayerTransformation, ov::pass::low_precision::MarkupBias, ov::pass::low_precision::PropagateThroughPrecisionPreserved< AttributeType >, ov::pass::low_precision::PropagateToInput< AttributeType >, ov::pass::low_precision::PullReshapeThroughDequantization, ov::pass::low_precision::PullTransposeThroughDequantization, ov::pass::low_precision::UpdateSharedPrecisionPreserved< AttributeType, ExpectedAttributeType >, ov::pass::transpose_sinking::TSBinaryBackward, ov::pass::transpose_sinking::TSConcatBackward, ov::pass::transpose_sinking::TSCumSumBackward, ov::pass::transpose_sinking::TSDataMovementBackward, ov::pass::transpose_sinking::TSForwardBase, ov::pass::transpose_sinking::TSFuse, ov::pass::transpose_sinking::TSGatherBackward, ov::pass::transpose_sinking::TSInterpolateBackward, ov::pass::transpose_sinking::TSReductionBackward, ov::pass::transpose_sinking::TSResetNoSinkingAttribute, ov::pass::transpose_sinking::TSSliceBackward, ov::pass::transpose_sinking::TSSplitBackward, ov::pass::transpose_sinking::TSSqueezeBackward, ov::pass::transpose_sinking::TSTileBackward, ov::pass::transpose_sinking::TSUnaryBackward, ov::pass::transpose_sinking::TSUnsqueezeBackward

class MatMulConstTransposesExtraction : public ov::pass::MatcherPass
#include <matmul_const_transposes_extraction.hpp>
class MatMulMultiplyFusion : public ov::pass::MatcherPass
#include <matmul_multiply_fusion.hpp>

MatMulMultiplyFusion transformation matches following graph:

    +----------+            +----------+
    |    A     |            |    B     |
    +----------+            +----------+
         |                       |
         -----------    ----------
                   |    |
                   v    v
                 +--------+
                 | MatMul |
                 +--------+
                     |
                     v
                +----------+     +----------+
                | Multiply |<----| Constant |
                +----------+     +----------+

and replaces with:

                      +-------+   +----------+
                      |   B   |   | Constant |
                      +-------+   +----------+
                           |            |
                           ------  ------
                                |  |
                                v  v
    +----------+            +----------+
    |    A     |            | Multiply |
    +----------+            +----------+
         |                       |
         -----------    ----------
                   |    |
                   v    v
                 +--------+
                 | MatMul |
                 +--------+

class MimicSetBatchSize : public ov::pass::ModelPass
#include <mimic_set_batch_size.hpp>

MimicSetBatchSize transformation relaxes hard-coded output batch dimension of Reshape operation. For Reshape with input shape [in_batch, …] and pattern value [out_batch, …] it generates a sub-graph which basically keeps ratio of input and output batch size and performs the following calculation:

scale = float(out_batch) / float(in_batch) modified_batch_dim = int(ceil(float(shape(input)[0]) * scale))

This transformation should be executed only while setBatchSize method call

class MishFusion : public ov::pass::MatcherPass
#include <mish_fusion.hpp>

MishFusion transformation replaces group of operations: x * tanh(log(exp(x) + 1)) to Mish op.

class MOCLegacyTransformations : public ov::pass::ModelPass
#include <moc_legacy_transformations.hpp>

This transformation is an entry point for OpenVINO transformations that will be applied inside MOC. This transformations container is filled with legacy transformations to reach parity between legacy front-ends and new frontends calling from the Model Optimizer. It contains transformations to avoid limitations of OpenVINO 1.X API such as unsupported INT64 for inputs, usage of NCHW layout that is critical for TensorFlow models.

class MOCTransformations : public ov::pass::ModelPass
#include <moc_transformations.hpp>

This transformation is an entry point for OpenVINO transformations that will be applied inside MOC. And in future this transformations container will be filled with transformations pipeline but now it remains empty.

Public Functions

inline explicit MOCTransformations(bool use_shapes, bool low_precision_enabled = true)

use_shapes = True enables transformations which are depends on shapes and also it enables ConstantFolding for all ShapeOf operations.

low_precision_enabled = True enables preserving mechanisms that helps to keep low_precision sub-graphs as is.

class ModelPass : public ov::pass::PassBase
#include <pass.hpp>

Base class for Model passes.

Subclassed by ov::pass::AlignMixedFP32FP16Types, ov::pass::ApplyTableOfEquivalence, ov::pass::ChangePlaceholderTypes, ov::pass::CommonOptimizations, ov::pass::ConstantFolding, ov::pass::ConvertCompressedOnlyToLegacy, ov::pass::ConvertFP32ToFP16, ov::pass::ConvertOpSet2ToOpSet1, ov::pass::ConvertOpSet3ToOpSet2, ov::pass::ConvertPrecision, ov::pass::FindBatch, ov::pass::FusedNamesCleanup, ov::pass::GraphRewrite, ov::pass::GroupedSliceToVSplitOptimization, ov::pass::GroupedStridedSliceOptimizer, ov::pass::Hash, ov::pass::InitNodeInfo, ov::pass::LSTMStatesBroadcast, ov::pass::LowLatency2, ov::pass::MOCLegacyTransformations, ov::pass::MOCTransformations, ov::pass::MakeStateful, ov::pass::MarkPrecisionSensitiveShapeOfSubgraphs, ov::pass::MarkSugraphsToKeepInMixedPrecision, ov::pass::MimicSetBatchSize, ov::pass::OptimizeLabelsUsedAsValues, ov::pass::POTTransformations, ov::pass::Pruning, ov::pass::PushConstantToSubgraph, ov::pass::RemoveMultiSubGraphOpDanglingParamsResults, ov::pass::ResolveNameCollisions, ov::pass::ReverseInputChannelsFusion, ov::pass::ReverseShapeAndTypeInfer, ov::pass::Serialize, ov::pass::SetBatchSize, ov::pass::SharedOpOptimization, ov::pass::SharedSqueeze, ov::pass::SharedStridedSliceEraser, ov::pass::ShrinkWeights, ov::pass::SimplifyShapeOfSubGraph, ov::pass::SmartReshape, ov::pass::StreamSerialize, ov::pass::StridedSliceOptimization, ov::pass::SymbolicOptimizations, ov::pass::SymbolicPropagation, ov::pass::UnrollIf, ov::pass::UnrollTensorIterator, ov::pass::UselessSliceEraser, ov::pass::Validate, ov::pass::VisualizeTree, ov::pass::low_precision::AlignQuantizationIntervals, ov::pass::low_precision::AlignQuantizationParameters, ov::pass::low_precision::LowPrecision, ov::pass::low_precision::MarkupAvgPoolPrecisionPreserved, ov::pass::low_precision::MarkupCanBeQuantized, ov::pass::low_precision::MarkupOptimizations, ov::pass::low_precision::MarkupPrecisions, ov::pass::low_precision::MarkupQuantizationGranularity, ov::pass::low_precision::PropagatePrecisions, ov::pass::low_precision::PropagateSharedValue< AttributeType >, ov::pass::transpose_sinking::TSGeneral

class MulFakeQuantizeFusion : public ov::pass::MatcherPass
#include <mul_fake_quantize_fusion.hpp>

MulFakeQuantizeFusion transformation replaces following graph: Mul->FakeQuantize to a single FakeQuantize Restrictions:

  • second input to Mul is a Constant

class MultiplyConvolutionBackpropDataFusion : public ov::pass::MatcherPass
#include <mul_conv_fusion.hpp>
class MultiplyConvolutionFusion : public ov::pass::MatcherPass
#include <mul_conv_fusion.hpp>

Multiply->Convolution fusion replaces following graph:

+—-&#8212;+ +——-&#8212;+ | Input

| | Constant | +—-&#8212;+ +——-&#8212;+ | |

| | v v +——-

&#8212;+ +——&#8212;+ | Multiply | | Weights | +——-&#8212;+ +——&#8212;+ | |

| | v v +————-

&#8212;+ | Convolution Op | +————-&#8212;+

to following:

                      +---------+   +----------+
                      | Weights |   | Constant |
                      +---------+   +----------+
                           |            |
                           ------  ------
                                |  |
                                v  v
     +-------+              +----------+
     | Input |              | Multiply |
     +-------+              +----------+
         |                       |
         -----------    ----------
                   |    |
                   v    v
              +----------------+
              | Convolution Op |
              +----------------+
where ‘Convolution Op’ is one of:
  • Convolution

  • ConvolutionBackpropData

  • GroupConvolution

  • GroupConvolutionBackpropData

Restrictions:

  • weights’ shape is static

  • if the constant input to Multiply has the same rank as ‘input’, the constant first dimension has to be 1

  • constant input to Multiply has to be broadcastable to weights when ‘Convolution Op’ is either Convolution or GroupConvolution

  • shape of a constant input to Multiply has to be in one of following forms: (1), (1, 1, …, 1), (C, 1, …, 1), (1, C, 1, …, 1) when ‘Convolution Op’ is either ConvolutionBackpropData or GroupConvolutionBackpropData

class MultiplyGroupConvolutionBackpropDataFusion : public ov::pass::MatcherPass
#include <mul_conv_fusion.hpp>
class MultiplyGroupConvolutionFusion : public ov::pass::MatcherPass
#include <mul_conv_fusion.hpp>
class MultiplyMultiplyFusion : public ov::pass::MatcherPass
#include <lin_op_sequence_fusion.hpp>
class MVN6Decomposition : public ov::pass::MatcherPass
#include <mvn6_decomposition.hpp>

MVN6Decomposition transformation into sub-graph x - ReduceMean(x, axes) if normalize_variance is false and into sub-graph (x - ReduceMean(x, axes)) / Sqrt(ReduceMean((x - ReduceMean(x, axes)) ^ 2)) if normalize_variance is true.

class MVNFusion : public ov::pass::GraphRewrite
#include <mvn_fusion.hpp>

MVNFusion transformation replaces various sub-graphs with a MVN op.

class MVNFusionWithConstantsInside : public ov::pass::MatcherPass
#include <mvn_fusion.hpp>

MVNFusion transformation replaces group of operations: gamma * (x - ReduceMean(x, axes)) / (Sqrt(ReduceMean((x - ReduceMean(x, axes)) ^ 2)) + eps) + beta to MVN op.

class MVNFusionWithoutConstants : public ov::pass::MatcherPass
#include <mvn_fusion.hpp>

MVNFusion transformation replaces group of operations: (x - ReduceMean(x, axes)) / (Sqrt(ReduceMean((x - ReduceMean(x, axes)) ^ 2)) + eps) to MVN op.

class NearestNeighborUpsamplingFusion : public ov::pass::MatcherPass
#include <nearest_neighbor_upsampling_fusion.hpp>

NearestNeighborUpsamplingFusion transformation fuses subgraph that uses the simpler operations, as ShapeOf, StridedSlice, Concat, Reshape, Mul to calculate Interpolate with mode=’nearest’.

class NodeRegistry
#include <graph_rewrite.hpp>

Register openvino node pointers into container. Can create and/or add existing node pointers into register.

Public Functions

template<typename T, class ...Args>
inline std::shared_ptr<T> make(Args&&... args)

Make new node and add it to register.

Template Parameters
  • TNode type.

  • ArgsNode ctor args types.

Parameters

args – New node ctor arguments.

Returns

Shared pointer to node of type T.

template<typename T>
inline std::shared_ptr<T> add(const std::shared_ptr<T> &node)

Add node to register.

Template Parameters

TNode type.

Parameters

nodeNode to add

Returns

Shared pointer to new node added of type T.

inline const std::vector<std::shared_ptr<Node>> &get() const

Get nodes container.

Returns

Const reference to nodes container.

inline void clear()

Clear register.

class NonZeroHorizontalFusion : public ov::pass::MatcherPass
#include <nonzero_horizontal_fusion.hpp>

NonZeroHorizontalFusion transformation makes horizontal fusion for equal NonZero layers.

class NopBroadcast : public ov::pass::MatcherPass
#include <nop_broadcast.hpp>

Optimizes out Broadcast(data, Maximum(shape, ones)) if labels on data and shape are equal Use case with data being empty should not be considered here since original graph has Maximum with ones.

class NopElimination : public ov::pass::GraphRewrite
#include <nop_elimination.hpp>
class NopSliceBeforeGatherElements : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

NopSliceBeforeGatherElements eliminates slice before GElements if slicing from 0 It is valid since GatherElements doesn’t support negative indices and Slice won’t affect indexing of elements in the original tensor that GatherElements would like to take.

class NormalizeL2Decomposition : public ov::pass::MatcherPass
#include <normalize_l2_decomposition.hpp>

Decomposes NormalizeL2 into subgraph.

class NormalizeL2Fusion : public ov::pass::MatcherPass
#include <normalize_l2_fusion.hpp>

NormalizeL2Fusion transformation replaces sub-graphs: x/(sqrt(max(reduce_sum(x[j0, …, jN]**2, axes), eps)) x/(sqrt(add(reduce_sum(x[j0, …, jN]**2, axes), eps)) x/(pow(max(reduce_sum(x[j0, …, jN]**2, axes), eps), 0.5) x/(pow(add(reduce_sum(x[j0, …, jN]**2, axes), eps), 0.5) x*(pow(max(reduce_sum(x[j0, …, jN]**2, axes), eps), -0.5) x*(pow(add(reduce_sum(x[j0, …, jN]**2, axes), eps), -0.5) with a NormalizeL2(x, axes, eps, eps_mode[MAX|ADD]) op.

class OptimizeLabelsUsedAsValues : public ov::pass::ModelPass
#include <label_optimization.hpp>

Collects sources where each symbol / label initially appeared (on shape or shape sub-graph) and attaches all value usages of this label to this initial source.

class PadFusion : public ov::pass::GraphRewrite
#include <pad_fusion.hpp>
class PadFusionAvgPool : public ov::pass::MatcherPass
#include <pad_fusion.hpp>

PadFusion transformation replaces following graph: Pad -> AvgPool to AvgPool, under following conditions.

  • pad mode is op::PadMode::CONSTANT

  • pad value is 0

  • exclude_pad in AvgPool is set to false or pads_begin, pads_end are set to zero

class PadFusionConvolution : public ov::pass::MatcherPass
#include <pad_fusion.hpp>

PadFusion transformation replaces following graph: Pad -> Convolution to Convolution, under following conditions.

class PadFusionConvolutionBackpropData : public ov::pass::MatcherPass
#include <pad_fusion.hpp>

PadFusion transformation replaces following graph: Pad -> ConvolutionBackpropData to ConvolutionBackpropData, under following conditions.

  • pad mode is op::PadMode::CONSTANT

  • pad value is 0

  • pads in ConvolutionBackpropData are greater than pads in Pad node

class PadFusionGroupConvolution : public ov::pass::MatcherPass
#include <pad_fusion.hpp>

PadFusion transformation replaces following graph: Pad -> GroupConvolution to GroupConvolution, under following conditions.

class PadFusionGroupConvolutionBackpropData : public ov::pass::MatcherPass
#include <pad_fusion.hpp>

PadFusion transformation replaces following graph: Pad -> GroupConvolutionBackpropData to GroupConvolutionBackpropData, under following conditions.

  • pad mode is op::PadMode::CONSTANT

  • pad value is 0

  • pads in GroupConvolutionBackpropData are greater than pads in Pad node

class PassBase
#include <pass.hpp>

Base class for transformation passes.

Subclassed by ngraph::pass::NodePass, ov::pass::MatcherPass, ov::pass::ModelPass

Public Functions

bool get_property(const PassPropertyMask &prop_mask) const

Check if this pass has all the pass properties.

void set_callback(const param_callback &callback)

Set callback for particular transformation type. This method set global callback. For more details see PassConfig class documentation.

Parameters

callback – lambda function that takes node and returns bool

inline virtual void set_pass_config(const std::shared_ptr<PassConfig> &pass_config)

Set PassConfig for particular transformation instance.

Parameters

pass_config – is a PassConfig shared_ptr

inline std::shared_ptr<PassConfig> get_pass_config()

Allows to access PassConfig shared instance.

Returns

Shared instance of PassConfig class

inline bool transformation_callback(const std::shared_ptr<const Node> &node)

Applies callback for given node. By default callback returns false.

Parameters

node – which will be used inside callback

Returns

result of callback execution for given node

class PassConfig
#include <pass_config.hpp>

Class representing a transformations config that is used for disabling/enabling transformations registered inside pass::Manager and also allows to set callback for all transformations or for particular transformation.

When pass::Manager is created all passes registered inside this manager including nested passes will share the same instance of PassConfig class. To work with this class first you need to get shared instance of this class by calling manager.get_pass_config() method. Then you will be able to disable/enable passes based on transformations type_info. For example:

pass::Manager manager;
manager.register_pass<CommonOptimizations>();
auto pass_config = manager.get_pass_config();
pass_config->disable<ConvertGELU>(); // this will disable nested pass inside
                                     // CommonOptimizations pipeline
manager.run_passes(f);
Sometimes it is needed to call transformation inside other transformation manually. And for that case before running transformation you need manually check that this pass is not disabled and then you need to set current PassConfig instance to this transformation. For example:
// Inside MatcherPass callback or inside FunctionPass run_on_function() method
// you need to call get_pass_config() method to get shared instance of PassConfig
auto pass_config = get_pass_config();

// Before running nested transformation you need to check is it disabled or not
if (!pass_config->is_disabled<ConvertGELU>()) {
    auto pass = ConvertGELU();
    pass->set_pass_config(pass_config);
    pass.apply(node);
}
Following this logic inside your transformations you will guaranty that transformations will be executed in a right way.

Public Functions

PassConfig()

Default constructor.

void disable(const DiscreteTypeInfo &type_info)

Disable transformation by its type_info.

Parameters

type_info – Transformation type_info

template<class T>
inline void disable()

Disable transformation by its class type (based on type_info)

void enable(const DiscreteTypeInfo &type_info)

Enable transformation by its type_info.

Parameters

type_info – Transformation type_info

template<class T>
inline void enable()

Enable transformation by its class type (based on type_info)

inline void set_callback(const param_callback &callback)

Set callback for all kind of transformations.

template<typename T, class ...Args>
inline void set_callback(const param_callback &callback)

Set callback for particular transformation class types.

Example below show how to set callback for one or multiple passes using this method.

pass_config->set_callback<ov::pass::ConvertBatchToSpace,
                          ov::pass::ConvertSpaceToBatch>(
         [](const_node_ptr &node) -> bool {
              // Disable transformations for cases when input shape rank is not
              equal to 4
              const auto input_shape_rank =
              node->get_output_partial_shape(0).rank().get_length();
              if (input_shape_rank != 4) {
                  return false;
              }
              return true;
          });
Note that inside transformations you must provide code that work with this callback. See example below:
if (transformation_callback(node)) {
    return false; // exit from transformation
}

param_callback get_callback(const DiscreteTypeInfo &type_info) const

Get callback for given transformation type_info.

In case if callback wasn’t set for given transformation type then global callback will be returned. But if even global callback wasn’t set then default callback will be returned.

Parameters

type_info – Transformation type_info

template<class T>
inline param_callback get_callback() const

Get callback for given transformation class type.

Returns

callback lambda function

inline bool is_disabled(const DiscreteTypeInfo &type_info) const

Check either transformation type is disabled or not.

Parameters

type_info – Transformation type_info

Returns

true if transformation type was disabled and false otherwise

template<class T>
inline bool is_disabled() const

Check either transformation class type is disabled or not.

Returns

true if transformation type was disabled and false otherwise

inline bool is_enabled(const DiscreteTypeInfo &type_info) const

Check either transformation type is force enabled or not.

Parameters

type_info – Transformation type_info

Returns

true if transformation type was force enabled and false otherwise

template<class T>
inline bool is_enabled() const

Check either transformation class type is force enabled or not.

Returns

true if transformation type was force enabled and false otherwise

class POTTransformations : public ov::pass::ModelPass
#include <pot_transformations.hpp>

This transformation is an entry point for OpenVINO transformations that will be executed inside POT.

class PReluFusion : public ov::pass::GraphRewrite
#include <prelu_fusion.hpp>

PReluFusion transformation replaces various sub-graphs with a PRelu op.

class PReluFusionAbsSubMulMulAdd : public ov::pass::MatcherPass
#include <prelu_fusion.hpp>

PReluFusionAbsSubMulMulAdd transformation replaces a sub-graph Op / | \ Relu | Abs | \ | | Subtract | | | Multiply | | | Multiply (0.5) \ / Add.

class PReluFusionMultiplyAdd : public ov::pass::MatcherPass
#include <prelu_fusion.hpp>

PReluFusionMultiplyAdd transformation replaces a sub-graph Op / \ Relu Multiply (-1) | | | Relu | | | Multiply \ / Add.

class PReluFusionMultiplySub : public ov::pass::MatcherPass
#include <prelu_fusion.hpp>

PReluFusionMultiplySub transformation replaces a sub-graph Op / \ Relu Multiply (-1) | | | Relu | | | Multiply \ / Sub.

class PReluFusionNegativeAdd : public ov::pass::MatcherPass
#include <prelu_fusion.hpp>

PReluFusionNegativeAdd transformation replaces a sub-graph Op / \ Relu Negative | | | Relu | | | Negative | | | Multiply \ / Add.

class PReluFusionNegativeSub : public ov::pass::MatcherPass
#include <prelu_fusion.hpp>

PReluFusionNegativeSub transformation replaces a sub-graph Op / \ Relu Negative | | | Relu | | | Multiply \ / Sub.

class PReluFusionNegReluMulAdd : public ov::pass::MatcherPass
#include <prelu_fusion.hpp>

PReluFusionNegReluMulAdd transformation replaces a sub-graph Op / \ Relu Negative | | | Relu | | | Multiply \ / Add.

class PrepareShapeOpsForEliminationAroundBE : public ov::pass::MatcherPass
#include <nop_elimination.hpp>

PrepareShapeOpsForEliminationAroundBE works on the subgraph like Reshape/Squeeze/Unsqueeze -> BinaryElementwiseOperation -> Reshape/Squeeze/Unsqueeze and prepares it for the following optimizations by moving bottom op up through Binary op.

class PropagateMasks : public ov::pass::GraphRewrite
#include <pruning.hpp>

Contains several MatcherPasses that initialize and propagate masks from Constant operation to the network output.

class Proposal1Scales : public ov::pass::MatcherPass
#include <proposal_scales_stridedslice.hpp>

ProposalScales transformation helps to silently avoid reshape issues on the scale-input of Proposal layer.

Expected sub-graph looks like: Parameter [batch, 3 or 4] -> Reshape [-1] -(in: 3)-> PriorBox

PriorBox operation accepts 3 or 4 values as scales from specification standpoint PriorBox uses first set (batch) of scale values to proceed in the plugins According to this we explicitly take first batch of scales with StridedSlice operation

Resulting sub-graph: Parameter [batch, 3 or 4] -> Reshape [-1] -> StridedSlice[0: 3 or 4] -(in: 3)-> PriorBox

class Proposal4Scales : public ov::pass::MatcherPass
#include <proposal_scales_stridedslice.hpp>
class Pruning : public ov::pass::ModelPass
#include <pruning.hpp>

This is just a sequence of passes that performs pruning transformations pipeline.

class PullReshapeThroughReduce : public ov::pass::MatcherPass
#include <pull_through_reduce.hpp>

PullReshapeThroughReduce transformation The transformation pulls Reshape operator though Reduce ops if possible. In the further processing such Reshape can be often skipped as nop.

class PullSqueezeThroughEltwise : public ov::pass::MatcherPass
#include <concat_reduce_fusion.hpp>

PullSqueezeThroughEltwise transformation propagates Squeeze up through binary elementwise operations:

class PullThroughReduce : public ov::pass::GraphRewrite
#include <pull_through_reduce.hpp>

PullThroughReduce transformation The transformation pulls Reshape or Unsqueeze operators though Reduce ops if possible. In the further processing such Reshape/Unsqueeze can be often skipped as nop.

class PullTransposeThroughFQUp : public ov::pass::MatcherPass
#include <pull_transpose_through_fq.hpp>
class PullUnsqueezeThroughReduce : public ov::pass::MatcherPass
#include <pull_through_reduce.hpp>

PullUnsqueezeThroughReduce transformation The transformation pulls Unsqueeze operator though Reduce ops if possible. In the further processing such Unsqueeze can be often skipped as nop.

class PushConstantToSubgraph : public ov::pass::ModelPass
#include <push_constant_to_subgraph.hpp>

PushConstantToSubgraph transformation detects MultiSubGraphOp inputs that can be constfoldable pushes that inputs to subgraphs.

class RandomUniformFusion : public ov::pass::MatcherPass
#include <random_uniform_fusion.hpp>

RandomUniformFusion transformation replaces RandomUniform -> Add or RandomUniform -> Mul subgraph with a RandomUniform and replaces min and max const with corrected values.

class ReduceL1Decomposition : public ov::pass::MatcherPass
#include <reduce_l1_decomposition.hpp>

Decomposes ReduceL1 into ReduceSum(abs(x)).

class ReduceL2Decomposition : public ov::pass::MatcherPass
#include <reduce_l2_decomposition.hpp>

Decomposes ReduceL2 into sqrt(ReduceSum(x * x)).

class ReduceMerge : public ov::pass::MatcherPass
#include <reduce_merge.hpp>

ReduceMerge transformation matches following graph:

+——-&#8212;+ +——-&#8212;+ | A | | B | +——-&#8212;+ +——-&#8212;+ | |

| | v v +—–

&#8212;+ +—–&#8212;+ | Reduce | | C | +—–&#8212;+ +—–&#8212;+ | | | —-&#8212; | | v v +——-&#8212;+ | Reduce | +——-&#8212;+

and replaces with:

      +----------+     +----------+
      |    B     |     |    C     |
      +----------+     +----------+
           |                |
           -------    -------
                 |    |
                 v    v
+——-&#8212;+ +—————-&#8212;+ | A | | Concat/Constant | +——-&#8212;+ +—————-&#8212;+ | | | —–&#8212; | | v v +——-&#8212;+ | Reduce | +——-&#8212;+

class ReduceReshapeFusion : public ov::pass::MatcherPass
#include <reduce_reshape_fusion.hpp>

ReduceReshapeFusion transformation Fuse ReduceOp(keep_dims=false)+Reshape to ReduceOp(keep_dims=true)

class ReluFakeQuantizeFusion : public ov::pass::MatcherPass
#include <relu_fake_quantize_fusion.hpp>

ReluFakeQuantizeFusion transformation replaces following graph: Relu -> FakeQuantize to FakeQuantize under following conditions:

  • ‘input_low’ input to FakeQuantize is a Constant

  • ’input_low’ has non negative values

class RemoveConcatZeroDimInput : public ov::pass::MatcherPass
#include <remove_concat_zero_dim_input.hpp>

RemoveConcatZeroDimInput transformation removes input of Concat if the tensor size is equal to 0.

class RemoveFilteringBoxesBySize : public ov::pass::MatcherPass
#include <remove_filtering_boxes_by_size.hpp>
class RemoveMultiSubGraphOpDanglingParamsResults : public ov::pass::ModelPass
#include <remove_multi_subgraph_op_dangling_params.hpp>
class ReplaceConcatReduceByMinOrMax : public ov::pass::MatcherPass
#include <concat_reduce_fusion.hpp>

ReplaceConcatReduceByMinOrMax transformation replaces Concat with 2 inputs and ReduceMin/Max by a single Minimum/Maximum with 2 inputs and inserts squeeze in case when Reduce has keep_dims = false.

class ReshapeAMatMul : public ov::pass::MatcherPass
#include <matmul_sr.hpp>

ReshapeAMatMul and ReshapeBMatMul transformations relax hard-coded Reshape followed by MatMul operation For 2D Reshape search patterns are:

  • MatMul(Reshape(any_input, any_input), any_input)

  • MatMul(any_input, Reshape(any_input, any_input))

class ReshapeBMatMul : public ov::pass::MatcherPass
#include <matmul_sr.hpp>
class ReshapeOptimizations : public ov::pass::MatcherPass
#include <reshape_optimizations.hpp>

Searches for Flatten-like Reshape operations and simplifies 2nd input of such Reshape using special zero feature.

class ReshapePRelu : public ov::pass::MatcherPass
#include <reshape_prelu.hpp>

ReshapePRelu reshape second input of PRelu (slope)

class ReshapeSequenceFusion : public ov::pass::MatcherPass
#include <reshape_sequence_fusion.hpp>

ReshapeSequenceFusion fuses sequence of Reshape operation into single Reshape or eliminates full redundant sequence.

class ReshapeSinkingMatMul : public ov::pass::MatcherPass
#include <reshape_sinking.hpp>

ReshapeSinkingMatMul transformation looks for MatMul followed by optional Add surrounded with Reshape operations which are only needed to merge and unmerge dimensions into MatMuls batch. In case of success upscales MatMul to work with multidimensional batch and updates Reshape operators to make batch propagate through freely.

class ReshapeTo1D : public ov::pass::MatcherPass
#include <reshape_to_1D.hpp>

ReshapeTo1D transformation looks for Reshape from nD to 1D tensor and replaces its pattern to [-1].

class ResolveNameCollisions : public ov::pass::ModelPass
#include <resolve_names_collisions.hpp>

ResolveNameCollisions transformation helps to fix names collisions if nodes with autogenerated names have conflicts with other node names.

Every transformation call can change the graph structure and create some additional operations, autogenerated name is used if new operation doesn’t have friendly name. This transformations should be called after the transformation pipeline in order to fix names collisions.

There is also an additional mode “resolve_all_names”, the logic of which is the same, but for all “friendly_names” in the model ov, not only for autogenerated.

class ReverseInputChannelsFusion : public ov::pass::ModelPass
#include <ric_fusion.hpp>

ReverseInputChannelsFusion.

class ReverseShapeAndTypeInfer : public ov::pass::ModelPass
#include <reverse_shape_and_type_infer.hpp>

Perform reverse shape and type infer to deduce input rank and type in certain cases.

class RNNCellDecomposition : public ov::pass::MatcherPass
#include <rnn_cell_decomposition.hpp>

RNNCellDecomposition transformation decomposes RNNCell layer with inputs X, H, W, R, B to Add, MatMul ops according to the formula:

  • Denotes dot product. f - is an activation functions.

Ht = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Wbi + Rbi)

class RPE_Fusion : public ov::pass::MatcherPass
#include <fuse_rotary_positional_embeddings.hpp>

Fuses special sub-graph into an internal Rotary Positional Embedding operation.

class ScaledDotProductAttentionDecomposition : public ov::pass::MatcherPass
#include <scaled_dot_product_attention_decomposition.hpp>
class SelectWithOneValueCondition : public ov::pass::MatcherPass
#include <select_with_one_value_condition.hpp>

SelectWithOneValueCondition transformation eliminates Select operation if the condition is constant and consists of al True or False elements.

class SequenceFusion : public ov::pass::MatcherPass
#include <sequence_fusion.hpp>

SequenceFusion transformation replaces a chain of Cells operations with single Sequence op.

Supported cells: GRUCell, LSTMCell, RNNCell, AUGRUCell Prerequisites: the source of W,R,B inputs must be the same or it can be different Constants with the same type, shape and value.

class Serialize : public ov::pass::ModelPass
#include <serialize.hpp>

Serialize transformation converts ov::Model into IR files.

Attention

  • dynamic shapes are not supported

class SetBatchSize : public ov::pass::ModelPass
#include <set_batch_size.hpp>

Generic caller for all the transformations responsible to make model reshape-able by batch dimension.

class ShapeOfConstFolding : public ov::pass::MatcherPass
#include <shape_of_const_folding.hpp>

ShapeOfConstFolding constantfolds ShapeOf->Constant subgraph.

class SharedOpOptimization : public ov::pass::ModelPass
#include <shared_ops_optimization.hpp>

SharedOpOptimization optimizes operations which are sourcing from same Output<Node> and perform the same action on the same data.

class SharedSqueeze : public ov::pass::ModelPass
#include <strided_slice_squeeze.hpp>

SharedSqueeze transformation looks for shared Squeezes and leaves only one Squeeze reconnecting all the outputs to it.

class SharedStridedSliceEraser : public ov::pass::ModelPass
#include <optimize_strided_slice.hpp>

SharedStridedSliceEraser transformation replaces group of StridedSlice operations with first StridedSlice in this group. All SrtideSlices in this group must be equal and consume the same output port.

class ShrinkWeights : public ov::pass::ModelPass
#include <pruning.hpp>

Based on masks in Constant operation it inserts Gather operations to shrink them. After this pass execution ConstantFolding is required.

class ShuffleChannelsFusion : public ov::pass::MatcherPass
#include <shuffle_channels_fusion.hpp>

ShuffleChannelsFusion transformation detects Reshape-Transpose-Reshape pattern and tries to fuse it into a single ShuffleChannels layer with axis = 1.

x’ = reshape(x, [N, group, C / group, H, W]) or reshape(x, [N, group, C / group, H * W]) x’’ = transpose(x’, [0, 2, 1, 3, 4]) or transpose(x’, [0, 2, 1, 3]) y = reshape(x’’, [N, C, H, W])

Param reshape_constants_check

the flag that defines the need for additional checks of reshapes constant Additional checks are required when ShuffleChannelsFusion using inside offline transformations and are not necessary when ShuffleChannelsFusion using inside CommonOptimizations

class SkipGatherBeforeTransposeAndReshape : public ov::pass::MatcherPass
#include <skip_gather_before_transpose_and_reshape.hpp>

SkipGatherBeforeTransposeAndReshape transformation removes Gather from the Gather->Transpose->Reshape sequence in case when input has batch=1 and gather has axis=0 and indices={0}. Also, this transformation corrects a transpose constant to save semantic.

class SliceToStridedSlice : public ov::pass::MatcherPass
#include <convert_slice_to_strided_slice.hpp>

SliceToStridedSlice transformation convert v8::Slice to v1::StridedSlice.

class SmartReshape : public ov::pass::ModelPass
#include <smart_reshape.hpp>
class SoftmaxDecomposition : public ov::pass::MatcherPass
#include <softmax_decomposition.hpp>

SoftmaxDecomposition transformation replaces softmax with following graph:

       +---------------+
       │               │
       │     input     │
       │               │
       +---------------+
           │      │
           │      v
           │ +-----------+
           │ │           │
           │ │ ReduceMax │
           │ │           │
           │ +-----------+
           │      │
           │      │
           v      v
       +---------------+
       │               │
       │      Sub      │
       │               │
       +---------------+
               |
               |
               v
       +---------------+
       │               │
       │      Exp      │
       │               │
       +---------------+
           │      │
           │      v
           │ +-----------+
           │ │           │
           │ │ ReduceSum │
           │ │           │
           │ +-----------+
           │      │
           │      │
           v      v
        +-------------+
        |             │
        |     Div     │
        │             │
        +-------------+
class SoftmaxFusion : public ov::pass::MatcherPass
#include <softmax_fusion.hpp>

SoftmaxFusion transformation replaces following graphs:

       +---------------+
       │               │
       │     input     │
       │               │
       +---------------+
           │      │
           │      v
           │ +-----------+
           │ │           │
           │ │ ReduceMax │
           │ │           │
           │ +-----------+
           │      │
           │      │
           v      v
       +---------------+
       │               │
       │      Sub      │
       │               │
       +---------------+
               |
               |
               v
       +---------------+
       │               │
       │      Exp      │
       │               │
       +---------------+
           │      │
           │      v
           │ +-----------+
           │ │           │
           │ │ ReduceSum │
           │ │           │
           │ +-----------+
           │      │
           │      │
           v      v
        +-------------+
        |             │
        |     Div     │
        │             │
        +-------------+
and +————&#8212;+ │ │ │ input │ │ │ +————&#8212;+

v +————&#8212;+ │ │ │ Exp │ │ │ +————&#8212;+ │ │ │ v │ +——–&#8212;+ │ │ │ │ │ ReduceSum │ │ │ │ │ +——–&#8212;+ │ │ │ │ v v +———-&#8212;+ | │ | Div │ │ │ +———-&#8212;+

to a single Softmax node

  • Restrictions:

    • ReduceMax and ReduceSum axes must be scalar constants and they have to point to the same axis

class SoftPlusDecomposition : public ov::pass::MatcherPass
#include <softplus_decomposition.hpp>

SoftPlusDecomposition transformation replaces SoftPlus op to group of operations: log(exp(x) + 1).

class SoftPlusFusion : public ov::pass::MatcherPass
#include <softplus_fusion.hpp>

SoftPlusFusion transformation replaces group of operations: log(exp(x) + 1) to SoftPlus op.

class SoftPlusToMishFusion : public ov::pass::MatcherPass
#include <softplus_to_mish_fusion.hpp>

SoftPlusToMishFusion transformation replaces group of operations: x * tanh(softplus(x)) to Mish op.

class SoftSignDecomposition : public ov::pass::MatcherPass
#include <softsign_decomposition.hpp>

SoftSignDecomposition transformation replaces SoftSign with the following graph.

  Input ---> Abs
    |         |
    |         |
    |         |
    |         V
    |        Add <--- 1
    |         |
    |         |
    V         |
  Divide <----|
    |
    |
    |
    V
  Output
class SpaceToBatchFusion : public ov::pass::MatcherPass
#include <space_to_batch_fusion.hpp>

SpaceToBatchFusion transformation replaces following graph: Transpose (or Reshape) -> Pad -> SpaceToDepth -> Transpose (or Reshape) to SpaceToBatch Restrictions:

  • input rank must be 4

  • Transpose permutation must be [1, 0, 2, 3]

  • pad value is 0, PadMode is CONSTANT

  • SpaceToDepthMode must be BLOCKS_FIRST

class SplitConcatPairToInterpolateFusion : public ov::pass::MatcherPass
#include <split_concat_pair_to_interpolate_fusion.hpp>

SplitConcatPairToInterpolateFusion transformation replaces group of operations: Split -> Concat to Interpolate op.

class SplitSqueezeConcatFusion : public ov::pass::MatcherPass
#include <split_squeeze_concat_fusion.hpp>

SplitSqueezeConcatFusion transformation replaces group of operations: Split -> Squeeze (multiple) -> Concat to Transpose -> Reshape ops.

class SqueezeStridedSlice : public ov::pass::MatcherPass
#include <strided_slice_squeeze.hpp>

StridedSliceSqueeze transformation looks for Squeeze -> SSe and corrects SS inputs and attributes for SS output to be squeeze-able.

class StreamSerialize : public ov::pass::ModelPass
#include <serialize.hpp>

StreamSerialize transformation converts ov::Model into single binary stream.

Attention

  • dynamic shapes are not supported

struct DataHeader
#include <serialize.hpp>
class StridedSliceOptimization : public ov::pass::ModelPass
#include <optimize_strided_slice.hpp>

StridedSliceOptimization transformation executes all transformations related to StridedSlice optimizations.

class StridedSliceSqueeze : public ov::pass::MatcherPass
#include <strided_slice_squeeze.hpp>

StridedSliceSqueeze transformation looks for SS -> Squeeze and corrects SS inputs and attributes for SS output to be squeeze-able.

class StridesOptimization : public ov::pass::BackwardGraphRewrite
#include <strides_optimization.hpp>

StridesOptimization transformation works backward on function and propagates strides up through the graph if possible.

class SubtractFusion : public ov::pass::MatcherPass
#include <subtract_fusion.hpp>

SubtractFusion transformation replaces a sub-graph Mul(y, -1) + x or x + Mul(y, -1) with Subtract(x,y)

class SupportedNodesStridesPropagation : public ov::pass::MatcherPass
#include <strides_optimization.hpp>

SupportedNodesStridesPropagation either propagates stride (greater than 1) from current node up through the graph or inserts pooling between current node and its consumers if the consumers have different StridesProp attributes.

class SwishFusion : public ov::pass::GraphRewrite
#include <swish_fusion.hpp>

SwishFusion transformation replaces various sub-graphs with a Swish op.

class SwishFusionWithBeta : public ov::pass::MatcherPass
#include <swish_fusion.hpp>

SwishFusionWithSigmoid replaces a sub-graphs x / (1.0 + exp(-x * beta)) with a Swish op.

class SwishFusionWithoutBeta : public ov::pass::MatcherPass
#include <swish_fusion.hpp>

SwishFusionWithSigmoid replaces a sub-graphs x / (1.0 + exp(-x)) with a Swish op.

class SwishFusionWithSigmoid : public ov::pass::MatcherPass
#include <swish_fusion.hpp>

SwishFusionWithSigmoid replaces a sub-graphs x * Sigmoid(x) with a Swish op.

class SwishFusionWithSigmoidWithBeta : public ov::pass::MatcherPass
#include <swish_fusion.hpp>

SwishFusionWithSigmoid replaces a sub-graphs x * Sigmoid(x * beta) with a Swish op.

class SymbolicOptimizations : public ov::pass::ModelPass
#include <symbolic_optimizations.hpp>

Runs optimizations which are based on symbolic shape inference.

class SymbolicPropagation : public ov::pass::ModelPass
#include <symbolic_optimizations.hpp>

Assigns labels / symbols to all tensors on shapes and values. Uses shape inference and other special rules to propagate labels / symbols.

class TransposeConvert : public ov::pass::MatcherPass
#include <transpose_sinking.hpp>

TransposeConvert transformation sinks Transpose through Convert.

class TransposeEltwise : public ov::pass::MatcherPass
#include <transpose_sinking.hpp>

TransposeEltwise transformation sinks Transpose through Eltwise.

class TransposeFQReduction : public ov::pass::MatcherPass
#include <transpose_sinking.hpp>

TransposeFQReduction transformation sinks Transpose through FakeQuantize in case it is followed by reduction or squeeze.

class TransposeFuse : public ov::pass::MatcherPass
#include <transpose_sinking.hpp>

TransposeFuse transformation eliminates 2 consequtive Transposes if they result in no changes to input or fuses them to single Transpose if input gets changed.

class TransposeMatMul : public ov::pass::MatcherPass
#include <matmul_sr.hpp>
class TransposeReduction : public ov::pass::MatcherPass
#include <transpose_sinking.hpp>

TransposeReduction transformation sinks Transpose through Reduce operations.

class TransposeReshapeEliminationForMatmul : public ov::pass::MatcherPass
#include <transpose_reshape_elimination_for_matmul.hpp>

TransposeReshapeEliminationForMatmul transformation eliminates Transpose and Reshape which were created to align input and output dimension ranks before second MatMul input and after MatMul output (for example, after Einsum Decomposition inside TensorFlow 1 and OpenVINO EinsumDecomposition transformation)

class TransposeSinking : public ov::pass::GraphRewrite
#include <transpose_sinking.hpp>

TransposeSinking transformation sinks Transposes through known operations.

class TransposeToReshape : public ov::pass::MatcherPass
#include <transpose_to_reshape.hpp>

TransposeToReshape transformation replaces suitable Transposes with Reshape operation or optimizes them out.

class UniqueDecomposition : public ov::pass::MatcherPass
#include <unique_decomposition.hpp>
class UnrollIf : public ov::pass::ModelPass
#include <unroll_if.hpp>

The transformation replaces ‘If’ operations with one of the internal functions (bodies) if the provided condition is constant. The condition is true: ‘If’ op is replaced with then_body The condition is false ‘If’ op is replaced with else_body.

class UnrollTensorIterator : public ov::pass::ModelPass
#include <unroll_tensor_iterator.hpp>

Unrolls the body of the TensorIterator layer. Multiple body copies, the number of which is determined by the number of iterations of the TensorIterator layer, are created and connected to each other and to the external network. If the number of TensorIterator iterations is greater than 1, then additional Concat and Split layers are added to the network.

class UnsupportedNodesStridesPropagation : public ov::pass::MatcherPass
#include <strides_optimization.hpp>

UnsupportedNodesStridesPropagation inserts pooling between current node and its consumers if the consumers have different StridesProp attributes.

class UselessSliceEraser : public ov::pass::ModelPass
#include <optimize_strided_slice.hpp>

UselessSliceEraser transformation removes Slice/StridedSlice operations with equal input and output shapes.

class Validate : public ov::pass::ModelPass
#include <validate.hpp>

The Validate pass performs sanity checks on attributes and inputs, and computes output shapes and element types for all computation nodes in a given computation graph.

The verification and inference is done via invoking each node’s specific implementation of ov::Node::validate_and_infer_types() function.

By default, the ov::pass::Manager runs this pass after executing every optimization pass. This is to ensure that any update to the graph by an optimization pass does not break the shape and data type requirement on a computation node. This default validation run can be changed via calling the ov::pass::Manager::set_per_pass_validation(bool) function.

class VisualizeTree : public ov::pass::ModelPass
#include <visualize_tree.hpp>

VisualizeTree pass allows to serialize ov::Model to xDot format.

class WeightsDequantizeToFakeQuantize : public ov::pass::MatcherPass
#include <weights_dequantize_to_fake_quantize.hpp>

WeightsDequantizeToFakeQuantize transformation replaces Constant (i8) -> Convert (to fp) -> Subtract (zp) -> Multiply (scale) -> with Constant (i8) -> Convert (to fp) -> FakeQuantize -> deducing levels and FakeQuantize limits according to actual values in the weights Constant.

class WrapInterpolateIntoTransposes : public ov::pass::MatcherPass
#include <wrap_interpolate_into_transposes.hpp>

WrapInterpolateIntoTransposes transformation replaces Interpolate with Transpose -> Interpolate -> Transpose when 1) the source Interpolate has the static input rank; 2) ‘axes’ input is a Constant; 3) number of axes is equal to input rank minus 2; 4) axes contain 0 or 1. The reason of this transformation is that now CPU plugin supports interpolation only with respect to spatial dimensions, but TensorFlow frontend gives Interpolate with axes {1, 2} for 4D tensors.

namespace low_precision

Typedefs

typedef std::tuple<std::shared_ptr<ov::Node>, std::shared_ptr<ov::Node>> FakeQuantizeDequantizationValues
typedef std::shared_ptr<LayerTransformation> LayerTransformationPtr

Enums

enum levels

Values:

enumerator int4
enumerator int4_narrow_range
enumerator int8
enumerator int8_narrow_range
enumerator int16
enumerator int16_narrow_range
enumerator int32
enumerator int32_narrow_range

Functions

inline bool operator==(const DataPrecision &value1, const DataPrecision &value2)
inline bool operator!=(const DataPrecision &value1, const DataPrecision &value2)
inline std::ostream &operator<<(std::ostream &os, const DataPrecision &value)
template<typename T>
std::shared_ptr<Node> make_op_pattern(const ov::NodeVector &args)
template<typename T, typename ...Args>
std::shared_ptr<Node> fold(Args&&... args)
std::shared_ptr<Node> foldConvert(const Output<Node> &node, const element::Type targetPrecision)
template<typename T, typename ...Args>
std::shared_ptr<Node> fold_reshape(Args&&... args)
template<typename T>
ov::Any getAttribute(const std::shared_ptr<Node> &node)
template<typename T>
ov::Any getAttribute(const Input<Node> &input)
template<typename T>
ov::Any getAttributeFromOutput(const Output<Node> &output)
bool isDisabled(const std::shared_ptr<Node> &node)
inline std::ostream &operator<<(std::ostream &os, const QuantizationDetails &value)

Variables

class LP_TRANSFORMATIONS_API AlignQuantizationIntervals
class LP_TRANSFORMATIONS_API AlignQuantizationParameters
class LP_TRANSFORMATIONS_API BaseMatcherPass
class LP_TRANSFORMATIONS_API ConvertSubtractConstant
class LP_TRANSFORMATIONS_API TypeRelaxedReplacer
class LP_TRANSFORMATIONS_API MarkupOptimizations
class LP_TRANSFORMATIONS_API LowPrecision
class LP_TRANSFORMATIONS_API MarkupAvgPoolPrecisionPreserved
class LP_TRANSFORMATIONS_API MarkupCanBeQuantized
class LP_TRANSFORMATIONS_API MarkupPrecisions
class LP_TRANSFORMATIONS_API MarkupQuantizationGranularity
class LP_TRANSFORMATIONS_API PropagatePrecisions
template<class AttributeType>
class LP_TRANSFORMATIONS_API PropagateSharedValue
class LP_TRANSFORMATIONS_API PullReshapeThroughDequantization
class LP_TRANSFORMATIONS_API PullTransposeThroughDequantization
class AddTransformation : public ov::pass::low_precision::EltwiseBaseTransformation
#include <add.hpp>

AddTransformation propagates dequantization subtraction from one input branch to another and propagates dequantization multiplication from the same branch through Add operation.

For more details about the transformation, refer to AddTransformation page in the Inference Engine Developer Guide.

class AlignQuantizationIntervals : public ov::pass::ModelPass
#include <align_quantization_intervals.hpp>

AlignQuantizationIntervals transformation marks precision preserved operations subgraph by IntervalsAlignmentAttribute after FakeQuantize operations.

For more details about the transformation, refer to AlignQuantizationIntervals page in the Inference Engine Developer Guide.

class AlignQuantizationParameters : public ov::pass::ModelPass
#include <align_quantization_parameters.hpp>

AlignQuantizationParameters transformation marks precision preserved operations subgraph by QuantizationAlignmentAttribute attribute after FakeQuantize operations.

For more details about the transformation, refer to AlignQuantizationParameters page in the Inference Engine Developer Guide.

class AssignAndReadValueTransformation : public ov::pass::low_precision::LayerTransformation
#include <assign_and_read_value.hpp>
class AvgPoolTransformation : public ov::pass::low_precision::LayerTransformation
#include <avg_pool.hpp>

AvgPoolTransformation propagates dequantization operations through AvgPool operation.

For more details about the transformation, refer to AvgPoolTransformation page in the Inference Engine Developer Guide.

class BatchToSpaceTransformation : public ov::pass::low_precision::LayerTransformation
#include <batch_to_space.hpp>

BatchToSpaceTransformation propagates dequantization operations through BatchToSpace operation.

For more details about the transformation, refer to BatchToSpaceTransformation page in the Inference Engine Developer Guide.

class ClampTransformation : public ov::pass::low_precision::LayerTransformation
#include <clamp.hpp>

ClampTransformation propagates dequantization operations through Clamp operation.

For more details about the transformation, refer to ClampTransformation page in the Inference Engine Developer Guide.

class CleanupTransformation : public ov::pass::low_precision::LayerTransformation
#include <cleanup_transformation.hpp>

Base class for cleanup low precision transformation.

Subclassed by ov::pass::low_precision::EliminateFakeQuantizeTransformation, ov::pass::low_precision::FoldConvertTransformation, ov::pass::low_precision::FuseConvertTransformation, ov::pass::low_precision::FuseElementwiseToFakeQuantizeTransformation, ov::pass::low_precision::MultiplyToGroupConvolutionTransformation

class ConcatTransformation : public ov::pass::low_precision::LayerTransformation
#include <concat.hpp>

ConcatTransformation propagates dequantization operations through Concat operation.

For more details about the transformation, refer to ConcatTransformation page in the Inference Engine Developer Guide.

class ConvertSubtractConstant : public ov::pass::MatcherPass
#include <convert_subtract_constant.hpp>

ConvertSubtractConstant marks Convert operations on constant subgraph by DISABLED_CONSTANT_FOLDING attribute to prevent constant folding.

For more details about the transformation, refer to ConvertSubtractConstant page in the Inference Engine Developer Guide.

class ConvertTransformation : public ov::pass::low_precision::LayerTransformation
#include <convert.hpp>
class ConvolutionBackpropDataTransformation : public ov::pass::low_precision::WeightableLayerTransformation
#include <convolution_backprop_data.hpp>

ConvolutionBackpropDataTransformation propagates dequantization operations through ConvolutionBackpropData operation.

For more details about the transformation, refer to ConvolutionBackpropDataTransformation page in the Inference Engine Developer Guide.

class ConvolutionTransformation : public ov::pass::low_precision::WeightableLayerTransformation
#include <convolution.hpp>

ConvolutionTransformation propagates dequantization operations through Convolution operation.

For more details about the transformation, refer to ConvolutionTransformation page in the Inference Engine Developer Guide.

Subclassed by ov::pass::low_precision::GroupConvolutionTransformation

template<typename AttributeType, typename OperationType = ov::pass::pattern::op::Label>
class CreateAttribute : public ov::pass::low_precision::BaseMatcherPass
#include <create_attribute.hpp>

CreateAttribute transformation marks OperationType operations by AttributeType attribute.

For more details about the transformation, refer to CreateAttribute page in the Inference Engine Developer Guide.

template<typename AttributeType, typename OperationType>
class CreatePrecisionsDependentAttribute : public ov::pass::MatcherPass
#include <create_precisions_dependent_attribute.hpp>

CreatePrecisionsDependentAttribute transformation marks OperationType operations by PrecisionPreservedAttribute and AttributeType attributes with the same shared part.

For more details about the transformation, refer to CreatePrecisionsDependentAttribute page in the Inference Engine Developer Guide.

class DataPrecision
#include <layer_transformation.hpp>
class DepthToSpaceTransformation : public ov::pass::low_precision::TransparentBaseTransformation
#include <depth_to_space.hpp>

DepthToSpaceTransformation propagates dequantization operations through DepthToSpace operation.

For more details about the transformation, refer to DepthToSpaceTransformation page in the Inference Engine Developer Guide.

class EliminateFakeQuantizeTransformation : public ov::pass::low_precision::CleanupTransformation
#include <eliminate_fake_quantize.hpp>

EliminateFakeQuantizeTransformation removes FakeQuantize operations.

For more details about the transformation, refer to EliminateFakeQuantizeTransformation page in the Inference Engine Developer Guide.

class EltwiseBaseTransformation : public ov::pass::low_precision::LayerTransformation
#include <eltwise_base_transformation.hpp>

EltwiseBaseTransformation is base class for element-wise LPT transformations.

Subclassed by ov::pass::low_precision::AddTransformation, ov::pass::low_precision::MultiplyPartialTransformation

class Exception : public std::exception
#include <ie_lpt_exception.hpp>

Subclassed by ov::pass::low_precision::InferenceEngineLptException

class FakeQuantizeDecompositionTransformation : public ov::pass::low_precision::LayerTransformation
#include <fake_quantize_decomposition.hpp>

FakeQuantizeDecompositionTransformation decomposes FakeQuantize operations to quantize (FakeQuantize with changes output intervals and low precision output type) and dequantize operations.

For more details about the transformation, refer to FakeQuantizeDecompositionTransformation page in the Inference Engine Developer Guide.

class FakeQuantizeDequantization
#include <fake_quantize_dequantization.hpp>
class FakeQuantizeTransformation : public ov::pass::low_precision::LayerTransformation
#include <fake_quantize.hpp>

FakeQuantizeTransformation fuses dequantization operations into FakeQuantize operation.

For more details about the transformation, refer to FakeQuantizeTransformation page in the Inference Engine Developer Guide.

class FoldConvertTransformation : public ov::pass::low_precision::CleanupTransformation
#include <fold_convert.hpp>

FoldConvertTransformation evaluates Convert operation on Subtract constant subgraph. Important notice: this transformation ignores DisableConstantFolding runtime attribute.

For more details about the transformation, refer to FoldConvertTransformation page in the Inference Engine Developer Guide.

class FoldFakeQuantizeTransformation : public ov::pass::low_precision::LayerTransformation
#include <fold_fake_quantize.hpp>

FoldFakeQuantizeTransformation evaluate FakeQuantize operations.

For more details about the transformation, refer to FoldFakeQuantizeTransformation page in the Inference Engine Developer Guide.

class FuseConvertTransformation : public ov::pass::low_precision::CleanupTransformation
#include <fuse_convert.hpp>

FuseConvertTransformation fuses Convert operation with Multiply, Subtract or Add operations.

For more details about the transformation, refer to FuseConvertTransformation page in the Inference Engine Developer Guide.

class FuseElementwiseToFakeQuantizeTransformation : public ov::pass::low_precision::CleanupTransformation
#include <fuse_elementwise_to_fake_quantize.hpp>

Base class for fuse elementwise to FakeQuantize low precision transformation.

Subclassed by ov::pass::low_precision::FuseMultiplyToFakeQuantizeTransformation, ov::pass::low_precision::FuseSubtractToFakeQuantizeTransformation

class FuseMultiplyToFakeQuantizeTransformation : public ov::pass::low_precision::FuseElementwiseToFakeQuantizeTransformation
#include <fuse_multiply_to_fake_quantize.hpp>

FuseMultiplyToFakeQuantizeTransformation fuses Multiply operation to FakeQuantize.

For more details about the transformation, refer to FuseMultiplyToFakeQuantizeTransformation page in the Inference Engine Developer Guide.

class FuseSubtractToFakeQuantizeTransformation : public ov::pass::low_precision::FuseElementwiseToFakeQuantizeTransformation
#include <fuse_subtract_to_fake_quantize.hpp>

FuseSubtractToFakeQuantizeTransformation fuses Subtract operation to FakeQuantize.

For more details about the transformation, refer to FuseSubtractToFakeQuantizeTransformation page in the Inference Engine Developer Guide.

class GatherTransformation : public ov::pass::low_precision::LayerTransformation
#include <gather.hpp>
class GroupConvolutionTransformation : public ov::pass::low_precision::ConvolutionTransformation
#include <group_convolution.hpp>

GroupConvolutionTransformation propagates dequantization operations through GroupConvolution operation.

For more details about the transformation, refer to GroupConvolutionTransformation page in the Inference Engine Developer Guide.

class InferenceEngineLptException : public ov::pass::low_precision::Exception
#include <ie_lpt_exception.hpp>
class InterpolateTransformation : public ov::pass::low_precision::LayerTransformation
#include <interpolate.hpp>

InterpolateTransformation propagates dequantization operations through Interpolate operation.

For more details about the transformation, refer to InterpolateTransformation page in the Inference Engine Developer Guide.

class LayerTransformation : public ov::pass::MatcherPass
#include <layer_transformation.hpp>

Base class for low precision transformation.

Subclassed by ov::pass::low_precision::AssignAndReadValueTransformation, ov::pass::low_precision::AvgPoolTransformation, ov::pass::low_precision::BatchToSpaceTransformation, ov::pass::low_precision::ClampTransformation, ov::pass::low_precision::CleanupTransformation, ov::pass::low_precision::ConcatTransformation, ov::pass::low_precision::ConvertTransformation, ov::pass::low_precision::EltwiseBaseTransformation, ov::pass::low_precision::FakeQuantizeDecompositionTransformation, ov::pass::low_precision::FakeQuantizeTransformation, ov::pass::low_precision::FoldFakeQuantizeTransformation, ov::pass::low_precision::GatherTransformation, ov::pass::low_precision::InterpolateTransformation, ov::pass::low_precision::MVNTransformation, ov::pass::low_precision::MatMulTransformation, ov::pass::low_precision::MaxPoolTransformation, ov::pass::low_precision::MoveFakeQuantize, ov::pass::low_precision::NormalizeL2Transformation, ov::pass::low_precision::PReluTransformation, ov::pass::low_precision::PadTransformation, ov::pass::low_precision::RecurrentCellTransformation, ov::pass::low_precision::ReduceBaseTransformation, ov::pass::low_precision::ReluTransformation, ov::pass::low_precision::ReshapeTransformation, ov::pass::low_precision::ShuffleChannelsTransformation, ov::pass::low_precision::SpaceToBatchTransformation, ov::pass::low_precision::SplitTransformation, ov::pass::low_precision::SqueezeTransformation, ov::pass::low_precision::StridedSliceTransformation, ov::pass::low_precision::SubtractTransformation, ov::pass::low_precision::TransparentBaseTransformation, ov::pass::low_precision::TransposeTransformation, ov::pass::low_precision::UnsqueezeTransformation, ov::pass::low_precision::WeightableLayerTransformation

class Params
#include <layer_transformation.hpp>
class PrecisionDetails
#include <layer_transformation.hpp>
class LowPrecision : public ov::pass::ModelPass
#include <low_precision.hpp>
class MarkupAvgPoolPrecisionPreserved : public ov::pass::ModelPass
#include <markup_avg_pool_precision_preserved.hpp>

MarkupAvgPoolPrecisionPreserved transformation marks AvgPool operations as precision preserved or not.

For more details about the transformation, refer to MarkupAvgPoolPrecisionPreserved page in the Inference Engine Developer Guide.

class MarkupBias : public ov::pass::MatcherPass
#include <markup_bias.hpp>

MarkupBias transformation marks biases after target layers.

For more details about the transformation, refer to MarkupBias page in the Inference Engine Developer Guide.

class MarkupCanBeQuantized : public ov::pass::ModelPass
#include <markup_can_be_quantized.hpp>

MarkupCanBeQuantized transformation marks Convolution, ConvolutionBackpropData, GroupConvolution and Concat operations as able to be quantized or not. If an operation is not quantized, then PrecisionsAttribute attribute instance is created with empty precisions.

For more details about the transformation, refer to MarkupCanBeQuantized page in the Inference Engine Developer Guide.

class MarkupOptimizations : public ov::pass::ModelPass
#include <low_precision.hpp>
class MarkupPrecisions : public ov::pass::ModelPass
#include <markup_precisions.hpp>

MarkupPrecisions transformation marks: 1) not supported operations by PrecisionsAttribute attribute with empty precisions, 2) operations with required precisions by PrecisionsAttribute attribute according to the provided restrictions, 3) precision preserved operations by PrecisionPreservedAttribute attribute.

For more details about the transformation, refer to MarkupPrecisions page in the Inference Engine Developer Guide.

class Restriction
#include <markup_precisions.hpp>
class RestrictionByVersion
#include <markup_precisions.hpp>
class MarkupQuantizationGranularity : public ov::pass::ModelPass
#include <markup_quantization_granularity.hpp>

MarkupPerTensorQuantization transformation marks operations as required per-tensor quantization according to the provided restrictions.

For more details about the transformation, refer to MarkupPerTensorQuantization page in the Inference Engine Developer Guide.

class PerTensorQuantization
#include <markup_quantization_granularity.hpp>
class MatMulTransformation : public ov::pass::low_precision::LayerTransformation
#include <mat_mul.hpp>

MatMulTransformation propagates dequantization operations through MatMul operation.

For more details about the transformation, refer to MatMulTransformation page in the Inference Engine Developer Guide.

class MaxPoolTransformation : public ov::pass::low_precision::LayerTransformation
#include <max_pool.hpp>

MaxPoolTransformation propagates dequantization operations through MaxPool operation.

For more details about the transformation, refer to MaxPoolTransformation page in the Inference Engine Developer Guide.

class MoveFakeQuantize : public ov::pass::low_precision::LayerTransformation
#include <move_fake_quantize.hpp>
class MultiplyPartialTransformation : public ov::pass::low_precision::EltwiseBaseTransformation
#include <multiply_partial.hpp>

MultiplyPartialTransformation propagates dequantization operations through Multiply operation.

For more details about the transformation, refer to MultiplyPartialTransformation page in the Inference Engine Developer Guide.

class MultiplyToGroupConvolutionTransformation : public ov::pass::low_precision::CleanupTransformation
#include <multiply_to_group_convolution.hpp>

MultiplyToGroupConvolutionTransformation replace quantized Multiply operations to GroupConvolution to speed up inference.

For more details about the transformation, refer to MultiplyToGroupConvolutionTransformation page in the Inference Engine Developer Guide.

class MultiplyTransformation : public ov::pass::low_precision::WeightableLayerTransformation
#include <multiply.hpp>

MultiplyTransformation propagates dequantization operations through Multiply operation.

For more details about the transformation, refer to MultiplyTransformation page in the Inference Engine Developer Guide.

class MVNTransformation : public ov::pass::low_precision::LayerTransformation
#include <mvn.hpp>

MVNTransformation propagates dequantization operations through MVN operation.

For more details about the transformation, refer to MVNTransformation page in the Inference Engine Developer Guide.

class NetworkHelper
#include <network_helper.hpp>

NetworkHelper class encapsulates manipulations with ov::Model.

class InsertDequantizationResult
#include <network_helper.hpp>
class NormalizeL2Transformation : public ov::pass::low_precision::LayerTransformation
#include <normalize_l2.hpp>

NormalizeL2Transformation propagates dequantization operations through NormalizeL2 operation.

For more details about the transformation, refer to NormalizeL2Transformation page in the Inference Engine Developer Guide.

class PadTransformation : public ov::pass::low_precision::LayerTransformation
#include <pad.hpp>

PadTransformation propagates dequantization operations through Pad operation.

For more details about the transformation, refer to PadTransformation page in the Inference Engine Developer Guide.

class PortQuantizationGranularityRestriction
#include <port_quantization_granularity_restriction.hpp>
class PrecisionsRestriction
#include <precisions_restriction.hpp>

PrecisionsRestriction defines a set of precision restrictions for each input port Common precision restriction can be also set for several ports. In this case, an operation will have the same precision for mentioned.

// One restriction for each port PrecisionsRestriction::create<ov::opset1::Convolution>({ {{0}, {ov::element::u8}}, {{1}, {ov::element::i8}}, }),

// Common precision restriction for several ports: // both inputs will have the same precision PrecisionsRestriction::create<ov::opset5::LSTMSequence>({ {{0, 1}, {ov::element::u8, ov::element::i8}} }),

class PReluTransformation : public ov::pass::low_precision::LayerTransformation
#include <prelu.hpp>

PReluTransformation propagates dequantization operations through PRelu operation.

For more details about the transformation, refer to PReluTransformation page in the Inference Engine Developer Guide.

class PropagatePrecisions : public ov::pass::ModelPass
#include <propagate_precisions.hpp>

PropagatePrecisions transformation propagates PrecisionsAttribute attribute instances precision preserved operations.

For more details about the transformation, refer to PropagatePrecisions page in the Inference Engine Developer Guide.

template<class AttributeType>
class PropagateSharedValue : public ov::pass::ModelPass
#include <propagate_shared_value.hpp>

PropagateSharedValue transformation propagates shared value AttributeType attribute instances through precision preserved operations.

For more details about the transformation, refer to PropagateSharedValue page in the Inference Engine Developer Guide.

template<typename AttributeType>
class PropagateThroughPrecisionPreserved : public ov::pass::MatcherPass
#include <propagate_through_precision_preserved.hpp>

PropagateThroughPrecisionPreserved transformation propagates AttributeType attribute instances through precision preserved operations.

For more details about the transformation, refer to PropagateThroughPrecisionPreserved page in the Inference Engine Developer Guide.

template<typename AttributeType>
class PropagateToInput : public ov::pass::MatcherPass
#include <propagate_to_input.hpp>

PropagateToInput transformation propagates AttributeType shared value attribute instances from parent output ports to consumers input ports.

For more details about the transformation, refer to PropagateToInput page in the Inference Engine Developer Guide.

class PullReshapeThroughDequantization : public ov::pass::MatcherPass
#include <pull_reshape_through_dequantization.hpp>

PullReshapeThroughDequantization propagates dequantization operations through Reshape operations. The transformation is used on constant subgraph weights to prepare a model for the next low precision transformations.

For more details about the transformation, refer to PullReshapeThroughDequantization page in the Inference Engine Developer Guide.

class PullTransposeThroughDequantization : public ov::pass::MatcherPass
#include <pull_transpose_through_dequantization.hpp>

PullTransposeThroughDequantization propagates dequantization operations through Transpose operations. The transformation is used on constant subgraph weights to prepare a model for the next low precision transformations.

For more details about the transformation, refer to PullTransposeThroughDequantization page in the Inference Engine Developer Guide.

class QuantizationDetails
#include <quantization_details.hpp>
class QuantizationGranularityRestriction
#include <quantization_granularity_restriction.hpp>
class RecurrentCellTransformation : public ov::pass::low_precision::LayerTransformation
#include <recurrent_cell.hpp>
class ReduceBaseTransformation : public ov::pass::low_precision::LayerTransformation
#include <reduce_base_transformation.hpp>

ReduceBaseTransformation: base class for Reduce*Transformation, detects dequantization operations in front of the Reduce* operation and propagates them through the Reduce* if possible.

Subclassed by ov::pass::low_precision::ReduceMaxTransformation, ov::pass::low_precision::ReduceMeanTransformation, ov::pass::low_precision::ReduceMinTransformation, ov::pass::low_precision::ReduceSumTransformation

class ReduceMaxTransformation : public ov::pass::low_precision::ReduceBaseTransformation
#include <reduce_max.hpp>

ReduceMaxTransformation propagates dequantization operations through ReduceMax operation.

For more details about the transformation, refer to ReduceMaxTransformation page in the Inference Engine Developer Guide.

class ReduceMeanTransformation : public ov::pass::low_precision::ReduceBaseTransformation
#include <reduce_mean.hpp>

ReduceMeanTransformation propagates dequantization operations through ReduceMean operation.

For more details about the transformation, refer to ReduceMeanTransformation page in the Inference Engine Developer Guide.

class ReduceMinTransformation : public ov::pass::low_precision::ReduceBaseTransformation
#include <reduce_min.hpp>

ReduceMinTransformation propagates dequantization operations through ReduceMin operation.

For more details about the transformation, refer to ReduceMinTransformation page in the Inference Engine Developer Guide.

class ReduceSumTransformation : public ov::pass::low_precision::ReduceBaseTransformation
#include <reduce_sum.hpp>

ReduceSumTransformation propagates dequantization operations through ReduceSum operation.

For more details about the transformation, refer to ReduceSumTransformation page in the Inference Engine Developer Guide.

class ReluTransformation : public ov::pass::low_precision::LayerTransformation
#include <relu.hpp>

ReluTransformation propagates dequantization operations through Relu operation.

For more details about the transformation, refer to ReluTransformation page in the Inference Engine Developer Guide.

class ReshapeTransformation : public ov::pass::low_precision::LayerTransformation
#include <reshape.hpp>

ReshapeTransformation propagates dequantization operations through Reshape operation.

For more details about the transformation, refer to ReshapeTransformation page in the Inference Engine Developer Guide.

class ShuffleChannelsTransformation : public ov::pass::low_precision::LayerTransformation
#include <shuffle_channels.hpp>

ShuffleChannelsTransformation propagates dequantization operations through ShuffleChannels operation.

For more details about the transformation, refer to ShuffleChannelsTransformation page in the Inference Engine Developer Guide.

class SpaceToBatchTransformation : public ov::pass::low_precision::LayerTransformation
#include <space_to_batch.hpp>

SpaceToBatchTransformation propagates dequantization operations through SpaceToBatch operation.

For more details about the transformation, refer to SpaceToBatchTransformation page in the Inference Engine Developer Guide.

class SplitTransformation : public ov::pass::low_precision::LayerTransformation
#include <split.hpp>

SplitTransformation propagates dequantization operations through Split operation.

For more details about the transformation, refer to SplitTransformation page in the Inference Engine Developer Guide.

Subclassed by ov::pass::low_precision::VariadicSplitTransformation

class SqueezeTransformation : public ov::pass::low_precision::LayerTransformation
#include <squeeze.hpp>

SqueezeTransformation propagates dequantization operations through Squeeze operation.

For more details about the transformation, refer to SqueezeTransformation page in the Inference Engine Developer Guide.

class StridedSliceTransformation : public ov::pass::low_precision::LayerTransformation
#include <strided_slice.hpp>

StridedSliceTransformation propagates dequantization operations through StridedSlice operation.

For more details about the transformation, refer to StridedSliceTransformation page in the Inference Engine Developer Guide.

class SubtractTransformation : public ov::pass::low_precision::LayerTransformation
#include <subtract.hpp>

SubtractTransformation propagates dequantization operations through Subtract operation.

For more details about the transformation, refer to SubtractTransformation page in the Inference Engine Developer Guide.

class TransformationContext
#include <transformation_context.hpp>

TransformationContext instance is used to pass model transformation context data between transformations.

class TransparentBaseTransformation : public ov::pass::low_precision::LayerTransformation
#include <transparent_base_transformation.hpp>

TransparentBaseTransformation is base type for precision preserved operation transformation.

Subclassed by ov::pass::low_precision::DepthToSpaceTransformation

class TransposeTransformation : public ov::pass::low_precision::LayerTransformation
#include <transpose.hpp>

TransposeTransformation propagates dequantization operations through Transpose operation.

For more details about the transformation, refer to TransposeTransformation page in the Inference Engine Developer Guide.

class TypeRelaxedReplacer : public ov::pass::GraphRewrite
#include <low_precision.hpp>
class UnsqueezeTransformation : public ov::pass::low_precision::LayerTransformation
#include <unsqueeze.hpp>

UnsqueezeTransformation propagates dequantization operations through Unsqueeze operation.

For more details about the transformation, refer to UnsqueezeTransformation page in the Inference Engine Developer Guide.

template<typename AttributeType, typename ExpectedAttributeType = AttributeType>
class UpdateSharedPrecisionPreserved : public ov::pass::MatcherPass
#include <update_shared_precision_preserved.hpp>

UpdateSharedPrecisionPreserved transformation updates shared AttributeType attribute instance value to true for precision preserved operations if ExpectedAttributeType exist.

For more details about the transformation, refer to UpdateSharedPrecisionPreserved page in the Inference Engine Developer Guide.

class VariadicSplitTransformation : public ov::pass::low_precision::SplitTransformation
#include <variadic_split.hpp>

VariadicSplitTransformation propagates dequantization operations through VariadicSplit operation.

For more details about the transformation, refer to VariadicSplitTransformation page in the Inference Engine Developer Guide.

class WeightableLayerTransformation : public ov::pass::low_precision::LayerTransformation
#include <weightable_layer_transformation.hpp>

WeightableLayerTransformation is base type for weightable operation transformation.

Subclassed by ov::pass::low_precision::ConvolutionBackpropDataTransformation, ov::pass::low_precision::ConvolutionTransformation, ov::pass::low_precision::MultiplyTransformation

struct CanBeTransformedParams
#include <weightable_layer_transformation.hpp>
namespace itt
namespace domains

Functions

OV_ITT_DOMAIN(LowPrecisionTransformations)
OV_ITT_DOMAIN(LPT)
OV_ITT_DOMAIN(LPT_LT)
namespace precision_set

Functions

LP_TRANSFORMATIONS_API const std::vector< element::Type > & get_int8_support ()
LP_TRANSFORMATIONS_API const std::vector< element::Type > & get_int8_int16_int32_support ()
namespace pattern

Typedefs

using RPatternValueMap = std::map<std::shared_ptr<Node>, OutputVector>
using PatternValueMap = std::map<std::shared_ptr<Node>, Output<Node>>
using PatternValueMaps = std::vector<PatternValueMap>
using PatternMap = std::map<std::shared_ptr<Node>, std::shared_ptr<Node>>

Functions

OPENVINO_API std::shared_ptr< Node > any_input ()
OPENVINO_API std::shared_ptr< Node > any_input (const pattern::op::ValuePredicate &pred)
PatternMap as_pattern_map(const PatternValueMap &pattern_value_map)
PatternValueMap as_pattern_value_map(const PatternMap &pattern_map)
template<typename T>
std::function<bool(std::shared_ptr<Node>)> has_class()
OPENVINO_API std::function< bool(Output< Node >)> consumers_count (size_t n)
OPENVINO_API std::function< bool(Output< Node >)> consumers_more_than (size_t n)
OPENVINO_API std::function< bool(Output< Node >)> has_static_dim (size_t pos)
OPENVINO_API std::function< bool(Output< Node >)> has_static_dims (const std::vector< size_t > &dims)
OPENVINO_API std::function< bool(Output< Node >)> has_static_shape ()
OPENVINO_API std::function< bool(Output< Node >)> has_static_rank ()
OPENVINO_API std::function< bool(Output< Node >)> rank_equals (const Dimension &expected_rank)
OPENVINO_API std::function< bool(Output< Node >)> type_matches (const element::Type &type)
OPENVINO_API std::function< bool(Output< Node >)> type_matches_any (const std::vector< element::Type > &types)
template<class T>
void collect_wrap_info(std::vector<DiscreteTypeInfo> &info)
template<class T, class ...Targs, typename std::enable_if<sizeof...(Targs) != 0, bool>::type = true>
void collect_wrap_info(std::vector<DiscreteTypeInfo> &info)
template<class ...Args>
std::shared_ptr<Node> wrap_type(const OutputVector &inputs, const pattern::op::ValuePredicate &pred)
template<class ...Args>
std::shared_ptr<Node> wrap_type(const OutputVector &inputs = {})
template<class ...Args>
std::shared_ptr<Node> wrap_type(const pattern::op::ValuePredicate &pred)
class Matcher
#include <matcher.hpp>

Matcher looks for node patterns in a computation graph. The patterns are described by an automaton that is described by an extended computation graph. The matcher executes by attempting to match the start node of the pattern to a computation graph value (output of a Node). In addition to determing if a match occurs, a pattern node may add graph nodes to a list of matched nodes, associate nodes with graph values, and start submatches. Submatches add match state changes to the enclosing match if the submatch succeeds; otherwise the state is reverted.

The default match behavior of a pattern node with a graph nodes is that the computation graph value is added to the end of the matched value list and the match succeeds if the node/pattern types match and the input values match. In the case of a commutative node, the inputs can match in any order. If the matcher is in strict mode, the graph value element type and shape must also match.

Pattern nodes that have different match behavior are in ov::pass::pattern::op and have descriptions of their match behavior.

Public Functions

inline Matcher(const Output<Node> &pattern_node, const std::string &name, bool strict_mode)

Constructs a Matcher object.

Parameters
  • pattern_node – is a pattern sub graph that will be matched against input graphs

  • name – is a string which is used for logging and disabling a matcher

  • strict_mode – forces a matcher to consider shapes and ET of nodes

bool match(const Output<Node> &graph_value)

Matches a pattern to graph_node.

Parameters

graph_value – is an input graph to be matched against

bool match(const Output<Node> &graph_value, const PatternMap &previous_matches)

Matches a pattern to graph_node.

Parameters
  • graph_value – is an input graph to be matched against

  • previous_matches – contains previous mappings from labels to nodes to use

size_t add_node(Output<Node> node)

Low-level helper to match recurring patterns.

Parameters
  • graph – is a graph to be matched against

  • pattern – is a recurring pattern

  • rpattern – specifies a node to recur from next

  • patterns – a map from labels to matches

MatcherState start_match()

Try a match.

class MatcherState
#include <matcher.hpp>
namespace op

Typedefs

using NodePredicate = std::function<bool(std::shared_ptr<Node>)>
using ValuePredicate = std::function<bool(const Output<Node> &value)>

Functions

OPENVINO_API ValuePredicate as_value_predicate (NodePredicate pred)
class Any : public ov::pass::pattern::op::Pattern
#include <any.hpp>

The graph value is to the matched value list. If the predicate is true for the node and the arguments match, the match succeeds.

Public Functions

inline Any(const element::Type &type, const PartialShape &s, ValuePredicate pred, const OutputVector &wrapped_values)

creates a Any node containing a sub-pattern described by

See also

type and

See also

shape.

inline Any(const Output<Node> &node, ValuePredicate pred, const OutputVector &wrapped_values)

creates a Any node containing a sub-pattern described by the type and shape of

See also

node.

class AnyOf : public ov::pass::pattern::op::Pattern
#include <any_of.hpp>

The graph value is added to the matched values list. If the predicate is true for the graph node, a submatch is performed on the input of AnyOf and each input of the graph node. The first match that succeeds results in a successful match. Otherwise the match fails.

AnyOf may be given a type and shape for use in strict mode.

Public Functions

inline AnyOf(const element::Type &type, const PartialShape &s, ValuePredicate pred, const OutputVector &wrapped_values)

creates a AnyOf node containing a sub-pattern described by

See also

type and

See also

shape.

inline AnyOf(const Output<Node> &node, ValuePredicate pred, const OutputVector &wrapped_values)

creates a AnyOf node containing a sub-pattern described by the type and shape of

See also

node.

class AnyOutput : public ov::pass::pattern::op::Pattern
#include <any_output.hpp>

Matches any output of a node.

Public Functions

inline AnyOutput(const std::shared_ptr<Node> &pattern)

creates an AnyOutput node matching any output of a node

Parameters

node – The node to match

class Branch : public ov::pass::pattern::op::Pattern
#include <branch.hpp>

A branch adds a loop to the pattern. The branch match is successful if the destination node pattern matches the graph value. The destination node is a node in the pattern graph that will not have been created some time after the Branch node is created; use set_destination to add it.

The branch destination is not stored as a shared pointer to prevent reference cycles. Thus the destination node must be referenced in some other way to prevent it from being deleted.

Public Functions

inline Branch()

Creates a Branch pattern.

class Capture : public ov::pass::pattern::op::Pattern
#include <capture.hpp>

Experimental for support of recurrent matches.

Capture adds the pattern value map to a list of pattern value maps and resets matches for pattern nodes not in the static node list. The match always succeeds.

Public Functions

inline std::set<Node*> get_static_nodes()

static nodes are retained after a capture. All other nodes are dropped

class Label : public ov::pass::pattern::op::Pattern
#include <label.hpp>

Fails if the predicate returns false on the graph value.

The graph value is added to the matched values list. If the Label is already associated with a value, the match succeeds if the value is the same as the graph value. Otherwise, the label is associated with the graph value and the match succeeds if the pattern input matches the graph value.

DEPRECATED: If no inputs are given to Label, a True node is serves as the input. If more than one inputs are given, an Or pattern of the inputs serves as the input.

Public Functions

inline Label(const element::Type &type, const PartialShape &s, const ValuePredicate pred, const OutputVector &wrapped_values)

creates a Label node containing a sub-pattern described by

this Label node can be bound only to the nodes in the input graph that match the pattern specified by

See also

type and

See also

shape.

See also

wrapped_nodes Example:

auto add = a + b; // a and b are op::Parameter in this example
auto label = std::make_shared<pattern::op::Label>(element::f32,
                                                  PartialShape{2,2},
                                                  nullptr,
                                                  OutputVector{add});

inline Label(const Output<Node> &value, const ValuePredicate pred, const OutputVector &wrapped_values)

creates a Label node containing a sub-pattern described by the type and shape of

this Label node can be bound only to the nodes in the input graph that match the pattern specified by

See also

node.

See also

wrapped_values Example:

auto add = a + b; // a and b are op::Parameter in this example
auto label = std::make_shared<pattern::op::Label>(add,
                                                  nullptr,
                                                  OutputVector{add});

class Or : public ov::pass::pattern::op::Pattern
#include <or.hpp>

A submatch on the graph value is performed on each input to the Or; the match succeeds on the first match. Otherwise the match fails.

Public Functions

inline Or(const OutputVector &patterns)

creates an Or node matching one of several sub-patterns in order. Does not add node to match list.

Parameters

patterns – The patterns to try for matching

class Pattern : public ov::Node
#include <pattern.hpp>

Subclassed by ov::pass::pattern::op::Any, ov::pass::pattern::op::AnyOf, ov::pass::pattern::op::AnyOutput, ov::pass::pattern::op::Branch, ov::pass::pattern::op::Capture, ov::pass::pattern::op::Label, ov::pass::pattern::op::Or, ov::pass::pattern::op::Skip, ov::pass::pattern::op::True, ov::pass::pattern::op::WrapType

Public Functions

inline Pattern(const OutputVector &patterns, ValuePredicate pred)

a base class for

See also

Skip and

See also

Label

class Skip : public ov::pass::pattern::op::Pattern
#include <skip.hpp>

The graph value is added to the matched value list. If the predicate is true, the match succeeds if the arguments match; if the predicate is false, the match succeeds if the pattern input matches the graph value.

class True : public ov::pass::pattern::op::Pattern
#include <true.hpp>

The match always succeeds.

Public Functions

inline True()

Always matches, does not add node to match list.

class WrapType : public ov::pass::pattern::op::Pattern
#include <wrap_type.hpp>
namespace transpose_sinking
class TSBinaryBackward : public ov::pass::MatcherPass
#include <ts_binary.hpp>

TSBinaryBackward transformation sinks Transpose through BinaryElementwiseArithmetic, BinaryElementwiseComparison, BinaryElementwiseLogical and PRelu operations in the backward direction.

class TSBinaryForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_binary.hpp>

TSBinaryForward transformation sinks Transpose through BinaryElementwiseArithmetic, BinaryElementwiseComparison, BinaryElementwiseLogical and PRelu operations in the forward direction.

class TSConcatBackward : public ov::pass::MatcherPass
#include <ts_concat.hpp>

TSConcatBackward transformation sinks Transpose through Concat operation in the backward direction.

class TSConcatForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_concat.hpp>

TSConcatForward transformation sinks Transpose through Concat operation in the forward direction.

class TSCumSumBackward : public ov::pass::MatcherPass
#include <ts_cumsum.hpp>

TSCumSumBackward transformation sinks Transpose through CumSum in the backward direction.

class TSCumSumForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_cumsum.hpp>

TSCumSumForward transformation sinks Transpose through CumSum in the forward direction.

class TSDataMovementBackward : public ov::pass::MatcherPass
#include <ts_data_movement.hpp>

TSDataMovementBackward transformation sinks Transpose through BatchToSpace, SpaceToBatch, ReverseSequence and Pad operations in the backward direction. These operations are categorized as “DataMovement” and are handled in a similar way in this transformation.

class TSDataMovementForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_data_movement.hpp>

TSDataMovementForward transformation sinks Transpose through BatchToSpace, SpaceToBatch, ReverseSequence and Pad operations in the forward direction. These operations are categorized as “DataMovement” and are handled in a similar way in this transformation.

class TSForwardBase : public ov::pass::MatcherPass
#include <ts_base.hpp>

TSForwardBase is a base class for all forward transformations.

Subclassed by ov::pass::transpose_sinking::TSBinaryForward, ov::pass::transpose_sinking::TSConcatForward, ov::pass::transpose_sinking::TSCumSumForward, ov::pass::transpose_sinking::TSDataMovementForward, ov::pass::transpose_sinking::TSGatherForward, ov::pass::transpose_sinking::TSInterpolateForward, ov::pass::transpose_sinking::TSReductionForward, ov::pass::transpose_sinking::TSShapeOfForward, ov::pass::transpose_sinking::TSSliceForward, ov::pass::transpose_sinking::TSSplitForward, ov::pass::transpose_sinking::TSSqueezeForward, ov::pass::transpose_sinking::TSTileForward, ov::pass::transpose_sinking::TSUnaryForward, ov::pass::transpose_sinking::TSUnsqueezeForward

class TSFuse : public ov::pass::MatcherPass
#include <ts_fuse.hpp>

TSFuse transformation eliminates 2 consecutive Transposes if they result in no changes to input or fuses them to single Transpose if input gets changed.

class TSGatherBackward : public ov::pass::MatcherPass
#include <ts_gather.hpp>

TSGatherBackward transformation sinks Transpose through Gather operation in the backward direction.

class TSGatherForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_gather.hpp>

TSGatherForward transformation sinks Transpose through Gather operations in the forward direction.

class TSGeneral : public ov::pass::ModelPass
#include <ts_general.hpp>

TSGeneral transformation combines TSGeneralForward and TSGeneralBackward transformations into single ModelPass pass and inserts ConstantFolding pass after them.

class TSGeneralBackward : public ov::pass::GraphRewrite
#include <ts_general.hpp>

TSGeneralBackward transformation combines all TransposeSinkingBackward* transformations into single GraphRewrite pass.

class TSGeneralForward : public ov::pass::GraphRewrite
#include <ts_general.hpp>

TSGeneralForward transformation combines all TransposeSinkingForward* transformations into single GraphRewrite pass.

class TSInterpolateBackward : public ov::pass::MatcherPass
#include <ts_interpolate.hpp>

TSInterpolateBackward transformation sinks Transpose through Interpolate operation in the backward direction.

class TSInterpolateForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_interpolate.hpp>

TSInterpolateForward transformation sinks Transpose through Interpolate operation in the forward direction.

class TSReductionBackward : public ov::pass::MatcherPass
#include <ts_reduction.hpp>

TSReductionBackward transformation sinks Transpose through Reduce operations in the backward direction.

class TSReductionForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_reduction.hpp>

TSReductionForward transformation sinks Transpose through Reduce operations in the forward direction.

class TSResetNoSinkingAttribute : public ov::pass::MatcherPass
#include <ts_reset_no_sinking_attribute.hpp>

TSResetNoSinkingAttribute transformation resets all NoSinkingAttribute runtime attributes in Transpose operations.

class TSShapeOfForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_shape_of.hpp>

TSShapeOfForward transformation sinks Transpose through ShapeOf in the forward direction.

It replaces:

+——&#8212;+ |Transpose| +——&#8212;+ | v +——&#8212;+ | ShapeOf | +——&#8212;+

with the following:

+——&#8212;+ | ShapeOf | +——&#8212;+ | v +—&#8212;+ |Gather| +—&#8212;+

class TSSliceBackward : public ov::pass::MatcherPass
#include <ts_slice.hpp>
class TSSliceForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_slice.hpp>
class TSSplitBackward : public ov::pass::MatcherPass
#include <ts_split.hpp>

TSSplitBackward transformation sinks Transpose through Split, VariadicSplit operations in the backward direction.

class TSSplitForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_split.hpp>

TSSplitForward transformation sinks Transpose through Split, VariadicSplit operations in the forward direction.

class TSSqueezeBackward : public ov::pass::MatcherPass
#include <ts_squeeze.hpp>

TSSqueezeBackward transformation sinks Transpose through Reshape, Squeeze operations in the backward direction.

class TSSqueezeForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_squeeze.hpp>

TSSqueezeForward transformation sinks Transpose through Reshape, Squeeze operations in the forward direction.

class TSTileBackward : public ov::pass::MatcherPass
#include <ts_tile.hpp>

TSTileBackward transformation sinks Transpose through Tile in the backward direction.

class TSTileForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_tile.hpp>

TSTileForward transformation sinks Transpose through Tile in the forward direction.

class TSUnaryBackward : public ov::pass::MatcherPass
#include <ts_unary.hpp>

TSUnaryBackward transformation sinks Transpose through UnaryElementwiseArithmetic, Clamp, Elu, SoftPlus, LogicalNot, Convert, IsInf, IsNaN, IsFinite in the backward direction.

class TSUnaryForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_unary.hpp>

TSUnaryForward transformation sinks Transpose through UnaryElementwiseArithmetic, Clamp, Elu, SoftPlus, LogicalNot, Convert, IsInf, IsNaN, IsFinite operations in the forward direction.

class TSUnsqueezeBackward : public ov::pass::MatcherPass
#include <ts_unsqueeze.hpp>

TSUnsqueezeBackward transformation sinks Transpose through Unsqueeze, Reshape operations in the backward direction.

class TSUnsqueezeForward : public ov::pass::transpose_sinking::TSForwardBase
#include <ts_unsqueeze.hpp>

TSUnsqueezeForward transformation sinks Transpose through Unsqueeze, Reshape operations in the forward direction.

namespace utils

Functions

TransposeInputsInfo GetFirstTransposeInput(const std::shared_ptr<ov::Node>&, bool const_transpose_order, const std::vector<size_t> &indices = {})

Finds node first input that is a transpose operation and returns filled TransposeInputsInfo for it.

bool IfNodeHasTransposeInputs(const ov::Output<ov::Node>&, const std::vector<size_t> &indices = {})

Checks if.

  • has any input node that is a transpose operation

ov::AxisVector ReverseTransposeOrder(const ov::AxisVector&)

Reverses order of transpose operation. Do it in a such way that if we had couple following one after another transposes (one would be reversed version of another) we will have no transpose as a result of that couple of transposes.

void SwapOutputNames(ov::Output<ov::Node>, ov::Output<ov::Node>)

Swaps @args output tensor names.

void SwapFriendlyNames(const std::shared_ptr<ov::Node>&, const std::shared_ptr<ov::Node>&)

Swaps @args friendly names.

std::shared_ptr<ov::Node> InsertBroadcastUnsqueeze(const ov::Output<ov::Node> &node, size_t n_dims)

Inserts Unsqueeze node as a child to

  • node with axes {0, 1, … N - 1}, where N =

  • n_dims

bool CheckTransposeConsumers(const ov::Output<ov::Node>&)

Checks if.

  • has consumers that are all the same Transpose operation and that sinking is enabled for all these Transpose ops. Otherwise returns false. If no consumers at all returns false.

bool RemoveTransposeConsumers(const std::shared_ptr<ov::Node> &node)

Removes all Transpose consumers for given node.

ov::Output<ov::Node> ChangeValuesOrder(const ov::Output<ov::Node> &input, const ov::AxisVector &transpose_axis_order, const std::shared_ptr<ov::opset10::Constant> &axis)

Inserts Gather operation which changes the order of values in.

  • input according to

  • transpose_axis_order along

  • axis.

Output<Node> ChangeAxes(const Output<Node> &input, const AxisVector &transpose_axis_order, const std::shared_ptr<ov::opset10::Constant> &axis)

Inserts Gather operation which changes the order of values in.

  • input according to

  • transpose_axis_order along

  • axis.

Output<Node> ChangeAxes(const Output<Node> &input, const std::shared_ptr<ov::opset10::Constant> &transpose_axis_order, const std::shared_ptr<ov::opset10::Constant> &axis)

Inserts Gather operation which changes the order of values in.

  • input according to

  • transpose_axis_order along

  • axis.

std::vector<size_t> GetOrderAfterReduction(const std::vector<size_t> &axes_values, const std::vector<size_t> &order_values)

Returns the updated axes order for case when the initial axes order has more elements than after TransposeSinking, e.g.:

before: Transpose(the initial axes order) -> ReduceMax after : ReduceMax -> Transpose (the updated axes order)

before: Unsqueeze -> Transpose (the initial axes order) after : Transpose (the updated axes order) -> Unsqueeze

std::vector<size_t> GetOrderBeforeReduction(const std::vector<size_t> &axes_values, const std::vector<size_t> &order_values)

Returns the updated axes order for case when the initial axes order has less elements than after TransposeSinking, e.g.:

before : ReduceMax -> Transpose (the updated axes order) after: Transpose(the initial axes order) -> ReduceMax

before: Transpose (the updated axes order) -> Unsqueeze after : Unsqueeze -> Transpose (the initial axes order)

struct TransposeInputsInfo
#include <ts_utils.hpp>
namespace sink_backward

Functions

ov::NodeVector InsertTransposeBeforeNode(const std::shared_ptr<ov::Node> &main_node, const std::shared_ptr<ov::opset10::Constant> &transpose_const, std::vector<size_t> input_indexes = {}, std::function<std::shared_ptr<ov::Node>(const ov::Output<ov::Node> &node, size_t n_dims)> InsertUnsqueeze = InsertBroadcastUnsqueeze)

Inserts transposes on inputs of.

  • main_node specified by

  • input_indexes with the order specified in

  • transpose_const. If

  • input_indexes is empty, then it inserts transposes for all inputs.

namespace sink_forward

Functions

bool UpdateInputTransposes(const std::shared_ptr<ov::Node> &main_node, const TransposeInputsInfo &transpose_input_info, std::vector<size_t> input_indexes = {})

Inserts reversed transposed on @args main_node inputs. Removes input transpose specified in.

  • transpose_input_info

void RemoveInputNode(const std::shared_ptr<ov::Node>&, size_t input_idx)

Removes.

  • input node

ov::NodeVector InsertOutputTransposes(const std::shared_ptr<ov::Node> &main_node, const TransposeInputsInfo &transpose_input_info)

Inserts transposes on each main_node output with the order specified in.

  • transpose_input_info

namespace preprocess

Enums

enum class ColorFormat

Color format enumeration for conversion.

Values:

enumerator UNDEFINED

Undefined color format.

enumerator NV12_SINGLE_PLANE

Image in NV12 format represented as separate tensors for Y and UV planes.

enumerator NV12_TWO_PLANES

Image in NV12 format represented as separate tensors for Y and UV planes.

enumerator I420_SINGLE_PLANE

Image in I420 (YUV) format as single tensor.

enumerator I420_THREE_PLANES

Image in I420 format represented as separate tensors for Y, U and V planes.

enumerator RGB

Image in RGB interleaved format (3 channels)

enumerator BGR

Image in BGR interleaved format (3 channels)

enumerator GRAY

Image in GRAY format (1 channel)

enumerator RGBX

Image in RGBX interleaved format (4 channels)

enumerator BGRX

Image in BGRX interleaved format (4 channels)

enum class ResizeAlgorithm

An enum containing all supported resize(interpolation) algorithms available in preprocessing.

Values:

enumerator RESIZE_LINEAR

Linear interpolation matching the TensorFlow behavior.

enumerator RESIZE_CUBIC

Cubic interpolation.

enumerator RESIZE_NEAREST

Nearest interpolation.

enumerator RESIZE_BILINEAR_PILLOW

Bilinear interpolation matching the Pillow behavior.

enumerator RESIZE_BICUBIC_PILLOW

Bicubic interpolation matching the Pillow behavior.

Functions

OPENVINO_API std::ostream & operator<< (std::ostream &str, const PrePostProcessor &prePostProcessor)

Inserts a human-readable representation of a PrePostProcessors into an output stream. The output to the stream is in “informal” notation and can be used for debugging purposes.

Parameters
  • str – The output stream targeted for insertion.

  • prePostProcessor – The shape to be inserted into output stream.

Returns

A reference to same output stream after insertion.

class InputInfo
#include <input_info.hpp>

Class holding preprocessing information for one input From preprocessing pipeline perspective, each input can be represented as:

Public Functions

InputInfo(InputInfo &&other) noexcept

Move constructor.

InputInfo &operator=(InputInfo &&other) noexcept

Move assignment operator.

~InputInfo()

Default destructor.

InputTensorInfo &tensor()

Get current input tensor information with ability to change specific data.

Returns

Reference to current input tensor structure

PreProcessSteps &preprocess()

Get current input preprocess information with ability to add more preprocessing steps.

Returns

Reference to current preprocess steps structure

InputModelInfo &model()

Get current input model information with ability to change original model’s input data.

Returns

Reference to current model’s input information structure

class InputModelInfo
#include <input_model_info.hpp>

Information about model’s input tensor. If all information is already included to loaded model, this info may not be needed. However it can be set to specify additional information about model, like ‘layout’.

Example of usage of model ‘layout’: Support model has input parameter with shape {1, 3, 224, 224} and user needs to resize input image to model’s dimensions. It can be done like this

<model has input parameter with shape {1, 3, 224, 224}>
auto proc = PrePostProcessor(function);
proc.input().preprocess().resize(ResizeAlgorithm::RESIZE_LINEAR);
proc.input().model().set_layout("NCHW");

Public Functions

~InputModelInfo()

Default destructor.

InputModelInfo &set_layout(const ov::Layout &layout)

Set layout for model’s input tensor This version allows chaining for Lvalue objects.

Parameters

layoutLayout for model’s input tensor.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

class InputTensorInfo
#include <input_tensor_info.hpp>

Information about user’s input tensor. By default, it will be initialized to same data (type/shape/etc) as model’s input parameter. User application can override particular parameters (like ‘element_type’) according to application’s data and specify appropriate conversions in pre-processing steps.

auto proc = PrePostProcessor(function);
proc.input().tensor().set_element_type(ov::element::u8);

Public Functions

~InputTensorInfo()

Default destructor.

InputTensorInfo &set_element_type(const ov::element::Type &type)

Set element type for user’s input tensor.

Parameters

type – Element type for user’s input tensor.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

InputTensorInfo &set_layout(const ov::Layout &layout)

Set layout for user’s input tensor.

Parameters

layoutLayout for user’s input tensor.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

InputTensorInfo &set_spatial_dynamic_shape()

By default, input image shape is inherited from model input shape. This method specifies that user’s input image has dynamic spatial dimensions (width & height). This can be useful for adding resize preprocessing from any input image to model’s expected dimensions.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

InputTensorInfo &set_spatial_static_shape(size_t height, size_t width)

By default, input image shape is inherited from model input shape. Use this method to specify different width and height of user’s input image. In case if input image size is not known, use set_spatial_dynamic_shape method.

Parameters
  • height – Set fixed user’s input image height.

  • width – Set fixed user’s input image width.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

InputTensorInfo &set_color_format(const ov::preprocess::ColorFormat &format, const std::vector<std::string> &sub_names = {})

Set color format for user’s input tensor.

In general way, some formats support multi-plane input, e.g. NV12 image can be represented as 2 separate tensors (planes): Y plane and UV plane. set_color_format API also allows to set sub_names for such parameters for convenient usage of plane parameters. During build stage, new parameters for each plane will be inserted to the place of original parameter. This means that all parameters located after will shift their positions accordingly (e.g. {param1, param2} will become {param1/Y, param1/UV, param2})

Parameters
  • format – Color format of input image.

  • sub_names – Optional list of sub-names assigned for each plane (e.g. {“Y”, “UV”}). If specified, number of sub-names shall match with number of planes. If not specified, friendly name and tensor name for plane parameters will be empty. It is not allowed to specify sub-names for single-plane inputs.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

InputTensorInfo &set_memory_type(const std::string &memory_type)

Set memory type runtime information for user’s input tensor.

Parameters

memory_type – Memory type. Refer to specific plugin’s documentation for exact string format

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

InputTensorInfo &set_shape(const ov::PartialShape &shape)

By default, input shape is inherited from model’s input shape. Use this method to specify different input data shape. If it is needed to change only input height & width of input image, consider define layout and use set_spatial_static_shape or ‘set_spatial_dynamic_shape’ instead. This method allows defining any custom input shape and can be useful for custom preprocessing operations.

Note

Methods ‘set_spatial_dynamic_shape’, ‘set_spatial_static_shape’ are also intended to modify input shape, using those methods together will throw ov::AssertFailure exception

Parameters

shape – New shape for input tensor.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

InputTensorInfo &set_from(const ov::Tensor &runtime_tensor)

Helper function to reuse element type and shape from user’s created tensor. Use this only in case if input tensor is already known and available before. Overwrites previously set element type & shape via set_element_type and set_shape. Tensor’s memory type is not reused, so if runtime_tensor represents remote tensor with particular memory type - you should still specify appropriate memory type manually using set_memory_type

Note

As for InputTensorInfo::set_shape, this method shall not be used together with methods ‘set_spatial_dynamic_shape’ and ‘set_spatial_static_shape’, otherwise ov::AssertFailure exception will be thrown

Parameters

runtime_tensor – User’s created tensor.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

class OutputInfo
#include <output_info.hpp>

Class holding postprocessing information for one output From postprocessing pipeline perspective, each output can be represented as:

Public Functions

OutputInfo(OutputInfo &&other) noexcept

Move constructor.

OutputInfo &operator=(OutputInfo &&other) noexcept

Move assignment operator.

~OutputInfo()

Default destructor.

OutputModelInfo &model()

Get current output model information with ability to change original model’s output data.

Returns

Reference to current model’s output information structure

PostProcessSteps &postprocess()

Get current output post-process information with ability to add more post-processing steps.

Returns

Reference to current preprocess steps structure

OutputTensorInfo &tensor()

Get current output tensor information with ability to change specific data.

Returns

Reference to current output tensor structure

class OutputModelInfo
#include <output_model_info.hpp>

Information about model’s output tensor. If all information is already included to loaded model, this info may not be needed. However it can be set to specify additional information about model’s output, like ‘layout’.

Example of usage of model’s ‘layout’: Suppose model has output result with shape {1, 3, 224, 224} and NHWC layout. User may need to transpose output picture to interleaved format {1, 224, 224, 3}. This can be done with the following code

<model has output result with shape {1, 3, 224, 224}>
auto proc = PrePostProcessor(function);
proc.output().model().set_layout("NCHW");
proc.output().postprocess().convert_layout("NHWC");
function = proc.build();

Public Functions

~OutputModelInfo()

Default destructor.

OutputModelInfo &set_layout(const ov::Layout &layout)

Set layout for model’s output tensor.

Parameters

layoutLayout for model’s output tensor.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

class OutputTensorInfo
#include <output_tensor_info.hpp>

Information about user’s desired output tensor. By default, it will be initialized to same data (type/shape/etc) as model’s output parameter. User application can override particular parameters (like ‘element_type’) according to application’s data and specify appropriate conversions in post-processing steps.

auto proc = PrePostProcessor(function);
auto& output = proc.output();
output.postprocess().<add steps + conversion to user's output element type>;
output.tensor().set_element_type(ov::element::u8);
function = proc.build();

Public Functions

~OutputTensorInfo()

Default destructor.

OutputTensorInfo &set_element_type(const ov::element::Type &type)

Set element type for user’s desired output tensor.

Parameters

type – Element type for user’s output tensor.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

OutputTensorInfo &set_layout(const ov::Layout &layout)

Set layout for user’s output tensor.

Parameters

layoutLayout for user’s output tensor.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

class PostProcessSteps
#include <postprocess_steps.hpp>

Postprocessing steps. Each step typically intends adding of some operation to output parameter User application can specify sequence of postprocessing steps in a builder-like manner.

auto proc = PrePostProcessor(function);
proc.output().postprocess().convert_element_type(element::u8);
function = proc.build();

Public Types

using CustomPostprocessOp = std::function<ov::Output<ov::Node>(const ov::Output<ov::Node> &node)>

Signature for custom postprocessing operation. Custom postprocessing operation takes one output node and produces one output node. For more advanced cases, client’s code can use transformation passes over ov::Model directly.

Param node

Output node for custom post-processing operation

Return

New node after applying custom post-processing operation

Public Functions

~PostProcessSteps()

Default destructor.

PostProcessSteps &convert_element_type(const ov::element::Type &type = {})

Add convert element type post-process operation.

Parameters

type – Desired type of output. If not specified, type will be obtained from ‘tensor’ output information

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

PostProcessSteps &convert_layout(const Layout &dst_layout = {})

Add ‘convert layout’ operation to specified layout.

Adds appropriate ‘transpose’ operation between model layout and user’s desired layout. Current implementation requires source and destination layout to have same number of dimensions

Example: when model data has output in ‘NCHW’ layout ([1, 3, 224, 224]) but user needs interleaved output image (‘NHWC’, [1, 224, 224, 3]). Post-processing may look like this:

auto proc = PrePostProcessor(function);
proc.output().model(OutputTensorInfo().set_layout("NCHW"); // model output is NCHW
proc.output().postprocess().convert_layout("NHWC"); // User needs output as NHWC
Parameters

dst_layout – New layout after conversion. If not specified - destination layout is obtained from appropriate tensor output properties.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

PostProcessSteps &convert_layout(const std::vector<uint64_t> &dims)

Add convert layout operation by direct specification of transposed dimensions.

Example: model produces output with shape [1, 3, 480, 640] and user’s needs interleaved output image [1, 480, 640, 3]. Post-processing may look like this:

 auto proc = PrePostProcessor(function);
proc.output().postprocess().convert_layout({0, 2, 3, 1});
function = proc.build();
Parameters

dims – Dimensions array specifying places for new axis. If not empty, array size (N) must match to input shape rank. Array values shall contain all values from 0 to N-1. If empty, no actual conversion will be added.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

PostProcessSteps &custom(const CustomPostprocessOp &postprocess_cb)

Add custom post-process operation. Client application can specify callback function for custom action.

Parameters

postprocess_cb – Client’s custom postprocess operation.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

class PrePostProcessor
#include <pre_post_process.hpp>

Main class for adding pre- and post- processing steps to existing ov::Model.

This is a helper class for writing easy pre- and post- processing operations on ov::Model object assuming that any preprocess operation takes one input and produces one output.

For advanced preprocessing scenarios, like combining several functions with multiple inputs/outputs into one, client’s code can use transformation passes over ov::Model

Public Functions

explicit PrePostProcessor(const std::shared_ptr<Model> &function)

Default constructor.

Parameters

function – Existing function representing loaded model

PrePostProcessor(PrePostProcessor&&) noexcept

Default move constructor.

PrePostProcessor &operator=(PrePostProcessor&&) noexcept

Default move assignment operator.

~PrePostProcessor()

Default destructor.

InputInfo &input()

Gets input pre-processing data structure. Should be used only if model/function has only one input Using returned structure application’s code is able to set user’s tensor data (e.g layout), preprocess steps, target model’s data.

Returns

Reference to model’s input information structure

InputInfo &input(const std::string &tensor_name)

Gets input pre-processing data structure for input identified by it’s tensor name.

Parameters

tensor_nameTensor name of specific input. Throws if tensor name is not associated with any input in a model

Returns

Reference to model’s input information structure

InputInfo &input(size_t input_index)

Gets input pre-processing data structure for input identified by it’s order in a model.

Parameters

input_indexInput index of specific input. Throws if input index is out of range for associated function

Returns

Reference to model’s input information structure

OutputInfo &output()

Gets output post-processing data structure. Should be used only if model/function has only one output Using returned structure application’s code is able to set model’s output data, post-process steps, user’s tensor data (e.g layout)

Returns

Reference to model’s output information structure

OutputInfo &output(const std::string &tensor_name)

Gets output post-processing data structure for output identified by it’s tensor name.

Parameters

tensor_nameTensor name of specific output. Throws if tensor name is not associated with any input in a model

Returns

Reference to model’s output information structure

OutputInfo &output(size_t output_index)

Gets output post-processing data structure for output identified by it’s order in a model.

Parameters

output_indexOutput index of specific output. Throws if output index is out of range for associated function

Returns

Reference to model’s output information structure

std::shared_ptr<Model> build()

Adds pre/post-processing operations to function passed in constructor.

Returns

Function with added pre/post-processing operations

class PreProcessSteps
#include <preprocess_steps.hpp>

Preprocessing steps. Each step typically intends adding of some operation to input parameter User application can specify sequence of preprocessing steps in a builder-like manner.

auto proc = PrePostProcessor(function);
proc.input().preprocess()
                       .mean(0.2f)     // Subtract 0.2 from each element
                       .scale(2.3f));   // then divide each element to 2.3

Public Types

using CustomPreprocessOp = std::function<Output<Node>(const Output<Node> &node)>

Signature for custom preprocessing operation. Custom preprocessing operation takes one input node and produces one output node. For more advanced cases, client’s code can use transformation passes over ov::Model directly.

Param node

Input node for custom preprocessing operation (output of previous preprocessing operation)

Return

New node after applying custom preprocessing operation

Public Functions

~PreProcessSteps()

Default destructor.

PreProcessSteps &convert_element_type(const ov::element::Type &type = {})

Add convert element type preprocess operation.

Parameters

type – Desired type of input.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

PreProcessSteps &convert_color(const ov::preprocess::ColorFormat &dst_format)

Converts color format for user’s input tensor. Requires source color format to be specified by InputTensorInfo::set_color_format.

Parameters

dst_format – Destination color format of input image

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

PreProcessSteps &scale(float value)

Add scale preprocess operation Divide each element of input by specified value.

Parameters

value – Scaling value.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

PreProcessSteps &scale(const std::vector<float> &values)

Add scale preprocess operation by specified array of scale values for each channel.

Parameters

values – Scaling values. Layout runtime info with channels dimension must be specified for input tensor

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

PreProcessSteps &mean(float value)

Add mean preprocess operation Subtract specified value from each element of input.

Parameters

value – Value to subtract from each element.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

PreProcessSteps &mean(const std::vector<float> &values)

Add mean preprocess operation by specified array of mean values for each channel.

Parameters

values – Mean values. Layout runtime info with channels dimension must be specified for input tensor

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

PreProcessSteps &custom(const CustomPreprocessOp &preprocess_cb)

Add custom preprocess operation Client application can specify callback function for custom action.

Parameters

preprocess_cb – Client’s custom preprocess operation.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner

PreProcessSteps &resize(ResizeAlgorithm alg, size_t dst_height, size_t dst_width)

Add resize operation to known dimensions - Lvalue version.

Parameters
  • alg – Resize algorithm.

  • dst_height – Desired height of resized image.

  • dst_width – Desired width of resized image.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

PreProcessSteps &resize(ResizeAlgorithm alg)

Add resize operation to model’s dimensions.

Parameters

alg – Resize algorithm.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

PreProcessSteps &crop(const std::vector<int> &begin, const std::vector<int> &end)

Crop input tensor between begin and end coordinates. Under the hood, inserts opset8::Slice operation to execution graph. It is recommended to use to together with ov::preprocess::InputTensorInfo::set_shape to set original input shape before cropping.

Parameters
  • begin – Begin indexes for input tensor cropping. Negative values represent counting elements from the end of input tensor

  • end – End indexes for input tensor cropping. End indexes are exclusive, which means values including end edge are not included in the output slice. Negative values represent counting elements from the end of input tensor

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

PreProcessSteps &convert_layout(const Layout &dst_layout = {})

Add ‘convert layout’ operation to specified layout.

Adds appropriate ‘transpose’ operation between user layout and target layout. Current implementation requires source and destination layout to have same number of dimensions

Example: when user data has ‘NHWC’ layout (example is RGB image, [1, 224, 224, 3]) but model expects planar input image (‘NCHW’, [1, 3, 224, 224]). Preprocessing may look like this:

auto proc = PrePostProcessor(model);
proc.input().tensor().set_layout("NHWC"); // User data is NHWC
proc.input().preprocess().convert_layout("NCHW")) // model expects input as NCHW
Parameters

dst_layout – New layout after conversion. If not specified - destination layout is obtained from appropriate model input properties.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

PreProcessSteps &convert_layout(const std::vector<uint64_t> &dims)

Add convert layout operation by direct specification of transposed dimensions.

Example: when user data has input RGB image {1x480x640x3} but model expects planar input image (‘NCHW’, [1, 3, 480, 640]). Preprocessing may look like this:

auto proc = PrePostProcessor(function);
proc.input().preprocess().convert_layout({0, 3, 1, 2});
Parameters

dims – Dimensions array specifying places for new axis. If not empty, array size (N) must match to input shape rank. Array values shall contain all values from 0 to N-1. If empty, no actual conversion will be added.

Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

PreProcessSteps &reverse_channels()

Reverse channels operation.

Adds appropriate operation which reverses channels layout. Operation requires layout having ‘C’ dimension Operation convert_color (RGB<->BGR) does reversing of channels also, but only for NHWC layout

Example: when user data has ‘NCHW’ layout (example is [1, 3, 224, 224] RGB order) but model expects BGR planes order. Preprocessing may look like this:

auto proc = PrePostProcessor(function);
proc.input().tensor().set_layout("NCHW"); // User data is NCHW
proc.input().preprocess().reverse_channels();
Returns

Reference to ‘this’ to allow chaining with other calls in a builder-like manner.

class TensorInfoMemoryType : public ov::RuntimeAttribute
#include <input_tensor_info.hpp>
namespace proxy
namespace reference

Typedefs

using Nearest_mode = op::v4::Interpolate::NearestMode
using Transform_mode = op::v4::Interpolate::CoordinateTransformMode
using InterpolateMode = op::v4::Interpolate::InterpolateMode
using ROIPoolingMode = op::v3::ROIAlign::PoolingMode
using AlignedMode = op::v9::ROIAlign::AlignedMode
using Reduction = ov::op::v12::ScatterElementsUpdate::Reduction
using custom_evaluate_function = std::function<void(const std::shared_ptr<Model> &function, const ov::TensorVector &inputs, ov::TensorVector &outputs)>

Enums

enum [anonymous]

Values:

enumerator idxLocation
enumerator idxConfidence
enumerator idxPriors
enumerator idxArmConfidence
enumerator idxArmLocation
enumerator numInputs
enum class FFTKind

Values:

enumerator Forward
enumerator Inverse
enum PSROIPoolingMode

Values:

enumerator AVG
enumerator BILINEAR
enum class CellType

Values:

enumerator RNN
enumerator GRU
enumerator LSTM
enumerator LSTM_v1
enumerator AUGRU
enum class DescriptorType

Values:

enumerator SINGLE_VALUE
enumerator SLICE

Functions

template<class T>
void abs(const T *in, T *out, const size_t count)

Reference implementation of Abs operator.

Parameters
  • inInput pointer to data.

  • outOutput pointer to results.

  • count – Number of elements in input buffer.

template<class T>
void acos(const T *arg, T *out, const size_t count)

Reference implementation of Acos operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

template<class T>
void acosh(const T *arg, T *out, const size_t count)

Reference implementation of Acosh operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

template<typename T>
void adaptive_avg_pool(const T *arg, T *out, const Shape &arg_shape, const Shape &out_shape)
template<typename T, typename IT>
void adaptive_max_pool_1d(const T *arg, T *out, IT *indices, size_t h_in, size_t h_out)
template<typename T, typename IT>
void adaptive_max_pool_2d(const T *arg, T *out, IT *indices, size_t h_in, size_t h_out, size_t w_in, size_t w_out)
template<typename T, typename IT>
void adaptive_max_pool_3d(const T *arg, T *out, IT *indices, size_t d_in, size_t d_out, size_t h_in, size_t h_out, size_t w_in, size_t w_out)
template<typename T, typename IT>
void adaptive_max_pool(const T *arg, T *out, IT *selected_indices, const Shape &arg_shape, const Shape &out_shape)
template<class T>
void add(const T *arg0, const T *arg1, T *out, const size_t count)
template<class T>
void add(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise Add operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<class T>
void logical_and(const T *arg0, const T *arg1, T *out, size_t count)
template<class T>
void logical_and(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise LogicalAnd operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<typename T>
void asin(const T *arg, T *out, const size_t count)

Reference implementation of Asin operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

template<class T>
void asinh(const T *arg, T *out, const size_t count)

Reference implementation of Asinh operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

template<class T>
void atan(const T *arg, T *out, const size_t count)

Reference implementation of Atan operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

template<typename X, typename Y, typename Z>
void atan2(const X *py, const Y *px, Z *pout, size_t count)
template<class T>
void atanh(const T *arg, T *out, const size_t count)

Reference implementation of Atanh operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

template<typename T, typename U, typename Functor>
void autobroadcast_binop(const T *arg0, const T *arg1, U *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec, Functor elementwise_functor)

Helper function to implement autobroadcasting elementwise binop references.

Template Parameters
  • T – Element type of the input tensors.

  • U – Element type of the output tensor.

  • Functor – Type of the functor for the elementwise operation. Must support operator()(T,T), and operator()(T,T) must return a value of type U.

Parameters
  • arg0 – Pointer to the buffer for left operand input tensor.

  • arg1 – Pointer to the buffer for right operand input tensor.

  • out – Pointer to the buffer for output tensor. This must be pre-allocated by the caller, and must be large enough to hold a tensor of the correct shape.

  • broadcast_spec – Specification of the auto-broadcasting scheme.

  • elementwise_functor – Functor implementing the elementwise operation to be applied across the input tensors. Must accept two arguments of type T, and return a value of type U.

template<typename T, typename U, typename Functor>
void autobroadcast_select(const U *arg0, const T *arg1, const T *arg2, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const Shape &arg2_shape, const op::AutoBroadcastSpec &broadcast_spec, Functor elementwise_functor)

Helper function to implement autobroadcasting elementwise ternaryop references.

Template Parameters
  • U – Element type of the selector tensor.

  • T – Element type of the input tensors.

  • Functor – Type of the functor for the elementwise operation. Must support operator()(U,T,T), and operator()(U,T,T) must return a value of type T.

Parameters
  • arg0 – Pointer to the buffer for selector tensor.

  • arg1 – Pointer to the buffer for left operand input tensor.

  • arg2 – Pointer to the buffer for right operand input tensor.

  • out – Pointer to the buffer for output tensor. This must be pre-allocated by the caller, and must be large enough to hold a tensor of the correct shape.

  • broadcast_spec – Specification of the auto-broadcasting scheme.

  • elementwise_functor – Functor implementing the elementwise operation to be applied across the input tensors. Must accept an argument of type U and two of type T, and return a value of type T.

template<typename T>
void avg_pool(const T *const arg, T *const out, const Shape &arg_shape, const Shape &out_shape, const Shape &window_shape, const Strides &window_movement_strides, const Shape &padding_below, const Shape &padding_above, const bool include_padding_in_avg_computation)
template<typename T>
static inline T norm(T val, T mean, T var, T eps)
template<typename T>
void batch_norm_inference(float eps, const T *in, const T *gamma, const T *beta, const T *mean, const T *variance, T *out, const Shape &in_shape)
template<typename T_IN, typename T_F>
void binary_convolution(const T_IN *in, const T_F *f, T_IN *out, const Shape &in_shape, const Shape &f_shape, const Shape &out_shape, const Strides &strides, const Strides &dilations, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const float pad_value)
template<class T, typename std::enable_if<std::is_same<typename std::decay<T>::type, char>::value>::type* = nullptr>
void bitwise_and(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise bitwise AND operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<class T>
void bitwise_not(const T *in, T *out, size_t count)

Reference implementation of BitwiseNot operator.

Parameters
  • inInput pointer to data.

  • outOutput pointer to results.

  • count – Number of elements in input buffer.

template<class T, typename std::enable_if<std::is_same<typename std::decay<T>::type, char>::value>::type* = nullptr>
void bitwise_or(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise bitwise OR operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<class T, typename std::enable_if<std::is_same<typename std::decay<T>::type, char>::value>::type* = nullptr>
void bitwise_xor(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise bitwise XOR operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

void broadcast(const char *arg, char *out, const Shape &in_shape, const Shape &out_shape, const AxisSet &broadcast_axes, size_t elem_size)
template<typename T, typename B, typename P>
void bucketize(const T *data, const B *buckets, P *out, const Shape &data_shape, const Shape &buckets_shape, bool with_right_bound)
template<class T, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
void ceiling(const T *arg, T *out, const size_t count)

Reference implementation of Ceiling operator (integral types).

Reference implementation of Ceiling operator (floating point types).

Parameters
  • argInput pointer to data.

  • outOutput pointer to results.

  • count – Number of elements in input buffer.

template<typename T>
void clamp(const T *arg, T *out, const T min, const T max, const size_t count)

Reference implementation of Clamp operator.

Parameters
  • arg – Pointer to input data.

  • out – Pointer to output data.

  • min – Minimum value used to clamp input data.

  • max – Maximum value used to clamp input data.

  • count – Number of elements in input buffer.

void concat(const std::vector<const char*> &args, char *out, const std::vector<Shape> &in_shapes, const Shape &out_shape, int64_t concatenation_axis, size_t elem_size)
template<typename T>
void constant(const T *arg0, T *out, size_t count)
template<typename TI, typename TO>
void convert(const TI *arg, TO *out, const size_t count)
size_t count_out_of_f16_range(const float *arg, size_t count)
void convert_from_f32_to_f16_with_clamp(const float *arg, float16 *out, size_t count)
template<typename T>
std::tuple<T, T, T> yuv_pixel_to_rgb(float y_val, float u_val, float v_val)
template<typename T>
void color_convert_nv12(const T *arg_y, const T *arg_uv, T *out_ptr, size_t batch_size, size_t image_h, size_t image_w, size_t stride_y, size_t stride_uv, ov::op::util::ConvertColorNV12Base::ColorConversion color_format)
template<typename T>
void color_convert_i420(const T *arg_y, const T *arg_u, const T *arg_v, T *out_ptr, size_t batch_size, size_t image_h, size_t image_w, size_t stride_y, size_t stride_uv, ov::op::util::ConvertColorI420Base::ColorConversion color_format)
template<ov::element::Type_t T>
inline bool color_convert_nv12(const std::shared_ptr<Node> &op, ov::TensorVector &outputs, const ov::TensorVector &inputs, ov::op::util::ConvertColorNV12Base::ColorConversion type)
template<ov::element::Type_t T>
inline bool color_convert_i420(const std::shared_ptr<Node> &op, ov::TensorVector &outputs, const ov::TensorVector &inputs, ov::op::util::ConvertColorI420Base::ColorConversion type)
template<typename T>
void convolution(const T *in, const T *f, T *out, const Shape &in_shape, const Shape &f_shape, const Shape &out_shape, const Strides &strides, const Strides &dilations, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end)
template<typename T>
void convolution_backprop_in(const T *delta_in, const T *filter, T *delta_out, const Shape &in_shape, const Shape &filter_shape, const Shape &out_shape, const Strides &in_dilation, const Strides &filter_dilation, const CoordinateDiff &forward_in_pad_bellow, const CoordinateDiff &forward_in_pad_above, const Strides &stride, const CoordinateDiff &output_padding)
template<typename T>
void copy(const T *arg, T *out, size_t count)
template<class T>
void cos(const T *arg, T *out, const size_t count)

Reference implementation of Cos operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

template<class T>
void cosh(const T *arg, T *out, size_t count)

Reference implementation of Cosh operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

template<typename T>
void ctc_greedy_decoder(const T *data, const T *sequence_masks, T *out, const Shape &data_shape, const Shape &sequence_masks_shape, const Shape &out_shape, const bool ctc_merge_repeated)
template<typename TF, typename TI, typename TCI, typename TSL>
void ctc_greedy_decoder_seq_len(const TF *data, const TI *sequence_length, const TI *blank_index, TCI *out1, TSL *out2, const Shape &data_shape, const Shape &out_shape, const bool ctc_merge_repeated)
template<typename T, typename U>
void CTCLoss(const T *logits, const Shape &logitsShape, const U *logitsLength, const U *labels, const U *labelsLength, const U *blankIndexP, const bool preprocessCollapseRepeated, const bool ctcMergeRepeated, const bool unique, T *output)
template<typename T, typename P>
void cumsum(const T *arg, const P axis, T *out, const Shape &tensor_shape, const bool exclusive, const bool reverse)
template<typename T>
void deformable_convolution(const T *in, const T *offsets, const T *filters, const T *mask, T *out, const Shape &in_shape, const Shape &o_shape, const Shape &f_shape, const Shape &m_shape, const Shape &out_shape, const Strides &strides, const Strides &dilation, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const int64_t groups, const int64_t deformable_groups, const bool bilinear_interpolation_pad)
template<typename T>
void deformable_convolution(const T *in, const T *offsets, const T *filters, T *out, const Shape &in_shape, const Shape &o_shape, const Shape &f_shape, const Shape &out_shape, const Strides &strides, const Strides &dilation, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const int64_t groups, const int64_t deformable_groups, const bool bilinear_interpolation_pad = false)
template<typename T>
void deformable_psroi_pooling(const T *data_input, const Shape &data_input_shape, const T *rois_input, const Shape &rois_input_shape, const T *offsets_input, const Shape &offsets_input_shape, T *output, const Shape &output_shape, const std::string &mode_str, const float spatial_scale, const int64_t spatial_bins_x, const int64_t spatial_bins_y, const float trans_std, const int64_t part_size)
void depth_to_space(const char *const in, const Shape &in_shape, char *const out, const Shape &out_shape, const size_t block_size, const op::v0::DepthToSpace::DepthToSpaceMode mode, const size_t elem_size)
template<typename T>
std::enable_if<std::is_integral<T>::value>::type divide(const T *arg0, const T *arg1, T *out, size_t count, bool pythondiv)
template<typename T>
std::enable_if<std::is_integral<T>::value>::type divide(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec, bool pythondiv)
template<typename T>
std::enable_if<std::is_floating_point<T>::value || std::is_same<T, bfloat16>::value || std::is_same<T, float16>::value>::type divide(const T *arg0, const T *arg1, T *out, size_t count, bool pythondiv)
template<typename T>
std::enable_if<std::is_floating_point<T>::value || std::is_same<T, bfloat16>::value || std::is_same<T, float16>::value>::type divide(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec, bool pythondiv)
void einsum(ov::TensorVector &outputs, const ov::TensorVector &inputs, const std::string &equation)
template<typename T>
void elu(const T *arg, T *out, size_t count, double alpha)
template<typename T, typename U>
void embeddingBagOffsetsSum(const T *emb_table, const U *indices, const U *offsets, const U *default_index, const T *weights, T *out, const size_t indices_count, const Shape &outShape)
template<typename T, typename U>
void embeddingBagPackedSum(const T *emb_table, const U *indices, const T *weights, T *out, const Shape &indicesShape, const Shape &outShape)
template<typename T, typename U>
void embeddingSegmentsSum(const T *embTable, const U *indices, const U *segmentIds, const U *defaultIndex, const T *weights, T *out, const Shape &embTableShape, const Shape &indicesShape, const Shape &outShape)
template<typename T>
void equal(const T *arg0, const T *arg1, char *out, size_t count)
template<typename T, typename U, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
void equal(const T *arg0, const T *arg1, U *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise Equal operator.

Used for integral types with custom equal function (reduce binary size).

Used for floating-point types to (avoid warning compare floating point with ==).

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<class T>
void erf(const T *arg, T *out, const size_t count)

Reference implementation of Erf operator.

Parameters
  • arg – Pointer to input data.

  • out – Pointer to output data.

  • count – Number of elements in input buffer.

template<class T>
void exp(const T *arg, T *out, size_t count)

Reference implementation of Exp operator.

Parameters
  • arg – Pointer to input data.

  • out – Pointer to output data.

  • count – Number of elements in input buffer.

void experimental_detectron_detection_output(const float *input_rois, const float *input_deltas, const float *input_scores, const float *input_im_info, const op::v6::ExperimentalDetectronDetectionOutput::Attributes &attrs, float *output_boxes, float *output_scores, int32_t *output_classes)
void experimental_detectron_detection_output_postprocessing(void *pboxes, void *pclasses, void *pscores, const element::Type output_type, const std::vector<float> &output_boxes, const std::vector<int32_t> &output_classes, const std::vector<float> &output_scores, const Shape &output_boxes_shape, const Shape &output_classes_shape, const Shape &output_scores_shape)
template<typename T>
void experimental_detectron_prior_grid_generator(const T *priors, const Shape &priors_shape, const Shape &feature_map_shape, const Shape &im_data_shape, T *output_rois, int64_t grid_h, int64_t grid_w, float stride_h, float stride_w)
void experimental_detectron_proposals_single_image(const float *im_info, const float *anchors, const float *deltas, const float *scores, const op::v6::ExperimentalDetectronGenerateProposalsSingleImage::Attributes &attrs, const Shape &im_info_shape, const Shape &anchors_shape, const Shape &deltas_shape, const Shape &scores_shape, float *output_rois, float *output_scores)
void experimental_detectron_proposals_single_image_postprocessing(void *prois, void *pscores, const element::Type output_type, const std::vector<float> &output_rois, const std::vector<float> &output_scores, const Shape &output_rois_shape, const Shape &output_scores_shape)
void experimental_detectron_roi_feature_extractor(const std::vector<std::vector<float>> &inputs, const std::vector<Shape> &input_shapes, const op::v6::ExperimentalDetectronROIFeatureExtractor::Attributes &attrs, float *output_rois_features, float *output_rois)
void experimental_detectron_roi_feature_extractor_postprocessing(void *prois_features, void *prois, const element::Type output_type, const std::vector<float> &output_roi_features, const std::vector<float> &output_rois, const Shape &output_roi_features_shape, const Shape &output_rois_shape)
template<typename T>
void experimental_detectron_topk_rois(const T *input_rois, const T *input_probs, const Shape &input_rois_shape, const Shape &input_probs_shape, size_t max_rois, T *output_rois)
template<typename T>
void extract_image_patches(const std::shared_ptr<op::v3::ExtractImagePatches> extImgPatches, const T *input, T *out, const Shape &inShape, const Shape &outShape)
template<typename T>
void eye(T *data, const Shape &out_shape, const int64_t diagonal_index)

Reference implementation of Eye operator.

Parameters
  • data – Pointer to output data.

  • out_shapeOutput data size.

  • diagonal_index – Eye diagonal index to populate matrix with ones

template<typename T, typename std::enable_if<std::is_same<T, float16>::value, bool>::type = true>
void fake_convert(const T *data, const T *scale, const T *shift, T *out, const Shape &data_shape, const Shape &scale_shape, const Shape &shift_shape, const std::string &destination_type)

Reference implementation of the FakeConvert operation specialized for float16 type. It emulates format specified by “destination_type” on the input type. FakeConvert performs element-wise quantization of floating-point input values into a set of values corresponding to a target low-precision floating-point type.

Reference implementation of the FakeConvert operation for floating-point types, with middle conversion to float16. It emulates format specified by “destination_type” on the input type. FakeConvert performs element-wise quantization of floating-point input values into a set of values corresponding to a target low-precision floating-point type.

Parameters
  • data – Pointer to the input data.

  • scale – Pointer to the input scale.

  • shift – Pointer to the input shift.

  • out – Pointer to the output data.

  • data_shapeShape of the input data.

  • scale_shapeShape of the input scale.

  • shift_shapeShape of the input shift.

  • destination_type – Name of the destination type.

template<typename T>
void fake_convert(const T *data, const T *scale, T *out, const Shape &data_shape, const Shape &scale_shape, const std::string &destination_type)

Reference implementation of the FakeConvert operation, with default shift input.

template<typename T>
void fake_quantize(const T *arg, const T *in_low, const T *in_high, const T *out_low, const T *out_high, T *out, const Shape &arg_shape, const Shape &in_low_shape, const Shape &in_high_shape, const Shape &out_low_shape, const Shape &out_high_shape, size_t levels, const op::AutoBroadcastSpec &broadcast)
void fft(const float *input_data, const Shape &input_data_shape, const int64_t *axes_data, const Shape &axes_data_shape, float *fft_result, const Shape &output_shape, FFTKind fft_kind)
void fft_postprocessing(ov::TensorVector &outputs, const ov::element::Type output_type, const std::vector<float> &fft_result)
std::vector<int64_t> canonicalize_axes(const int64_t *axes_data, const Shape &axes_data_shape, int64_t complex_data_rank)
template<class T, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
void floor(const T *arg, T *out, const size_t count)

Reference implementation of Floor operator (integral types).

Reference implementation of Floor operator (floating point types).

Parameters
  • argInput pointer to data.

  • outOutput pointer to results.

  • count – Number of elements in input buffer.

template<typename T>
void floor_mod(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)
void function(const std::shared_ptr<Model> &function, const ov::TensorVector &inputs, ov::TensorVector &outputs)
template<typename T, typename U>
void gather(const T *const data, const U *const indices, T *out, const Shape &data_shape, const Shape &indices_shape, const Shape &out_shape, size_t axis, size_t batch_dims = 0)
template<typename T, typename U>
void gather_elements(const T *data, const U *indices, T *out, const Shape &data_shape, const Shape &indices_shape, const Shape &out_shape, int64_t axis)
template<typename T, typename U>
void gather_nd(const T *const params, const U *const indices, T *const out, const Shape &params_shape, const Shape &indices_shape, const Shape &out_shape, const int batch_dims = 0)

Implementation find maximum length of slice of input params which might be copied to out index by index. +—-&#8212;+———–&#8212;+—-&#8212;+ | batch | indices[:-1] | slice | | shape | shape | shape | +—-&#8212;+———–&#8212;+—-&#8212;+

void gather_tree(const char *step_ids, const char *parent_ids, const char *max_seq_len, const char *end_token, char *out, const Shape &step_ids_shape, const Shape &parent_ids_shape, const Shape &max_seq_len_shape, const Shape &end_token_shape, const element::Type &type)
template<typename T>
void gelu(const T *arg, T *out, op::GeluApproximationMode mode, size_t count)
void generate_proposals(const std::vector<float> &im_info, const std::vector<float> &anchors, const std::vector<float> &deltas, const std::vector<float> &scores, const op::v9::GenerateProposals::Attributes &attrs, const Shape &im_info_shape, const Shape &anchors_shape, const Shape &deltas_shape, const Shape &scores_shape, std::vector<float> &output_rois, std::vector<float> &output_scores, std::vector<int64_t> &num_rois)
void generate_proposals_postprocessing(void *prois, void *pscores, void *proi_num, const element::Type &output_type, const element::Type &roi_num_type, const std::vector<float> &output_rois, const std::vector<float> &output_scores, const std::vector<int64_t> &num_rois, const Shape &output_rois_shape, const Shape &output_scores_shape)
template<typename T>
void greater(const T *arg0, const T *arg1, char *out, size_t count)
template<typename T, typename U>
void greater(const T *arg0, const T *arg1, U *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise Greater operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg0_shapeInput 0 shape.

  • arg1_shapeInput 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<typename T>
void greater_eq(const T *arg0, const T *arg1, char *out, size_t count)
template<typename T, typename U>
void greater_eq(const T *arg0, const T *arg1, U *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)
template<typename DATA_ET, typename GRID_ET>
void grid_sample(DATA_ET *output, const DATA_ET *data, const GRID_ET *grid, const Shape &data_shape, const Shape &grid_shape, const bool align_corners, const ov::op::v9::GridSample::InterpolationMode interpolation_mode, const ov::op::v9::GridSample::PaddingMode padding_mode)
template<typename T>
void grn(const T *data, T *out, float bias, const Shape &data_shape)
void validate_group_convolution_parameters(const Shape &in_shape, const Shape &f_shape, const Shape &out_shape, const Strides &strides, const Strides &dilations, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end)
template<typename INPUT, typename FILTER, typename OUTPUT, typename ACCU = typename widen<OUTPUT>::type>
void group_convolution(const INPUT *in, const FILTER *f, OUTPUT *out, const Shape &in_shape, const Shape &filter_shape, const Shape &out_shape, const Strides &strides, const Strides &dilation, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end)
void infer_backward_conv_output_shape(const Shape &in_spatial_shape, const Shape &f_spatial_shape, Shape &out_spatial_shape, const Strides &strides, const Strides &dilations, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end)
void validate_convolution_backprop_data_parameters(const Shape &in_shape, const Shape &f_shape, const Shape &out_shape, const Strides &strides, const Strides &dilations, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end)
void validate_group_convolution_backprop_data_parameters(const Shape &in_shape, const Shape &f_shape, const Shape &out_shape, const Strides &strides, const Strides &dilations, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end)
template<typename T>
void group_convolution_backprop_data(const T *in, const T *f, T *out, const Shape &in_shape, const Shape &filter_shape, const Shape &out_shape, const Strides &strides, const Strides &dilation, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const CoordinateDiff &output_padding)
template<typename T>
void group_normalization(const T *const data, const T *const scale, const T *const bias, T *const out, const Shape &data_shape, const size_t num_groups, const double epsilon)
template<typename T>
void gru_cell(const T *X, const Shape &X_shape, const T *H, const Shape &H_shape, const T *W, const Shape &W_shape, const T *R, const Shape &R_shape, const T *B, const Shape &B_shape, T *dst_data, const std::string &activation_f, const std::string &activation_g, float clip, bool linear_before_reset, const T *A = nullptr)
template<typename T>
void hard_sigmoid(const T *arg, const T alpha, const T beta, T *out, size_t count)
template<typename T>
void hsigmoid(const T *arg, T *out, size_t count)
template<typename T>
void hswish(const T *arg, T *out, size_t count)
void if_reference(const std::vector<std::shared_ptr<Model>> &body, const std::vector<op::util::MultiSubGraphOp::MultiSubgraphOutputDescriptionVector> &out_descs, const std::vector<op::util::MultiSubGraphOp::MultiSubgraphInputDescriptionVector> &input_descs, ov::TensorVector &out, const ov::TensorVector &args)
inline void pad_input_data(const uint8_t *data_ptr, uint8_t *padded_data_ptr, size_t type_size, const ov::Shape &input_shape, const ov::Shape &padded_input_shape, const std::vector<size_t> &pads_begin)
inline PartialShape get_padded_input_shape(const PartialShape &input_shape, const op::v0::Interpolate::Attributes &attrs)
inline std::vector<float> get_scales(const PartialShape &input_data_partial_shape, const Shape &out_shape, const op::v0::Interpolate::Attributes &attrs)
inline op::v4::Interpolate::InterpolateAttrs transform_v0_to_v4(const PartialShape &input_partial_shape, const op::v0::Interpolate::Attributes &attrs_v0)
template<typename T>
void interpolate(const T *input_data, const Shape &input_data_shape, const std::vector<float> &scales, const std::vector<int64_t> &axes, T *out, const Shape &out_shape, const op::v4::Interpolate::InterpolateAttrs &attrs)
template<typename T>
void interpolate(T *input_data, const PartialShape &input_data_shape, T *out, const Shape &out_shape, const op::v0::Interpolate::Attributes &attrs)
void irdft(const std::vector<float> &input_data, const Shape &input_data_shape, const std::vector<int64_t> &axes_data, float *irdft_result, const Shape &fft_output_shape, const Shape &irdft_output_shape, const int64_t last_signal_size)
template<typename T, typename U>
std::enable_if<std::is_floating_point<T>::value, void>::type is_finite(const T *input, U *output, size_t count)
template<typename T, typename U>
std::enable_if<std::is_class<T>::value, void>::type is_finite(const T *input, U *output, size_t count)
template<typename T, typename U>
std::enable_if<std::is_floating_point<T>::value, void>::type is_inf(const T *input, U *output, size_t count, const ov::op::v10::IsInf::Attributes &attributes)
template<typename T, typename U>
std::enable_if<std::is_class<T>::value, void>::type is_inf(const T *input, U *output, size_t count, const ov::op::v10::IsInf::Attributes &attributes)
template<typename T, typename U>
std::enable_if<std::is_floating_point<T>::value, void>::type is_nan(const T *input, U *output, size_t count)
template<typename T, typename U>
std::enable_if<std::is_class<T>::value, void>::type is_nan(const T *input, U *output, size_t count)
template<typename T>
void less(const T *arg0, const T *arg1, char *out, const size_t count)
template<typename T, typename U>
void less(const T *arg0, const T *arg1, U *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise Less operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg0_shapeInput 0 shape.

  • arg1_shapeInput 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<typename T>
void less_eq(const T *arg0, const T *arg1, char *out, const size_t count)
template<typename T, typename U>
void less_eq(const T *arg0, const T *arg1, U *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise LessEqual operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg0_shapeInput 0 shape.

  • arg1_shapeInput 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<typename T>
void log(const T *arg, T *out, size_t count)
template<typename T>
void log_softmax(const T *arg, T *out, const Shape &shape, const AxisSet &axes)
template<class T>
void logical_not(const T *arg, T *out, const size_t count)

Reference implementation of LogicalNot operator.

Parameters
  • arg – Pointer to input data.

  • out – Pointer to output data.

  • count – Number of elements in input buffer.

static inline void reduce_logical_and(const char *arg, char *out, const Shape &in_shape, const AxisSet &reduction_axes)

Reference implementation of ReduceLogicalAnd operator.

Parameters
  • inInput pointer to data.

  • outOutput pointer to results.

  • in_shapeInput shape.

  • reduction_axes – Axes on which reduction is applied.

static inline void reduce_logical_or(const char *arg, char *out, const Shape &in_shape, const AxisSet &reduction_axes)

Reference implementation of ReduceLogicalOr operator.

Parameters
  • inInput pointer to data.

  • outOutput pointer to results.

  • in_shapeInput shape.

  • reduction_axes – Axes on which reduction is applied.

void loop(const std::shared_ptr<Model> &body, const op::util::OutputDescriptionVector &out_descs, const op::util::InputDescriptionVector &input_descs, const op::v5::Loop::SpecialBodyPorts &special_ports, ov::TensorVector &out, const ov::TensorVector &args)
static size_t point_to_flat_idx(const Shape &shape, const std::vector<size_t> &point)
static std::vector<size_t> slice_indices(const Shape &full_shape, const std::vector<size_t> &begin, const Shape &slice_shape)
template<typename T>
static T sum_region_across_axes(const T *arg, const std::vector<size_t> &indices)
template<typename T>
void lrn(const T *arg, const AxisSet &axes, T *out, const Shape &arg_shape, double dalpha, double dbeta, double dbias, size_t size)
template<typename T>
void lstm_cell(const T *X, const Shape &X_shape, const T *H, const Shape &H_shape, const T *C, const Shape &C_shape, const T *W, const Shape &W_shape, const T *R, const Shape &R_shape, const T *B, const Shape &B_shape, T *out_Ht, T *out_Ct, const std::string &activation_f, const std::string &activation_g, const std::string &activation_h, float clip)
template<typename T>
void lstm_cell_v1(const T *X, const Shape &X_shape, const T *H, const Shape &H_shape, const T *C, const Shape &C_shape, const T *W, const Shape &W_shape, const T *R, const Shape &R_shape, const T *B, const Shape &B_shape, const T *P, const Shape &P_shape, T *out_Ht, T *out_Ct, const std::string &activation_f, const std::string &activation_g, const std::string &activation_h, float clip, const ov::op::LSTMWeightsFormat weight_format, bool input_forget)
template<typename T>
void matmul(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const Shape &out_shape, bool transpose_arg0, bool transpose_arg1)

Reference kernel for matmul computation.

Template Parameters

T – Type of input and output tensors.

Parameters
  • arg0 – Pointer to the buffer for left operand input tensor.

  • arg1 – Pointer to the buffer for right operand input tensor.

  • out – Pointer to the buffer for output tensor. This must be pre-allocated by the caller, and must be large enough to hold a tensor of the correct shape.

  • arg0_shapeShape of arg0.

  • arg1_shapeShape of arg1.

  • out_shapeShape of out.

  • transpose_arg0 – Flag to indicate if transpose on arg0.

  • transpose_arg1 – Flag to indicate if transpose on arg1.

void matrix_nms(const float *boxes_data, const Shape &boxes_data_shape, const float *scores_data, const Shape &scores_data_shape, const op::v8::MatrixNms::Attributes &attrs, float *selected_outputs, const Shape &selected_outputs_shape, int64_t *selected_indices, const Shape &selected_indices_shape, int64_t *valid_outputs)
template<typename Values_t, typename Indices_t>
void max_pool(const Values_t *data, Values_t *values, Indices_t *indices, const Shape &data_shape, const Shape &out_shape, const Shape &kernel, const Strides &strides, const Strides &dilations, const Shape &pads_begin, const Shape &pads_end, const int64_t axis = 0)
template<typename Value_t>
void max_pool(const Value_t *data, Value_t *values, const Shape &data_shape, const Shape &out_shape, const Shape &kernel, const Strides &strides, const Shape &pads_begin, const Shape &pads_end)
template<typename T>
void maximum(const T *arg0, const T *arg1, T *out, size_t count)
template<typename T>
void maximum(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise Maximum operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg0_shapeInput 0 shape.

  • arg1_shapeInput 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<typename T>
void minimum(const T *arg0, const T *arg1, T *out, size_t count)
template<typename T>
void minimum(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)
template<typename T>
void mish(const T *arg, T *out, const size_t count)

Reference implementation of Mish operator.

Parameters
  • arg – Pointer to input data.

  • out – Pointer to output data.

  • count – Number of elements in input buffer.

template<class InputIt, class OutputIt>
void mod(InputIt arg0, InputIt arg1, OutputIt out, const Shape &arg_shape0, const Shape &arg_shape1, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise Mod operator.

Parameters
  • arg0 – Iterator to input 0 data.

  • arg1 – Iterator to input 1 data.

  • out – Iterator to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

void multiclass_nms(const float *boxes_data, const Shape &boxes_data_shape, const float *scores_data, const Shape &scores_data_shape, const int64_t *roisnum_data, const Shape &roisnum_data_shape, const op::util::MulticlassNmsBase::Attributes &attrs, float *selected_outputs, const Shape &selected_outputs_shape, int64_t *selected_indices, const Shape &selected_indices_shape, int64_t *valid_outputs)
template<typename T>
void multiply(const T *arg0, const T *arg1, T *out, const size_t count)
template<typename T>
void multiply(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise Multiply operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<typename T>
void mvn(const T *arg, T *out, const Shape &in_shape, const bool normalize_variance, const AxisSet &reduction_axes, const double eps)
template<typename T>
void mvn_6(const T *arg, T *out, const Shape &in_shape, AxisSet reduction_axes, bool normalize_variance, double eps, op::MVNEpsMode eps_mode)
template<typename T>
AxisSet mvn_6_reduction_axes(const ov::Tensor &axes_input, size_t rank)
template<typename T>
void negate(const T *arg, T *out, const size_t count)

Reference implementation of Negative operator.

Parameters
  • arg – Pointer to input data.

  • out – Pointer to output data.

  • count – Number of elements in input buffer.

void nms_rotated(const float *boxes_data, const Shape &boxes_data_shape, const float *scores_data, const Shape &scores_data_shape, int64_t max_output_boxes_per_class, float iou_threshold, float score_threshold, float soft_nms_sigma, int64_t *selected_indices, const Shape &selected_indices_shape, float *selected_scores, const Shape &selected_scores_shape, int64_t *valid_outputs, bool sort_result_descending, bool clockwise = true)
void non_max_suppression5(const float *boxes_data, const Shape &boxes_data_shape, const float *scores_data, const Shape &scores_data_shape, int64_t max_output_boxes_per_class, float iou_threshold, float score_threshold, float soft_nms_sigma, int64_t *selected_indices, const Shape &selected_indices_shape, float *selected_scores, const Shape &selected_scores_shape, int64_t *valid_outputs, const bool sort_result_descending)
void nms5_postprocessing(ov::TensorVector &outputs, const element::Type output_type, const std::vector<int64_t> &selected_indices, const std::vector<float> &selected_scores, int64_t valid_outputs, const element::Type selected_scores_type)
void non_max_suppression(const float *boxes_data, const Shape &boxes_data_shape, const float *scores_data, const Shape &scores_data_shape, int64_t max_output_boxes_per_class, float iou_threshold, float score_threshold, float soft_nms_sigma, int64_t *selected_indices, const Shape &selected_indices_shape, float *selected_scores, const Shape &selected_scores_shape, int64_t *valid_outputs, const bool sort_result_descending)
void nms_postprocessing(ov::TensorVector &outputs, const element::Type output_type, const std::vector<int64_t> &selected_indices, const std::vector<float> &selected_scores, int64_t valid_outputs, const element::Type selected_scores_type)
template<typename T>
size_t non_zero_get_count(const T *arg, const Shape &arg_shape)

Return number of non-zero entries in the input argument.

Parameters
  • argInput tensor

  • arg_shapeInput tensor shape Output number of non-zero entries in arg

template<typename T, typename U>
void non_zero(const T *arg, U *out, const Shape &arg_shape)

Return indices of non-zero entries in input argument.

Parameters
  • argInput tensor

  • arg_shapeInput tensor shape

  • outOutput containing indices of non-zero entries in arg

template<typename T>
void normalize_l2(const T *data, T *out, const Shape &data_shape, const AxisSet &reduction_axes, float eps, op::EpsMode eps_mode)
template<typename T, typename U>
void not_equal(const T *arg0, const T *arg1, U *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)
template<typename INPUT_TYPE>
void one_hot(const INPUT_TYPE *indices, const Shape &indices_shape, char *out, const size_t out_elem_size, const size_t depth, const int64_t one_hot_axis, const char *on_value, const char *off_value)
template<class T>
void logical_or(const T *arg0, const T *arg1, T *out, const size_t count)
template<class T>
void logical_or(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise LogicalOr operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

void pad(const char *data, const char *pad_value, char *out, const size_t elem_size, const Shape &data_shape, const Shape &out_shape, const CoordinateDiff &padding_below, const CoordinateDiff &padding_above, const op::PadMode pad_mode)
template<typename T>
void power(const T *arg0, const T *arg1, T *out, size_t count)
template<typename T>
void power(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise Power operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg0_shapeInput 0 shape.

  • arg1_shapeInput 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<typename T>
void prelu(const T *arg, const T *slope, T *out, const Shape &arg_shape, const Shape &slope_shape)
static inline float clip_great(float x, float threshold)
static inline float clip_less(float x, float threshold)
template<typename T>
void prior_box(const T *data, const T *img, float *dst_data, const Shape &out_shape, const op::v8::PriorBox::Attributes &attrs)
template<typename T>
void prior_box_clustered(const T *data, const T *img, float *dst_data, const Shape &out_shape, const op::v0::PriorBoxClustered::Attributes &attrs)
template<typename T>
void proposal_v0(const T *class_probs, const T *bbox_deltas, const T *image_shape, T *output, const Shape &class_probs_shape, const Shape &bbox_deltas_shape, const Shape &image_shape_shape, const Shape &output_shape, const op::v0::Proposal::Attributes &attrs)
template<typename T>
void proposal_v4(const T *class_probs, const T *bbox_deltas, const T *image_shape, T *output, T *out_probs, const Shape &class_probs_shape, const Shape &bbox_deltas_shape, const Shape &image_shape_shape, const Shape &output_shape, const Shape &out_probs_shape, const op::v0::Proposal::Attributes &attrs)
template<typename T>
void psroi_pooling(const T *input, const Shape &input_shape, const T *rois, const Shape &rois_shape, T *output, const Shape &output_shape, const std::string &mode_str, float spatial_scale, int spatial_bins_x, int spatial_bins_y)
std::pair<uint64_t, uint64_t> random_uniform(const uint64_t *out_shape, const char *min_val, const char *max_val, char *out, const Shape &out_shape_shape, const element::Type &elem_type, uint64_t seed, uint64_t seed2, std::pair<uint64_t, uint64_t> prev_state)
template<typename T>
std::enable_if<ov::is_floating_point<T>()>::type range(const T start, const T step, const size_t num_elem, T *out)

Reference implementation for Range operator (floating-point types).

Parameters
  • start – Start value.

  • step – Step is difference value for consecutive values.

  • num_elem – Number of elements to generate

  • out – Pointer to output data.

template<typename T>
std::enable_if<std::is_integral<T>::value>::type range(const T start, const T step, const size_t num_elem, T *out)

Reference implementation for Range operator (integral types).

Parameters
  • start – Start value.

  • step – Step is difference value for consecutive values.

  • num_elem – Number of elements to generate

  • out – Pointer to output data.

void rdft(const std::vector<float> &input_data, const Shape &input_data_shape, const std::vector<int64_t> &axes_data, const Shape &output_fft_shape, float *rdft_result)
template<class InputIt, class OutputIt>
void reduce_l1(InputIt in, OutputIt out, const Shape &in_shape, const AxisSet &reduction_axes)

Reference implementation of ReduceL1 operator.

Parameters
  • inInput iterator to data.

  • outOutput iterator to results.

  • in_shapeInput shape.

  • reduction_axes – Axes on which reduction is applied.

template<class InputIt, class OutputIt>
void reduce_l2(InputIt in, OutputIt out, const Shape &in_shape, const AxisSet &reduction_axes)

Reference implementation of ReduceL2 operator.

Parameters
  • inInput iterator to data.

  • outOutput iterator to results.

  • in_shapeInput shape.

  • reduction_axes – Axes on which reduction is applied.

template<class T>
void reduce_max(const T *in, T *out, const Shape &in_shape, const AxisSet &reduction_axes)

Reference implementation of ReduceMax operator.

Parameters
  • inInput pointer to data.

  • outOutput pointer to results.

  • in_shapeInput shape.

  • reduction_axes – Axes on which reduction is applied.

template<class T>
void reduce_mean(const T *in, T *out, const Shape &in_shape, const AxisSet &reduction_axes)

Reference implementation of ReduceMean operator.

Parameters
  • inInput pointer to data.

  • outOutput pointer to results.

  • in_shapeInput shape.

  • reduction_axes – Axes on which reduction is applied.

template<class T>
void reduce_min(const T *in, T *out, const Shape &in_shape, const AxisSet &reduction_axes)

Reference implementation of ReduceMin operator.

Parameters
  • inInput pointer to data.

  • outOutput pointer to results.

  • in_shapeInput shape.

  • reduction_axes – Axes on which reduction is applied.

template<typename T>
void reduce_prod(const T *arg, T *out, const Shape &in_shape, const AxisSet &reduction_axes)

Reference implementation of ReduceProduct operator.

Parameters
  • inInput pointer to data.

  • outOutput pointer to results.

  • in_shapeInput shape.

  • reduction_axes – Axes on which reduction is applied.

template<typename T>
void reduce_sum(const T *in, T *out, const Shape &in_shape, const AxisSet &reduction_axes)

Reference implementation of ReduceSum operator.

Parameters
  • inInput pointer to data.

  • outOutput pointer to results.

  • in_shapeInput shape.

  • reduction_axes – Axes on which reduction is applied.

static inline int entry_index(int width, int height, int coords, int classes, int outputs, int batch, int location, int entry)
template<typename T>
static inline T sigmoid(float x)
template<typename T>
static inline void softmax_generic(const T *src_data, T *dst_data, int batches, int channels, int height, int width)
template<typename T>
void region_yolo(const T *input, T *output, const Shape &input_shape, const int coords, const int classes, const int regions, const bool do_softmax, const std::vector<int64_t> &mask)
template<class T, typename std::enable_if<ov::is_floating_point<T>() || std::is_signed<T>::value>::type* = nullptr>
void relu(const T *arg, T *out, const size_t count)

Reference implementation of ReLU operator (signed values).

Reference implementation of ReLU operator (unsigned).

Parameters
  • arg – Pointer to input data.

  • out – Pointer to output data.

  • count – Number of elements in input buffer.

void reorg_yolo(const char *arg, char *out, const Shape &in_shape, int64_t stride, const size_t elem_size)
inline void reshape(const char *in, char *out, const Shape &in_shape, size_t elem_size)

Basic reshape operation, without axes reorder.

Parameters
  • in – Pointer to input data.

  • out – Pointer to output data.

  • in_shapeInput data shape.

  • out_shapeOutput data shape.

  • elem_size – Single data element size im bytes.

void reshape(const char *in, char *out, const Shape &in_shape, const AxisVector &axes_order, const Shape &out_shape, size_t elem_size)

Permutes data shape and axes.

Parameters
  • in – Pointer to input data.

  • out – Pointer to output data.

  • in_shapeInput data shape.

  • axes_order – Axes order.

  • out_shapeOutput data shape.

  • elem_size – Single data element size im bytes.

template<typename T>
void result(const T *arg, T *out, size_t count)
void reverse(const char *arg, char *out, const Shape &arg_shape, const Shape &out_shape, const AxisSet &reversed_axes, size_t elem_size)
template<typename T, typename U>
void reverse_sequence(const T *arg, T *out, const Shape &arg_shape, size_t batch_axis, size_t sequence_axis, const U *sequence_lengths)
template<typename T>
void rnn_cell(const T *X, const Shape &X_shape, const T *H, const Shape &H_shape, const T *W, const Shape &W_shape, const T *R, const Shape &R_shape, const T *B, const Shape &B_shape, T *dst_data, const std::string &activation_f, float clip)
template<typename T>
void roi_align(const T *feature_maps, const T *rois, const int64_t *batch_indices, T *out, const Shape &feature_maps_shape, const Shape &rois_shape, const Shape &batch_indices_shape, const Shape &out_shape, const int pooled_height, const int pooled_width, const int sampling_ratio, const float spatial_scale, const ROIPoolingMode &pooling_mode, const AlignedMode &aligned_mode = AlignedMode::ASYMMETRIC)
template<typename T>
void roi_pooling(const T *feature_maps, const T *rois, T *output, const Shape &feature_maps_shape, const Shape &rois_shape, const Shape &output_shape, const float spatial_scale, const std::string &pooling_method)
size_t shift_pos(size_t pos_in_spanned_data, size_t dim_shift, size_t spanned_shape_size, size_t dim_size)
void roll(const char *arg, const int64_t *shift, const int64_t *axes, char *out, const Shape &arg_shape, const Shape &shift_shape, const Shape &axes_shape, size_t elem_size)
template<typename T>
T round_to_nearest_even(T value)

Rounding algorithm for ov::op::v5::Round::RoundMode::HALF_TO_EVEN.

Template Parameters

T – Value type.

Parameters

value – Value for rounding.

Returns

Rounded value.

template<typename T>
T round_half_away_zero(T value)

Rounding algorithm for ov::op::v5::Round::RoundMode::HALF_AWAY_FROM_ZERO.

Template Parameters

T – Value type.

Parameters

value – Value for rounding.

Returns

Rounded value.

template<typename T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
void round(const T *arg, T *out, const size_t count, const op::v5::Round::RoundMode mode)

Reference implementation of Round operator.

Used when T is OpenVINO floating type.

Used when T is OpenVINO integral type.

Parameters
  • argInput buffer pointer with data to round.

  • outOutput buffer pointer with rounded results.

  • count – Number of elements in input tensor.

  • mode – Rounding mode.

  • argInput buffer pointer with data to round.

  • outOutput buffer pointer with rounded results.

  • count – Number of elements in input tensor.

  • mode – Rounding mode.

template<typename T>
size_t normalize_index(const T idx, const size_t dim_value)
template<typename DataType, typename IndicesType>
void scatter_elem_update_with_reduction(const DataType *input_data, const IndicesType *indices, const DataType *updates, const int64_t axis, DataType *out_buf, const Shape &data_shape, const Shape &indices_shape, const ov::op::v12::ScatterElementsUpdate::Reduction reduction_type, const bool use_init_val)
template<typename DataType, typename IndicesType>
void scatter_elem_update(const DataType *input_data, const IndicesType *indices, const DataType *updates, const int64_t axis, DataType *out_buf, const Shape &data_shape, const Shape &indices_shape, const Reduction reduction_type = Reduction::NONE, const bool use_init_val = true)
template<typename T>
T reduction_neutral_value(const Reduction reduction_type)
template<typename T>
std::function<T(const T, const T)> reduction_functor_for(const Reduction reduction_type)
template<>
std::function<char(const char, const char)> reduction_functor_for<char>(const Reduction reduction_type)
template<typename T>
std::enable_if<std::is_floating_point<T>::value || std::is_class<T>::value, T>::type arithmetic_mean(const T accumulator, const int32_t N)
template<typename T>
std::enable_if<std::is_integral<T>::value, T>::type arithmetic_mean(const T accumulator, const int32_t N)
template<typename dataType, typename indicesType>
void scatterNdUpdate(const dataType *const inputData, const indicesType *const indices, const dataType *const updates, dataType *const outBuf, const Shape &dataShape, const Shape &indicesShape, const Shape &updatesShape)
static const CoordinateTransformBasic get_target_shape(const Shape &data_shape, const Coordinate &start_corner, const Coordinate &end_corner)
static void scatter_update(const char *input_data, const int64_t *indices, const char *updates, const int64_t axis, char *out_buf, const size_t elem_size, const Shape &data_shape, const Shape &indices_shape, const Shape &updates_shape)
template<typename T>
void select(const char *arg0, const T *arg1, const T *arg2, T *out, size_t arg0_count, size_t arg1_count, size_t arg2_count, size_t out_count)
template<typename T>
void select(const char *arg0, const T *arg1, const T *arg2, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const Shape &arg2_shape, const op::AutoBroadcastSpec &broadcast_spec)
template<typename T>
void selu(const T *arg, const T *alpha, const T *lambda, T *out, size_t size_arg, size_t size_alpha, size_t size_lambda)
template<typename T, typename U>
void cell_pass(CellType type, const std::vector<const char*> &inputs, const std::vector<Shape> &shapes, const std::vector<char*> &outputs, const CellArgs &args, bool is_reverse)
template<typename T, typename U>
void lstm_sequence(const char *X, const Shape &X_shape, const char *H, const Shape &H_shape, const char *C, const Shape &C_shape, const char *seq_lengths, const Shape &seq_lengths_shape, const char *W, const Shape &W_shape, const char *R, const Shape &R_shape, const char *B, const Shape &B_shape, char *Y, char *Ho, char *Co, const std::string &activation_f, const std::string &activation_g, const std::string &activation_h, float clip, op::RecurrentSequenceDirection direction)
template<typename T, typename U>
void lstm_sequence_v1(const char *X, const Shape &X_shape, const char *H, const Shape &H_shape, const char *C, const Shape &C_shape, const char *seq_lengths, const Shape &seq_lengths_shape, const char *W, const Shape &W_shape, const char *R, const Shape &R_shape, const char *B, const Shape &B_shape, const char *P, const Shape &P_shape, char *Y, char *Ho, char *Co, const std::string &activation_f, const std::string &activation_g, const std::string &activation_h, float clip, const ov::op::LSTMWeightsFormat weight_format, bool input_forget, op::RecurrentSequenceDirection direction)
template<typename T, typename U>
void gru_sequence(const char *X, const Shape &X_shape, const char *H, const Shape &H_shape, const char *seq_lengths, const Shape &seq_lengths_shape, const char *W, const Shape &W_shape, const char *R, const Shape &R_shape, const char *B, const Shape &B_shape, char *Y, char *Ho, const std::string &activation_f, const std::string &activation_g, const float clip, const op::RecurrentSequenceDirection direction, const bool linear_before_reset, const char *A = nullptr)
template<typename T, typename U>
void rnn_sequence(const char *X, const Shape &X_shape, const char *H, const Shape &H_shape, const char *seq_lengths, const Shape &seq_lengths_shape, const char *W, const Shape &W_shape, const char *R, const Shape &R_shape, const char *B, const Shape &B_shape, char *Y, char *Ho, const std::string &activation_f, float clip, const op::RecurrentSequenceDirection direction)
template<typename T>
inline void shape_of(const Shape &arg_shape, T *out)
void shuffle_channels(const char *arg, char *out, const Shape &data_shape, size_t elem_size, const int64_t axis, const int64_t group)
template<class T>
void sigmoid(const T *arg, T *out, const size_t count)
template<typename T>
void sign(const T *arg, T *out, size_t count)

Reference implementation of Sign operator.

Parameters
  • arg – Pointer to input data.

  • out – Pointer to output data.

  • count – Number of elements in input buffer.

template<typename T>
void sin(const T *arg, T *out, const size_t count)

Reference implementation of Sin operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

template<class T>
void sinh(const T *arg, T *out, size_t count)

Reference implementation of Sinh operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

void slice(const char *data, const Shape &data_shape, char *out, const Shape &out_shape, size_t elem_size, const std::vector<int64_t> &starts, const std::vector<int64_t> &steps, const std::vector<int64_t> &axes)
void slice(const char *arg, char *out, const Shape &arg_shape, const Coordinate &lower_bounds, const Coordinate &upper_bounds, const Strides &strides, const Shape &out_shape, size_t elem_size)
template<typename T>
void softmax(const T *arg, T *out, const Shape &shape, const AxisSet &axes)
template<typename T>
void softplus(const T *arg, T *out, size_t count)
template<typename T>
void softsign(const T *arg, T *out, size_t count)
void space_to_depth(const char *const in, const Shape &in_shape, char *const out, const Shape &out_shape, const size_t block_size, const op::v0::SpaceToDepth::SpaceToDepthMode mode, const size_t elem_size)
void split(const char *data, const Shape &data_shape, size_t elem_size, int64_t axis, size_t num_splits, char **out_data)

Reference implementation of Split operator.

Parameters
  • data – Pointer to input data.

  • data_shapeInput data shape.

  • elem_size – Size of single element type.

  • axis – Axis used for split input data.

  • num_splits – Number of splits

  • out_data – Pointer to output data pointers (must have size of num_splits)

template<class T>
void sqrt(const T *arg, T *out, const size_t count)

Reference implementation of Sqrt operator.

Parameters
  • arg – Pointer to input data.

  • out – Pointer to output data.

  • count – Number of elements in input buffer.

template<typename T>
void squared_difference(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)
void strided_slice(const char *arg, char *out, const Shape &arg_shape, const op::util::SlicePlan &sp, size_t elem_type)
template<class T>
void subtract(const T *arg0, const T *arg1, T *out, size_t count)
template<class T>
void subtract(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise Subtract operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

template<class T>
void swish(const T *arg, const T beta, T *out, const size_t count)

Reference implementation of Swish operator.

Parameters
  • argInput pointer to data.

  • beta – Beta parameter value.

  • outOutput pointer to results.

  • count – Number of elements in input buffer.

template<class T>
void tan(const T *arg, T *out, const size_t count)

Reference implementation of Tan operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

template<class T>
void tanh(const T *arg, T *out, const size_t count)

Reference implementation of Tanh operator.

Parameters
  • argInput buffer pointer with input data.

  • outOutput buffer pointer with results.

  • count – Number of elements in input buffer.

void tensor_iterator(uint64_t num_iterations, const std::shared_ptr<Model> &body, const op::util::OutputDescriptionVector &out_descs, const op::util::InputDescriptionVector &input_descs, ov::TensorVector &out, const ov::TensorVector &args, const custom_evaluate_function &evaluate = nullptr)
void tile(const char *arg, char *out, const Shape &in_shape, const Shape &out_shape, const size_t elem_size, const std::vector<int64_t> &repeats)
template<bool D, typename T, typename U>
inline bool compare_max(const std::tuple<T, U> &a, const std::tuple<T, U> &b)
template<typename T, typename U>
inline bool compare_indices_ascending(const std::tuple<T, U> &a, const std::tuple<T, U> &b)
template<typename T, typename U>
void topk(const T *arg, U *out_indices, T *out_values, const Shape &in_shape, const Shape &out_shape, const size_t axis, const size_t k, const bool compute_max, const op::TopKSortType sort = op::TopKSortType::NONE)

Reference implementation fo TopK operator.

Parameters
  • arg – Pointer to input data.

  • out_indices – Pointer to output indicies.

  • out_values – Pointer to output values.

  • in_shapeInput data shape.

  • out_shapeOutput data (values, indicies) shape.

  • axis – Axis for search of top K elements.

  • k – Number to find of top elements.

  • compute_max – Select mode of find max or min.

  • sort – Sorting type.

void transpose(const char *data, char *out, const Shape &data_shape, size_t element_size, const std::vector<int64_t> &axes_order, const Shape &out_shape)

Reference implementation of Transpose operator.

Parameters
  • data – Pointer to input data.

  • out – Pointer to output data.

  • data_shapeInput data shape.

  • element_size – Element size in bytes for input and output.

  • axes_order – Transpose order.

  • out_shapeOutput data shape.

template<typename Data_t, typename Index_t, typename Count_t = int64_t>
UniqueElements<Index_t, Count_t> find_unique_elements(const Data_t *data, const Shape &data_shape, std::unique_ptr<int64_t> axis, const bool sorted)
template<typename Index_t, typename Count_t = int64_t>
std::tuple<Shape, Shape, Shape> make_tensor_shapes(const UniqueElements<Index_t, Count_t> &unique_elements, const Shape &data_shape, std::unique_ptr<int64_t> axis)
template<typename Data_t, typename Index_t, typename Count_t = int64_t>
void unique(Data_t *out_unique_elements, Index_t *out_indices, Index_t *out_rev_indices, Count_t *out_counts, const Data_t *data, const Shape &data_shape, const Shape &out_shape, const UniqueElements<Index_t, Count_t> &descriptors)
template<typename Iterator, typename value = typename details::from_iterator<Iterator>::stored_value, details::Required<details::IsRandomAccessIt<Iterator>::value> = true>
constexpr auto span(Iterator first, Iterator second) -> Span<value>
template<typename Container, typename = details::void_t<decltype(std::declval<Container>().data()), decltype(std::declval<Container>().size())>>
constexpr auto span(const Container &c) -> Span<const typename Container::value_type>
template<typename Container, typename = details::void_t<decltype(std::declval<Container>().data()), decltype(std::declval<Container>().size())>>
constexpr auto span(Container &c) -> Span<typename Container::value_type>
template<typename Element>
constexpr auto span(const Element *data, std::size_t size) -> Span<const Element>
template<typename Element>
constexpr auto span(Element *data, std::size_t size) -> Span<Element>
template<typename T>
void logical_xor(const T *arg0, const T *arg1, T *out, const size_t count)
template<typename T>
void logical_xor(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const op::AutoBroadcastSpec &broadcast_spec)

Reference implementation of binary elementwise LogicalXor operator.

Parameters
  • arg0 – Pointer to input 0 data.

  • arg1 – Pointer to input 1 data.

  • out – Pointer to output data.

  • arg_shape0Input 0 shape.

  • arg_shape1Input 1 shape.

  • broadcast_spec – Broadcast specification mode.

Variables

constexpr auto nms_rotated_postprocessing = ov::reference::nms_postprocessing
const uint32_t crush_resistance_const_lower_value = 0x9E3779B9
const uint32_t crush_resistance_const_upper_value = 0xBB67AE85
const uint64_t statistic_maximizing_multiplier_n = 0xD2511F53
const uint64_t statistic_maximizing_multiplier_counter = 0xCD9E8D57
const size_t rounds_number = 10
const uint64_t skip_const = 256
struct CellArgs
#include <sequences.hpp>
struct convert_types
#include <random_uniform.hpp>
class GetNearestPixel
#include <interpolate.hpp>

Calculation of nearest pixel.

Public Functions

inline GetNearestPixel()

Constructs calculation of a nearest pixel in the default mode.

inline GetNearestPixel(Nearest_mode mode)

Constructs calculation of nearest pixel for the specified mode.

Parameters

mode – the mode of the calculation of the nearest pixel

inline int64_t operator()(float original, bool is_downsample) const

Performing the nearest pixel calculation.

Parameters
  • original – original coordinate

  • is_downsample – true if it has downsample and false otherwise

Returns

the nearest pixel

class GetOriginalCoordinate
#include <interpolate.hpp>

Calculation of the source coordinate using the resized coordinate.

Public Functions

inline GetOriginalCoordinate()

Constructs calculation of a nearest pixel in the default mode.

inline GetOriginalCoordinate(Transform_mode mode)

Constructs calculation of the source coordinate.

Parameters

mode – the mode of the calculation of the source coordinate.

inline float operator()(float x_resized, float x_scale, float length_resized, float length_original) const

Performing the source coordinate calculation.

Parameters
  • x_resized – resized coordinate

  • x_scale – scale for the considered axis

  • length_resized – length of the resized axis

  • length_original – original length of the axis

Returns

the source coordinate

template<typename T>
class InterpolateEval
#include <interpolate.hpp>

Class to perform interpolation calculation.

Public Functions

inline InterpolateEval(const op::v4::Interpolate::InterpolateAttrs &attrs)

Constructs interpolation calculation using Interpolate attributes.

Parameters

attrs – Interpolate-4 attributes.

inline void operator()(const T *input_data, const Shape &input_data_shape, const std::vector<float> &scales, const std::vector<int64_t> &axes, T *out, const Shape &out_shape)

Performing interpolation calculation.

Parameters
  • input_data – pointer to input data

  • input_data_shape – shape of the input data

  • scales – scale factors for each interpolated axis

  • axes – axes to interpolate

  • out – pointer to memory block for output data

  • out_shape – shape of output data

class InterpolateEvalHelper
#include <interpolate.hpp>

Helper class to implent non-template parts of the interpolation calculation.

struct ICoords
#include <interpolate.hpp>
struct InfoForGenericLinearONNXMode
#include <interpolate.hpp>
struct InfoForLinearMode
#include <interpolate.hpp>
struct LinearModeInnerIterationResult
#include <interpolate.hpp>
template<typename dataType>
class referenceDetectionOutput
#include <detection_output.hpp>
template<typename T>
struct RoundingDirectionGuard
#include <scatter_elements_update.hpp>
template<typename Element>
class Span
#include <span.hpp>

Span should mimic std::span.

Public Functions

inline Span subspan(std::size_t offset, std::size_t size = std::numeric_limits<std::size_t>::max()) const

return sub part of span starting from offset and not greater than size

inline Span &drop_front(std::size_t number_of_elements)

drop number of elements from front

inline Span &drop_back(std::size_t number_of_elements)

drop number of elements from back

template<typename Index_t, typename Count_t>
struct TensorSlice
#include <unique.hpp>

Public Members

Index_t idx = 0

The index of the current element in the original input tensor. It never changes even if the elements get sorted. This value is used as a mapping between a unique element in the first output tensor and the position of this element in the original input tensor.

Index_t rev_idx = -1

The rev_idx is a mapping between every element in the original input and the location of a unique element in the first output tensor. More than one Element can have the same rev_idx.

Count_t count = 1

The number of occurrences of a given element in the input tensor. This value is different than one only for duplicates found in the input tensor.

DescriptorType descriptor_type = DescriptorType::SINGLE_VALUE

Indicates if this object points to a single value in the input tensor (rather than a slice of the tensor)

template<typename Index_t, typename Count_t>
struct UniqueElements
#include <unique.hpp>

Public Members

std::vector<TensorSlice<Index_t, Count_t>> all_tensor_elements

Contains descriptors of all elements in the input tensor. Possibly sorted by value.

std::vector<TensorSlice<Index_t, Count_t>> unique_tensor_elements

Subset of all tensor elements. First occurrences of the unique values.

int64_t axis = 0

Axis (optional). Used to gather unique elements over a given dimension.

template<typename T>
struct widen
#include <helpers.hpp>
template<>
struct widen<double>
#include <helpers.hpp>
template<>
struct widen<float>
#include <helpers.hpp>
namespace adaptive_pool

Functions

inline size_t window_start(size_t idx, size_t arg_shape, size_t out_shape)
inline size_t window_end(size_t idx, size_t arg_shape, size_t out_shape)
template<typename T>
T avg_div(const T sum, size_t n)
template<typename T>
void adaptive_avg_pool_1d(const T *arg, T *out, size_t h_in, size_t h_out)
template<typename T>
void adaptive_avg_pool_2d(const T *arg, T *out, size_t h_in, size_t h_out, size_t w_in, size_t w_out)
template<typename T>
void adaptive_avg_pool_3d(const T *arg, T *out, size_t d_in, size_t d_out, size_t h_in, size_t h_out, size_t w_in, size_t w_out)
namespace detail

Functions

inline void set_u1(uint8_t *buf, size_t idx, uint8_t val)
inline uint8_t get_u1(const uint8_t *buf, size_t idx)
inline void set_u4(uint8_t *buf, size_t idx, uint8_t val)
inline uint8_t get_u4(const uint8_t *buf, size_t idx)
inline void set_i4(uint8_t *buf, size_t idx, int8_t val)
inline int8_t get_i4(const uint8_t *buf, size_t idx)
template<typename TO, typename TI>
TO get_value(const uint8_t *buf, size_t idx, element::Type from_type)
template<typename TI, typename TO>
void lp_convert(const TI *arg, TO *out, size_t count, element::Type_t src_type, element::Type_t dst_type)
template<typename TI, typename TO>
std::enable_if<!std::is_same<TO, char>::value, TO>::type convert(const TI v)
template<typename TI, typename TO>
std::enable_if<std::is_same<TO, char>::value, TO>::type convert(const TI v)
namespace details

Typedefs

template<bool check>
using Required = typename std::enable_if<check, bool>::type
template<typename ...Args>
using void_t = void

Functions

inline uint8_t extract_bit(uint8_t val, uint8_t bit)
template<typename T>
inline bool xnor(T a, T b)
template<typename T_IN, typename T_F>
void binary_convolve_3D_channels(const ConvolutionParams &p, const T_IN *batch, const Shape &batch_shape, const T_F *filter, const Shape &filter_shape, T_IN *&out, const float pad_value)
template<typename Iterator>
std::vector<size_t> get_indices_offsets(const Iterator beg, const Iterator end, size_t last_slice_size)
template<typename T>
void dot(const T *arg0, const T *arg1, T *out, const Shape &arg0_shape, const Shape &arg1_shape, const Shape &out_shape)
std::vector<size_t> get_transpose_order(const Shape &input_shape)
inline std::vector<float> generate_anchors(const op::v0::Proposal::Attributes &attrs, const unsigned int anchor_count)
template<typename T>
static void enumerate_proposals(const T *bottom4d, const T *d_anchor4d, const float *anchors, std::vector<ProposalBox<T>> &proposals, const unsigned int num_anchors, const unsigned int bottom_H, const unsigned int bottom_W, const float img_H, const float img_W, const float min_box_H, const float min_box_W, const int feat_stride, const float box_coordinate_scale, const float box_size_scale, float coordinates_offset, bool initial_clip, bool swap_xy, bool clip_before_nms)
template<typename T>
static void nms(const int num_boxes, const std::vector<ProposalBox<T>> &proposals, std::vector<unsigned int> &index_out, int &num_out, const int base_index, const float nms_thresh, const int max_num_out, T coordinates_offset)
template<typename T>
static void retrieve_rois(const int num_rois, const int item_index, const int num_proposals, const std::vector<ProposalBox<T>> &proposals, const std::vector<unsigned int> &roi_indices, T *rois, int post_nms_topn_, bool normalize, float img_h, float img_w, bool clip_after_nms, T *probs = nullptr)
template<typename T>
static void proposal_exec(const T *class_probs, const T *bbox_deltas, const T *image_shape, T *output, T *out_probs, const Shape &class_probs_shape, const Shape &bbox_deltas_shape, const Shape &image_shape_shape, const Shape &output_shape, const Shape &out_probs_shape, const op::v0::Proposal::Attributes &attrs)
template<typename T, typename std::enable_if<std::is_floating_point<T>::value, bool>::type = true>
bool isfinite(T x)
template<class T, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
constexpr T kahan_summation(const T in, const T prev_sum, T&)

Performs one element summation based on Kahan algorithm to significantly reduce (integral types).

Parameters
  • in – Value to add with previous value of summation.

  • prev_sum – Previous value of summation (accumulator).

Returns

Compensate sum.

template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T kahan_summation(const T in, const T prev_sum, T &compensation)

Performs one element summation based on Kahan algorithm to significantly reduce (floating point types).

Parameters
  • in – Value to add with previous value of summation.

  • prev_sum – Previous value of summation (accumulator).

  • compensation – Accumulates the summation error.

Returns

Compensate sum.

template<typename It>
struct from_iterator
#include <span.hpp>
template<typename, typename = size_t>
struct is_complete : public std::false_type
#include <span.hpp>
template<typename T>
struct is_complete<T, decltype(sizeof(T))> : public std::true_type
#include <span.hpp>
template<typename It>
struct IsRandomAccessIt
#include <span.hpp>
template<typename T>
struct ProposalBox
#include <proposal.hpp>
namespace fake_convert_details

Functions

template<bool INVERT, typename T>
inline T scale_shift(T val, T scale, T shift)
template<typename T>
void apply_scale_shift(T *out, const T *data, const T *scale, const T *shift, const Shape &data_shape, const Shape &scale_shape, const Shape &shift_shape, const bool invert = false)

Apply scale and shift for the data input. The scale_shape and shift_shape should be equal and numpy-broadcastable to the data_shape.

Parameters
  • data – Pointer to the input data.

  • scale – Pointer to the input scale.

  • shift – Pointer to the input shift.

  • data_shapeShape of the input data.

  • scale_shapeShape of the input scale.

  • shift_shapeShape of the input shift.

  • invert – Flag denoting applying scale before (if false) or after conversion (if true).

void apply_conversion(const float16 *data, float16 *out, size_t element_count, const std::string &destination_type)

Call conversion of fp16 value to the desired destination type.

Parameters
  • arg – Pointer to the input data.

  • out – Pointer to the otuput data.

  • count – Number of elements in the data input.

  • destination_type – Name of the destination type.

namespace fake_quantize_details

Functions

template<typename T>
static inline T quantize(const T arg, const T in_low, const T in_high, const T out_low, const T out_high, const T levels_minus_one)
static std::vector<size_t> compute_strides(const ov::Shape &out_shape, const ov::Shape &shape)
static std::tuple<size_t, size_t> get_inner_stride(size_t num_output_elements, const ov::Shape &output_shape, const ov::Shape &shape, size_t current_output_inner_stride)
template<typename T, typename F>
static void fake_quantize_non_unit_inner_stride(const T *arg, const T *in_low, const T *in_high, const T *out_low, const T *out_high, T *out, const Shape &arg_shape, T levels_minus_one, size_t input_inner_stride, const F &get_outer_strides)
template<typename T, typename F>
static void fake_quantize_unit_inner_stride(const T *arg, const T *in_low, const T *in_high, const T *out_low, const T *out_high, T *out, const Shape &arg_shape, T levels_minus_one, size_t input_inner_stride, const F &get_outer_strides)
template<typename T, typename F>
static void fake_quantize_unit_output_intervals_inner_stride(const T *arg, const T *in_low, const T *in_high, const T *out_low, const T *out_high, T *out, const Shape &arg_shape, T levels_minus_one, size_t input_inner_stride, const F &get_outer_strides)
template<typename T, typename F>
static void fake_quantize_unit_input_intervals_inner_stride(const T *arg, const T *in_low, const T *in_high, const T *out_low, const T *out_high, T *out, const Shape &arg_shape, T levels_minus_one, size_t input_inner_stride, const F &get_outer_strides)
template<typename T, typename F>
static void transform(const T *first1, const T *const last1, const T *first2, const T *first3, T *out, const F &f)
template<typename T, typename F>
static void transform(const T *first1, const T *const last1, const T *first2, const T *first3, const T *first4, const T *first5, T *out, const F &f)
template<typename T, typename F1, typename F2>
static void fake_quantize_loop(const Shape &arg_shape, const T *arg, const T *in_low, const T *in_high, const T *out_low, const T *out_high, T *out, size_t input_inner_stride, const F1 &get_outer_strides, const F2 &quantize_loop)
template<typename T, typename F>
void fake_quantize_non_unit_inner_stride(const T *arg, const T *in_low, const T *in_high, const T *out_low, const T *out_high, T *out, const Shape &arg_shape, T levels_minus_one, size_t input_inner_stride, const F &get_outer_strides)
template<typename T, typename F>
void fake_quantize_unit_inner_stride(const T *arg, const T *in_low, const T *in_high, const T *out_low, const T *out_high, T *out, const Shape &arg_shape, T levels_minus_one, size_t input_inner_stride, const F &get_outer_strides)
template<typename T, typename F>
void fake_quantize_unit_output_intervals_inner_stride(const T *arg, const T *in_low, const T *in_high, const T *out_low, const T *out_high, T *out, const Shape &arg_shape, T levels_minus_one, size_t input_inner_stride, const F &get_outer_strides)
template<typename T, typename F>
void fake_quantize_unit_input_intervals_inner_stride(const T *arg, const T *in_low, const T *in_high, const T *out_low, const T *out_high, T *out, const Shape &arg_shape, T levels_minus_one, size_t input_inner_stride, const F &get_outer_strides)
namespace fft_common

Functions

std::vector<int64_t> reverse_shape_of_emulated_complex_tensor(const Shape &shape)
std::vector<int64_t> compute_strides(const std::vector<int64_t> &v)
std::vector<int64_t> coords_from_index(int64_t index, const std::vector<int64_t> &strides)
int64_t offset_from_coords_and_strides(const std::vector<int64_t> &coords, const std::vector<int64_t> &strides)
std::vector<int64_t> reverse_fft_axes(const std::vector<int64_t> &axes, int64_t complex_data_rank)
namespace func

Functions

template<class T, typename std::enable_if<std::is_unsigned<T>::value>::type* = nullptr>
constexpr T abs(const T num)
template<class T, typename std::enable_if<std::is_signed<T>::value || ov::is_floating_point<T>()>::type* = nullptr>
T abs(const T num)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T acos(const T in)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T acosh(const T in)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T asinh(const T in)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T atan(const T in)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T atanh(const T in)
template<class T, typename std::enable_if<std::is_same<typename std::decay<T>::type, char>::value>::type* = nullptr>
T bitwise_not(const T in)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T cos(const T in)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T cosh(const T in)
template<class T>
bool equal(const T lhs, const T rhs)
template<class T, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
T erf(const T v)
void emulate_f8e5m2_on_fp16(const float16 *const arg_f, float16 *out_f, size_t count)

Emulation of conversion fp16 value to f8e5m2 format.

Parameters
  • arg_f – Pointer to the input data.

  • out_f – Pointer to the otuput data.

  • count – Number of elements in the data input.

void emulate_f8e4m3_on_fp16(const float16 *arg_f, float16 *out_f, size_t count)

Emulation of conversion fp16 value to f8e4m3 format.

Exponent denormal values 0 -7 Exponent normal values 1..15 -6..8 (7 - exponent) Exponent NaN values 15 8

Parameters
  • arg_f – Pointer to the input data.

  • out_f – Pointer to the otuput data.

  • count – Number of elements in the data input.

template<class T, typename std::enable_if<std::is_unsigned<T>::value>::type* = nullptr>
constexpr T floor_mod(const T x, const T y)
template<class T, typename std::enable_if<ov::is_floating_point<T>() || std::is_signed<T>::value>::type* = nullptr>
T floor_mod(const T x, const T y)
template<class T>
constexpr bool less(const T lhs, const T rhs)
template<class T>
constexpr bool less_eq(const T lhs, const T rhs)
template<class T>
T max(const T a, const T b)
template<class T>
T min(const T a, const T b)
template<class T, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
constexpr T mod(const T x, const T y)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T mod(const T x, const T y)
template<class T, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
std::pair<T, T> mod_interval_value(const T v1, const T v2, const T m)

Estimates division remainder [v1, v2] % m = [r0, r1] as interval.

Assumes that 0 <= v1 <= v2 and m != 0, in other cases result is undefined behaviour. The result interval estimate minimum and maximum but is not true that value can be any value between min and max. e.g.

  • [4,6] % 5 = [0, 4], but in fact accurate result is set of [0,1,4]

Parameters
  • v1 – Minimum of value interval.

  • v2 – Maximum of value interval.

  • m – Modulo divisor.

Returns

Remainder of division as interval range.

template<class T, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
std::pair<T, T> mod_interval(const T v1, const T v2, const T m1, const T m2)

Estimates division reminder of [v1, v2] & [m1, m2] = [r0, r1] as interval.

  • Assumes that 0 <= v1 <= v2 and 0 < m1 <= m2, in other cases result is undefined behaviour.

Parameters
  • v1 – Minimum of value interval.

  • v2 – Maximum of value interval.

  • m1 – Minimum of modulo divisor.

  • m2 – Maximum of modulo divisor.

Returns

Remainder of division as interval range.

template<class T>
constexpr T multiply(const T a, const T b)
template<class T>
bool not_equal(const T lhs, const T rhs)
template<class T>
T power(const T x, const T y)
template<class T>
T prelu(const T x, const T y)
template<class T>
bool is_negative(const T v)
template<class T, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
T sigmoid(const T value)
template<class T, typename std::enable_if<std::is_unsigned<T>::value>::type* = nullptr>
constexpr T sign(const T v)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T sinh(const T in)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T sqrt(const T in)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T tan(const T in)
template<class T, typename std::enable_if<ov::is_floating_point<T>()>::type* = nullptr>
T tanh(const T in)
template<class T>
T logical_xor(const T a, const T b)
namespace internal

Functions

inline void row_major_strides(const Shape &shape, size_t *strides, size_t size) noexcept
template<typename C, typename T>
inline T value_with_padding_or(const C &arr, size_t padding, size_t idx, T &&default_value)
template<int A0, int A1, typename T, typename U, typename Functor>
inline void numpy_autobroadcast_binop(const T *arg0, const T *arg1, U *out, const Shape &shape0, const Shape &shape1, const size_t *strides0, const size_t *strides1, const size_t padding0, const size_t padding1, const Shape &output_shape, const size_t axis, const size_t stride, Functor elementwise_functor)
inline size_t calculate_fixed_axis(size_t axis, const size_t *strides)
namespace interpolate_pil

Functions

template<typename T_out, typename T_in>
T_out round_up(T_in x)
template<typename T_out, typename T_in>
T_out clip(const T_in &x, const T_out &min = std::numeric_limits<T_out>::min(), const T_out &max = std::numeric_limits<T_out>::max())
static inline double bilinear_filter(double x, double)
static inline double bicubic_filter(double x, double a)
static inline int precompute_coeffs(int in_size, float in0, float in1, int out_size, const filter &filterp, std::vector<int> &bounds, std::vector<double> &kk)
template<typename T>
void imaging_resample_horizontal(T *im_out, Shape im_out_shape, const T *im_in, Shape im_in_shape, int offset, int ksize, std::vector<int> &bounds, std::vector<double> &kk)
template<typename T>
void imaging_resample_vertical(T *im_out, Shape im_out_shape, const T *im_in, Shape im_in_shape, int offset, int ksize, std::vector<int> &bounds, std::vector<double> &kk)
template<typename T>
void imaging_resample_inner(const T *im_in, size_t im_in_xsize, size_t im_in_ysize, size_t xsize, size_t ysize, const filter &filterp, float *box, T *im_out)
struct filter
#include <interpolate_pil.hpp>
namespace iou_rotated

Functions

static inline float dot_2d(const Point2D &A, const Point2D &B)
static inline float cross_2d(const Point2D &A, const Point2D &B)
static inline void get_rotated_vertices(const RotatedBox &box, Point2D (&pts)[4])
static inline int get_intersection_points(const Point2D (&pts1)[4], const Point2D (&pts2)[4], Point2D (&intersections)[24])
static inline int convex_hull_graham(const Point2D (&p)[24], const int num_in, Point2D (&q)[24], bool shift_to_zero = false)
static inline float polygon_area(const Point2D (&q)[24], const int &m)
static inline float rotated_boxes_intersection(const RotatedBox &box1, const RotatedBox &box2)
struct Point2D
#include <nms_rotated_util.hpp>
struct RotatedBox
#include <nms_rotated_util.hpp>
namespace kernel
namespace multinomial

Functions

template<typename T, typename U, typename V>
void multinomial(const T *probs, const Shape &probs_shape, const U *num_samples, const Shape &num_samples_shape, V *output, const Shape &output_shape, const bool with_replacement, const bool log_probs, const uint64_t global_seed, const uint64_t op_seed)

Multinomial operation creates a sequence of indices of classes sampled from the multinomial distribution.

Template Parameters
  • T – Data type of the probs’ values.

  • U – Data type of num_samples’ values.

  • V – Data type of output’s values.

Parameters
  • probsInput tensor containing at each index poisition probability/log probability of sampling a given class.

  • probs_shapeShape of the ‘probs’ tensor.

  • num_samples – Scalar or 1D tensor with a single value that determines the number of samples to generate per batch.

  • num_samples_shapeShape of the ‘num_samples’ tensor.

  • outputOutput tensor for the generated class indices.

  • output_shapeShape of the ‘output’ tensor.

  • with_replacement – Boolean that determines whether a sampled class can appear more than once in the output.

  • log_probs – Boolean that determines whether to treat input probabilities as log probabilities.

  • global_seed – First seed value (key) of Phillox random number generation algorithm. (See RandomUniform for details)

  • op_seed – Second seed value (counter) of Phillox random number generation algorithm. (See RandomUniform for details)

namespace nms_common

Functions

void nms_common_postprocessing(void *prois, void *pscores, void *pselected_num, const element::Type &output_type, const std::vector<float> &selected_outputs, const std::vector<int64_t> &selected_indices, const std::vector<int64_t> &valid_outputs, const element::Type &selected_outputs_type)
struct BoxInfo
#include <nms_common.hpp>
struct Rectangle
#include <nms_common.hpp>
namespace runtime
namespace streams

Namespace for streams in streams executor.

Variables

static constexpr Property<Num, PropertyMutability::RW> num = {"NUM_STREAMS"}

The number of executor logical partitions.

static constexpr Num AUTO = {-1}

Creates bare minimum of streams to improve the performance.

static constexpr Num NUMA = {-2}

Creates as many streams as needed to accommodate NUMA and avoid associated penalties.

struct Num
#include <properties.hpp>

Class to represent number of streams in streams executor.

Public Types

using Base = std::tuple<int32_t>

NumStreams is representable as int32_t.

namespace symbol
namespace util

Functions

bool get_labels(const ov::PartialShape &shape, ov::TensorLabel &labels)

Collects labels from shape. Labels of static dimensions are guaranteed to be ov::no_labels.

Parameters
  • shapeShape object to collect labels from

  • labels – TensorLabel object to collect labels to

Returns

Status of collecting the labels (false if rank is static else true)

bool get_labels(const ov::Output<ov::Node> &output, ov::TensorLabel &labels)

Collects labels from tensor of Output object.

Parameters
  • outputOutput object to collect labels from

  • labels – TensorLabel object to collect labels to

Returns

Status of collecting the labels (false if tensor has no labels else true)

bool are_unique_and_equal_labels(const ov::TensorLabel &lhs, const ov::TensorLabel &rhs)

Compares.

Parameters
  • lhs – TensorLabel object to compare

  • rhs – TensorLabel object to compare

Returns

true if labels are unique and equal between lhs and rhs else false

bool dims_are_equal(const ov::Dimension &lhs, const ov::Dimension &rhs)

Compares dimensions: if dimensions are static compares values of dimensions, if dimensions are dynamic compares their respective labels using TableOfEquivalence.

Parameters
Returns

true if static dimensions are equal and dynamic dimensions have equal labels else false

namespace threading

Typedefs

using Task = std::function<void()>

OpenVINO Task Executor can use any copyable callable without parameters and output as a task. It would be wrapped into std::function object.

template<typename T>
using ThreadLocal = tbb::enumerable_thread_specific<T>

A wrapper class to keep object to be thread local.

Template Parameters

T – A type of object to keep thread local.

template<typename T>
using ThreadSafeQueue = tbb::concurrent_queue<T>
template<typename T>
using ThreadSafeBoundedQueue = tbb::concurrent_bounded_queue<T>

Enums

enum StreamCreateType

Values:

enumerator STREAM_WITHOUT_PARAM
enumerator STREAM_WITH_CORE_TYPE
enumerator STREAM_WITH_NUMA_ID
enumerator STREAM_WITH_OBSERVE

Functions

void get_cur_stream_info(const int stream_id, const bool cpu_reservation, const std::vector<std::vector<int>> org_proc_type_table, const std::vector<std::vector<int>> streams_info_table, StreamCreateType &stream_type, int &concurrency, int &core_type, int &numa_node_id, int &max_threads_per_core)

Get current stream information.

Parameters
  • stream_id[in] stream id

  • cpu_reservation[in] cpu reservation

  • org_proc_type_table[in] available processors in the platform

  • streams_info_table[in] streams information table

  • stream_type[out] stream create type

  • concurrency[out] the number of threads created at the same time

  • core_type[out] core type

  • numa_node_id[out] numa node id

  • max_threads_per_core[out] the max number of threads per cpu core

void reserve_cpu_by_streams_info(const std::vector<std::vector<int>> _streams_info_table, const int _numa_nodes, std::vector<std::vector<int>> &_cpu_mapping_table, std::vector<std::vector<int>> &_proc_type_table, std::vector<std::vector<int>> &_stream_processors, const int _cpu_status)

Reserve cpu resource by streams info.

Parameters
  • _streams_info_table[in] streams info table

  • _numa_nodes[in] number of numa nodes

  • _cpu_mapping_table[out] CPU mapping table for each processor

  • _proc_type_table[out] summary table of number of processors per type

  • _stream_processors[out] processors grouped in stream which is used in core binding in cpu streams executor

  • _cpu_status[in] set cpu status

Returns

void update_proc_type_table(const std::vector<std::vector<int>> _cpu_mapping_table, const int _numa_nodes, std::vector<std::vector<int>> &_proc_type_table)

Update proc_type_table.

Parameters
  • _cpu_mapping_table[in] CPU mapping table for each processor

  • _numa_nodes[in] total number for nodes in system

  • _proc_type_table[out] summary table of number of processors per type

Returns

OPENVINO_API std::shared_ptr< ExecutorManager > executor_manager ()

Variables

static constexpr Property<size_t, PropertyMutability::RW> big_core_streams = {"BIG_CORE_STREAMS"}

Number of streams in Performance-core(big core)

static constexpr Property<size_t, PropertyMutability::RW> small_core_streams = {"SMALL_CORE_STREAMS"}

Number of streams in Efficient-core(small core) on hybrid cores machine.

static constexpr Property<size_t, PropertyMutability::RW> threads_per_stream_big = {"THREADS_PER_STREAM_BIG"}

Number of threads per stream in big cores.

static constexpr Property<size_t, PropertyMutability::RW> threads_per_stream_small = {"THREADS_PER_STREAM_SMALL"}

Number of threads per stream in small cores on hybrid cores machine.

static constexpr Property<size_t, PropertyMutability::RW> small_core_offset = {"SMALL_CORE_OFFSET"}

Small core start offset when binding cpu cores.

class AsyncInferRequestThreadSafeDefault : public IInferRequestInternal
#include <ie_infer_async_request_thread_safe_default.hpp>

Base class with default implementation of asynchronous multi staged inference request. To customize pipeline stages derived class should change the content of AsyncInferRequestThreadSafeDefault::_pipeline member container. It consists of pairs of tasks and executors which will run the task. The class is recommended to be used by plugins as a base class for asynchronous inference request implementation.

Example

Here is an example of asynchronous inference request implementation for some accelerator device. It uses 5 different executors to run different stages of a synchronous inference request.

Note

To synchronize derived context with stages derived class should call AsyncInferRequestThreadSafeDefault::StopAndWait() function in destructor.

Public Types

using Ptr = std::shared_ptr<AsyncInferRequestThreadSafeDefault>

A shared pointer to AsyncInferRequestThreadSafeDefault.

Public Functions

inline AsyncInferRequestThreadSafeDefault(const IInferRequestInternal::Ptr &request, const ITaskExecutor::Ptr &taskExecutor, const ITaskExecutor::Ptr &callbackExecutor)

Wraps a IInferRequestInternal::Ptr implementation and constructs a AsyncInferRequestThreadSafeDefault::_pipeline where taskExecutor is used to run IInferRequestInternal::Infer asynchronously.

Parameters
  • request[in] The synchronous request

  • taskExecutor[in] The task executor

  • callbackExecutor[in] The callback executor

inline ~AsyncInferRequestThreadSafeDefault()

Destroys the object, stops AsyncInferRequestThreadSafeDefault::_pipeline and waits for a finish.

inline StatusCode Wait(int64_t millis_timeout) override

Waits for completion of all pipeline stages If the pipeline raises an exception it will be rethrown here.

Parameters

millis_timeout – A timeout is ms to wait or special enum value of InferRequest::WaitMode

Returns

A status code

class CPUStreamsExecutor : public ov::threading::IStreamsExecutor
#include <cpu_streams_executor.hpp>

CPU Streams executor implementation. The executor splits the CPU into groups of threads, that can be pinned to cores or NUMA nodes. It uses custom threads to pull tasks from single queue.

Public Types

using Ptr = std::shared_ptr<CPUStreamsExecutor>

A shared pointer to a CPUStreamsExecutor object.

Public Functions

explicit CPUStreamsExecutor(const Config &config)

Constructor.

Parameters

config – Stream executor parameters

~CPUStreamsExecutor() override

A class destructor.

virtual void run(Task task) override

Execute ov::Task inside task executor context.

Parameters

task – A task to start

virtual void execute(Task task) override

Execute the task in the current thread using streams executor configuration and constraints.

Parameters

task – A task to start

virtual int get_stream_id() override

Return the index of current stream.

Returns

An index of current stream. Or throw exceptions if called not from stream thread

virtual int get_numa_node_id() override

Return the id of current NUMA Node Return 0 when current stream cross some NUMA Nodes.

Returns

ID of current NUMA Node, or throws exceptions if called not from stream thread

virtual int get_socket_id() override

Return the id of current socket Return 0 when current stream cross some sockets.

Returns

ID of current socket, or throws exceptions if called not from stream thread

class ExecutableNetworkThreadSafeDefault : public IExecutableNetworkInternal
#include <ie_executable_network_thread_safe_default.hpp>

This class provides optimal thread safe default implementation. The class is recommended to be used as a base class for Executable Network implementation during plugin development.

Public Types

typedef std::shared_ptr<ExecutableNetworkThreadSafeDefault> Ptr

A shared pointer to a ExecutableNetworkThreadSafeDefault object.

Public Functions

inline explicit ExecutableNetworkThreadSafeDefault(const ITaskExecutor::Ptr &taskExecutor = std::make_shared<CPUStreamsExecutor>(IStreamsExecutor::Config{"Default"}), const ITaskExecutor::Ptr &callbackExecutor = std::make_shared<CPUStreamsExecutor>(IStreamsExecutor::Config{"Callback"}))

Constructs a new instance.

Parameters
  • taskExecutor[in] The task executor used

  • callbackExecutor[in] The callback executor

inline IInferRequestInternal::Ptr CreateInferRequest() override

Given optional implementation of creating asynchronous inference request to avoid need for it to be implemented by plugin.

Returns

shared_ptr for the created asynchronous inference request

interface ExecutorManager
#include <executor_manager.hpp>

Interface for tasks execution manager. This is global point for getting task executor objects by string id. It’s necessary in multiple asynchronous requests for having unique executors to avoid oversubscription. E.g. There 2 task executors for CPU device: one - in FPGA, another - in OneDNN. Parallel execution both of them leads to not optimal CPU usage. More efficient to run the corresponding tasks one by one via single executor.

Public Functions

virtual std::shared_ptr<ov::threading::ITaskExecutor> get_executor(const std::string &id) = 0

Returns executor by unique identificator.

Parameters

id – An unique identificator of device (Usually string representation of TargetDevice)

Returns

A shared pointer to existing or newly ITaskExecutor

virtual std::shared_ptr<ov::threading::IStreamsExecutor> get_idle_cpu_streams_executor(const ov::threading::IStreamsExecutor::Config &config) = 0

Returns idle cpu streams executor.

Parameters

config – Streams executor config

Returns

pointer to streams executor config

virtual void set_property(const ov::AnyMap &properties) = 0

Allows to configure executor manager.

Parameters

properties – map with configuration

virtual ov::Any get_property(const std::string &name) const = 0

Returns configuration.

Parameters

name – property name

Returns

Property value

virtual void execute_task_by_streams_executor(ov::threading::IStreamsExecutor::Config::PreferredCoreType core_type, ov::threading::Task task) = 0

create a temporary executor to execute the specific task

Parameters
  • core_type – cpu core type

  • task – task to be performed

class ImmediateExecutor : public ov::threading::ITaskExecutor
#include <immediate_executor.hpp>

Task executor implementation that just run tasks in current thread during calling of run() method.

Public Types

using Ptr = std::shared_ptr<ImmediateExecutor>

A shared pointer to a ImmediateExecutor object.

Public Functions

~ImmediateExecutor() override = default

Destroys the object.

inline virtual void run(Task task) override

Execute ov::Task inside task executor context.

Parameters

task – A task to start

interface IStreamsExecutor : public virtual ov::threading::ITaskExecutor
#include <istreams_executor.hpp>

Interface for Streams Task Executor. This executor groups worker threads into so-called streams.

CPU

The executor executes all parallel tasks using threads from one stream. With proper pinning settings it should reduce cache misses for memory bound workloads.

NUMA

On NUMA hosts GetNumaNodeId() method can be used to define the NUMA node of current stream

Subclassed by ov::threading::CPUStreamsExecutor

Public Types

enum ThreadBindingType

Defines inference thread binding type.

Values:

enumerator NONE

Don’t bind the inference threads.

enumerator CORES

Bind inference threads to the CPU cores (round-robin)

enumerator NUMA

Bind to the NUMA nodes (default mode for the non-hybrid CPUs on the Win/MacOS, where the ‘CORES’ is not implemeneted)

enumerator HYBRID_AWARE

Let the runtime bind the inference threads depending on the cores type (default mode for the hybrid CPUs)

using Ptr = std::shared_ptr<IStreamsExecutor>

A shared pointer to IStreamsExecutor interface

Public Functions

~IStreamsExecutor() override

A virtual destructor.

virtual int get_stream_id() = 0

Return the index of current stream.

Returns

An index of current stream. Or throw exceptions if called not from stream thread

virtual int get_numa_node_id() = 0

Return the id of current NUMA Node Return 0 when current stream cross some NUMA Nodes.

Returns

ID of current NUMA Node, or throws exceptions if called not from stream thread

virtual int get_socket_id() = 0

Return the id of current socket Return 0 when current stream cross some sockets.

Returns

ID of current socket, or throws exceptions if called not from stream thread

virtual void execute(Task task) = 0

Execute the task in the current thread using streams executor configuration and constraints.

Parameters

task – A task to start

struct Config
#include <istreams_executor.hpp>

Defines IStreamsExecutor configuration.

Public Functions

void set_property(const ov::AnyMap &properties)

Sets configuration.

Parameters

properties – map of properties

void set_property(const std::string &key, const ov::Any &value)

Sets configuration.

Parameters
  • key – property name

  • value – property value

ov::Any get_property(const std::string &key) const

Return configuration value.

Parameters

key – configuration key

Returns

configuration value wrapped into ov::Any

inline Config(std::string name = "StreamsExecutor", int streams = 1, int threadsPerStream = 0, ThreadBindingType threadBindingType = ThreadBindingType::NONE, int threadBindingStep = 1, int threadBindingOffset = 0, int threads = 0, PreferredCoreType threadPreferredCoreType = PreferredCoreType::ANY, std::vector<std::vector<int>> streamsInfoTable = {}, bool cpuReservation = false)

A constructor with arguments.

Parameters
  • name[in] The executor name

  • streams[in] Number of streams.

  • threadsPerStream[in] Number of threads per stream that executes ov_parallel calls.

  • threadBindingType[in]

  • threadBindingStep[in]

  • threadBindingOffset[in]

  • threads[in]

  • threadPreferBigCores[in]

void update_executor_config(int stream_nums, int threads_per_stream, PreferredCoreType core_type, bool cpu_pinning)

Modify _streams_info_table and related configuration according to user-specified parameters, bind threads to cpu cores if cpu_pinning is true.

Parameters
  • stream_nums – Number of streams specified by user

  • threads_per_stream – Number of threads per stream specified by user

  • core_type – Cpu type (Big/Little/Any) specified by user

  • cpu_pinning – Whether to bind the threads to cpu cores

void set_config_zero_stream()

Set _streams_info_table and _cpu_reservation in cpu streams executor config when nstreams = 0, that is, only create one thread with TBB.

Public Members

std::string _name

Used by ITT to name executor threads.

int _streams = 1

Number of streams.

int _threadsPerStream = 0

Number of threads per stream that executes ov_parallel calls.

ThreadBindingType _threadBindingType = ThreadBindingType::NONE

Thread binding to hardware resource type. No binding by default

int _threadBindingStep = 1

In case of CORES binding offset type thread binded to cores with defined step

int _threadBindingOffset = 0

In case of CORES binding offset type thread binded to cores starting from offset

int _threads = 0

Number of threads distributed between streams. Reserved. Should not be used.

int _big_core_streams = 0

Number of streams in Performance-core(big core)

int _small_core_streams = 0

Number of streams in Efficient-core(small core)

int _threads_per_stream_big = 0

Threads per stream in big cores.

int _threads_per_stream_small = 0

Threads per stream in small cores.

int _small_core_offset = 0

Calculate small core start offset when binding cpu cores.

bool _enable_hyper_thread = true

enable hyper thread

enum ov::threading::IStreamsExecutor::Config::PreferredCoreType _threadPreferredCoreType = PreferredCoreType::ANY

In case of HYBRID_AWARE hints the TBB to affinitize.

Public Static Functions

static Config make_default_multi_threaded(const Config &initial, const bool fp_intesive = true)

Create appropriate multithreaded configuration filing unconfigured values from initial configuration using hardware properties.

Parameters
  • initial – Inital configuration

  • fp_intesive – additional hint for the the (Hybrid) core-types selection logic whether the executor should be configured for floating point intensive work (as opposite to int8 intensive)

Returns

configured values

interface ITaskExecutor
#include <itask_executor.hpp>

Interface for Task Executor. OpenVINO uses ov::ITaskExecutor interface to run all asynchronous internal tasks. Different implementations of task executors can be used for different purposes:

  • To improve cache locality of memory bound CPU tasks some executors can limit task’s affinity and maximum concurrency.

  • The executor with one worker thread can be used to serialize access to acceleration device.

  • Immediate task executor can be used to satisfy ov::ITaskExecutor interface restrictions but run tasks in current thread.

Synchronization

It is ov::ITaskExecutor user responsibility to wait for task execution completion. The c++11 standard way to wait task completion is to use std::packaged_task or std::promise with std::future. Here is an example of how to use std::promise to wait task completion and process task’s exceptions:

    // std::promise is move only object so to satisfy copy callable constraint we use std::shared_ptr
    auto promise = std::make_shared<std::promise<void>>();
    // When the promise is created we can get std::future to wait the result
    auto future = promise->get_future();
    // Rather simple task
    ov::threading::Task task = [] {
        std::cout << "Some Output" << std::endl;
    };
    // Create an executor
    ov::threading::ITaskExecutor::Ptr taskExecutor =
        std::make_shared<ov::threading::CPUStreamsExecutor>(ov::threading::IStreamsExecutor::Config{});
    if (taskExecutor == nullptr) {
        // ProcessError(e);
        return;
    }
    // We capture the task and the promise. When the task is executed in the task executor context
    // we munually call std::promise::set_value() method
    taskExecutor->run([task, promise] {
        std::exception_ptr currentException;
        try {
            task();
        } catch(...) {
            // If there is some exceptions store the pointer to current exception
            currentException = std::current_exception();
        }

        if (nullptr == currentException) {
            promise->set_value();  //  <-- If there is no problems just call std::promise::set_value()
        } else {
            promise->set_exception(currentException);    //  <-- If there is an exception forward it to std::future object
        }
    });
    // To wait the task completion we call std::future::wait method
    future.wait();  //  The current thread will be blocked here and wait when std::promise::set_value()
                    //  or std::promise::set_exception() method will be called.

    // If the future store the exception it will be rethrown in std::future::get method
    try {
        future.get();
    } catch(std::exception& /*e*/) {
        // ProcessError(e);
    }

Note

Implementation should guaranty thread safety of all methods

Subclassed by ov::threading::IStreamsExecutor, ov::threading::ImmediateExecutor

Public Types

using Ptr = std::shared_ptr<ITaskExecutor>

A shared pointer to ITaskExecutor interface

Public Functions

virtual ~ITaskExecutor() = default

Destroys the object.

virtual void run(Task task) = 0

Execute ov::Task inside task executor context.

Parameters

task – A task to start

virtual void run_and_wait(const std::vector<Task> &tasks)

Execute all of the tasks and waits for its completion. Default run_and_wait() method implementation uses run() pure virtual method and higher level synchronization primitives from STL. The task is wrapped into std::packaged_task which returns std::future. std::packaged_task will call the task and signal to std::future that the task is finished or the exception is thrown from task Than std::future is used to wait for task execution completion and task exception extraction.

Note

run_and_wait() does not copy or capture tasks!

Parameters

tasks – A vector of tasks to execute

template<typename T>
class ThreadSafeBoundedPriorityQueue
#include <thread_safe_containers.hpp>
template<typename T>
class ThreadSafeQueueWithSize
#include <thread_safe_containers.hpp>
namespace util

Typedefs

using FilePath = std::string

Enums

enum class LOG_TYPE

Values:

enumerator _LOG_TYPE_ERROR
enumerator _LOG_TYPE_WARNING
enumerator _LOG_TYPE_INFO
enumerator _LOG_TYPE_DEBUG

Functions

template<typename T>
std::string join(const T &v, const std::string &sep = ", ")
template<typename T>
std::string vector_to_string(const T &v)
std::string to_lower(const std::string &s)
std::string to_upper(const std::string &s)
size_t hash_combine(const std::vector<size_t> &list)
inline std::string ltrim(const std::string &s)

trim from start (in place)

Parameters

s – - string to trim

inline std::string rtrim(const std::string &s)

trim from end (in place)

Parameters

s – - string to trim

inline std::string trim(const std::string &s)

Trims std::string from both ends (in place)

Parameters

s – A reference to a std::tring to trim

Returns

A reference to a trimmed std::string

inline bool ends_with(const std::string &src, const char *with)

check string end with given substring

Parameters
  • src – - string to check

  • with – - given substring

Returns

true if string end with given substring

template<typename T>
inline bool ends_with(const std::basic_string<T> &str, const std::basic_string<T> &suffix)

check string/wstring end with given substring

Parameters
  • src – - string/wstring to check

  • with – - given substring

Returns

true if string end with given substring

std::vector<std::string> split(const std::string &s, char delimiter, bool trim = false)
template<typename T>
T ceil_div(const T &x, const T &y)
template<typename T, typename A, typename V>
bool contains(const std::vector<T, A> &vec, const V &v)
std::string filter_lines_by_prefix(const std::string &str, const std::string &prefix)
constexpr const char *find_last(ConstString s, size_t offset, char ch)
constexpr const char *find_last(ConstString s, char ch)
constexpr const char *get_file_name(ConstString s)
std::string getenv_string(const char *env_var)

Get the names environment variable as a string.

Parameters

env_var – The string name of the environment variable to get.

Returns

Returns string by value or an empty string if the environment variable is not set.

int32_t getenv_int(const char *env_var, int32_t default_value = -1)

Get the names environment variable as an integer. If the value is not a valid integer then an exception is thrown.

Parameters
  • env_var – The string name of the environment variable to get.

  • default_value – The value to return if the environment variable is not set.

Returns

Returns value or default_value if the environment variable is not set.

bool getenv_bool(const char *env_var, bool default_value = false)

Get the names environment variable as a boolean. If the value is not a valid boolean then an exception is thrown. Valid booleans are one of 1, 0, on, off, true, false All values are case insensitive. If the environment variable is not set the default_value is returned.

Parameters
  • env_var – The string name of the environment variable to get.

  • default_value – The value to return if the environment variable is not set.

Returns

Returns the boolean value of the environment variable.

std::string sanitize_path(const std::string &path)

Remove path components which would allow traversing up a directory tree.

Parameters

path – A path to file

Returns

A sanitized path

std::string get_file_name(const std::string &path)

Returns the name with extension for a given path.

Parameters

path – The path to the output file

std::string get_absolute_file_path(const std::string &path)

Interface function to get absolute path of file.

Parameters

path – - path to file, can be relative to current working directory

Throws

runtime_error – if absolute path can’t be resolved

Returns

Absolute path of file

bool is_absolute_file_path(const std::string &path)

Interface function to check path to file is absolute or not.

Parameters

path – - path to file, can be relative to current working directory

Throws

runtime_error – if any error occurred

Returns

True if path is absolute and False otherwise

void create_directory_recursive(const std::string &path)

Interface function to create directorty recursively by given path.

Parameters

path – - path to file, can be relative to current working directory

Throws

runtime_error – if any error occurred

bool directory_exists(const std::string &path)

Interface function to check if directory exists for given path.

Parameters

path – - path to directory

Returns

true if directory exists, false otherwise

inline int64_t file_size(const char *path)

Returns file size for file.

Parameters

path[in] The file name

Returns

file size

inline bool file_exists(const char *path)

Returns file size for file.

Parameters

path[in] The file name

Returns

file size

inline int64_t file_size(const std::string &path)

Returns file size for file.

Parameters

path[in] The file name

Returns

file size

inline bool file_exists(const std::string &path)

Returns true if file exists.

Parameters

path[in] The file name

Returns

true if file exists

std::string get_file_ext(const std::string &path)
std::string get_directory(const std::string &path)
std::string path_join(const std::vector<std::string> &paths)
void iterate_files(const std::string &path, const std::function<void(const std::string &file, bool is_dir)> &func, bool recurse = false, bool include_links = false)
void convert_path_win_style(std::string &path)
std::string get_ov_lib_path()
inline std::string from_file_path(const FilePath &path)
inline FilePath to_file_path(const std::string &path)
inline std::string get_ov_library_path()
template<typename C, typename = typename std::enable_if<(std::is_same<C, char>::value || std::is_same<C, wchar_t>::value)>::type>
inline std::basic_string<C> make_plugin_library_name(const std::basic_string<C> &path, const std::basic_string<C> &input)
FilePath get_plugin_path(const std::string &plugin)

Format plugin path (canonicalize, complete to absolute or complete to file name) for further dynamic loading by OS.

Parameters

plugin – - Path (absolute or relative) or name of a plugin. Depending on platform, plugin is wrapped with shared library suffix and prefix to identify library full name

Returns

absolute path or file name with extension (to be found in ENV)

FilePath get_compiled_plugin_path(const std::string &plugin)

Find the plugins which are located together with OV library.

Parameters

plugin – - Path (absolute or relative) or name of a plugin. Depending on platform, plugin is wrapped with shared library suffix and prefix to identify library full name

Returns

absolute path or file name with extension (to be found in ENV)

FilePath get_plugin_path(const std::string &plugin, const std::string &xml_path, bool as_abs_only = false)

Format plugin path (canonicalize, complete to absolute or complete to file name) for further dynamic loading by OS.

Parameters
  • plugin – - Path (absolute or relative) or name of a plugin. Depending on platform, plugin is wrapped with shared library suffix and prefix to identify library full name

  • xml_path – - Path (absolute or relative) to XML configuration file

  • as_abs_only – - Bool value, allows return file names or not

Returns

absolute path or file name with extension (to be found in ENV)

std::vector<uint8_t> load_binary(const std::string &path)

load binary data from file

Parameters

path – - binary file path to load

Returns

binary vector

void save_binary(const std::string &path, std::vector<uint8_t> binary)

save binary data to file

Parameters

path – - binary file path to store

void save_binary(const std::string &path, const char *binary, size_t bin_size)
const char *trim_file_name(const char *const fname)

Trim OpenVINO project file name path if OpenVINO project directory found.

Function use OV_NATIVE_PARENT_PROJECT_ROOT_DIR definition with project directory name defines ‘openvino_dir_name’. The input file name is scanned for OV_NATIVE_PARENT_PROJECT_ROOT_DIR, if found returns pointer to trimmed name otherwise returns input pointer.

e.g: OV_NATIVE_PARENT_PROJECT_ROOT_DIR = openvino

  • /home/user/openvino/src/example.cpp -> src/example.cpp

  • ../../../../openvino/src/example.cpp -> src/example.cpp

Parameters

fname – Pointer to OpenVINO file name path.

Returns

Pointer to trimmed file name path.

void default_logger_handler_func(const std::string &s)
std::shared_ptr<void> load_shared_object(const char *path)

Loads a library with the name specified.

Parameters

path – Full or relative path to the plugin library

Returns

Reference to shared object

void *get_symbol(const std::shared_ptr<void> &shared_object, const char *symbolName)

Searches for a function symbol in the loaded module.

Parameters
  • shared_object – shared object reference

  • symbolName – Name of the function to find

Throws

Exception – if the function is not found

Returns

A pointer to the function if found

ov::Tensor make_tensor(const std::shared_ptr<ov::ITensor> &tensor, const std::shared_ptr<void> &so)
template<class T, class U = T>
constexpr bool is_max(const T &value)

Check if value of type T has got maximum value of type U.

Template Parameters
  • TInput value type

  • U – Type to get its minimum for comparision. Default same as T.

Parameters

valueInput value.

Returns

True if input value has got maximum value of type U otherwise false.

template<class T, class U = T>
constexpr bool is_min(const T &value)

Check if value of type T has got minimum value of type U.

Template Parameters
  • TInput value type.

  • U – Type to get its minimum for comparision. Default same as T.

Parameters

valueInput value.

Returns

True if input value has got minimum value of type U otherwise false.

template<typename T>
struct AsTypePtr
template<typename In> shared_ptr< In > >
#include <type.hpp>

Casts a std::shared_ptr<Value> to a std::shared_ptr<Type> if it is of type Type, nullptr otherwise

template<class T>
struct Cast
#include <shape_infer_type_utils.hpp>

Transform tensor data by cast them to type T.

Template Parameters

T – Type of returned value.

class ConstString
#include <const_string.hpp>
template<class C>
struct FileTraits

OS specific file traits.

template<>
struct FileTraits<char>
#include <file_util.hpp>
template<>
struct FileTraits<wchar_t>
#include <file_util.hpp>
template<class T>
struct InTypeRange
#include <shape_infer_type_utils.hpp>

Check if input data is in [T::min(), T::max()] and then cast it to T.

Template Parameters

T – Type of returned value and used to specified min, max of valid value range.

Throws ov::AssertFailure

if input value not in type range.

class Logger
#include <log.hpp>
class LogHelper
#include <log.hpp>
namespace dim

Functions

template<class TDim, typename std::enable_if<std::is_arithmetic<TDim>::value>::type* = nullptr>
constexpr bool is_static(const TDim)
template<class TDim, typename std::enable_if<!std::is_arithmetic<TDim>::value>::type* = nullptr>
constexpr bool is_static(const TDim &d)
template<class TDim>
constexpr std::enable_if<std::is_arithmetic<TDim>::value, TDim>::type get_length(const TDim &d)
template<class TDim>
constexpr std::enable_if<!std::is_arithmetic<TDim>::value, typenameTDim::value_type>::type get_length(const TDim &d)
template<class T>
constexpr bool is_inf_bound(const T dim)

Checks if dimension length is infinite bound (undefined).

Template Parameters

T – Type of dimension length.

Parameters

dimDimension length value.

Returns

True if dimension length has infinite bound, otherwise false.

template<class T, class U = typename Dimension::value_type>
constexpr std::enable_if<std::is_same<T, size_t>::value, U>::type value_convert(const T dim)

Convert static dimension length to ov::Dimension::value_type.

As static dimension length type is size_t (bit-length depends on architecture) the maximum value (undefined) is convert to ov::Dimension infinite bound.

Template Parameters
  • T – Static dimension type (size_t)

  • U – ov::Dimension::value_type

Parameters

dimDimension length to convert.

Returns

Converted input value to ov::Dimension::value_type.

template<class T, class U = typename Dimension::value_type>
constexpr std::enable_if<std::is_same<T, U>::value, U>::type value_convert(const T dim)

Conversion of dimension when input type is same as ov::Dimension::value_type.

Return value as it is.

Template Parameters
  • TDimension type same as ov::Dimension::value_type.

  • U – Dimension::value_type.

Parameters

dimDimension length to convert.

Returns

Same value as input.

template<class T>
constexpr auto dilated(const T dim, const T dilation) -> T

Calculate dilated dimension value.

Parameters
  • dimDimension size value.

  • dilation – Dilation value

Returns

Dilated dimension value.

template<class TDim>
constexpr auto dilated(const TDim &dim, const typename TDim::value_type dilation) -> TDim

Calculate dilated dimension.

Template Parameters

TDimDimension type.

Parameters
  • dimDimension.

  • dilation – Dilation value.

Returns

Return dimension after dilation.

template<class TDim>
constexpr std::enable_if<std::is_arithmetic<TDim>::value, TDim>::type padded(const TDim dim, const int64_t pad_num)

Calculate padded dimension size as dim size + padding size.

Template Parameters

TDimDimension type as dimension class value type or any arithmetic value.

Parameters
  • dimDimension size value.

  • pad_num – Number of padding to add.

Returns

Padded dimension value or infinite bound.

template<class TDim>
std::enable_if<std::is_class<TDim>::value, TDim>::type padded(const TDim &dim, const int64_t pad_num)

Calculate padded dimension size as dim + padding size.

Note

the Dimension + operator cannot be used if padding is ‘-1’ which result add dynamic dimension.

Template Parameters

TDimDimension type as dimension class.

Parameters
  • dimDimension.

  • pad_num – Number padding to add.

Returns

Padded dimension.

template<class TDim, class T = typename std::conditional<std::is_arithmetic<TDim>::value, size_t, typename Dimension::value_type>::type>
inline std::pair<T, T> padding(const TDim &dim, const int64_t kernel_size, const int64_t dilation, int64_t stride)

Calculate dimension padding required by filter/kernel properties.

Provides pair of padding values as left padding is total value of required padding divided by 2 and right as total required padding minus left padding.

Parameters
  • dim – input dimension to calculate its padding.

  • filter_size – Kernel size for input dimension.

  • dilation – Kernel dilation.

  • stride – Kernel stride.

Returns

Pair of left, right padding values for input dimension.

template<class TDim>
auto ceil_div(const TDim &dim, const typename TDim::value_type divisor) -> TDim

Divide dimension using ceil rounding.

Template Parameters
Parameters
Returns

Divided dimension with bounds round up.

template<class TDim>
auto floor_div(const TDim &dim, const typename TDim::value_type divisor) -> TDim

Divide dimension using floor rounding.

Template Parameters
Parameters
Returns

Divided dimension with bound round down.

template<class TDim, typename std::enable_if<!std::is_same<Dimension, typename std::decay<TDim>::type>::value>::type* = nullptr>
bool is_empty(TDim &&d)

Check if dimension is empty.

For static dimension the empty dimension is equal to zero dimension.

For iv::Dimension the empty means that has no dimension at all.

Template Parameters
Parameters
Returns

true if dimension is empty otherwise false.

Returns

true if dimension is empty otherwise false.

template<class TDim>
bool is_divisible(const TDim &quotient, const typename TDim::value_type dividend)

Check if dimension is evenly divisible.

Template Parameters

TDimDimension type.

Parameters
  • quotientDimension to check.

  • dividend – Dividend to check.

Returns

true if dimension is divisible other wise false.

template<class TDim>
void scale(TDim &d, float scale)

Scale dimension size by floating point value.

Template Parameters

TDimDimension type.

Parameters
  • dDimension to scale.

  • scale – Scale value for dimension.

Variables

constexpr int64_t inf_bound = -1

Infinite bound value for dimension.