Group Execution model utilities¶
- group ov_dev_exec_model
Contains
ExecutionNodeand its properties.Variables
-
static const char ORIGINAL_NAMES[] = "originalLayersNames"¶
Used to get a string of layer names separated by a comma from the original IR, which were fused/merged to the current executable primitive.
-
static const char IMPL_TYPE[] = "primitiveType"¶
Used to get a type of the executable primitive.
-
static const char OUTPUT_PRECISIONS[] = "outputPrecisions"¶
Used to get output precisions of the executable primitive.
-
static const char PERF_COUNTER[] = "execTimeMcs"¶
Used to get a value of execution time of the executable primitive.
-
static const char OUTPUT_LAYOUTS[] = "outputLayouts"¶
Used to get output layouts of primitive.
-
static const char EXECUTION_ORDER[] = "execOrder"¶
Used to get an execution order of primitive.
-
static const char LAYER_TYPE[] = "layerType"¶
Used to get a type of primitive.
-
static const char RUNTIME_PRECISION[] = "runtimePrecision"¶
Used to get runtime precision of the executable primitive.
-
class ExecutionNode : public ov::op::Op
- #include <exec_model_info.hpp>
The Execution node which is used to represent node in execution graph.
It contains the following type of information in node runtime information:
ExecGraphInfoSerialization::ORIGINAL_NAMES
ExecGraphInfoSerialization::IMPL_TYPE
ExecGraphInfoSerialization::OUTPUT_PRECISIONS
ExecGraphInfoSerialization::PERF_COUNTER
ExecGraphInfoSerialization::OUTPUT_LAYOUTS
ExecGraphInfoSerialization::EXECUTION_ORDER
ExecGraphInfoSerialization::LAYER_TYPE
ExecGraphInfoSerialization::RUNTIME_PRECISION
Public Functions
-
ExecutionNode()
A default constructor with no node inputs and 0 output ports.
-
ExecutionNode(const ov::OutputVector &arguments, size_t output_size = 1)
Constructs a new execution node with a given parameters.
- Parameters
arguments – [in] Inputs nodes
output_size – [in] A number of output ports
-
std::shared_ptr<ov::Node> clone_with_new_inputs(const ov::OutputVector &inputs) const override
Creates a new execution node with the same state, but different input nodes.
- Parameters
inputs – [in] The input nodes
- Returns
A newly created execution node
-
virtual bool visit_attributes(ov::AttributeVisitor&) override
Visits attributes of the node.
- Parameters
visitor – [in] An attribute visitor
- Returns
Returns
trueif an operation has completed successfully
-
namespace ov
transformation aligns elementwise constant inputs ranks with its output rank
A namespace with const values for Execution Graph parameters names.
Executable Model Info is represented in ov::Model format with general ExecutionNode nodes inside including connections between the nodes. Each node describes an executable hardware-specific primitive and stores its parameters within ExecutionNode::get_rt_info map. There is a list of general keys for the parameters map.
OpenVINO C++ API.
Resolves transpose_b key from MatMul operation if corresponding input is constant or FakeQuantize by inserting Transpose.
Functions
- LP_TRANSFORMATIONS_API void mark_as_bias (const std::shared_ptr< Node > &node)
- LP_TRANSFORMATIONS_API bool marked_as_bias (const std::shared_ptr< const Node > &node)
-
std::ostream &operator<<(std::ostream &out, const Mask &mask)
-
void postpone_fp16_compression(RTMap &rt_info)
-
bool is_fp16_compression_postponed(const RTMap &rt_info)
-
void do_not_postpone_fp16_compression(RTMap &rt_info)
getFusedNames return string with operation names separated by coma in alphabetical order
- Parameters
node – [in] The node will be used to get FusedNames attribute
getFusedNamesVector return vector of fused names sorted in alphabetical order
- Parameters
node – [in] The node will be used to get FusedNames attribute
- Returns
vector of strings
-
bool has_nms_selected_indices(const Node *node)
-
void set_nms_selected_indices(Node *node)
getPrimitivesPriority return string with primitive priorities value
- Parameters
node – [in] The node will be used to get PrimitivesPriority attribute
-
bool is_sinking_node(const Node *node)
-
std::shared_ptr<ov::MappedMemory> load_mmap_object(const std::string &path)
Returns mapped memory for a file from provided path. Instead of reading files, we can map the memory via mmap for Linux in order to avoid time-consuming reading and reduce memory consumption.
- Parameters
path – Path to a file which memory will be mmaped.
- Returns
MappedMemory shared ptr object which keep mmaped memory and control the lifetime.
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const AxisSet &axis_set)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const AxisVector &axis_vector)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const Coordinate &coordinate)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const CoordinateDiff &coordinate_diff)
- OPENVINO_API std::ostream & operator<< (std::ostream &str, const Dimension &dimension)
Insert a human-readable representation of a dimension into an output stream.
Inserts the string
?ifdimensionis dynamic; else insertsdimension.get_length().- Parameters
str – The output stream targeted for insertion.
dimension – The dimension to be inserted into
str.
- Returns
A reference to
strafter insertion.
-
template<typename Type, typename Value>
std::enable_if<std::is_convertible<Value, std::string>::value, Type>::type as_enum(const Value &value) Returns the enum value matching the string.
-
template<typename Value>
const std::string &as_string(Value value) Returns the string matching the enum value.
-
static inline std::ostream &write_all_to_stream(std::ostream &str)
-
template<typename T, typename ...TS>
std::ostream &write_all_to_stream(std::ostream &str, T &&arg, TS&&... args)
-
template<class T, typename std::enable_if<!std::is_same<typename std::decay<T>::type, std::string>::value>::type* = nullptr>
std::string stringify(T &&arg)
-
template<class T, typename std::enable_if<std::is_same<typename std::decay<T>::type, std::string>::value>::type* = nullptr>
T &stringify(T &&arg)
-
void create_extensions(std::vector<Extension::Ptr>&)
The entry point for library with OpenVINO extensions.
- Parameters
vector – of extensions
- OPENVINO_API void traverse_nodes (const std::shared_ptr< const Model > &p, const std::function< void(const std::shared_ptr< Node > &)> &f)
- OPENVINO_API void traverse_nodes (const Model *p, const std::function< void(const std::shared_ptr< Node > &)> &f)
- OPENVINO_API void traverse_nodes (const NodeVector &subgraph_results, const std::function< void(const std::shared_ptr< Node > &)> &f, const NodeVector &subgraph_params={})
Visit each node in a sub-graph of the entire graph.
Traverses a sub-graph starting from subgraph_results moving up towards parameter nodes. Traversal stops if it hits a node in subgraph_params.
Most useful for finding parameters of a graph directly from the result nodes and not from function parameters or extracting a subgraph relevant to the computation of certain outputs
- OPENVINO_API void replace_node (const std::shared_ptr< Node > &target, const std::shared_ptr< Node > &replacement, const std::vector< int64_t > &output_order)
Replace the node
targetwith the nodereplacement, i.e., redirect all users and control dependencies oftargettoreplacement.This is primarily used in graph-rewriting passes. For example, we might “fuse” two Concat operations as follows:
(Step 0: Original graph)
A B | | v v N0[Concat, concatenation_axis=3] C | | v v N1[Concat, concatenation_axis=3] | | v v some_user another_user
(Step 1: Construct replacement)
shared_ptr<Node> new_N1 = make_shared<op::Concat>({A,B,C},3);
A————————————-—. | | | B————-—)—. | | | | v v | | N0[Concat, concatenation_axis=3] C–—)—)—. | | | | | v v v v v N1[Concat, concatenation_axis=3] new_N1[Concat, concatenation_axis=3] | | v v some_user another_user
(Step 2: Replace N1 with new_N1)
replace_node(N1, new_N1);
A————————————-—. | | | B————-—)—. | | | | v v | | N0[Concat, concatenation_axis=3] C–—)—)—. | | | | | v v v v v N1[Concat, concatenation_axis=3] new_N1[Concat, concatenation_axis=3] | | v v some_user another_user
(Step 3: N0 and N1 are now dead, nodes will be freed)
[happens automatically, once all shared_ptrs to N1 are released]
A————————————-—. | B————-—)—. | | | | C–—)—)—. | | | v v v new_N1[Concat, concatenation_axis=3] | | v v some_user another_user
NOTE 1: replace_node is not type-safe (the graph is not revalidated). For example, the following is allowed, even if node
some_userrequires an input of shape 2x2:(Before) A(shape=2x2) B(shape=3x3) | v some_user(requires 2x2 input)
(After — graph is now invalid)
NOTE 2: it is possible to insert a cycle into the graph with replace_node, resulting in an invalid graph. Care must be taken to avoid this. One common example is when you are attempting to insert a new nodereplace_node(A, B); A(shape=2x2) B(shape=3x3) | v some_user(requires 2x2 input)
M“after”a nodeN`. For example, you might expect this to work:shared_ptr<Node> M = make_shared<SomeUnaryOp>(N); replace_node(M, N);
The problem is that at replacement time, N itself is a user of M. So we end up introducing a cycle as follows:
other users…N | v
other users…||| vvv N------------>M | v
To avoid the cycle, a valid way to perform the above desired insertion would be,||| vvv .----. | | | | N `----->M | v other users...auto new_N = N->clone_with_new_inputs(N->input_values()); shared_ptr<Node> M = make_shared<SomeUnaryOp>(new_N); replace_node(N, M);
- OPENVINO_API void replace_node (const std::shared_ptr< Node > &target, const OutputVector &replacement_values)
Replace target.outputs[i] with replacement_values[i] and transfer control dependents and.
- OPENVINO_API void replace_node (const std::shared_ptr< Node > &target, const std::shared_ptr< Node > &replacement)
- OPENVINO_API void replace_nodes (const std::shared_ptr< Model > &f, const std::unordered_map< std::shared_ptr< op::v0::Parameter >, std::shared_ptr< op::v0::Parameter > > ¶meter_replacement_map, const std::unordered_map< std::shared_ptr< Node >, std::shared_ptr< Node > > &body_replacement_map)
Replace multiple nodes in a function.
Limitations:
No check is made that the replaced nodes in
parameter_replacement_mapare actually among the bound parameters off. (If a parameter appears in the map that is not bound byf, it will be silently ignored.)If a parameter node appears as a key in both
parameter_replacement_mapand inbody_replacement_map, behavior is unspecified.
- Parameters
f – Model where replacement is taking place.
parameter_replacement_map – A mapping from parameter shared pointers to parameter shared pointers. For each pair (k,v) in the map, parameter k is replaced by parameter v, except if k==v or k is not a parameter bound by f, in which case the pair (k,v) is ignored.
body_replacement_map – A mapping from node shared pointers to node shared pointers. For each pair (k,v) in the map, node k is replaced by node v, except if k==v, the pair (k,v) is ignored. Note that if k is a parameter, its users will be redirected to v, but k will not be replaced in the function’s parameter list.
Topological sort of nodes needed to compute root_nodes.
- OPENVINO_API std::shared_ptr< ov::Model > clone_model (const ov::Model &model, std::unordered_map< Node *, std::shared_ptr< Node > > &node_map)
- OPENVINO_API std::shared_ptr< ov::Model > clone_model (const ov::Model &model)
input model is cloned and returned
- OPENVINO_API bool compare_constants (const std::shared_ptr< Node > &n1, const std::shared_ptr< Node > &n2)
- OPENVINO_API bool replace_output_update_name (Output< Node > node, const Output< Node > &node_input)
- OPENVINO_API bool replace_node_update_name (const std::shared_ptr< Node > &target, const std::shared_ptr< Node > &replacement)
- OPENVINO_API void serialize (const std::shared_ptr< const ov::Model > &m, const std::string &xml_path, const std::string &bin_path="", ov::pass::Serialize::Version version=ov::pass::Serialize::Version::UNSPECIFIED)
Serialize given model into IR. The generated .xml and .bin files will be saved into provided paths. This method serializes model “as-is” that means no weights compression and other possible transformations are applied. It is recommended to use ov::save_model function instead of ov::serialize, because it is aligned with default model conversion flow.
- OPENVINO_API void save_model (const std::shared_ptr< const ov::Model > &model, const std::string &output_model, bool compress_to_fp16=true)
Save given model into IR. Floating point weights are compressed to FP16 by default. This method saves a model to IR applying all necessary transformations that usually applied in model conversion flow provided by mo tool. Paricularly, floatting point weights are compressed to FP16.
- Parameters
model – Model which will be converted to IR representation.
output_model – Path to the output model file, must have extension .xml
compress_to_fp16 – Whether to compress floatting point weights to FP16 (true by default)
- OPENVINO_API std::ostream & operator<< (std::ostream &str, const Interval &interval)
- OPENVINO_API std::ostream & operator<< (std::ostream &, const Model &)
- OPENVINO_API ov::Dimension get_batch (const std::shared_ptr< const ov::Model > &f)
Helper method to get associated batch size for a Model.
Checks layout of each parameter in a Model and extracts value for N (B) dimension. All values are then merged and returned
- Throws
ov::AssertFailure – with details in case of error. Possible errors are:
There is no parameter with layout set. Model shall have at least one parameter with layout with ‘N’ dimension. Recommended fix is to use
Parameter::set_layoutAPI, e.g.model->get_parameters()[some_index]->set_layout("NCHW");Several parameters have conflicting N dimension, e.g. param1 NCHW{1,3,224,224} and param2 NCHW{2,3,224,224}. This is ambiguous, most probably first dimension is incorrectly marked as ‘batch’ (N) in some layout. User shall fix it before using of ‘get_batch’ (in example above correct layout for param2 from ‘NCHW’ to ‘CHWN’)
- Parameters
f – Model where to look for a batch_size value
- Returns
Dimension representing current batch size. Can represent a number or be a dynamic
- OPENVINO_API void set_batch (const std::shared_ptr< ov::Model > &model, ov::Dimension batch_size)
Helper method to set batch size to a Model.
Checks layout of each parameter in a Model and sets value for N (B) dimension. Then performs validation and type propagation
- Throws
ov::AssertFailure – with details in case of error. Possible errors are:
There is no parameter with N dimension in layout. Model shall have at least one parameter with layout with ‘N’ dimension. Recommended fix is to use
Parameter::set_layoutAPI, e.g.model->get_parameters()[some_index]->set_layout("NCHW");Several parameters have conflicting N dimension, e.g. param1 NCHW{1,3,224,224} and param2 NCHW{3,224,224,1}. This is ambiguous (1 != 3), most probably some dimension is incorrectly marked as ‘batch’ (N) in some layout. User shall fix it before using of ‘set_batch’ (in example above correct layout for param2 from ‘NCHW’ to ‘CHWN’)
Validation fails after setting batch_size. Model becomes in inconsistent state after new batch size value is applied. Possible reason could be that layout was not set for some parameters, or batch size can’t be applied to model at all
- Parameters
model – model where to set batch_size value
batch_size – Batch size value. For dynamic batch size, Dimension::dynamic() can be passed.
- OPENVINO_API std::string node_validation_failure_loc_string (const Node *node)
- OPENVINO_API std::ostream & operator<< (std::ostream &, const Node &)
- OPENVINO_API std::ostream & operator<< (std::ostream &, const Node *)
- void OPENVINO_API check_new_args_count (const Node *const node, const OutputVector &new_args)
Check new arguments size if match node inputs count.
This check is required in cloning ov::Node.
- Parameters
node – Pointer to node.
new_args – Vector with new outputs to check.
- OPENVINO_API std::ostream & operator<< (std::ostream &out, const Input< Node > &input)
- OPENVINO_API std::ostream & operator<< (std::ostream &out, const Input< const Node > &input)
- OPENVINO_API std::ostream & operator<< (std::ostream &out, const Output< Node > &output)
- OPENVINO_API std::ostream & operator<< (std::ostream &out, const Output< const Node > &output)
- OPENVINO_API OutputVector as_output_vector (const NodeVector &args)
- OPENVINO_API NodeVector as_node_vector (const OutputVector &values)
- OPENVINO_API ResultVector as_result_vector (const OutputVector &values)
Returns a ResultVector referencing values.
- OPENVINO_API PartialShape operator+ (const PartialShape &s1, const PartialShape &s2)
Elementwise addition of two PartialShape objects.
If
s1ors2has dynamic rank, returns PartialShape::dynamic().If
s1 ands2` both have static rank, and their ranks are unequal, throws std::invalid_argument.If
s1ands2both have static rank, and their ranks are equal, returns a new shape whoseith dimension iss1[i] + s2[i].
- Parameters
s1 – Left operand for addition.
s2 – Right operand for addition.
- Throws
std::invalid_argument – If
s1ands2have inconsistent ranks.- Returns
The result of elementwise adding
s1tos2(see description).
- OPENVINO_API std::ostream & operator<< (std::ostream &str, const PartialShape &shape)
Inserts a human-readable representation of a PartialShape into an output stream.
The output to the stream is in “informal” notation. In other words:
If
shapehas dynamic rank, inserts the string?.If
shapehas static rank, inserts the string{, then inserts each dimension ofshapeinto the output stream separated by commas, then inserts}.
PartialShape s1{PartialShape::dynamic())}; PartialShape s2{}; PartialShape s3{1,Dimension::dynamic(),2,3}; PartialShape s4{2,3,4}; std::cout << s1 << std::endl << s2 << std::endl << s3 << std::endl << s4 << std::endl;
? {} {1,?,2,3} {2,3,4}- Parameters
str – The output stream targeted for insertion.
shape – The shape to be inserted into
str.
- Returns
A reference to
strafter insertion.
- OPENVINO_API void copy_runtime_info (const std::shared_ptr< ov::Node > &from, const std::shared_ptr< ov::Node > &to)
- OPENVINO_API void copy_runtime_info (const std::shared_ptr< ov::Node > &from, ov::NodeVector to)
- OPENVINO_API void copy_runtime_info (const ov::NodeVector &from, const std::shared_ptr< ov::Node > &to)
- OPENVINO_API void copy_runtime_info (const ov::NodeVector &from, ov::NodeVector to)
- OPENVINO_API void copy_output_runtime_info (const ov::OutputVector &from, ov::OutputVector to)
- OPENVINO_API std::ostream & operator<< (std::ostream &os, const RuntimeAttribute &attrubute)
-
template<typename ForwardIt>
size_t shape_size(ForwardIt start_dim, const ForwardIt end_dim) Number of elements in a subset of dimensions of a shape. Returns a product of dimensions in a range [start_dim;end_dim)
-
template<typename SHAPE_TYPE>
size_t shape_size(const SHAPE_TYPE &shape) Number of elements in spanned by a shape.
-
template<typename SHAPE_TYPE>
std::vector<size_t> row_major_strides(const SHAPE_TYPE &shape) Row-major strides for a shape.
-
template<typename SHAPE_TYPE>
size_t row_major_stride(const SHAPE_TYPE &shape, size_t axis)
-
template<typename SHAPE_TYPE>
inline bool is_scalar(const SHAPE_TYPE &shape)
-
template<typename SHAPE_TYPE>
inline bool is_vector(const SHAPE_TYPE &shape)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const Shape &shape)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const Strides &strides)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const DiscreteTypeInfo &info)
- bool ::type is_type (Value value)
- Type *::type as_type (Value value)
-
template<typename T, typename U>
auto as_type_ptr(const U &value) -> decltype(::ov::util::AsTypePtr<U>::template call<T>(value)) Casts a std::shared_ptr<Value> to a std::shared_ptr<Type> if it is of type Type, nullptr otherwise
- OPENVINO_API PartialShape infer_convolution_forward (const Node *node, const PartialShape &data_batch_shape, const Strides &data_dilation, const CoordinateDiff &data_padding_below, const CoordinateDiff &data_padding_above, const PartialShape &filters_shape, const Strides &filter_strides, const Strides &filter_dilation)
- OPENVINO_API void infer_auto_padding (const Shape &image_shape, const Shape &filter_shape, const Strides &filter_strides, const Strides &filter_dilations, const op::PadType pad_type, CoordinateDiff &padding_above, CoordinateDiff &padding_below)
- OPENVINO_API int64_t normalize_axis (const Node *node, std::int64_t axis, const Rank &tensor_rank)
Handle out of range axis.
- Parameters
node – [in] The node with requested axis.
axis – [in] The requested axis value.
tensor_rank – [in] The corresponding tensor rank.
- Returns
Checking if axis is in range [-tensor_rank, tensor_rank-1], otherwise returns error. If negative axis, it counts from the last to the first axis, by adding tensor_rank to axis.
- OPENVINO_API std::vector< size_t > normalize_axes (const std::string &node_description, const std::vector< int64_t > &axes, const Rank &tensor_rank)
Handle out of range axes in vector.
- Parameters
node_description – [in] The name of node with requested axes.
axes – [in] The requested vector of axes.
tensor_rank – [in] The corresponding tensor rank.
- Returns
If any negative axis in vector, it counts from the last to the first axis, by adding tensor_rank to axis.
- OPENVINO_API int64_t normalize_axis (const std::string &node_description, std::int64_t axis, const Rank &tensor_rank)
Handle out of range axis.
- Parameters
node_description – [in] The node with requested axis.
axis – [in] The requested axis value.
tensor_rank – [in] The corresponding tensor rank.
- Returns
Checking if axis is in range [-tensor_rank, tensor_rank-1], otherwise returns error. If negative axis, it counts from the last to the first axis, by adding tensor_rank to axis.
- OPENVINO_API int64_t normalize_axis (const Node *node, std::int64_t axis, std::uint64_t tensor_rank, std::int64_t axis_range_min, std::int64_t axis_range_max)
Handle out of range axis.
- Parameters
node – [in] The node with requested axis.
axis – [in] The requested axis value.
tensor_rank – [in] The corresponding tensor rank.
axis_range_min – [in] The min value of accepted range for axis.
axis_range_max – [in] The max value of accepted range for axis.
- Returns
Checking if axis is in range [axis_range_min, axis_range_max], otherwise returns error. If negative axis, it counts from the last to the first axis, by adding tensor_rank to axis.
- OPENVINO_API int64_t normalize_axis (const std::string &node_description, std::int64_t axis, std::uint64_t tensor_rank, std::int64_t axis_range_min, std::int64_t axis_range_max)
Handle out of range axis.
- Parameters
node_description – [in] The name of node with requested axis.
axis – [in] The requested axis value.
tensor_rank – [in] The corresponding tensor rank.
axis_range_min – [in] The min value of accepted range for axis.
axis_range_max – [in] The max value of accepted range for axis.
- Returns
Checking if axis is in range [axis_range_min, axis_range_max], otherwise returns error. If negative axis, it counts from the last to the first axis, by adding tensor_rank to axis.
- OPENVINO_API void normalize_axes (const Node *node, const int64_t &tensor_rank, std::vector< int64_t > &axes)
Handle out of range axes in vector. If any negative axis in vector, it counts from the last to the first axis, by adding tensor_rank to axis. Changes axes vector inplace.
- Parameters
node – [in] The node with requested axes.
tensor_rank – [in] The corresponding tensor rank.
axes – [inout] The requested vector of axes.
- OPENVINO_API bool evaluate_as_partial_shape (const Output< Node > &output, PartialShape &pshape)
Evaluates lower and upper value estimations for the output tensor. Estimation would be represented as partial shape object using Dimension(min, max) for each element.
- Parameters
output – Node output pointing to the tensor for estimation.
pshape – Resulting estimation would be stored in this PartialShape.
- Returns
boolean status if value evaluation was successful.
- OPENVINO_API std::shared_ptr< op::v0::Constant > get_constant_from_source (const Output< Node > &source)
Runs an estimation of source tensor. If it succeeded to calculate both bounds and they are the same returns Constant operation from the resulting bound, otherwise nullptr.
- OPENVINO_API bool default_label_evaluator (const Node *node, TensorLabelVector &output_labels)
Propagates value label from 0 input to the only output through an operation. Not applicable for operations which require values interaction (example: mathematical operations). Could be used for movement operations (example: gathering, shape change)
- Parameters
node – Operation to be performed
output_labels – Vector of TensorLabel objects representing resulting value labels
- Returns
boolean status if label evaluation was successful.
- OPENVINO_API void generate_transpose_default_order (std::vector< int64_t > &axes_order, const size_t length)
Generates transpose default axes order at end of input vector.
Default axes order is decreasing sequence numbers which start from
length - 1.- Parameters
axes_order – Vector where default order will be generated.
length – Sequence length of axes order.
- OPENVINO_API bool is_valid_axes_order (const std::vector< int64_t > &axes_order, const size_t size)
Check if vector of axes order has got valid values.
Axes order has to be unique numbers in range of [0, size).
- Parameters
axes_order – Vector with axes order to check.
size – Input for transpose rank size.
- Returns
true if axes order is valid otherwise false.
- OPENVINO_API bool has_no_labels (const TensorLabel &labels)
Checks label tensor if there is no label.
- Parameters
labels – Label tensor for check.
- Returns
True if there is no labels, otherwise false.
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const Version &version)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const std::map< std::string, Version > &versions)
- OPENVINO_API_C (const Version) get_openvino_version() noexcept
Gets the current OpenVINO version.
- Returns
The current OpenVINO version
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v1::BinaryConvolution::BinaryConvolutionMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v0::DepthToSpace::DepthToSpaceMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v9::GridSample::InterpolationMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v9::GridSample::PaddingMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v0::Interpolate::InterpolateMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::LSTMWeightsFormat &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v8::MatrixNms::DecayFunction &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v8::MatrixNms::SortResultType &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v1::NonMaxSuppression::BoxEncodingType &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v3::NonMaxSuppression::BoxEncodingType &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v5::NonMaxSuppression::BoxEncodingType &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v9::NonMaxSuppression::BoxEncodingType &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v1::Reverse::Mode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v5::Round::RoundMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::v0::SpaceToDepth::SpaceToDepthMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::util::InterpolateBase::InterpolateMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::util::InterpolateBase::CoordinateTransformMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::util::InterpolateBase::NearestMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::util::InterpolateBase::ShapeCalcMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const op::util::MulticlassNmsBase::SortResultType &type)
- void OPENVINO_API mark_as_precision_sensitive (ov::Input< ov::Node > node_input)
- void OPENVINO_API unmark_as_precision_sensitive (ov::Input< ov::Node > node_input)
- bool OPENVINO_API is_precision_sensitive (const ov::Input< ov::Node > &node_input)
- OPENVINO_API void set_up_symbolic_info (const std::shared_ptr< ov::Model > &model, const std::shared_ptr< ov::TableOfEquivalence > &table)
- OPENVINO_API void set_up_symbolic_info (const ov::Output< ov::Node > &output, const std::shared_ptr< ov::TableOfEquivalence > &table)
- OPENVINO_API void populate_tensor_with_missing_labels (ov::descriptor::Tensor &tensor)
- OPENVINO_API bool skip_invalidation (const ov::descriptor::Tensor &tensor)
- OPENVINO_API std::shared_ptr< ov::TableOfEquivalence > table_of_equivalence (const std::shared_ptr< ov::Model > &model)
- OPENVINO_API std::shared_ptr< ov::TableOfEquivalence > table_of_equivalence (const ov::descriptor::Tensor &tensor)
- OPENVINO_API void remove_symbolic_info (const std::shared_ptr< ov::Model > &model, bool outermost_model=true)
- const OPENVINO_API OpSet & get_opset1 ()
Returns opset1.
- const OPENVINO_API OpSet & get_opset2 ()
Returns opset2.
- const OPENVINO_API OpSet & get_opset3 ()
Returns opset3.
- const OPENVINO_API OpSet & get_opset4 ()
Returns opset4.
- const OPENVINO_API OpSet & get_opset5 ()
Returns opset5.
- const OPENVINO_API OpSet & get_opset6 ()
Returns opset6.
- const OPENVINO_API OpSet & get_opset7 ()
Returns opset7.
- const OPENVINO_API OpSet & get_opset8 ()
Returns opset8.
- const OPENVINO_API OpSet & get_opset9 ()
Returns opset9.
- const OPENVINO_API OpSet & get_opset10 ()
Returns opset10.
- const OPENVINO_API OpSet & get_opset11 ()
Returns opset11.
- const OPENVINO_API OpSet & get_opset12 ()
Returns opset12.
- const OPENVINO_API OpSet & get_opset13 ()
Returns opset13.
- const OPENVINO_API std::map< std::string, std::function< const ov::OpSet &()> > & get_available_opsets ()
Returns map of available opsets.
-
std::size_t coordinate_index(const Coordinate &c, const Shape &s)
-
size_t coordinate_offset(const std::vector<size_t> &coordinate, const std::vector<size_t> &strides)
Calculate offset from begin of buffer based on coordinate and strides.
If coordinates and strides have different sizes then result is undefined behaviour.
- Parameters
coordinate – Vector with multi-dimension coordinates.
strides – Vector with multi-dimension strides
- Returns
Offset of element from start of buffer.
-
template<class T>
constexpr bool is_floating_point() Check if T is OpenVINO floating point precision.
- Returns
True if OpenVino floating point precision.
-
OV_ITT_DOMAIN(OV_PP_CAT(TYPE_LIST_, ov_eval))
-
template<class TContainer>
constexpr auto make_tensor_accessor(const TContainer &c) -> TensorAccessor<TContainer> Makes TensorAccessor for specific tensor container.
See also
TensorAccessor for supported types.
- Template Parameters
TContainer – Type of tensor containers
- Parameters
c – Container of tensors.
- Returns
TensorContainer for specific type.
-
auto make_tensor_accessor() -> const TensorAccessor<void>&
Makes empty TensorAccessor which return empty tensor for any port number.
- Returns
TensorAccessor to return empty tensor.
-
template<class T, class TResult = std::vector<T>, class UnaryOperation>
TResult get_raw_data_as(const element::Type_t et, const void *const ptr, const size_t size, UnaryOperation &&func) Get the raw data as TResult object.
- Template Parameters
T – TResult data type.
TResult – Type of return object, must support creation of std::inserter. Default std::vector<T>.
UnaryOperation – Unary function object applied on data with signature (T f(const U u)).
- Parameters
et – Element type of input data.
ptr – Pointer to data of type et.
size – Data size as number of elements.
func – Unary operation function object.
- Throws
ov::AssertionFailure – for not supported element type.
- Returns
Object of TResult with data from input pointer and transformed by unary operation.
-
template<class T, class TResult = std::vector<T>, class UnaryOperation = ov::util::Cast<T>>
TResult get_tensor_data_as(const Tensor &t, UnaryOperation &&func = ov::util::Cast<T>()) Get data from ov:tensor as object TResult.
- Template Parameters
T – TResult data type.
TResult – Type of return object, must support creation of std::inserter. Default std::vector<T>.
UnaryOperation – Unary function object applied on data with signature (T f(const U u)).
- Parameters
t – Input tensor.
func – Unary operation function object.
- Returns
Object of TResult with data from tensor.
- FRONTEND_API void shutdown ()
Shut down the OpenVINO by deleting all static-duration objects allocated by the library and releasing dependent resources.
You might want to use this function if you are developing a dynamically-loaded library which should clean up all resources after itself when the library is unloaded.
Note
This function should be used by advanced user to control unload the resources.
Returns set of nodes from original model which are determined as supported after applied transformation pipeline.
- Parameters
model – Original model
transform – Transformation pipeline function
is_node_supported – Function returning whether node is supported or not
- Returns
Set of strings which contains supported node names
-
std::shared_ptr<ITensor> make_tensor(const element::Type type, const Shape &shape, const Allocator &allocator = {})
Constructs Tensor using element type and shape. Allocate internal host storage using default allocator.
-
std::shared_ptr<ITensor> make_tensor(const element::Type type, const Shape &shape, void *host_ptr, const Strides &strides = {})
Constructs Tensor using element type and shape. Wraps allocated host memory.
Note
Does not perform memory allocation internally
Constructs region of interest (ROI) tensor form another tensor.
Note
Does not perform memory allocation internally
Note
A Number of dimensions in
beginandendmust match number of dimensions inother.get_shape()- Parameters
other – original tensor
begin – start coordinate of ROI object inside of the original object.
end – end coordinate of ROI object inside of the original object.
-
ov::Tensor make_tensor(const ov::SoPtr<ITensor> &tensor)
Constructs public ov::Tensor class.
-
bool check_open_mp_env_vars(bool include_omp_num_threads = true)
Checks whether OpenMP environment variables are defined.
- Parameters
include_omp_num_threads – [in] Indicates if the omp number threads is included
- Returns
Trueif any OpenMP environment variable is defined,falseotherwise
-
std::vector<int> get_available_numa_nodes()
Returns available CPU NUMA nodes (on Linux, and Windows [only with TBB], single node is assumed on all other OSes)
- Returns
NUMA nodes
-
std::vector<int> get_available_cores_types()
Returns available CPU cores types (on Linux, and Windows) and ONLY with TBB, single core type is assumed otherwise.
- Returns
Vector of core types
-
int get_number_of_cpu_cores(bool big_cores_only = false)
Returns number of CPU physical cores on Linux/Windows (which is considered to be more performance friendly for servers) (on other OSes it simply relies on the original parallel API of choice, which usually uses the logical cores). call function with ‘false’ to get #phys cores of all types call function with ‘true’ to get #phys ‘Big’ cores number of ‘Little’ = ‘all’ - ‘Big’.
- Parameters
big_cores_only – [in] Additionally limits the number of reported cores to the ‘Big’ cores only.
- Returns
Number of physical CPU cores.
-
int get_number_of_logical_cpu_cores(bool big_cores_only = false)
Returns number of CPU logical cores on Linux/Windows (on other OSes it simply relies on the original parallel API of choice, which uses the ‘all’ logical cores). call function with ‘false’ to get #logical cores of all types call function with ‘true’ to get #logical ‘Big’ cores number of ‘Little’ = ‘all’ - ‘Big’.
- Parameters
big_cores_only – [in] Additionally limits the number of reported cores to the ‘Big’ cores only.
- Returns
Number of logical CPU cores.
-
int get_number_of_blocked_cores()
Returns number of blocked CPU cores. Please note that this is a temporary interface for performance optimization on a specific platform. May be removed in future release.
- Returns
Number of blocked CPU cores.
-
bool with_cpu_x86_sse42()
Checks whether CPU supports SSE 4.2 capability.
- Returns
Trueis SSE 4.2 instructions are available,falseotherwise
-
bool with_cpu_x86_avx()
Checks whether CPU supports AVX capability.
- Returns
Trueis AVX instructions are available,falseotherwise
-
bool with_cpu_x86_avx2()
Checks whether CPU supports AVX2 capability.
- Returns
Trueis AVX2 instructions are available,falseotherwise
-
bool with_cpu_x86_avx2_vnni()
Checks whether CPU supports AVX2_VNNI capability.
- Returns
Trueis AVX2_VNNI instructions are available,falseotherwise
-
bool with_cpu_x86_avx512f()
Checks whether CPU supports AVX 512 capability.
- Returns
Trueis AVX512F (foundation) instructions are available,falseotherwise
-
bool with_cpu_x86_avx512_core()
Checks whether CPU supports AVX 512 capability.
- Returns
Trueis AVX512F, AVX512BW, AVX512DQ instructions are available,falseotherwise
-
bool with_cpu_x86_avx512_core_vnni()
Checks whether CPU supports AVX 512 VNNI capability.
- Returns
Trueis AVX512F, AVX512BW, AVX512DQ, AVX512_VNNI instructions are available,falseotherwise
-
bool with_cpu_x86_bfloat16()
Checks whether CPU supports BFloat16 capability.
- Returns
Trueis tAVX512_BF16 instructions are available,falseotherwise
-
bool with_cpu_x86_avx512_core_fp16()
Checks whether CPU supports fp16 capability.
- Returns
Trueis tAVX512_FP16 instructions are available,falseotherwise
-
bool with_cpu_x86_avx512_core_amx_int8()
Checks whether CPU supports AMX int8 capability.
- Returns
Trueis tAMX_INT8 instructions are available,falseotherwise
-
bool with_cpu_x86_avx512_core_amx_bf16()
Checks whether CPU supports AMX bf16 capability.
- Returns
Trueis tAMX_BF16 instructions are available,falseotherwise
-
bool with_cpu_x86_avx512_core_amx()
Checks whether CPU supports AMX capability.
- Returns
Trueis tAMX_INT8 or tAMX_BF16 instructions are available,falseotherwise
-
bool is_cpu_map_available()
Checks whether cpu_mapping Available.
- Returns
Trueis CPU mapping is available,falseotherwise
-
int get_num_numa_nodes()
Get number of numa nodes.
- Returns
Number of numa nodes
-
int get_num_sockets()
Get number of sockets.
- Returns
Number of sockets
-
std::vector<std::vector<int>> get_proc_type_table()
Returns a table of number of processor types on Linux/Windows.
Processor table of one socket CPU desktop ALL_PROC | MAIN_CORE_PROC | EFFICIENT_CORE_PROC | HYPER_THREADING_PROC 32 8 16 8 // Total number of one socket
- Returns
A table about number of CPU cores of different types defined with ColumnOfProcessorTypeTable The following are two example of processor type table.
Processor table of two socket CPUs XEON server ALL_PROC | MAIN_CORE_PROC | EFFICIENT_CORE_PROC | HYPER_THREADING_PROC 96 48 0 48 // Total number of two sockets 48 24 0 24 // Number of socket one 48 24 0 24 // Number of socket two
-
std::vector<std::vector<int>> get_org_proc_type_table()
Returns a table of original number of processor types without filtering other plugins occupying CPU resources. The difference from get_proc_type_table: This is used to get the configuration of current machine. For example, GPU plugin occupies all Pcores, there is only one type core in proc_type_table from get_proc_type_table(). If user wants to get the real configuration of this machine which should be got from get_org_proc_type_table.
- Returns
A table about number of CPU cores of different types defined with ColumnOfProcessorTypeTable
-
void reserve_available_cpus(const std::vector<std::vector<int>> streams_info_table, std::vector<std::vector<int>> &stream_processors, const int cpu_status = NOT_USED)
Get and reserve available cpu ids.
- Parameters
streams_info_table – [in] streams information table.
stream_processors – [in] processors grouped in stream which is used in core binding in cpu streams executor
cpu_status – [in] set cpu status
-
void set_cpu_used(const std::vector<int> &cpu_ids, const int used)
Set CPU_MAP_USED_FLAG of cpu_mapping.
- Parameters
cpu_ids – [in] cpus in cpu_mapping.
used – [in] update CPU_MAP_USED_FLAG of cpu_mapping with this flag bit
-
int get_socket_by_numa_node(int numa_node_id)
Get socket id by current numa node id.
- Parameters
numa_node_id – [in] numa node id
- Returns
socket id
-
int get_org_socket_id(int socket_id)
Get original socket id by current socket id, the input socket id is recalculated after filtering (like numactl), while the original socket id is the original id before filtering.
- Parameters
socket_id – [in] socket id
- Returns
socket id
-
int get_org_numa_id(int numa_node_id)
Get original numa node id by current numa node id, the input numa node id is recalculated after filtering (like numactl), while the original numa node id is the original id before filtering.
- Parameters
numa_node_id – [in] numa node id
- Returns
numa node id
-
class Allocator
- #include <allocator.hpp>
Wraps allocator implementation to provide safe way to store allocater loaded from shared library And constructs default based on
newdeletec++ calls allocator if created without parameters Accepts any std::pmr::memory_resource like allocator.Public Functions
-
~Allocator()
Destructor preserves unloading order of implementation object and reference to library.
-
Allocator()
Default constructor.
-
Allocator(const Allocator &other) = default
Default copy constructor.
- Parameters
other – other Allocator object
-
Allocator &operator=(const Allocator &other) = default
Default copy assignment operator.
- Parameters
other – other Allocator object
- Returns
reference to the current object
-
Allocator(Allocator &&other) = default
Default move constructor.
- Parameters
other – other Allocator object
-
Allocator &operator=(Allocator &&other) = default
Default move assignment operator.
- Parameters
other – other Allocator object
- Returns
reference to the current object
-
OPENVINO_SUPPRESS_DEPRECATED_START Allocator(const AllocatorImpl::Ptr &impl)
Constructs Allocator from the initialized std::shared_ptr.
- Parameters
impl – Initialized shared pointer
Initialize allocator using any allocator like object.
- Template Parameters
A – Type of allocator
- Parameters
a – allocator object
- OPENVINO_SUPPRESS_DEPRECATED_END void * allocate (const size_t bytes, const size_t alignment=alignof(max_align_t))
Allocates memory.
- Parameters
bytes – The size in bytes at least to allocate
alignment – The alignment of storage
- Throws
Exception – if specified size and alignment is not supported
- Returns
Handle to the allocated resource
-
void deallocate(void *ptr, const size_t bytes = 0, const size_t alignment = alignof(max_align_t))
Releases the handle and all associated memory resources which invalidates the handle.
- Parameters
ptr – The handle to free
bytes – The size in bytes that was passed into allocate() method
alignment – The alignment of storage that was passed into allocate() method
-
bool operator==(const Allocator &other) const
Compares with other Allocator.
- Parameters
other – Other instance of allocator
- Returns
trueif and only if memory allocated from one Allocator can be deallocated from the other and vice versa
-
~Allocator()
-
interface AllocatorImpl : public std::enable_shared_from_this<AllocatorImpl>
- #include <allocator.hpp>
Tries to act like std::pmr::memory_resource
- Deprecated:
This class will be removed in 2024.0 release
Public Types
-
using Ptr = std::shared_ptr<AllocatorImpl>
A smart pointer containing AllocatorImpl object.
Public Functions
-
virtual void *allocate(const size_t bytes, const size_t alignment = alignof(max_align_t)) = 0
Allocates memory.
- Parameters
bytes – The size in bytes at least to allocate
alignment – The alignment of storage
- Throws
Exception – if specified size and alignment is not supported
- Returns
Handle to the allocated resource
-
virtual void deallocate(void *handle, const size_t bytes, size_t alignment = alignof(max_align_t)) = 0
Releases the handle and all associated memory resources which invalidates the handle.
- Parameters
handle – The handle to free
bytes – The size in bytes that was passed into allocate() method
alignment – The alignment of storage that was passed into allocate() method
-
virtual bool is_equal(const AllocatorImpl &other) const = 0
Compares with other AllocatorImpl.
- Parameters
other – Other instance of allocator
- Returns
trueif and only if memory allocated from one AllocatorImpl can be deallocated from the other and vice versa
-
class Any
- #include <any.hpp>
This class represents an object to work with different types.
Public Functions
-
Any() = default
Default constructor.
-
Any &operator=(const Any &other)
Сopy assignment operator.
- Parameters
other – other Any object
- Returns
reference to the current object
-
Any &operator=(Any &&other) = default
Default move assignment operator.
- Parameters
other – other Any object
- Returns
reference to the current object
-
~Any()
Destructor preserves unloading order of implementation object and reference to library.
-
template<typename T, typename std::enable_if<!std::is_same<decay_t<T>, Any>::value && !std::is_abstract<decay_t<T>>::value && !std::is_convertible<decay_t<T>, Base::Ptr>::value, bool>::type = true>
inline Any(T &&value) Constructor creates any with object.
- Template Parameters
T – Any type
- Parameters
value – object
-
Any(const char *str)
Constructor creates string any from char *.
- Parameters
str – char array
-
Any(const std::nullptr_t)
Empty constructor.
-
const std::type_info &type_info() const
Returns type info
- Returns
type info
-
bool empty() const
Checks that any contains a value
- Returns
false if any contains a value else false
-
template<class T>
inline bool is() const Check that stored type can be casted to specified type. If internal type supports Base.
- Template Parameters
T – Type of value
- Returns
true if type of value is correct. Return false if any is empty
Dynamic cast to specified type
- Template Parameters
T – type
- Returns
casted object
Dynamic cast to specified type
- Template Parameters
T – type
- Returns
casted object
Dynamic cast to specified type
- Template Parameters
T – type
- Returns
casted object
-
template<class T>
inline std::enable_if<std::is_same<T, std::string>::value, T>::type &as() Dynamic cast to specified type
- Template Parameters
T – type
- Returns
casted object
-
template<class T>
inline const T &as() const Dynamic cast to specified type
- Template Parameters
T – type
- Returns
const reference to caster object
-
bool operator==(const Any &other) const
The comparison operator for the Any.
- Parameters
other – object to compare
- Returns
true if objects are equal
-
bool operator==(const std::nullptr_t&) const
The comparison operator for the Any.
- Parameters
other – object to compare
- Returns
true if objects are equal
-
bool operator!=(const Any &other) const
The comparison operator for the Any.
- Parameters
other – object to compare
- Returns
true if objects aren’t equal
-
void print(std::ostream &stream) const
Prints underlying object to the given output stream. Uses operator<< if it is defined, leaves stream unchanged otherwise. In case of empty any or nullptr stream immediately returns.
- Parameters
stream – Output stream object will be printed to.
-
void read(std::istream &stream)
Read into underlying object from the given input stream. Uses operator>> if it is defined, leaves stream unchanged otherwise. In case of empty any or nullptr stream immediately returns.
- Parameters
stream – Output stream object will be printed to.
-
void *addressof()
Returns address to internal value if any is not empty and
nullptrinstead.- Returns
address to internal stored value
-
const void *addressof() const
Returns address to internal value if any is not empty and
nullptrinstead.- Returns
address to internal stored value
-
Any() = default
-
class AssertFailure : public ov::Exception
- #include <except.hpp>
Base class for check failure exceptions.
Subclassed by ov::NodeValidationFailure, ov::NotImplemented, ov::frontend::GeneralFailure, ov::frontend::InitializationFailure, ov::frontend::NotImplementedFailure, ov::frontend::OpConversionFailure, ov::frontend::OpValidationFailure
-
template<typename AT>
class AttributeAdapter - #include <attribute_adapter.hpp>
An AttributeAdapter “captures” an attribute as an AT& and makes it available as a ValueAccessor<VAT>.
-
template<>
class AttributeAdapter<AxisVector> : public ov::IndirectVectorValueAccessor<AxisVector, std::vector<int64_t>> - #include <axis_vector.hpp>
-
template<>
class AttributeAdapter<bool> : public ov::DirectValueAccessor<bool> - #include <attribute_adapter.hpp>
Access a bool as a bool.
-
template<>
class AttributeAdapter<Coordinate> : public ov::IndirectVectorValueAccessor<Coordinate, std::vector<int64_t>> - #include <coordinate.hpp>
-
template<>
class AttributeAdapter<CoordinateDiff> : public ov::IndirectVectorValueAccessor<CoordinateDiff, std::vector<int64_t>> - #include <coordinate_diff.hpp>
-
template<>
class AttributeAdapter<double> : public ov::DirectValueAccessor<double> - #include <attribute_adapter.hpp>
Access a double as a double.
-
template<>
class AttributeAdapter<float> : public ov::IndirectScalarValueAccessor<float, double> - #include <attribute_adapter.hpp>
-
template<>
class AttributeAdapter<int16_t> : public ov::IndirectScalarValueAccessor<int16_t, int64_t> - #include <attribute_adapter.hpp>
Access an int16_t as an int64_t.
-
template<>
class AttributeAdapter<int32_t> : public ov::IndirectScalarValueAccessor<int32_t, int64_t> - #include <attribute_adapter.hpp>
Access an int32_t as an int64_t.
-
template<>
class AttributeAdapter<int64_t> : public ov::DirectValueAccessor<int64_t> - #include <attribute_adapter.hpp>
Access an int64_t as an int64_t.
-
template<>
class AttributeAdapter<int8_t> : public ov::IndirectScalarValueAccessor<int8_t, int64_t> - #include <attribute_adapter.hpp>
Access an int8_t and an int64_t.
-
template<>
class AttributeAdapter<Layout> : public ov::ValueAccessor<std::string> - #include <layout.hpp>
Public Functions
-
virtual const std::string &get() override
Returns the value.
-
virtual void set(const std::string &value) override
Sets the value.
-
virtual const std::string &get() override
- template<> Type > : public ov::EnumAttributeAdapterBase< ngraph::reduction::Type >
- #include <distributed.hpp>
- template<> AutoBroadcastSpec > : public ov::VisitorAdapter
- #include <attr_types.hpp>
- template<> AutoBroadcastType > : public ov::EnumAttributeAdapterBase< op::AutoBroadcastType >
- #include <attr_types.hpp>
- template<> BroadcastModeSpec > : public ov::VisitorAdapter
- #include <attr_types.hpp>
- template<> BroadcastType > : public ov::EnumAttributeAdapterBase< op::BroadcastType >
- #include <attr_types.hpp>
- template<> EpsMode > : public ov::EnumAttributeAdapterBase< op::EpsMode >
- #include <attr_types.hpp>
- template<> GeluApproximationMode > : public ov::EnumAttributeAdapterBase< op::GeluApproximationMode >
- #include <gelu.hpp>
- template<> LSTMWeightsFormat > : public ov::EnumAttributeAdapterBase< op::LSTMWeightsFormat >
- #include <lstm_cell.hpp>
- template<> MVNEpsMode > : public ov::EnumAttributeAdapterBase< op::MVNEpsMode >
- #include <mvn.hpp>
- template<> PadMode > : public ov::EnumAttributeAdapterBase< op::PadMode >
- #include <attr_types.hpp>
- template<> PadType > : public ov::EnumAttributeAdapterBase< op::PadType >
- #include <attr_types.hpp>
- template<> RecurrentSequenceDirection > : public ov::EnumAttributeAdapterBase< op::RecurrentSequenceDirection >
- #include <attr_types.hpp>
- template<> RoundingType > : public ov::EnumAttributeAdapterBase< op::RoundingType >
- #include <attr_types.hpp>
- template<> TopKMode > : public ov::EnumAttributeAdapterBase< op::TopKMode >
- #include <attr_types.hpp>
- template<> TopKSortType > : public ov::EnumAttributeAdapterBase< op::TopKSortType >
- #include <attr_types.hpp>
- template<> CoordinateTransformMode > : public ov::EnumAttributeAdapterBase< op::util::InterpolateBase::CoordinateTransformMode >
- #include <interpolate_base.hpp>
- template<> InterpolateMode > : public ov::EnumAttributeAdapterBase< op::util::InterpolateBase::InterpolateMode >
- #include <interpolate_base.hpp>
- template<> NearestMode > : public ov::EnumAttributeAdapterBase< op::util::InterpolateBase::NearestMode >
- #include <interpolate_base.hpp>
- template<> ShapeCalcMode > : public ov::EnumAttributeAdapterBase< op::util::InterpolateBase::ShapeCalcMode >
- #include <interpolate_base.hpp>
- template<> SortResultType > : public ov::EnumAttributeAdapterBase< op::util::MulticlassNmsBase::SortResultType >
- #include <multiclass_nms_base.hpp>
- template<> DepthToSpaceMode > : public ov::EnumAttributeAdapterBase< op::v0::DepthToSpace::DepthToSpaceMode >
- #include <depth_to_space.hpp>
- template<> InterpolateMode > : public ov::EnumAttributeAdapterBase< op::v0::Interpolate::InterpolateMode >
- #include <interpolate.hpp>
- template<> SpaceToDepthMode > : public ov::EnumAttributeAdapterBase< op::v0::SpaceToDepth::SpaceToDepthMode >
- #include <space_to_depth.hpp>
- template<> Reduction > : public ov::EnumAttributeAdapterBase< op::v12::ScatterElementsUpdate::Reduction >
- #include <scatter_elements_update.hpp>
- template<> BinaryConvolutionMode > : public ov::EnumAttributeAdapterBase< op::v1::BinaryConvolution::BinaryConvolutionMode >
- #include <binary_convolution.hpp>
- template<> BoxEncodingType > : public ov::EnumAttributeAdapterBase< op::v1::NonMaxSuppression::BoxEncodingType >
- #include <non_max_suppression.hpp>
- template<> Mode > : public ov::EnumAttributeAdapterBase< op::v1::Reverse::Mode >
- #include <reverse.hpp>
- template<> BoxEncodingType > : public ov::EnumAttributeAdapterBase< op::v3::NonMaxSuppression::BoxEncodingType >
- #include <non_max_suppression.hpp>
- template<> PoolingMode > : public ov::EnumAttributeAdapterBase< op::v3::ROIAlign::PoolingMode >
- #include <roi_align.hpp>
- template<> SpecialBodyPorts > : public ov::DirectValueAccessor< op::v5::Loop::SpecialBodyPorts >
- #include <loop.hpp>
- template<> BoxEncodingType > : public ov::EnumAttributeAdapterBase< op::v5::NonMaxSuppression::BoxEncodingType >
- #include <non_max_suppression.hpp>
- template<> RoundMode > : public ov::EnumAttributeAdapterBase< op::v5::Round::RoundMode >
- #include <round.hpp>
- template<> DecayFunction > : public ov::EnumAttributeAdapterBase< op::v8::MatrixNms::DecayFunction >
- #include <matrix_nms.hpp>
- template<> SortResultType > : public ov::EnumAttributeAdapterBase< op::v8::MatrixNms::SortResultType >
- #include <matrix_nms.hpp>
- template<> InterpolationMode > : public ov::EnumAttributeAdapterBase< op::v9::GridSample::InterpolationMode >
- #include <grid_sample.hpp>
- template<> PaddingMode > : public ov::EnumAttributeAdapterBase< op::v9::GridSample::PaddingMode >
- #include <grid_sample.hpp>
- template<> BoxEncodingType > : public ov::EnumAttributeAdapterBase< op::v9::NonMaxSuppression::BoxEncodingType >
- #include <non_max_suppression.hpp>
- template<> AlignedMode > : public ov::EnumAttributeAdapterBase< op::v9::ROIAlign::AlignedMode >
- #include <roi_align.hpp>
- template<> PoolingMode > : public ov::EnumAttributeAdapterBase< op::v9::ROIAlign::PoolingMode >
- #include <roi_align.hpp>
- template<> AxisSet > : public ov::ValueAccessor< std::vector< int64_t > >
- #include <axis_set.hpp>
Public Functions
-
virtual const std::vector<int64_t> &get() override
Returns the value.
-
virtual void set(const std::vector<int64_t> &value) override
Sets the value.
-
virtual const std::vector<int64_t> &get() override
- template<> Dimension > : public ov::DirectValueAccessor< ov::Dimension >
- #include <dimension.hpp>
- template<> Type > : public ov::ValueAccessor< std::string >
- #include <element_type.hpp>
Public Functions
-
virtual const std::string &get() override
Returns the value.
-
virtual void set(const std::string &value) override
Sets the value.
-
virtual const std::string &get() override
- template<> Type_t > : public ov::EnumAttributeAdapterBase< ov::element::Type_t >
- #include <element_type.hpp>
- template<> TypeVector > : public ov::DirectValueAccessor< ov::element::TypeVector >
- #include <element_type.hpp>
- template<> NodeVector > : public ov::VisitorAdapter
- #include <node.hpp>
- template<> FrameworkNodeAttrs > : public ov::DirectValueAccessor< ov::op::util::FrameworkNodeAttrs >
- #include <framework_node.hpp>
- template<> PartialShape > : public ov::DirectValueAccessor< ov::PartialShape >
- #include <partial_shape.hpp>
- template<> Shape > : public ov::IndirectVectorValueAccessor< ov::Shape, std::vector< int64_t > >
- #include <shape.hpp>
-
template<>
class AttributeAdapter<ParameterVector> : public ov::VisitorAdapter - #include <parameter.hpp>
-
template<>
class AttributeAdapter<ResultVector> : public ov::VisitorAdapter - #include <result.hpp>
- template<> string > > : public ov::DirectValueAccessor< std::set< std::string > >
- #include <attribute_adapter.hpp>
- template<> AlignedBuffer > > : public ov::DirectValueAccessor< std::shared_ptr< ngraph::runtime::AlignedBuffer > >
- #include <aligned_buffer.hpp>
- template<> Variable > > : public ov::DirectValueAccessor< std::shared_ptr< op::util::Variable > >
- #include <variable.hpp>
- template<> Model > > : public ov::DirectValueAccessor< std::shared_ptr< ov::Model > >
- #include <model.hpp>
- template<> Node > > : public ov::VisitorAdapter
- #include <node.hpp>
Visits a reference to a node that has been registered with the visitor.
- template<> string > : public ov::DirectValueAccessor< std::string >
- #include <attribute_adapter.hpp>
Access a string as a string.
- template<> vector< double > > : public ov::DirectValueAccessor< std::vector< double > >
- #include <attribute_adapter.hpp>
Access a vector<double>
- template<> vector< float > > : public ov::DirectValueAccessor< std::vector< float > >
- #include <attribute_adapter.hpp>
Access a vector<float>
- template<> vector< int16_t > > : public ov::DirectValueAccessor< std::vector< int16_t > >
- #include <attribute_adapter.hpp>
Access a vector<int16_t>
- template<> vector< int32_t > > : public ov::DirectValueAccessor< std::vector< int32_t > >
- #include <attribute_adapter.hpp>
Access a vector<int32_t>
- template<> vector< int64_t > > : public ov::DirectValueAccessor< std::vector< int64_t > >
- #include <attribute_adapter.hpp>
Access a vector<int64_t>
- template<> vector< int8_t > > : public ov::DirectValueAccessor< std::vector< int8_t > >
- #include <attribute_adapter.hpp>
Access a vector<int8_t>
Note: These class bodies cannot be defined with templates because of interactions between dllexport and templates on Windows.
- template<> InputDescription > > > : public ov::DirectValueAccessor< std::vector< std::shared_ptr< op::util::MultiSubGraphOp::InputDescription > > >
- #include <multi_subgraph_base.hpp>
- template<> OutputDescription > > > : public ov::DirectValueAccessor< std::vector< std::shared_ptr< op::util::MultiSubGraphOp::OutputDescription > > >
- #include <multi_subgraph_base.hpp>
- template<> string > > : public ov::DirectValueAccessor< std::vector< std::string > >
- #include <attribute_adapter.hpp>
Access a vector<string>
- template<> vector< uint16_t > > : public ov::DirectValueAccessor< std::vector< uint16_t > >
- #include <attribute_adapter.hpp>
Access a vector<uint16_t>
- template<> vector< uint32_t > > : public ov::DirectValueAccessor< std::vector< uint32_t > >
- #include <attribute_adapter.hpp>
Access a vector<uint32_t>
- template<> vector< uint64_t > > : public ov::DirectValueAccessor< std::vector< uint64_t > >
- #include <attribute_adapter.hpp>
Access a vector<uint64_t>
- template<> vector< uint8_t > > : public ov::DirectValueAccessor< std::vector< uint8_t > >
- #include <attribute_adapter.hpp>
Access a vector<uint8_t>
-
template<>
class AttributeAdapter<Strides> : public ov::IndirectVectorValueAccessor<Strides, std::vector<int64_t>> - #include <strides.hpp>
-
template<>
class AttributeAdapter<uint16_t> : public ov::IndirectScalarValueAccessor<uint16_t, int64_t> - #include <attribute_adapter.hpp>
Access a uint16_t as an int64_t.
-
template<>
class AttributeAdapter<uint32_t> : public ov::IndirectScalarValueAccessor<uint32_t, int64_t> - #include <attribute_adapter.hpp>
Access a uint32_t as an int64_t.
-
template<>
class AttributeAdapter<uint64_t> : public ov::IndirectScalarValueAccessor<uint64_t, int64_t> - #include <attribute_adapter.hpp>
Access a uint64_t as an int64_t.
-
template<>
class AttributeAdapter<uint8_t> : public ov::IndirectScalarValueAccessor<uint8_t, int64_t> - #include <attribute_adapter.hpp>
Access a uint8_t as an int64_t.
-
class AttributeVisitor
- #include <attribute_visitor.hpp>
Visits the attributes of a node, primarily for serialization-like tasks.
Attributes are the node parameters that are always compile-time constants. Values computed from the graph topology and attributes during compilation are not attributes.
Attributes have a wide variety of types, but serialization formats are more restricted. We assume serialization easily supports scalar types of bool 64-bit signed, string, and double, and has specialized ways to support numeric arrays and raw data+size. The visitor and adapter convert between the limited serialization types and the unlimited attribute types.
A visitor is passed to an op’s visit_attributes method. The visit_attributes method calls the template method visitor.on_attribute<AT>(const std::string& name, AT& value) on each attribute. The visitor can read or write the attribute’s value. The on_attribute method creates an AttributeAdapter<AT> for the value and passes it to one of the visitors on_adapter methods. The on_adapter methods expect a reference to a ValueAccessor<VAT> or a VisitorAdapter. A ValueAccessor<VAT> has get/set methods that can be used to read/write the attribute value as type VAT. These methods are triggered by deriving AttributeAdapter<AT> from ValueAccessor<VAT>. For more complex cases, such as structs, the on_adapter method for VisitorAdapter passes the name and visitor to the adapter, so that the adapter can perform additional work such as visiting struct members or sequence values.
When a node visits an attribute with structure, the node’s on_attribute passes a name for the entire attribute, but the struct will have its own methods to be visited. Similarly, a vector will have a sequence of members to be visited. The adapter may use the visitor methods start_struct/finish_struct and start_vector/next_vector/finish_vector to inidicate nexted members.
The visitor method get_name_with_context creates a generic nested version of the name. Visitors can override according to their serialization requirements.
Attributes that are shared_ptr<Node> are special. They must have been already been registered with the visitor using register_node, which needs a shared pointer to a node and a string ID. The ID string will be used to serialize the node or find the node during deserialization.
Subclassed by ov::frontend::FWVisitor, ov::frontend::FWVisitorInputAttributes
Public Functions
-
virtual void on_adapter(const std::string &name, ValueAccessor<void> &adapter) = 0
handles all specialized on_adapter methods implemented by the visitor.
The adapter implements get_type_info(), which can be used to determine the adapter directly or via is_type and as_type on any platform
-
virtual void on_adapter(const std::string &name, VisitorAdapter &adapter)
Hook for adapters that need visitor access.
Provides API to handle openvino Function attribute type, accessed as ValueAccessor.
- Parameters
name – attribute name
adapter – reference to a Function ValueAccessor<VAT>
-
template<typename AT>
inline void on_attribute(const std::string &name, AT &value) The generic visitor. There must be a definition of AttributeAdapter<T> that can convert to a ValueAccessor
-
inline const std::vector<std::string> &get_context() const
- Returns
The nested context of visits
-
virtual std::string get_name_with_context()
- Returns
context prepended to names
-
virtual void start_structure(const std::string &name)
Start visiting a nested structure.
-
virtual std::string finish_structure()
Finish visiting a nested structure.
Associate a node with an id.
No node may be used as an attribute unless it has already been registered with an ID. References to nodes are visited with a ValueAccessor of their ID.
-
virtual std::shared_ptr<Node> get_registered_node(node_id_t id)
Returns the node with the given id, or nullptr if there is no registered node.
Returns the id for the node, or -1 if the node is not registered.
-
virtual void on_adapter(const std::string &name, ValueAccessor<void> &adapter) = 0
-
class AvgPoolPrecisionPreservedAttribute : public ov::PrecisionPreservedAttribute
- #include <avg_pool_precision_preserved_attribute.hpp>
AvgPoolPrecisionPreservedAttribute is utility attribute which is used only during
AvgPooloperation precision preserved property definition.For more details about the attribute, refer to AvgPoolPrecisionPreservedAttribute page in the Inference Engine Developer Guide.
-
class AxisSet : public std::set<size_t>
- #include <axis_set.hpp>
A set of axes.
-
class AxisVector : public std::vector<size_t>
- #include <axis_vector.hpp>
A vector of axes.
-
class BaseOpExtension : public ov::Extension
- #include <op_extension.hpp>
The base interface for OpenVINO operation extensions.
Subclassed by ov::OpExtension< T >
Public Functions
-
virtual const ov::DiscreteTypeInfo &get_type_info() const = 0
Returns the type info of operation.
- Returns
-
virtual ov::OutputVector create(const ov::OutputVector &inputs, ov::AttributeVisitor &visitor) const = 0
Method creates an OpenVINO operation.
- Parameters
inputs – vector of input ports
visitor – attribute visitor which allows to read necessaty arguments
- Returns
vector of output ports
-
virtual std::vector<ov::Extension::Ptr> get_attached_extensions() const = 0
Returns extensions that should be registered together with this extension class object.
Attached extensions may include frontend extensions that OpenVINO op to framework ops or necessary transformations that should be applied to the network which consist of target op.
- Returns
-
~BaseOpExtension() override
Destructor.
-
virtual const ov::DiscreteTypeInfo &get_type_info() const = 0
-
class bfloat16
- #include <bfloat16.hpp>
-
class BiasAttribute : public ov::RuntimeAttribute
- #include <bias_attribute.hpp>
-
class Busy : public ov::Exception
- #include <exception.hpp>
Thrown in case of calling the InferRequest methods while the request is busy with compute operation.
-
class Cancelled : public ov::Exception
- #include <exception.hpp>
Thrown in case of cancelled asynchronous operation.
-
struct CheckLocInfo
- #include <except.hpp>
-
class CompiledModel
- #include <compiled_model.hpp>
This class represents a compiled model.
A model is compiled by a specific device by applying multiple optimization transformations, then mapping to compute kernels.
Public Functions
-
CompiledModel() = default
Default constructor.
-
~CompiledModel()
Destructor that preserves unloading order of an implementation object and reference to library.
-
std::shared_ptr<const Model> get_runtime_model() const
Gets runtime model information from a device. This object represents an internal device-specific model that is optimized for a particular accelerator. It contains device-specific nodes, runtime information and can be used only to understand how the source model is optimized and which kernels, element types, and layouts are selected for optimal inference.
- Returns
A model containing Executable Graph Info.
-
const std::vector<ov::Output<const ov::Node>> &inputs() const
Gets all inputs of a compiled model. Inputs are represented as a vector of outputs of the ov::op::v0::Parameter operations. They contain information about input tensors such as tensor shape, names, and element type.
- Returns
std::vector of model inputs.
-
const ov::Output<const ov::Node> &input() const
Gets a single input of a compiled model. The input is represented as an output of the ov::op::v0::Parameter operation. The input contains information about input tensor such as tensor shape, names, and element type.
Note
If a model has more than one input, this method throws ov::Exception.
- Returns
Compiled model input.
-
const ov::Output<const ov::Node> &input(size_t i) const
Gets input of a compiled model identified by
i. The input contains information about input tensor such as tensor shape, names, and element type.Note
The method throws ov::Exception if input with the specified index
iis not found.- Parameters
i – Index of input.
- Returns
Compiled model input.
-
const ov::Output<const ov::Node> &input(const std::string &tensor_name) const
Gets input of a compiled model identified by
tensor_name. The input contains information about input tensor such as tensor shape, names, and element type.Note
The method throws ov::Exception if input with the specified tensor name
tensor_nameis not found.- Parameters
tensor_name – The input tensor name.
- Returns
Compiled model input.
-
const std::vector<ov::Output<const ov::Node>> &outputs() const
Get all outputs of a compiled model. Outputs are represented as a vector of output from the ov::op::v0::Result operations. Outputs contain information about output tensors such as tensor shape, names, and element type.
- Returns
std::vector of model outputs.
-
const ov::Output<const ov::Node> &output() const
Gets a single output of a compiled model. The output is represented as an output from the ov::op::v0::Result operation. The output contains information about output tensor such as tensor shape, names, and element type.
Note
If a model has more than one output, this method throws ov::Exception.
- Returns
Compiled model output.
-
const ov::Output<const ov::Node> &output(size_t i) const
Gets output of a compiled model identified by
index. The output contains information about output tensor such as tensor shape, names, and element type.Note
The method throws ov::Exception if output with the specified index
indexis not found.- Parameters
i – Index of input.
- Returns
Compiled model output.
-
const ov::Output<const ov::Node> &output(const std::string &tensor_name) const
Gets output of a compiled model identified by
tensor_name. The output contains information about output tensor such as tensor shape, names, and element type.Note
The method throws ov::Exception if output with the specified tensor name
tensor_nameis not found.- Parameters
tensor_name – Output tensor name.
- Returns
Compiled model output.
-
InferRequest create_infer_request()
Creates an inference request object used to infer the compiled model. The created request has allocated input and output tensors (which can be changed later).
- Returns
InferRequest object
-
void export_model(std::ostream &model_stream)
Exports the current compiled model to an output stream
std::ostream. The exported model can also be imported via the ov::Core::import_model method.See also
- Parameters
model_stream – Output stream to store the model to.
-
void set_property(const AnyMap &properties)
Sets properties for the current compiled model.
- Parameters
properties – Map of pairs: (property name, property value).
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<void, Properties...> set_property(Properties&&... properties) Sets properties for the current compiled model.
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types.- Parameters
properties – Optional pack of pairs: (property name, property value).
-
Any get_property(const std::string &name) const
Gets properties for current compiled model.
The method is responsible for extracting information that affects compiled model inference. The list of supported configuration values can be extracted via CompiledModel::get_property with the ov::supported_properties key, but some of these keys cannot be changed dynamically, for example, ov::device::id cannot be changed if a compiled model has already been compiled for a particular device.
-
template<typename T, PropertyMutability mutability>
inline T get_property(const ov::Property<T, mutability> &property) const Gets properties related to device behaviour.
The method extracts information that can be set via the set_property method.
- Template Parameters
T – Type of a returned value.
- Parameters
property – Property object.
- Returns
Value of property.
-
RemoteContext get_context() const
Returns pointer to device-specific shared context on a remote accelerator device that was used to create this CompiledModel.
- Returns
A context.
-
bool operator!() const noexcept
Checks if the current CompiledModel object is not initialized.
- Returns
trueif the current CompiledModel object is not initialized;false, otherwise.
-
explicit operator bool() const noexcept
Checks if the current CompiledModel object is initialized.
- Returns
trueif the current CompiledModel object is initialized;false, otherwise.
-
CompiledModel() = default
-
class Coordinate : public std::vector<size_t>
- #include <coordinate.hpp>
Coordinates for a tensor element.
-
class CoordinateDiff : public std::vector<std::ptrdiff_t>
- #include <coordinate_diff.hpp>
A difference (signed) of tensor element coordinates.
-
class CoordinateIterator
- #include <coordinate_transform.hpp>
A useful class that allows to iterate over the tensor coordinates. For example, for tensor with dimensions {2, 3} this iterator produces the following coordinates: {0,0}, {0,1}, {0,2}, {1,0}, {1,1}, {2,2}.
- Deprecated:
Public Functions
-
inline CoordinateIterator(const Shape &target_shape)
Coordinates iterator constructor.
- Parameters
target_shape – The target shape for coordinates iteration
-
void operator++()
The postfix operation increment the iterator by one.
-
CoordinateIterator operator++(int)
The prefix operation increment the iterator by one.
-
void operator+=(size_t n)
Increments iterator n times.
- Parameters
n – number of elements it should be advanced
-
const Coordinate &operator*() const noexcept
Iterator dereferencing operator returns reference to current pointed coordinate.
-
bool operator!=(const CoordinateIterator &it) const noexcept
Checks for iterator inequality.
- Parameters
it – second iterator to compare
-
bool operator==(const CoordinateIterator &it) const noexcept
Checks for iterator equality.
- Parameters
it – second iterator to compare
-
size_t advance(size_t axis) noexcept
Increments iterator using specified axis of the shape n times.
- Parameters
axis – index used for iteration
Public Static Functions
-
static const CoordinateIterator &end()
Useful function to build the last iterator. Returns a singleton that points to the last iterator.
-
class CoordinateTransform : protected ov::CoordinateTransformBasic
- #include <coordinate_transform.hpp>
Class which allows to calculate item index with given coordinates in tensor and helps to iterate over the subset of coordinates. Tensor items should be placed in memory in row-major order.
- Deprecated:
Public Functions
-
size_t index(const Coordinate &c) const
The tensor element index calculation by given coordinate.
- Parameters
c – tensor element coordinate
-
bool has_source_coordinate(const Coordinate &c) const
Checks that coordinate belongs to given coordinates subset.
- Parameters
c – tensor element coordinate
-
Coordinate to_source_coordinate(const Coordinate &c) const
Convert a target-space coordinate to a source-space coordinate.
- Parameters
c – tensor element coordinate
-
CoordinateIterator begin() const noexcept
Returns an iterator to the first coordinate of the tensor.
-
const CoordinateIterator &end() const noexcept
Returns an iterator to the coordinate following the last element of the tensor.
-
class CoordinateTransformBasic
- #include <coordinate_transform.hpp>
Class which allows to calculate item index with given coordinates in tensor and helps to iterate over all coordinates. Tensor items should be placed in memory in row-major order.
Subclassed by ov::CoordinateTransform
Public Functions
-
size_t index(const Coordinate &c) const
The tensor element index calculation by given coordinate.
- Deprecated:
- Parameters
c – tensor element coordinate
-
CoordinateIterator begin() const noexcept
Returns an iterator to the first coordinate of the tensor.
-
const CoordinateIterator &end() const noexcept
Returns an iterator to the coordinate following the last element of the tensor.
-
size_t index(const Coordinate &c) const
-
class Core
- #include <core.hpp>
This class represents an OpenVINO runtime Core entity.
User applications can create several Core class instances, but in this case the underlying plugins are created multiple times and not shared between several Core instances. The recommended way is to have a single Core instance per application.
Unnamed Group
-
CompiledModel compile_model(const std::string &model_path, const AnyMap &properties = {})
Reads and loads a compiled model from the IR/ONNX/PDPD file to the default OpenVINO device selected by the AUTO plugin.
This can be more efficient than using the Core::read_model + Core::compile_model(model_in_memory_object) flow, especially for cases when caching is enabled and a cached model is available.
- Parameters
model_path – Path to a model.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model.
Unnamed Group
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::string &model_path, Properties&&... properties) Reads and loads a compiled model from IR / ONNX / PDPD file to the default OpenVINO device selected by AUTO plugin.
This can be more efficient than using read_model + compile_model(Model) flow especially for cases when caching is enabled and cached model is available
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types- Parameters
model_path – path to model with string or wstring
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation
- Returns
A compiled model
Unnamed Group
-
CompiledModel compile_model(const std::string &model_path, const std::string &device_name, const AnyMap &properties = {})
Reads a model and creates a compiled model from the IR/ONNX/PDPD file.
This can be more efficient than using the Core::read_model + Core::compile_model(model_in_memory_object) flow, especially for cases when caching is enabled and a cached model is available.
- Parameters
model_path – Path to a model.
device_name – Name of a device to load a model to.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model.
Unnamed Group
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::string &model_path, const std::string &device_name, Properties&&... properties) Reads a model and creates a compiled model from the IR/ONNX/PDPD file.
This can be more efficient than using read_model + compile_model(Model) flow especially for cases when caching is enabled and cached model is available.
- Template Parameters
Properties – Should be a pack of
std::pair<std::string, ov::Any>types.- Parameters
model_path – Path to a model.
device_name – Name of a device to load a model to.
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model.
Public Functions
-
explicit Core(const std::string &xml_config_file = {})
Constructs an OpenVINO Core instance with devices and their plugins description.
There are two ways how to configure device plugins:
(default) Use XML configuration file in case of dynamic libraries build;
Use strictly defined configuration in case of static libraries build.
- Parameters
xml_config_file – Path to the .xml file with plugins to load from. If path contains only file name with extension, file will be searched in a folder with OpenVINO runtime shared library. If the XML configuration file is not specified, default OpenVINO Runtime plugins are loaded from:
(dynamic build) default
plugins.xmlfile located in the same folder as OpenVINO runtime shared library;(static build) statically defined configuration. In this case path to the .xml file is ignored.
-
std::map<std::string, Version> get_versions(const std::string &device_name) const
Returns device plugins version information. Device name can be complex and identify multiple devices at once like
HETERO:CPU,GPU; in this case, std::map contains multiple entries, each per device.- Parameters
device_name – Device name to identify a plugin.
- Returns
A vector of versions.
-
std::shared_ptr<ov::Model> read_model(const std::string &model_path, const std::string &bin_path = {}) const
Reads models from IR / ONNX / PDPD / TF / TFLite file formats.
- Parameters
model_path – Path to a model.
bin_path – Path to a data file. For IR format (*.bin):
if
bin_pathis empty, will try to read a bin file with the same name as xml andif the bin file with the same name is not found, will load IR without weights. For the following file formats the
bin_pathparameter is not used:ONNX format (*.onnx)
PDPD (*.pdmodel)
TF (*.pb)
TFLite (*.tflite)
- Returns
A model.
-
std::shared_ptr<ov::Model> read_model(const std::string &model, const Tensor &weights) const
Reads models from IR / ONNX / PDPD / TF / TFLite formats.
Note
Created model object shares the weights with the
weightsobject. Thus, do not createweightson temporary data that can be freed later, since the model constant data will point to an invalid memory.- Parameters
model – String with a model in IR / ONNX / PDPD / TF / TFLite format.
weights – Shared pointer to a constant tensor with weights. Reading ONNX / PDPD / TF / TFLite models does not support loading weights from the
weightstensors.
- Returns
A model.
Creates and loads a compiled model from a source model to the default OpenVINO device selected by the AUTO plugin.
Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources).
- Parameters
model – Model object acquired from Core::read_model.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model.
Creates and loads a compiled model from a source model to the default OpenVINO device selected by AUTO plugin.
Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources)
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types- Parameters
model – Model object acquired from Core::read_model
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation
- Returns
A compiled model
Creates a compiled model from a source model object.
Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources).
- Parameters
model – Model object acquired from Core::read_model.
device_name – Name of a device to load a model to.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model.
Creates a compiled model from a source model object.
Users can create as many compiled models as they need and use them simultaneously (up to the limitation of the hardware resources)
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types- Parameters
model – Model object acquired from Core::read_model
device_name – Name of device to load model to
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation
- Returns
A compiled model
-
CompiledModel compile_model(const std::string &model, const ov::Tensor &weights, const std::string &device_name, const AnyMap &properties = {})
Reads a model and creates a compiled model from the IR/ONNX/PDPD memory.
Note
Created model object shares the weights with the
weightsobject. Thus, do not createweightson temporary data that can be freed later, since the model constant data will point to an invalid memory.- Parameters
model – String with a model in IR/ONNX/PDPD format.
weights – Shared pointer to a constant tensor with weights. Reading ONNX/PDPD models does not support loading weights from the
weightstensors.device_name – Name of a device to load a model to.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model.
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> compile_model(const std::string &model, const ov::Tensor &weights, const std::string &device_name, Properties&&... properties) Reads a model and creates a compiled model from the IR/ONNX/PDPD memory.
Note
Created model object shares the weights with the
weightsobject. Thus, do not createweightson temporary data that can be freed later, since the model constant data will point to an invalid memory.- Parameters
model – String with a model in IR/ONNX/PDPD format.
weights – Shared pointer to a constant tensor with weights. Reading ONNX/PDPD models does not support loading weights from the
weightstensors.device_name – Name of a device to load a model to.
- Template Parameters
Properties – Should be a pack of
std::pair<std::string, ov::Any>types.- Returns
A compiled model.
Creates a compiled model from a source model within a specified remote context.
- Parameters
model – Model object acquired from Core::read_model.
context – A reference to a RemoteContext object.
properties – Optional map of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model object.
Creates a compiled model from a source model within a specified remote context.
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types- Parameters
model – Model object acquired from Core::read_model
context – Pointer to RemoteContext object
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation
- Returns
A compiled model object
- OPENVINO_SUPPRESS_DEPRECATED_START void add_extension (const std::shared_ptr< InferenceEngine::IExtension > &extension)
Registers OpenVINO 1.0 extension to a Core object.
- Deprecated:
This method is deprecated. Please use other Core::add_extension methods.
- Parameters
extension – Pointer to the already loaded extension.
- OPENVINO_SUPPRESS_DEPRECATED_END void add_extension (const std::string &library_path)
Registers an extension to a Core object.
- Parameters
library_path – Path to the library with ov::Extension.
Registers an extension to a Core object.
- Parameters
extension – Pointer to the extension.
Registers extensions to a Core object.
- Parameters
extensions – Vector of loaded extensions.
-
template<class T, typename std::enable_if<std::is_base_of<ov::Extension, T>::value, bool>::type = true>
inline void add_extension(const T &extension) Registers an extension to a Core object.
- Parameters
extension – Extension class that is inherited from the ov::Extension class.
-
template<class T, class ...Targs, typename std::enable_if<std::is_base_of<ov::Extension, T>::value, bool>::type = true>
inline void add_extension(const T &extension, Targs... args) Registers extensions to a Core object.
- Parameters
extension – Extension class that is inherited from the ov::Extension class.
args – A list of extensions.
-
template<class T, typename std::enable_if<std::is_base_of<ov::op::Op, T>::value, bool>::type = true>
inline void add_extension() Registers a custom operation inherited from ov::op::Op.
-
template<class T, class ...Targs, typename std::enable_if<std::is_base_of<ov::op::Op, T>::value && sizeof...(Targs), bool>::type = true>
inline void add_extension() Registers custom operations inherited from ov::op::Op.
-
CompiledModel import_model(std::istream &model_stream, const std::string &device_name, const AnyMap &properties = {})
Imports a compiled model from the previously exported one.
- Parameters
model_stream – std::istream input stream containing a model previously exported using the ov::CompiledModel::export_model method.
device_name – Name of a device to import a compiled model for. Note, if
device_namedevice was not used to compile the original mode, an exception is thrown.properties – Optional map of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model.
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> import_model(std::istream &model_stream, const std::string &device_name, Properties&&... properties) Imports a compiled model from the previously exported one.
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types.- Parameters
model_stream – Model stream.
device_name – Name of a device to import a compiled model for. Note, if
device_namedevice was not used to compile the original mode, an exception is thrown.properties – Optional pack of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model.
-
CompiledModel import_model(std::istream &model_stream, const RemoteContext &context, const AnyMap &properties = {})
Imports a compiled model from the previously exported one with the specified remote context.
- Parameters
model_stream – std::istream input stream containing a model previously exported from ov::CompiledModel::export_model
context – A reference to a RemoteContext object. Note, if the device from
contextwas not used to compile the original mode, an exception is thrown.properties – Optional map of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model.
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<CompiledModel, Properties...> import_model(std::istream &model_stream, const RemoteContext &context, Properties&&... properties) Imports a compiled model from the previously exported one with the specified remote context.
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types.- Parameters
model_stream – Model stream.
context – Pointer to a RemoteContext object.
properties – Optional pack of pairs: (property name, property value) relevant only for this load operation.
- Returns
A compiled model.
Query device if it supports the specified model with specified properties.
- Parameters
device_name – Name of a device to query.
model – Model object to query.
properties – Optional map of pairs: (property name, property value).
- Returns
An object containing a map of pairs an operation name -> a device name supporting this operation.
Queries a device if it supports the specified model with specified properties.
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types.- Parameters
device_name – Name of a device to query.
model – Model object to query.
properties – Optional pack of pairs: (property name, property value) relevant only for this query operation.
- Returns
An object containing a map of pairs an operation name -> a device name supporting this operation.
-
void set_property(const AnyMap &properties)
Sets properties for all the registered devices, acceptable keys can be found in openvino/runtime/properties.hpp.
- Parameters
properties – Map of pairs: (property name, property value).
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<void, Properties...> set_property(Properties&&... properties) Sets properties for all the registered devices, acceptable keys can be found in openvino/runtime/properties.hpp.
- Template Parameters
Properties – Should be a pack of
std::pair<std::string, ov::Any>types.- Parameters
properties – Optional pack of pairs: property name, property value.
-
void set_property(const std::string &device_name, const AnyMap &properties)
Sets properties for a device, acceptable keys can be found in openvino/runtime/properties.hpp.
- Parameters
device_name – Name of a device.
properties – Map of pairs: (property name, property value).
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<void, Properties...> set_property(const std::string &device_name, Properties&&... properties) Sets properties for a device, acceptable keys can be found in openvino/runtime/properties.hpp.
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types.- Parameters
device_name – Name of a device.
properties – Optional pack of pairs: (property name, property value).
-
Any get_property(const std::string &device_name, const std::string &name) const
Gets properties related to device behaviour.
The method extracts information that can be set via the set_property method.
- Parameters
device_name – Name of a device to get a property value.
name – Property name.
- Returns
Value of a property corresponding to the property name.
-
Any get_property(const std::string &device_name, const std::string &name, const AnyMap &arguments) const
Gets properties related to device behaviour.
The method extracts information that can be set via the set_property method.
- Parameters
device_name – Name of a device to get a property value.
name – Property name.
arguments – Additional arguments to get a property.
- Returns
Value of a property corresponding to the property name.
-
inline Any get_property(const std::string &name) const
Gets properties related to core behaviour.
The method extracts information that can be set via the set_property method.
- Parameters
name – Property name.
- Returns
Value of a property corresponding to the property name.
-
template<typename T, PropertyMutability M>
inline T get_property(const std::string &device_name, const ov::Property<T, M> &property) const Gets properties related to device behaviour.
The method is needed to request common device or system properties. It can be device name, temperature, and other devices-specific values.
-
template<typename T, PropertyMutability M>
inline T get_property(const std::string &device_name, const ov::Property<T, M> &property, const AnyMap &arguments) const Gets properties related to device behaviour.
The method is needed to request common device or system properties. It can be device name, temperature, other devices-specific values.
-
template<typename T, PropertyMutability M, typename ...Args>
inline util::EnableIfAllStringAny<T, Args...> get_property(const std::string &device_name, const ov::Property<T, M> &property, Args&&... args) const Gets properties related to device behaviour.
The method is needed to request common device or system properties. It can be device name, temperature, other devices-specific values.
- Template Parameters
T – Type of a returned value.
M – Property mutability.
Args – Set of additional arguments ended with property object variable.
- Parameters
device_name – Name of a device to get a property value.
property – Property object.
args – Optional pack of pairs: (argument name, argument value) ended with property object.
- Returns
Property value.
-
std::vector<std::string> get_available_devices() const
Returns devices available for inference. Core objects go over all registered plugins and ask about available devices.
- Returns
A vector of devices. The devices are returned as { CPU, GPU.0, GPU.1, GNA }. If there is more than one device of a specific type, they are enumerated with the .# suffix. Such enumerated device can later be used as a device name in all Core methods like Core::compile_model, Core::query_model, Core::set_property and so on.
-
void register_plugin(const std::string &plugin, const std::string &device_name, const ov::AnyMap &config = {})
Register a new device and plugin that enables this device inside OpenVINO Runtime.
Note
For security purposes it suggested to specify absolute path to register plugin.
- Parameters
plugin – Path (absolute or relative) or name of a plugin. Depending on platform,
pluginis wrapped with shared library suffix and prefix to identify library full name. For example, on Linux platform, plugin name specified asplugin_namewill be wrapped aslibplugin_name.so. Plugin search algorithm:If
pluginpoints to an exact library path (absolute or relative), it will be used.If
pluginspecifies file name (libplugin_name.so) or plugin name (plugin_name), it will be searched by file name (libplugin_name.so) in CWD or in paths pointed by PATH/LD_LIBRARY_PATH/DYLD_LIBRARY_PATH environment variables depending on the platform.
device_name – Device name to register a plugin for.
config – Plugin configuration options
-
void unload_plugin(const std::string &device_name)
Unloads the previously loaded plugin identified by
device_namefrom OpenVINO Runtime. The method is needed to remove loaded plugin instance and free its resources. If plugin for a specified device has not been created before, the method throws an exception.Note
This method does not remove plugin from the plugins known to OpenVINO Core object.
- Parameters
device_name – Device name identifying plugin to remove from OpenVINO Runtime.
-
void register_plugins(const std::string &xml_config_file)
Registers a device plugin to the OpenVINO Runtime Core instance using an XML configuration file with plugins description.
The XML file has the following structure:
<ie> <plugins> <plugin name="" location=""> <extensions> <extension location=""/> </extensions> <properties> <property key="" value=""/> </properties> </plugin> </plugins> </ie>
nameidentifies name of a device enabled by a plugin.locationspecifies absolute path to dynamic library with a plugin. The path can also be relative to XML file directory. It allows having common config for different systems with different configurations.propertiesare set to a plugin via the ov::Core::set_property method.extensionsare set to a plugin via the ov::Core::add_extension method.
Note
For security purposes it suggested to specify absolute path to register plugin.
- Parameters
xml_config_file – A path to .xml file with plugins to register.
-
RemoteContext create_context(const std::string &device_name, const AnyMap &remote_properties)
Creates a new remote shared context object on the specified accelerator device using specified plugin-specific low-level device API parameters (device handle, pointer, context, etc.).
- Parameters
device_name – Name of a device to create a new shared context on.
remote_properties – Map of device-specific shared context remote properties.
- Returns
Reference to a created remote context.
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<RemoteContext, Properties...> create_context(const std::string &device_name, Properties&&... remote_properties) Creates a new shared context object on specified accelerator device using specified plugin-specific low level device API properties (device handle, pointer, etc.)
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types- Parameters
device_name – Name of a device to create new shared context on.
remote_properties – Pack of device-specific shared context remote properties.
- Returns
A shared pointer to a created remote context.
-
RemoteContext get_default_context(const std::string &device_name)
Gets a pointer to default (plugin-supplied) shared context object for the specified accelerator device.
- Parameters
device_name – Name of a device to get a default shared context from.
- Returns
Reference to a default remote context.
-
CompiledModel compile_model(const std::string &model_path, const AnyMap &properties = {})
-
class Decompression : public ov::RuntimeAttribute
- #include <decompression.hpp>
Decompression class represents runtime info attribute that marks operation as used as decompression for Compressed Only format.
-
class DequantizationNode : public ov::RuntimeAttribute
- #include <dequantization_node.hpp>
DequantizationNode class represents runtime info attribute that marks operation that are part of dequantization subgraph.
-
class DeviceIDParser
- #include <device_id_parser.hpp>
Class parses device name and id.
-
class Dimension
- #include <dimension.hpp>
Class representing a dimension, which may be dynamic (undetermined until runtime), in a shape or shape-like object.
Static dimensions may be implicitly converted from value_type. A dynamic dimension is constructed with Dimension() or Dimension::dynamic().
Public Functions
-
Dimension(value_type dimension)
Construct a static dimension.
- Parameters
dimension – Value of the dimension.
-
Dimension(value_type min_dimension, value_type max_dimension)
Construct a dynamic dimension with bounded range.
- Parameters
min_dimension – The lower inclusive limit for the dimension
max_dimension – The upper inclusive limit for the dimension
-
Dimension(const std::string &str)
Construct a dimension from string.
- Parameters
str – String to parse to dimension.
-
Dimension() = default
Construct a dynamic dimension with range [0, …].
-
inline bool is_static() const
Check whether this dimension is static.
- Returns
trueif the dimension is static, elsefalse.
-
inline bool is_dynamic() const
Check whether this dimension is dynamic.
- Returns
falseif the dimension is static, elsetrue.
-
value_type get_length() const
Convert this dimension to
value_type. This dimension must be static and non-negative.- Throws
std::invalid_argument – If this dimension is dynamic or negative.
-
inline const Interval &get_interval() const
Return the interval of valid lengths.
-
bool same_scheme(const Dimension &dim) const
Check whether this dimension represents the same scheme as the argument (both dynamic, or equal).
- Parameters
dim – The other dimension to compare this dimension to.
- Returns
trueif this dimension anddimare both dynamic, or if they are both static and equal; otherwise,false.
-
bool compatible(const Dimension &d) const
Check whether this dimension is capable of being merged with the argument dimension.
Two dimensions are considered compatible if it is possible to merge them. (See Dimension::merge.)
- Parameters
d – The dimension to compare this dimension with.
- Returns
trueif this dimension is compatible withd, elsefalse.
-
bool relaxes(const Dimension &d) const
Check whether this dimension is a relaxation of the argument.
A dimension
d1relaxes (or is a relaxation of)d2ifd1andd2are static and equal, ord1is dynamic.d1.relaxes(d2)is equivalent tod2.refines(d1).- Parameters
d – The dimension to compare this dimension with.
- Returns
trueif this dimension relaxesd, elsefalse.
-
bool refines(const Dimension &d) const
Check whether this dimension is a refinement of the argument.
A dimension
d2refines (or is a refinement of)d1ifd1andd2are static and equal, ord2is dynamic.d1.refines(d2)is equivalent tod2.relaxes(d1).- Parameters
d – The dimension to compare this dimension with.
- Returns
trueif this dimension relaxesd, elsefalse.
-
Dimension operator+(const Dimension &dim) const
Addition operator for Dimension.
- Parameters
dim – Right operand for addition.
- Returns
Smallest interval dimension enclosing inputs
-
Dimension operator-(const Dimension &dim) const
Subtraction operator for Dimension.
- Parameters
dim – Right operand for subtraction.
- Returns
Smallest interval dimension enclosing inputs
-
Dimension operator/(const value_type divisor) const
Division operator for Dimension divided by a value_type parameter.
- Parameters
divisor – Right operand for division.
- Returns
Smallest interval dimension enclosing inputs
-
inline Dimension &operator/=(const value_type divisor)
Divided-into operator for Dimension.
- Parameters
divisor – Right operand for multiplication.
- Returns
A reference to
*this, after updating*thisto the value*this * dim.
-
Dimension operator*(const Dimension &dim) const
Multiplication operator for Dimension.
- Parameters
dim – Right operand for multiplicaiton.
- Returns
Smallest interval containing all “produces” which are 0 if either of
thisordimhas length0, else unbounded if either is unbounded, else product of lengths.
-
inline Dimension &operator+=(const Dimension &dim)
Add-into operator for Dimension.
- Parameters
dim – Right operand for addition.
- Returns
A reference to
*this, after updating*thisto the value*this + dim.
-
inline Dimension &operator*=(const Dimension &dim)
Multiply-into operator for Dimension.
- Parameters
dim – Right operand for multiplication.
- Returns
A reference to
*this, after updating*thisto the value*this * dim.
-
std::string to_string() const
String representation of Dimension.
Public Static Functions
-
static bool merge(Dimension &dst, const Dimension &d1, const Dimension &d2)
Try to merge two Dimension objects together.
If
d1is dynamic, writesd2todstand returnstrue.If
d2is dynamic, writesd1todstand returnstrue.If
d1andd2are static and equal, writesd1todstand returnstrue.If
d1andd2are both static and unequal, leavesdstunchanged and returnsfalse.
- Parameters
dst – [out] Reference to write the merged Dimension into.
d1 – First dimension to merge.
d2 – Second dimension to merge.
- Returns
trueif merging succeeds, elsefalse.
-
static bool broadcast_merge(Dimension &dst, const Dimension &d1, const Dimension &d2)
Try to merge two Dimension objects together with implicit broadcasting of unit-sized dimension to non unit-sized dimension.
-
static inline Dimension dynamic()
Create a dynamic dimension.
- Returns
A dynamic dimension.
-
Dimension(value_type dimension)
-
template<typename AT>
class DirectValueAccessor : public ov::ValueAccessor<AT> - #include <attribute_adapter.hpp>
-
class DisableCleanupAttribute : public ov::RuntimeAttribute
- #include <disable_cleanup_attribute.hpp>
-
class DisableFP16Compression : public ov::RuntimeAttribute
- #include <disable_fp16_compression.hpp>
DisableFP16Compression class represents runtime info attribute that marks operation as prohibited to convert to lower precision (e.g. to FP16) and they should be inferred precisely in the original precision.
-
struct DiscreteTypeInfo
- #include <type.hpp>
Type information for a type system without inheritance; instances have exactly one type not related to any other type.
Supports three functions, ov::is_type<Type>, ov::as_type<Type>, and ov::as_type_ptr<Type> for type-safe dynamic conversions via static_cast/static_ptr_cast without using C++ RTTI. Type must have a static type_info member and a virtual get_type_info() member that returns a reference to its type_info member.
- template<> bf16 >
- #include <element_type_traits.hpp>
- template<> boolean >
- #include <element_type_traits.hpp>
- template<> f16 >
- #include <element_type_traits.hpp>
- template<> f32 >
- #include <element_type_traits.hpp>
- template<> f64 >
- #include <element_type_traits.hpp>
- template<> i16 >
- #include <element_type_traits.hpp>
- template<> i32 >
- #include <element_type_traits.hpp>
- template<> i4 >
- #include <element_type_traits.hpp>
- template<> i64 >
- #include <element_type_traits.hpp>
- template<> i8 >
- #include <element_type_traits.hpp>
- template<> nf4 >
- #include <element_type_traits.hpp>
- template<> string >
- #include <element_type_traits.hpp>
- template<> u1 >
- #include <element_type_traits.hpp>
- template<> u16 >
- #include <element_type_traits.hpp>
- template<> u32 >
- #include <element_type_traits.hpp>
- template<> u4 >
- #include <element_type_traits.hpp>
- template<> u64 >
- #include <element_type_traits.hpp>
- template<> u8 >
- #include <element_type_traits.hpp>
-
template<typename AT>
class EnumAttributeAdapterBase : public ov::ValueAccessor<std::string> - #include <attribute_adapter.hpp>
Access an enum via a string.
- Template Parameters
AT – The attribute type enum class
Public Functions
-
inline virtual const std::string &get() override
Returns the value.
-
inline virtual void set(const std::string &value) override
Sets the value.
-
template<typename T>
class EnumMask - #include <enum_mask.hpp>
Public Types
-
using value_type = typename std::underlying_type<T>::type
Make sure the template type is an enum.
Extract the underlying type of the enum.
Public Functions
-
constexpr EnumMask() = default
Some bit operations are not safe for signed values, we require enum type to use unsigned underlying type.
-
inline bool is_any_set(const EnumMask &p) const
Check if any of the input parameter enum bit mask match.
-
inline bool is_set(const EnumMask &p) const
Check if all of the input parameter enum bit mask match.
-
inline bool is_any_clear(const EnumMask &p) const
Check if any of the input parameter enum bit mask does not match.
-
inline bool is_clear(const EnumMask &p) const
Check if all of the input parameter enum bit mask do not match.
-
using value_type = typename std::underlying_type<T>::type
-
template<typename EnumType>
class EnumNames - #include <enum_names.hpp>
Uses a pairings defined by EnumTypes::get() to convert between strings and enum values.
-
class Exception : public std::runtime_error
- #include <except.hpp>
Base error for ov runtime errors.
Subclassed by ngraph::builder::numpy_autobroadcast_incompatible_shapes, ov::AssertFailure, ov::Busy, ov::Cancelled, ov::op::util::error::UnknownActivationFunction
-
class Extension
- #include <extension.hpp>
The class provides the base interface for OpenVINO extensions.
Subclassed by ov::BaseOpExtension, ov::frontend::ConversionExtensionBase, ov::frontend::DecoderTransformationExtension, ov::frontend::ProgressReporterExtension, ov::frontend::TelemetryExtension
-
class float16
- #include <float16.hpp>
-
class FusedNames : public ov::RuntimeAttribute
- #include <fused_names_attribute.hpp>
FusedName class represents runtime info attribute that stores all operation names that was fully or partially fused into node.
Public Functions
-
FusedNames() = default
A default constructor
-
inline explicit FusedNames(const std::string &name)
Constructs a new object consisting of a single name *.
- Parameters
name – [in] The name
-
void fuseWith(const FusedNames &names)
Unites current set of already fused names with another FusedNames object.
- Parameters
names – [in] Another object to fuse with
-
std::string getNames() const
return string with operation names separated by coma in alphabetical order
-
std::vector<std::string> getVectorNames() const
return vector of fused names sorted in alphabetical order
- Returns
vector if strings
-
FusedNames() = default
-
class IAsyncInferRequest : public ov::IInferRequest
- #include <iasync_infer_request.hpp>
Base class with default implementation of asynchronous multi staged inference request. To customize pipeline stages derived class should change the content of IAsyncInferRequest::m_pipeline member container. It consists of pairs of tasks and executors which will run the task. The class is recommended to be used by plugins as a base class for asynchronous inference request implementation.
- Example
Here is an example of asynchronous inference request implementation for some accelerator device. It uses 5 different executors to run different stages of a synchronous inference request.
Note
To synchronize derived context with stages derived class should call IAsyncInferRequest::stop_and_wait() function in destructor.
Public Functions
-
virtual void start_async()
Start inference of specified input(s) in asynchronous mode.
Note
The method returns immediately. Inference starts also immediately.
-
virtual void wait()
Waits for the result to become available.
-
virtual bool wait_for(const std::chrono::milliseconds &timeout)
Waits for the result to become available. Blocks until specified timeout has elapsed or the result becomes available, whichever comes first.
- Parameters
timeout – - maximum duration in milliseconds to block for
- Returns
A true if results are ready.
-
virtual void cancel()
Cancel current inference request execution.
-
virtual void set_callback(std::function<void(std::exception_ptr)> callback)
Set callback function which will be called on success or failure of asynchronous request.
- Parameters
callback – - function to be called with the following description:
-
virtual void infer() override
Infers specified input(s) in synchronous mode.
Note
blocks all method of InferRequest while request is ongoing (running or waiting in queue)
-
virtual std::vector<ov::ProfilingInfo> get_profiling_info() const override
Queries performance measures per layer to identify the most time consuming operation.
Note
Not all plugins provide meaningful data.
- Returns
Vector of profiling information for operations in a model.
-
virtual ov::SoPtr<ov::ITensor> get_tensor(const ov::Output<const ov::Node> &port) const override
Gets an input/output tensor for inference.
Note
If the tensor with the specified
portis not found, an exception is thrown.- Parameters
port – Port of the tensor to get.
- Returns
Tensor for the port
port.
-
virtual void set_tensor(const ov::Output<const ov::Node> &port, const ov::SoPtr<ov::ITensor> &tensor) override
Sets an input/output tensor to infer.
- Parameters
port – Port of the input or output tensor.
tensor – Reference to a tensor. The element_type and shape of a tensor must match the model’s input/output element_type and size.
-
virtual std::vector<ov::SoPtr<ov::ITensor>> get_tensors(const ov::Output<const ov::Node> &port) const override
Gets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of
tensorsmust match the batch size. The current version supports setting tensors to model inputs only. Ifportis associated with output (or any other non-input node), an exception is thrown.- Parameters
port – Port of the input tensor.
tensors – Input tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.
- Returns
vector of tensors
-
virtual void set_tensors(const ov::Output<const ov::Node> &port, const std::vector<ov::SoPtr<ov::ITensor>> &tensors) override
Sets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of
tensorsmust match the batch size. The current version supports setting tensors to model inputs only. Ifportis associated with output (or any other non-input node), an exception is thrown.- Parameters
port – Port of the input tensor.
tensors – Input tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.
-
virtual std::vector<ov::SoPtr<ov::IVariableState>> query_state() const override
Gets state control interface for the given infer request.
State control essential for recurrent models.
- Returns
Vector of Variable State objects.
-
virtual const std::shared_ptr<const ov::ICompiledModel> &get_compiled_model() const override
Gets pointer to compiled model (usually synchronous request holds the compiled model)
- Returns
Pointer to the compiled model
-
class ICompiledModel : public std::enable_shared_from_this<ICompiledModel>
- #include <icompiled_model.hpp>
OpenVINO ICompiledModel interface.
Public Functions
Constructor for ICompiledModel interface.
- Parameters
model – OpenVINO model representation
plugin – Pointer to plugin
task_executor – Task executor (CPUStreamsExecutor by default)
callback_executor – Callback executor (CPUStreamsExecutor by default)
Constructor for ICompiledModel interface with remote context.
- Parameters
model – OpenVINO model representation
plugin – Pointer to plugin
context – Remote context
task_executor – Task executor (CPUStreamsExecutor by default)
callback_executor – Callback executor (CPUStreamsExecutor by default)
-
virtual const std::vector<ov::Output<const ov::Node>> &outputs() const
Gets all outputs from compiled model.
- Returns
model outputs
-
virtual const std::vector<ov::Output<const ov::Node>> &inputs() const
Gets all inputs from compiled model.
- Returns
model inputs
-
virtual std::shared_ptr<ov::IAsyncInferRequest> create_infer_request() const
Create infer request.
- Returns
Asynchronous infer request interface
-
virtual void export_model(std::ostream &model) const = 0
Export compiled model to stream.
- Parameters
model – output stream
-
virtual std::shared_ptr<const ov::Model> get_runtime_model() const = 0
Returns runtime model.
- Returns
OpenVINO Model which represents runtime graph
-
virtual void set_property(const ov::AnyMap &properties) = 0
Allows to set property.
- Parameters
properties – new plugin properties
-
ov::SoPtr<ov::IRemoteContext> get_context() const
Creates device specific remote context.
- Returns
OpenVINO RemoteContext
-
interface ICore
- #include <icore.hpp>
Minimal ICore interface to allow plugin to get information from Core OpenVINO class.
Subclassed by InferenceEngine::ICore
Public Functions
-
virtual std::shared_ptr<ov::Model> read_model(const std::string &model, const ov::Tensor &weights, bool frontend_mode = false) const = 0
Reads IR xml and bin (with the same name) files.
- Parameters
model – string with IR
weights – shared pointer to constant blob with weights
frontend_mode – read network without post-processing or other transformations
- Returns
shared pointer to ov::Model
-
virtual std::shared_ptr<ov::Model> read_model(const std::string &model_path, const std::string &bin_path) const = 0
Reads IR xml and bin files.
- Parameters
model_path – path to IR file
bin_path – path to bin file, if path is empty, will try to read bin file with the same name as xml and if bin file with the same name was not found, will load IR without weights.
- Returns
shared pointer to ov::Model
Creates a compiled mdel from a model object.
Users can create as many models as they need and use them simultaneously (up to the limitation of the hardware resources)
- Parameters
model – OpenVINO Model
device_name – Name of device to load model to
config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation
- Returns
A pointer to compiled model
Creates a compiled model from a model object.
Users can create as many models as they need and use them simultaneously (up to the limitation of the hardware resources)
- Parameters
model – OpenVINO Model
context – “Remote” (non-CPU) accelerator device-specific execution context to use
config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation
- Returns
A pointer to compiled model
-
virtual ov::SoPtr<ov::ICompiledModel> compile_model(const std::string &model_path, const std::string &device_name, const ov::AnyMap &config) const = 0
Creates a compiled model from a model file.
Users can create as many models as they need and use them simultaneously (up to the limitation of the hardware resources)
- Parameters
model_path – Path to model
device_name – Name of device to load model to
config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation
- Returns
A pointer to compiled model
-
virtual ov::SoPtr<ov::ICompiledModel> compile_model(const std::string &model_str, const ov::Tensor &weights, const std::string &device_name, const ov::AnyMap &config) const = 0
Creates a compiled model from a model memory.
Users can create as many models as they need and use them simultaneously (up to the limitation of the hardware resources)
- Parameters
model_str – String data of model
weights – Model’s weights
device_name – Name of device to load model to
config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation
- Returns
A pointer to compiled model
-
virtual ov::SoPtr<ov::ICompiledModel> import_model(std::istream &model, const std::string &device_name, const ov::AnyMap &config = {}) const = 0
Creates a compiled model from a previously exported model.
- Parameters
model – model stream
device_name – Name of device load executable model on
config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation*
- Returns
A pointer to compiled model
-
virtual ov::SoPtr<ov::ICompiledModel> import_model(std::istream &modelStream, const ov::SoPtr<ov::IRemoteContext> &context, const ov::AnyMap &config = {}) const = 0
Creates a compiled model from a previously exported model.
- Parameters
model – model stream
context – Remote context
config – Optional map of pairs: (config parameter name, config parameter value) relevant only for this load operation*
- Returns
A pointer to compiled model
Query device if it supports specified network with specified configuration.
- Parameters
model – OpenVINO Model
device_name – A name of a device to query
config – Optional map of pairs: (config parameter name, config parameter value)
- Returns
An object containing a map of pairs a layer name -> a device name supporting this layer.
-
virtual std::vector<std::string> get_available_devices() const = 0
Returns devices available for neural networks inference.
- Returns
A vector of devices. The devices are returned as { CPU, GPU.0, GPU.1, MYRIAD } If there more than one device of specific type, they are enumerated with .# suffix.
-
virtual ov::SoPtr<ov::IRemoteContext> create_context(const std::string &device_name, const AnyMap &args) const = 0
Create a new shared context object on specified accelerator device using specified plugin-specific low level device API parameters (device handle, pointer, etc.)
- Parameters
device_name – Name of a device to create new shared context on.
params – Map of device-specific shared context parameters.
- Returns
A shared pointer to a created remote context.
-
virtual ov::SoPtr<ov::IRemoteContext> get_default_context(const std::string &device_name) const = 0
Get a pointer to default shared context object for the specified device.
- Parameters
device_name – - A name of a device to get create shared context from.
- Returns
A shared pointer to a default remote context.
-
virtual Any get_property(const std::string &device_name, const std::string &name, const AnyMap &arguments) const = 0
Gets properties related to device behaviour.
- Parameters
device_name – Name of a device to get a property value.
name – Property name.
arguments – Additional arguments to get a property.
- Returns
Value of a property corresponding to the property name.
-
template<typename T, PropertyMutability M>
inline T get_property(const std::string &device_name, const Property<T, M> &property) const Gets properties related to device behaviour.
-
template<typename T, PropertyMutability M>
inline T get_property(const std::string &device_name, const Property<T, M> &property, const AnyMap &arguments) const Gets properties related to device behaviour.
-
virtual AnyMap get_supported_property(const std::string &full_device_name, const AnyMap &properties) const = 0
Get only properties that are supported by specified device.
- Parameters
full_device_name – Name of a device (can be either virtual or hardware)
properties – Properties that can contains configs that are not supported by device
- Returns
map of properties that are supported by device
-
virtual ~ICore()
Default virtual destructor.
-
virtual std::shared_ptr<ov::Model> read_model(const std::string &model, const ov::Tensor &weights, bool frontend_mode = false) const = 0
-
class IInferRequest
- #include <iinfer_request.hpp>
An internal API of inference request to be implemented by plugin.
Subclassed by ov::IAsyncInferRequest, ov::ISyncInferRequest
Public Functions
-
virtual void infer() = 0
Infers specified input(s) in synchronous mode.
Note
blocks all method of InferRequest while request is ongoing (running or waiting in queue)
-
virtual std::vector<ov::ProfilingInfo> get_profiling_info() const = 0
Queries performance measures per layer to identify the most time consuming operation.
Note
Not all plugins provide meaningful data.
- Returns
Vector of profiling information for operations in a model.
-
virtual ov::SoPtr<ov::ITensor> get_tensor(const ov::Output<const ov::Node> &port) const = 0
Gets an input/output tensor for inference.
Note
If the tensor with the specified
portis not found, an exception is thrown.- Parameters
port – Port of the tensor to get.
- Returns
Tensor for the port
port.
-
virtual void set_tensor(const ov::Output<const ov::Node> &port, const ov::SoPtr<ov::ITensor> &tensor) = 0
Sets an input/output tensor to infer.
- Parameters
port – Port of the input or output tensor.
tensor – Reference to a tensor. The element_type and shape of a tensor must match the model’s input/output element_type and size.
-
virtual std::vector<ov::SoPtr<ov::ITensor>> get_tensors(const ov::Output<const ov::Node> &port) const = 0
Gets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of
tensorsmust match the batch size. The current version supports setting tensors to model inputs only. Ifportis associated with output (or any other non-input node), an exception is thrown.- Parameters
port – Port of the input tensor.
tensors – Input tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.
- Returns
vector of tensors
-
virtual void set_tensors(const ov::Output<const ov::Node> &port, const std::vector<ov::SoPtr<ov::ITensor>> &tensors) = 0
Sets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of
tensorsmust match the batch size. The current version supports setting tensors to model inputs only. Ifportis associated with output (or any other non-input node), an exception is thrown.- Parameters
port – Port of the input tensor.
tensors – Input tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.
-
virtual std::vector<ov::SoPtr<ov::IVariableState>> query_state() const = 0
Gets state control interface for the given infer request.
State control essential for recurrent models.
- Returns
Vector of Variable State objects.
-
virtual const std::shared_ptr<const ov::ICompiledModel> &get_compiled_model() const = 0
Gets pointer to compiled model (usually synchronous request holds the compiled model)
- Returns
Pointer to the compiled model
-
virtual void infer() = 0
-
template<typename AT, typename VAT>
class IndirectScalarValueAccessor : public ov::ValueAccessor<VAT> - #include <attribute_adapter.hpp>
-
template<typename AT, typename VAT>
class IndirectVectorValueAccessor : public ov::ValueAccessor<VAT> - #include <attribute_adapter.hpp>
-
class InferRequest
- #include <infer_request.hpp>
This is a class of infer request that can be run in asynchronous or synchronous manners.
Public Functions
-
InferRequest() = default
Default constructor.
-
InferRequest(const InferRequest &other) = default
Default copy constructor.
- Parameters
other – Another InferRequest object.
-
InferRequest &operator=(const InferRequest &other) = default
Default copy assignment operator.
- Parameters
other – Another InferRequest object.
- Returns
Reference to the current object.
-
InferRequest(InferRequest &&other) = default
Default move constructor.
- Parameters
other – Another InferRequest object.
-
InferRequest &operator=(InferRequest &&other) = default
Default move assignment operator.
- Parameters
other – Another InferRequest object.
- Returns
Reference to the current object.
-
~InferRequest()
Destructor that preserves unloading order of implementation object and reference to the library.
Note
To preserve destruction order inside the default generated assignment operator,
_implis stored before_so. Use the destructor to remove implementation object before referencing to the library explicitly.
-
void set_tensor(const std::string &tensor_name, const Tensor &tensor)
Sets an input/output tensor to infer on.
- Parameters
tensor_name – Name of the input or output tensor.
tensor – Reference to the tensor. The element_type and shape of the tensor must match the model’s input/output element_type and size.
-
void set_tensor(const ov::Output<const ov::Node> &port, const Tensor &tensor)
Sets an input/output tensor to infer.
- Parameters
port – Port of the input or output tensor. Use the following methods to get the ports:
tensor – Reference to a tensor. The element_type and shape of a tensor must match the model’s input/output element_type and size.
-
void set_tensor(const ov::Output<ov::Node> &port, const Tensor &tensor)
Sets an input/output tensor to infer.
- Parameters
port – Port of the input or output tensor. Use the following methods to get the ports:
tensor – Reference to a tensor. The element_type and shape of a tensor must match the model’s input/output element_type and size.
-
void set_tensors(const std::string &tensor_name, const std::vector<Tensor> &tensors)
Sets a batch of tensors for input data to infer by tensor name. Model input must have batch dimension, and the number of
tensorsmust match the batch size. The current version supports setting tensors to model inputs only. Iftensor_nameis associated with output (or any other non-input node), an exception is thrown.- Parameters
tensor_name – Name of the input tensor.
tensors – Input tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.
-
void set_tensors(const ov::Output<const ov::Node> &port, const std::vector<Tensor> &tensors)
Sets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of
tensorsmust match the batch size. The current version supports setting tensors to model inputs only. Ifportis associated with output (or any other non-input node), an exception is thrown.- Parameters
port – Port of the input tensor.
tensors – Input tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.
-
void set_input_tensor(size_t idx, const Tensor &tensor)
Sets an input tensor to infer.
- Parameters
idx – Index of the input tensor. If
idxis greater than the number of model inputs, an exception is thrown.tensor – Reference to the tensor. The element_type and shape of the tensor must match the model’s input/output element_type and size.
-
void set_input_tensor(const Tensor &tensor)
Sets an input tensor to infer models with single input.
Note
If model has several inputs, an exception is thrown.
- Parameters
tensor – Reference to the input tensor.
-
void set_input_tensors(const std::vector<Tensor> &tensors)
Sets a batch of tensors for single input data. Model input must have batch dimension, and the number of
tensorsmust match the batch size.- Parameters
tensors – Input tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.
-
void set_input_tensors(size_t idx, const std::vector<Tensor> &tensors)
Sets a batch of tensors for input data to infer by input name. Model input must have batch dimension, and number of
tensorsmust match the batch size.- Parameters
idx – Name of the input tensor.
tensors – Input tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.
-
void set_output_tensor(size_t idx, const Tensor &tensor)
Sets an output tensor to infer.
Note
Index of the input preserved accross ov::Model, ov::CompiledModel, and ov::InferRequest.
- Parameters
idx – Index of the output tensor.
tensor – Reference to the output tensor. The type of the tensor must match the model output element type and shape.
-
void set_output_tensor(const Tensor &tensor)
Sets an output tensor to infer models with single output.
Note
If model has several outputs, an exception is thrown.
- Parameters
tensor – Reference to the output tensor.
-
Tensor get_tensor(const std::string &tensor_name)
Gets an input/output tensor for inference by tensor name.
- Parameters
tensor_name – Name of a tensor to get.
- Returns
The tensor with name
tensor_name. If the tensor is not found, an exception is thrown.
-
Tensor get_tensor(const ov::Output<const ov::Node> &port)
Gets an input/output tensor for inference.
Note
If the tensor with the specified
portis not found, an exception is thrown.- Parameters
port – Port of the tensor to get.
- Returns
Tensor for the port
port.
-
Tensor get_tensor(const ov::Output<ov::Node> &port)
Gets an input/output tensor for inference.
Note
If the tensor with the specified
portis not found, an exception is thrown.- Parameters
port – Port of the tensor to get.
- Returns
Tensor for the port
port.
-
Tensor get_input_tensor(size_t idx)
Gets an input tensor for inference.
- Parameters
idx – Index of the tensor to get.
- Returns
Tensor with the input index
idx. If the tensor with the specifiedidxis not found, an exception is thrown.
-
Tensor get_input_tensor()
Gets an input tensor for inference.
- Returns
The input tensor for the model. If model has several inputs, an exception is thrown.
-
Tensor get_output_tensor(size_t idx)
Gets an output tensor for inference.
- Parameters
idx – Index of the tensor to get.
- Returns
Tensor with the output index
idx. If the tensor with the specifiedidxis not found, an exception is thrown.
-
Tensor get_output_tensor()
Gets an output tensor for inference.
- Returns
Output tensor for the model. If model has several outputs, an exception is thrown.
-
void infer()
Infers specified input(s) in synchronous mode.
Note
It blocks all methods of InferRequest while request is ongoing (running or waiting in a queue). Calling any method leads to throwning the ov::Busy exception.
-
void cancel()
Cancels inference request.
-
std::vector<ProfilingInfo> get_profiling_info() const
Queries performance measures per layer to identify the most time consuming operation.
Note
Not all plugins provide meaningful data.
- Returns
Vector of profiling information for operations in a model.
-
void start_async()
Starts inference of specified input(s) in asynchronous mode.
Note
It returns immediately. Inference starts also immediately. Calling any method while the request in a running state leads to throwning the ov::Busy exception.
-
void wait()
Waits for the result to become available. Blocks until the result becomes available.
-
bool wait_for(const std::chrono::milliseconds timeout)
Waits for the result to become available. Blocks until the specified timeout has elapsed or the result becomes available, whichever comes first.
- Parameters
timeout – Maximum duration, in milliseconds, to block for.
- Returns
True if inference request is ready and false, otherwise.
-
void set_callback(std::function<void(std::exception_ptr)> callback)
Sets a callback std::function that is called on success or failure of an asynchronous request.
Warning
Do not capture strong references to OpenVINO runtime objects into callback. Following objects should not be captured like:
ov::ExecutableNetwork
ov::Core As specified objects implement shared reference concept do not capture this objects by value. It can lead to memory leaks or undefined behaviour! Try to use weak references or pointers.
- Parameters
callback – callback object which will be called on when inference finish.
-
std::vector<VariableState> query_state()
Gets state control interface for the given infer request.
State control essential for recurrent models.
- Returns
Vector of Variable State objects.
-
void reset_state()
Resets all internal variable states for relevant infer request to a value specified as default for the corresponding
ReadValuenode.
-
CompiledModel get_compiled_model()
Returns a compiled model that creates this inference request.
- Returns
Compiled model object.
-
bool operator!() const noexcept
Checks if the current InferRequest object is not initialized.
- Returns
True if the current InferRequest object is not initialized; false, otherwise.
-
explicit operator bool() const noexcept
Checks if the current InferRequest object is initialized.
- Returns
True if the current InferRequest object is initialized; false, otherwise.
-
bool operator!=(const InferRequest &other) const noexcept
Compares whether this request wraps the same impl underneath.
- Parameters
other – Another inference request.
- Returns
True if the current InferRequest object does not wrap the same impl as the operator’s arg.
-
bool operator==(const InferRequest &other) const noexcept
Compares whether this request wraps the same impl underneath.
- Parameters
other – Another inference request.
- Returns
True if the current InferRequest object wraps the same impl as the operator’s arg.
-
InferRequest() = default
-
template<typename NodeType>
class Input - #include <node_input.hpp>
-
template<>
class Input<const Node> - #include <node_input.hpp>
A handle for one of a node’s inputs.
Public Functions
-
Input(const Node *node, size_t index)
Constructs a Input.
- Parameters
node – Pointer to the node for the input handle.
index – The index of the input.
-
const Node *get_node() const
- Returns
A pointer to the node referenced by this input handle.
-
size_t get_index() const
- Returns
The index of the input referred to by this input handle.
-
const element::Type &get_element_type() const
- Returns
The element type of the input referred to by this input handle.
-
const Shape &get_shape() const
- Returns
The shape of the input referred to by this input handle.
-
const PartialShape &get_partial_shape() const
- Returns
The partial shape of the input referred to by this input handle.
-
Output<Node> get_source_output() const
- Returns
A handle to the output that is connected to this input.
-
descriptor::Tensor &get_tensor() const
- Returns
A reference to the tensor descriptor for this input.
-
std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
- Returns
A shared pointer to the tensor descriptor for this input.
-
bool get_is_relevant_to_shapes() const
- Returns
true if this input is relevant to its node’s output shapes; else false.
-
bool get_is_relevant_to_values() const
- Returns
true if this input is relevant to its node’s output values; else false.
-
const RTMap &get_rt_info() const
- Returns
The constant reference to runtime info map
-
Input(const Node *node, size_t index)
-
template<>
class Input<Node> - #include <node_input.hpp>
A handle for one of a node’s inputs.
Public Functions
-
Input(Node *node, size_t index)
Constructs a Input.
- Parameters
node – Pointer to the node for the input handle.
index – The index of the input.
-
Node *get_node() const
- Returns
A pointer to the node referenced by this input handle.
-
size_t get_index() const
- Returns
The index of the input referred to by this input handle.
-
const element::Type &get_element_type() const
- Returns
The element type of the input referred to by this input handle.
-
const Shape &get_shape() const
- Returns
The shape of the input referred to by this input handle.
-
const PartialShape &get_partial_shape() const
- Returns
The partial shape of the input referred to by this input handle.
-
Output<Node> get_source_output() const
- Returns
A handle to the output that is connected to this input.
-
descriptor::Tensor &get_tensor() const
- Returns
A reference to the tensor descriptor for this input.
-
std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
- Returns
A shared pointer to the tensor descriptor for this input.
-
bool get_is_relevant_to_shapes() const
- Returns
true if this input is relevant to its node’s output shapes; else false.
-
bool get_is_relevant_to_values() const
- Returns
true if this input is relevant to its node’s output values; else false.
-
void replace_source_output(const Output<Node> &new_source_output) const
Replaces the source output of this input.
- Parameters
new_source_output – A handle for the output that will replace this input’s source.
-
RTMap &get_rt_info()
- Returns
The reference to runtime info map
-
const RTMap &get_rt_info() const
- Returns
The constant reference to runtime info map
-
Input(Node *node, size_t index)
-
class Interval
- #include <interval.hpp>
Interval arithmetic.
An interval is the set of integers from m_min_val through m_max_val. The value s_max acts like infinity. The addition, subtraction, or multiplication of intervals is the smallest interval containing the sums, differences, or products of elements of the two intervals. An empty interval is canonicalized to [s_max, s_max].
Public Functions
-
Interval() = default
Interval of everything.
-
Interval(const Interval &interval) = default
Copy constructor.
-
Interval(value_type min_val, value_type max_val)
Closed interval {x|min_val <= x <= max_val}.
-
Interval(value_type val)
Single-valued interval; just contains val.
-
inline size_type size() const
The number of elements in the interval. Zero if max < min.
-
inline bool empty() const
Returns true if the interval has no elements.
-
inline value_type get_min_val() const
the inclusive lower bound of the interval
-
inline void set_min_val(value_type val)
Set the inclusive lower bound of the interval.
-
inline value_type get_max_val() const
the inclusive upper bound of the interval
-
inline void set_max_val(value_type val)
Set the inclusive upper bound of the interval.
-
inline bool has_upper_bound() const
True if the upper bound is finite.
-
bool operator==(const Interval &interval) const
True if min and max bounds match.
-
Interval operator+(const Interval &interval) const
The interval whose elements are a sum of an element from each interval.
-
Interval &operator+=(const Interval &interval)
Extend this interval to sums of elements in this interval and interval.
-
Interval operator-(const Interval &interval) const
The interval whose elements are a difference of an element from each interval.
-
Interval &operator-=(const Interval &interval)
Extend this interval to differences of elements in this interval and interval.
-
Interval operator*(const Interval &interval) const
The smallest interval whose elements are a product of an element from each interval.
-
Interval &operator*=(const Interval &interval)
Extend this interval to products of elements in this interval and interval.
-
Interval operator&(const Interval &interval) const
The interval that is the intersection of this interval and interval.
-
Interval &operator&=(const Interval &interval)
Change this interval to only include elements also in interval.
-
inline bool contains(value_type value) const
True if this interval includes value.
-
bool contains(const Interval &interval) const
True if this interval includes all the values in interval.
Public Static Attributes
-
static constexpr value_type s_max = {std::numeric_limits<value_type>::max()}
The value used for no upper bound.
-
Interval() = default
-
class IntervalsAlignmentAttribute : public SharedAttribute<IntervalsAlignmentSharedValue>
- #include <intervals_alignment_attribute.hpp>
IntervalsAlignmentAttribute defines subgraph with the same quantization intervals alignment. FakeQuantize operations are included. The attribute is used by quantization operations.
For more details about the attribute, refer to IntervalsAlignmentAttribute page in the Inference Engine Developer Guide.
- #include <intervals_alignment_attribute.hpp>
IntervalsAlignmentSharedValue is used by IntervalsAlignmentAttribute as attribute shared value.
- #include <intervals_alignment_attribute.hpp>
-
class IPlugin : public std::enable_shared_from_this<IPlugin>
- #include <iplugin.hpp>
OpenVINO Plugin Interface 2.0.
Public Functions
-
void set_version(const Version &version)
Sets a plugin version.
- Parameters
version – A version to set
-
const Version &get_version() const
Returns a plugin version.
- Returns
A constant ov::Version object
Compiles model from ov::Model object.
- Parameters
model – A model object acquired from ov::Core::read_model or source construction
properties – A ov::AnyMap of properties relevant only for this load operation
- Returns
Created Compiled Model object
-
virtual std::shared_ptr<ov::ICompiledModel> compile_model(const std::string &model_path, const ov::AnyMap &properties) const
Compiles model from ov::Model object.
- Parameters
model_path – A path to model (path can be converted from unicode representation)
properties – A ov::AnyMap of properties relevant only for this load operation
- Returns
Created Compiled Model object
Compiles model from ov::Model object, on specified remote context.
- Parameters
model – A model object acquired from ov::Core::read_model or source construction
properties – A ov::AnyMap of properties relevant only for this load operation
context – A pointer to plugin context derived from RemoteContext class used to execute the model
- Returns
Created Compiled Model object
-
virtual void set_property(const ov::AnyMap &properties) = 0
Sets properties for plugin, acceptable keys can be found in openvino/runtime/properties.hpp.
- Parameters
properties – ov::AnyMap of properties
-
virtual ov::Any get_property(const std::string &name, const ov::AnyMap &arguments) const = 0
Gets properties related to plugin behaviour.
- Parameters
name – Property name.
arguments – Additional arguments to get a property.
- Returns
Value of a property corresponding to the property name.
-
virtual ov::SoPtr<ov::IRemoteContext> create_context(const ov::AnyMap &remote_properties) const = 0
Creates a remote context instance based on a map of properties.
- Parameters
remote_properties – Map of device-specific shared context remote properties.
- Returns
A remote context object
-
virtual ov::SoPtr<ov::IRemoteContext> get_default_context(const ov::AnyMap &remote_properties) const = 0
Provides a default remote context instance if supported by a plugin.
- Parameters
remote_properties – Map of device-specific shared context remote properties.
- Returns
The default context.
-
virtual std::shared_ptr<ov::ICompiledModel> import_model(std::istream &model, const ov::AnyMap &properties) const = 0
Creates an compiled model from an previously exported model using plugin implementation and removes OpenVINO Runtime magic and plugin name.
- Parameters
model – Reference to model output stream
properties – A ov::AnyMap of properties
- Returns
An Compiled model
-
virtual std::shared_ptr<ov::ICompiledModel> import_model(std::istream &model, const ov::SoPtr<ov::IRemoteContext> &context, const ov::AnyMap &properties) const = 0
Creates an compiled model from an previously exported model using plugin implementation and removes OpenVINO Runtime magic and plugin name.
- Parameters
model – Reference to model output stream
context – A pointer to plugin context derived from RemoteContext class used to execute the network
properties – A ov::AnyMap of properties
- Returns
An Compiled model
Queries a plugin about supported layers in model.
- Parameters
model – Model object to query.
properties – Optional map of pairs: (property name, property value).
- Returns
An object containing a map of pairs an operation name -> a device name supporting this operation.
- virtual OPENVINO_SUPPRESS_DEPRECATED_START void add_extension (const std::shared_ptr< InferenceEngine::IExtension > &extension)
Registers legacy extension within plugin.
- Deprecated:
This method allows to load legacy Inference Engine Extensions and will be removed in 2024.0 release
- Parameters
extension – - pointer to already loaded legacy extension
- OPENVINO_SUPPRESS_DEPRECATED_END void set_core (const std::weak_ptr< ov::ICore > &core)
Sets pointer to ICore interface.
- Parameters
core – Pointer to Core interface
-
std::shared_ptr<ov::ICore> get_core() const
Gets reference to ICore interface.
- Returns
Reference to ICore interface
-
bool is_new_api() const
Provides an information about used API.
- Returns
true if new API is used
-
const std::shared_ptr<ov::threading::ExecutorManager> &get_executor_manager() const
Gets reference to tasks execution manager.
- Returns
Reference to ExecutorManager interface
-
void set_version(const Version &version)
-
class IRemoteContext : public std::enable_shared_from_this<IRemoteContext>
- #include <iremote_context.hpp>
Subclassed by InferenceEngine::IRemoteContextWrapper
Public Functions
-
virtual const ov::AnyMap &get_property() const = 0
Returns a map of device-specific parameters required for low-level operations with underlying object. Parameters include device/context handles, access flags, etc. Contents of the map returned depend on remote execution context that is currently set on the device (working scenario). Abstract method.
- Returns
A map of name/Any elements.
-
virtual ov::SoPtr<ov::IRemoteTensor> create_tensor(const ov::element::Type &type, const ov::Shape &shape, const ov::AnyMap ¶ms = {}) = 0
Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.
- Parameters
type – Defines the element type of the tensor.
shape – Defines the shape of the tensor.
params – Map of the low-level tensor object parameters.
- Returns
Pointer to a plugin object that implements the RemoteTensor interface.
-
virtual ov::SoPtr<ov::ITensor> create_host_tensor(const ov::element::Type type, const ov::Shape &shape)
This method is used to create a host tensor object friendly for the device in current context. For example, GPU context may allocate USM host memory (if corresponding extension is available), which could be more efficient than regular host memory.
-
virtual const ov::AnyMap &get_property() const = 0
-
class IRemoteTensor : public ITensor
- #include <iremote_tensor.hpp>
Subclassed by ov::RemoteBlobTensor
Public Functions
-
virtual const AnyMap &get_properties() const = 0
Returns additional information associated with tensor.
- Returns
Map of property names to properties
-
virtual const AnyMap &get_properties() const = 0
-
class ISyncInferRequest : public ov::IInferRequest
- #include <isync_infer_request.hpp>
Interface for syncronous infer request.
Public Functions
Constructs syncronous inference request.
- Parameters
compiled_model – pointer to compiled model
-
virtual ov::SoPtr<ov::ITensor> get_tensor(const ov::Output<const ov::Node> &port) const override
Gets an input/output tensor for inference.
Note
If the tensor with the specified
portis not found, an exception is thrown.- Parameters
port – Port of the tensor to get.
- Returns
Tensor for the port
port.
-
virtual void set_tensor(const ov::Output<const ov::Node> &port, const ov::SoPtr<ov::ITensor> &tensor) override
Sets an input/output tensor to infer.
- Parameters
port – Port of the input or output tensor.
tensor – Reference to a tensor. The element_type and shape of a tensor must match the model’s input/output element_type and size.
-
virtual std::vector<ov::SoPtr<ov::ITensor>> get_tensors(const ov::Output<const ov::Node> &port) const override
Gets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of
tensorsmust match the batch size. The current version supports setting tensors to model inputs only. Ifportis associated with output (or any other non-input node), an exception is thrown.- Parameters
port – Port of the input tensor.
tensors – Input tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.
- Returns
vector of tensors
-
virtual void set_tensors(const ov::Output<const ov::Node> &port, const std::vector<ov::SoPtr<ov::ITensor>> &tensors) override
Sets a batch of tensors for input data to infer by input port. Model input must have batch dimension, and the number of
tensorsmust match the batch size. The current version supports setting tensors to model inputs only. Ifportis associated with output (or any other non-input node), an exception is thrown.- Parameters
port – Port of the input tensor.
tensors – Input tensors for batched infer request. The type of each tensor must match the model input element type and shape (except batch dimension). Total size of tensors must match the input size.
-
virtual const std::vector<ov::Output<const ov::Node>> &get_inputs() const override
Gets inputs for infer request.
- Returns
vector of input ports
-
virtual const std::vector<ov::Output<const ov::Node>> &get_outputs() const override
Gets outputs for infer request.
- Returns
vector of output ports
-
virtual const std::shared_ptr<const ov::ICompiledModel> &get_compiled_model() const override
Gets pointer to compiled model (usually synchronous request holds the compiled model)
- Returns
Pointer to the compiled model
-
class ITensorAccessor
- #include <tensor_data_accessor.hpp>
Interface for data accessor.
Subclassed by ov::TensorAccessor< TContainer >
-
interface IVariableState : public std::enable_shared_from_this<IVariableState>
- #include <ivariable_state.hpp>
Minimal interface for variable state implementation.
Public Functions
-
virtual const std::string &get_name() const
Gets a variable state name.
- Returns
A string representing variable state name
-
virtual void reset()
Reset internal variable state for relevant infer request, to a value specified as default for according
ReadValuenode.
-
virtual const std::string &get_name() const
-
class KeepConstPrecision : public ov::RuntimeAttribute
- #include <keep_const_precision.hpp>
KeepConstPrecision class represents runtime info attribute that marks a Constant as prohibitted to fuse precision in ConvertPrecision.
-
class Layout
- #include <layout.hpp>
ov::Layout represents the text information of tensor’s dimensions/axes. E.g. layout
NCHWmeans that 4D tensor{-1, 3, 480, 640}will have:0:
N = -1: batch dimension is dynamic1:
C = 3: number of channels is ‘3’2:
H = 480: image height is 4803:
W = 640: image width is 640
Examples:
ov::Layoutcan be specified for:Preprocessing purposes. E.g.
To apply normalization (means/scales) it is usually required to set ‘C’ dimension in a layout.
To resize the image to specified width/height it is needed to set ‘H’ and ‘W’ dimensions in a layout
To transpose image - source and target layout can be set (see
ov::preprocess::PreProcessSteps::convert_layout)
To set/get model’s batch (see
ov::get_batch/ov::set_batch) it is required in general to specify ‘N’ dimension in layout for appropriate inputs
Refer also to
ov::layoutnamespace for various additional helper functions ofov::LayoutPublic Functions
-
Layout()
Constructs a dynamic Layout with no layout information.
-
inline Layout(const char *layoutStr)
Constructs a Layout with static or dynamic layout information based on string representation.
- Parameters
layoutStr – The string used to construct Layout from. The string representation can be in the following form:
can define order and meaning for dimensions “NCHW”
partial layout specialization:
”NC?” defines 3 dimensional layout, first two NC, 3rd one is not defined
”N…C” defines layout with dynamic rank where 1st dimension is N, last one is C
”NC…” defines layout with dynamic rank where first two are NC, others are not defined
only order of dimensions “adbc” (0312)
Advanced syntax can be used for multi-character names like “[N,C,H,W,…,CustomName]”
-
bool operator==(const Layout &rhs) const
Comparison operator (equal)
-
bool operator!=(const Layout &rhs) const
Comparison operator (not equal)
-
bool has_name(const std::string &dimensionName) const
Checks if dimension with specified name is in layout.
- Returns
trueif layout has information about dimension index with a given name
-
std::int64_t get_index_by_name(const std::string &dimensionName) const
Gets index of dimension with a specified name.
- Throws
ov::AssertFailure – if dimension name is not found in a layout
- Returns
Index of given dimension name
-
std::string to_string() const
String representation of Layout.
-
inline bool empty() const
Returns ‘true’ if layout has no information, i.e. equals to Layout()
Public Static Functions
-
static Layout scalar()
Constructs layout representing scalar.
-
class LayoutAttribute : public ov::RuntimeAttribute
- #include <layout.hpp>
-
class MappedMemory
- #include <mmap_object.hpp>
This class represents a mapped memory. Instead of reading files, we can map the memory via mmap for Linux or MapViewOfFile for Windows. The MappedMemory class is a abstraction to handle such memory with os-dependent details.
-
class Mask : public std::vector<std::set<uint64_t>>, public std::enable_shared_from_this<Mask>
- #include <mask_attribute.hpp>
each element in vector represents dimension and each element in set is an id of dimensions which contains zeros.
-
struct MemBandwidthPressure
- #include <performance_heuristics.hpp>
-
class MemorySolver
- #include <memory_solver.hpp>
Helps to solve issue of optimal memory allocation only for particular execution order.
It works with abstract data description where
Example:
Mem(offset) | |____| Box {4, 5} | |_____________| Box {2, 6} | |____| Box {3, 4} | |____| Box {2, 3} | |____| Box {6, 7} |_____________________________________ 1 2 3 4 5 6 7 8 9 ExecOrder
Boxes which has an ExecOrder-axis intersection should have no Mem-axis intersections. The goal is to define a minimal required memory blob to store all boxes with such constraints and specify all corresponding position on Mem axis(through offset field).
NOTE! Exec order is predefined.
Public Functions
-
inline int64_t solve()
Solve memory location with maximal reuse.
- Returns
Size of common memory blob required for storing all
-
inline int64_t get_offset(int id) const
Provides calculated offset for specified box id
-
inline int64_t max_depth()
Additional info. Max sum of box sizes required for any time stamp.
-
inline int64_t max_top_depth()
Additional info. Max num of boxes required for any time stamp.
Public Static Functions
-
static inline int normalize_boxes(std::vector<Box> &boxes)
Performes inplace normalization of the input boxes.
- Returns
lifespan of all boxes
-
struct Box
- #include <memory_solver.hpp>
Representation of edge (size and live time)
Public Members
-
int start
Execution order index of first use. The data will be produced here.
-
int finish
The execution order index of last use. After that data will be released. -1 is a reserved value for “till to end”. The data will be alive to very end of execution.
-
int64_t size
Size of data. In abstract unit of measure (byte, simd, cache line, …)
-
int64_t id
Box identifier, unique for each box. Will be used to querying calculated offset.
-
int start
-
inline int64_t solve()
-
class Model : public std::enable_shared_from_this<Model>
- #include <model.hpp>
A user-defined model.
Public Functions
-
explicit Model(const ov::OutputVector &results, const std::string &name = "")
Constructs a Model. Lists of parameters and variables will be generated automatically based on traversing the graph from the results.
-
Model(const ov::OutputVector &results, const ov::SinkVector &sinks, const std::string &name = "")
Constructs a Model. Lists of parameters and variables will be generated automatically based on traversing the graph from the results and the sinks.
-
size_t get_output_size() const
Return the number of outputs for this Model.
-
const ov::element::Type &get_output_element_type(size_t i) const
Return the element type of output i.
-
const Shape &get_output_shape(size_t i) const
Return the shape of element i.
-
const PartialShape &get_output_partial_shape(size_t i) const
Return the partial shape of element i.
-
const std::string &get_name() const
Get the unique name of the model.
- Returns
A const reference to the model’s unique name.
-
void set_friendly_name(const std::string &name)
Sets a friendly name for a model. This does not overwrite the unique name of the model and is retrieved via get_friendly_name(). Used mainly for debugging.
- Parameters
name – is the friendly name to set
-
const std::string &get_friendly_name() const
Gets the friendly name for a model. If no friendly name has been set via set_friendly_name then the model’s unique name is returned.
- Returns
A const reference to the model’s friendly name.
-
size_t get_graph_size() const
Returns the sum of the size of all nodes in the graph plus the size of all constant data. This has little value beyond comparing the relative size of graphs and should not be considered the actual memory consumption of a graph.
-
bool is_dynamic() const
Returns true if any of the op’s defined in the model contains partial shape.
Replace the
parameter_indexth parameter of the model withparameter.All users of the
parameter_indexth parameter are redirected toparameter, and theparameter_indexth entry in the model parameter list is replaced withparameter.- Parameters
parameter_index – The index of the parameter to replace.
parameter – The parameter to substitute for the
parameter_indexth parameter.
-
inline const ov::ParameterVector &get_parameters() const
Return the model parameters.
-
inline const ov::ResultVector &get_results() const
Return a list of model’s outputs.
Index for parameter, or -1.
-
int64_t get_result_index(const ov::Output<ov::Node> &value) const
Return the index of this model’s Result represented by the “value” Output object. This method returns -1 if an the passed output is not related to the Results of a model.
-
int64_t get_result_index(const ov::Output<const ov::Node> &value) const
Return the index of this model’s Result represented by the “value” Output object. This method returns -1 if an the passed output is not related to the Results of a model.
-
bool evaluate(const ov::HostTensorVector &output_tensors, const ov::HostTensorVector &input_tensors, ov::EvaluationContext &evaluation_context) const
Evaluate the model on inputs, putting results in outputs.
- Deprecated:
Use evaluate with ov::Tensor instead
- Parameters
output_tensors – Tensors for the outputs to compute. One for each result
input_tensors – Tensors for the inputs. One for each inputs.
evaluation_context – Storage of additional settings and attributes that can be used when evaluating the model. This additional information can be shared across nodes.
-
bool evaluate(const ov::HostTensorVector &output_tensors, const ov::HostTensorVector &input_tensors) const
Evaluate the model on inputs, putting results in outputs.
- Deprecated:
Use evaluate with ov::Tensor instead
- Parameters
output_tensors – Tensors for the outputs to compute. One for each result
input_tensors – Tensors for the inputs. One for each inputs.
-
bool evaluate(ov::TensorVector &output_tensors, const ov::TensorVector &input_tensors, ov::EvaluationContext &evaluation_context) const
Evaluate the model on inputs, putting results in outputs.
- Parameters
output_tensors – Tensors for the outputs to compute. One for each result
input_tensors – Tensors for the inputs. One for each inputs.
evaluation_context – Storage of additional settings and attributes that can be used when evaluating the model. This additional information can be shared across nodes.
-
bool evaluate(ov::TensorVector &output_tensors, const ov::TensorVector &input_tensors) const
Evaluate the model on inputs, putting results in outputs.
- Parameters
output_tensors – Tensors for the outputs to compute. One for each result
input_tensors – Tensors for the inputs. One for each inputs.
-
inline const ov::SinkVector &get_sinks() const
Return a list of model’s sinks.
-
void add_sinks(const ov::SinkVector &sinks)
Add new sink nodes to the list. Method doesn’t validate graph, it should be done manually after all changes.
- Parameters
sinks – new sink nodes
Delete sink node from the list of sinks. Method doesn’t delete node from graph.
- Parameters
sink – Sink to delete
-
void add_results(const ov::ResultVector &results)
Add new Result nodes to the list. Method doesn’t validate graph, it should be done manually after all changes.
- Parameters
results – new Result nodes
Delete Result node from the list of results. Method will not delete node from graph.
- Parameters
result – Result node to delete
-
void add_parameters(const ov::ParameterVector ¶ms)
Add new Parameter nodes to the list.
Method doesn’t change or validate graph, it should be done manually. For example, if you want to replace
ReadValuenode byParameter, you should do the following steps:replace node
ReadValuebyParameterin graphcall add_parameter() to add new input to the list
call graph validation to check correctness of changes
- Parameters
params – new Parameter nodes
Delete Parameter node from the list of parameters. Method will not delete node from graph. You need to replace Parameter with other operation manually. Attention: Indexing of parameters can be changed.
Possible use of method is to replace input by variable. For it the following steps should be done:
Parameternode should be replaced byReadValuecall remove_parameter(param) to remove input from the list
check if any parameter indexes are saved/used somewhere, update it for all inputs because indexes can be changed
call graph validation to check all changes
- Parameters
param – Parameter node to delete
-
void add_variables(const ov::op::util::VariableVector &variables)
Add new variables to the list. Method doesn’t validate graph, it should be done manually after all changes.
- Parameters
variables – new variables to add
-
void remove_variable(const ov::op::util::Variable::Ptr &variable)
Delete variable from the list of variables. Method doesn’t delete nodes that used this variable from the graph.
- Parameters
variable – Variable to delete
-
inline const ov::op::util::VariableVector &get_variables() const
Return a list of model’s variables.
-
ov::op::util::Variable::Ptr get_variable_by_id(const std::string &variable_id) const
Return a variable by specified variable_id.
-
inline RTMap &get_rt_info()
Returns a runtime info.
- Returns
reference to ov::AnyMap with runtime info
-
inline const RTMap &get_rt_info() const
Returns a constant runtime info.
- Returns
reference to const ov::AnyMap with runtime info
-
template<class T, class ...Args, typename std::enable_if<!std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(Args... args) const Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.
- Template Parameters
T – the type of returned value
Args – types of variadic arguments
- Parameters
args – path to the runtime attribute
- Returns
constant reference to value from runtime info
-
template<class T, class ...Args, typename std::enable_if<std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(Args... args) const Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.
- Template Parameters
T – the type of returned value
Args – types of variadic arguments
- Parameters
args – path to the runtime attribute
- Returns
constant reference to value from runtime info
-
template<class T, typename std::enable_if<!std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(const std::vector<std::string> &args) const Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.
- Template Parameters
T – the type of returned value
- Parameters
args – vector with path to the runtime attribute
- Returns
constant reference to value from runtime info
-
template<class T, typename std::enable_if<std::is_same<T, ov::Any>::value, bool>::type = true>
inline const T &get_rt_info(const std::vector<std::string> &args) const Returns a runtime attribute for the path, throws an ov::Exception if path doesn’t exist.
- Template Parameters
T – the type of returned value
- Parameters
args – vector with path to the runtime attribute
- Returns
constant reference to value from runtime info
-
template<class ...Args>
inline bool has_rt_info(Args... args) const Checks if given path exists in runtime info.
- Template Parameters
Args – types of variadic arguments
- Parameters
args – path to the runtime attribute
- Returns
true if path exists, otherwise false
-
bool has_rt_info(const std::vector<std::string> &args) const
Checks if given path exists in runtime info.
- Parameters
args – vector with path to the runtime attribute
- Returns
true if path exists, otherwise false
-
template<class T, class ...Args>
inline void set_rt_info(const T &argument, Args... args) Add value inside the runtime info.
- Template Parameters
T – type of new value
Args – types of variadic arguments
- Parameters
argument – value for the runtime info
args – path to the runtime attribute
-
template<class T>
inline void set_rt_info(const T &argument, const std::vector<std::string> &args) Add value inside the runtime info.
- Template Parameters
T – type of new value
- Parameters
argument – value for the runtime info
args – vector with path to the runtime attribute
-
explicit Model(const ov::OutputVector &results, const std::string &name = "")
-
class NmsSelectedIndices : private ov::RuntimeAttribute
- #include <nms_selected_indices.hpp>
-
class Node : public std::enable_shared_from_this<Node>
- #include <node.hpp>
Nodes are the backbone of the graph of Value dataflow. Every node has zero or more nodes as arguments and one value, which is either a tensor or a (possibly empty) tuple of values.
Subclassed by ov::op::Op, ov::pass::pattern::op::Pattern
Public Functions
-
virtual void validate_and_infer_types()
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual const ov::op::AutoBroadcastSpec &get_autob() const
- Returns
the autobroadcasr spec
-
virtual bool has_evaluate() const
Allows to get information about availability of evaluate method for the current operation.
-
virtual bool evaluate(const ov::HostTensorVector &output_values, const ov::HostTensorVector &input_values) const
Evaluates the op on input_values putting results in output_values.
- Deprecated:
Use evaluate with ov::Tensor instead
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
- Returns
true if successful
-
virtual bool evaluate(const ov::HostTensorVector &output_values, const ov::HostTensorVector &input_values, const EvaluationContext &evaluationContext) const
Evaluates the op on input_values putting results in output_values.
- Deprecated:
Use evaluate with ov::Tensor instead
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
evaluation_context – Storage of additional settings and attributes that can be used when evaluating the op.
- Returns
true if successful
-
virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values) const
Evaluates the op on input_values putting results in output_values.
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
- Returns
true if successful
-
virtual bool evaluate(ov::TensorVector &output_values, const ov::TensorVector &input_values, const ov::EvaluationContext &evaluationContext) const
Evaluates the op on input_values putting results in output_values.
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
evaluation_context – Storage of additional settings and attributes that can be used when evaluating the op.
- Returns
true if successful
-
inline virtual OutputVector decompose_op() const
Decomposes the FusedOp into a sub-graph consisting of core openvino ops.
- Returns
A vector of nodes comprising the sub-graph. The order of output tensors must match the match output tensors of the FusedOp
-
virtual const type_info_t &get_type_info() const = 0
Returns the NodeTypeInfo for the node’s class. During transition to type_info, returns a dummy type_info for Node if the class has not been updated yet.
-
void set_arguments(const NodeVector &arguments)
Sets/replaces the arguments with new arguments.
-
void set_arguments(const OutputVector &arguments)
Sets/replaces the arguments with new arguments.
-
void set_argument(size_t position, const Output<Node> &argument)
Sets/replaces the arguments with new arguments.
-
void set_output_size(size_t output_size)
Sets the number of outputs.
-
virtual std::string description() const
Get the string name for the type of the node, such as
AddorMultiply. The class name, must not contain spaces as it is used for codegen.- Returns
A const reference to the node’s type name
-
const std::string &get_name() const
Get the unique name of the node.
- Returns
A const reference to the node’s unique name.
-
void set_friendly_name(const std::string &name)
Sets a friendly name for a node. This does not overwrite the unique name of the node and is retrieved via get_friendly_name(). Used mainly for debugging. The friendly name may be set exactly once.
- Parameters
name – is the friendly name to set
-
const std::string &get_friendly_name() const
Gets the friendly name for a node. If no friendly name has been set via set_friendly_name then the node’s unique name is returned.
- Returns
A const reference to the node’s friendly name.
-
virtual std::ostream &write_description(std::ostream &os, uint32_t depth = 0) const
Writes a description of a node to a stream.
- Parameters
os – The stream; should be returned
depth – How many levels of inputs to describe
- Returns
The stream os
-
const std::vector<std::shared_ptr<Node>> &get_control_dependencies() const
Get control dependencies registered on the node.
-
const std::vector<Node*> &get_control_dependents() const
Get nodes dependent on this node.
This node cannot execute until node executes.
Remove the dependency of this node on node.
-
void clear_control_dependencies()
Remove all dependencies from this node.
-
void clear_control_dependents()
Remove this node as a dependency from all dependent nodes.
This node absorbs the control dependencies of source_node.
This node becomes a dependent of every node dependent on source_node.
This node’s control dependencies are replaced by replacement.
-
size_t get_output_size() const
Returns the number of outputs from the node.
-
const element::Type &get_element_type() const
Checks that there is exactly one output and returns its element type.
-
const Shape &get_output_shape(size_t i) const
Returns the shape for output i.
-
const PartialShape &get_output_partial_shape(size_t i) const
Returns the partial shape for output i.
-
Output<const Node> get_default_output() const
Return the output to use when converting to an Output<Node> with no index specified. Throws when not supported.
-
virtual size_t get_default_output_index() const
Returns the output of the default output, or throws if there is none.
-
size_t no_default_index() const
Throws no default.
-
const Shape &get_shape() const
Checks that there is exactly one output and returns its shape.
-
descriptor::Tensor &get_output_tensor(size_t i) const
Returns the tensor for output or input i.
-
size_t get_input_size() const
Returns the number of inputs for the op.
-
const Shape &get_input_shape(size_t i) const
Returns the shape of input i.
-
const PartialShape &get_input_partial_shape(size_t i) const
Returns the partial shape of input i.
True if this and node have one output with same element type and shape.
-
NodeVector get_users(bool check_is_used = false) const
Get all the nodes that uses the current node.
-
inline bool operator<(const Node &other) const
Use instance ids for comparison instead of memory addresses to improve determinism.
-
std::vector<Input<Node>> inputs()
- Returns
A vector containing a handle for each of this node’s inputs, in order.
-
std::vector<Input<const Node>> inputs() const
- Returns
A vector containing a handle for each of this node’s inputs, in order.
-
std::vector<Output<Node>> input_values() const
- Returns
A vector containing the values for each input
-
std::vector<Output<Node>> outputs()
- Returns
A vector containing a handle for each of this node’s outputs, in order.
-
std::vector<Output<const Node>> outputs() const
- Returns
A vector containing a handle for each of this node’s outputs, in order.
-
Input<Node> input(size_t input_index)
- Throws
std::out_of_range – if the node does not have at least
input_index+1inputs.- Returns
A handle to the
input_indexth input of this node.
-
Input<const Node> input(size_t input_index) const
- Throws
std::out_of_range – if the node does not have at least
input_index+1inputs.- Returns
A handle to the
input_indexth input of this node.
-
virtual void validate_and_infer_types()
-
class NodeValidationFailure : public ov::AssertFailure
- #include <node.hpp>
Public Functions
- template<> OPENVINO_API void create (const char *file, int line, const char *check_string, std::pair< const Node *, const std::vector< PartialShape > * > &&ctx, const std::string &explanation)
Specialization to throw the
NodeValidationFailurefor shape inference usingPartialShape- Parameters
check_loc_info – Exception location details to print.
ctx – NodeValidationFailure context which got pointer to node and input shapes used for shape inference.
explanation – Exception explanation string.
-
class NonconvertibleDivide : public ov::RuntimeAttribute
- #include <nonconvertible_divide.hpp>
NonconvertibleDivide class represents runtime info attribute that marks a Divide as prohibitted to transform it to power.
-
class NotImplemented : public ov::AssertFailure
- #include <except.hpp>
Exception class to be thrown on not implemented code.
-
class NoTransposeSinkingAttr : public ov::RuntimeAttribute
- #include <transpose_sinking_attr.hpp>
NoTransposeSinkingAttr class represents runtime info attribute that marks transpose operation should not be moved be backward sinking propagation.
-
class OldApiMapElementType : public ov::RuntimeAttribute
- #include <old_api_map_element_type_attribute.hpp>
OldApiMapElementType class represents runtime info attribute that stores legacy type that is required for obtaining IR in old API.
Public Functions
-
OldApiMapElementType() = default
A default constructor
-
inline OldApiMapElementType(const ov::element::Type &value)
Constructs a new OldApiMapElementType object.
- Parameters
value – [in] The object that stores values of OldApiMapElementType.
-
OldApiMapElementType() = default
-
class OldApiMapOrder : public ov::RuntimeAttribute
- #include <old_api_map_order_attribute.hpp>
OldApiMapOrder class represents runtime info attribute that stores order of the transpose that is required for obtaining IR in old API.
OldApiMapOrder stores the following information. Parameter: Order of the transpose which should be applied to Parameter with old API layout to obtain Parameter with new API layout.
Result: Order of the transpose which should be applied to Result with new API layout to obtain Result with old API layout.
Public Functions
-
OldApiMapOrder() = default
A default constructor
-
inline OldApiMapOrder(const std::vector<uint64_t> &value)
Constructs a new OldApiMapOrder object.
- Parameters
value – [in] The object that stores values of OldApiMapOrder.
-
OldApiMapOrder() = default
-
template<class T>
class OpExtension : public ov::BaseOpExtension - #include <op_extension.hpp>
The default implementation of OpenVINO operation extensions.
Public Functions
-
inline OpExtension()
Default constructor.
-
inline virtual const ov::DiscreteTypeInfo &get_type_info() const override
Returns the type info of operation.
- Returns
-
inline virtual ov::OutputVector create(const ov::OutputVector &inputs, ov::AttributeVisitor &visitor) const override
Method creates an OpenVINO operation.
- Parameters
inputs – vector of input ports
visitor – attribute visitor which allows to read necessaty arguments
- Returns
vector of output ports
-
inline virtual std::vector<ov::Extension::Ptr> get_attached_extensions() const override
Returns extensions that should be registered together with this extension class object.
Attached extensions may include frontend extensions that OpenVINO op to framework ops or necessary transformations that should be applied to the network which consist of target op.
- Returns
-
inline OpExtension()
-
class OpSet
- #include <opset.hpp>
Run-time opset information.
Subclassed by ngraph::OpSet
Public Functions
-
template<typename OP_TYPE>
inline void insert(const std::string &name) Insert OP_TYPE into the opset with a special name and the default factory.
-
template<typename OP_TYPE>
inline void insert() Insert OP_TYPE into the opset with the default name and factory.
-
ov::Node *create_insensitive(const std::string &name) const
Create the op named name using it’s factory.
-
inline bool contains_type(const NodeTypeInfo &type_info) const
Return true if OP_TYPE is in the opset.
-
template<typename OP_TYPE>
inline bool contains_type() const Return true if OP_TYPE is in the opset.
-
inline bool contains_type(const std::string &name) const
Return true if name is in the opset.
-
inline bool contains_type_insensitive(const std::string &name) const
Return true if name is in the opset.
-
inline bool contains_op_type(const Node *node) const
Return true if node’s type is in the opset.
-
template<typename OP_TYPE>
-
template<class T>
class optional - #include <ov_optional.hpp>
Store optional object of type T (basic version of std::optional).
Note
If cpp17 used this class should be replaced by std::optional.
- Template Parameters
T – Type of stored object.
-
class OriginalPrecisionAttribute : public ov::RuntimeAttribute
- #include <original_precision_attribute.hpp>
OriginalPrecisionAttribute stores the original precision of the node to pass this information to plugins.
-
template<typename NodeType>
class Output - #include <node_output.hpp>
-
template<>
class Output<const Node> - #include <node_output.hpp>
A handle for one of a node’s outputs.
Public Functions
-
Output(const Node *node, size_t index)
Constructs a Output.
- Parameters
node – A pointer to the node for the output handle.
index – The index of the output.
Constructs a Output.
- Parameters
node – A
shared_ptrto the node for the output handle.index – The index of the output.
Constructs a Output, referencing the zeroth output of the node.
- Parameters
node – A
shared_ptrto the node for the output handle.
-
Output() = default
A null output.
-
const Node *get_node() const
- Returns
A pointer to the node referred to by this output handle.
- Returns
A
shared_ptrto the node referred to by this output handle.
-
size_t get_index() const
- Returns
The index of the output referred to by this output handle.
-
descriptor::Tensor &get_tensor() const
- Returns
A reference to the tensor descriptor for this output.
-
std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
- Returns
A shared point to the tensor ptr for this output.
-
const element::Type &get_element_type() const
- Returns
The element type of the output referred to by this output handle.
-
const Shape &get_shape() const
- Returns
The shape of the output referred to by this output handle.
-
const PartialShape &get_partial_shape() const
- Returns
The partial shape of the output referred to by this output handle.
-
const RTMap &get_rt_info() const
- Returns
The constant reference to runtime info map
-
const std::unordered_set<std::string> &get_names() const
- Returns
The tensor names associated with this output
-
std::string get_any_name() const
- Returns
Any tensor name associated with this output
-
Output(const Node *node, size_t index)
-
template<>
class Output<Node> - #include <node_output.hpp>
A handle for one of a node’s outputs.
Public Functions
-
Output(Node *node, size_t index)
Constructs a Output.
- Parameters
node – A pointer to the node for the output handle.
index – The index of the output.
Constructs a Output.
- Parameters
node – A
shared_ptrto the node for the output handle.index – The index of the output.
Constructs a Output, referencing the zeroth output of the node.
- Parameters
node – A
shared_ptrto the node for the output handle.
-
Output() = default
A null output.
-
Node *get_node() const
- Returns
A pointer to the node referred to by this output handle.
- Returns
A
shared_ptrto the node referred to by this output handle.
-
size_t get_index() const
- Returns
The index of the output referred to by this output handle.
-
descriptor::Tensor &get_tensor() const
- Returns
A reference to the tensor descriptor for this output.
-
std::shared_ptr<descriptor::Tensor> get_tensor_ptr() const
- Returns
A shared point to the tensor ptr for this output.
- Returns
Set new tensor desc shared pointer to this output
-
const element::Type &get_element_type() const
- Returns
The element type of the output referred to by this output handle.
-
const Shape &get_shape() const
- Returns
The shape of the output referred to by this output handle.
-
const PartialShape &get_partial_shape() const
- Returns
The partial shape of the output referred to by this output handle.
-
RTMap &get_rt_info()
- Returns
The reference to runtime info map
-
const RTMap &get_rt_info() const
- Returns
The constant reference to runtime info map
-
const std::unordered_set<std::string> &get_names() const
- Returns
The tensor names associated with this output
-
std::string get_any_name() const
- Returns
Any tensor names associated with this output
-
void set_names(const std::unordered_set<std::string> &names)
- Returns
Set tensor names associated with this output
-
void add_names(const std::unordered_set<std::string> &names)
- Returns
Add tensor names associated with this output
-
std::set<Input<Node>> get_target_inputs() const
- Returns
A set containing handles for all inputs targeted by the output referenced by this output handle.
-
Output(Node *node, size_t index)
-
class PartialShape
- #include <partial_shape.hpp>
Class representing a shape that may be partially or totally dynamic.
A PartialShape may have:
Dynamic rank. (Informal notation:
?)Static rank, but dynamic dimensions on some or all axes. (Informal notation examples:
{1,2,?,4},{?,?,?})Static rank, and static dimensions on all axes. (Informal notation examples:
{1,2,3,4},{6},{})
Public Functions
-
PartialShape(std::initializer_list<Dimension> init)
Constructs a shape with static rank from an initializer list of Dimension.
Examples:
PartialShape s{2,3,4}; // rank=3, all dimensions static PartialShape s{}; // rank=0 PartialShape s{2,Dimension::dynamic(),3}; // rank=3, dimension 1 dynamic
- Parameters
init – The Dimension values for the constructed shape.
-
PartialShape(std::vector<Dimension> dimensions)
Constructs a PartialShape with static rank from a vector of Dimension.
- Parameters
dimensions – The Dimension values for the constructed shape.
-
PartialShape(const std::vector<Dimension::value_type> &dimensions)
Constructs a PartialShape with static rank from a vector of dimensions values.
- Parameters
dimensions – The Dimension values for the constructed shape.
-
PartialShape()
Constructs a static PartialShape with zero rank (the shape of a scalar).
-
PartialShape(const Shape &shape)
Constructs a static PartialShape from a PartialShape.
- Parameters
shape – The PartialShape to convert into PartialShape.
-
PartialShape(const std::string &shape)
Constructs a static PartialShape from a string.
- Parameters
shape – The string to parse into PartialShape.
-
bool is_static() const
Check if this shape is static.
A shape is considered static if it has static rank, and all dimensions of the shape are static.
- Returns
trueif this shape is static, elsefalse.
-
inline bool is_dynamic() const
Check if this shape is dynamic.
A shape is considered static if it has static rank, and all dimensions of the shape are static.
- Returns
falseif this shape is static, elsetrue.
-
inline Rank rank() const
Get the rank of the shape.
- Returns
The rank of the shape. This will be Rank::dynamic() if the rank of the shape is dynamic.
-
bool compatible(const PartialShape &s) const
Check whether this shape is compatible with the argument, i.e., whether it is possible to merge them.
Two shapes are compatible if
one or both of them has dynamic rank, or
both shapes have dynamic and equal rank, and their dimensions are elementwise compatible (see Dimension::compatible()).
- Parameters
s – The shape to be checked for compatibility with this shape.
- Returns
trueif this shape is compatible withs, elsefalse.
-
bool same_scheme(const PartialShape &s) const
Check whether this shape represents the same scheme as the argument.
Two shapes
s1ands2represent the same scheme ifthey both have dynamic rank, or
they both have static and equal rank
r, and for everyifrom0tor-1,s1[i]represents the same scheme ass2[i](see Dimension::same_scheme()).
- Parameters
s – The shape whose scheme is being compared with this shape.
- Returns
trueif this shape represents the same scheme ass, elsefalse.
-
bool relaxes(const PartialShape &s) const
Check whether this shape is a relaxation of the argument.
Intuitively, a PartialShape
s1is said to relaxs2(or is a relaxation ofs2) if it is “more permissive” thans2. In other words,s1is a relaxation ofs2if anything you can form by plugging things into the dynamic dimensions ofs2is also something you can form by plugging things into the dynamic dimensions ofs1, but not necessarily the other way around.s1.relaxes(s2)is equivalent tos2.refines(s1).Formally, PartialShape
s1is said to relax PartialShapes2if:For every
ifrom0tor-1, eithers1[i]contains s2[i].
- Parameters
s – The shape which is being compared against this shape.
- Returns
trueif this shape relaxess, elsefalse.
-
bool refines(const PartialShape &s) const
Check whether this shape is a refinement of the argument.
Intuitively, a PartialShape
s1is said to relaxs2(or is a relaxation ofs2) if it is “less permissive” thans2. In other words,s1is a relaxation ofs2if anything you can form by plugging things into the dynamic dimensions ofs1is also something you can form by plugging things into the dynamic dimensions ofs2, but not necessarily the other way around.s1.refines(s2)is equivalent tos2.relaxes(s1).Formally, PartialShape
s1is said to refine PartialShapes2if:s2has dynamic rank, ors1ands2both have static rankr, and for everyifrom0tor-1, eithers2[i]is dynamic, ors1[i]==s2[i].
- Parameters
s – The shape which is being compared against this shape.
- Returns
trueif this shape refiness, elsefalse.
-
bool merge_rank(const Rank &r)
Checks that this shape’s rank is compatible with
r, and, if this shape’s rank is dynamic andris static, updates this shape to have a rank ofrwith dimensions all dynamic.- Returns
trueif this shape’s rank is compatible withr, elsefalse.
-
Shape to_shape() const
Convert a static PartialShape to a PartialShape.
- Throws
std::invalid_argument – If this PartialShape is dynamic.
- Returns
A new PartialShape
swheres[i] = size_t((*this)[i]).
-
bool all_non_negative() const
Returns
trueif all static dimensions of the tensor are non-negative, elsefalse.
-
const Dimension &operator[](size_t i) const
Index operator for PartialShape.
- Parameters
i – The index of the dimension being selected.
- Returns
A reference to the
ith Dimension of this shape.
-
Dimension &operator[](size_t i)
Index operator for PartialShape.
- Parameters
i – The index of the dimension being selected.
- Returns
A reference to the
ith Dimension of this shape.
-
inline explicit operator std::vector<Dimension>() const
Returns a vector of the dimensions. This has no meaning if dynamic.
-
Shape get_max_shape() const
Get the max bounding shape.
-
Shape get_min_shape() const
Get the min bounding shape.
-
Shape get_shape() const
Get the unique shape.
-
inline iterator begin() noexcept
Returns a read/write iterator that points to the first element in the shape. Iteration is done in ordinary element order.
-
inline const_iterator begin() const noexcept
Returns a read-only (constant) iterator that points to the first element in the shape. Iteration is done in ordinary element order.
-
inline iterator end() noexcept
Returns a read/write iterator that points one past the last element in the shape. Iteration is done in ordinary element order.
-
inline const_iterator end() const noexcept
Returns a read-only (constant) iterator that points one past the last element in the shape. Iteration is done in ordinary element order.
-
inline reverse_iterator rbegin() noexcept
Returns a read/write reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.
-
inline const_reverse_iterator rbegin() const noexcept
Returns a read-only (constant) reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.
-
inline reverse_iterator rend() noexcept
Returns a read/write reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.
-
inline const_reverse_iterator rend() const noexcept
Returns a read-only (constant) reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.
-
inline const_iterator cbegin() const noexcept
Returns a read-only (constant) iterator that points to the first element in the shape. Iteration is done in ordinary element order.
-
inline const_iterator cend() const noexcept
Returns a read-only (constant) iterator that points one past the last element in the shape. Iteration is done in ordinary element order.
-
inline const_reverse_iterator crbegin() const noexcept
Returns a read-only (constant) reverse iterator that points to the last element in the shape. Iteration is done in reverse element order.
-
inline const_reverse_iterator crend() const noexcept
Returns a read-only (constant) reverse iterator that points to one before the first element in the shape. Iteration is done in reverse element order.
-
inline void resize(size_t count)
Resizes dimensions container to contain count elements.
-
inline size_t size() const
Returns size of dimension vector. Requires rank to be static.
-
inline iterator insert(iterator position, const Dimension &val)
Returns a read/write iterator that points to the inserted element in the shape.
-
inline void insert(iterator position, size_t n, const Dimension &val)
Inserts count copies of the value before position.
-
template<class InputIterator>
inline void insert(iterator position, InputIterator first, InputIterator last) Inserts elements from range [first, last) before position.
-
inline void reserve(size_t n)
Requests that the dimensions vector capacity be enough to contain n elements.
-
inline void push_back(const Dimension &val)
push element to the end of partial shape
-
template<class ...Args>
inline void emplace_back(Args&&... args) emplace element to the end of partial shape
-
std::string to_string() const
String representation of PartialShape.
Public Static Functions
-
static PartialShape dynamic(Rank r = Rank::dynamic())
Construct a PartialShape with the given rank and all dimensions (if any) dynamic.
- Returns
A PartialShape with the given rank, and all dimensions (if any) dynamic.
-
static bool merge_into(PartialShape &dst, const PartialShape &src)
Try to merge one shape into another.
Merges
srcintodst, returningtrueon success andfalseon failure. Iffalseis returned, the effect ondstis unspecified.To merge two partial shapes
s1ands2is to find the most permissive partial shapesthat is no more permissive thans1ors2, ifsexists. For example:merge(?,?) -> ? merge(?,{?,?}) -> {?,?} merge({?,?},{?,?}) -> {?,?} merge({1,2,3,4},?) -> {1,2,3,4} merge({1,2},{1,?}) -> {1,2} merge({1,2,?,?},{1,?,3,?}) -> {1,2,3,?} merge({1,2,3},{1,2,3}) -> {1,2,3} merge({1,?},{2,?}) fails [dimension 0 constraints are inconsistent] merge({?,?},{?,?,?}) fails [ranks are inconsistent]This function (merge_into) performs the “merge” operation described above on
dstandsrc, but overwritesdstwith the result and returnstrueif merging is successful; if merging is unsuccessful, the function returnsfalseand may make unspecified changes todst.- Parameters
dst – [inout] The shape that
srcwill be merged into.src – The shape that will be merged into
dst.
- Returns
trueif merging succeeds, elsefalse.
-
static bool broadcast_merge_into(PartialShape &dst, const PartialShape &src, const ov::op::AutoBroadcastSpec &autob)
Try to merge one shape into another along with implicit broadcasting.
Friends
- friend OPENVINO_API std::ostream & operator<< (std::ostream &str, const PartialShape &shape)
Inserts a human-readable representation of a PartialShape into an output stream.
The output to the stream is in “informal” notation. In other words:
If
shapehas dynamic rank, inserts the string?.If
shapehas static rank, inserts the string{, then inserts each dimension ofshapeinto the output stream separated by commas, then inserts}.
PartialShape s1{PartialShape::dynamic())}; PartialShape s2{}; PartialShape s3{1,Dimension::dynamic(),2,3}; PartialShape s4{2,3,4}; std::cout << s1 << std::endl << s2 << std::endl << s3 << std::endl << s4 << std::endl;
? {} {1,?,2,3} {2,3,4}- Parameters
str – The output stream targeted for insertion.
shape – The shape to be inserted into
str.
- Returns
A reference to
strafter insertion.
- friend OPENVINO_API PartialShape operator+ (const PartialShape &s1, const PartialShape &s2)
Elementwise addition of two PartialShape objects.
If
s1ors2has dynamic rank, returns PartialShape::dynamic().If
s1 ands2` both have static rank, and their ranks are unequal, throws std::invalid_argument.If
s1ands2both have static rank, and their ranks are equal, returns a new shape whoseith dimension iss1[i] + s2[i].
- Parameters
s1 – Left operand for addition.
s2 – Right operand for addition.
- Throws
std::invalid_argument – If
s1ands2have inconsistent ranks.- Returns
The result of elementwise adding
s1tos2(see description).
-
class PrecisionPreservedAttribute : public SharedAttribute<bool>
- #include <precision_preserved_attribute.hpp>
PrecisionPreservedAttribute defines the precision preserved operation. If the attribute is absent, then an operation is not precision preserved.
For more details about the attribute, refer to PrecisionPreservedAttribute page in the Inference Engine Developer Guide.
Subclassed by ov::AvgPoolPrecisionPreservedAttribute
-
class PrecisionsAttribute : public SharedAttribute<std::vector<ov::element::Type>>
- #include <precisions_attribute.hpp>
PrecisionsAttribute defines precision which is required for input/output port or an operation.
For more details about the attribute, refer to PrecisionsAttribute page in the Inference Engine Developer Guide.
-
class PrecisionSensitive : public ov::RuntimeAttribute
- #include <precision_sensitive_attribute.hpp>
PrecisionSensitive class represents runtime info attribute that marks input to an operation as a precision sensitive and disables compression to FP16 of the subgraph before this input.
-
class PreprocessingAttribute : public ov::RuntimeAttribute
- #include <preprocessing_attribute.hpp>
-
class PrimitivesPriority : public ov::RuntimeAttribute
- #include <primitives_priority_attribute.hpp>
-
struct ProfilingInfo
- #include <profiling_info.hpp>
Represents basic inference profiling information per operation.
If the operation is executed using tiling, the sum time per each tile is indicated as the total execution time. Due to parallel execution, the total execution time for all nodes might be greater than the total inference time.
Public Types
-
enum class Status
Defines the general status of a node.
Values:
-
enumerator NOT_RUN
A node is not executed.
-
enumerator OPTIMIZED_OUT
A node is optimized out during graph optimization phase.
-
enumerator EXECUTED
A node is executed.
-
enumerator NOT_RUN
Public Members
-
Status status
Defines the node status.
-
std::chrono::microseconds real_time
The absolute time, in microseconds, that the node ran (in total).
-
std::chrono::microseconds cpu_time
The net host CPU time that the node ran.
-
std::string node_name
Name of a node.
-
std::string exec_type
Execution type of a unit.
-
std::string node_type
Node type.
-
enum class Status
-
template<typename T, PropertyMutability mutability_ = PropertyMutability::RW>
class Property : public util::BaseProperty<T, PropertyMutability::RW> - #include <properties.hpp>
This class is used to bind property name with value type.
- Template Parameters
T – type of value used to set or get property
- template<typename T> RO > : public util::BaseProperty< T, PropertyMutability::RO >
- #include <properties.hpp>
This class is used to bind read-only property name with value type.
- Template Parameters
T – type of value used to pass or get property
-
struct PropertyName : public std::string
- #include <properties.hpp>
This class is used to return property name and its mutability attribute.
Public Functions
-
inline PropertyName(const std::string &str, PropertyMutability mutability = PropertyMutability::RW)
Constructs property name object.
- Parameters
str – property name
mutability – property mutability
-
inline bool is_mutable() const
check property mutability
- Returns
true if property is mutable
-
inline PropertyName(const std::string &str, PropertyMutability mutability = PropertyMutability::RW)
-
class QuantizationAlignmentAttribute : public SharedAttribute<bool>
- #include <quantization_alignment_attribute.hpp>
QuantizationAlignmentAttribute defines subgraph with the same quantization alignment. FakeQuantize operations are not included. The attribute is used by quantization operations.
For more details about the attribute, refer to QuantizationAlignmentAttribute page in the Inference Engine Developer Guide.
-
class QuantizationGranularityAttribute : public ov::RuntimeAttribute
- #include <quantization_granularity_attribute.hpp>
QuantizationGranularityAttribute defines quantization granularity of operation inputs.
For more details about the attribute, refer to QuantizationGranularityAttribute page in the Inference Engine Developer Guide.
-
class QuantizationModeAttribute : public ov::RuntimeAttribute
- #include <quantization_mode_attribute.hpp>
-
struct RawNodeOutput
- #include <node.hpp>
-
class RemoteBlobTensor : public ov::IRemoteTensor
- #include <remote_utils.hpp>
Tensor what contains InferenceEngine::RemoteBlob inside Blob owns the memory.
Public Functions
-
inline virtual const AnyMap &get_properties() const override
Returns additional information associated with tensor.
- Returns
Map of property names to properties
-
inline virtual const AnyMap &get_properties() const override
-
class RemoteContext
- #include <remote_context.hpp>
This class represents an abstraction
for remote (non-CPU) accelerator device-specific inference context. Such context represents a scope on the device within which compiled models and remote memory tensors can exist, function, and exchange data.
Subclassed by ov::intel_gpu::ocl::ClContext
Public Functions
-
RemoteContext() = default
Default constructor.
-
RemoteContext(const RemoteContext &other) = default
Default copy constructor.
- Parameters
other – Another RemoteContext object.
-
RemoteContext &operator=(const RemoteContext &other) = default
Default copy assignment operator.
- Parameters
other – Another RemoteContext object.
- Returns
Reference to the current object.
-
RemoteContext(RemoteContext &&other) = default
Default move constructor.
- Parameters
other – Another RemoteContext object.
-
RemoteContext &operator=(RemoteContext &&other) = default
Default move assignment operator.
- Parameters
other – Another RemoteContext object.
- Returns
Reference to the current object.
-
operator bool() const noexcept
Checks if current RemoteContext object is initialized.
- Returns
trueif current RemoteContext object is initialized,false- otherwise
-
~RemoteContext()
Destructor that preserves unloading order of implementation object and reference to the library.
-
template<typename T>
inline bool is() const noexcept Checks if the RemoteContext object can be cast to the type T.
- Template Parameters
T – Type to be checked. Must represent a class derived from RemoteContext.
- Returns
True if this object can be dynamically cast to the type T*; false, otherwise.
-
template<typename T>
inline const T as() const Casts this RemoteContext object to the type T.
- Template Parameters
T – Type to cast to. Must represent a class derived from RemoteContext.
- Returns
T Object.
-
RemoteTensor create_tensor(const element::Type &type, const Shape &shape, const AnyMap ¶ms = {})
Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.
- Parameters
type – Defines the element type of the tensor.
shape – Defines the shape of the tensor.
params – Map of the low-level tensor object parameters.
- Returns
Pointer to a plugin object that implements the RemoteTensor interface.
-
AnyMap get_params() const
Returns a map of device-specific parameters required for low-level operations with the underlying object. Parameters include device/context handles, access flags, etc. Content of the returned map depends on a remote execution context that is currently set on the device (working scenario). Abstract method.
- Returns
A map of name/parameter elements.
-
Tensor create_host_tensor(const element::Type type, const Shape &shape)
This method is used to create a host tensor object friendly for the device in current context. For example, GPU context may allocate USM host memory (if corresponding extension is available), which could be more efficient than regular host memory.
Public Static Functions
-
static void type_check(const RemoteContext &remote_context, const std::map<std::string, std::vector<std::string>> &type_info = {})
Internal method: checks remote type.
- Parameters
remote_context – Remote context which type is checked.
type_info – Map with remote object runtime info.
- Throws
Exception – if type check with the specified parameters failed.
-
RemoteContext() = default
-
class RemoteTensor : public ov::Tensor
- #include <remote_tensor.hpp>
Remote memory access and interoperability API.
Subclassed by ov::intel_gpu::ocl::ClBufferTensor, ov::intel_gpu::ocl::ClImage2DTensor, ov::intel_gpu::ocl::USMTensor
Public Functions
-
void *data(const element::Type) = delete
Access to host memory is not available for RemoteTensor. To access a device-specific memory, cast to a specific RemoteTensor derived object and work with its properties or parse device memory properties via RemoteTensor::get_params.
- Returns
Nothing, throws an exception.
-
ov::AnyMap get_params() const
Returns a map of device-specific parameters required for low-level operations with underlying object. Parameters include device/context/surface/buffer handles, access flags, etc. Content of the returned map depends on remote execution context that is currently set on the device (working scenario). Abstract method.
- Returns
A map of name/parameter elements.
Public Static Functions
-
static void type_check(const Tensor &tensor, const std::map<std::string, std::vector<std::string>> &type_info = {})
Checks OpenVINO remote type.
-
void *data(const element::Type) = delete
-
template<class TShape>
struct result_shape - #include <utils.hpp>
Get correct return type of input shape when call
shape_infer.The input shapes are vector like std::vector<TShape>, where
TShapecan bestd::vector<const size_t>This will provide correct return especially for static shape which can work as reference to dimension or hold them.- Template Parameters
TShape – Type of input shape.
- template<> Shape >
- #include <utils.hpp>
Get correct result shape for ov::Shape which is same type.
-
template<>
struct result_shape<PartialShape> - #include <utils.hpp>
Get correct result shape for PartialShape which is same type.
-
class RoundingGuard
- #include <rounding_guard.hpp>
Set current round direction for scoped block.
Round direction can be one of:
FE_DOWNWARD
FE_TONEAREST
FE_TOWARDZERO
FE_UPWARD see std <cfenv> header for details.
-
class RuntimeAttribute
- #include <runtime_attribute.hpp>
Subclassed by SharedAttribute< IntervalsAlignmentSharedValue >, SharedAttribute< bool >, SharedAttribute< std::vector< ov::element::Type > >, SharedAttribute< T >, ov::BiasAttribute, ov::Decompression, ov::DequantizationNode, ov::DisableCleanupAttribute, ov::DisableFP16Compression, ov::FusedNames, ov::KeepConstPrecision, ov::LayoutAttribute, ov::NmsSelectedIndices, ov::NoTransposeSinkingAttr, ov::NonconvertibleDivide, ov::OldApiMapElementType, ov::OldApiMapOrder, ov::OriginalPrecisionAttribute, ov::PrecisionSensitive, ov::PreprocessingAttribute, ov::PrimitivesPriority, ov::QuantizationGranularityAttribute, ov::QuantizationModeAttribute, ov::ShapeSubgraph, ov::StridesPropagation, ov::SymbolicInfo, ov::frontend::tensorflow::GraphIterator, ov::pass::DisableConstantFolding, ov::pass::DisableFoldSubgraphEmptyInputs, ov::pass::DisableRemoveConcatZeroDimInput, ov::preprocess::TensorInfoMemoryType
-
template<class T, Direction D = Direction::FORWARD>
class SeqGen - #include <sequnce_generator.hpp>
Infinite generator of sequence increasing values.
Start value can be specified.
- Template Parameters
T – Type of sequence values (must support
++or ‘—’ operators).
-
class Shape : public std::vector<size_t>
- #include <shape.hpp>
Shape for a tensor.
-
class ShapeSubgraph : public ov::RuntimeAttribute
- #include <is_shape_subgraph.hpp>
ShapeSubgraph class represents runtime info attribute that marks shape subgraphs. Information whether the node belongs to the shape path or to the data path is needed during evaluate and CF.
-
template<class T>
struct SoPtr - #include <so_ptr.hpp>
This class instantiate object using shared library.
- Template Parameters
T – An type of object SoPtr can hold
Public Functions
-
SoPtr() = default
Default constructor.
-
inline ~SoPtr()
Destructor preserves unloading order of implementation object and reference to library.
Constructs an object with existing shared object reference and loaded pointer.
- Parameters
ptr – pointer to the loaded object
so – Existing reference to library
Constructs an object with existing shared object reference.
- Parameters
ptr – pointer to the loaded object
Constructs an object with existing shared object reference.
- Parameters
ptr – pointer to the loaded object
-
template<typename U>
inline SoPtr(const SoPtr<U> &that) The copy-like constructor, can create So Pointer that dereferenced into child type if T is derived of U.
- Parameters
that – copied SoPtr object
-
inline T *operator->() const noexcept
Standard pointer operator.
- Returns
underlined interface with disabled Release method
Public Members
-
std::shared_ptr<T> _ptr
Gets a smart pointer to the custom object.
-
std::shared_ptr<void> _so
The shared object or dynamic loaded library.
-
class Strides : public std::vector<size_t>
- #include <strides.hpp>
Strides for a tensor.
-
class StridesPropagation : public ov::RuntimeAttribute
- #include <strides_property.hpp>
-
class SymbolicInfo : public ov::RuntimeAttribute
- #include <symbolic_info.hpp>
SymbolicInfo class represents runtime info attribute that instructs ov::Output objects to skip invalidation of partial values and labels during partial value propagation and keeps shared_ptr to TableOfEquivalence.
-
class Tensor
- #include <tensor.hpp>
Tensor API holding host memory It can throw exceptions safely for the application, where it is properly handled.
Subclassed by ov::RemoteTensor
Public Functions
-
Tensor() = default
Default constructor.
Copy constructor with adding new shared object.
- Parameters
other – Original tensor
so – Shared object
-
Tensor(const Tensor &other) = default
Default copy constructor.
- Parameters
other – other Tensor object
-
Tensor &operator=(const Tensor &other) = default
Default copy assignment operator.
- Parameters
other – other Tensor object
- Returns
reference to the current object
-
Tensor &operator=(Tensor &&other) = default
Default move assignment operator.
- Parameters
other – other Tensor object
- Returns
reference to the current object
-
~Tensor()
Destructor preserves unloading order of implementation object and reference to library.
-
Tensor(const element::Type &type, const Shape &shape, const Allocator &allocator = {})
Constructs Tensor using element type and shape. Allocate internal host storage using default allocator.
-
Tensor(const element::Type &type, const Shape &shape, void *host_ptr, const Strides &strides = {})
Constructs Tensor using element type and shape. Wraps allocated host memory.
Note
Does not perform memory allocation internally
-
Tensor(const ov::Output<const ov::Node> &port, const Allocator &allocator = {})
Constructs Tensor using port from node. Allocate internal host storage using default allocator.
- Parameters
port – port from node
allocator – allocates memory for internal tensor storage
-
Tensor(const ov::Output<const ov::Node> &port, void *host_ptr, const Strides &strides = {})
Constructs Tensor using port from node. Wraps allocated host memory.
Note
Does not perform memory allocation internally
- Parameters
port – port from node
host_ptr – Pointer to pre-allocated host memory with initialized objects
strides – Optional strides parameters in bytes. Strides are supposed to be computed automatically based on shape and element size
-
Tensor(const Tensor &other, const Coordinate &begin, const Coordinate &end)
Constructs region of interest (ROI) tensor form another tensor.
Note
Does not perform memory allocation internally
Note
A Number of dimensions in
beginandendmust match number of dimensions inother.get_shape()- Parameters
other – original tensor
begin – start coordinate of ROI object inside of the original object.
end – end coordinate of ROI object inside of the original object.
-
void set_shape(const ov::Shape &shape)
Set new shape for tensor, deallocate/allocate if new total size is bigger than previous one.
Note
Memory allocation may happen
- Parameters
shape – A new shape
-
const Shape &get_shape() const
- Returns
A tensor shape
-
void copy_to(ov::Tensor dst) const
Copy tensor, destination tensor should have the same element type and shape.
- Parameters
dst – destination tensor
-
bool is_continuous() const
Reports whether the tensor is continuous or not.
- Returns
true if tensor is continuous
-
size_t get_size() const
Returns the total number of elements (a product of all the dims or 1 for scalar)
- Returns
The total number of elements
-
size_t get_byte_size() const
Returns the size of the current Tensor in bytes.
- Returns
Tensor’s size in bytes
-
void *data(const element::Type &type = {}) const
Provides an access to the underlaying host memory.
Note
If type parameter is specified, the method throws an exception if specified type’s fundamental type does not match with tensor element type’s fundamental type
- Parameters
type – Optional type parameter.
- Returns
A host pointer to tensor memory
-
template<typename T, typename datatype = typename std::decay<T>::type>
inline T *data() const Provides an access to the underlaying host memory casted to type
TNote
Throws exception if specified type does not match with tensor element type
- Returns
A host pointer to tensor memory casted to specified type
T.
-
bool operator!() const noexcept
Checks if current Tensor object is not initialized.
- Returns
trueif current Tensor object is not initialized,false- otherwise
-
explicit operator bool() const noexcept
Checks if current Tensor object is initialized.
- Returns
trueif current Tensor object is initialized,false- otherwise
-
template<typename T>
inline std::enable_if<std::is_base_of<Tensor, T>::value, bool>::type is() const noexcept Checks if the Tensor object can be cast to the type T.
- Template Parameters
T – Type to be checked. Must represent a class derived from the Tensor
- Returns
true if this object can be dynamically cast to the type const T*. Otherwise, false
-
Tensor() = default
-
template<class TContainer>
class TensorAccessor : public ov::ITensorAccessor - #include <tensor_data_accessor.hpp>
Tensor data accessor functor.
Creates the ov::Tensor found in tensors container. This accessor does not take ownership of tensors container. Supports following containers:
std::unordered_map<size_t, ov::Tensor>
- Template Parameters
TContainer – Type of tensor container.
Public Functions
-
inline constexpr TensorAccessor(const TContainer *tensors)
Construct a new Tensor Accessor object for tensors container.
- Parameters
tensors – Pointer to container with tensors.
-
virtual Tensor operator()(size_t port) const override
Get tensor for given port number.
- Parameters
port – Port number to get data.
- Returns
Tensor to data or empty tensor if data not found.
-
virtual Tensor operator()(size_t port) const
Get tensor at port.
- Parameters
port – Number of data port (operator input) to get tensor.
- Returns
Tensor to data at port.
-
class TensorRemoteBlob : public InferenceEngine::RemoteBlob, public ov::legacy_convert::TensorHolder
- #include <remote_utils.hpp>
Create InferenceEngine::RemoteBlob from the Tensor.
Public Functions
-
inline virtual AnyMap getParams() const override
Returns a map of device-specific parameters required for low-level operations with underlying object. Parameters include device/context/surface/buffer handles, access flags, etc. Contents of the map returned depend on remote execution context that is currently set on the device (working scenario). Abstract method.
- Returns
A map of name/parameter elements.
-
inline virtual std::string getDeviceName() const noexcept override
Returns name of the device on which underlying object is allocated. Abstract method.
- Returns
A device name string in the same format as that in plugin metric.
-
inline virtual std::shared_ptr<InferenceEngine::RemoteContext> getContext() const noexcept override
Returns device context which underlying object belongs to. Abstract method.
- Returns
Pointer to plugin-specific context class object, which is derived from RemoteContext. Dynamic casting should be used if it is necessary to retrieve a pointer to original class.
-
inline virtual void allocate() noexcept override
Allocates memory to store the data.
Abstract method.
-
inline virtual bool deallocate() noexcept override
Releases previously allocated data.
Abstract method.
- Returns
Trueif deallocation happens successfully,falseotherwise.
-
inline virtual InferenceEngine::LockedMemory<void> rwmap() noexcept override
Gets read/write access to the memory in virtual space of the process. The function returns object which retains mapped memory. The memory been addressed in the MemoryBlob in general case can be allocated on remote device. This function maps remote memory to the memory in the virtual process space and after destruction of the LockedMemory will upload changed content to the accelerator.
To avoid extra copy of data, you can use rmap() and wmap() functions.
In case of memory originally allocated on the host, this function returns LockedMemory which will transparently refer to original memory address. No extra copy will happen
In general case, pointer received from that LockedMemory becomes invalid just after destruction of LockedMemory instance. Keep Locked memory alive while you need to address memory in the process on the host.
Abstract method.
- Returns
A LockedMemory object
-
inline virtual InferenceEngine::LockedMemory<const void> rmap() const noexcept override
Gets read only access to the memory in virtual space of the process. The function returns object which retains mapped memory.
The memory been addressed in the MemoryBlob in general case can be allocated on remote device. This function copies remote memory to the memory in the virtual process space and after destruction of the LockedMemory it will not upload host memory back, because it is expected that content is not changed.
To have an ability change content, you can use rwmap() and wmap() functions.
In case of memory originally allocated on the host, this function returns LockedMemory which will transparently refer to original memory address. No extra copy will happen
In general case, pointer received from that LockedMemory becomes invalid just after destruction of LockedMemory instance. Keep Locked memory alive while you need to address memory in the process on the host.
Abstract method.
- Returns
A LockedMemory object
-
inline virtual InferenceEngine::LockedMemory<void> wmap() noexcept override
Gets “write only direction” access to the memory in virtual space of the process. The function returns object which retains memory to be uploaded on device.
The memory been addressed in the MemoryBlob in general case can be allocated on remote device. This function does not copy of the content from the device to the memory in the virtual process space, the content of the memory just after calling of this function is not specified. After destruction of the LockedMemory, content will be upload host memory. In the same time there is no abilities to restrict reading from the memory, you need to care of reading from memory got by wmap(), it might have sense in some cases like filling of content and before uploading to device
To access data stored in the blob, you can use rwmap() and rmap() functions.
In case of memory originally allocated on the host, this function returns LockedMemory which will transparently refer to original memory address. No extra copy will happen
In general case, pointer received from that LockedMemory becomes invalid just after destruction of LockedMemory instance. Keep Locked memory alive while you need to address memory in the process on the host.
Abstract method.
- Returns
A LockedMemory object
-
inline virtual const std::shared_ptr<InferenceEngine::IAllocator> &getAllocator() const noexcept override
Gets the allocator for allocator-based blobs.
- Returns
The allocator for allocator-based blobs or if there is none then a nullptr.
-
inline virtual void *getHandle() const noexcept override
Gets the handle to allocated memory.
- Returns
The handle to allocated memory for allocator-based blobs or if there is none then a nullptr.
-
inline virtual AnyMap getParams() const override
-
struct TensorTransform : public ov::element::NotSupported<void>
- #include <utils.hpp>
-
template<typename VAT>
class ValueAccessor - #include <attribute_adapter.hpp>
Provides access to an attribute of type AT as a value accessor type VAT.
Provides access to values via get/set methods from an m_value, typically from ValueReference.
The m_buffer holds a VAT, which may be wider than the attribute AT. For example, serializers that only support int64_t integers would use a ValueAccessor<vector<int64_t>> to reference a vector<int8_t> attribute. Destruction moves the value back to the attribute if it was changed.
- Template Parameters
VAT – The adapter value type; may be wider than the value being accessed.
Subclassed by ov::DirectValueAccessor< bool >, ov::DirectValueAccessor< double >, ov::DirectValueAccessor< int64_t >, ov::DirectValueAccessor< op::v5::Loop::SpecialBodyPorts >, ov::DirectValueAccessor< ov::Dimension >, ov::DirectValueAccessor< ov::PartialShape >, ov::DirectValueAccessor< ov::element::TypeVector >, ov::DirectValueAccessor< ov::op::util::FrameworkNodeAttrs >, ov::DirectValueAccessor< std::set< std::string > >, ov::DirectValueAccessor< std::shared_ptr< ngraph::runtime::AlignedBuffer > >, ov::DirectValueAccessor< std::shared_ptr< op::util::Variable > >, ov::DirectValueAccessor< std::shared_ptr< ov::Model > >, ov::DirectValueAccessor< std::vector< double > >, ov::DirectValueAccessor< std::vector< float > >, ov::DirectValueAccessor< std::vector< int16_t > >, ov::DirectValueAccessor< std::vector< int32_t > >, ov::DirectValueAccessor< std::vector< int8_t > >, ov::DirectValueAccessor< std::vector< std::shared_ptr< op::util::MultiSubGraphOp::InputDescription > > >, ov::DirectValueAccessor< std::vector< std::shared_ptr< op::util::MultiSubGraphOp::OutputDescription > > >, ov::DirectValueAccessor< std::vector< std::string > >, ov::DirectValueAccessor< std::vector< uint16_t > >, ov::DirectValueAccessor< std::vector< uint32_t > >, ov::DirectValueAccessor< std::vector< uint64_t > >, ov::DirectValueAccessor< std::vector< uint8_t > >, ov::IndirectScalarValueAccessor< float, double >, ov::IndirectScalarValueAccessor< int16_t, int64_t >, ov::IndirectScalarValueAccessor< int32_t, int64_t >, ov::IndirectScalarValueAccessor< int8_t, int64_t >, ov::IndirectScalarValueAccessor< uint16_t, int64_t >, ov::IndirectScalarValueAccessor< uint32_t, int64_t >, ov::IndirectScalarValueAccessor< uint64_t, int64_t >, ov::IndirectScalarValueAccessor< uint8_t, int64_t >, ov::IndirectScalarValueAccessor< AT, VAT >, ov::IndirectVectorValueAccessor< AT, VAT >
-
template<>
class ValueAccessor<void*> : public ov::ValueAccessor<void> - #include <attribute_adapter.hpp>
-
template<>
class ValueAccessor<void> - #include <attribute_adapter.hpp>
ValueAccessor<void> provides an accessor for values that do not have get/set methods via AttributeVisitor.on_adapter.
All ValueAccessors must be derived from ValueAccessor<void> so that an AttributeVisitor only needs to implement a subset of the on_adapter methods.
Subclassed by ov::ValueAccessor< void * >, ov::VisitorAdapter
Public Functions
-
virtual const DiscreteTypeInfo &get_type_info() const = 0
type info enables identification of the value accessor, as well as is_type and as_type.
-
virtual const DiscreteTypeInfo &get_type_info() const = 0
-
class VariableState
- #include <variable_state.hpp>
VariableState class.
Public Functions
-
VariableState() = default
Default constructor.
-
~VariableState()
Destructor that preserves unloading order of implementation object and reference to the library.
-
void reset()
Resets internal variable state for relevant infer request to a value specified as default for the corresponding ReadValue node.
-
std::string get_name() const
Gets the name of the current variable state. If length of an array is not enough, the name is truncated by len, null terminator is inserted as well.
variable_idfrom the correspondingReadValueis used as variable state name.- Returns
A string representing state name.
-
Tensor get_state() const
Returns the value of the variable state.
- Returns
A tensor representing a state.
-
void set_state(const Tensor &state)
Sets the new state for the next inference.
- Parameters
state – The current state to set.
-
VariableState() = default
-
struct Version
- #include <version.hpp>
Represents version information that describes plugins and the OpemVINO library.
Public Members
-
const char *buildNumber
A null terminated string with build number.
-
const char *description
A null terminated description string.
-
const char *buildNumber
-
class VisitorAdapter : public ov::ValueAccessor<void>
- #include <attribute_adapter.hpp>
Adapters will see visitor.
Subclassed by ngraph::FactoryAttributeAdapter< BASE_TYPE >, ov::AttributeAdapter< ParameterVector >, ov::AttributeAdapter< ResultVector >, ov::AttributeAdapter< op::AutoBroadcastSpec >, ov::AttributeAdapter< op::BroadcastModeSpec >, ov::AttributeAdapter< ov::NodeVector >, ov::AttributeAdapter< std::shared_ptr< ov::Node > >
-
namespace batch_util
Functions
-
namespace cmp
Enums
-
enum Bound
Enumerate bounds to compare.
Values:
-
enumerator NONE
-
enumerator LOWER
-
enumerator UPPER
-
enumerator BOTH
-
enumerator NONE
Functions
-
template<class T, class U, typename std::enable_if<((std::is_signed<T>::value || std::is_same<T, float16>::value || std::is_same<T, bfloat16>::value) && (std::is_signed<U>::value || std::is_same<U, float16>::value || std::is_same<U, bfloat16>::value)) || (std::is_unsigned<T>::value && std::is_unsigned<U>::value)>::type* = nullptr>
constexpr bool lt(T a, U b) noexcept Compare two values (a < b) in safe way against lossy integer conversion.
- Template Parameters
T – Type of a value.
U – Type of b value.
- Parameters
a – Value a.
b – Value b.
- Returns
true if a less b otherwise false.
-
template<class T, class U>
constexpr bool gt(T a, U b) noexcept Compare two values (a > b) in safe way against lossy integer conversion.
- Template Parameters
T – Type of a value.
U – Type of b value.
- Parameters
a – Value a.
b – Value b.
- Returns
true if a > b otherwise false.
-
template<class T, Bound BMode = Bound::NONE>
class Between - #include <compare.hpp>
Compare if value is between lower and upper bounds.
The Between comparator has four modes to check value:
Bound::None (lower, upper)
Bound::LOWER [lower, upper)
Bound::UPPER (lower, upper]
Bound::BOTH [lower, upper]
- Template Parameters
T – Value type to compare.
BMode – Compare bounds mode.
-
template<class T>
class Equal - #include <compare.hpp>
Compare if value is equal to expected.
- Template Parameters
T – Value type to compare.
-
template<class T>
class Less - #include <compare.hpp>
Compare if value is less to expected.
- Template Parameters
T – Value type to compare.
-
enum Bound
-
namespace coordinates
-
namespace descriptor
Functions
- OPENVINO_API std::string get_ov_tensor_legacy_name (const Tensor &tensor)
- OPENVINO_API void set_ov_tensor_legacy_name (Tensor &tensor, const std::string &tensor_name)
- OPENVINO_API std::ostream & operator<< (std::ostream &, const ov::descriptor::Tensor &)
-
class Input
- #include <input.hpp>
Public Functions
-
Input(Node *node, size_t index, Output &output)
- Parameters
node – The node that owns this input
index – The position of this tensor in all input tensors
output – The output that supplies a value for this input
-
Input(Node *node, size_t index)
Create an Input that is not connected to an output.
- Parameters
node – The node that owns this input
index – The position of this tensor in all input tensors
-
std::shared_ptr<Node> get_node() const
- Returns
the node that this is an input of
-
inline Node *get_raw_pointer_node() const
- Returns
the raw pointer to the node that this is an input of
-
inline size_t get_index() const
- Returns
the position within all supplied tensors of this input
-
inline const Output &get_output() const
- Returns
the connected output
-
inline Output &get_output()
- Returns
the connected output
-
inline bool has_output() const
- Returns
true if an output is connected to the input.
-
const Tensor &get_tensor() const
- Returns
the tensor of the connected output
-
Tensor &get_tensor()
- Returns
the tensor of the connected output
Replace the current output that supplies a value for this input with output i of node.
-
void replace_output(Output &output)
Replace the current output that supplies a value for this input with output.
-
void remove_output()
Remove the output from this input. The node will not be valid until another output is supplied.
-
inline bool get_is_relevant_to_shape() const
See Node::set_input_is_relevant_to_shape for more details.
- Returns
true if the value of this input is relevant to the output shapes of the corresponding node. (Usually this is false.)
-
inline bool get_is_relevant_to_value() const
See Node::set_input_is_relevant_to_value for more details.
- Returns
true if the value of this input is relevant to the output value of the corresponding node. (Usually this is true.)
-
const Shape &get_shape() const
- Returns
the shape of the connected output
-
const PartialShape &get_partial_shape() const
- Returns
the partial shape of the connected output
-
Input(Node *node, size_t index, Output &output)
-
class Output
- #include <output.hpp>
Public Functions
- Parameters
node – Node that owns this output.
index – Position of the output tensor in all output tensors
tensor – The tensor where the value will be written
-
const Shape &get_shape() const
- Returns
the shape of the output
-
const PartialShape &get_partial_shape() const
- Returns
the partial shape of the output
-
class Tensor
- #include <tensor.hpp>
Compile-time descriptor of a first-class value that is a tensor.
Public Functions
-
void set_value_label(const TensorLabel &value_label)
sets value label description
-
void invalidate_values()
unsets bound value descriptions
-
inline TensorLabel get_value_label() const
gets upper bound value description
-
inline bool has_and_set_bound() const
checks if lower and upper bound are set and point to the same HostTensor
-
void set_value_label(const TensorLabel &value_label)
-
namespace detail
Functions
- template<class T> static auto collect_attached_extensions_onnx (std::vector< ov::Extension::Ptr > &res) -> decltype(typename T::template __openvino_framework_map_helper_onnx< T >().get(), void())
- template<class T> static auto collect_attached_extensions_paddle (std::vector< ov::Extension::Ptr > &res) -> decltype(typename T::template __openvino_framework_map_helper_paddle< T >().get(), void())
- template<class T> static auto collect_attached_extensions_tensorflow (std::vector< ov::Extension::Ptr > &res) -> decltype(typename T::template __openvino_framework_map_helper_tensorflow< T >().get(), void())
-
namespace device
Namespace with device properties.
Enums
-
enum class Type
Enum to define possible device types.
Values:
-
enumerator INTEGRATED
Device is integrated into host system.
-
enumerator DISCRETE
Device is not integrated into host system.
-
enumerator INTEGRATED
Variables
-
static constexpr Property<std::string> id = {"DEVICE_ID"}
the property for setting of required device to execute on values: device id starts from “0” - first device, “1” - second device, etc
-
static constexpr Priorities priorities = {"MULTI_DEVICE_PRIORITIES"}
Device Priorities config option, with comma-separated devices listed in the desired priority.
-
static constexpr Properties properties = {"DEVICE_PROPERTIES"}
Property to pass set of property values to specified device
Usage Example:
core.compile_model("HETERO" ov::device::priorities("GPU", "CPU"), ov::device::properties("CPU", ov::enable_profiling(true)), ov::device::properties("GPU", ov::enable_profiling(false)));
-
static constexpr Property<std::string, PropertyMutability::RO> full_name = {"FULL_DEVICE_NAME"}
Read-only property to get a std::string value representing a full device name.
-
static constexpr Property<std::string, PropertyMutability::RO> architecture = {"DEVICE_ARCHITECTURE"}
Read-only property which defines the device architecture.
-
static constexpr Property<UUID, PropertyMutability::RO> uuid = {"DEVICE_UUID"}
Read-only property which defines the UUID of the device.
-
static constexpr Property<LUID, PropertyMutability::RO> luid = {"DEVICE_LUID"}
Read-only property which defines the LUID of the device.
-
static constexpr Property<Type, PropertyMutability::RO> type = {"DEVICE_TYPE"}
Read-only property to get a type of device. See Type enum definition for possible return values.
-
static constexpr Property<std::map<element::Type, float>, PropertyMutability::RO> gops = {"DEVICE_GOPS"}
Read-only property which defines Giga OPS per second count (GFLOPS or GIOPS) for a set of precisions supported by specified device.
-
static constexpr Property<float, PropertyMutability::RO> thermal = {"DEVICE_THERMAL"}
Read-only property to get a float of device thermal.
-
static constexpr Property<std::vector<std::string>, PropertyMutability::RO> capabilities = {"OPTIMIZATION_CAPABILITIES"}
Read-only property to get a std::vector<std::string> of capabilities options per device.
-
struct LUID
- #include <properties.hpp>
Structure which defines format of LUID.
Public Members
-
std::array<uint8_t, MAX_LUID_SIZE> luid
Array with luid for a device.
Public Static Attributes
-
static const uint64_t MAX_LUID_SIZE = 8
Max size of luid array (64 bits)
-
std::array<uint8_t, MAX_LUID_SIZE> luid
-
struct Priorities : public ov::Property<std::string>
- #include <properties.hpp>
Type for device Priorities config option, with comma-separated devices listed in the desired priority.
-
struct Properties : public ov::Property<std::map<std::string, std::map<std::string, Any>>>
- #include <properties.hpp>
Type for property to pass set of properties to specified device.
Public Functions
-
inline std::pair<std::string, Any> operator()(const AnyMap &config) const
Constructs property.
- Parameters
configs – set of property values with names
- Returns
Pair of string key representation and type erased property value.
-
inline std::pair<std::string, Any> operator()(const std::string &device_name, const AnyMap &config) const
Constructs property.
- Parameters
device_name – device plugin alias
config – set of property values with names
- Returns
Pair of string key representation and type erased property value.
-
template<typename ...Properties>
inline util::EnableIfAllStringAny<std::pair<std::string, Any>, Properties...> operator()(const std::string &device_name, Properties&&... configs) const Constructs property.
- Template Parameters
Properties – Should be the pack of
std::pair<std::string, ov::Any>types- Parameters
device_name – device plugin alias
configs – Optional pack of pairs: (config parameter name, config parameter value)
- Returns
Pair of string key representation and type erased property value.
-
inline std::pair<std::string, Any> operator()(const AnyMap &config) const
-
struct UUID
- #include <properties.hpp>
Structure which defines format of UUID.
Public Members
-
std::array<uint8_t, MAX_UUID_SIZE> uuid
Array with uuid for a device.
Public Static Attributes
-
static const uint64_t MAX_UUID_SIZE = 16
Max size of uuid array (128 bits)
-
std::array<uint8_t, MAX_UUID_SIZE> uuid
-
namespace capability
Namespace with possible values for ov::device::capabilities property.
Variables
-
static constexpr const auto FP32 = "FP32"
Device supports fp32 inference.
-
static constexpr const auto BF16 = "BF16"
Device supports bf16 inference.
-
static constexpr const auto FP16 = "FP16"
Device supports fp16 inference.
-
static constexpr const auto INT8 = "INT8"
Device supports int8 inference.
-
static constexpr const auto INT16 = "INT16"
Device supports int16 inference.
-
static constexpr const auto BIN = "BIN"
Device supports binary inference.
-
static constexpr const auto WINOGRAD = "WINOGRAD"
Device supports winograd optimization.
-
static constexpr const auto EXPORT_IMPORT = "EXPORT_IMPORT"
Device supports compiled model export and import.
-
static constexpr const auto FP32 = "FP32"
-
enum class Type
-
namespace element
Typedefs
-
using TypeVector = std::vector<Type>
Enums
-
enum class Type_t
Enum to define possible element types.
Values:
-
enumerator undefined
Undefined element type.
-
enumerator dynamic
Dynamic element type.
-
enumerator boolean
boolean element type
-
enumerator bf16
bf16 element type
-
enumerator f16
f16 element type
-
enumerator f32
f32 element type
-
enumerator f64
f64 element type
-
enumerator i4
i4 element type
-
enumerator i8
i8 element type
-
enumerator i16
i16 element type
-
enumerator i32
i32 element type
-
enumerator i64
i64 element type
-
enumerator u1
binary element type
-
enumerator u4
u4 element type
-
enumerator u8
u8 element type
-
enumerator u16
u16 element type
-
enumerator u32
u32 element type
-
enumerator u64
u64 element type
-
enumerator nf4
nf4 element type
-
enumerator string
string element type
-
enumerator undefined
Functions
-
template<typename T>
Type from()
- template<> OPENVINO_API Type from< char > ()
- template<> OPENVINO_API Type from< bool > ()
- template<> OPENVINO_API Type from< float > ()
- template<> OPENVINO_API Type from< double > ()
- template<> OPENVINO_API Type from< int8_t > ()
- template<> OPENVINO_API Type from< int16_t > ()
- template<> OPENVINO_API Type from< int32_t > ()
- template<> OPENVINO_API Type from< int64_t > ()
- template<> OPENVINO_API Type from< uint8_t > ()
- template<> OPENVINO_API Type from< uint16_t > ()
- template<> OPENVINO_API Type from< uint32_t > ()
- template<> OPENVINO_API Type from< uint64_t > ()
- template<> OPENVINO_API Type from< ov::bfloat16 > ()
- template<> OPENVINO_API Type from< ov::float16 > ()
- template<> OPENVINO_API Type from< std::string > ()
- OPENVINO_API Type fundamental_type_for (const Type &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &out, const ov::element::Type &obj)
- OPENVINO_API std::istream & operator>> (std::istream &out, ov::element::Type &obj)
-
template<class T>
bool is_max_of(const element::Type_t &type, const T &value) Check if value has got maximum value of ov::element::Type_t.
- Template Parameters
T – Input value type.
- Parameters
type – ov::element type to get its maximum.
value – Input value for check.
- Returns
True if input value has got maximum number specified by ov::element type otherwise false.
-
template<class T>
bool is_min_of(const element::Type_t type, const T &value) Check if value has got minimum value of ov::element::Type_t.
- Template Parameters
T – Input value type.
- Parameters
type – ov::element type to get its minimum.
value – Input value for check.
- Returns
True if input value has got minimum number specified by ov::element type otherwise false.
-
template<class T, class U = T>
U get_value_or_limit_of(const element::Type_t &type, const T &value) Checks input value for element type maximum or minimum and return limit or value.
- Template Parameters
- Parameters
type – Type of ov::element::Type_t
value – Input value for check.
- Returns
If value is maximum or minimum get limit of U otherwise value as U.
-
template<Type_t...>
struct IfTypeOf Primary template defines suppoted element types.
The list of element types is used to check if runtime value of element type is one in the list. Base on this check the Visitor::visit function is called for specific element type.
- Template Parameters
List – of supported ov::element types.
-
template<Type_t ET, Type_t... Others>
struct IfTypeOf<ET, Others...> - #include <element_visitor.hpp>
Applies visitor action for supported element type defined by template parameters.
- Template Parameters
ET – Current ov::element type used for check with input.
Others – Others supported ov::element.
Public Static Functions
-
template<class Visitor, class ...Args>
static inline auto apply(Type_t et, Args&&... args) -> typename Visitor::result_type Applies visitor action if input element type is same as ET.
Uses Visitor::visit<ET> function if
et == ET, otherwise check input element type against Others.- Template Parameters
Visitor – Visitor class implementing visit function.
Args – Types of visit parameters.
- Parameters
et – Input element type.
args – Visitor arguments.
- Returns
Value of result type returned by Visitor.
-
template<>
struct IfTypeOf<> - #include <element_visitor.hpp>
Applies visitor action for not supported ov::element type.
Public Static Functions
-
template<class Visitor, class ...Args>
static inline auto apply(Type_t et, Args&&... args) -> typename Visitor::result_type Applies visitor default action if input element type is not not supported by IfTypeOf.
Uses Visitor::visit non-template function.
- Template Parameters
Visitor – Visitor class implementing visit function.
Args – Types of visit parameters.
- Parameters
et – Input element type.
args – Visitor arguments.
- Returns
Value of result type returned by Visitor.
-
template<class Visitor, class ...Args>
-
template<class R, R... value>
struct NoAction - #include <element_visitor.hpp>
Helper visitor which defines no action for not supported type.
- Template Parameters
R – Type of return value.
value – Default value returned.
-
template<>
struct NoAction<void> - #include <element_visitor.hpp>
Helper visitor which defines no action for not supported type if result is void type.
-
template<class R>
struct NotSupported - #include <element_visitor.hpp>
Helper visitor which throws ov::Exception for not supported element type.
- Template Parameters
R – Type of return value.
-
class Type
- #include <element_type.hpp>
Base class to define element type.
Public Functions
Public Static Functions
-
static bool merge(element::Type &dst, const element::Type &t1, const element::Type &t2)
Merges two element types t1 and t2, writing the result into dst and returning true if successful, else returning false.
To “merge” two element types t1 and t2 is to find the least restrictive element type t that is no more restrictive than t1 and t2, if t exists. More simply:
merge(dst,element::Type::dynamic,t) writes t to dst and returns true
merge(dst,t,element::Type::dynamic) writes t to dst and returns true
merge(dst,t1,t2) where t1, t2 both static and equal writes t1 to dst and returns true
merge(dst,t1,t2) where t1, t2 both static and unequal does nothing to dst, and returns false
-
static bool merge(element::Type &dst, const element::Type &t1, const element::Type &t2)
-
using TypeVector = std::vector<Type>
-
namespace exec_model_info
Variables
-
static const char ORIGINAL_NAMES[] = "originalLayersNames"
Used to get a string of layer names separated by a comma from the original IR, which were fused/merged to the current executable primitive.
-
static const char IMPL_TYPE[] = "primitiveType"
Used to get a type of the executable primitive.
-
static const char OUTPUT_PRECISIONS[] = "outputPrecisions"
Used to get output precisions of the executable primitive.
-
static const char PERF_COUNTER[] = "execTimeMcs"
Used to get a value of execution time of the executable primitive.
-
static const char OUTPUT_LAYOUTS[] = "outputLayouts"
Used to get output layouts of primitive.
-
static const char EXECUTION_ORDER[] = "execOrder"
Used to get an execution order of primitive.
-
static const char LAYER_TYPE[] = "layerType"
Used to get a type of primitive.
-
static const char RUNTIME_PRECISION[] = "runtimePrecision"
Used to get runtime precision of the executable primitive.
-
class ExecutionNode : public ov::op::Op
- #include <exec_model_info.hpp>
The Execution node which is used to represent node in execution graph.
It contains the following type of information in node runtime information:
ExecGraphInfoSerialization::ORIGINAL_NAMES
ExecGraphInfoSerialization::IMPL_TYPE
ExecGraphInfoSerialization::OUTPUT_PRECISIONS
ExecGraphInfoSerialization::PERF_COUNTER
ExecGraphInfoSerialization::OUTPUT_LAYOUTS
ExecGraphInfoSerialization::EXECUTION_ORDER
ExecGraphInfoSerialization::LAYER_TYPE
ExecGraphInfoSerialization::RUNTIME_PRECISION
Public Functions
-
ExecutionNode()
A default constructor with no node inputs and 0 output ports.
-
ExecutionNode(const ov::OutputVector &arguments, size_t output_size = 1)
Constructs a new execution node with a given parameters.
- Parameters
arguments – [in] Inputs nodes
output_size – [in] A number of output ports
-
std::shared_ptr<ov::Node> clone_with_new_inputs(const ov::OutputVector &inputs) const override
Creates a new execution node with the same state, but different input nodes.
- Parameters
inputs – [in] The input nodes
- Returns
A newly created execution node
-
virtual bool visit_attributes(ov::AttributeVisitor&) override
Visits attributes of the node.
- Parameters
visitor – [in] An attribute visitor
- Returns
Returns
trueif an operation has completed successfully
-
static const char ORIGINAL_NAMES[] = "originalLayersNames"
-
namespace frontend
Typedefs
-
template<typename OVOpType = void>
using OpExtension = ov::frontend::OpExtensionBase<ov::frontend::ConversionExtension, OVOpType>
-
using FrontEndFactory = std::function<FrontEnd::Ptr()>
-
using FrontEndVersion = uint64_t
Each frontend plugin is responsible to export get_api_version function returning version of frontend API used for this plugin If version is not matched with OV_FRONTEND_API_VERSION - plugin will not be loaded by FrontEndManager.
-
using NamedOutputVector = std::vector<NamedOutput>
-
using CreatorFunction = std::function<OutputVector(const NodeContext&)>
-
using CreatorFunctionNamed = std::function<std::map<std::string, OutputVector>(const NodeContext&)>
-
using CreatorFunctionNamedAndIndexed = std::function<NamedOutputVector(const NodeContext&)>
Functions
-
inline const ov::OpSet &get_opset_by_name(const std::string &opset_name)
The helper function to return an instance of OpSet class initialized with operations from provided opset by name.
- Parameters
opset_name – Opset name (opsetN) to initialize OpSet class.
-
inline std::shared_ptr<ov::Node> create_ov_node_by_name(const std::string &ov_type_name)
The helper function to create an instance of ov::Node class initialized by provided type name. Expected formats:
opsetN::OpName
opsetN.OpName
OpName
- Parameters
ov_type_name – Type name of created ov::Node.
-
inline OutputVector indexed_from_named(const NamedOutputVector &outputs)
-
inline NamedOutputVector named_from_indexed(const OutputVector &outputs)
-
class ComplexTypeMark : public ov::op::util::FrameworkNode
- #include <complex_type_mark.hpp>
Public Functions
-
inline virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
inline virtual void validate_and_infer_types() override
-
class ConversionExtension : public ov::frontend::ConversionExtensionBase
- #include <conversion.hpp>
-
class ConversionExtensionBase : public ov::Extension
- #include <conversion.hpp>
Subclassed by ov::frontend::ConversionExtension
-
class DecoderBase : public ov::frontend::IDecoder
- #include <decoder.hpp>
Public Functions
-
virtual ov::Any get_attribute(const std::string &name) const = 0
Get attribute value by name.
- Parameters
name – Attribute name
- Returns
Shared pointer to appropriate value converted to openvino data type if it exists, ‘nullptr’ otherwise
-
virtual size_t get_input_size() const = 0
Get a number of inputs.
-
virtual void get_input_node(size_t input_port_idx, std::string &producer_name, std::string &producer_output_port_name, size_t &producer_output_port_index) const = 0
Get a producer name and its output port index.
-
virtual const std::string &get_op_type() const = 0
Get operation type.
-
virtual const std::string &get_op_name() const = 0
Get node name.
-
virtual ~DecoderBase()
Destructor.
-
virtual ov::Any get_attribute(const std::string &name) const = 0
-
class DecoderTransformationExtension : public ov::Extension
- #include <decoder_transformation.hpp>
Holds a transformation that is applied just after the original model graph is decoded. This class is a holder for transformation. The transformation can be specified as FunctionPass or MathcerPass derivatives or as a function that can be used to build corresponding FunctionPass or MatcherPass object. The type of the extension is determined in the moment of creation by calling corresponding ctor.
Public Functions
Create a custom functional pass where code of the pass is implemented as a function.
-
explicit DecoderTransformationExtension(const std::function<void(ov::pass::MatcherPass*)> &matcher_pass_initializer)
Create a custom matcher pass where the code of matcher pass initialization is a given function.
-
template<typename Transformation, typename std::enable_if<std::is_base_of<ov::pass::PassBase, Transformation>::value, bool>::type = true>
inline explicit DecoderTransformationExtension(const Transformation &transformation) Register existing transformation object which will be copied and kept for further registration.
-
struct ExtensionHolder
- #include <holder.hpp>
-
class FrontEnd
- #include <frontend.hpp>
An interface for identifying a frontend for a particular framework. Provides an ability to load and convert of input model.
Public Functions
-
FrontEnd()
Default constructor.
-
template<typename ...Types>
inline bool supported(const Types&... vars) const Validates if FrontEnd can recognize model with parameters specified. Same parameters should be used to load model.
- Parameters
vars – Any number of parameters of any type. What kind of parameters are accepted is determined by each FrontEnd individually, typically it is std::string containing path to the model file. For more information please refer to specific FrontEnd documentation.
- Returns
true if model recognized, false - otherwise.
-
template<typename ...Types>
inline InputModel::Ptr load(const Types&... vars) const Loads an input model by any specified arguments. Each FrontEnd separately defines what arguments it can accept.
-
virtual std::shared_ptr<ov::Model> convert(const InputModel::Ptr &model) const
Completely convert and normalize entire Model, throws if it is not possible.
Completely convert the remaining, not converted part of a Model.
- Parameters
partiallyConverted – partially converted OV Model
-
virtual std::shared_ptr<ov::Model> convert_partially(const InputModel::Ptr &model) const
Convert only those parts of the model that can be converted leaving others as-is wrapped by FrameworkNode. Converted parts are normalized by additional transformations like it is done in convert method. If part of the graph cannot be converted, it is not guaranteed that the converted regions are completely normalized. Normalize should be called for each completely converted parts individually in this case.
-
virtual std::shared_ptr<ov::Model> decode(const InputModel::Ptr &model) const
Convert operations with one-to-one mapping with decoding nodes. Each decoding node is an OV node representing a single FW operation node with all attributes represented in FW-independent way.
Runs normalization passes on Model that was loaded with partial conversion.
- Parameters
Model – partially converted OV Model
-
virtual std::string get_name() const
Gets name of this FrontEnd. Can be used by clients if frontend is selected automatically by FrontEndManager::load_by_model.
- Returns
Current frontend name. Empty string if not implemented
Register base extension in the FrontEnd.
- Parameters
extension – base extension
Register base extensions in the FrontEnd.
- Parameters
extensions – vector of extensions
-
void add_extension(const std::string &library_path)
Registers extension.
- Parameters
library_path – path to library with ov::Extension
-
FrontEnd()
-
class FrontEndManager
- #include <manager.hpp>
Frontend management class, loads available frontend plugins on construction Allows load of frontends for particular framework, register new and list available frontends This is a main frontend entry point for client applications.
Public Functions
-
FrontEndManager()
Default constructor. Searches and loads of available frontends.
-
FrontEndManager(FrontEndManager&&) noexcept
Default move constructor.
-
FrontEndManager &operator=(FrontEndManager&&) noexcept
Default move assignment operator.
-
~FrontEndManager()
Default destructor.
-
FrontEnd::Ptr load_by_framework(const std::string &framework)
Loads frontend by name of framework and capabilities.
- Parameters
framework – Framework name. Throws exception if name is not in list of available frontends
- Returns
Frontend interface for further loading of models
-
template<typename ...Types>
inline FrontEnd::Ptr load_by_model(const Types&... vars) Loads frontend by model fragments described by each FrontEnd documentation. Selects and loads appropriate frontend depending on model file extension and other file info (header)
- Parameters
vars – Any number of parameters of any type. What kind of parameters are accepted is determined by each FrontEnd individually, typically it is std::string containing path to the model file. For more information please refer to specific FrontEnd documentation.
- Returns
Frontend interface for further loading of model. Returns ‘nullptr’ if no suitable frontend is found
-
std::vector<std::string> get_available_front_ends()
Gets list of registered frontends. Any not loaded frontends will be loaded by this call.
-
void register_front_end(const std::string &name, FrontEndFactory creator)
Register frontend with name and factory creation method.
- Parameters
name – Name of front end
creator – Creation factory callback. Will be called when frontend is about to be created
-
void register_front_end(const std::string &name, const std::string &library_path)
Register frontend with name and factory loaded from provided library.
- Parameters
name – Name of front end
library_path – Path (absolute or relative) or name of a frontend library. If name is provided, depending on platform, it will be wrapped with shared library suffix and prefix to identify library full name
-
FrontEndManager()
-
struct FrontEndPluginInfo
- #include <manager.hpp>
Each frontend plugin is responsible to export get_front_end_data function returning heap-allocated pointer to this structure. Will be used by FrontEndManager during loading of plugins.
-
class FWVisitor : public ov::AttributeVisitor
- #include <op.hpp>
Public Functions
-
inline virtual void on_adapter(const std::string &name, ValueAccessor<void> &adapter) override
handles all specialized on_adapter methods implemented by the visitor.
The adapter implements get_type_info(), which can be used to determine the adapter directly or via is_type and as_type on any platform
-
inline virtual void on_adapter(const std::string &name, ValueAccessor<void> &adapter) override
-
class FWVisitorInputAttributes : public ov::AttributeVisitor
- #include <op.hpp>
Public Functions
-
inline virtual void on_adapter(const std::string &name, ValueAccessor<void> &adapter) override
handles all specialized on_adapter methods implemented by the visitor.
The adapter implements get_type_info(), which can be used to determine the adapter directly or via is_type and as_type on any platform
-
inline virtual void on_adapter(const std::string &name, ValueAccessor<void> &adapter) override
-
class GeneralFailure : public ov::AssertFailure
- #include <exception.hpp>
-
class IDecoder
- #include <decoder.hpp>
Plays a role of node, block and module decoder.
Subclassed by ov::frontend::DecoderBase
-
class InitializationFailure : public ov::AssertFailure
- #include <exception.hpp>
-
class InputModel
- #include <input_model.hpp>
InputModel class represents an original, not yet converted model graph in a framework format given services to find places of interest in a graph or specialize/edit the model before conversion.
Editing requests may affect ability to convert the original model to OV Model. Aim to provide these editing capabilities is to unlock conversion for models that are not natively supported “as-is” because of undefined shapes, types or operations.
Specific front-end implementation is supposed to have a lazy implementation for all methods, not doing a complete load of a model without an explicit method call. For example, the list of all inputs are not pre-fetched by InputModel derived class instance creation, but only when get_inputs method is called. But it is not an obligation, the most convenient way should be chosen depending on the framework model representation.
All editing requests affect the model representation that is held behind the scene successive method calls observe a new graph structure.
Note
Class methods are divided into several groups: searching for places, naming and annotation, topology editing, setting tensor properties.
Public Functions
-
virtual std::vector<Place::Ptr> get_inputs() const
Returns all inputs for a model An input is a place in a graph where data is supposed to flow inside graph from outside. It can be a tensor, port, operation; which kind of place can be an output is FW dependent. Usually framework models have a dedicated artifact to code model input, it can be a tensor without producer, that writes to it in ONNX, or a special operation like Placeholder in TensorFlow.
- Returns
A vector of input place references
-
virtual std::vector<Place::Ptr> get_outputs() const
Returns all output for a model An output is a terminal place in a graph where data escapes the flow. It can be a tensor, port, operation; which kind of place can be an output is FW dependent. In comparison to a graph input, the output is less formally defined thing and determination of initial list of outputs may include some conventions defined by a frontend itself, not a framework. For example, all output ports without consumers may be considered as outputs.
- Returns
A vector of output place references
-
virtual Place::Ptr get_place_by_tensor_name(const std::string &tensor_name) const
Returns a tensor place by a tensor name following framework conventions, or nullptr if a tensor with this name doesn’t exist.
- Parameters
tensor_name – Name of tensor
- Returns
Tensor place corresponding to specified tensor name or nullptr if not exists
-
virtual Place::Ptr get_place_by_operation_name(const std::string &operation_name) const
Returns an operation place by an operation name following framework conventions, or nullptr if an operation with this name doesn’t exist.
- Parameters
operation_name – Name of operation
- Returns
Place representing operation or nullptr if not exists
-
virtual Place::Ptr get_place_by_operation_name_and_input_port(const std::string &operation_name, int input_port_index)
Returns an input port place by operation name and appropriate port index.
- Parameters
operation_name – Name of operation
input_port_index – Index of input port for this operation
- Returns
Place representing input port of operation or nullptr if not exists
-
virtual Place::Ptr get_place_by_operation_name_and_output_port(const std::string &operation_name, int output_port_index)
Returns an output port place by operation name and appropriate port index.
- Parameters
operation_name – Name of operation
output_port_index – Index of output port for this operation
- Returns
Place representing output port of operation or nullptr if not exists
-
virtual void set_name_for_tensor(const Place::Ptr &tensor, const std::string &new_name)
Sets name for tensor. Overwrites existing names of this place.
- Parameters
tensor – Tensor place
new_name – New name for this tensor
-
virtual void add_name_for_tensor(const Place::Ptr &tensor, const std::string &new_name)
Adds new name for tensor.
- Parameters
tensor – Tensor place
new_name – New name to be added to this place
-
virtual void set_name_for_operation(const Place::Ptr &operation, const std::string &new_name)
Sets name for operation. Overwrites existing names of this place.
- Parameters
operation – Operation place
new_name – New name for this operation
-
virtual void free_name_for_tensor(const std::string &name)
Unassign specified name from tensor place(s)
- Parameters
name – Name of tensor
-
virtual void free_name_for_operation(const std::string &name)
Unassign specified name from operation place(s)
- Parameters
name – Name of operation
-
virtual void set_name_for_dimension(const Place::Ptr &place, size_t shape_dim_index, const std::string &dim_name)
Set name for a particular dimension of a place (e.g. batch dimension)
-
virtual void cut_and_add_new_input(const Place::Ptr &place, const std::string &new_name_optional = "")
Cut immediately before this place and assign this place as new input; prune all nodes that don’t contribute to any output.
- Parameters
place – New place to be assigned as input
new_name_optional – Optional new name assigned to this input place
-
virtual void cut_and_add_new_output(const Place::Ptr &place, const std::string &new_name_optional = "")
Cut immediately after this place and assign this place as new output; prune all nodes that don’t contribute to any output.
- Parameters
place – New place to be assigned as output
new_name_optional – Optional new name assigned to this output place
-
virtual Place::Ptr add_output(const Place::Ptr &place)
Assign this place as new output or add necessary nodes to represent a new output.
- Parameters
place – Anchor point to add an output
- Returns
new output place, may be the same as a given place
-
virtual void remove_output(const Place::Ptr &place)
Removes any sinks directly attached to this place with all inbound data flow if it is not required by any other output.
- Parameters
place – Model place
-
virtual void override_all_outputs(const std::vector<Place::Ptr> &outputs)
Replaces all existing outputs with new ones removing all data flow that is not required for new outputs.
- Parameters
outputs – Vector with places that will become new outputs; may intersect existing outputs.
outputs – Array of new output places
-
virtual void override_all_inputs(const std::vector<Place::Ptr> &inputs)
Modifies the graph to use new inputs instead of existing ones. New inputs should completely satisfy all existing outputs.
- Parameters
inputs – Array of new input places
-
virtual void extract_subgraph(const std::vector<Place::Ptr> &inputs, const std::vector<Place::Ptr> &outputs)
Leaves only subgraph that are defined by new inputs and new outputs.
- Parameters
inputs – Array of new input places
outputs – Array of new output places
-
virtual void set_partial_shape(const Place::Ptr &place, const ov::PartialShape &shape)
Defines all possible shape that may be used for this place; place should be uniquely refer to some data. This partial shape will be converted to corresponding shape of results OV nodes and will define shape inference when the model is converted to OV.
- Parameters
place – Model place
shape – Partial shape for this place
-
virtual ov::PartialShape get_partial_shape(const Place::Ptr &place) const
Returns current partial shape used for this place.
- Parameters
place – Model place
- Returns
Partial shape for this place
-
virtual void set_element_type(const Place::Ptr &place, const ov::element::Type &type)
Sets new element type for a place.
- Parameters
place – Model place
type – New element type
-
virtual ov::element::Type get_element_type(const Place::Ptr &place) const
Returns current element type used for this place.
- Parameters
place – Model place
- Returns
Element type for this place
-
virtual void set_tensor_value(const Place::Ptr &place, const void *value)
Freezes a tensor with statically defined value or replace existing value for already constant node or tensor.
- Parameters
place – Tensor place
value – Value for tensor place representing a memory buffer
-
virtual void set_tensor_partial_value(const Place::Ptr &place, const void *min_value, const void *max_value)
Defines partial value (lower bound and upper bound) for a tensor place TODO: more details for min_value and max_value format; who defines shape?
- Parameters
place – Tensor place
min_value – Lower bound of partial value for tensor place
max_value – Upper bound of partial value for tensor place
-
virtual std::vector<Place::Ptr> get_inputs() const
-
struct NamedOutput
- #include <node_context.hpp>
-
class NodeContext
- #include <node_context.hpp>
Public Functions
-
inline virtual size_t get_input_size() const
Returns a number of inputs.
-
inline virtual size_t get_input_size(const std::string &port_name) const
Returns a number of inputs.
-
inline virtual Output<Node> get_input(int idx) const
Returns exactly one input with a given idx; throws if there is no inputs or there are more than one input.
-
inline virtual Output<Node> get_input(const std::string &name, int idx) const
Returns exactly one input with a given name and idx; throws if there is no inputs or there are more than one input.
-
inline virtual Output<Node> get_input(const std::string &name) const
Returns exactly one input with a given name; throws if there is no inputs or there are more than one input.
-
inline virtual Any get_values_from_const_input(int idx) const
Returns values from Constant input with the given index as ov::Any. Throws an exception if the input cannot be represented as Constant.
-
template<class T>
inline T get_attribute(const std::string &name) const Returns node attribute by name.
-
template<class T>
inline T get_attribute(const std::string &name, const T &def) const Returns node attribute by name. Returns ‘def’ value if attribute does not exist.
-
inline bool has_attribute(const std::string &name) const
Check if an attribute of a given name exist.
-
virtual ov::Any get_attribute_as_any(const std::string &name) const = 0
Returns node attribute by name as ov::Any.
-
inline virtual size_t get_subgraph_size() const
Returns the number of sub-graphs that can be enumerated with get_subgraph.
-
inline virtual std::shared_ptr<Model> get_subgraph(int idx) const
Returns subgraph converted on demand by the first access If there is no query for specific sub-graph it shouldn’t be converted idx should be in range 0..get_subgraph_size()-1.
-
inline virtual size_t get_input_size() const
-
class NotImplementedFailure : public ov::AssertFailure
- #include <exception.hpp>
-
class OpConversionFailure : public ov::AssertFailure
- #include <exception.hpp>
-
class OpConversionFunction
- #include <op.hpp>
-
class OpConversionFunctionInputAttributes
- #include <op.hpp>
-
class OpConversionFunctionNamed
- #include <op.hpp>
-
template<typename BaseConversionType, typename OVOpType = void>
class OpExtensionBase : public BaseConversionType - #include <op.hpp>
-
template<typename BaseConversionType>
class OpExtensionBase<BaseConversionType, void> : public BaseConversionType - #include <op.hpp>
-
class OpValidationFailure : public ov::AssertFailure
- #include <exception.hpp>
-
class Place
- #include <place.hpp>
An interface for identifying a place in a graph and iterate over it; can refer to an operation node, tensor, port etc.
Place can refer to Tensor, Input Edge, Input Port, Operation, Output Port, Output Edge
[Tensor A] | | [Input Edge] | V ------------------- [ [Input Port 0] ] [ ] [ Operation A ] [ ] [ [Output Port 0] ] ------------------- | | [Output Edge] | V [Tensor B] | | [Input Edge] | V ------------------- [ [Input Port 0] ] [ ] [ Operation B ] [ ] [ [Output Port 0] ] ------------------- | | [Output Edge] | V [Tensor C]
Note
Each front end implementation provides specialization of this interface to represent a place in a model graph. Various methods in the front end classes accept and retrieve instances of Place to point to particular node part which should be modified or satisfies some criteria. For example, this class is used to report model inputs and outputs, for searching operations and tensors by name, for setting shape etc.
Public Functions
-
virtual std::vector<std::string> get_names() const
All associated names (synonyms) that identify this place in the graph in a framework specific way.
- Returns
A vector of strings each representing a name that identifies this place in the graph. Can be empty if there are no names associated with this place or name cannot be attached.
-
virtual std::vector<Ptr> get_consuming_operations() const
Returns references to all operation nodes that consume data from this place.
Note
It can be called for any kind of graph place searching for the first consuming operations. It is optional if place has only one output port
- Returns
A vector with all operation node references that consumes data from this place
-
virtual std::vector<Ptr> get_consuming_operations(int output_port_index) const
Returns references to all operation nodes that consume data from this place for specified output port.
Note
It can be called for any kind of graph place searching for the first consuming operations.
- Parameters
output_port_index – If place is an operational node it specifies which output port should be considered.
- Returns
A vector with all operation node references that consumes data from this place
-
virtual std::vector<Ptr> get_consuming_operations(const std::string &outputName) const
Returns references to all operation nodes that consume data from this place for specified output port.
Note
It can be called for any kind of graph place searching for the first consuming operations.
- Parameters
outputName – If a given place is itself an operation node, this specifies name of output port group
- Returns
A vector with all operation node references that consumes data from this place
-
virtual std::vector<Ptr> get_consuming_operations(const std::string &outputName, int outputPortIndex) const
Returns references to all operation nodes that consume data from this place for specified output port.
Note
It can be called for any kind of graph place searching for the first consuming operations.
- Parameters
outputName – If a given place is itself an operation node, this specifies name of output port group, each group can have multiple ports
outputPortIndex – If place is an operational node it specifies which output port should be considered.
- Returns
A vector with all operation node references that consumes data from this place
-
virtual Ptr get_target_tensor() const
Returns a tensor place that gets data from this place; applicable for operations, output ports and output edges which have only one output port.
- Returns
A tensor place which hold the resulting value for this place
-
virtual Ptr get_target_tensor(const std::string &outputName) const
Returns a tensor place that gets data from this place; applicable for operations.
- Parameters
outputName – Name of output port group
- Returns
A tensor place which hold the resulting value for this place
-
virtual Ptr get_target_tensor(const std::string &outputName, int outputPortIndex) const
Returns a tensor place that gets data from this place; applicable for operations.
- Parameters
outputName – Name of output port group, each group can have multiple ports
outputPortIndex – Output port index if the current place is an operation node and has multiple output ports
- Returns
A tensor place which hold the resulting value for this place
-
virtual Ptr get_target_tensor(int output_port_index) const
Returns a tensor place that gets data from this place; applicable for operations.
- Parameters
output_port_index – Output port index if the current place is an operation node and has multiple output ports
- Returns
A tensor place which hold the resulting value for this place
-
virtual Ptr get_source_tensor() const
Returns a tensor place that supplies data for this place; applicable for operations, input ports and input edges which have only one input port.
- Returns
A tensor place which supplies data for this place
-
virtual Ptr get_source_tensor(int input_port_index) const
Returns a tensor place that supplies data for this place; applicable for operations.
- Parameters
input_port_index – Input port index for operational nodes.
- Returns
A tensor place which supplies data for this place
-
virtual Ptr get_source_tensor(const std::string &inputName) const
Returns a tensor place that supplies data for this place; applicable for operations.
- Parameters
inputName – Name of input port group
- Returns
A tensor place which supplies data for this place
-
virtual Ptr get_source_tensor(const std::string &inputName, int inputPortIndex) const
Returns a tensor place that supplies data for this place; applicable for operations.
- Parameters
inputName – If a given place is itself an operation node, this specifies name of output port group, each group can have multiple ports
inputPortIndex – Input port index for operational nodes.
- Returns
A tensor place which supplies data for this place
-
virtual Ptr get_producing_operation() const
Get an operation node place that immediately produces data for this place; applicable if place has only one input port.
- Returns
An operation place that produces data for this place
-
virtual Ptr get_producing_operation(int input_port_index) const
Get an operation node place that immediately produces data for this place.
- Parameters
input_port_index – If a given place is itself an operation node, this specifies a port index
- Returns
An operation place that produces data for this place
-
virtual Ptr get_producing_operation(const std::string &inputName) const
Get an operation node place that immediately produces data for this place.
- Parameters
inputName – If a given place is itself an operation node, this specifies name of output port group
- Returns
An operation place that produces data for this place
-
virtual Ptr get_producing_operation(const std::string &inputName, int inputPortIndex) const
Get an operation node place that immediately produces data for this place.
- Parameters
inputName – If a given place is itself an operation node, this specifies name of output port group, each group can have multiple ports
inputPortIndex – If a given place is itself an operation node, this specifies a port index
- Returns
An operation place that produces data for this place
-
virtual Ptr get_producing_port() const
Returns a port that produces data for this place.
-
virtual Ptr get_input_port() const
For operation node returns reference to an input port; applicable if operation node has only one input port.
- Returns
Input port place or nullptr if not exists
-
virtual Ptr get_input_port(int input_port_index) const
For operation node returns reference to an input port with specified index.
- Parameters
input_port_index – Input port index
- Returns
Appropriate input port place or nullptr if not exists
-
virtual Ptr get_input_port(const std::string &input_name) const
For operation node returns reference to an input port with specified name; applicable if port group has only one input port.
- Parameters
input_name – Name of port group
- Returns
Appropriate input port place or nullptr if not exists
-
virtual Ptr get_input_port(const std::string &input_name, int input_port_index) const
For operation node returns reference to an input port with specified name and index.
- Parameters
input_name – Name of port group, each group can have multiple ports
input_port_index – Input port index in a group
- Returns
Appropriate input port place or nullptr if not exists
-
virtual Ptr get_output_port() const
For operation node returns reference to an output port; applicable for operations with only one output port.
- Returns
Appropriate output port place or nullptr if not exists
-
virtual Ptr get_output_port(int output_port_index) const
For operation node returns reference to an output port with specified index.
- Parameters
output_port_index – Output port index
- Returns
Appropriate output port place or nullptr if not exists
-
virtual Ptr get_output_port(const std::string &output_name) const
For operation node returns reference to an output port with specified name; applicable if port group has only one output port.
- Parameters
output_name – Name of output port group
- Returns
Appropriate output port place or nullptr if not exists
-
virtual Ptr get_output_port(const std::string &output_name, int output_port_index) const
For operation node returns reference to an output port with specified name and index.
- Parameters
output_name – Name of output port group, each group can have multiple ports
output_port_index – Output port index
- Returns
Appropriate output port place or nullptr if not exists
-
virtual std::vector<Place::Ptr> get_consuming_ports() const
Returns all input ports that consume data flows through this place.
-
virtual bool is_input() const
Returns true if this place is input for a model.
-
virtual bool is_output() const
Returns true if this place is output for a model.
-
virtual bool is_equal(const Ptr &another) const
Returns true if another place is the same as this place.
- Parameters
another – Another place object
-
virtual bool is_equal_data(const Ptr &another) const
Returns true if another place points to the same data.
Note
The same data means all places on path: output port -> output edge -> tensor -> input edge -> input port.
- Parameters
another – Another place object
-
virtual std::vector<std::string> get_names() const
-
class ProgressReporterExtension : public ov::Extension
- #include <progress_reporter.hpp>
Public Types
-
using progress_notifier_callback = std::function<void(float, unsigned int, unsigned int)>
A progress reporting callback signature. A FunctionObject that matches this signature should be passed to the constructor of this extension. The extension will then invoke this as a callback each time the progress needs to be reported. The callback itself is responsible for consuming the reported values.
- Param progress
A float value in the range [0.0, 1.0] indicating the total progress of an operation.
- Param total_steps
The total number of steps that a given instance of this extension is tracking
- Param completed_completed
The current number of completed steps (out of the total number of steps to take)
Public Functions
-
inline ProgressReporterExtension()
The default constructor which creates a reporter that doesn’t report progress.
-
void report_progress(float progress, unsigned int total_steps, unsigned int completed_steps) const
The main method of this extension used to report the progress. This method forwards its arguments to the callback stored in this class.
- Parameters
progress – A float value in the range [0.0, 1.0] indicating the total progress of an operation.
total_steps – The total number of steps that a given instance of this extension is tracking
completed_steps – The current number of completed steps (out of the total number of steps to take)
-
using progress_notifier_callback = std::function<void(float, unsigned int, unsigned int)>
-
class TelemetryExtension : public ov::Extension
- #include <telemetry.hpp>
Provides callback to report telemetry information back to Python code.
-
namespace tensorflow
-
class GraphIterator : private ov::RuntimeAttribute
- #include <graph_iterator.hpp>
Abstract representation for an input model graph that gives nodes in topologically sorted order.
Public Functions
-
virtual size_t size() const = 0
Get a number of operation nodes in the graph.
-
virtual void reset() = 0
Set iterator to the start position.
-
virtual void next() = 0
Move to the next node in the graph.
-
virtual bool is_end() const = 0
Returns true if iterator goes out of the range of available nodes.
-
virtual std::shared_ptr<DecoderBase> get_decoder() const = 0
Return a pointer to a decoder of the current node.
-
virtual ~GraphIterator() = default
Destructor.
-
virtual std::shared_ptr<GraphIterator> get_body_graph_iterator(const std::string &func_name) const = 0
Checks if the main model graph contains a function of the requested name in the library Returns GraphIterator to this function and nullptr, if it does not exist.
-
virtual std::vector<std::string> get_input_names() const = 0
Returns a vector of input names in the original order.
-
virtual std::vector<std::string> get_output_names() const = 0
Returns a vector of output names in the original order.
-
inline virtual std::map<std::string, std::string> get_input_names_map() const
Returns a map from internal tensor name to (user-defined) external name for inputs.
-
inline virtual std::map<std::string, std::string> get_output_names_map() const
Returns a map from internal tensor name to (user-defined) external name for outputs.
-
virtual size_t size() const = 0
-
class GraphIterator : private ov::RuntimeAttribute
-
namespace type
-
struct List
- #include <decoder.hpp>
-
struct PyNone
- #include <decoder.hpp>
-
struct Str
- #include <decoder.hpp>
-
struct Tensor
- #include <decoder.hpp>
-
struct List
-
template<typename OVOpType = void>
-
namespace helpers
Functions
-
template<typename ACT, typename ...T, size_t N_ARGS = NumOfLambdaArgs<ACT>::value>
std::enable_if<N_ARGS == sizeof...(T) + 2, void>::type call_with_args(const ACT &body, size_t g_id, size_t iwork, T... arg)
-
template<typename T>
struct NumOfLambdaArgs - #include <parallel.hpp>
- template<typename C, typename R, typename... Args> *)(Args...) const >
- #include <parallel.hpp>
-
template<typename ACT, typename ...T, size_t N_ARGS = NumOfLambdaArgs<ACT>::value>
-
namespace hint
Namespace with hint properties.
Enums
-
enum class Priority
Enum to define possible priorities hints.
Values:
-
enumerator LOW
Low priority.
-
enumerator MEDIUM
Medium priority.
-
enumerator HIGH
High priority.
-
enumerator DEFAULT
Default priority is MEDIUM.
-
enumerator LOW
-
enum class PerformanceMode
Enum to define possible performance mode hints.
Values:
-
enumerator OPENVINO_ENUM_DEPRECATED
Undefined value, performance setting may vary from device to device.
-
enumerator LATENCY
Optimize for latency.
-
enumerator THROUGHPUT
Optimize for throughput.
-
enumerator CUMULATIVE_THROUGHPUT
Optimize for cumulative throughput.
-
enumerator OPENVINO_ENUM_DEPRECATED
-
enum class SchedulingCoreType
This enum contains definition of core type can be used for CPU tasks on different devices.
Values:
-
enumerator ANY_CORE
Any processors can be used.
-
enumerator PCORE_ONLY
Only processors of performance-cores can be used.
-
enumerator ECORE_ONLY
Only processors of efficient-cores can be used.
-
enumerator ANY_CORE
-
enum class ExecutionMode
Enum to define possible execution mode hints.
Values:
-
enumerator PERFORMANCE
Optimize for max performance, may apply properties which slightly affect accuracy.
-
enumerator ACCURACY
Optimize for max accuracy.
-
enumerator PERFORMANCE
Variables
-
static constexpr Property<element::Type, PropertyMutability::RW> inference_precision = {"INFERENCE_PRECISION_HINT"}
Hint for device to use specified precision for inference.
-
static constexpr Property<Priority> model_priority = {"MODEL_PRIORITY"}
High-level OpenVINO model priority hint Defines what model should be provided with more performant bounded resource first.
-
static constexpr Property<PerformanceMode> performance_mode = {"PERFORMANCE_HINT"}
High-level OpenVINO Performance Hints unlike low-level properties that are individual (per-device), the hints are something that every device accepts and turns into device-specific settings.
-
static constexpr Property<SchedulingCoreType> scheduling_core_type = {"SCHEDULING_CORE_TYPE"}
This property defines CPU core type which can be used during inference.
Developer can use this property to select specific CPU cores for inference. Please refer SchedulingCoreType for all definition of core type.
The following code is an example to only use efficient-cores for inference on hybrid CPU. If user sets this configuration on a platform with only performance-cores, CPU inference will still run on the performance-cores.
ie.set_property(ov::hint::scheduling_core_type(ov::hint::SchedulingCoreType::ECORE_ONLY));
-
static constexpr Property<bool> enable_cpu_pinning = {"ENABLE_CPU_PINNING"}
This property allows CPU threads pinning during inference.
Developer can use this property to use or not use CPU threads pinning during inference. If user does not explicitly set value for this property, OpenVINO may choose any desired value based on internal logic.
The following code is example to use this property.
ie.set_property(ov::hint::enable_cpu_pinning(true)); ie.set_property(ov::hint::enable_cpu_pinning(false));
-
static constexpr Property<bool> enable_hyper_threading = {"ENABLE_HYPER_THREADING"}
This property define if using hyper threading during inference.
Developer can use this property to use or not use CPU pinning during inference. If user does not explicitly set value for this property, OpenVINO may choose any desired value based on internal logic.
The following code is example to use this property.
ie.set_property(ov::hint::enable_hyper_threading(true)); ie.set_property(ov::hint::enable_hyper_threading(false));
-
static constexpr Property<uint32_t> num_requests = {"PERFORMANCE_HINT_NUM_REQUESTS"}
(Optional) property that backs the (above) Performance Hints by giving additional information on how many inference requests the application will be keeping in flight usually this value comes from the actual use-case (e.g. number of video-cameras, or other sources of inputs)
-
static constexpr Property<std::shared_ptr<ov::Model>> model = {"MODEL_PTR"}
This key identifies shared pointer to the ov::Model, required for some properties (ov::max_batch_size and ov::optimal_batch_size)
-
static constexpr Property<bool, PropertyMutability::RW> allow_auto_batching = {"ALLOW_AUTO_BATCHING"}
Special key for auto batching feature configuration. Enabled by default.
-
static constexpr Property<ExecutionMode> execution_mode = {"EXECUTION_MODE_HINT"}
High-level OpenVINO Execution hint unlike low-level properties that are individual (per-device), the hints are something that every device accepts and turns into device-specific settings Execution mode hint controls preferred optimization targets (performance or accuracy) for given model.
-
enum class Priority
-
namespace intel_auto
Namespace with Intel AUTO specific properties.
Enums
-
enum class SchedulePolicy
Enum to define the policy of scheduling inference request to target device in cumulative throughput mode on AUTO.
Values:
-
enumerator ROUND_ROBIN
-
enumerator DEVICE_PRIORITY
-
enumerator DEFAULT
Default schedule policy is DEVICE_PRIORITY.
-
enumerator ROUND_ROBIN
Variables
-
static constexpr Property<bool> enable_startup_fallback = {"ENABLE_STARTUP_FALLBACK"}
auto device setting that enable/disable CPU as acceleration (or helper device) at the beginning
-
static constexpr Property<bool> enable_runtime_fallback = {"ENABLE_RUNTIME_FALLBACK"}
auto device setting that enable/disable runtime fallback to other devices when infer fails on current selected device
-
static constexpr Property<SchedulePolicy> schedule_policy = {"SCHEDULE_POLICY"}
High-level OpenVINO model policy hint Defines what scheduling policy should be used in AUTO CUMULATIVE_THROUGHPUT or MULTI case.
-
enum class SchedulePolicy
-
namespace intel_cpu
Namespace with Intel CPU specific properties.
Variables
-
static constexpr Property<bool> denormals_optimization = {"CPU_DENORMALS_OPTIMIZATION"}
This property define whether to perform denormals optimization.
Computation with denormals is very time consuming. FTZ(Flushing denormals to zero) and DAZ(Denormals as zero) could significantly improve the performance, but it does not comply with IEEE standard. In most cases, this behavior has little impact on model accuracy. Users could enable this optimization if no or acceptable accuracy drop is seen. The following code enables denormals optimization
ie.set_property(ov::denormals_optimization(true)); // enable denormals optimization
The following code disables denormals optimization
ie.set_property(ov::denormals_optimization(false)); // disable denormals optimization
-
static constexpr Property<float> sparse_weights_decompression_rate = {"CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE"}
This property defines threshold for sparse weights decompression feature activation.
Sparse weights decompression feature allows to pack weights for Matrix Multiplication operations directly in the CPU plugin at the model compilation stage and store non-zero values in a special packed format. Then, during the execution of the model, the weights are unpacked and used in the computational kernel. Since the weights are loaded from DDR/L3 cache in the packed format this significantly decreases memory consumption and as a consequence improve inference performance. The following code allows to set the sparse rate value.
core.set_property(ov::intel_cpu::sparse_weights_decompression_rate(0.8));
-
static constexpr Property<bool> denormals_optimization = {"CPU_DENORMALS_OPTIMIZATION"}
-
namespace intel_gna
Namespace with Intel GNA specific properties.
Enums
-
enum class ExecutionMode
Enum to define software acceleration mode.
Values:
-
enumerator AUTO
Uses Intel GNA if available, otherwise uses software execution mode on CPU.
-
enumerator HW
Uses Intel GNA if available, otherwise raises an error.
-
enumerator HW_WITH_SW_FBACK
Uses Intel GNA if available, otherwise raises an error. If the hardware queue is not empty, automatically falls back to CPU in the bit-exact mode.
-
enumerator SW_EXACT
Executes the GNA-compiled graph on CPU performing calculations in the same precision as the Intel GNA in the bit-exact mode.
-
enumerator SW_FP32
Executes the GNA-compiled graph on CPU but substitutes parameters and calculations from low precision to floating point
-
enumerator AUTO
-
enum class HWGeneration
Enum to define HW compile and execution targets.
Values:
-
enumerator UNDEFINED
GNA HW generation is undefined.
-
enumerator GNA_1_0
GNA HW generation 1.0.
-
enumerator GNA_1_0_E
GNA HW generation 1.0 embedded.
-
enumerator GNA_2_0
GNA HW generation 2.0.
-
enumerator GNA_3_0
GNA HW generation 3.0.
-
enumerator GNA_3_1
GNA HW generation 3.1.
-
enumerator GNA_3_5
GNA HW generation 3.5.
-
enumerator GNA_3_5_E
GNA HW generation 3.5 embedded.
-
enumerator GNA_3_6
GNA HW generation 3.6.
-
enumerator GNA_4_0
GNA HW generation 4.0.
-
enumerator UNDEFINED
-
enum class PWLDesignAlgorithm
Enum to define PWL design algorithm.
Values:
-
enumerator UNDEFINED
PWL approximation algorithm is undefined.
-
enumerator RECURSIVE_DESCENT
Recursive Descent Algorithm.
-
enumerator UNIFORM_DISTRIBUTION
Uniform distribution algorithm.
-
enumerator UNDEFINED
Variables
-
static constexpr Property<std::string, PropertyMutability::RO> library_full_version = {"GNA_LIBRARY_FULL_VERSION"}
Property to get an std::string of GNA Library version, usually in the form <API_REVISION>.<RELEASE_LINE>.<RELEASE>.<BUILD>
-
static constexpr Property<std::map<std::string, float>> scale_factors_per_input = {"GNA_SCALE_FACTOR_PER_INPUT"}
Scale factor provided by the user to use static quantization. This option should be used with floating point value serialized to string with . (dot) as a decimal separator.
In the case of multiple inputs, individual scale factors can be provided using the map where key is layer name and value is scale factor. The input name shall not contain symbol “:”. Example:
ov::Core core; auto model = core.read_model(model_path); std::map<std::string, float> scale_factors; for (auto& input : model->inputs()) { scale_factors[input.get_any_name()] = 1.0f; } core.set_property("GNA", ov::intel_gna::scale_factors_per_input(scale_factors));
-
static constexpr Property<std::string> firmware_model_image_path = {"GNA_FIRMWARE_MODEL_IMAGE"}
if turned on, dump GNA firmware model into specified file
-
static constexpr Property<ExecutionMode> execution_mode = {"GNA_DEVICE_MODE"}
GNA proc_type setting that should be one of AUTO, HW, GNA_HW_WITH_SW_FBACK, GNA_SW_EXACT or SW_FP32.
-
static constexpr Property<HWGeneration> execution_target = {"GNA_HW_EXECUTION_TARGET"}
The option to override the GNA HW execution target. May be one of GNA_2_0, GNA_3_0, GNA_3_5. By default (in case of no value set) the behavior depends on GNA HW availability: If GNA HW is present, use the option corresponding to this HW. If HW is not present, use the option corresponding to the latest fully supported GNA HW generation. A fully supported GNA HW generation means it must be supported by both the OV GNA Plugin and the core GNA Library. Currently, the latest supported GNA HW generation corresponds to GNA_3_5.
-
static constexpr Property<HWGeneration> compile_target = {"GNA_HW_COMPILE_TARGET"}
The option to override the GNA HW compile target. May be one of GNA_2_0, GNA_3_0, GNA_3_5. By default the same as execution_target.
-
static constexpr Property<bool> memory_reuse = {"GNA_COMPACT_MODE"}
if enabled produced minimum memory footprint for compiled model in GNA memory, default value is true
-
static constexpr Property<PWLDesignAlgorithm> pwl_design_algorithm = {"GNA_PWL_DESIGN_ALGORITHM"}
The option to set PWL design algorithm. By default the optimized algorithm called “Recursive Descent Algorithm for Finding
the Optimal Minimax Piecewise Linear Approximation of Convex Functions” is used. If value is UNIFORM_DISTRIBUTION then simple uniform distribution is used to create PWL approximation of activation functions. Uniform distribution usually gives poor approximation with the same number of segments.
-
static constexpr Property<float> pwl_max_error_percent = {"GNA_PWL_MAX_ERROR_PERCENT"}
The option to allow to specify the maximum error percent that the optimized algorithm finding will be used to find PWL functions. By default (in case of NO value set), 1.0 value is used.
-
enum class ExecutionMode
-
namespace intel_gpu
Namespace with Intel GPU specific properties.
Typedefs
-
using gpu_handle_param = void*
Enums
-
enum class ContextType
Enum to define the type of the shared context.
Values:
-
enumerator OCL
Pure OpenCL context.
-
enumerator VA_SHARED
Context shared with a video decoding device.
-
enumerator OCL
Enum to define the type of the shared memory buffer.
Values:
Shared OpenCL buffer blob.
Shared OpenCL 2D image blob.
Shared USM pointer allocated by user.
Shared USM pointer type with host allocation type allocated by plugin.
Shared USM pointer type with device allocation type allocated by plugin.
Shared video decoder surface or D3D 2D texture blob.
Shared D3D buffer blob.
Variables
-
static constexpr Property<uint64_t, PropertyMutability::RO> device_total_mem_size = {"GPU_DEVICE_TOTAL_MEM_SIZE"}
Read-only property which defines size of memory in bytes available for the device. For iGPU it returns host memory size, for dGPU - dedicated gpu memory size.
-
static constexpr Property<std::string, PropertyMutability::RO> uarch_version = {"GPU_UARCH_VERSION"}
Read-only property to get microarchitecture identifier in major.minor.revision format.
-
static constexpr Property<int32_t, PropertyMutability::RO> execution_units_count = {"GPU_EXECUTION_UNITS_COUNT"}
Read-only property to get count of execution units for current GPU.
-
static constexpr Property<std::map<std::string, uint64_t>, PropertyMutability::RO> memory_statistics{"GPU_MEMORY_STATISTICS"}
Read-only property to get statistics of GPU memory allocated by engine for each allocation type It contains information about current memory usage.
-
static constexpr Property<bool> enable_loop_unrolling = {"GPU_ENABLE_LOOP_UNROLLING"}
Turning on this key enables to unroll recurrent layers such as TensorIterator or Loop with fixed iteration count. This key is turned on by default. Turning this key on will achieve better inference performance for loops with not too many iteration counts (less than 16, as a rule of thumb). Turning this key off will achieve better performance for both graph loading time and inference time with many iteration counts (greater than 16). Note that turning this key on will increase the graph loading time in proportion to the iteration counts. Thus, this key should be turned off if graph loading time is considered to be most important target to optimize.
-
static constexpr Property<bool> disable_winograd_convolution = {"GPU_DISABLE_WINOGRAD_CONVOLUTION"}
Turning on this key disables winograd convolution. Winograd convolution has different characteristics for accuracy and performance compared to other convolution implementations.
-
static constexpr Property<ContextType> context_type = {"CONTEXT_TYPE"}
Shared device context type: can be either pure OpenCL (OCL) or shared video decoder (VA_SHARED) context.
-
static constexpr Property<gpu_handle_param> ocl_context = {"OCL_CONTEXT"}
This key identifies OpenCL context handle in a shared context or shared memory blob parameter map.
-
static constexpr Property<int> ocl_context_device_id = {"OCL_CONTEXT_DEVICE_ID"}
This key identifies ID of device in OpenCL context if multiple devices are present in the context.
-
static constexpr Property<int> tile_id = {"TILE_ID"}
In case of multi-tile system, this key identifies tile within given context.
-
static constexpr Property<gpu_handle_param> ocl_queue = {"OCL_QUEUE"}
This key identifies OpenCL queue handle in a shared context.
-
static constexpr Property<gpu_handle_param> va_device = {"VA_DEVICE"}
This key identifies video acceleration device/display handle in a shared context or shared memory blob parameter map.
This key identifies type of internal shared memory in a shared memory blob parameter map.
-
static constexpr Property<gpu_handle_param> mem_handle = {"MEM_HANDLE"}
This key identifies OpenCL memory handle in a shared memory blob parameter map.
-
static constexpr Property<gpu_handle_param> dev_object_handle = {"DEV_OBJECT_HANDLE"}
This key identifies video decoder surface handle in a shared memory blob parameter map.
-
static constexpr Property<uint32_t> va_plane = {"VA_PLANE"}
This key identifies video decoder surface plane in a shared memory blob parameter map.
-
namespace capability
Possible return value for ov::device::capabilities property.
Variables
-
static constexpr const auto HW_MATMUL = "GPU_HW_MATMUL"
Device has hardware block for matrix multiplication.
-
static constexpr const auto HW_MATMUL = "GPU_HW_MATMUL"
-
namespace hint
Typedefs
-
using ThrottleLevel = ov::hint::Priority
This enum represents the possible value of ov::intel_gpu::hint::queue_throttle property:
LOW is used for CL_QUEUE_THROTTLE_LOW_KHR OpenCL throttle hint
MEDIUM (DEFAULT) is used for CL_QUEUE_THROTTLE_MED_KHR OpenCL throttle hint
HIGH is used for CL_QUEUE_THROTTLE_HIGH_KHR OpenCL throttle hint
Variables
-
static constexpr Property<ThrottleLevel> queue_throttle = {"GPU_QUEUE_THROTTLE"}
This key instructs the GPU plugin to use OpenCL queue throttle hints as defined in https://www.khronos.org/registry/OpenCL/specs/opencl-2.1-extensions.pdf, chapter 9.19. This option should be used with ov::intel_gpu::hint::ThrottleLevel values.
-
static constexpr Property<ov::hint::Priority> queue_priority = {"GPU_QUEUE_PRIORITY"}
This key instructs the GPU plugin to use the OpenCL queue priority hint as defined in https://www.khronos.org/registry/OpenCL/specs/opencl-2.1-extensions.pdf. This option should be used with ov::hint::Priority:
LOW is used for CL_QUEUE_PRIORITY_LOW_KHR OpenCL priority hint
MEDIUM (DEFAULT) is used for CL_QUEUE_PRIORITY_MED_KHR OpenCL priority hint
HIGH is used for CL_QUEUE_PRIORITY_HIGH_KHR OpenCL priority hint
-
static constexpr Property<ov::hint::Priority> host_task_priority = {"GPU_HOST_TASK_PRIORITY"}
This key instructs the GPU plugin which cpu core type of TBB affinity used in load network. This option has 3 types of levels: HIGH, LOW, and ANY. It is only affected on Hybrid CPUs.
LOW - instructs the GPU Plugin to use LITTLE cores if they are available
MEDIUM (DEFAULT) - instructs the GPU Plugin to use any available cores (BIG or LITTLE cores)
HIGH - instructs the GPU Plugin to use BIG cores if they are available
-
static constexpr Property<int64_t> available_device_mem = {"AVAILABLE_DEVICE_MEM_SIZE"}
This key identifies available device memory size in bytes.
-
using ThrottleLevel = ov::hint::Priority
-
namespace memory_type
These keys instruct the GPU plugin to use surface/buffer memory type.
Variables
-
static constexpr auto surface = "GPU_SURFACE"
Native video decoder surface.
-
static constexpr auto surface = "GPU_SURFACE"
-
namespace ocl
Namespace with Intel GPU OpenCL specific remote objects.
Typedefs
-
using gpu_handle_param = void*
Shortcut for defining a handle parameter.
-
class ClBufferTensor : public ov::RemoteTensor
- #include <ocl.hpp>
This class represents an abstraction for GPU plugin remote tensor which can be shared with user-supplied OpenCL buffer. The plugin object derived from this class can be obtained with ClContext::create_tensor() call.
Note
User can obtain OpenCL buffer handle from this class.
Subclassed by ov::intel_gpu::ocl::D3DBufferTensor
Public Functions
-
inline cl_mem get()
Returns the underlying OpenCL memory object handle.
- Returns
underlying OpenCL memory object handle
-
inline operator cl_mem()
OpenCL memory handle conversion operator.
- Returns
cl_mem
-
inline operator cl::Buffer()
Standard Khronos cl::Buffer wrapper conversion operator.
- Returns
cl::Bufferobject
Public Static Functions
-
static inline void type_check(const Tensor &tensor)
Checks that type defined runtime parameters are presented in remote object.
- Parameters
tensor – a tensor to check
-
inline cl_mem get()
-
class ClContext : public ov::RemoteContext
- #include <ocl.hpp>
This class represents an abstraction for GPU plugin remote context which is shared with OpenCL context object. The plugin object derived from this class can be obtained either with CompiledModel::get_context() or Core::create_context() calls.
Subclassed by ov::intel_gpu::ocl::D3DContext, ov::intel_gpu::ocl::VAContext
Public Functions
-
inline ClContext(Core &core, cl_context ctx, int ctx_device_id = 0)
Constructs context object from user-supplied OpenCL context handle.
- Parameters
core – A reference to OpenVINO Runtime Core object
ctx – A OpenCL context to be used to create shared remote context
ctx_device_id – An ID of device to be used from ctx
-
inline ClContext(Core &core, cl_command_queue queue)
Constructs context object from user-supplied OpenCL context handle.
Note
Only latency mode is supported for such context sharing case.
- Parameters
core – A reference to OpenVINO Runtime Core object
queue – An OpenCL queue to be used to create shared remote context. Queue will be reused inside the plugin.
-
inline cl_context get()
Returns the underlying OpenCL context handle.
- Returns
cl_context
-
inline operator cl_context()
OpenCL context handle conversion operator for the ClContext object.
- Returns
cl_context
-
inline operator cl::Context()
Standard Khronos cl::Context wrapper conversion operator for the ClContext object.
- Returns
cl::Contextobject
-
inline std::pair<ClImage2DTensor, ClImage2DTensor> create_tensor_nv12(const cl::Image2D &nv12_image_plane_y, const cl::Image2D &nv12_image_plane_uv)
This function is used to construct a NV12 compound tensor object from two cl::Image2D wrapper objects. The resulting compound contains two remote tensors for Y and UV planes of the surface.
- Parameters
nv12_image_plane_y – cl::Image2D object containing Y plane data.
nv12_image_plane_uv – cl::Image2D object containing UV plane data.
- Returns
A pair of remote tensors for each plane
-
inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl_mem buffer)
This function is used to obtain remote tensor object from user-supplied cl_mem object.
-
inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl::Buffer &buffer)
This function is used to obtain remote tensor object from user-supplied cl::Buffer object.
-
inline ClImage2DTensor create_tensor(const element::Type type, const Shape &shape, const cl::Image2D &image)
This function is used to obtain remote tensor object from user-supplied cl::Image2D object.
-
inline USMTensor create_tensor(const element::Type type, const Shape &shape, void *usm_ptr)
This function is used to obtain remote tensor object from user-supplied USM pointer.
-
inline USMTensor create_usm_host_tensor(const element::Type type, const Shape &shape)
This function is used to allocate USM tensor with host allocation type.
-
inline USMTensor create_usm_device_tensor(const element::Type type, const Shape &shape)
This function is used to allocate USM tensor with device allocation type.
-
RemoteTensor create_tensor(const element::Type &type, const Shape &shape, const AnyMap ¶ms = {})
Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.
- Parameters
type – Defines the element type of the tensor.
shape – Defines the shape of the tensor.
params – Map of the low-level tensor object parameters.
- Returns
Pointer to a plugin object that implements the RemoteTensor interface.
Public Static Functions
-
static inline void type_check(const RemoteContext &remote_context)
Checks that type defined runtime parameters are presented in remote object.
- Parameters
remote_context – A remote context to check
-
inline ClContext(Core &core, cl_context ctx, int ctx_device_id = 0)
-
class ClImage2DTensor : public ov::RemoteTensor
- #include <ocl.hpp>
This class represents an abstraction for GPU plugin remote tensor which can be shared with user-supplied OpenCL 2D Image. The plugin object derived from this class can be obtained with ClContext::create_tensor() call.
Note
User can obtain OpenCL image handle from this class.
Subclassed by ov::intel_gpu::ocl::D3DSurface2DTensor, ov::intel_gpu::ocl::VASurfaceTensor
Public Functions
-
inline cl_mem get()
Returns the underlying OpenCL memory object handle.
- Returns
underlying OpenCL memory object handle
-
inline operator cl_mem()
OpenCL memory handle conversion operator.
- Returns
cl_mem
-
inline operator cl::Image2D()
Standard Khronos cl::Image2D wrapper conversion operator for the ClContext object.
- Returns
cl::Image2Dobject
Public Static Functions
-
static inline void type_check(const Tensor &tensor)
Checks that type defined runtime parameters are presented in remote object.
- Parameters
tensor – a tensor to check
-
inline cl_mem get()
-
class D3DBufferTensor : public ov::intel_gpu::ocl::ClBufferTensor
- #include <dx.hpp>
This class represents an abstraction for GPU plugin remote tensor which is shared with Direct3D 11 buffer. The plugin object derived from this class can be obtained with D3DContext::create_tensor() call.
Note
User can also obtain OpenCL buffer handle from this class.
Public Functions
-
inline operator ID3D11Buffer*()
ID3D11Buffer conversion operator for the D3DContext object.
- Returns
Pointer to underlying ID3D11Buffer interface
Public Static Functions
-
static inline void type_check(const Tensor &tensor)
Checks that type defined runtime parameters are presented in remote object.
- Parameters
tensor – a tensor to check
-
inline operator ID3D11Buffer*()
-
class D3DContext : public ov::intel_gpu::ocl::ClContext
- #include <dx.hpp>
This class represents an abstraction for GPU plugin remote context which is shared with Direct3D 11 device. The plugin object derived from this class can be obtained either with CompiledModel::get_context() or Core::create_context() calls.
Note
User can also obtain OpenCL context handle from this class.
Public Functions
-
inline operator ID3D11Device*()
ID3D11Device conversion operator for the D3DContext object.
- Returns
Pointer to underlying ID3D11Device interface
-
inline D3DContext(Core &core, ID3D11Device *device, int target_tile_id = -1)
Constructs D3DContext remote context object from ID3D11Device.
- Parameters
core – OpenVINO Runtime Core object instance
device – A pointer to ID3D11Device to be used to create a remote context
target_tile_id – Desired tile id within given context for multi-tile system. Default value (-1) means that root device should be used
-
inline std::pair<D3DSurface2DTensor, D3DSurface2DTensor> create_tensor_nv12(const size_t height, const size_t width, ID3D11Texture2D *nv12_surf)
This function is used to obtain a NV12 tensor from NV12 DXGI video decoder output. The resulting tensor contains two remote tensors for Y and UV planes of the surface.
- Parameters
height – Height of Y plane
width – Width of Y plane
nv12_surf – A ID3D11Texture2D instance to create NV12 tensor from
- Returns
A pair of remote tensors for each plane
-
inline D3DBufferTensor create_tensor(const element::Type type, const Shape &shape, ID3D11Buffer *buffer)
This function is used to obtain remote tensor object from ID3D11Buffer.
-
inline D3DSurface2DTensor create_tensor(const element::Type type, const Shape &shape, ID3D11Texture2D *surface, uint32_t plane = 0)
This function is used to obtain remote tensor object from ID3D11Texture2D.
Note
The underlying ID3D11Texture2D can also be a plane of output surface of DXGI video decoder
- Parameters
- Returns
D3DSurface2DTensor tensor
-
inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl_mem buffer)
This function is used to obtain remote tensor object from user-supplied cl_mem object.
-
inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl::Buffer &buffer)
This function is used to obtain remote tensor object from user-supplied cl::Buffer object.
-
inline ClImage2DTensor create_tensor(const element::Type type, const Shape &shape, const cl::Image2D &image)
This function is used to obtain remote tensor object from user-supplied cl::Image2D object.
Public Static Functions
-
static inline void type_check(const RemoteContext &remote_context)
Checks that type defined runtime parameters are presented in remote object.
- Parameters
remote_context – A remote context to check
-
inline operator ID3D11Device*()
-
class D3DSurface2DTensor : public ov::intel_gpu::ocl::ClImage2DTensor
- #include <dx.hpp>
This class represents an abstraction for GPU plugin remote tensor which is shared with Direct3D 11 2D texture. The plugin object derived from this class can be obtained with D3DContext::create_tensor() call.
Note
User can also obtain OpenCL 2D image handle from this class.
Public Functions
-
inline operator ID3D11Texture2D*()
ID3D11Texture2D conversion operator for the D3DContext object.
- Returns
Pointer to underlying ID3D11Texture2D interface
-
inline uint32_t plane()
Returns plane ID of underlying video decoder surface, or 0 if no video surface was shared.
- Returns
Plane ID
Public Static Functions
-
static inline void type_check(const Tensor &remote_tensor)
Checks that type defined runtime parameters are presented in remote object.
- Parameters
remote_tensor – remote tensor to check
-
inline operator ID3D11Texture2D*()
-
class USMTensor : public ov::RemoteTensor
- #include <ocl.hpp>
This class represents an abstraction for GPU plugin remote tensor which can be shared with user-supplied USM device pointer. The plugin object derived from this class can be obtained with ClContext::create_tensor() call.
Note
User can obtain USM pointer from this class.
Public Functions
-
inline void *get()
Returns the underlying USM pointer.
- Returns
underlying USM pointer
Public Static Functions
-
static inline void type_check(const Tensor &tensor)
Checks that type defined runtime parameters are presented in remote object.
- Parameters
tensor – a tensor to check
-
inline void *get()
-
class VAContext : public ov::intel_gpu::ocl::ClContext
- #include <va.hpp>
This class represents an abstraction for GPU plugin remote context which is shared with VA display object. The plugin object derived from this class can be obtained either with CompiledModel::get_context() or Core::create_context() calls.
Note
User can also obtain OpenCL context handle from this class.
Public Functions
-
inline operator VADisplay()
VADisplayconversion operator for the VAContext object.- Returns
Underlying
VADisplayobject handle
-
inline VAContext(Core &core, VADisplay device, int target_tile_id = -1)
Constructs remote context object from VA display handle.
- Parameters
core – OpenVINO Runtime Core object
device – A
VADisplayto create remote context fromtarget_tile_id – Desired tile id within given context for multi-tile system. Default value (-1) means that root device should be used
-
inline std::pair<VASurfaceTensor, VASurfaceTensor> create_tensor_nv12(const size_t height, const size_t width, const VASurfaceID nv12_surf)
This function is used to obtain a NV12 tensor from NV12 VA decoder output. The resulting tensor contains two remote tensors for Y and UV planes of the surface.
- Parameters
height – A height of Y plane
width – A width of Y plane
nv12_surf – NV12
VASurfaceIDto create NV12 from
- Returns
A pair of remote tensors for each plane
-
inline VASurfaceTensor create_tensor(const element::Type type, const Shape &shape, const VASurfaceID surface, const uint32_t plane = 0)
This function is used to create remote tensor from VA surface handle.
-
inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl_mem buffer)
This function is used to obtain remote tensor object from user-supplied cl_mem object.
-
inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl::Buffer &buffer)
This function is used to obtain remote tensor object from user-supplied cl::Buffer object.
-
inline ClImage2DTensor create_tensor(const element::Type type, const Shape &shape, const cl::Image2D &image)
This function is used to obtain remote tensor object from user-supplied cl::Image2D object.
-
inline USMTensor create_tensor(const element::Type type, const Shape &shape, void *usm_ptr)
This function is used to obtain remote tensor object from user-supplied USM pointer.
-
RemoteTensor create_tensor(const element::Type &type, const Shape &shape, const AnyMap ¶ms = {})
Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.
- Parameters
type – Defines the element type of the tensor.
shape – Defines the shape of the tensor.
params – Map of the low-level tensor object parameters.
- Returns
Pointer to a plugin object that implements the RemoteTensor interface.
Public Static Functions
-
static inline void type_check(const RemoteContext &remote_context)
Checks that type defined runtime parameters are presented in remote object.
- Parameters
remote_context – A remote context to check
-
inline operator VADisplay()
-
class VASurfaceTensor : public ov::intel_gpu::ocl::ClImage2DTensor
- #include <va.hpp>
This class represents an abstraction for GPU plugin remote tensor which is shared with VA output surface. The plugin object derived from this class can be obtained with VAContext::create_tensor() call.
Note
User can also obtain OpenCL 2D image handle from this class.
Public Functions
-
inline operator VASurfaceID()
VASurfaceID conversion operator for the VASurfaceTensor object.
- Returns
VASurfaceIDhandle
-
inline uint32_t plane()
Returns plane ID of underlying video decoder surface.
- Returns
Plane ID
Public Static Functions
-
static inline void type_check(const Tensor &tensor)
Checks that type defined runtime parameters are presented in remote object.
- Parameters
tensor – a tensor to check
-
inline operator VASurfaceID()
-
using gpu_handle_param = void*
-
using gpu_handle_param = void*
-
namespace internal
Variables
-
static constexpr Property<std::vector<PropertyName>, PropertyMutability::RO> supported_properties{"INTERNAL_SUPPORTED_PROPERTIES"}
Read-only property to get a std::vector<PropertyName> of supported internal properties.
-
static constexpr Property<std::vector<PropertyName>, PropertyMutability::RO> caching_properties = {"CACHING_PROPERTIES"}
Read-only property to get a std::vector<PropertyName> of properties which should affect the hash calculation for model cache.
-
static constexpr Property<bool, PropertyMutability::RW> exclusive_async_requests = {"EXCLUSIVE_ASYNC_REQUESTS"}
Allow to create exclusive_async_requests with one executor.
-
static constexpr Property<std::string, PropertyMutability::WO> config_device_id = {"CONFIG_DEVICE_ID"}
the property for setting of required device for which config to be updated values: device id starts from “0” - first device, “1” - second device, etc note: plugin may have different devices naming convention
-
static constexpr Property<ov::threading::IStreamsExecutor::ThreadBindingType, PropertyMutability::RW> cpu_bind_thread{"CPU_BIND_THREAD"}
The name for setting CPU affinity per thread option.
It is passed to Core::get_property()
The following options are implemented only for the TBB as a threading option ov::threading::IStreamsExecutor::ThreadBindingType::NUMA (pinning threads to NUMA nodes, best for real-life, contented cases) on the Windows and MacOS* this option behaves as YES ov::threading::IStreamsExecutor::ThreadBindingType::HYBRID_AWARE (let the runtime to do pinning to the cores types, e.g. prefer the “big” cores for latency tasks) on the hybrid CPUs this option is default
Also, the settings are ignored, if the OpenVINO compiled with OpenMP and any affinity-related OpenMP’s environment variable is set (as affinity is configured explicitly)
-
static constexpr Property<size_t, PropertyMutability::RW> threads_per_stream = {"THREADS_PER_STREAM"}
Limit #threads that are used by IStreamsExecutor to execute
parallel_forcalls.
-
static constexpr Property<std::string, PropertyMutability::RO> compiled_model_runtime_properties{"COMPILED_MODEL_RUNTIME_PROPERTIES"}
It contains compiled_model_runtime_properties information to make plugin runtime can check whether it is compatible with the cached compiled model, the result is returned by get_property() calling.
The information details are defined by plugin itself, each plugin may require different runtime contents. For example, CPU plugin will contain OV version, while GPU plugin will contain OV and GPU driver version, etc. Core doesn’t understand its content and only read it from plugin and write it into blob header.
-
static constexpr Property<bool, PropertyMutability::RO> compiled_model_runtime_properties_supported{"COMPILED_MODEL_RUNTIME_PROPERTIES_SUPPORTED"}
Check whether the attached compiled_model_runtime_properties is supported by this device runtime.
-
static constexpr Property<std::vector<PropertyName>, PropertyMutability::RO> supported_properties{"INTERNAL_SUPPORTED_PROPERTIES"}
-
namespace itt
-
namespace domains
Functions
-
OV_ITT_DOMAIN(ov_eval)
-
OV_ITT_DOMAIN(ov_eval)
-
namespace domains
-
namespace layout
Functions
- OPENVINO_API bool has_batch (const Layout &layout)
Checks if layout has ‘batch’ dimension.
- OPENVINO_API std::int64_t batch_idx (const Layout &layout)
Returns ‘batch’ dimension index.
- Throws
ov::AssertFailure – if dimension doesn’t exist.
- OPENVINO_API bool has_channels (const Layout &layout)
Checks if layout has ‘channels’ dimension.
- Throws
ov::AssertFailure – if dimension doesn’t exist.
- OPENVINO_API std::int64_t channels_idx (const Layout &layout)
Returns ‘channels’ dimension index.
- Throws
ov::AssertFailure – if dimension doesn’t exist.
- OPENVINO_API bool has_depth (const Layout &layout)
Checks if layout has ‘depth’ dimension.
- OPENVINO_API std::int64_t depth_idx (const Layout &layout)
Returns ‘depth’ dimension index.
- Throws
ov::AssertFailure – if dimension doesn’t exist.
- OPENVINO_API bool has_height (const Layout &layout)
Checks if layout has ‘height’ dimension.
- OPENVINO_API std::int64_t height_idx (const Layout &layout)
Returns ‘height’ dimension index.
- Throws
ov::AssertFailure – if dimension doesn’t exist.
- OPENVINO_API bool has_width (const Layout &layout)
Checks if layout has ‘width’ dimension.
- OPENVINO_API std::int64_t width_idx (const Layout &layout)
Returns ‘width’ dimension index.
- Throws
ov::AssertFailure – if dimension doesn’t exist.
- OPENVINO_API void set_layout (ov::Output< ov::Node > output, const ov::Layout &layout)
Sets Layout of port.
- OPENVINO_API ov::Layout get_layout (const ov::Output< ov::Node > &output)
Gets Layout of port.
- Returns
layout from port and empty layout in other case
- OPENVINO_API ov::Layout get_layout (const ov::Output< const ov::Node > &output)
Gets Layout of port.
- Returns
layout from port and empty layout in other case
-
namespace legacy_convert
-
-
class TensorHolder
- #include <remote_utils.hpp>
Subclassed by ov::TensorRemoteBlob
-
class TensorHolder
-
namespace log
Namespace with log level property and its possible values.
Enums
-
enum class Level
Enum to define possible log levels.
Values:
-
enumerator NO
disable any logging
-
enumerator ERR
error events that might still allow the application to continue running
-
enumerator WARNING
potentially harmful situations which may further lead to ERROR
-
enumerator INFO
informational messages that display the progress of the application at coarse-grained level
-
enumerator DEBUG
fine-grained events that are most useful to debug an application.
-
enumerator TRACE
finer-grained informational events than the DEBUG
-
enumerator NO
-
enum class Level
-
namespace op
Enums
-
enum class GeluApproximationMode
Specifies the approximation to calculate Gelu.
Values:
-
enumerator TANH
-
enumerator ERF
-
enumerator TANH
-
enum class LSTMWeightsFormat
Values:
-
enumerator FICO
-
enumerator ICOF
-
enumerator IFCO
-
enumerator IFOC
-
enumerator IOFC
-
enumerator FICO
-
enum class MVNEpsMode
Specifies how eps is applied in MVN.
Values:
-
enumerator INSIDE_SQRT
-
enumerator OUTSIDE_SQRT
-
enumerator INSIDE_SQRT
-
enum class PadMode
Modes for the
Padoperator.Values:
-
enumerator CONSTANT
-
enumerator EDGE
-
enumerator REFLECT
-
enumerator SYMMETRIC
-
enumerator CONSTANT
-
enum class PadType
Padding Type used for
ConvolutionandPoolingFollows ONNX padding type definitions EXPLICIT - Pad dimensions are explicity specified SAME_LOWER - Pad dimensions computed to match input shape Ceil(num_dims/2) at the beginning and Floor(num_dims/2) at the end SAME_UPPER - Pad dimensions computed to match input shape Floor(num_dims/2) at the beginning and Ceil(num_dims/2) at the end VALID - No padding AUTO - Deprecated. User should not use it in the future NOTSET - Deprecated. User should not use it in the future
Values:
-
enumerator EXPLICIT
-
enumerator SAME_LOWER
-
enumerator SAME_UPPER
-
enumerator VALID
-
enumerator AUTO
-
enumerator NOTSET
-
enumerator EXPLICIT
-
enum class RoundingType
Rounding Type used for
Poolingoperators.Values:
-
enumerator FLOOR
-
enumerator CEIL
-
enumerator FLOOR
-
enum class AutoBroadcastType
Specifies the algorithm to use for implicit broadcasting of a tensor to align with another tensor.
NONE - No implicit broadcasting of tensor NUMPY - Numpy-style implicit broadcasting (https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html) Right-align dimensions of the two tensors, with missing dimensions treated as size 1 dimensions. After alignment, for each dimension, their sizes should either match or one of them should be of size 1. Size 1 dimension will be implicitly broadcast to match the other size.
E.g., A: Shape(2, 1, 6) B: Shape( 3, 1) Result: Shape(2, 3, 6)
Result: Shape(2, 3, 6) PDPD - PaddlePaddle-style implicit broadcasting (https://github.com/PaddlePaddle/Paddle/blob/release/1.5/paddle/ fluid/operators/elementwise/elementwise_op.h#L126) Broadcast B to match the shape of A, where axis is the start dimension index to align B with A. If axis is -1 (default), i axis = rank(A) - rank(B). The trailing dimensions of size 1 for B will be ignored.A: Shape(2, 1, 6) B: Shape( 3, 1)
E.g., A: Shape(2, 3, 4, 5) B: Shape( 3, 4 ) with axis =1 Result: Shape(2, 3, 4, 5)
Result: Shape(2, 3, 4, 5)A: Shape(2, 3, 4, 5) B: Shape( 3, 1 ) with axis = 1
Values:
-
enumerator NONE
-
enumerator EXPLICIT
-
enumerator NUMPY
-
enumerator PDPD
-
enumerator NONE
-
enum class BroadcastType
BroadcastType specifies rules used for mapping of input tensor axes to output shape axes.
EXPLICIT - Mapping of the input data shape to output shape based on axes_mapping input. NUMPY - Numpy broadcasting rules, aligned with ONNX Broadcasting. (https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md) PDPD - PaddlePaddle-style implicit broadcasting. For more informaction see AutoBroadcastType documentation. BIDIRECTIONAL - The broadcast rule is similar to numpy.array(input) * numpy.ones(target_shape). Dimensions are right alignment.
Note
Broadcasting rules are different for Broadcast op and for element-wise ops. AutoBroadcastType::NUMPY is equivalent of BroadcastType::BIDIRECTIONAL according to spec.
Values:
-
enumerator NONE
-
enumerator EXPLICIT
-
enumerator NUMPY
-
enumerator PDPD
-
enumerator BIDIRECTIONAL
-
enumerator NONE
-
enum class EpsMode
Specifies how eps is combined with L2 value.
Values:
-
enumerator ADD
-
enumerator MAX
-
enumerator ADD
-
enum class TopKSortType
Values:
-
enumerator NONE
-
enumerator SORT_INDICES
-
enumerator SORT_VALUES
-
enumerator NONE
-
enum class TopKMode
Values:
-
enumerator MAX
-
enumerator MIN
-
enumerator MAX
-
enum class RecurrentSequenceDirection
This class defines possible recurrent sequence directions.
Values:
-
enumerator FORWARD
-
enumerator REVERSE
-
enumerator BIDIRECTIONAL
-
enumerator FORWARD
Functions
- std::unordered_map< size_t, std::pair< ov::Tensor, ov::Tensor > > OPENVINO_API convert_input_types (OutputVector &inputs, const element::TypeVector &types)
- ov::TensorVector OPENVINO_API get_output_tensors_of_original_type (const ov::TensorVector &fake_output_tensors, const element::TypeVector &types)
- void OPENVINO_API reset_input_types (const std::unordered_map< size_t, std::pair< ov::Tensor, ov::Tensor > > &original_input_vals, OutputVector &inputs)
- bool OPENVINO_API convert_outputs_to_fake_type (ov::TensorVector &outputs, ov::TensorVector &original_outputs, bool is_upper)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const GeluApproximationMode &type)
-
ov::op::util::LSTMWeightsFormat convert_lstm_weights_enums(LSTMWeightsFormat format)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const MVNEpsMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const v12::ScatterElementsUpdate::Reduction &reduction)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const PadMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const PadType &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const RoundingType &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const AutoBroadcastType &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const BroadcastType &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const EpsMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const TopKSortType &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const TopKMode &type)
- OPENVINO_API std::ostream & operator<< (std::ostream &s, const RecurrentSequenceDirection &direction)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> copy_shape_infer(const Node *op, const std::vector<TShape> &input_shapes)
-
template<class OpType, class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> eltwise_shape_infer(const OpType *op, const std::vector<T> &input_shapes)
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const util::FFTBase *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::GatherBase *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::ConvertColorI420Base *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::MulticlassNmsBase *op, const std::vector<TShape> &input_shapes, const bool static_output = !std::is_same<PartialShape, TShape>::value, const bool ignore_bg_class = false)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::ConvertColorNV12Base *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::PadBase *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> reduce_shape_infer(const util::ReductionBase *op, bool keep_dims, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::ArithmeticReductionKeepDims *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::LogicalReductionKeepDims *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::ScatterElementsUpdateBase *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::ScatterNDBase *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const util::TopKBase *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor()) TopK shape inference.
- Template Parameters
TShape – Type of shape.
- Parameters
op – Pointer to TopK operator.
input_shapes – Input shapes of TopK.
constant_data – Map of constant data. DEfault empty.
- Returns
Vector of output shapes for
-
template<class TShape, class TData, class TRes = std::vector<TData>, class UnaryOperation = ov::util::Cast<TData>, typename std::enable_if<!std::is_same<TShape, ov::PartialShape>::value>::type* = nullptr>
std::unique_ptr<TRes> get_input_const_data_as(const ov::Node *op, size_t idx, const ITensorAccessor &tensor_accessor, UnaryOperation &&func = ov::util::Cast<TData>()) Get the operator’s input const as pointer to vector of specified type.
The behaviour depends on shape type. The default output type is std::vector<TData> can be replace by other type which if is possible to construct it from constant data vector.
The behaviour depends on shape type. The default output type is std::vector<TData> can be replace by other type which if is possible to construct it from constant data vector.
- Template Parameters
TShape – Shape type which enabled this version (not ov::PartialShape)
TData – Type use to cast input’s data.
TRes – Result type which has got default type as std::vector<TData>.
UnaryOperation – Unary function object applied on data with signature (Ret f(const TData &a)).
TShape – Shape type which enabled this version (ov::PartialShape)
TData – Type use to cast input’s data.
TRes – Result type which has got default type as std::vector<TData>.
UnaryOperation – Unary function object applied on data with signature (Ret f(const TData &a)).
- Parameters
- Returns
Pointer to constant data or nullptr if input has no constant data.
- Returns
Pointer to constant data or nullptr if input has no constant data.
-
template<class TShape, class TDimValue = typename TShape::value_type::value_type, class UnaryOperation = ov::util::InTypeRange<TDimValue>, typename std::enable_if<!std::is_same<TShape, ov::PartialShape>::value>::type* = nullptr>
ov::optional<TShape> get_input_const_data_as_shape(const ov::Node *op, size_t port, const ITensorAccessor &tensor_accessor, UnaryOperation &&func = ov::util::InTypeRange<TDimValue>()) Get the input const data as shape object.
The input data can be processed by unary operation. By default is validated and casted to shape’s dimension type.
- Template Parameters
- Parameters
- Returns
Unique pointer to shape created from input data.
-
inline element::Type get_input_const_element_type(const ov::Node *const op, size_t port, const ITensorAccessor &ta)
-
template<class TShape, class TData, class TResult = std::vector<std::pair<TData, TData>>>
ov::optional<TResult> get_input_bounds(const ov::Node *op, size_t port, const ITensorAccessor &ta) Get the input bounds from constant input or try evaluate bunds and return them as vector of pairs (lower, upper).
-
ov::Shape infer_broadcast_shape(const ov::Node *const op, const ov::Shape &first, const ov::Shape &second)
Inference broadcast shape for element wise operator according to broadcast specification stored in operator.
- Parameters
op – Pointer to operator.
first – First input shape.
second – Second input shape.
- Returns
Result shape from inputs with applied broadcast specification.
-
ov::Shape infer_broadcast_shape(const ov::Node *const op, const ov::TensorVector &inputs)
Inference broadcast shape from input tensor shapes for element wise operator according to broadcast specification stored in operator.
- Parameters
op – Pointer to operator.
inputs – Tensors vector to get theirs shapes.
- Returns
Result shape from input tensors shape with applied broadcast specification.
-
struct AutoBroadcastSpec
- #include <attr_types.hpp>
Implicit broadcast specification.
-
struct BroadcastModeSpec
- #include <attr_types.hpp>
Implicit broadcast specification.
-
class Op : public ov::Node
- #include <op.hpp>
Root of all actual ops.
Subclassed by ov::exec_model_info::ExecutionNode, ov::op::Sink, ov::op::internal::NonMaxSuppressionIEInternal, ov::op::internal::RPE, ov::op::util::BinaryElementwiseArithmetic, ov::op::util::BinaryElementwiseBitwise, ov::op::util::BinaryElementwiseComparison, ov::op::util::BinaryElementwiseLogical, ov::op::util::BroadcastBase, ov::op::util::ConvertColorI420Base, ov::op::util::ConvertColorNV12Base, ov::op::util::ConvolutionBase, ov::op::util::DetectionOutputBase, ov::op::util::EmbeddingBagOffsetsBase, ov::op::util::EmbeddingBagPackedBase, ov::op::util::FFTBase, ov::op::util::GatherBase, ov::op::util::GatherNDBase, ov::op::util::IndexReduction, ov::op::util::InterpolateBase, ov::op::util::MaxPoolBase, ov::op::util::MultiSubGraphOp, ov::op::util::MulticlassNmsBase, ov::op::util::PadBase, ov::op::util::RNNCellBase, ov::op::util::ReadValueBase, ov::op::util::ReductionBase, ov::op::util::ScatterBase, ov::op::util::ScatterElementsUpdateBase, ov::op::util::ScatterNDBase, ov::op::util::ShapeOfBase, ov::op::util::TopKBase, ov::op::util::UnaryElementwiseArithmetic, ov::op::v0::BatchNormInference, ov::op::v0::CTCGreedyDecoder, ov::op::v0::Concat, ov::op::v0::Constant, ov::op::v0::Convert, ov::op::v0::CumSum, ov::op::v0::DepthToSpace, ov::op::v0::FakeQuantize, ov::op::v0::HardSigmoid, ov::op::v0::Interpolate, ov::op::v0::LRN, ov::op::v0::MVN, ov::op::v0::MatMul, ov::op::v0::NormalizeL2, ov::op::v0::PRelu, ov::op::v0::PSROIPooling, ov::op::v0::Parameter, ov::op::v0::PriorBox, ov::op::v0::PriorBoxClustered, ov::op::v0::Proposal, ov::op::v0::ROIPooling, ov::op::v0::Range, ov::op::v0::RegionYolo, ov::op::v0::ReorgYolo, ov::op::v0::Result, ov::op::v0::ReverseSequence, ov::op::v0::Selu, ov::op::v0::ShuffleChannels, ov::op::v0::SpaceToDepth, ov::op::v0::Squeeze, ov::op::v0::Tile, ov::op::v0::Unsqueeze, ov::op::v10::IsFinite, ov::op::v10::IsInf, ov::op::v10::IsNaN, ov::op::v10::Unique, ov::op::v12::GroupNormalization, ov::op::v13::BitwiseNot, ov::op::v13::FakeConvert, ov::op::v13::Multinomial, ov::op::v13::NMSRotated, ov::op::v13::ScaledDotProductAttention, ov::op::v1::AvgPool, ov::op::v1::BatchToSpace, ov::op::v1::ConvertLike, ov::op::v1::DeformablePSROIPooling, ov::op::v1::GatherTree, ov::op::v1::LogicalNot, ov::op::v1::NonMaxSuppression, ov::op::v1::OneHot, ov::op::v1::Reshape, ov::op::v1::Reverse, ov::op::v1::Select, ov::op::v1::Softmax, ov::op::v1::SpaceToBatch, ov::op::v1::Split, ov::op::v1::StridedSlice, ov::op::v1::Transpose, ov::op::v1::VariadicSplit, ov::op::v3::Bucketize, ov::op::v3::EmbeddingSegmentsSum, ov::op::v3::ExtractImagePatches, ov::op::v3::NonMaxSuppression, ov::op::v3::NonZero, ov::op::v3::ROIAlign, ov::op::v4::CTCLoss, ov::op::v4::Range, ov::op::v4::Swish, ov::op::v5::BatchNormInference, ov::op::v5::LogSoftmax, ov::op::v5::NonMaxSuppression, ov::op::v6::CTCGreedyDecoderSeqLen, ov::op::v6::ExperimentalDetectronDetectionOutput, ov::op::v6::ExperimentalDetectronGenerateProposalsSingleImage, ov::op::v6::ExperimentalDetectronPriorGridGenerator, ov::op::v6::ExperimentalDetectronROIFeatureExtractor, ov::op::v6::ExperimentalDetectronTopKROIs, ov::op::v6::GatherElements, ov::op::v6::MVN, ov::op::v7::Einsum, ov::op::v7::Roll, ov::op::v8::AdaptiveAvgPool, ov::op::v8::AdaptiveMaxPool, ov::op::v8::MatrixNms, ov::op::v8::PriorBox, ov::op::v8::RandomUniform, ov::op::v8::Slice, ov::op::v8::Softmax, ov::op::v9::Eye, ov::op::v9::GenerateProposals, ov::op::v9::GridSample, ov::op::v9::NonMaxSuppression, ov::op::v9::ROIAlign
-
class Sink : public ov::op::Op
- #include <sink.hpp>
Root of nodes that can be sink nodes.
Subclassed by ov::op::util::AssignBase
-
class TemporaryReplaceOutputType
- #include <type_relaxed.hpp>
Set another type for a specified output for the period of time when an instance of the class exists. When the execution leaves the scope where an onject of TemporaryReplaceOutputType is defined, the type of the output is set to its original value. Used when initialized TypeRelaxed<BaseOp> operation in case when inputs have types that are not compatible with BaseOp infer function. In this case before TypeRelaxed is constructed the BaseOp contructor requires modified data types. So it should be
-
template<typename BaseOp>
class TypeRelaxed : public BaseOp, public ov::op::TypeRelaxedBase - #include <type_relaxed.hpp>
Relaxes tensor element type requirements for BaseOp inputs and outputs This class template should be used with Node descendant class. Defines a new operation by extending the original BaseOp operation with ability to accept inputs and provide outputs with element type that is unusual for BaseOp. For example, TypeRelaxed<opset1::Add> can accept mixed-precision inputs and provide another type of output. New types are provided as inputs attributes for TypeRelaxed template and fixed. There is no any deduction logic for types are provided as a part of this class and it should be implemented outside if required.
Public Functions
-
template<typename ...Args>
inline TypeRelaxed(const element::TypeVector &_input_data_types, const element::TypeVector &_output_data_types, Args&&... args) Creating a new TypeRelaxed operation by calling one of the original op ctors forwarding arguments directly.
-
template<typename ...Args>
-
class TypeRelaxedBase
- #include <type_relaxed.hpp>
A base class for templated TypeRelaxed that maintains overridden input types and output types for an operation.
Subclassed by ov::op::TypeRelaxed< BaseOp >
Public Functions
-
inline const element::Type &get_overridden_output_type(size_t outputIndex = 0) const
This method may look similar to Node::get_output_element_type, but it is not the same thing, because get_output_element_type returns the result of type inference, so it is completely deduced from an operation inputs and attributes, and get_overridden_output_type returns value of the attribute that is used to deduce output type. In some cases they don’t match: get_overridden_output_type may return element::undefined for some index i, and get_output_element_type will return some real type for the same index i.
- Returns
Data type that will be set for output with a given index outputIndex. If output with a specified index outputIndex hasn’t been set before, element::undefined will returned. Undefined means no type override happens for a given outputIndex and it will deduced as original operation defineds in its infer function.
-
inline void set_overridden_output_type(const element::Type &element_type, size_t outputIndex = 0)
Set data type that overrides the original data type for output port with outputIndex index In case if outputIndex is out of range of known outputs (and this class cannot detect the real number of outputs for original operation), the number of overridden outputs is changed according to a given outputIndex value.
-
inline const element::Type &get_origin_input_type(size_t inputIndex = 0) const
- Returns
Data type that will be set for input when original shape/type inference function is called. If index inputIndex hasn’t been set before, element::undefined will returned. Undefined means that the type from input tensor descriptor is used for a given index.
-
inline void set_origin_input_type(const element::Type &element_type, size_t inputIndex = 0)
Set data type that overrides the original data type for input port with inputIndex index. In case if inputIndex is out of range of known inputs (and this class cannot detect the real number of inputs for original operation), the number of overridden inputs is changed according to a given inputIndex value. All new entries except one added at inputIndex position are undefined.
-
inline const element::Type &get_overridden_output_type(size_t outputIndex = 0) const
-
namespace convolution
Functions
-
template<class TOp, class TShape, typename std::enable_if<std::is_base_of<util::ConvolutionBackPropBase, TOp>::value>::type* = nullptr>
size_t calculate_num_spatial(const TOp *op, const std::vector<TShape> &input_shapes, const result_shape_t<TShape> &out_spatial_shape)
-
template<class TOp, class TShape, class TIter>
void apply_auto_pad(const TOp *op, const TShape &data_shape, const TShape &filters_shape, const result_shape_t<TShape> &out_spatial_shape, TIter pads_begin, TIter pads_end) Apply auto padding for backward convolution.
The auto padding can be applied only if inputs and attributes of operator are validated. The input shapes must have got static ranks.
- Parameters
-
template<class TShape>
void apply_padding(const util::ConvolutionBackPropBase *op, const std::vector<TShape> &input_shapes, const result_shape_t<TShape> &out_spatial_shape, CoordinateDiff &pads_begin, CoordinateDiff &pads_end) Apply auto padding for back propagation convolutions.
-
template<class TOp, class TShape, class TContainer, typename std::enable_if<std::is_base_of<ov::op::util::ConvolutionBackPropBase, TOp>::value>::type* = nullptr>
void append_spatial_shape(const TOp *op, const TShape &data_shape, const TShape &filters_shape, const TContainer &pads_begin, const TContainer &pads_end, result_shape_t<TShape> &out_shape) Append spatial dimension at end of output shape of back propagation convolution.
-
template<class TConv>
constexpr size_t filter_non_spatial_dims_count() Provides convolution filter non spatial dimension count.
Note
If specific convolution operator requires different value provide specialization for this operator.
- Template Parameters
TConv – Type of convolution operator.
- Returns
Default value for convolution operators (2).
-
template<class TOp>
bool is_auto_pad(const TOp *op) Checks if Op property auto_pad is set to same lower or upper.
- Template Parameters
TOp – Type of operator (must have get_auto_pad member function).
- Parameters
op – Pointer to operator.
- Returns
True if auto pad enabled.
-
inline void resize_empty_padding(const size_t num_spatial, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)
Resize paddings if empty to number of spatial dimensions.
- Parameters
num_spatial – Number of spatial dimensions.
pads_begin – Begin padding to resize.
pads_end – End padding to resize.
-
inline size_t num_spatial_from_attr(const util::ConvolutionBase *op)
-
template<class TOp, class TShape, typename std::enable_if<std::is_base_of<util::ConvolutionFwdPropBase, TOp>::value>::type* = nullptr>
size_t calculate_num_spatial(const TOp *op, const std::vector<TShape> &input_shapes)
-
template<class TOp, class TShape, class TIter, typename std::enable_if<std::is_base_of<util::ConvolutionFwdPropBase, TOp>::value || std::is_base_of<util::DeformableConvolutionBase, TOp>::value>::type* = nullptr>
void apply_auto_pad(const TOp *op, const TShape &data_shape, const TShape &filters_shape, TIter pads_begin, TIter pads_end) Apply auto padding for forward convolution.
The auto padding can be applied only if inputs and attributes of operator are validated. The input shapes must have got static ranks.
-
template<class TOp, class TShape, typename std::enable_if<std::is_base_of<util::ConvolutionFwdPropBase, TOp>::value || std::is_base_of<util::DeformableConvolutionBase, TOp>::value>::type* = nullptr>
void apply_padding(const TOp *op, const TShape &data_shape, const TShape &filters_shape, CoordinateDiff &pads_begin, CoordinateDiff &pads_end) Apply padding to forward propagation convolution besed on padding.
-
template<class TOp, class TShape, class TRShape = result_shape_t<TShape>, typename std::enable_if<std::is_base_of<util::ConvolutionFwdPropBase, TOp>::value || std::is_base_of<util::DeformableConvolutionBase, TOp>::value>::type* = nullptr>
void append_spatial_shape(const TOp *op, const TShape &data_shape, const TShape &filters_shape, CoordinateDiff &pads_begin, CoordinateDiff &pads_end, TRShape &out_shape) Append spatial dimension at end of output shape of forward propagation convolution.
-
template<>
constexpr size_t filter_non_spatial_dims_count<v1::GroupConvolutionBackpropData>() Defines non-spatial dimension for filters for group convolution back propagation operator.
- Returns
Value of non-spatial filter dimensions (3).
-
template<>
constexpr size_t filter_non_spatial_dims_count<v1::GroupConvolution>() Defines non-spatial dimension for filters for group convolution operator.
- Returns
Value of non-spatial filter dimensions (3).
Variables
-
constexpr auto num_spatial_undefined = util::num_spatial_undefined
-
constexpr size_t spatial_dim_offset = 2
-
namespace validate
Functions
-
template<class TShape>
void data_shape(const v1::BinaryConvolution *op, const TShape &data_shape) Specific check of data shape for binary convolution data shape must be rank 4.
The shape_infer is same as for Convolution operator except this check.
See also
convolution_shape_inference.hpp
-
template<class TShape>
void filter_shape(const ov::op::util::ConvolutionBackPropBase *op, const TShape &filters_shape, const TShape &data_shape)
-
template<class TShape>
void data_shape(const ov::op::util::ConvolutionBase *op, const TShape &data_shape)
-
template<class TShape>
void filter_shape(const ov::op::util::ConvolutionBase *op, const TShape &filters_shape, const TShape &data_shape)
-
inline void common_attributes(const util::ConvolutionBase *op, const size_t num_spatial, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end)
-
inline void common_attributes(const util::ConvolutionBackPropBase *op, const size_t num_spatial, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end)
-
template<class TShape>
-
template<class TOp, class TShape, typename std::enable_if<std::is_base_of<util::ConvolutionBackPropBase, TOp>::value>::type* = nullptr>
-
namespace deformable_conv
Functions
-
template<class TShape>
size_t calculate_num_spatial(const util::DeformableConvolutionBase *op, const std::vector<TShape> &input_shapes)
-
namespace validate
Functions
-
template<class TDeformableConv>
void group_attribute(const TDeformableConv *op, int64_t group, const std::string &name)
-
template<class TDeformableConv, class TDim>
void group_divisible_dimension(const TDeformableConv *op, const TDim &dim, const std::string name)
-
template<class TDeformableConv, class TDim>
void deformable_group_divisible_dimension(const TDeformableConv *op, const TDim &dim, const std::string name)
-
template<class TDeformableConv>
-
template<class TShape>
-
namespace detectron
-
namespace validate
Functions
-
std::pair<std::vector<PartialShape>, element::Type> all_inputs_same_floating_type(const Node *const op)
Validates if all op’s inputs have got same floating type and return inputs shapes and element type.
- Parameters
op – Pointer to detector operator.
- Returns
Input shapes and element type as pair.
-
std::pair<std::vector<PartialShape>, element::Type> all_inputs_same_floating_type(const Node *const op)
-
namespace validate
-
namespace eye
Variables
-
constexpr std::array<char const*, 4> shape_names = {"'num_rows'", "'num_columns'", "'diagonal_index'", "'batch_shape'"}
-
constexpr std::array<char const*, 4> shape_names = {"'num_rows'", "'num_columns'", "'diagonal_index'", "'batch_shape'"}
-
namespace fft
Functions
-
template<class TRShape, typename std::enable_if<std::is_same<TRShape, PartialShape>::value>::type* = nullptr>
void apply_dims_from_sizes(const util::FFTBase *op, TRShape &output_shape, const std::vector<int64_t> &axes, const ITensorAccessor &ta)
-
template<class TRShape, typename std::enable_if<std::is_same<TRShape, PartialShape>::value>::type* = nullptr>
-
namespace gather_nd
-
namespace internal
Functions
-
class AUGRUCell : public ov::op::util::RNNCellBase
- #include <augru_cell.hpp>
AUGRUCell operation.
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class AUGRUSequence : public ov::op::util::RNNCellBase
- #include <augru_sequence.hpp>
AUGRUSequence operation.
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class GenerateProposalsIEInternal : public ov::op::v9::GenerateProposals
- #include <generate_proposals_ie_internal.hpp>
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class MulticlassNmsIEInternal : public opset9::MulticlassNms
- #include <multiclass_nms_ie_internal.hpp>
-
template<typename BaseNmsOp>
class NmsStaticShapeIE : public BaseNmsOp - #include <nms_static_shape_ie.hpp>
-
class NonMaxSuppressionIEInternal : public ov::op::Op
- #include <nms_ie_internal.hpp>
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class RPE : public ov::op::Op
- #include <rotary_positional_embeddings.hpp>
Rotary Positional Embeddings operation Internal operation which may change in the future.
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class AUGRUCell : public ov::op::util::RNNCellBase
-
namespace interpolate
Functions
-
template<class T, class U, typename std::enable_if<std::is_same<T, U>::value>::type* = nullptr>
constexpr bool is_same_instance(const T &lhs, const U &rhs)
-
template<class TContainer>
void resize_padding(const ov::op::util::InterpolateBase *op, size_t input_rank, TContainer &pads_begin, TContainer &pads_end) Resize padding to input rank.
- Template Parameters
TContainer – Pads container type.
- Parameters
op – Pointer to base of interpolate.
input_rank – Expected padding size.
pads_begin – Begin padding container.
pads_end – End padding container.
-
template<class TShape, class TInputIter, class TRShape = result_shape_t<TShape>>
TRShape make_padded_shape(const TShape &input, TInputIter pads_begin, TInputIter pads_end) Makes padded shape from input shapes and padding values.
Note
The input shape must be static rank and padding count must match input shape rank.
-
template<class TShape, class TRes = std::vector<int64_t>>
std::unique_ptr<TRes> get_axes(const Node *const op, size_t port, bool has_axes, size_t rank, const ITensorAccessor &ta) Get the axes for interpolate from constant input or default value if op has no axes input.
- Parameters
op – Pointer to operator.
port – Axes input port number.
has_axes – Flag if op has input with axes.
rank – input shape used for axes values validation.
ta – Tensor accessor for input data.
- Returns
Not null pointer with axes values or null pointer if can’t get axes from input.
-
template<class TShape, class TContainer>
void set_undefined_dim_on_axes(TShape &out, TContainer &axes) Set the undefined dimensions on specified axes.
- Parameters
out – Output shape to update.
axes – List of axes for update.
-
template<class TShape, class TContainer>
void update_dims_with_sizes_on_axes(TShape &out_shape, const TContainer &axes, const Node *const op, const size_t port, const ITensorAccessor &ta) Update output shape with dimension size from input on specified axes.
-
template<class TShape>
void update_dims_with_scales_on_axes(TShape &out_shape, const std::vector<int64_t> &axes, const Node *const op, const size_t port, const ITensorAccessor &ta) Update output shape by scaling dimensions on axes.
-
namespace validate
Functions
-
template<class TShape>
void input_rank_1d(const Node *const op, const std::vector<TShape> &shapes, size_t port) Validates that input at port number from is 1-D rank.
- Template Parameters
TShape –
- Parameters
op – Pointer to operator.
shapes – Vector of op’s input shapes.
port – Port number.
-
template<class TShape>
void are_inputs_except_first_1d(const Node *const op, const std::vector<TShape> &shapes) Validates that inputs from 2nd to last are compatible 1-D rank.
- Template Parameters
TShape –
- Parameters
op – Pointer to operator.
shapes – Vector of op’s input shapes.
-
template<class TContainer>
void axes_values(const Node *const op, const TContainer &axes, size_t rank) Check if axes values in range [0,rank].
- Template Parameters
TContainer – Type of axes container.
- Parameters
op – Pointer to operator.
axes – Container with axes to check.
rank – Maximum value for axes values.
-
inline void input_elements_num(const Node *const op, const std::string &input_name, size_t element_count, size_t exp_count)
Check if number of elements in input is same as expected.
- Parameters
op – Pointer to operator.
input_name – Input name.
element_count – Element count in tested input.
exp_count – Expected element count on tested input.
-
template<class TShape>
-
template<class T, class U, typename std::enable_if<std::is_same<T, U>::value>::type* = nullptr>
-
namespace multiclass_nms
-
namespace validate
-
namespace validate
-
namespace multinomial
-
namespace validate
Functions
-
void input_types(const Node *op)
-
void input_types(const Node *op)
-
namespace validate
-
namespace nms
Functions
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Node *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Node *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta, const bool static_output)
-
namespace validate
Functions
-
template<class TShape>
bool scalar(const TShape &shape)
-
template<class TShape>
bool scalar_or_1d_tensor_with_1_element(const TShape &shape)
-
template<class TShape>
void boxes_shape(const Node *const op, const std::vector<TShape> &input_shapes)
-
template<class TShape>
void scores_shape(const Node *const op, const std::vector<TShape> &input_shapes)
-
template<class TShape>
void num_batches(const Node *const op, const std::vector<TShape> &input_shapes)
-
template<class TShape>
-
template<class T, class TRShape = result_shape_t<T>>
-
namespace pad
Functions
-
constexpr bool is_inf_padding(const std::pair<int64_t, int64_t> &pad_bounds)
-
constexpr bool is_inf_padding(const std::pair<int64_t, int64_t> &pad_bounds)
-
namespace pooling
Functions
-
template<>
inline void valid_dilated_kernel_with_padding(const v1::AvgPool *op, const size_t kernel, const size_t pad_begin, const size_t pad_end, const size_t axis)
-
template<class TContainer>
void resize_empty_padding(const size_t num_spatial, TContainer &pads_begin, TContainer &pads_end) Resize paddings if empty to number of spatial dimensions.
- Parameters
num_spatial – Number of spatial dimensions.
pads_begin – Begin padding to resize.
pads_end – End padding to resize.
-
template<class TOp, class TShape, class TContainer>
void apply_padding(const TOp *op, const TShape &data_shape, const Strides &dilations, TContainer &pads_begin, TContainer &pads_end) Apply pooling operator padding depends on auto pad value.
- Parameters
op – Pointer to Pooling operator to apply padding.
data_shape – Shape infer data input shape.
dilations – Kernel dilations.
pads_begin – Padding begin to update.
pads_end – Padding end to update.
-
template<class TOp, class TDim>
void valid_dilated_kernel_with_dim(const TOp *op, const size_t kernel, const TDim &dim, const size_t axis)
-
template<class TOp>
void valid_dilated_kernel_with_padding(const TOp *op, const size_t kernel, const size_t pad_begin, const size_t pad_end, const size_t axis)
-
template<class TOp, class TShape, class TContainer, class TRShape>
void append_spatial_shape(const TOp *op, const TShape &data_shape, const TContainer &pads_begin, const TContainer &pads_end, const Strides &dilations, TRShape &out_shape) Append spatial shape to the end of output shape for pooling operator shape inference result.
-
template<class TOp, class TShape, class TContainer, class TRShape = result_shape_t<TShape>>
TRShape out_shape_infer(const TOp *op, const TShape &data_shape, const TContainer &pads_begin, const TContainer &pads_end, const Strides &dilations) Shape inference helper used for pooling operators such Max Pool, Avg Pool.
-
template<class TShape, class TOp, class TRShape = result_shape_t<TShape>, typename std::enable_if<std::is_same<TOp, v8::AdaptiveAvgPool>::value || std::is_same<TOp, v8::AdaptiveMaxPool>::value>::type* = nullptr>
TRShape out_shape_infer(const TOp *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta) Shape inference helper used for adaptive pooling operators.
Variables
-
constexpr size_t spatial_dim_offset = 2
-
namespace validate
Functions
-
template<class TOp, class TContainer>
void padding(const TOp *op, const TContainer &pads_begin, const TContainer &pads_end)
-
template<class TOp, class TContainer>
-
template<>
-
namespace prior_box
Functions
-
template<class TDim, class TOp, typename std::enable_if<std::is_same<v0::PriorBox, TOp>::value || std::is_same<v8::PriorBox, TOp>::value>::type* = nullptr>
TDim number_of_priors(const TOp *const op)
-
template<class TDim>
TDim number_of_priors(const v0::PriorBoxClustered *const op)
-
template<class TOp, class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const TOp *const op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
Variables
-
constexpr std::array<char const*, 2> input_names = {"output size", "image"}
-
namespace validate
Functions
-
inline std::vector<PartialShape> inputs_et(const Node *const op)
-
inline std::vector<PartialShape> inputs_et(const Node *const op)
-
template<class TDim, class TOp, typename std::enable_if<std::is_same<v0::PriorBox, TOp>::value || std::is_same<v8::PriorBox, TOp>::value>::type* = nullptr>
-
namespace proposal
-
namespace psroi_pooling
-
namespace validate
Functions
-
template<class TROIPooling, class TShape>
void feat_input_shape(const TROIPooling *op, const TShape feat_shape)
-
template<class TROIPooling>
void output_group_attr(const TROIPooling *op)
-
template<class TROIPooling>
void bins_attr(const TROIPooling *op)
-
template<class TROIPooling>
void mode_attr(const TROIPooling *op)
-
template<class TROIPooling, class TShape>
-
namespace validate
-
namespace reshape
Functions
-
template<class TDim, typename std::enable_if<std::is_same<typename std::decay<TDim>::type, Dimension>::value>::type* = nullptr>
TDim resolve_minus_one_dim(const Product<TDim> &product)
-
template<class TShape>
std::pair<TShape, int64_t> get_pattern_and_minus_one_idx(const Node *const op, const std::vector<std::pair<int64_t, int64_t>> &bounds) Get the pattern and minus one idx from input bounds.
- Parameters
op – Pointer to reshape node.
bounds – Vector of reshape pattern bounds.
- Returns
Pair which got bounds converted to shape and
minus_oneindex in pattern (-1 if not found).
-
template<class TShape, typename std::enable_if<std::is_same<TShape, PartialShape>::value>::type* = nullptr>
void set_pattern_labels(const Node *const op, TShape &shape) Set the pattern labels on pattern shape if this input is labeled.
Shapes other than PartialShape have no labels.
- Parameters
op – Pointer to reshape node.
shape – Pointer to shape for labels set.
-
template<class T, class U = void>
struct Product - #include <reshape_shape_inference.hpp>
- template<class T> type >
- #include <reshape_shape_inference.hpp>
Helper to resolve the input and output product for ov::Dimension (dynamic) dimensions.
- template<class T> type >
- #include <reshape_shape_inference.hpp>
Helper to resolve the input and output product for static dimensions.
-
template<class TDim, typename std::enable_if<std::is_same<typename std::decay<TDim>::type, Dimension>::value>::type* = nullptr>
-
namespace rnn
Functions
-
template<class TShape>
void validate_inputs_rank(const op::util::RNNCellBase *op, const std::vector<TShape> &input_shapes, const std::vector<Rank> &expected_ranks)
-
template<class TShape>
-
namespace roi_align
Functions
-
namespace validate
Functions
-
inline element::Type data_and_roi_et(const Node *const op)
Validates ROIs align input data and ROIs element type.
- Parameters
op – Pointer to ROIs align node.
- Returns
Valid ROIs align output element type.
-
inline void batch_indicies_et(const Node *const op)
Check ROIs align batch indicies input element type.
- Parameters
op – Pointer to ROIs align node.
-
inline element::Type data_and_roi_et(const Node *const op)
-
namespace validate
-
namespace roi_pooling
-
namespace validate
Functions
-
template<class TROIPooling, class TShape>
void feat_intput_shape(const TROIPooling *op, const TShape &feat_shape)
-
template<class TROIPooling, class TShape>
void rois_input_shape(const TROIPooling *op, const TShape rois_shape)
-
template<class TROIPooling>
void output_roi_attr(const TROIPooling *op)
-
template<class TROIPooling>
void scale_attr(const TROIPooling *op)
-
template<class TROIPooling>
void method_attr(const TROIPooling *op)
-
template<class TROIPooling, class TShape>
-
namespace validate
-
namespace shape_of
-
namespace ShapeInferRange
Functions
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> range_shape_infer(const Node *op, const std::vector<T> &input_shapes, bool output_is_integral, bool step_allows_zero, const ITensorAccessor &tensor_accessor)
-
template<class T, class TRShape = result_shape_t<T>>
-
namespace slice
Typedefs
-
using Bounds = std::pair<int64_t, int64_t>
Alias to dimension bounds for slice.
Functions
-
inline int64_t get_sliced_value(const int64_t dim, const int64_t start, const int64_t stop, const int64_t step)
Get sliced value in step for given dimension value and start, stop, step.
Note
This function cannot be use for step 0 (division by 0)
- Parameters
dim – Dimension value.
start – Start of slice.
stop – Stop of slice.
step – Step of slice.
- Returns
-1 for infinite number otherwise [0..int64_max] for finit step.
-
constexpr bool is_bounds_zero_crossing(const Bounds b)
Check if bounds can cross 0 value (rising edge).
- Parameters
b – Input interval bounds for check.
- Returns
True if lower bound is negative and upper is not negative, otherwise false.
-
template<class TDim>
constexpr bool is_lb_within_dim(const int64_t lb, const TDim &dim) Check if lower bound is within dimension.
Check valid only if bounds can cross zero value (lb is negative).
- Parameters
lb – Lower bound for check.
dim – Dimension used to check lower bound.
- Returns
True if lower bound is within dimension length, otherwise false.
-
template<class TDim>
constexpr bool is_ub_within_dim(const int64_t ub, const TDim &dim) Check if upper bound is within dimension.
Check valid only if bounds can cross zero value (up is not negative).
- Parameters
ub – Upper bound for check.
dim – Dimension used to check upper bound.
- Returns
True if upper bound is within dimension length, otherwise false.
Variables
-
constexpr std::array<char const*, 4> shape_names = {"start", "stop", "step", "axes"}
-
struct AxesMap
- #include <slice_shape_inference.hpp>
Public Members
-
bool is_valid = {}
Flag indicates current axes map has valid data (unique).
-
std::map<size_t, size_t> m = {}
Map axis value to index of start, stop order.
-
bool is_valid = {}
-
using Bounds = std::pair<int64_t, int64_t>
-
namespace util
Typedefs
-
using ActivationFunctionType = std::shared_ptr<Node> (*)(const std::shared_ptr<Node>&, float, float)
-
using InputDescriptionVector = std::vector<util::SubGraphOp::InputDescription::Ptr>
-
using OutputDescriptionVector = std::vector<util::SubGraphOp::OutputDescription::Ptr>
-
using VariableVector = std::vector<Variable::Ptr>
-
using VariableMap = std::unordered_map<Variable::Ptr, VariableValue::Ptr>
Enums
-
enum class LSTMWeightsFormat
Values:
-
enumerator FICO
-
enumerator ICOF
-
enumerator IFCO
-
enumerator IFOC
-
enumerator IOFC
-
enumerator FICO
-
enum class LSTMPeepholesFormat
Values:
-
enumerator FIO
-
enumerator IOF
-
enumerator IFO
-
enumerator FIO
Functions
-
template<class T>
bool normalize_single_value(std::vector<T> vec, float &value, bool check_value_range = true)
-
float cast_eps_to_float(double eps_d)
Convert epsilon value from double to float type.
If the value is too large, the epsilon is converted to std::numeric_limits<float>::min() or std::numeric_limits<float>::min(), otherwise static cast to float is called. The adjustment is made for positive values only, for negative it works as static cast.
- Parameters
eps – Original value of the epsilon (double).
- Returns
Epsilon value as float.
-
bool check_for_broadcast(const PartialShape &ref_shape, const PartialShape &other_shape)
-
bool shapes_equal_except_dynamic_expected_batch(const PartialShape &expected, const PartialShape &actual)
-
void visit_shape_path(ov::Node *node, std::unordered_set<ov::Node*> &visited, std::function<void(ov::Node*)> func)
Traverses a shapeOf subgraph starting from the node and not including the ShapeOf nodes, and calls “func” for each ov::Node.
- Parameters
node – The node from which constant path is started.
visited – Set of nodes which were visited.
func – The function which is called for each visited node.
-
void visit_constant_path(ov::Node *node, std::unordered_set<ov::Node*> &visited, std::function<void(ov::Node*)> func)
Traverses a constant path starting from “node”, and calls “func” for each ov::Node. If the function was called for non-constant subgraph, exception is thrown.
- Parameters
node – The node from which constant path is started.
visited – Set of nodes which were visited.
func – The function which is called for each visited node.
-
template<typename T>
ov::pass::pattern::op::ValuePredicate constant_predicate(std::function<bool(const std::vector<T>&)> predicate)
-
ActivationFunction get_activation_func_by_name(const std::string &func_name)
Gets the activation function by name.
- Parameters
func_name – [in] The function name
- Throws
UnknownActivationFunction – When provided func_name is unknown.
- Returns
The activation function object.
-
AxisSet get_normalized_axes_from_tensor(const Node *const node, const Tensor &tensor, const Rank &rank)
Get the normalized axes as ov::AxisSet from raw tensor data.
- Parameters
node – A node pointer used for detailed description if normalization fails.
tensor – Tensor with axes for normalization.
rank – Rank value to normalize axes.
- Returns
Normalized axes as set.
-
std::tuple<element::Type, PartialShape> validate_and_infer_elementwise_args(Node *node)
- OPENVINO_API bool is_unary_elementwise_arithmetic (const Node *node)
- OPENVINO_API bool is_binary_elementwise_arithmetic (const Node *node)
- OPENVINO_API bool is_binary_elementwise_comparison (const Node *node)
- OPENVINO_API bool is_binary_elementwise_logical (const Node *node)
- OPENVINO_API bool supports_auto_broadcast (const Node *node)
- OPENVINO_API bool is_op (const Node *node)
- OPENVINO_API bool is_parameter (const Node *node)
- OPENVINO_API bool is_output (const Node *node)
- OPENVINO_API bool is_sink (const Node *node)
- OPENVINO_API bool is_constant (const Node *node)
- OPENVINO_API bool is_commutative (const Node *node)
- OPENVINO_API bool is_unary_elementwise_arithmetic (const std::shared_ptr< Node > &node)
- OPENVINO_API bool is_binary_elementwise_arithmetic (const std::shared_ptr< Node > &node)
- OPENVINO_API bool is_binary_elementwise_comparison (const std::shared_ptr< Node > &node)
- OPENVINO_API bool is_binary_elementwise_logical (const std::shared_ptr< Node > &node)
- OPENVINO_API bool supports_auto_broadcast (const std::shared_ptr< Node > &node)
- OPENVINO_API bool is_op (const std::shared_ptr< Node > &node)
- OPENVINO_API bool is_parameter (const std::shared_ptr< Node > &node)
- OPENVINO_API bool is_output (const std::shared_ptr< Node > &node)
- OPENVINO_API bool is_sink (const std::shared_ptr< Node > &node)
- OPENVINO_API bool is_constant (const std::shared_ptr< Node > &node)
- OPENVINO_API bool is_commutative (const std::shared_ptr< Node > &node)
- OPENVINO_API void validate_seq_input_rank_dimension (const std::vector< ov::PartialShape > &input)
Validates static rank and dimension for provided input parameters. Additionally input_size dimension is checked for X and W inputs. Applies to LSTM, GRU and RNN Sequences.
- Parameters
input – [in] Vector with RNNSequence-like op inputs in following order: X, initial_hidden_state, sequence_lengths, W, R and B.
- std::shared_ptr< Node > OPENVINO_API convert_lstm_node_format (const Output< Node > &node, LSTMWeightsFormat from_format, LSTMWeightsFormat to_format=LSTMWeightsFormat::FICO, int64_t axis=0)
Change data format of provided node.
- Parameters
node – [in] The input node to be permuted.
from_format – [in] Original node weights format.
to_format – [in] Weights format to convert to.
- Returns
Node representing reshaped tensor according to
to_formatweights format.
- std::shared_ptr< Node > OPENVINO_API convert_lstm_peepholes_format (const Output< Node > &node, LSTMPeepholesFormat from_format, LSTMPeepholesFormat to_format=LSTMPeepholesFormat::FIO, int64_t axis=0)
-
template<typename T, class TRShape = result_shape_t<T>>
void validate_target_shape_none(const ov::Node *op, const T &arg_shape, const AxisVector &axes_mapping_val, const TRShape &target_input_shape)
-
template<typename T, class TRShape = result_shape_t<T>>
void validate_target_shape_numpy(const ov::Node *op, const T &arg_shape, const TRShape &target_input_shape)
-
template<typename T, class TRShape = result_shape_t<T>>
void set_result_shape_pdpd(const ov::Node *op, const T &arg0_shape, const TRShape &target_input_shape, TRShape &result_shape, const ov::op::BroadcastModeSpec &broadcast_spec)
-
template<typename T, class TRShape = result_shape_t<T>>
void set_result_shape_bidirectional(const ov::Node *op, const T &arg_shape, TRShape &target_input_shape, TRShape &result_shape)
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> broadcast_base_shape_infer(const ov::op::util::BroadcastBase *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
-
template<class TShape>
size_t num_spatial_from_shapes(const TShape &data_shape, const TShape &filter_shape, const size_t filter_non_spatial_dims_count) Get num of spatial form convolution operator.
Tries get value from operator member if is not deduced (has -1 value) then tries evaluate it from input shapes.
- Template Parameters
TConv – Convolution type (this function must be a friend of TConv to access private member).
TShape – Shape type.
- Parameters
- Returns
Value of spatial dimension number or infinite bound (-1) if cannot evaluate.
-
inline bool is_attr_validation_required(const ConvolutionBase *op)
Checks if validation attributes is required.
- Parameters
op – Pointer to convolution base operator.
- Returns
True if internal number of spatial dimension not defined otherwise false.
-
inline size_t get_num_spatial(const ConvolutionBase *op)
Get the num spatil object.
- Parameters
op –
- Returns
size_t
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DeformableConvolutionBase *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)
-
template<typename T, typename V = typename std::iterator_traits<typename T::iterator>::value_type::value_type>
void compute_num_classes(const DetectionOutputBase *op, const DetectionOutputBase::AttributesBase &attrs, const std::vector<T> &input_shapes, V &num_classes, V &num_prior_boxes)
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer_base(const DetectionOutputBase *op, const DetectionOutputBase::AttributesBase &attrs, const std::vector<T> &input_shapes, int64_t attribute_num_classes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ov::op::util::EmbeddingBagOffsetsBase *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ov::op::util::EmbeddingBagPackedBase *op, const std::vector<TShape> &input_shapes)
Variables
-
constexpr size_t num_spatial_undefined = std::numeric_limits<size_t>::max()
-
constexpr size_t spatial_dim_offset = 2
-
class ActivationFunction
- #include <activation_functions.hpp>
Class representing activation function used in RNN cells.
Public Functions
Calls stored activation function with provided node argument.
-
class ArithmeticReduction : public ov::op::util::ReductionBase
- #include <arithmetic_reduction.hpp>
Abstract base class for arithmetic reduction operations, i.e., operations where chosen axes of the input tensors are eliminated (reduced out) by repeated application of a particular binary arithmetic operation.
Subclassed by ov::op::util::ArithmeticReductionKeepDims
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class ArithmeticReductionKeepDims : public ov::op::util::ArithmeticReduction
- #include <arithmetic_reductions_keep_dims.hpp>
Subclassed by ov::op::v1::ReduceMax, ov::op::v1::ReduceMean, ov::op::v1::ReduceMin, ov::op::v1::ReduceProd, ov::op::v1::ReduceSum, ov::op::v4::ReduceL1, ov::op::v4::ReduceL2
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
inline virtual bool get_keep_dims() const override
- Returns
If set to 1 it holds axes that are used for reduction. For each such axis, output dimension is equal to 1.
-
virtual void validate_and_infer_types() override
-
class AssignBase : public ov::op::Sink, public ov::op::util::VariableExtension
- #include <assign_base.hpp>
Subclassed by ov::op::v3::Assign, ov::op::v6::Assign
Public Functions
-
inline explicit AssignBase(const OutputVector &arguments)
Constructs an AssignBase operation.
-
inline explicit AssignBase(const OutputVector &arguments)
-
class BinaryElementwiseArithmetic : public ov::op::Op
- #include <binary_elementwise_arithmetic.hpp>
Abstract base class for elementwise binary arithmetic operations, i.e., operations where the same scalar binary arithmetic operation is applied to each corresponding pair of elements in the two input tensors. Implicit broadcast of input tensors is supported through one of the AutoBroadcast modes.
For example, if the underlying arithmetic operation (determined by the subclass) is \(\mathit{op}(x,y)\), the input tensors \([[x_0,y_0],[z_0,w_0]]\) and \([[x_1,y_1],[z_1,w_1]]\) will be mapped to \([[\mathit{op}(x_0,x_1),\mathit{op}(y_0,y_1)],[\mathit{op}(z_0,z_1),\mathit{op}(w_0,w_1)]]\).
Inputs
Type
Description
arg0\(N[d_1,\dots,d_n]~(n \geq 0)\)
A tensor of any shape. The element type \(N\) may be any numeric type.
arg1\(N[d_1,\dots,d_n]~(n \geq 0)\)
A tensor of the same element type as
arg0.autobAuto broadcast specification.
Type
Description
\(N[d_1,\dots,d_n]\)
The tensor \(T\), where \(T[i_1,\dots,i_n] = \mathit{op}(\texttt{arg0}[i_1,\dots,i_n],\texttt{arg1}[i_1,\dots,i_n])\). This will always have the same shape and element type as the input tensors (after auto broadcasting).
Subclassed by ov::op::v0::SquaredDifference, ov::op::v1::Add, ov::op::v1::Divide, ov::op::v1::FloorMod, ov::op::v1::Maximum, ov::op::v1::Minimum, ov::op::v1::Mod, ov::op::v1::Multiply, ov::op::v1::Power, ov::op::v1::Subtract
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
inline virtual const AutoBroadcastSpec &get_autob() const override
- Returns
the autobroadcasr spec
-
virtual void validate_and_infer_types() override
-
class BinaryElementwiseBitwise : public ov::op::Op
- #include <binary_elementwise_bitwise.hpp>
Subclassed by ov::op::v13::BitwiseAnd, ov::op::v13::BitwiseOr, ov::op::v13::BitwiseXor
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual const AutoBroadcastSpec &get_autob() const override
- Returns
the autobroadcasr spec
-
virtual void validate_and_infer_types() override
-
class BinaryElementwiseComparison : public ov::op::Op
- #include <binary_elementwise_comparison.hpp>
Abstract base class for elementwise binary comparison operations, i.e., operations where the same scalar binary comparison operation is applied to each corresponding pair of elements in two input tensors. Implicit broadcast of input tensors is supported through one of the AutoBroadcast modes.
For example, if the underlying comparison operation (determined by the subclass) is \(\mathit{op}(x,y)\), the input tensors \([[x_0,y_0],[z_0,w_0]]\) and \([[x_1,y_1],[z_1,w_1]]\) will be mapped to \([[\mathit{op}(x_0,x_1),\mathit{op}(y_0,y_1)],[\mathit{op}(z_0,z_1),\mathit{op}(w_0,w_1)]]\).
Inputs
Type
Description
arg0\(E[d_1,\dots,d_n]~(n \geq 0)\)
A tensor of any shape and element type.
arg1\(E[d_1,\dots,d_n]~(n \geq 0)\)
A tensor of the same shape and element type as
arg0.autobAuto broadcast specification.
Type
Description
\(\texttt{bool}[d_1,\dots,d_n]\)
The tensor \(T\), where \(T[i_1,\dots,i_n] = \mathit{op}(\texttt{arg0}[i_1,\dots,i_n],\texttt{arg1}[i_1,\dots,i_n])\). This will always have the same shape as the input tensors, and the element type
bool.Subclassed by ov::op::v1::Equal, ov::op::v1::Greater, ov::op::v1::GreaterEqual, ov::op::v1::Less, ov::op::v1::LessEqual, ov::op::v1::NotEqual
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
inline virtual const AutoBroadcastSpec &get_autob() const override
- Returns
the autobroadcasr spec
-
virtual void validate_and_infer_types() override
-
class BinaryElementwiseLogical : public ov::op::Op
- #include <binary_elementwise_logical.hpp>
Abstract base class for elementwise binary logical operations, i.e., operations where the same scalar binary logical operation is applied to each corresponding pair of elements in two boolean input tensors. Implicit broadcast of input tensors is supported through one of the AutoBroadcast modes.
For example, if the underlying operation (determined by the subclass) is \(\mathit{op}(x,y)\), the input tensors \([[x_0,y_0],[z_0,w_0]]\) and \([[x_1,y_1],[z_1,w_1]]\) will be mapped to \([[\mathit{op}(x_0,x_1),\mathit{op}(y_0,y_1)],[\mathit{op}(z_0,z_1),\mathit{op}(w_0,w_1)]]\).
Inputs
Type
Description
arg0\(\texttt{bool}[d_1,\dots,d_n]~(n \geq 0)\)
A tensor of any shape, with element type
bool.arg1\(\texttt{bool}[d_1,\dots,d_n]~(n \geq 0)\)
A tensor of the same shape and element type as
arg0.autobAuto broadcast specification.
Type
Description
\(\texttt{bool}[d_1,\dots,d_n]\)
The tensor \(T\), where \(T[i_1,\dots,i_n] = \mathit{op}(\texttt{arg0}[i_1,\dots,i_n],\texttt{arg1}[i_1,\dots,i_n])\). This will always have the same shape as the input tensors, and the element type
bool.Subclassed by ov::op::v0::Xor, ov::op::v1::LogicalAnd, ov::op::v1::LogicalOr, ov::op::v1::LogicalXor
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
inline virtual const AutoBroadcastSpec &get_autob() const override
- Returns
the autobroadcasr spec
-
virtual void validate_and_infer_types() override
-
class BroadcastBase : public ov::op::Op
- #include <broadcast_base.hpp>
Subclassed by ov::op::v1::Broadcast, ov::op::v3::Broadcast
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual std::pair<bool, AxisSet> get_broadcast_axes() const
- Returns
true and the AxisSet if broadcast axes can be fully determined.
-
virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override
Evaluates the op on input_values putting results in output_values.
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
- Returns
true if successful
-
virtual void validate_and_infer_types() override
-
struct ClipNegative
- #include <reverse_shape_inference.hpp>
Clip if value type T is less than 0, otherwise cast to AxisSet::value_type.
-
class ConvertColorI420Base : public ov::op::Op
- #include <convert_color_i420_base.hpp>
Base class for color conversion operation from I420 to RGB/BGR format. Input:
Operation expects input shape in NHWC layout.
Input NV12 image can be represented in a two ways: a) Single plane: NV12 height dimension is 1.5x bigger than image height. ‘C’ dimension shall be 1 b) Three separate planes: Y, U and V. In this case b1) Y plane has height same as image height. ‘C’ dimension equals to 1 b2) U plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 1. b3) V plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 1.
Supported element types: u8 or any supported floating-point type. Output:
Output node will have NHWC layout and shape HxW same as image spatial dimensions.
Number of output channels ‘C’ will be 3
Conversion of each pixel from I420 (YUV) to RGB space is represented by following formulas: R = 1.164 * (Y - 16) + 1.596 * (V - 128) G = 1.164 * (Y - 16) - 0.813 * (V - 128) - 0.391 * (U - 128) B = 1.164 * (Y - 16) + 2.018 * (U - 128) Then R, G, B values are clipped to range (0, 255)
Subclassed by ov::op::v8::I420toBGR, ov::op::v8::I420toRGB
Public Types
-
enum class ColorConversion : int
Exact conversion format details Currently supports conversion from I420 to RGB or BGR.
Values:
-
enumerator I420_TO_RGB
-
enumerator I420_TO_BGR
-
enumerator I420_TO_RGB
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
class ConvertColorNV12Base : public ov::op::Op
- #include <convert_color_nv12_base.hpp>
Base class for color conversion operation from NV12 to RGB/BGR format. Input:
Operation expects input shape in NHWC layout.
Input NV12 image can be represented in a two ways: a) Single plane: NV12 height dimension is 1.5x bigger than image height. ‘C’ dimension shall be 1 b) Two separate planes: Y and UV. In this case b1) Y plane has height same as image height. ‘C’ dimension equals to 1 b2) UV plane has dimensions: ‘H’ = image_h / 2; ‘W’ = image_w / 2; ‘C’ = 2.
Supported element types: u8 or any supported floating-point type. Output:
Output node will have NHWC layout and shape HxW same as image spatial dimensions.
Number of output channels ‘C’ will be 3
Conversion of each pixel from NV12 (YUV) to RGB space is represented by following formulas: R = 1.164 * (Y - 16) + 1.596 * (V - 128) G = 1.164 * (Y - 16) - 0.813 * (V - 128) - 0.391 * (U - 128) B = 1.164 * (Y - 16) + 2.018 * (U - 128) Then R, G, B values are clipped to range (0, 255)
Subclassed by ov::op::v8::NV12toBGR, ov::op::v8::NV12toRGB
Public Types
-
enum class ColorConversion : int
Exact conversion format details Currently supports conversion from NV12 to RGB or BGR, in future can be extended with NV21_to_RGBA/BGRA, etc.
Values:
-
enumerator NV12_TO_RGB
-
enumerator NV12_TO_BGR
-
enumerator NV12_TO_RGB
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
class ConvolutionBackPropBase : public ov::op::util::ConvolutionBase
- #include <convolution_backprop_base.hpp>
Base class for operations like back propagation convolution.
Subclassed by ov::op::v1::ConvolutionBackpropData, ov::op::v1::GroupConvolutionBackpropData
Public Functions
-
ConvolutionBackPropBase() = default
Constructs a conversion operation.
-
inline ConvolutionBackPropBase(const OutputVector &arguments, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT, const CoordinateDiff &output_padding = {})
Constructs a conversion operation.
- Parameters
strides – Convolution strides.
pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.
pads_end – Amount of padding to be added to the end along each axis.
dilations – The distance in width and height between the weights in the filters tensor.
auto_pad – Specifies how the automatic calculation of padding should be done.
output_padding – The output padding adds additional amount of paddings per each spatial axis in the output tensor. clang-format on
-
ConvolutionBackPropBase() = default
-
class ConvolutionBase : public ov::op::Op
- #include <convolution_base.hpp>
Base class for operations like convolutions.
Subclassed by ov::op::util::ConvolutionBackPropBase, ov::op::util::ConvolutionFwdPropBase, ov::op::util::DeformableConvolutionBase
Public Functions
-
ConvolutionBase() = default
Constructs a conversion operation.
-
inline ConvolutionBase(const OutputVector &arguments, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT)
Constructs a conversion operation.
- Parameters
strides – Convolution strides.
pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.
pads_end – Amount of padding to be added to the end along each axis.
dilations – The distance in width and height between the weights in the filters tensor.
auto_pad – Specifies how the automatic calculation of padding should be done.
-
ConvolutionBase() = default
-
class ConvolutionFwdPropBase : public ov::op::util::ConvolutionBase
- #include <convolution_base.hpp>
Base class for operations like back propagation convolution.
Subclassed by ov::op::v1::BinaryConvolution, ov::op::v1::Convolution, ov::op::v1::GroupConvolution
Public Functions
-
ConvolutionFwdPropBase() = default
Constructs a conversion operation.
-
inline ConvolutionFwdPropBase(const OutputVector &arguments, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT)
Constructs a conversion operation.
- Parameters
strides – Convolution strides.
pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.
pads_end – Amount of padding to be added to the end along each axis.
dilations – The distance in width and height between the weights in the filters tensor.
auto_pad – Specifies how the automatic calculation of padding should be done.
-
ConvolutionFwdPropBase() = default
-
class DeformableConvolutionBase : public ov::op::util::ConvolutionBase
- #include <deformable_convolution_base.hpp>
Base class for operations DeformableConvolution v1 and DeformableConvolution v8.
Subclassed by ov::op::v1::DeformableConvolution, ov::op::v8::DeformableConvolution
Public Functions
-
DeformableConvolutionBase() = default
Constructs a conversion operation.
-
DeformableConvolutionBase(const OutputVector &arguments, const Strides &strides, const CoordinateDiff &pads_begin, const CoordinateDiff &pads_end, const Strides &dilations, const PadType &auto_pad = PadType::EXPLICIT, int64_t group = 1, int64_t deformable_group = 1)
Constructs a conversion operation.
- Parameters
strides – Convolution strides.
pads_begin – Amount of padding to be added to the beginning along each axis. For example in case of a 2D input the value of (1, 2) means that 1 element will be added to the top and 2 elements to the left.
pads_end – Amount of padding to be added to the end along each axis.
dilations – The distance in width and height between the weights in the filters tensor.
auto_pad – Specifies how the automatic calculation of padding should be done.
group – The number of groups which both output and input should be split into.
deformable_group – The number of groups which deformable values and output should be split into along the channel axis.
-
DeformableConvolutionBase() = default
-
class DetectionOutputBase : public ov::op::Op
- #include <detection_output_base.hpp>
DetectionOutputBase basic class for DetectionOutput v0 and v8.
Subclassed by ov::op::v0::DetectionOutput, ov::op::v8::DetectionOutput
-
struct AttributesBase
- #include <detection_output_base.hpp>
Subclassed by ov::op::v0::DetectionOutput::Attributes
-
struct AttributesBase
-
class EmbeddingBagOffsetsBase : public ov::op::Op
- #include <embeddingbag_offsets_base.hpp>
Returns embeddings for given indices.
Subclassed by ov::op::v3::EmbeddingBagOffsetsSum
Public Functions
-
EmbeddingBagOffsetsBase() = default
Constructs a EmbeddingBagOffsetsBase operation.
-
EmbeddingBagOffsetsBase(const Output<Node> &emb_table, const Output<Node> &indices, const Output<Node> &offsets, const Output<Node> &default_index, const Output<Node> &per_sample_weights)
Constructs a EmbeddingBagOffsetsBase operation.
EmbeddingBagOffsetsBase constructs an output tensor by replacing every index in a given input tensor with a row (from the weights matrix) at that index
- Parameters
emb_table – tensor containing the embedding lookup table of the module of shape [num_emb, emb_dim1, emb_dim2, …] and of type T
indices – tensor of shape [num_indices] and of type T_IND. Required
offsets – tensor of shape [batch] and of type T_IND containing the starting index positions of each “bag” in indices. Required.
per_sample_weigths – tensor of the same shape as indices and of type T. Each value in this tensor are multiplied with each value pooled from embedding table for each index. Optional.
default_index – scalar of type T_IND containing default index in embedding table to fill empty “bags”. If not provided empty “bags” are filled with zeros. Optional.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
EmbeddingBagOffsetsBase() = default
-
class EmbeddingBagPackedBase : public ov::op::Op
- #include <embeddingbag_packed_base.hpp>
Returns embeddings for given indices.
Subclassed by ov::op::v3::EmbeddingBagPackedSum
Public Functions
-
EmbeddingBagPackedBase() = default
Constructs a EmbeddingBagPackedBase operation.
-
EmbeddingBagPackedBase(const Output<Node> &emb_table, const Output<Node> &indices, const Output<Node> &per_sample_weights)
Constructs a EmbeddingBagPackedBase operation.
EmbeddingBagPackedBase constructs an output tensor by replacing every index in a given input tensor with a row (from the weights matrix) at that index
- Parameters
emb_table – Tensor containing the embedding lookup table of the module of shape [num_emb, emb_dim1, emb_dim2, …] and of type T
indices – Tensor of shape
[batch, indices_per_bag]and of type T_IND. Required.per_sample_weigths – tensor of the same shape as indices and of type T. Each value in this tensor are multiplied with each value pooled from embedding table for each index. Optional.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
EmbeddingBagPackedBase() = default
-
class FFTBase : public ov::op::Op
- #include <fft_base.hpp>
Base class for operations DFT and DFT.
Subclassed by ov::op::v7::DFT, ov::op::v7::IDFT, ov::op::v9::IRDFT, ov::op::v9::RDFT
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class FrameworkNode : public ov::op::util::MultiSubGraphOp
- #include <framework_node.hpp>
Subclassed by ov::frontend::ComplexTypeMark
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class FrameworkNodeAttrs
- #include <framework_node.hpp>
-
class GatherBase : public ov::op::Op
- #include <gather_base.hpp>
GatherBase basic class for Gather v1 and v7.
Subclassed by ov::op::v1::Gather, ov::op::v7::Gather, ov::op::v8::Gather
Public Functions
-
GatherBase(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &axis, const int64_t batch_dims = 0)
- Parameters
data – The tensor from which slices are gathered
indices – Tensor with indexes to gather
axis – The tensor is a dimension index to gather data from
batch_dims – The number of batch dimension in data and indices tensors
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
GatherBase(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &axis, const int64_t batch_dims = 0)
-
class GatherNDBase : public ov::op::Op
- #include <gather_nd_base.hpp>
GatherNDBase basic class for GatherND v5 and v8.
Subclassed by ov::op::v5::GatherND, ov::op::v8::GatherND
-
template<class T>
struct GetK - #include <topk_shape_inference.hpp>
-
template<class T>
struct GetNotNegative - #include <one_hot_shape_inference.hpp>
-
class IndexReduction : public ov::op::Op
- #include <index_reduction.hpp>
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class InterpolateBase : public ov::op::Op
- #include <interpolate_base.hpp>
Subclassed by ov::op::v11::Interpolate, ov::op::v4::Interpolate
Public Types
-
enum class ShapeCalcMode
PartialShape calculation mode.
SIZES - output shape for interpolated axes is calculated using input
sizesSCALES - output shape for interpolated axes is calculated using inputscalesValues:
-
enumerator SIZES
-
enumerator SCALES
-
enumerator SIZES
-
enum class InterpolateMode
Interpolation mode.
NEAREST - nearest interpolation LINEAR - linear interpolation as in TensorFlow LINEAR_ONNX - linear interpolation as in ONNX CUBIC - cubic interpolation BILINEAR_PILLOW - bilinear interpolation as in Pillow BICUBIC_PILLOW - bicubic interpolation as in Pillow
Values:
-
enumerator NEAREST
-
enumerator LINEAR
-
enumerator LINEAR_ONNX
-
enumerator CUBIC
-
enumerator BILINEAR_PILLOW
-
enumerator BICUBIC_PILLOW
-
enumerator NEAREST
-
enum class CoordinateTransformMode
Mode of the calculation of the source coordinate from resized one.
These modes are modes from ONNXRuntime.
Values:
-
enumerator HALF_PIXEL
-
enumerator PYTORCH_HALF_PIXEL
-
enumerator ASYMMETRIC
-
enumerator TF_HALF_PIXEL_FOR_NN
-
enumerator ALIGN_CORNERS
-
enumerator HALF_PIXEL
-
enum class NearestMode
Rounding modes for the NEAREST interpolation.
Values:
-
enumerator ROUND_PREFER_FLOOR
-
enumerator ROUND_PREFER_CEIL
-
enumerator FLOOR
-
enumerator CEIL
-
enumerator SIMPLE
-
enumerator ROUND_PREFER_FLOOR
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
struct InterpolateAttrs
- #include <interpolate_base.hpp>
-
enum class ShapeCalcMode
-
class LogicalReduction : public ov::op::util::ReductionBase
- #include <logical_reduction.hpp>
Abstract base class for logical reduction operations, i.e., operations where chosen axes of the input tensors are eliminated (reduced out) by repeated application of a particular binary logical operation.
Subclassed by ov::op::util::LogicalReductionKeepDims
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class LogicalReductionKeepDims : public ov::op::util::LogicalReduction
- #include <logical_reduction_keep_dims.hpp>
Subclassed by ov::op::v1::ReduceLogicalAnd, ov::op::v1::ReduceLogicalOr
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
inline virtual bool get_keep_dims() const override
- Returns
If set to 1 it holds axes that are used for reduction. For each such axis, output dimension is equal to 1.
-
virtual void validate_and_infer_types() override
-
class MaxPoolBase : public ov::op::Op
- #include <max_pool_base.hpp>
Subclassed by ov::op::v1::MaxPool, ov::op::v8::MaxPool
Public Functions
-
MaxPoolBase(const Output<Node> &arg, const Strides &strides, const Shape &pads_begin, const Shape &pads_end, const Shape &kernel, const op::RoundingType rounding_mode = op::RoundingType::FLOOR, const PadType auto_pad = op::PadType::EXPLICIT)
- Parameters
arg – The node producing the input data batch tensor.
strides – The strides.
pads_begin – The beginning of padding shape.
pads_end – The end of padding shape.
kernel – The kernel shape.
rounding_mode – Whether to use ceiling or floor rounding type while computing output shape.
auto_pad – The pad type for automatically computing padding sizes.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
inline const Shape &get_kernel() const
- Returns
The kernel shape.
-
inline const Strides &get_strides() const
- Returns
The strides.
-
inline const Shape &get_pads_begin() const
- Returns
The beginning of padding shape.
-
inline const Shape &get_pads_end() const
- Returns
The end of padding shape.
-
inline PadType get_auto_pad() const
- Returns
The pad type for pooling.
-
inline op::RoundingType get_rounding_type() const
- Returns
The ceiling mode being used for output shape computations
-
MaxPoolBase(const Output<Node> &arg, const Strides &strides, const Shape &pads_begin, const Shape &pads_end, const Shape &kernel, const op::RoundingType rounding_mode = op::RoundingType::FLOOR, const PadType auto_pad = op::PadType::EXPLICIT)
-
class MulticlassNmsBase : public ov::op::Op
- #include <multiclass_nms_base.hpp>
Base class for operations MulticlassNMS v8 and MulticlassNMS v9.
Subclassed by ov::op::v8::MulticlassNms, ov::op::v9::MulticlassNms
Public Functions
-
MulticlassNmsBase() = default
Constructs a conversion operation.
-
MulticlassNmsBase(const OutputVector &arguments, const Attributes &attrs)
Constructs a MulticlassNmsBase operation.
- Parameters
arguments – Node list producing the box coordinates, scores, etc.
attrs – Attributes of the operation
-
inline const Attributes &get_attrs() const
Returns attributes of the operation MulticlassNmsBase.
-
struct Attributes
- #include <multiclass_nms_base.hpp>
Structure that specifies attributes of the operation.
-
MulticlassNmsBase() = default
-
class MultiSubGraphOp : public ov::op::Op
- #include <multi_subgraph_base.hpp>
Abstract base class for sub-graph based ops, i.e ops that have some sub-graphs.
Subclassed by ov::op::util::FrameworkNode, ov::op::util::SubGraphOp, ov::op::v8::If
Public Functions
-
inline virtual const std::shared_ptr<Model> &get_function(size_t index) const
Gets internal sub-graph by index in MultiSubGraphOp.
- Parameters
index – sub-graph’s index in op
- Returns
pointer to Model with sub-graph
-
inline virtual const std::vector<std::shared_ptr<Model>> &get_functions() const
Gets internal sub-graphs.
- Returns
a vector of pointers to sub-graph Models
Adds sub-graph to MultiSubGraphOp.
- Parameters
index – index of new sub-graph
func – func new sub_graph as Model
-
inline const MultiSubgraphInputDescriptionVector &get_input_descriptions(int index) const
Gets vector with connections between operation inputs and internal sub-graph parameters.
- Parameters
index – index of internal sub-graph
- Returns
vector of input descriptions
-
inline MultiSubgraphInputDescriptionVector &get_input_descriptions(int index)
Gets vector with connections between operation inputs and internal sub-graph parameters.
- Parameters
index – index of internal sub-graph
- Returns
vector of input descriptions
-
inline const MultiSubgraphOutputDescriptionVector &get_output_descriptions(int index) const
Gets vector with connections between operation outputs and internal sub-graph results.
- Parameters
index – index of internal sub-graph
- Returns
vector of output descriptions
-
inline MultiSubgraphOutputDescriptionVector &get_output_descriptions(int index)
Gets vector with connections between operation outputs and internal sub-graph results.
- Parameters
index – index of internal sub-graph
- Returns
vector of output descriptions
-
inline void set_input_descriptions(int index, const MultiSubgraphInputDescriptionVector &inputs)
Sets vector with connections between operation inputs and internal sub-graph parameters.
- Parameters
index – index of internal sub-graph
inputs – vector of input descriptions
-
inline void set_output_descriptions(int index, const MultiSubgraphOutputDescriptionVector &outputs)
Sets vector with connections between operation outputs and internal sub-graph results.
- Parameters
index – index of internal sub-graph
outputs – vector of input descriptions
-
virtual void set_invariant_inputs(const Output<Node> &value, const ov::ParameterVector &bodies_parameters)
Set input decriptions for MultiSubGraphOp input.
- Parameters
value – The value supplied as an input to the block.
bodies_parameters – vector of bodies parameters.
-
virtual Output<Node> set_body_outputs(const ResultVector &bodies_results)
Set output decriptions for MultiSubGraphOp output.
- Parameters
bodies_results – vector of bodies results for one output.
- Returns
value Output node for bodies_results.
-
inline virtual size_t get_internal_subgraphs_size() const
Get number of internal sub-graphs.
- Returns
Number of sub-graphs.
-
inline virtual size_t get_input_descriptions_size() const
Get number of input descriptions.
- Returns
Number of input descriptions
-
inline virtual size_t get_output_descriptions_size() const
Get number of output descriptions.
- Returns
Number of output descriptions
-
class BodyOutputDescription : public ov::op::util::MultiSubGraphOp::OutputDescription
- #include <multi_subgraph_base.hpp>
Produces an output from a specific iteration.
Public Functions
-
BodyOutputDescription(uint64_t body_value_index, uint64_t output_index, int64_t iteration = -1)
Constructs a new instance.
- Parameters
body_value_index – A body value that produces the output
output_index – The SubGraphOp output index
iteration – which iteration (typically -1, final) will supply the value
-
BodyOutputDescription(uint64_t body_value_index, uint64_t output_index, int64_t iteration = -1)
-
class ConcatOutputDescription : public ov::op::util::MultiSubGraphOp::OutputDescription
- #include <multi_subgraph_base.hpp>
Produces an output by concatenating an output from each iteration.
Public Functions
-
ConcatOutputDescription(uint64_t body_value_index, uint64_t output_index, int64_t start, int64_t stride, int64_t part_size, int64_t end, int64_t axis)
Constructs a new instance.
- Parameters
body_value_index – A body value that produces the output
output_index – The MultiSubGraphOp output index
start – First index for slices
stride – Step amount for slices
part_size – Width of slices
end – Last index for slices
axis – Axis being sliced
-
ConcatOutputDescription(uint64_t body_value_index, uint64_t output_index, int64_t start, int64_t stride, int64_t part_size, int64_t end, int64_t axis)
-
class InputDescription
- #include <multi_subgraph_base.hpp>
Abstract class describes a connection between a MultiSubGraphOp input and the body.
Subclassed by ov::op::util::MultiSubGraphOp::InvariantInputDescription, ov::op::util::MultiSubGraphOp::MergedInputDescription, ov::op::util::MultiSubGraphOp::SliceInputDescription
-
class InvariantInputDescription : public ov::op::util::MultiSubGraphOp::InputDescription
- #include <multi_subgraph_base.hpp>
Produces an input.
Public Functions
-
InvariantInputDescription(uint64_t input_index, uint64_t body_parameter_index)
Constructs a new instance.
- Parameters
input_index – Position of the MultiSubGraphOp input
body_parameter_index – Body parameter to receive input
-
InvariantInputDescription(uint64_t input_index, uint64_t body_parameter_index)
-
class MergedInputDescription : public ov::op::util::MultiSubGraphOp::InputDescription
- #include <multi_subgraph_base.hpp>
Describes a body input initialized from a MultiSubGraphOp input on the first iteration, and then a body output thereafter.
Public Functions
-
MergedInputDescription(uint64_t input_index, uint64_t body_parameter_index, uint64_t body_value_index)
Constructs a new instance.
- Parameters
input_index – Position of the MultiSubGraphOp input supplying a value to body_parameter for the initial iteration.
body_parameter_index – Body parameter position to receive input.
body_value_index – Body value to supply body_parameter for successive iterations.
-
MergedInputDescription(uint64_t input_index, uint64_t body_parameter_index, uint64_t body_value_index)
-
class OutputDescription
- #include <multi_subgraph_base.hpp>
Abstract class describes how a MultiSubGraphOp output is produced from the body.
Subclassed by ov::op::util::MultiSubGraphOp::BodyOutputDescription, ov::op::util::MultiSubGraphOp::ConcatOutputDescription
-
class SliceInputDescription : public ov::op::util::MultiSubGraphOp::InputDescription
- #include <multi_subgraph_base.hpp>
Describes a body input formed from slices of an input to MultiSubGraphOp.
Public Functions
-
SliceInputDescription(uint64_t input_index, uint64_t body_parameter_index, int64_t start, int64_t stride, int64_t part_size, int64_t end, int64_t axis)
Constructs a new instance.
- Parameters
input_index – Position of the MultiSubGraphOp input
body_parameter_index – Body parameter position to receive input
start – First index for slices
stride – Step amount for slices
part_size – Width of slices
end – Last index for slices
axis – Axis being sliced
-
SliceInputDescription(uint64_t input_index, uint64_t body_parameter_index, int64_t start, int64_t stride, int64_t part_size, int64_t end, int64_t axis)
-
inline virtual const std::shared_ptr<Model> &get_function(size_t index) const
-
class PadBase : public ov::op::Op
- #include <pad_base.hpp>
Subclassed by ov::op::v12::Pad, ov::op::v1::Pad
Public Functions
-
PadBase(const Output<Node> &arg, const Output<Node> &pads_begin, const Output<Node> &pads_end, const Output<Node> &arg_pad_value, PadMode pad_mode)
Constructs a generic padding operation.
- Parameters
arg – The output producing input tensor to be padded.
pads_begin – The output which specifies the number of padding elements added before position 0 on each axis of arg.
pads_end – The output which specifies the number of padding elements after the last element on each axis.
arg_pad_value – The scalar output with the value used for padding if pad_mode is CONSTANT
pad_mode – The padding mode
-
PadBase(const Output<Node> &arg, const Output<Node> &pads_begin, const Output<Node> &pads_end, PadMode pad_mode)
Constructs a generic padding operation.
- Parameters
arg – The output producing input tensor to be padded.
pads_begin – The output which specifies the number of padding elements added
pads_end – The output which specifies the number of padding elements after the last element on each axis.
pad_mode – The padding mode
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
CoordinateDiff get_pads_begin() const
return The node which specifies the number of padding elements added at the beginning of each axis
-
CoordinateDiff get_pads_end() const
return The node which specifies the number of padding elements added at the end of each axis
-
inline PadMode get_pad_mode() const
- Returns
The padding mode.
-
PadBase(const Output<Node> &arg, const Output<Node> &pads_begin, const Output<Node> &pads_end, const Output<Node> &arg_pad_value, PadMode pad_mode)
-
class ReadValueBase : public ov::op::Op, public ov::op::util::VariableExtension
- #include <read_value_base.hpp>
Subclassed by ov::op::v3::ReadValue, ov::op::v6::ReadValue
Public Functions
-
inline explicit ReadValueBase(const OutputVector &arguments)
Constructs an AssignBase operation.
-
inline explicit ReadValueBase(const OutputVector &arguments)
-
class ReductionBase : public ov::op::Op
- #include <reduction_base.hpp>
Subclassed by ov::op::util::ArithmeticReduction, ov::op::util::LogicalReduction
Public Functions
-
bool reduction_axes_constant() const
- Returns
true if reduction axes are constant else false.
-
const AxisSet get_reduction_axes() const
- Throws
CheckFailure – if the reduction axes are not constant. (Use reduction_axes_constant to check.)
- Returns
The axis positions (0-based) to be eliminated through reduction.
-
void set_reduction_axes(const AxisSet &reduction_axes)
Change the reduction axes.
-
bool reduction_axes_constant() const
-
class RNNCellBase : public ov::op::Op
- #include <rnn_cell_base.hpp>
Base class for all recurrent network cells.
Note
It holds all common attributes.
Subclassed by ov::op::internal::AUGRUCell, ov::op::internal::AUGRUSequence, ov::op::v0::LSTMCell, ov::op::v0::LSTMSequence, ov::op::v0::RNNCell, ov::op::v3::GRUCell, ov::op::v4::LSTMCell, ov::op::v5::GRUSequence, ov::op::v5::LSTMSequence, ov::op::v5::RNNSequence
Public Functions
-
RNNCellBase(const OutputVector &args, std::size_t hidden_size, float clip, const std::vector<std::string> &activations, const std::vector<float> &activations_alpha, const std::vector<float> &activations_beta)
Constructs a RNNCellBase class.
- Parameters
hidden_size – [in] The number of hidden units for recurrent cell.
clip – [in] The value defining clipping range [-clip, clip] on input of activation functions.
activations – [in] The vector of activation functions used inside recurrent cell.
activations_alpha – [in] The vector of alpha parameters for activation functions in order respective to activation list.
activations_beta – [in] The vector of beta parameters for activation functions in order respective to activation list.
-
void validate_input_rank_dimension(const std::vector<PartialShape> &input)
Validates static rank and dimension for provided input parameters. Additionally input_size dimension is checked for X and W inputs.
- Parameters
input – [in] Vector with RNN-Cell op inputs in following order: X, initial_hidden_state, W, R and B.
-
RNNCellBase(const OutputVector &args, std::size_t hidden_size, float clip, const std::vector<std::string> &activations, const std::vector<float> &activations_alpha, const std::vector<float> &activations_beta)
-
class ScatterBase : public ov::op::Op
- #include <scatter_base.hpp>
Base class for ScatterXXX operators.
Subclassed by ov::op::v3::ScatterUpdate
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class ScatterElementsUpdateBase : public ov::op::Op
- #include <scatter_elements_update_base.hpp>
Subclassed by ov::op::v12::ScatterElementsUpdate, ov::op::v3::ScatterElementsUpdate
Public Functions
-
ScatterElementsUpdateBase(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &updates, const Output<Node> &axis)
The common base class for all ScatterElementsUpdate operator versions.
- Parameters
data – Input data
indices – Data entry index that will be updated
updates – Update values
axis – Axis to scatter on
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
ScatterElementsUpdateBase(const Output<Node> &data, const Output<Node> &indices, const Output<Node> &updates, const Output<Node> &axis)
-
class ScatterNDBase : public ov::op::Op
- #include <scatter_nd_base.hpp>
Base class for ScatterNDXXX operators.
Subclassed by ov::op::v3::ScatterNDUpdate
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class ShapeOfBase : public ov::op::Op
- #include <shape_of_base.hpp>
Subclassed by ov::op::v0::ShapeOf, ov::op::v3::ShapeOf
Public Functions
-
inline explicit ShapeOfBase(const OutputVector &arguments)
Constructs an ShapeOfBase operation.
-
inline explicit ShapeOfBase(const OutputVector &arguments)
-
class SubGraphOp : public ov::op::util::MultiSubGraphOp
- #include <sub_graph_base.hpp>
Abstract base class for sub-graph based ops, i.e ops that have only one sub-graph.
Subclassed by ov::op::v0::TensorIterator, ov::op::v5::Loop
Public Functions
-
inline const std::vector<std::shared_ptr<InputDescription>> &get_input_descriptions() const
- Returns
a reference to the input descriptions.
-
inline std::vector<std::shared_ptr<InputDescription>> &get_input_descriptions()
- Returns
a reference to the input descriptions. Can add input descriptions before validation.
-
inline const std::vector<std::shared_ptr<OutputDescription>> &get_output_descriptions() const
- Returns
a reference to the output descriptions.
-
inline std::vector<std::shared_ptr<OutputDescription>> &get_output_descriptions()
- Returns
a reference to the output descriptions. Can add output descriptions before validation.
Indicate that a body parameter comes from slices of a value.
- Parameters
parameter – The parameter to receive the slices
value – The value to be sliced. This will be added as an input to SubGraphOp.
start – First index on axis of the slicing
stride – Stepping of the slice
part_size – Size of the slice on axis
end – The last index on axis of the slicing
axis – The axis to slice along
Indicates that a body parameter has an initial value in the first iteration and computed value thereafter.
- Parameters
body_parameter – [in] The body parameter
initial_value – Value for the parameter in first iteration. This will be added as an input to Loop.
successive_value – Value for the parameter in successive iterations. The value is what is active in the most recent completed iteration.
Indicates that a body parameter has an invariant value during iteration that may depend on values computed outside of the iteration.
- Parameters
body_parameter – The body parameter
value – The value supplied as an input to the block
-
virtual Output<Node> get_iter_value(const Output<Node> &body_value, int64_t iteration = -1)
Gets a value for a particular iteration point.
- Parameters
body_value – The value
iteration – The iteration that supplies the value. Negative values are from the last iteration. Default value -1 (the last iteration).
- Returns
The iterator value.
-
virtual Output<Node> get_concatenated_slices(const Output<Node> &value, int64_t start, int64_t stride, int64_t part_size, int64_t end, int64_t axis)
Concatenates slices from all iterations.
- Parameters
value – The value supplying slice values from each iteration.
start – First index on axis of the slicing
stride – Stepping of the slice
part_size – Size of the slice on axis
end – The last index on axis of the slicing
axis – The axis to slice along
- Returns
The concatenated slices.
-
inline const std::vector<std::shared_ptr<InputDescription>> &get_input_descriptions() const
-
class TopKBase : public ov::op::Op
- #include <topk_base.hpp>
Subclassed by ov::op::v11::TopK, ov::op::v1::TopK, ov::op::v3::TopK
Public Functions
-
TopKBase(const Output<Node> &data, const Output<Node> &k, const int64_t axis, const std::string &mode, const std::string &sort, const element::Type &index_element_type = element::i32)
The common base class for all TopK operator versions.
- Parameters
data – The input tensor
k – Specifies how many maximum/minimum elements should be computed
axis – The axis along which TopK should be computed
mode – Specifies whether the maximum or minimum elements are selected
sort – Specifies the order of output elements and/or indices Accepted values: none, index, value
index_element_type – Specifies the type of produced indices
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
uint64_t get_axis() const
Returns axis value after normalization.
Note
If input rank required to normalization is dynamic, the exception is thrown
-
inline int64_t get_provided_axis() const
Returns axis value before normalization.
-
size_t get_k() const
Returns the value of K, if available.
Note
If the second input to this op is a constant, the value is retrieved and returned. If the input is not constant(dynamic) this method returns 0
-
inline virtual size_t get_default_output_index() const override
Returns the output of the default output, or throws if there is none.
-
TopKBase(const Output<Node> &data, const Output<Node> &k, const int64_t axis, const std::string &mode, const std::string &sort, const element::Type &index_element_type = element::i32)
-
class UnaryElementwiseArithmetic : public ov::op::Op
- #include <unary_elementwise_arithmetic.hpp>
Abstract base class for elementwise unary arithmetic operations, i.e., operations where the same scalar arithmetic operation is applied to each element.
For example, if the underlying operation (determined by the subclass) is \(\mathit{op}(x)\), the input tensor \([[x,y],[z,w]]\) will be mapped to \([[\mathit{op}(x),\mathit{op}(y)],[\mathit{op}(z),\mathit{op}(w)]]\).
Inputs
Type
Description
arg\(N[d_1,\dots,d_n]~(n \geq 0)\)
A tensor of any shape. The element type \(N\) may be any numeric type.
Type
Description
\(N[d_1,\dots,d_n]\)
The tensor \(T\), where \(T[i_1,\dots,i_n] = \mathit{op}(\texttt{arg}[i_1,\dots,i_n])\). This will always have the same shape and element type as the input tensor.
Subclassed by ov::op::v0::Abs, ov::op::v0::Acos, ov::op::v0::Asin, ov::op::v0::Atan, ov::op::v0::Ceiling, ov::op::v0::Clamp, ov::op::v0::Cos, ov::op::v0::Cosh, ov::op::v0::Elu, ov::op::v0::Erf, ov::op::v0::Exp, ov::op::v0::Floor, ov::op::v0::GRN, ov::op::v0::Gelu, ov::op::v0::Log, ov::op::v0::Negative, ov::op::v0::Relu, ov::op::v0::Sigmoid, ov::op::v0::Sign, ov::op::v0::Sin, ov::op::v0::Sinh, ov::op::v0::Sqrt, ov::op::v0::Tan, ov::op::v0::Tanh, ov::op::v3::Acosh, ov::op::v3::Asinh, ov::op::v3::Atanh, ov::op::v4::HSwish, ov::op::v4::Mish, ov::op::v4::SoftPlus, ov::op::v5::HSigmoid, ov::op::v5::Round, ov::op::v7::Gelu, ov::op::v9::SoftSign
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class Variable
- #include <variable.hpp>
-
class VariableContext
- #include <variable_context.hpp>
VariableContext stores and manages a evaluation context for Variables.
Public Functions
-
VariableContext() = default
Constructs an uninitialized VariableContext.
-
inline explicit VariableContext(const VariableMap &variable_values)
Constructor for VariableContext.
- Parameters
variable_values – The values associated with a particular Variables.
-
inline void reset_variable_context() const
Sets the reset flags for all stored Variables to true.
-
inline void set_variable_values(const VariableMap &variable_values)
Sets the new values for Variables.
- Parameters
variable_values – The new values associated with a particular Variable.
-
inline void set_variable_value(const Variable::Ptr &variable, const VariableValue::Ptr &variable_value)
Changes/sets the values for Variable.
- Parameters
variable – New or stored Variable.
variable_value – The values associated with the variable.
-
inline void remove_variable_value(const Variable::Ptr &variable)
Removes context for a particular Variable.
- Parameters
variable – The variable for which the context will be cleared.
-
inline const VariableMap &get_variable_values() const
Returns the current values for Variables.
-
inline VariableValue::Ptr get_variable_value(const Variable::Ptr &variable) const
Returns the value for specified Variable.
-
VariableContext() = default
-
class VariableExtension
- #include <variable_extension.hpp>
Subclassed by ov::op::util::AssignBase, ov::op::util::ReadValueBase
Public Functions
-
inline virtual std::shared_ptr<Variable> get_variable() const
Returns variable connected to this node.
Sets a new variable to be connected to this node.
- Parameters
variable – New variable to be connected to this node.
-
inline virtual void set_variable_id(const std::string &variable_id)
Sets the identifier to a variable.
- Parameters
variable_id – New identifier of the variable.
-
virtual std::string get_variable_id() const = 0
Returns the identifier of corresponding variable.
-
inline virtual std::shared_ptr<Variable> get_variable() const
-
struct VariableInfo
- #include <variable.hpp>
-
class VariableValue
- #include <variable_value.hpp>
VariableValue stores data and state (reset flag) for a Variable, and provides an interface for changing them.
Public Functions
-
VariableValue()
Constructs an uninitialized VariableValue.
-
explicit VariableValue(ngraph::HostTensorPtr value)
Constructor for VariableValue.
- Deprecated:
This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead
- Parameters
value – The data for Variable.
-
VariableValue(ngraph::HostTensorPtr value, bool reset)
Constructor for VariableValue.
- Deprecated:
This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead
- Parameters
value – Data for Variable.
reset – The current state of the reset flag.
-
ngraph::HostTensorPtr get_value() const
Returns the current stored data.
- Deprecated:
This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead
-
void set_value(const ngraph::HostTensorPtr &value)
Sets new values for Variable.
- Deprecated:
This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead
- Parameters
value – New data for Variable.
-
void set_reset(bool reset)
Sets the reset flag to a new state.
- Parameters
reset – The new state of the reset flag.
-
bool get_reset() const
Returns the current reset flag state.
-
VariableValue(const ov::Tensor &value, bool reset)
Constructor for VariableValue.
- Deprecated:
This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead
- Parameters
value – Data for Variable.
reset – The current state of the reset flag.
-
const ov::Tensor &get_state() const
Returns the current stored data.
- Deprecated:
This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead
-
void set_state(const ov::Tensor &value)
Sets new values for Variable.
- Deprecated:
This method is deprecated and will be removed in 2024.0 release. Please use method with ov::Tensor instead
- Parameters
value – New data for Variable.
-
VariableValue()
-
namespace detail
Functions
-
namespace embedding
Functions
-
template<class TShape, class TRShape = result_shape_t<TShape>>
TRShape out_shape_infer(const ov::Node *op, const TShape &emb_table_shape, const TShape &dim_shape_src) Return a copy of the
emb_table_shapewith the first dimension replaced by the first dimension from thedim_shape_src- Template Parameters
TShape – Shape type
- Parameters
op – Pointer to operator.
emb_table_shape – The shape to be copied
dim_shape_src – The shape to copy the first dimension from, with dynamic or static rank > 1
- Returns
The copy of the
emb_table_shapewith the first dimsnsion overwritten bydim_shape_src[0]if the rank is static, otherwise fully dynamic shape with dynamic rank.
-
template<class TShape, class TRShape = result_shape_t<TShape>>
-
namespace error
-
namespace fft_common_validation
Enums
-
enum FFTKind
Values:
-
enumerator RealInput
-
enumerator ComplexInput
-
enumerator RealInput
Functions
-
template<class T>
void validate_input_rank(const ov::op::util::FFTBase *op, const std::vector<T> &input_shapes, const T &input_shape, const T &axes_shape, int64_t input_rank, FFTKind fft_kind)
-
template<class T>
void validate_axes(const ov::op::util::FFTBase *op, const std::vector<T> &input_shapes, const T &axes_shape, std::vector<int64_t> &axes, int64_t input_rank, FFTKind fft_kind)
-
enum FFTKind
-
using ActivationFunctionType = std::shared_ptr<Node> (*)(const std::shared_ptr<Node>&, float, float)
-
namespace v0
Functions
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Concat *op, const std::vector<T> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const CTCGreedyDecoder *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DepthToSpace *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DetectionOutput *op, const std::vector<TShape> &input_shapes)
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const FakeQuantize *op, const std::vector<T> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Interpolate *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor)
-
template<class T>
std::vector<result_shape_t<T>> shape_infer(const LSTMCell *op, const std::vector<T> &input_shapes)
-
template<class TShape>
std::vector<result_shape_t<TShape>> shape_infer(const LSTMSequence *op, const std::vector<TShape> &input_shapes)
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const MatMul *op, const std::vector<T> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const PriorBoxClustered *const op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const PriorBox *const op, const std::vector<TShape> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Proposal *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const PSROIPooling *op, const std::vector<TShape> &input_shapes)
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Range *op, const std::vector<T> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const RegionYolo *op, const std::vector<TShape> &input_shapes)
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const ReorgYolo *op, const std::vector<T> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ReverseSequence *op, const std::vector<TShape> &input_shapes)
-
template<class TShape>
std::vector<result_shape_t<TShape>> shape_infer(const RNNCell *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ROIPooling *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ShapeOf *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ShuffleChannels *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ov::op::v0::SpaceToDepth *op, const std::vector<TShape> &input_shapes)
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const Squeeze *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor()) Do Squeeze shape inference.
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Tile *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
-
template<class TOp>
void check_unsqueeze_axes_rank(const TOp *op, const Rank &rank)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const Unsqueeze *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
-
class Abs : public ov::op::util::UnaryElementwiseArithmetic
- #include <abs.hpp>
Elementwise absolute value operation.
Public Functions
-
Abs() = default
Constructs an absolute value operation.
-
Abs(const Output<Node> &arg)
Constructs an absolute value operation.
Output
[d1, ...]- Parameters
arg – Output that produces the input tensor.
[d1, ...]
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Abs() = default
-
class Acos : public ov::op::util::UnaryElementwiseArithmetic
- #include <acos.hpp>
Elementwise inverse cosine (arccos) operation.
Public Functions
-
Acos() = default
Constructs an arccos operation.
-
Acos(const Output<Node> &arg)
Constructs an arccos operation.
Output
[d1, ...]- Parameters
arg – Output that produces the input tensor.
[d1, ...]
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Acos() = default
-
class Asin : public ov::op::util::UnaryElementwiseArithmetic
- #include <asin.hpp>
Elementwise inverse sine (arcsin) operation.
Public Functions
-
Asin() = default
Constructs an arcsin operation.
-
Asin(const Output<Node> &arg)
Constructs an arcsin operation.
Output
[d1, ...]- Parameters
arg – Output that produces the input tensor.
[d1, ...]
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Asin() = default
-
class Atan : public ov::op::util::UnaryElementwiseArithmetic
- #include <atan.hpp>
Elementwise inverse tangent (arctan) operation.
Public Functions
-
Atan() = default
Constructs an arctan operation.
-
Atan(const Output<Node> &arg)
Constructs an arctan operation.
Output
[d1, ...]- Parameters
arg – Output that produces the input tensor.
[d1, ...]
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Atan() = default
-
class BatchNormInference : public ov::op::Op
- #include <batch_norm.hpp>
BatchNormInference operation.
Public Functions
-
BatchNormInference(const Output<Node> &input, const Output<Node> &gamma, const Output<Node> &beta, const Output<Node> &mean, const Output<Node> &variance, double epsilon)
- Parameters
input – [., C, …]
gamma – gamma scaling for normalized value. [C]
beta – bias added to the scaled normalized value [C]
mean – value for mean normalization [C]
variance – value for variance normalization [C]
epsilon – Avoids divsion by 0 if input has 0 variance
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
BatchNormInference(const Output<Node> &input, const Output<Node> &gamma, const Output<Node> &beta, const Output<Node> &mean, const Output<Node> &variance, double epsilon)
-
class Ceiling : public ov::op::util::UnaryElementwiseArithmetic
- #include <ceiling.hpp>
Elementwise ceiling operation.
Public Functions
-
Ceiling() = default
Constructs a ceiling operation.
-
Ceiling(const Output<Node> &arg)
Constructs a ceiling operation.
- Parameters
arg – Node that produces the input tensor.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Ceiling() = default
-
class Clamp : public ov::op::util::UnaryElementwiseArithmetic
- #include <clamp.hpp>
Performs a clipping operation on all elements of the input node.
All input values that are outside of the <min;max> range are set to ‘min’ or ‘max’ depending on which side of the <min;max> range they are. The values that fall into this range remain unchanged.
Public Functions
-
Clamp(const Output<Node> &data, const double min, const double max)
Constructs a Clamp node.
- Parameters
data – - Node producing the input tensor
min – - the lower bound of the <min;max> range
max – - the upper bound of the <min;max> range
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Clamp(const Output<Node> &data, const double min, const double max)
-
class Concat : public ov::op::Op
- #include <concat.hpp>
Concatenation operation.
Public Functions
-
Concat() = default
Constructs a concatenation operation.
-
Concat(const OutputVector &args, int64_t axis)
Constructs a concatenation operation.
- Parameters
args – The outputs producing the input tensors.
axis – The axis along which to concatenate the input tensors.
-
Concat(const NodeVector &args, int64_t axis)
Constructs a concatenation operation.
- Parameters
args – The nodes producing the input tensors.
axis – The axis along which to concatenate the input tensors.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
inline int64_t get_concatenation_axis() const
- Returns
The concatenation axis.
-
inline int64_t get_axis() const
- Returns
The concatenation axis.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Concat() = default
-
class Constant : public ov::op::Op
- #include <constant.hpp>
Class for constants.
Public Functions
Initialize a constant from tensor.
- Parameters
tensor – The tensor with data
Constructs a tensor constant with the supplied data.
- Parameters
type – The element type of the tensor constant.
shape – The shape of the tensor constant.
data – A pointer to pre-allocated shared data.
-
OPENVINO_SUPPRESS_DEPRECATED_END Constant(const ov::Tensor &tensor)
Initialize a constant from ov::Tensor.
- Parameters
tensor – The ov::Tensor with data
-
template<typename T>
inline Constant(const element::Type &type, const Shape &shape, const std::vector<T> &values) Constructs a tensor constant.
- Parameters
type – The element type of the tensor constant.
shape – The shape of the tensor constant.
values – A vector of literals for initializing the tensor constant. The size of values must match the size of the shape.
-
template<class T, class = typename std::enable_if<std::is_fundamental<T>::value>::type>
inline Constant(const element::Type &type, const Shape &shape, T value) Constructs a uniform tensor constant.
- Parameters
type – The element type of the tensor constant.
shape – The shape of the tensor constant.
value – A scalar for initializing the uniform tensor constant. The value is broadcast to the specified shape.
-
Constant(const element::Type &type, const Shape &shape, const std::vector<std::string> &values)
Constructs a tensor constant This constructor is mainly to support deserialization of constants.
- Parameters
type – The element type of the tensor constant.
shape – The shape of the tensor constant.
values – A list of string values to use as the constant data.
-
Constant(const element::Type &type, const Shape &shape, const void *data)
Constructs a tensor constant with the supplied data.
- Parameters
type – The element type of the tensor constant.
shape – The shape of the tensor constant.
data – A void* to constant data.
-
inline virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override
Evaluates the op on input_values putting results in output_values.
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
- Returns
true if successful
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Shape get_shape_val() const
Returns the value of the constant node as a Shape object Can only be used on element::i64 nodes and interprets negative values as zeros.
-
Strides get_strides_val() const
Returns the value of the constant node as a Strides object Can only be used on element::i64 nodes and interprets negative values as zeros.
-
Coordinate get_coordinate_val() const
Returns the value of the constant node as a Coordinate object Can only be used on element::i64 nodes and interprets negative values as zeros.
-
CoordinateDiff get_coordinate_diff_val() const
Returns the value of the constant node as a CoordinateDiff object Can only be used on element::i64 nodes.
-
AxisVector get_axis_vector_val() const
Returns the value of the constant node as an AxisVector object Can only be used on element::i64 nodes and interprets negative values as zeros.
-
AxisSet get_axis_set_val() const
Returns the value of the constant node as an AxisSet object Can only be used on element::i64 nodes and interprets negative values as zeros. Repeated values are allowed.
-
size_t get_byte_size() const
Return data size in bytes.
-
std::vector<std::string> get_value_strings() const
- Returns
The initialization literals for the tensor constant.
-
template<typename T>
inline std::vector<T> cast_vector(int64_t num_elements = -1) const Return the Constant’s value as a vector cast to type T.
- Template Parameters
T – Type to which data vector’s entries will be cast.
- Parameters
num_elements – (Optional) Number of elements to cast. In default case returns all elements
- Returns
Constant’s data vector.
Public Static Functions
Wrapper around constructing a shared_ptr of a Constant.
- Parameters
type – The element type of the tensor constant.
shape – The shape of the tensor constant.
values – A vector of values to use as the constant data.
Wrapper around constructing a shared_ptr of a Constant.
- Parameters
type – The element type of the tensor constant.
shape – The shape of the tensor constant.
values – An initializer_list of values to use as the constant data.
-
static inline std::shared_ptr<Constant> create(const element::Type &type, const Shape &shape, const void *memory)
Wrapper around constructing a shared_ptr of a Constant.
- Parameters
type – The element type of the tensor constant.
shape – The shape of the tensor constant.
memory – An continues memory chunk which contains the constant data.
-
class Convert : public ov::op::Op
- #include <convert.hpp>
Elementwise type conversion operation.
Public Functions
-
Convert() = default
Constructs a conversion operation.
-
Convert(const Output<Node> &arg, const ov::element::Type &destination_type)
Constructs a conversion operation.
- Parameters
arg – Node that produces the input tensor.
destination_type – Element type for the output tensor.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Convert() = default
-
class Cos : public ov::op::util::UnaryElementwiseArithmetic
- #include <cos.hpp>
Elementwise cosine operation.
Public Functions
-
Cos() = default
Constructs a cosine operation.
-
Cos(const Output<Node> &arg)
Constructs a cosine operation.
- Parameters
arg – Node that produces the input tensor.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Cos() = default
-
class Cosh : public ov::op::util::UnaryElementwiseArithmetic
- #include <cosh.hpp>
Elementwise hyperbolic cosine (cosh) operation.
Public Functions
-
Cosh() = default
Constructs a hyperbolic cosine operation.
-
Cosh(const Output<Node> &arg)
Constructs a hyperbolic cosine operation.
- Parameters
arg – Node that produces the input tensor.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Cosh() = default
-
class CTCGreedyDecoder : public ov::op::Op
- #include <ctc_greedy_decoder.hpp>
CTCGreedyDecoder operation.
Public Functions
-
CTCGreedyDecoder(const Output<Node> &input, const Output<Node> &seq_len, const bool ctc_merge_repeated)
Constructs a CTCGreedyDecoder operation.
- Parameters
input – Logits on which greedy decoding is performed
seq_len – Sequence lengths
ctc_merge_repeated – Whether to merge repeated labels
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
CTCGreedyDecoder(const Output<Node> &input, const Output<Node> &seq_len, const bool ctc_merge_repeated)
-
class CumSum : public ov::op::Op
- #include <cum_sum.hpp>
Tensor cumulative sum operation.
Compute the cumulative sum of the input tensor along the axis specified.
Public Functions
-
CumSum() = default
Constructs a cumulative summation operation.
-
CumSum(const Output<Node> &arg, const Output<Node> &axis, const bool exclusive = false, const bool reverse = false)
Constructs a cumulative summation operation.
- Parameters
arg – The tensor to be summed.
axis – zero dimension tensor specifying axis position along which cumulative sum must be performed
exclusive – if set to true, the top element is not included
reverse – if set to true, will perform the sums in reverse direction
-
CumSum(const Output<Node> &arg, const bool exclusive = false, const bool reverse = false)
Constructs a cumulative summation operation with axis = 0.
- Parameters
arg – The tensor to be summed
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
CumSum() = default
-
class DepthToSpace : public ov::op::Op
- #include <depth_to_space.hpp>
DepthToSpace permutes data from the depth dimension of the input blob into spatial dimensions.
Output node produces a tensor with shape: [N, C/(blocksize * blocksize), H * blocksize, W * blocksize]
Note
Values from the depth dimension (assuming NCHW layout) are moved in spatial blocks to the height and width dimensions.
Public Functions
-
DepthToSpace(const Output<Node> &data, const DepthToSpaceMode &mode, std::size_t block_size = 1)
Constructs a DepthToSpace operation.
- Parameters
data – Node producing the input tensor
mode – Specifies how the input depth dimension is split to block coordinates
block_size – The size of the block of values to be moved
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
DepthToSpace(const Output<Node> &data, const DepthToSpaceMode &mode, std::size_t block_size = 1)
-
class DetectionOutput : public ov::op::util::DetectionOutputBase
- #include <detection_output.hpp>
Layer which performs non-max suppression to generate detection output using location and confidence predictions.
Public Functions
-
DetectionOutput(const Output<Node> &box_logits, const Output<Node> &class_preds, const Output<Node> &proposals, const Output<Node> &aux_class_preds, const Output<Node> &aux_box_preds, const Attributes &attrs)
Constructs a DetectionOutput operation.
- Parameters
box_logits – Box logits
class_preds – Class predictions
proposals – Proposals
aux_class_preds – Auxilary class predictions
aux_box_preds – Auxilary box predictions
attrs – Detection Output attributes
-
DetectionOutput(const Output<Node> &box_logits, const Output<Node> &class_preds, const Output<Node> &proposals, const Attributes &attrs)
Constructs a DetectionOutput operation.
- Parameters
box_logits – Box logits
class_preds – Class predictions
proposals – Proposals
attrs – Detection Output attributes
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
struct Attributes : public ov::op::util::DetectionOutputBase::AttributesBase
- #include <detection_output.hpp>
-
DetectionOutput(const Output<Node> &box_logits, const Output<Node> &class_preds, const Output<Node> &proposals, const Output<Node> &aux_class_preds, const Output<Node> &aux_box_preds, const Attributes &attrs)
-
class Elu : public ov::op::util::UnaryElementwiseArithmetic
- #include <elu.hpp>
Exponential Linear Unit x < 0 => f(x) = alpha * (exp(x) - 1.) x >= 0 => f(x) = x.
Public Functions
-
Elu(const Output<Node> &data, const double alpha)
Constructs an Elu operation.
- Parameters
data – Input tensor
alpha – Multiplier for negative values
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
Elu(const Output<Node> &data, const double alpha)
-
class Erf : public ov::op::util::UnaryElementwiseArithmetic
- #include <erf.hpp>
Elementwise erf operation.
Public Functions
-
Erf() = default
Constructs a floor operation.
-
Erf(const Output<Node> &arg)
Constructs a floor operation.
- Parameters
arg – Node that produces the input tensor.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Erf() = default
-
class Exp : public ov::op::util::UnaryElementwiseArithmetic
- #include <exp.hpp>
Elementwise natural exponential (exp) operation.
Public Functions
-
Exp() = default
Constructs an exponential operation.
-
Exp(const Output<Node> &arg)
Constructs an exponential operation.
- Parameters
arg – Node that produces the input tensor.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Exp() = default
-
class FakeQuantize : public ov::op::Op
- #include <fake_quantize.hpp>
Class performing element-wise linear quantization.
Note
Input floating point values are quantized into a discrete set of floating point values.
Public Functions
-
FakeQuantize(const Output<Node> &data, const Output<Node> &input_low, const Output<Node> &input_high, const Output<Node> &output_low, const Output<Node> &output_high, std::size_t levels, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))
Constructs a FakeQuantize operation node.
- Parameters
data – [in] The input data tensor.
input_low – [in] The minimum limit for input values.
input_high – [in] The maximum limit for input values.
output_low – [in] The minimum quantized value.
output_high – [in] The maximum quantized value.
levels – [in] The number of quantization levels.
auto_broadcast – [in] AutoBroadcast mode to be used for broadcasting limit values
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
FakeQuantize(const Output<Node> &data, const Output<Node> &input_low, const Output<Node> &input_high, const Output<Node> &output_low, const Output<Node> &output_high, std::size_t levels, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))
-
class Floor : public ov::op::util::UnaryElementwiseArithmetic
- #include <floor.hpp>
Elementwise floor operation.
Public Functions
-
Floor() = default
Constructs a floor operation.
-
Floor(const Output<Node> &arg)
Constructs a floor operation.
- Parameters
arg – Node that produces the input tensor.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Floor() = default
-
class Gelu : public ov::op::util::UnaryElementwiseArithmetic
- #include <gelu.hpp>
Gaussian Error Linear Unit f(x) = 0.5 * x * (1 + erf( x / sqrt(2) )
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual void validate_and_infer_types() override
-
class GRN : public ov::op::util::UnaryElementwiseArithmetic
- #include <grn.hpp>
Global Response Normalization with L2 norm (across channels only).
Public Functions
-
GRN(const Output<Node> &data, float bias)
Constructs a GRN operation.
- Parameters
data – - Node producing the input tensor
bias – - The bias added to the variance.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
GRN(const Output<Node> &data, float bias)
-
class HardSigmoid : public ov::op::Op
- #include <hard_sigmoid.hpp>
Parameterized, bounded sigmoid-like, piecewise linear function. min(max(alpha*x + beta, 0), 1)
Public Functions
-
HardSigmoid(const Output<Node> &data, const Output<Node> &alpha, const Output<Node> &beta)
Constructs a HardSigmoid operation.
- Parameters
data – Input tensor.
alpha – [in] A scalar value representing the alpha parameter.
beta – [in] A scalar value representing the beta parameter.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
HardSigmoid(const Output<Node> &data, const Output<Node> &alpha, const Output<Node> &beta)
-
class Interpolate : public ov::op::Op
- #include <interpolate.hpp>
Layer which performs bilinear interpolation.
Public Functions
-
Interpolate(const Output<Node> &image, const Output<Node> &output_shape, const Attributes &attrs)
Constructs a Interpolate operation.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
struct Attributes
- #include <interpolate.hpp>
Structure that specifies attributes for interpolation.
-
Interpolate(const Output<Node> &image, const Output<Node> &output_shape, const Attributes &attrs)
-
class Log : public ov::op::util::UnaryElementwiseArithmetic
- #include <log.hpp>
Elementwise natural log operation.
Public Functions
-
Log() = default
Constructs a natural log operation.
-
Log(const Output<Node> &arg)
Constructs a natural log operation.
- Parameters
arg – Node that produces the input tensor.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Log() = default
-
class LRN : public ov::op::Op
- #include <lrn.hpp>
Elementwise Local Response Normalization (LRN) operation.
Inputs
Type
Description
arg\(N[n, c, d_1,\dots,d_n]~(n \geq 0)\)
A tensor of any shape and numeric element type.
Type
Description
\(N[n, c, d_1,\dots,d_n]\)
The tensor \(T\), where \(T[n, c, d_1,\dots,d_n] = \frac{N[n,i,d_1,\dots,d_n]}{ (bias + alpha * (\sum_{i=max(0,(nsize-1)/2)}^{min(C, (nsize-1)/2)+1} N[n,i,d_1,\dots,d_n]^{2}) ^ {2})}\)
Public Functions
-
LRN() = default
Constructs a LRN operation.
-
LRN(const Output<Node> &arg, double alpha, double beta, double bias, size_t size)
Constructs a LRN operation.
- Parameters
arg – Node that produces the input tensor.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
LRN() = default
-
class LSTMCell : public ov::op::util::RNNCellBase
- #include <lstm_cell.hpp>
Class for single lstm cell node.
See also
LSTMSequence, RNNCell, GRUCell
Note
Following implementation supports:
peepholesGers & Schmidhuber (2000) https://ieeexplore.ieee.org/document/861302Coupling input and forget gates.
Note
It calculates following equations:
it = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Pi (.) Ct-1 + Wbi + Rbi) ft = f(Xt*(Wf^T) + Ht-1*(Rf^T) + Pf (.) Ct-1 + Wbf + Rbf) ct = g(Xt*(Wc^T) + Ht-1*(Rc^T) + Wbc + Rbc) Ct = ft (.) Ct-1 + it (.) ct ot = f(Xt*(Wo^T) + Ht-1*(Ro^T) + Po (.) Ct + Wbo + Rbo) Ht = ot (.) h(Ct) * - Is a dot product, (.) - is a Hadamard product (element-wise), f, g, h - are activation functions.
Note
This class represents only single cell (for current time step) and not the whole LSTM Sequence layer
Public Functions
-
LSTMCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &initial_cell_state, const Output<Node> &W, const Output<Node> &R, std::size_t hidden_size, LSTMWeightsFormat weights_format = LSTMWeightsFormat::IFCO, const std::vector<std::string> &activations = std::vector<std::string>{"sigmoid", "tanh", "tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f, bool input_forget = false)
Constructs LSTMCell node.
- Parameters
X – [in] The input tensor with shape: [batch_size, input_size].
initial_hidden_state – [in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].
initial_cell_state – [in] The cell state tensor at current time step with shape: [batch_size, hidden_size].
W – [in] The gate weights tensor with shape: [4*hidden_size, input_size].
R – [in] The recurrence weights tensor with shape: [4*hidden_size, hidden_size].
hidden_size – [in] The number of hidden units for recurrent cell.
weights_format – [in] The order of gates in weights tensors. The default format is IFCO since it is used by DNNL.
activations – [in] The vector of activation functions used inside recurrent cell.
activations_alpha – [in] The vector of alpha parameters for activation functions in order respective to activation list.
activations_beta – [in] The vector of beta parameters for activation functions in order respective to activation list.
clip – [in] The value defining clipping range [-clip, clip] on input of activation functions.
input_forget – [in] Controls coupling input and forget gates.
-
LSTMCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &initial_cell_state, const Output<Node> &W, const Output<Node> &R, const Output<Node> &B, std::size_t hidden_size, LSTMWeightsFormat weights_format = LSTMWeightsFormat::IFCO, const std::vector<std::string> &activations = std::vector<std::string>{"sigmoid", "tanh", "tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f, bool input_forget = false)
Constructs LSTMCell node.
- Parameters
X – [in] The input tensor with shape: [batch_size, input_size].
initial_hidden_state – [in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].
initial_cell_state – [in] The cell state tensor at current time step with shape: [batch_size, hidden_size].
W – [in] The weight tensor with shape: [4*hidden_size, input_size].
R – [in] The recurrence weight tensor with shape: [4*hidden_size, hidden_size].
B – [in] The bias tensor for gates with shape: [4*hidden_size].
hidden_size – [in] The number of hidden units for recurrent cell.
weights_format – [in] The order of gates in weights tensors. The default format is IFCO since it is used by DNNL.
activations – [in] The vector of activation functions used inside recurrent cell.
activations_alpha – [in] The vector of alpha parameters for activation functions in order respective to activation list.
activations_beta – [in] The vector of beta parameters for activation functions in order respective to activation list.
clip – [in] The value defining clipping range [-clip, clip] on input of activation functions.
input_forget – [in] Controls coupling input and forget gates.
-
LSTMCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &initial_cell_state, const Output<Node> &W, const Output<Node> &R, const Output<Node> &B, const Output<Node> &P, std::size_t hidden_size, LSTMWeightsFormat weights_format = LSTMWeightsFormat::IFCO, const std::vector<std::string> &activations = std::vector<std::string>{"sigmoid", "tanh", "tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f, bool input_forget = false)
Constructs LSTMCell node.
- Parameters
X – [in] The input tensor with shape: [batch_size, input_size].
initial_hidden_state – [in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].
initial_cell_state – [in] The cell state tensor at current time step with shape: [batch_size, hidden_size].
W – [in] The weight tensor with shape: [4*hidden_size, input_size].
R – [in] The recurrence weight tensor with shape: [4*hidden_size, hidden_size].
B – [in] The bias tensor for gates with shape: [4*hidden_size].
P – [in] The weight tensor for peepholes with shape: [3*hidden_size] - 3 equals to only iof gates. The order is: input, output, forget gates.
hidden_size – [in] The number of hidden units for recurrent cell.
weights_format – [in] The order of gates in weights tensors. The default format is IFCO since it is used by DNNL.
activations – [in] The vector of activation functions used inside recurrent cell.
activations_alpha – [in] The vector of alpha parameters for activation functions in order respective to activation list.
activations_beta – [in] The vector of beta parameters for activation functions in order respective to activation list.
clip – [in] The value defining clipping range [-clip, clip] on input of activation functions.
input_forget – [in] Controls coupling input and forget gates.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
class LSTMSequence : public ov::op::util::RNNCellBase
- #include <lstm_sequence.hpp>
Class for lstm sequence node.
Note
It follows notation and equations defined as in ONNX standard: https://github.com/onnx/onnx/blob/master/docs/Operators.md#LSTM
Public Functions
-
inline virtual size_t get_default_output_index() const override
Returns the output of the default output, or throws if there is none.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
inline virtual size_t get_default_output_index() const override
-
class MatMul : public ov::op::Op
- #include <matmul.hpp>
Operator performing Matrix Multiplication.
Public Functions
-
MatMul(const Output<Node> &A, const Output<Node> &B, const bool &transpose_a = false, const bool &transpose_b = false)
Constructs an Matrix Multiplication operation.
- Parameters
A – Matrix A
B – Matrix B
transpose_a – If matrix A should be transposed.
transpose_b – If matrix B should be transposed.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
MatMul(const Output<Node> &A, const Output<Node> &B, const bool &transpose_a = false, const bool &transpose_b = false)
-
class MVN : public ov::op::Op
- #include <mvn.hpp>
Operator performing Mean Variance Normalization.
Public Functions
-
MVN(const Output<Node> &data, bool across_channels = true, bool normalize_variance = true, double eps = 1e-9)
Constructs an MVN operation.
- Parameters
data – Input tensor with data
normalize_variance – flag that denotes whether to perform variance normalization.
across_channels – flag that denotes if mean values are shared across channels.
eps – the number to be added to the variance to avoid division by zero when normalizing the value
-
MVN(const Output<Node> &data, AxisSet reduction_axes, bool normalize_variance = true, double eps = 1e-9)
Constructs an MVN operation.
- Parameters
data – Input tensor with data
reduction_axes – A list of axes, along which to reduce.
normalize_variance – flag that denotes whether to perform variance normalization.
eps – the number to be added to the variance to avoid division by zero when normalizing the value
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
MVN(const Output<Node> &data, bool across_channels = true, bool normalize_variance = true, double eps = 1e-9)
-
class Negative : public ov::op::util::UnaryElementwiseArithmetic
- #include <negative.hpp>
Elementwise negative operation.
Public Functions
-
Negative() = default
Constructs a negative operation.
-
Negative(const Output<Node> &arg)
Constructs a negative operation.
- Parameters
arg – Node that produces the input tensor.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Negative() = default
-
template<class TShape>
struct NegativeToZero - #include <tile_shape_inference.hpp>
-
class NormalizeL2 : public ov::op::Op
- #include <normalize_l2.hpp>
Normalization with L2 norm.
Public Functions
-
NormalizeL2(const Output<Node> &data, const Output<Node> &axes, float eps, EpsMode eps_mode)
Constructs a NormalizeL2 operation.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
NormalizeL2(const Output<Node> &data, const Output<Node> &axes, float eps, EpsMode eps_mode)
-
class Parameter : public ov::op::Op
- #include <parameter.hpp>
A model parameter.
Parameters are nodes that represent the arguments that will be passed to user-defined models. Model creation requires a sequence of parameters. Basic graph operations do not need parameters attached to a model.
Public Functions
-
Parameter() = default
Constructions a tensor-typed parameter node.
-
Parameter(const ov::element::Type &element_type, const PartialShape &pshape)
Constructions a tensor-typed parameter node.
- Parameters
element_type – The element type of the parameter.
pshape – The partial shape of the parameter.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
Parameter() = default
-
class PRelu : public ov::op::Op
- #include <prelu.hpp>
Parametrized Relu x < 0 => f(x) = x * slope x >= 0 => f(x) = x.
Public Functions
-
PRelu(const Output<Node> &data, const Output<Node> &slope)
Constructs a PRelu operation.
- Parameters
data – Input tensor
slope – Multipliers for negative values
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
PRelu(const Output<Node> &data, const Output<Node> &slope)
-
class PriorBox : public ov::op::Op
- #include <prior_box.hpp>
Layer which generates prior boxes of specified sizes normalized to input image size.
Public Functions
-
PriorBox(const Output<Node> &layer_shape, const Output<Node> &image_shape, const Attributes &attrs)
Constructs a PriorBox operation.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
struct Attributes
- #include <prior_box.hpp>
-
PriorBox(const Output<Node> &layer_shape, const Output<Node> &image_shape, const Attributes &attrs)
-
class PriorBoxClustered : public ov::op::Op
- #include <prior_box_clustered.hpp>
Layer which generates prior boxes of specified sizes normalized to input image size.
Public Functions
-
PriorBoxClustered(const Output<Node> &layer_shape, const Output<Node> &image_shape, const Attributes &attrs)
Constructs a PriorBoxClustered operation.
- Parameters
layer_shape – Shape of layer for which prior boxes are computed
image_shape – Shape of image to which prior boxes are scaled
attrs – PriorBoxClustered attributes
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
struct Attributes
- #include <prior_box_clustered.hpp>
-
PriorBoxClustered(const Output<Node> &layer_shape, const Output<Node> &image_shape, const Attributes &attrs)
-
class Proposal : public ov::op::Op
- #include <proposal.hpp>
Proposal operation.
Subclassed by ov::op::v4::Proposal
Public Functions
-
Proposal(const Output<Node> &class_probs, const Output<Node> &bbox_deltas, const Output<Node> &image_shape, const Attributes &attrs)
Constructs a Proposal operation.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
void set_attrs(Attributes attrs)
Set the Proposal operator attributes.
- Parameters
attrs – Attributes to be set.
-
struct Attributes
- #include <proposal.hpp>
-
Proposal(const Output<Node> &class_probs, const Output<Node> &bbox_deltas, const Output<Node> &image_shape, const Attributes &attrs)
-
class PSROIPooling : public ov::op::Op
- #include <psroi_pooling.hpp>
PSROIPooling operation.
Public Functions
-
PSROIPooling(const Output<Node> &input, const Output<Node> &coords, const size_t output_dim, const size_t group_size, const float spatial_scale, int spatial_bins_x, int spatial_bins_y, const std::string &mode)
Constructs a PSROIPooling operation.
- Parameters
input – Input feature map {N, C, …}
coords – Coordinates of bounding boxes
output_dim – Output channel number
group_size – Number of groups to encode position-sensitive scores
spatial_scale – Ratio of input feature map over input image size
spatial_bins_x – Numbers of bins to divide the input feature maps over width
spatial_bins_y – Numbers of bins to divide the input feature maps over height
mode – Mode of pooling - Avg or Bilinear
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
void set_output_dim(size_t output_dim)
Set the output channel dimension size.
- Parameters
output_dim – Channel dimension size.
-
void set_group_size(size_t group_size)
Set the output groups number.
- Parameters
group_size – Number of groups.
-
void set_spatial_scale(float scale)
Set the spatial scale.
- Parameters
scale – Spatial scale value.
-
void set_spatial_bins_x(int x)
Set the number of bins over image width.
- Parameters
x – Number of bins over width (x) axis.
-
void set_spatial_bins_y(int y)
Set the number of bins over image height.
- Parameters
y – Number of bins over height (y) axis.
-
void set_mode(std::string mode)
Set the pooling mode.
- Parameters
mode – Pooling mode name.
-
PSROIPooling(const Output<Node> &input, const Output<Node> &coords, const size_t output_dim, const size_t group_size, const float spatial_scale, int spatial_bins_x, int spatial_bins_y, const std::string &mode)
-
class Range : public ov::op::Op
- #include <range.hpp>
Range operation, analogous to
range()in Python.Public Functions
-
Range() = default
Constructs an unitialized range operation.
-
Range(const Output<Node> &start, const Output<Node> &stop, const Output<Node> &step)
Constructs a range operation.
- Parameters
start – The tensor producing the start value. Must be a scalar of integer element type, and same element type as
stopandstep.stop – The tensor producing the stop value. Must be a scalar of integer element type, and same element type as
startandstep.step – The tensor producing the step value. Must be a scalar of integer element type, and same element type as
startandstop.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Range() = default
-
class RegionYolo : public ov::op::Op
- #include <region_yolo.hpp>
RegionYolo operation.
Public Functions
-
RegionYolo(const Output<Node> &input, const size_t coords, const size_t classes, const size_t regions, const bool do_softmax, const std::vector<int64_t> &mask, const int axis, const int end_axis, const std::vector<float> &anchors = std::vector<float>{})
Constructs a RegionYolo operation.
- Parameters
input – [in] Input
coords – [in] Number of coordinates for each region
classes – [in] Number of classes for each region
regions – [in] Number of regions
do_softmax – [in] Compute softmax
mask – [in] Mask
axis – [in] Axis to begin softmax on
end_axis – [in] Axis to end softmax on
anchors – [in] A flattened list of pairs
[width, height]that describes prior box sizes.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
RegionYolo(const Output<Node> &input, const size_t coords, const size_t classes, const size_t regions, const bool do_softmax, const std::vector<int64_t> &mask, const int axis, const int end_axis, const std::vector<float> &anchors = std::vector<float>{})
-
class Relu : public ov::op::util::UnaryElementwiseArithmetic
- #include <relu.hpp>
Elementwise Relu operation.
-
class ReorgYolo : public ov::op::Op
- #include <reorg_yolo.hpp>
ReorgYolo operation.
Public Functions
-
ReorgYolo(const Output<Node> &input, const size_t stride)
Constructs a ReorgYolo operation.
- Parameters
input – Input
stride – Stride to reorganize input by
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
ReorgYolo(const Output<Node> &input, const size_t stride)
-
class Result : public ov::op::Op
- #include <result.hpp>
Result operation.
Public Functions
-
Result() = default
Allows a value to be used as a function result.
-
Result(const Output<Node> &arg)
Allows a value to be used as a function result.
- Parameters
arg – Node that produces the input tensor.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override
Evaluates the op on input_values putting results in output_values.
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
- Returns
true if successful
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Result() = default
-
class ReverseSequence : public ov::op::Op
- #include <reverse_sequence.hpp>
ReverseSequence operation.
Public Functions
-
ReverseSequence(const Output<Node> &arg, const Output<Node> &seq_lengths, int64_t batch_axis = 0, int64_t seq_axis = 1)
Constructs a ReverseSequence operation.
- Parameters
arg – tensor with input data to reverse
seq_lengths – 1D tensor of integers with sequence lengths in the input tensor.
batch_axis – index of the batch dimension.
seq_axis – index of the sequence dimension.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
ReverseSequence(const Output<Node> &arg, const Output<Node> &seq_lengths, int64_t batch_axis = 0, int64_t seq_axis = 1)
-
class RNNCell : public ov::op::util::RNNCellBase
- #include <rnn_cell.hpp>
Class for single RNN cell node.
See also
LSTMSequence, LSTMCell, GRUCell
Note
It follows notation and equations defined as in ONNX standard: https://github.com/onnx/onnx/blob/master/docs/Operators.md#RNN
Note
It calculates following equations:
Ht = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Wbi + Rbi) * - Is a dot product, f - is activation functions.
Note
This class represents only single cell (for current time step) and not the whole RNN Sequence layer
Public Functions
-
RNNCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &W, const Output<Node> &R, std::size_t hidden_size, const std::vector<std::string> &activations = std::vector<std::string>{"tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f)
Constructs RNNCell node.
- Parameters
X – [in] The input tensor with shape: [batch_size, input_size].
initial_hidden_state – [in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].
W – [in] The weight tensor with shape: [hidden_size, input_size].
R – [in] The recurrence weight tensor with shape: [hidden_size, hidden_size].
hidden_size – [in] The number of hidden units for recurrent cell.
activations – [in] The vector of activation functions used inside recurrent cell.
activations_alpha – [in] The vector of alpha parameters for activation functions in order respective to activation list.
activations_beta – [in] The vector of beta parameters for activation functions in order respective to activation list.
clip – [in] The value defining clipping range [-clip, clip] on input of activation functions.
-
RNNCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &W, const Output<Node> &R, const Output<Node> &B, std::size_t hidden_size, const std::vector<std::string> &activations = std::vector<std::string>{"tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f)
Constructs RNNCell node.
- Parameters
X – [in] The input tensor with shape: [batch_size, input_size].
initial_hidden_state – [in] The hidden state tensor at current time step with shape: [batch_size, hidden_size].
W – [in] The weight tensor with shape: [hidden_size, input_size].
R – [in] The recurrence weight tensor with shape: [hidden_size, hidden_size].
B – [in] The bias tensor for input gate with shape: [hidden_size].
hidden_size – [in] The number of hidden units for recurrent cell.
activations – [in] The vector of activation functions used inside recurrent cell.
activations_alpha – [in] The vector of alpha parameters for activation functions in order respective to activation list.
activations_beta – [in] The vector of beta parameters for activation functions in order respective to activation list.
clip – [in] The value defining clipping range [-clip, clip] on input of activation functions.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
RNNCell(const Output<Node> &X, const Output<Node> &initial_hidden_state, const Output<Node> &W, const Output<Node> &R, std::size_t hidden_size, const std::vector<std::string> &activations = std::vector<std::string>{"tanh"}, const std::vector<float> &activations_alpha = {}, const std::vector<float> &activations_beta = {}, float clip = 0.f)
-
class ROIPooling : public ov::op::Op
- #include <roi_pooling.hpp>
ROIPooling operation.
Public Functions
-
ROIPooling(const Output<Node> &input, const Output<Node> &coords, const Shape &output_size, const float spatial_scale, const std::string &method = "max")
Constructs a ROIPooling operation.
- Parameters
input – Input feature map {N, C, H, W}
coords – Coordinates of bounding boxes
output_size – Height/Width of ROI output features
spatial_scale – Ratio of input feature map over input image size
method – Method of pooling - Max or Bilinear
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
void set_output_roi(Shape output_size)
Set the output ROI feature map (pooled_h, pooled_w).
- Parameters
output_size – Shape with pooling attributes pooled_h and pooled_w sizes.
-
const Shape &get_output_roi() const
Get the output ROI feature map shape (H x W)
- Returns
Shape with pooled_h and pooled_w attributes.
-
void set_spatial_scale(float scale)
Set the spatial scale value.
- Parameters
scale – Scale value to set.
-
void set_method(std::string method_name)
Set the method of pooling.
- Parameters
method_name – Pooling method name.
-
ROIPooling(const Output<Node> &input, const Output<Node> &coords, const Shape &output_size, const float spatial_scale, const std::string &method = "max")
-
class Selu : public ov::op::Op
- #include <selu.hpp>
Performs a SELU activation function on all elements of the input node.
Public Functions
-
Selu(const Output<Node> &data, const Output<Node> &alpha, const Output<Node> &lambda)
Constructs a Selu node.
- Parameters
data – - Node producing the input tensor
alpha – - Alpha coefficient of SELU operation
lambda – - Lambda coefficient of SELU operation
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
Selu(const Output<Node> &data, const Output<Node> &alpha, const Output<Node> &lambda)
-
class ShapeOf : public ov::op::util::ShapeOfBase
- #include <shape_of.hpp>
Operation that returns the shape of its input argument as a tensor.
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
virtual void validate_and_infer_types() override
-
class ShuffleChannels : public ov::op::Op
- #include <shuffle_channels.hpp>
Permutes data in the channel dimension of the input.
Public Functions
-
ShuffleChannels(const Output<Node> &data, const int64_t axis = 1, const int64_t group = 1)
Constructs a ShuffleChannels node.
- Parameters
data – Node producing the input tensor.
axis – Channel dimension index in the data tensor. A negative value means that the index should be calculated from the back of the input data shape.
group – Number of group the channel dimension should be split into.
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
ShuffleChannels(const Output<Node> &data, const int64_t axis = 1, const int64_t group = 1)
-
class Sigmoid : public ov::op::util::UnaryElementwiseArithmetic
- #include <sigmoid.hpp>
Sigmoid operation.
Public Functions
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
virtual bool has_evaluate() const override
-
class Sign : public ov::op::util::UnaryElementwiseArithmetic
- #include <sign.hpp>
Elementwise sign operation.
-
class Sin : public ov::op::util::UnaryElementwiseArithmetic
- #include <sin.hpp>
Elementwise sine operation.
Inputs
Type
Description
arg\(N[d_1,\dots,d_n]~(n \geq 0)\)
A tensor of any shape and numeric element type.
Type
Description
\(N[d_1,\dots,d_n]\)
The tensor \(T\), where \(T[i_1,\dots,i_n] = \sin(\texttt{arg}[i_1,\dots,i_n])\)
-
class Sinh : public ov::op::util::UnaryElementwiseArithmetic
- #include <sinh.hpp>
Elementwise hyperbolic sine (sinh) operation.
-
class SpaceToDepth : public ov::op::Op
- #include <space_to_depth.hpp>
SpaceToDepth permutes input tensor blocks of spatial data into depth dimension.
Note
Values from the height and width dimensions are moved to the depth dimension.
Output node produces a tensor with shape: [N, C * blocksize * blocksize, H / blocksize, W / blocksize]
Public Functions
-
SpaceToDepth(const Output<Node> &data, const SpaceToDepthMode &mode, std::size_t block_size = 1)
Constructs a SpaceToDepth operation.
- Parameters
data – - Node producing the input tensor
mode – Specifies how the output depth dimension is gathered from block coordinates and the old depth dimension.
block_size – - the size of the block of values to be moved
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
SpaceToDepth(const Output<Node> &data, const SpaceToDepthMode &mode, std::size_t block_size = 1)
-
class Sqrt : public ov::op::util::UnaryElementwiseArithmetic
- #include <sqrt.hpp>
Elementwise square root operation.
Inputs
Type
Description
arg\(N[d_1,\dots,d_n]~(n \geq 0)\)
A tensor of any shape and numeric element type.
Type
Description
\(N[d_1,\dots,d_n]\)
The tensor \(T\), where \(T[i_1,\dots,i_n] = \sqrt{\texttt{arg}[i_1,\dots,i_n]}\)
-
class SquaredDifference : public ov::op::util::BinaryElementwiseArithmetic
- #include <squared_difference.hpp>
Calculates an element-wise squared difference between two tensors.
y[i] = (x1[i] - x2[i])^2
Public Functions
-
inline SquaredDifference()
Constrcuts an uninitialized squared difference operation.
-
SquaredDifference(const Output<Node> &x1, const Output<Node> &x2, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec(AutoBroadcastType::NUMPY))
Constructs the squared difference operation.
- Parameters
x1 – First input tensor
x2 – Second input tensor
auto_broadcast – Auto broadcast specification
-
inline SquaredDifference()
-
class Squeeze : public ov::op::Op
- #include <squeeze.hpp>
Squeeze operation.
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
virtual void validate_and_infer_types() override
-
class Tan : public ov::op::util::UnaryElementwiseArithmetic
- #include <tan.hpp>
Elementwise tangent operation.
Inputs
Type
Description
arg\(N[d_1,\dots,d_n]~(n \geq 0)\)
A tensor of any shape and numeric element type.
Type
Description
\(N[d_1,\dots,d_n]\)
The tensor \(T\), where \(T[i_1,\dots,i_n] = \tan(\texttt{arg}[i_1,\dots,i_n])\)
-
class Tanh : public ov::op::util::UnaryElementwiseArithmetic
- #include <tanh.hpp>
Elementwise hyperbolic tangent operation.
-
class TensorIterator : public ov::op::util::SubGraphOp
- #include <tensor_iterator.hpp>
Iterate a body over tensors, accumulating into tensors.
Public Functions
-
inline std::shared_ptr<Model> get_body() const
- Returns
the body of the iteration
- Parameters
body – set the body of the iteration
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
inline std::shared_ptr<Model> get_body() const
-
class Tile : public ov::op::Op
- #include <tile.hpp>
Dynamic Tiling operation which repeats a tensor multiple times along each dimension.
Public Functions
-
Tile(const Output<Node> &data, const Output<Node> &repeats)
Perform dynamic padding of a tensor.
- Parameters
data – The node producing input tensor to be padded.
repeats – The node producing the per-dimension replication factor
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override
Evaluates the op on input_values putting results in output_values.
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
- Returns
true if successful
-
Tile(const Output<Node> &data, const Output<Node> &repeats)
-
class Unsqueeze : public ov::op::Op
- #include <unsqueeze.hpp>
Unsqueeze operation.
Public Functions
-
virtual void validate_and_infer_types() override
Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.
Throws if the node is invalid.
-
virtual bool evaluate(ov::TensorVector &outputs, const ov::TensorVector &inputs) const override
Evaluates the op on input_values putting results in output_values.
- Parameters
output_values – Tensors for the outputs to compute. One for each result
input_values – Tensors for the inputs. One for each inputs.
- Returns
true if successful
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
virtual void validate_and_infer_types() override
-
class Xor : public ov::op::util::BinaryElementwiseLogical
- #include <xor.hpp>
Elementwise logical-xor operation.
Public Functions
-
Xor(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec())
Constructs a logical-xor operation.
Output
[d0, ...]
-
virtual bool has_evaluate() const override
Allows to get information about availability of evaluate method for the current operation.
-
Xor(const Output<Node> &arg0, const Output<Node> &arg1, const AutoBroadcastSpec &auto_broadcast = AutoBroadcastSpec())
-
namespace lstm_cell
Variables
-
constexpr size_t gates_count = 4
-
constexpr size_t num_state_nodes = 2
-
constexpr size_t peepholes_count = 3
-
constexpr size_t gates_count = 4
-
template<class T, class TRShape = result_shape_t<T>>
-
namespace v1
Functions
-
template<class TShape, class TContainer, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const AvgPool *op, const std::vector<TShape> &input_shapes, TContainer &pads_begin, TContainer &pads_end)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const BatchToSpace *op, const std::vector<TShape> &input_shapes, const ITensorAccessor &tensor_accessor = make_tensor_accessor())
-
template<class T, class TRShape = result_shape_t<T>>
std::vector<TRShape> shape_infer(const ov::op::v1::Broadcast *op, const std::vector<T> &input_shapes, const ITensorAccessor &ta = make_tensor_accessor())
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const ConvolutionBackpropData *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end, const ITensorAccessor &ta = make_tensor_accessor())
-
template<class TOp, class TShape, class TRShape = result_shape_t<TShape>, typename std::enable_if<std::is_same<TOp, Convolution>::value || std::is_same<TOp, BinaryConvolution>::value>::type* = nullptr>
std::vector<TRShape> shape_infer(const TOp *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DeformableConvolution *op, const std::vector<TShape> &input_shapes, CoordinateDiff &pads_begin, CoordinateDiff &pads_end)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const DeformablePSROIPooling *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<TRShape> shape_infer(const GatherTree *op, const std::vector<TShape> &input_shapes)
-
template<class TShape, class TRShape = result_shape_t<TShape>>
std::vector<
-
template<class TShape, class TContainer, class TRShape = result_shape_t<TShape>>
-
enum class GeluApproximationMode
-
static const char ORIGINAL_NAMES[] = "originalLayersNames"¶