The Intel nGraph C++ API. More...
Namespaces | |
builder | |
Convenience functions that create addional graph nodes to implement commonly-used recipes, for example auto-broadcast. | |
descriptor | |
Descriptors are compile-time representations of objects that will appear at run-time. | |
onnx_import | |
ONNX importer features namespace. Functions in this namespace make it possible to use ONNX models. | |
op | |
Ops used in graph-building. | |
runtime | |
The objects used for executing the graph. | |
Typedefs | |
using | HostTensorPtr = std::shared_ptr< runtime::HostTensor > |
using | HostTensor = runtime::HostTensor |
using | HostTensorVector = std::vector< HostTensorPtr > |
using | ResultVector = std::vector< std::shared_ptr< op::v0::Result > > |
using | NodeMap = std::unordered_map< ngraph::Node *, std::shared_ptr< ngraph::Node > > |
Alias useful for cloning. | |
using | NodeTypeInfo = Node::type_info_t |
using | RawNodeOutputMap = std::map< RawNodeOutput, Output< Node > > |
using | ParameterVector = std::vector< std::shared_ptr< op::Parameter > > |
using | SinkVector = std::vector< std::shared_ptr< op::Sink > > |
using | NodeVector = std::vector< std::shared_ptr< Node > > |
using | OutputVector = std::vector< Output< Node > > |
using | matcher_pass_callback = std::function< bool(ngraph::pattern::Matcher &m)> |
using | graph_rewrite_callback = std::function< bool(ngraph::pattern::Matcher &m)> |
using | recurrent_graph_rewrite_callback = std::function< bool(ngraph::pattern::RecurrentMatcher &m)> |
using | handler_callback = std::function< bool(const std::shared_ptr< Node > &node)> |
using | Rank = Dimension |
Alias for Dimension, used when the value represents the number of axes in a shape, rather than the size of one dimension in a shape. More... | |
using | VariantTypeInfo = DiscreteTypeInfo |
Enumerations | |
enum class | LOG_TYPE { _LOG_TYPE_ERROR , _LOG_TYPE_WARNING , _LOG_TYPE_INFO , _LOG_TYPE_DEBUG } |
Functions | |
template<typename A , typename B > | |
A | copy_from (B &b) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const AxisSet &axis_set) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const AxisVector &axis_vector) |
static std::ostream & | write_all_to_stream (std::ostream &str) |
template<typename T , typename... TS> | |
static std::ostream & | write_all_to_stream (std::ostream &str, const T &arg, TS &&... args) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const Coordinate &coordinate) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const CoordinateDiff &coordinate_diff) |
NGRAPH_API std::ostream & | operator<< (std::ostream &str, const Dimension &dimension) |
Insert a human-readable representation of a dimension into an output stream. More... | |
template<typename Type , typename Value > | |
std::enable_if< std::is_convertible< Value, std::string >::value, Type >::type | as_enum (const Value &value) |
Returns the enum value matching the string. | |
template<typename Value > | |
const std::string & | as_string (Value value) |
Returns the string matching the enum value. | |
NGRAPH_API std::string | getenv_string (const char *env_var) |
Get the names environment variable as a string. More... | |
NGRAPH_API int32_t | getenv_int (const char *env_var, int32_t default_value=-1) |
Get the names environment variable as an integer. If the value is not a valid integer then an exception is thrown. More... | |
NGRAPH_API bool | getenv_bool (const char *env_var, bool default_value=false) |
Get the names environment variable as a boolean. If the value is not a valid boolean then an exception is thrown. Valid booleans are one of 1, 0, on, off, true, false All values are case insensitive. If the environment variable is not set the default_value is returned. More... | |
NGRAPH_API std::mutex & | get_registry_mutex () |
NGRAPH_API void | traverse_nodes (const std::shared_ptr< const Function > p, std::function< void(std::shared_ptr< Node >)> f) |
NGRAPH_API void | traverse_nodes (const Function *p, std::function< void(std::shared_ptr< Node >)> f) |
NGRAPH_API void | traverse_nodes (const NodeVector &subgraph_results, std::function< void(std::shared_ptr< Node >)> f, const NodeVector &subgraph_params={}) |
Visit each node in a sub-graph of the entire graph. More... | |
NGRAPH_API void | replace_node (std::shared_ptr< Node > target, std::shared_ptr< Node > replacement, const std::vector< int64_t > &output_order) |
Replace the node target with the node replacement , i.e., redirect all users and control dependencies of target to replacement . More... | |
NGRAPH_API void | replace_node (const std::shared_ptr< Node > &target, const OutputVector &replacement_values) |
NGRAPH_API void | replace_node (std::shared_ptr< Node > target, std::shared_ptr< Node > replacement) |
NGRAPH_API void | replace_nodes (const std::shared_ptr< Function > &f, const std::unordered_map< std::shared_ptr< op::v0::Parameter >, std::shared_ptr< op::v0::Parameter >> ¶meter_replacement_map, const std::unordered_map< std::shared_ptr< Node >, std::shared_ptr< Node >> &body_replacement_map) |
Replace multiple nodes in a function. More... | |
NGRAPH_API NodeVector | find_common_args (std::shared_ptr< Node > target, std::shared_ptr< Node > replacement) |
template<typename T > | |
std::vector< std::shared_ptr< Node > > | topological_sort (T root_nodes) |
Topological sort of nodes needed to compute root_nodes. | |
template<typename T > | |
std::vector< std::shared_ptr< Node > > | subgraph_topological_sort (T nodes) |
Topological sort of just nodes. | |
template<typename T > | |
void | validate_nodes_and_infer_types (const T &nodes) |
NGRAPH_API bool | is_post_dominated (Node *X, Node *Y) |
NGRAPH_API bool | is_equal_to_const_value (std::string const_value, const Output< Node > &reduce_constant) |
NGRAPH_API std::vector< std::shared_ptr< ngraph::Node > > | clone_nodes (const std::vector< std::shared_ptr< ngraph::Node >> &nodes, NodeMap &node_map) |
NGRAPH_API std::list< std::shared_ptr< ngraph::Node > > | clone_nodes (const std::vector< std::shared_ptr< ngraph::Node >> &nodes, RawNodeOutputMap &node_map) |
NGRAPH_API std::shared_ptr< ngraph::Function > | clone_function (const ngraph::Function &func, NodeMap &node_map) |
NGRAPH_API std::shared_ptr< ngraph::Function > | clone_function (const ngraph::Function &func) |
NGRAPH_API std::pair< std::shared_ptr< op::Result >, std::shared_ptr< op::v0::Parameter > > | insert_result_parameter_split (const std::shared_ptr< Node > &src_node, const std::shared_ptr< Node > &dst_node) |
NGRAPH_API void | insert_new_node_between (const std::shared_ptr< Node > &src_node, const std::shared_ptr< Node > &dst_node, const std::shared_ptr< Node > &new_node) |
NGRAPH_API std::shared_ptr< Node > | make_zero (const element::Type &element_type, const Shape &shape) |
NGRAPH_API std::shared_ptr< Node > | make_constant_from_string (std::string val, const element::Type &element_type, const Shape &shape) |
NGRAPH_API bool | is_zero (const Output< Node > &reduce_constant) |
NGRAPH_API NodeVector | get_subgraph_outputs (const NodeVector &nodes, const NodeVector &exclusions, bool ignore_unused=false, bool ignore_output_duplicates=true) |
NGRAPH_API NodeVector | extract_subgraph (const NodeVector &results, const NodeVector &args) |
NGRAPH_API bool | is_one (const Output< Node > &reduce_constant) |
NGRAPH_API bool | compare_constants (const std::shared_ptr< Node > &n1, const std::shared_ptr< Node > &n2) |
NGRAPH_API bool | is_used (Node *node) |
NGRAPH_API size_t | get_user_count (Node *node) |
NGRAPH_API bool | possibly_overwritten (Node *node) |
NGRAPH_API bool | is_strided (const Strides &strides) |
NGRAPH_API bool | is_valid_rank (const std::shared_ptr< Node > &node, std::vector< size_t > valid_ranks) |
NGRAPH_API void | plot_graph (std::shared_ptr< Function > f, const std::string &filename, std::function< void(const Node &node, std::vector< std::string > &attributes)>=nullptr) |
NGRAPH_API std::vector< Input< Node > > | get_inputs_from (Node &src, Node &dst) |
NGRAPH_API std::vector< Output< Node > > | get_outputs_to (Node &src, Node &dst) |
NGRAPH_API bool | check_for_cycles (const ngraph::Function *func, ngraph::NodeVector &cycle_nodes, bool &is_bkwd_cycle) |
NGRAPH_API bool | replace_output_update_name (Output< Node > node, const Output< Node > &node_input) |
NGRAPH_API bool | replace_node_update_name (std::shared_ptr< Node > target, std::shared_ptr< Node > replacement) |
NGRAPH_API std::ostream & | operator<< (std::ostream &str, const Interval &interval) |
constexpr const char * | find_last (ConstString s, size_t offset, char ch) |
constexpr const char * | find_last (ConstString s, char ch) |
constexpr const char * | get_file_name (ConstString s) |
constexpr const char * | trim_file_name (ConstString root, ConstString s) |
NGRAPH_API void | default_logger_handler_func (const std::string &s) |
template<typename T > | |
NullLogger && | operator<< (NullLogger &&logger, T &&) |
template<typename T > | |
NullLogger && | operator<< (NullLogger &&logger, const T &) |
NullLogger && | operator<< (NullLogger &&logger, std::basic_ostream< char, std::char_traits< char >> &(&)(std::basic_ostream< char, std::char_traits< char >> &)) |
NGRAPH_API void | get_version (size_t &major, size_t &minor, size_t &patch, std::string &extra) |
Function to query parsed version information of the version of ngraph which contains this function. Version information strictly follows Semantic Versioning http://semver.org. More... | |
NGRAPH_API std::string | node_validation_failure_loc_string (const Node *node) |
const std::shared_ptr< Node > & | check_single_output_arg (const std::shared_ptr< Node > &node, size_t i) |
NGRAPH_API const NodeVector & | check_single_output_args (const NodeVector &args) |
NGRAPH_API OutputVector | as_output_vector (const NodeVector &args) |
NGRAPH_API NodeVector | as_node_vector (const OutputVector &values) |
NGRAPH_API ResultVector | as_result_vector (const OutputVector &values) |
Returns a ResultVector referencing values. | |
NGRAPH_API std::ostream & | operator<< (std::ostream &, const Node &) |
NGRAPH_API std::ostream & | operator<< (std::ostream &, const Node *) |
template<typename T > | |
void | check_new_args_count (const Node *node, T new_args) |
NGRAPH_API std::ostream & | operator<< (std::ostream &out, const Input< Node > &input) |
NGRAPH_API std::ostream & | operator<< (std::ostream &out, const Input< const Node > &input) |
NGRAPH_API std::ostream & | operator<< (std::ostream &out, const Output< Node > &output) |
NGRAPH_API std::ostream & | operator<< (std::ostream &out, const Output< const Node > &output) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v1::BinaryConvolution::BinaryConvolutionMode &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v0::DepthToSpace::DepthToSpaceMode &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v0::Interpolate::InterpolateMode &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v4::Interpolate::InterpolateMode &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v4::Interpolate::CoordinateTransformMode &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v4::Interpolate::NearestMode &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v4::Interpolate::ShapeCalcMode &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::LSTMWeightsFormat &type) |
NGRAPH_API std::shared_ptr< Node > | operator- (const Output< Node > &arg0) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v1::NonMaxSuppression::BoxEncodingType &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v3::NonMaxSuppression::BoxEncodingType &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v5::NonMaxSuppression::BoxEncodingType &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v1::Reverse::Mode &type) |
std::ostream & | operator<< (std::ostream &s, const op::v3::ROIAlign::PoolingMode &mode) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v5::Round::RoundMode &type) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const op::v0::SpaceToDepth::SpaceToDepthMode &type) |
const NGRAPH_API OpSet & | get_opset1 () |
const NGRAPH_API OpSet & | get_opset2 () |
const NGRAPH_API OpSet & | get_opset3 () |
const NGRAPH_API OpSet & | get_opset4 () |
const NGRAPH_API OpSet & | get_opset5 () |
const NGRAPH_API OpSet & | get_opset6 () |
PartialShape | operator+ (const PartialShape &s1, const PartialShape &s2) |
Elementwise addition of two PartialShape objects. More... | |
NGRAPH_API std::ostream & | operator<< (std::ostream &str, const PartialShape &shape) |
Inserts a human-readable representation of a PartialShape into an output stream. More... | |
NGRAPH_API void | set_provenance_enabled (bool enabled) |
NGRAPH_API bool | get_provenance_enabled () |
NGRAPH_API void | copy_runtime_info (std::shared_ptr< ngraph::Node > from, std::shared_ptr< ngraph::Node > to) |
NGRAPH_API void | copy_runtime_info (std::shared_ptr< ngraph::Node > from, ngraph::NodeVector to) |
NGRAPH_API void | copy_runtime_info (const ngraph::NodeVector &from, std::shared_ptr< ngraph::Node > to) |
NGRAPH_API void | copy_runtime_info (const ngraph::NodeVector &from, ngraph::NodeVector to) |
template<typename SHAPE_TYPE > | |
size_t | shape_size (const SHAPE_TYPE &shape) |
Number of elements in spanned by a shape. | |
template<typename SHAPE_TYPE > | |
std::vector< size_t > | row_major_strides (const SHAPE_TYPE &shape) |
Row-major strides for a shape. | |
template<typename SHAPE_TYPE > | |
size_t | row_major_stride (const SHAPE_TYPE &shape, size_t axis) |
template<typename SHAPE_TYPE > | |
bool | is_scalar (const SHAPE_TYPE &shape) |
template<typename SHAPE_TYPE > | |
bool | is_vector (const SHAPE_TYPE &shape) |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const Shape &shape) |
template<typename AXIS_VALUES > | |
AXIS_VALUES | project (const AXIS_VALUES &axis_values, const AxisSet &axes) |
template<> | |
NGRAPH_API PartialShape | project (const PartialShape &shape, const AxisSet &axes) |
template<typename AXIS_VALUES > | |
AXIS_VALUES | reduce (const AXIS_VALUES &axis_values, const AxisSet &deleted_axes, bool keep_dims) |
template<> | |
NGRAPH_API PartialShape | reduce (const PartialShape &shape, const AxisSet &deleted_axes, bool keep_dims) |
template<typename AXIS_VALUES , typename AXIS_VALUE > | |
AXIS_VALUES | inject_pairs (const AXIS_VALUES &axis_values, std::vector< std::pair< size_t, AXIS_VALUE >> new_axis_pos_value_pairs) |
template<> | |
NGRAPH_API PartialShape | inject_pairs (const PartialShape &shape, std::vector< std::pair< size_t, Dimension >> new_axis_pos_value_pairs) |
template<typename AXIS_VALUES , typename AXIS_VALUE > | |
AXIS_VALUES | inject (const AXIS_VALUES &axis_values, size_t new_axis_pos, AXIS_VALUE new_axis_val) |
SlicePlan NGRAPH_API | make_slice_plan (const Shape &input_shape, const std::vector< int64_t > &begins, const std::vector< int64_t > &ends, const std::vector< int64_t > &strides, const AxisSet &lower_bounds_mask, const AxisSet &upper_bounds_mask, const AxisSet &new_axis_mask, const AxisSet &shrink_axis_mask, const AxisSet &ellipsis_mask) |
NGRAPH_API std::shared_ptr< Function > | specialize_function (std::shared_ptr< Function > f, const std::vector< element::Type > ¶meter_element_types, const std::vector< PartialShape > ¶meter_shapes, const std::vector< void * > ¶meter_values) |
Creates a "specialized" clone of a function. The partial shapes and element types of the function's parameters may be narrowed to more specific shapes and element types, and constant values may optionally be substituted for any or all of the parameters. More... | |
NGRAPH_API std::ostream & | operator<< (std::ostream &s, const Strides &strides) |
size_t | compiler_byte_size (element::Type_t et) |
Return the number of bytes in the compile-time representation of the element type. | |
bool ::type | is_type (Value value) |
template<typename Type , typename Value > | |
std::enable_if< std::is_convertible< decltype(static_cast< Type * >std::declval< Value >))), Type * >::value, Type * >::type | as_type (Value value) |
Casts a Value* to a Type* if it is of type Type, nullptr otherwise. | |
template<typename Type , typename Value > | |
std::enable_if< std::is_convertible< decltype(std::static_pointer_cast< Type >std::declval< Value >))), std::shared_ptr< Type > >::value, std::shared_ptr< Type > >::type | as_type_ptr (Value value) |
template<typename T > | |
std::string | join (const T &v, const std::string &sep=", ") |
template<typename T > | |
std::string | vector_to_string (const T &v) |
NGRAPH_API size_t | hash_combine (const std::vector< size_t > &list) |
NGRAPH_API void | dump (std::ostream &out, const void *, size_t) |
NGRAPH_API std::string | to_lower (const std::string &s) |
NGRAPH_API std::string | to_upper (const std::string &s) |
NGRAPH_API std::string | trim (const std::string &s) |
NGRAPH_API std::vector< std::string > | split (const std::string &s, char delimiter, bool trim=false) |
template<typename T > | |
std::string | locale_string (T x) |
template<typename T > | |
T | parse_string (const std::string &s) |
Parses a string containing a literal of the underlying type. | |
template<> | |
NGRAPH_API float | parse_string< float > (const std::string &s) |
template<> | |
NGRAPH_API double | parse_string< double > (const std::string &s) |
template<> | |
NGRAPH_API int8_t | parse_string< int8_t > (const std::string &s) |
template<> | |
NGRAPH_API uint8_t | parse_string< uint8_t > (const std::string &s) |
template<typename T > | |
std::vector< T > | parse_string (const std::vector< std::string > &ss) |
Parses a list of strings containing literals of the underlying type. | |
template<typename T > | |
T | ceil_div (const T &x, const T &y) |
template<typename T > | |
T | subtract_or_zero (T x, T y) |
NGRAPH_API void * | ngraph_malloc (size_t size) |
NGRAPH_API void | ngraph_free (void *) |
NGRAPH_API size_t | round_up (size_t size, size_t alignment) |
bool | is_valid_permutation (ngraph::AxisVector permutation, ngraph::Rank rank=Rank::dynamic()) |
template<typename T > | |
T | apply_permutation (T input, ngraph::AxisVector order) |
template NGRAPH_API AxisVector | apply_permutation< AxisVector > (AxisVector input, AxisVector order) |
template NGRAPH_API Coordinate | apply_permutation< Coordinate > (Coordinate input, AxisVector order) |
template NGRAPH_API Strides | apply_permutation< Strides > (Strides input, AxisVector order) |
template NGRAPH_API Shape | apply_permutation< Shape > (Shape input, AxisVector order) |
template<> | |
NGRAPH_API PartialShape | apply_permutation (PartialShape input, AxisVector order) |
NGRAPH_API AxisVector | get_default_order (size_t rank) |
NGRAPH_API AxisVector | get_default_order (const Shape &shape) |
NGRAPH_API void | parse_version_string (std::string version, size_t &major, size_t &minor, size_t &patch, std::string &extra) |
Function to query parsed version information of the version of ngraph which contains this function. Version information strictly follows Semantic Versioning http://semver.org. More... | |
template<typename T > | |
T | double_to_int (double x, double float_to_int_converter(double)) |
NGRAPH_API Strides | conv_default_strides (const Node *node, const PartialShape &data_batch_shape, const PartialShape &filters_shape) |
NGRAPH_API CoordinateDiff | conv_default_padding (const Node *node, const PartialShape &data_batch_shape, const PartialShape &filters_shape) |
NGRAPH_API PartialShape | infer_windowed_reduction_output_shape (const Node *node, const PartialShape &data_shape, const Strides &data_dilation, const CoordinateDiff &data_padding_below, const CoordinateDiff &data_padding_above, const PartialShape &window_shape, const Strides &window_strides, const Strides &window_dilation, bool is_window_all_in_padding_allowed, bool ceil_mode=false) |
NGRAPH_API PartialShape | infer_convolution_forward (const Node *node, const PartialShape &data_batch_shape, const Strides &data_dilation, const CoordinateDiff &data_padding_below, const CoordinateDiff &data_padding_above, const PartialShape &filters_shape, const Strides &filter_strides, const Strides &filter_dilation) |
NGRAPH_API PartialShape | infer_batched_pooling_forward (const Node *node, const PartialShape &data_batch_shape, const CoordinateDiff &data_padding_below, const CoordinateDiff &data_padding_above, const PartialShape &window_shape, const Strides &window_strides, bool is_window_all_in_padding_allowed, bool ceil_mode=false) |
NGRAPH_API std::tuple< element::Type, PartialShape, PartialShape > | infer_batch_norm_forward (const Node *node, element::Type input_element_type, element::Type gamma_element_type, element::Type beta_element_type, element::Type mean_element_type, element::Type variance_element_type, const PartialShape &input_shape, const PartialShape &gamma_shape, const PartialShape &beta_shape, const PartialShape &mean_shape, const PartialShape &variance_shape) |
NGRAPH_API std::tuple< element::Type, PartialShape, PartialShape > | infer_batch_norm_forward (const Node *node, element::Type input_element_type, element::Type gamma_element_type, element::Type beta_element_type, const PartialShape &input_shape, const PartialShape &gamma_shape, const PartialShape &beta_shape) |
NGRAPH_API bool | try_apply_auto_padding (const PartialShape &image_shape, const Shape &filter_shape, const Strides &filter_strides, const Strides &filter_dilations, const op::PadType pad_type, CoordinateDiff &padding_above, CoordinateDiff &padding_below) |
Apply auto padding to padding_above and padding_below inputs if all needed informations are known. More... | |
NGRAPH_API void | infer_auto_padding (const Shape &image_shape, const Shape &filter_shape, const Strides &filter_strides, const Strides &filter_dilations, const op::PadType pad_type, CoordinateDiff &padding_above, CoordinateDiff &padding_below) |
NGRAPH_API PartialShape | infer_slice_shape (const Node *node, const PartialShape &input_shape, const std::vector< int64_t > &begin, const std::vector< int64_t > &end, const std::vector< int64_t > &strides, const AxisSet &begin_mask, const AxisSet &end_mask, const AxisSet &new_axis_mask, const AxisSet &shrink_axis_mask, const AxisSet &ellipsis_mask) |
NGRAPH_API int64_t | normalize_axis (const Node *node, std::int64_t axis, const Rank &tensor_rank) |
Handle out of range axis. More... | |
NGRAPH_API std::vector< size_t > | normalize_axes (const std::string &node_description, const std::vector< int64_t > &axes, const Rank &tensor_rank) |
Handle out of range axes in vector. More... | |
NGRAPH_API int64_t | normalize_axis (const std::string &node_description, std::int64_t axis, const Rank &tensor_rank) |
Handle out of range axis. More... | |
NGRAPH_API int64_t | normalize_axis (const Node *node, std::int64_t axis, std::uint64_t tensor_rank, std::int64_t axis_range_min, std::int64_t axis_range_max) |
Handle out of range axis. More... | |
NGRAPH_API int64_t | normalize_axis (const std::string &node_description, std::int64_t axis, std::uint64_t tensor_rank, std::int64_t axis_range_min, std::int64_t axis_range_max) |
Handle out of range axis. More... | |
NGRAPH_API std::pair< bool, uint64_t > | maximum_value (const Output< Node > &value) |
Try to compute the maximum value of value. More... | |
NGRAPH_API void | evaluate_nodes (std::map< RawNodeOutput, HostTensorPtr > &value_map, std::map< RawNodeOutput, HostTensorPtr > &output_tensor_map, const OutputVector &outputs) |
Evaluates outputs, treating values in value_map as already computed. value_map is updated. More... | |
NGRAPH_API HostTensorPtr | evaluate_lower_bound (const Output< Node > &output) |
Evaluates lower value estimation of the output tensor. Traverses graph up to deduce estimation through it. More... | |
NGRAPH_API HostTensorPtr | evaluate_upper_bound (const Output< Node > &output) |
Evaluates lower value estimation of the output tensor. Traverses graph up to deduce estimation through it. More... | |
NGRAPH_API std::pair< HostTensorPtr, HostTensorPtr > | evaluate_both_bounds (const Output< Node > &output) |
Evaluates lower and upper value estimations of the output tensor. Traverses graph up to deduce estimation through it. More... | |
NGRAPH_API bool | evaluate_as_partial_shape (const Output< Node > &output, PartialShape &pshape) |
Evaluates lower and upper value estimations for the output tensor. Estimation would be represented as partial shape object using Dimension(min, max) for each element. More... | |
NGRAPH_API bool | default_upper_bound_evaluator (const Node *node, const HostTensorVector &output_values) |
Estimates upper bound for node output tensors using only upper bounds of the nodes inputs. More... | |
NGRAPH_API bool | default_lower_bound_evaluator (const Node *node, const HostTensorVector &output_values) |
Estimates lower bound for node output tensors using only lower bounds of the nodes inputs. More... | |
NGRAPH_API bool | interval_bound_evaluator (const Node *node, const HostTensorVector &lower_output_values, const HostTensorVector &upper_output_values) |
Estimates both bounds for node output tensors using both bounds of inputs. Works for operations with two inputs (in_1 and in_2). Brute forces all the pairs of bounds for inputs and evaluates all of them: {in_1_lower, in_2 lower}, {in_1_lower, in_2 upper}, {in_1_upper, in_2_lower}, {in_1_upper, in_2_upper}. Lower and upper values are selected from all the outputs calculated using input pairs. More... | |
NGRAPH_API bool | host_tensor_is_positive (const HostTensorPtr &bound) |
Checks if all the elements of the bound HostTensor are positive. | |
NGRAPH_API bool | has_and_set_equal_bounds (const Output< Node > &source) |
Checks if lower and upper bounds of the corresponding tensor are set (not nullptr) and pointers are the same. It doesn't check if lower and upper values are the same relying only on pointers comparison. | |
NGRAPH_API std::shared_ptr< op::Constant > | get_constant_from_source (const Output< Node > &source) |
Runs an estimation of source tensor. If it succeeded to calculate both bounds and they are the same returns Constant operation from the resulting bound, otherwise nullptr. | |
NGRAPH_API std::shared_ptr< op::Constant > | get_constant_max_of_type (element::Type_t t) |
Returns a Constant storing scalar value equal to std::numeric_limits<t>::max() | |
NGRAPH_API std::shared_ptr< op::Constant > | get_constant_min_of_type (element::Type_t t) |
Returns a Constant storing scalar value equal to std::numeric_limits<t>::min() | |
NGRAPH_API bool | validate_host_tensor_vector (const HostTensorVector &v, const size_t &size) |
Checks if size of HostTensorVector is the same as passed size attribute. Then checks that all the HostTensorPtrs are not equal to nullptr. | |
Variables | |
template class NGRAPH_API | VariantImpl< int64_t > |
The Intel nGraph C++ API.
Top level nGraph namespace.
using ngraph::Rank = typedef Dimension |
Alias for Dimension, used when the value represents the number of axes in a shape, rather than the size of one dimension in a shape.
XXX: THIS TYPE IS EXPERIMENTAL AND THE ENTIRE DESIGN IS SUBJECT TO CHANGE.
std::enable_if< std::is_convertible<decltype(std::static_pointer_cast<Type>std::declval<Value>))), std::shared_ptr<Type> >::value, std::shared_ptr<Type> >::type ngraph::as_type_ptr | ( | Value | value | ) |
Casts a std::shared_ptr<Value> to a std::shared_ptr<Type> if it is of type Type, nullptr otherwise
NGRAPH_API bool ngraph::check_for_cycles | ( | const ngraph::Function * | func, |
ngraph::NodeVector & | cycle_nodes, | ||
bool & | is_bkwd_cycle | ||
) |
Checks the func for graph cycles starting from results going backwards, then from parameters going forward. It returns true if a cycle is found and the first cycle encountered.
NGRAPH_API bool ngraph::default_lower_bound_evaluator | ( | const Node * | node, |
const HostTensorVector & | output_values | ||
) |
Estimates lower bound for node output tensors using only lower bounds of the nodes inputs.
node | Operation to be performed |
output_values | Vector of HostTensorPtrs representing resulting lower value estimations |
NGRAPH_API bool ngraph::default_upper_bound_evaluator | ( | const Node * | node, |
const HostTensorVector & | output_values | ||
) |
Estimates upper bound for node output tensors using only upper bounds of the nodes inputs.
node | Operation to be performed |
output_values | Vector of HostTensorPtrs representing resulting upper value estimations |
NGRAPH_API bool ngraph::evaluate_as_partial_shape | ( | const Output< Node > & | output, |
PartialShape & | pshape | ||
) |
Evaluates lower and upper value estimations for the output tensor. Estimation would be represented as partial shape object using Dimension(min, max) for each element.
output | Node output pointing to the tensor for estimation. |
pshape | Resulting estimation would be stored in this PartialShape. |
NGRAPH_API std::pair<HostTensorPtr, HostTensorPtr> ngraph::evaluate_both_bounds | ( | const Output< Node > & | output | ) |
Evaluates lower and upper value estimations of the output tensor. Traverses graph up to deduce estimation through it.
output | Node output pointing to the tensor for estimation. |
Evaluates lower value estimation of the output tensor. Traverses graph up to deduce estimation through it.
Node | output pointing to the tensor for estimation. |
NGRAPH_API void ngraph::evaluate_nodes | ( | std::map< RawNodeOutput, HostTensorPtr > & | value_map, |
std::map< RawNodeOutput, HostTensorPtr > & | output_tensor_map, | ||
const OutputVector & | outputs | ||
) |
Evaluates outputs, treating values in value_map as already computed. value_map is updated.
value_map | Key is RawNodeOutput in graph, value is the computed value. Updated by the function. |
output_tensor_map | Tensors to use for particular outputs |
outputs | Root set of values to try to compute |
Evaluates lower value estimation of the output tensor. Traverses graph up to deduce estimation through it.
output | Tensor to be estimated. |
src
. dst
. NGRAPH_API void ngraph::get_version | ( | size_t & | major, |
size_t & | minor, | ||
size_t & | patch, | ||
std::string & | extra | ||
) |
Function to query parsed version information of the version of ngraph which contains this function. Version information strictly follows Semantic Versioning http://semver.org.
major | Returns the major part of the version |
minor | Returns the minor part of the version |
patch | Returns the patch part of the version |
extra | Returns the extra part of the version. This includes everything following the patch version number. |
NGRAPH_API bool ngraph::getenv_bool | ( | const char * | env_var, |
bool | default_value = false |
||
) |
Get the names environment variable as a boolean. If the value is not a valid boolean then an exception is thrown. Valid booleans are one of 1, 0, on, off, true, false All values are case insensitive. If the environment variable is not set the default_value is returned.
env_var | The string name of the environment variable to get. |
default_value | The value to return if the environment variable is not set. |
NGRAPH_API int32_t ngraph::getenv_int | ( | const char * | env_var, |
int32_t | default_value = -1 |
||
) |
Get the names environment variable as an integer. If the value is not a valid integer then an exception is thrown.
env_var | The string name of the environment variable to get. |
default_value | The value to return if the environment variable is not set. |
NGRAPH_API std::string ngraph::getenv_string | ( | const char * | env_var | ) |
Get the names environment variable as a string.
env_var | The string name of the environment variable to get. |
NGRAPH_API bool ngraph::interval_bound_evaluator | ( | const Node * | node, |
const HostTensorVector & | lower_output_values, | ||
const HostTensorVector & | upper_output_values | ||
) |
Estimates both bounds for node output tensors using both bounds of inputs. Works for operations with two inputs (in_1 and in_2). Brute forces all the pairs of bounds for inputs and evaluates all of them: {in_1_lower, in_2 lower}, {in_1_lower, in_2 upper}, {in_1_upper, in_2_lower}, {in_1_upper, in_2_upper}. Lower and upper values are selected from all the outputs calculated using input pairs.
node | Operation to be performed |
output_values | Vector of HostTensorPtrs representing resulting lower value estimations |
Try to compute the maximum value of value.
NGRAPH_API std::vector<size_t> ngraph::normalize_axes | ( | const std::string & | node_description, |
const std::vector< int64_t > & | axes, | ||
const Rank & | tensor_rank | ||
) |
Handle out of range axes in vector.
[in] | node_description | The name of node with requested axes. |
[in] | axes | The requested vector of axes. |
[in] | tensor_rank | The corresponding tensor rank. |
NGRAPH_API int64_t ngraph::normalize_axis | ( | const Node * | node, |
std::int64_t | axis, | ||
const Rank & | tensor_rank | ||
) |
Handle out of range axis.
[in] | node | The node with requested axis. |
[in] | axis | The requested axis value. |
[in] | tensor_rank | The corresponding tensor rank. |
NGRAPH_API int64_t ngraph::normalize_axis | ( | const Node * | node, |
std::int64_t | axis, | ||
std::uint64_t | tensor_rank, | ||
std::int64_t | axis_range_min, | ||
std::int64_t | axis_range_max | ||
) |
Handle out of range axis.
[in] | node | The node with requested axis. |
[in] | axis | The requested axis value. |
[in] | tensor_rank | The corresponding tensor rank. |
[in] | axis_range_min | The min value of accepted range for axis. |
[in] | axis_range_max | The max value of accepted range for axis. |
NGRAPH_API int64_t ngraph::normalize_axis | ( | const std::string & | node_description, |
std::int64_t | axis, | ||
const Rank & | tensor_rank | ||
) |
Handle out of range axis.
[in] | node_description | The node with requested axis. |
[in] | axis | The requested axis value. |
[in] | tensor_rank | The corresponding tensor rank. |
NGRAPH_API int64_t ngraph::normalize_axis | ( | const std::string & | node_description, |
std::int64_t | axis, | ||
std::uint64_t | tensor_rank, | ||
std::int64_t | axis_range_min, | ||
std::int64_t | axis_range_max | ||
) |
Handle out of range axis.
[in] | node_description | The name of node with requested axis. |
[in] | axis | The requested axis value. |
[in] | tensor_rank | The corresponding tensor rank. |
[in] | axis_range_min | The min value of accepted range for axis. |
[in] | axis_range_max | The max value of accepted range for axis. |
PartialShape ngraph::operator+ | ( | const PartialShape & | s1, |
const PartialShape & | s2 | ||
) |
Elementwise addition of two PartialShape objects.
s1 | Left operand for addition. |
s2 | Right operand for addition. |
s1
to s2
(see description). std::invalid_argument | If s1 and s2 have inconsistent ranks. |
s1
or s2
has dynamic rank, returns PartialShape::dynamic(). s1 and
s2` both have static rank, and their ranks are unequal, throws std::invalid_argument. s1
and s2
both have static rank, and their ranks are equal, returns a new shape whose i
th dimension is s1[i] + s2[i]
. NGRAPH_API std::ostream& ngraph::operator<< | ( | std::ostream & | str, |
const Dimension & | dimension | ||
) |
Insert a human-readable representation of a dimension into an output stream.
str | The output stream targeted for insertion. |
dimension | The dimension to be inserted into str . |
str
after insertion.Inserts the string ?
if dimension
is dynamic; else inserts dimension.get_length()
.
NGRAPH_API std::ostream& ngraph::operator<< | ( | std::ostream & | str, |
const PartialShape & | shape | ||
) |
Inserts a human-readable representation of a PartialShape into an output stream.
str | The output stream targeted for insertion. |
shape | The shape to be inserted into str . |
str
after insertion.The output to the stream is in "informal" notation. In other words:
shape
has dynamic rank, inserts the string ?
. shape
has static rank, inserts the string {
, then inserts each dimension of shape
into the output stream separated by commas, then inserts }
.Example:
NGRAPH_API float ngraph::parse_string< float > | ( | const std::string & | s | ) |
template specializations for float and double to handle INFINITY, -INFINITY and NaN values.
NGRAPH_API int8_t ngraph::parse_string< int8_t > | ( | const std::string & | s | ) |
template specializations for int8_t and uint8_t to handle the fact that default implementation ends up treating values as characters so that the number "0" turns into the parsed value 48, which is it's ASCII value
NGRAPH_API void ngraph::parse_version_string | ( | std::string | version, |
size_t & | major, | ||
size_t & | minor, | ||
size_t & | patch, | ||
std::string & | extra | ||
) |
Function to query parsed version information of the version of ngraph which contains this function. Version information strictly follows Semantic Versioning http://semver.org.
version | The major part of the version |
major | Returns the major part of the version |
minor | Returns the minor part of the version |
patch | Returns the patch part of the version |
extra | Returns the extra part of the version. This includes everything following the patch version number. |
NGRAPH_API void ngraph::replace_node | ( | const std::shared_ptr< Node > & | target, |
const OutputVector & | replacement_values | ||
) |
Replace target.outputs[i] with replacement_values[i] and transfer control dependents and provenance from target to the node(s) in replacement_values.
NGRAPH_API void ngraph::replace_node | ( | std::shared_ptr< Node > | target, |
std::shared_ptr< Node > | replacement, | ||
const std::vector< int64_t > & | output_order | ||
) |
Replace the node target
with the node replacement
, i.e., redirect all users and control dependencies of target
to replacement
.
target | Node to be replaced. |
replacement | Node to replace target with. |
output_order | Vector determines order of replacement node's outputs. |
This is primarily used in graph-rewriting passes. For example, we might "fuse" two Concat operations as follows:
(Step 0: Original graph)
A B | | v v N0[Concat, concatenation_axis=3] C | | v v N1[Concat, concatenation_axis=3] | | v v some_user another_user
(Step 1: Construct replacement)
shared_ptr<Node> new_N1 = make_shared<op::Concat>({A,B,C},3);
A-------------------------------------—. | | | B-------------—)–. | | | | v v | | N0[Concat, concatenation_axis=3] C--—)–)–. | | | | | v v v v v N1[Concat, concatenation_axis=3] new_N1[Concat, concatenation_axis=3] | | v v some_user another_user
(Step 2: Replace N1 with new_N1)
replace_node(N1, new_N1);
A-------------------------------------—. | | | B-------------—)–. | | | | v v | | N0[Concat, concatenation_axis=3] C--—)–)–. | | | | | v v v v v N1[Concat, concatenation_axis=3] new_N1[Concat, concatenation_axis=3] | | v v some_user another_user
(Step 3: N0 and N1 are now dead, nodes will be freed)
[happens automatically, once all shared_ptrs to N1 are released]
A-------------------------------------—. | B-------------—)–. | | | | C--—)–)–. | | | v v v new_N1[Concat, concatenation_axis=3] | | v v some_user another_user
NOTE 1: replace_node is not type-safe (the graph is not revalidated). For example, the following is allowed, even if node some_user
requires an input of shape 2x2:
(Before) A(shape=2x2) B(shape=3x3) | v some_user(requires 2x2 input)
(After – graph is now invalid)
replace_node(A, B); A(shape=2x2) B(shape=3x3) | v some_user(requires 2x2 input)
NOTE 2: it is possible to insert a cycle into the graph with replace_node, resulting in an invalid graph. Care must be taken to avoid this. One common example is when you are attempting to insert a new node M
"after"a node
N`. For example, you might expect this to work:
shared_ptr<Node> M = make_shared<SomeUnaryOp>(N); replace_node(M, N);
The problem is that at replacement time, N itself is a user of M. So we end up introducing a cycle as follows:
N | v
other users...
||| vvv N------------>M | v
other users...
||| vvv .----. | | | | N `----->M | v other users...
To avoid the cycle, a valid way to perform the above desired insertion would be,
auto new_N = N->clone_with_new_inputs(N->input_values()); shared_ptr<Node> M = make_shared<SomeUnaryOp>(new_N); replace_node(N, M);
NGRAPH_API void ngraph::replace_nodes | ( | const std::shared_ptr< Function > & | f, |
const std::unordered_map< std::shared_ptr< op::v0::Parameter >, std::shared_ptr< op::v0::Parameter >> & | parameter_replacement_map, | ||
const std::unordered_map< std::shared_ptr< Node >, std::shared_ptr< Node >> & | body_replacement_map | ||
) |
Replace multiple nodes in a function.
f | Function where replacement is taking place. |
parameter_replacement_map | A mapping from parameter shared pointers to parameter shared pointers. For each pair (k,v) in the map, parameter k is replaced by parameter v, except if k==v or k is not a parameter bound by f, in which case the pair (k,v) is ignored. |
body_replacement_map | A mapping from node shared pointers to node shared pointers. For each pair (k,v) in the map, node k is replaced by node v, except if k==v, the pair (k,v) is ignored. Note that if k is a parameter, its users will be redirected to v, but k will not be replaced in the function's parameter list. |
Limitations:
parameter_replacement_map
are actually among the bound parameters of f
. (If a parameter appears in the map that is not bound by f
, it will be silently ignored.)parameter_replacement_map
and in body_replacement_map
, behavior is unspecified. NGRAPH_API std::shared_ptr<Function> ngraph::specialize_function | ( | std::shared_ptr< Function > | f, |
const std::vector< element::Type > & | parameter_element_types, | ||
const std::vector< PartialShape > & | parameter_shapes, | ||
const std::vector< void * > & | parameter_values | ||
) |
Creates a "specialized" clone of a function. The partial shapes and element types of the function's parameters may be narrowed to more specific shapes and element types, and constant values may optionally be substituted for any or all of the parameters.
f | The function to be cloned. |
parameter_element_types | The new parameter element types to substitute. Length must be equal to the number of parameters of f. |
parameter_shapes | The new parameter shapes to substitute. Length must be equal to the number of parameters of f. |
parameter_values | Parameter values to substitute. Length must be equal to the number of parameters of f, with nullptr indicating that no substitution is to be made for the corresponding parameter. |
CheckFailure | if parameter_element_types, parameter_shapes is not valid (see details). |
NodeValidationError | if node validation fails as the clone is being constructed. |
Creates a "specialized" clone of an nGraph Function.
For example, suppose that a function f has three parameters with partial shapes:
Shape specialization would allow us to create a clone of f where the shapes are (for example):
But not (for example):
Note that validation errors can potentially occur during cloning. For example:
If we specialize n to the shape {1,2,3}
and m to the shape {4,5,6}
, cloning will fail because when we reconstruct the new x node, it will see that the shapes are inconsistent for elementwise add.
Specialization of element types is also possible: element::dynamic
can be specialized to a concrete element type or left dynamic; but a concrete element type can only be specialized to itself (e.g., specialization does not allow you to change element::i32
to element::i64
).
Finally, it is possible to specialize parameter values. If the ith element of parameter_values
is not nullptr
, and fully static element type and shape has been specified for the ith parameter, a Constant
node will be created and substituted for the ith parameter, with its data drawn from parameter_values[i]
. Note that the Parameter node remains (in order to maintain the arity of the function), but will no longer have any users.
It is required that:
TODO(amprocte): convert this to a pass.
NGRAPH_API void ngraph::traverse_nodes | ( | const NodeVector & | subgraph_results, |
std::function< void(std::shared_ptr< Node >)> | f, | ||
const NodeVector & | subgraph_params = {} |
||
) |
Visit each node in a sub-graph of the entire graph.
subgraph_results | The output nodes of the sub-graph |
f | Function to execute at each node in the traversal |
subgraph_params | Input nodes of the sub-graph (optional) |
Traverses a sub-graph starting from subgraph_results moving up towards parameter nodes. Traversal stops if it hits a node in subgraph_params.
Most useful for finding parameters of a graph directly from the result nodes and not from function parameters or extracting a subgraph relevant to the computation of certain outputs
NGRAPH_API bool ngraph::try_apply_auto_padding | ( | const PartialShape & | image_shape, |
const Shape & | filter_shape, | ||
const Strides & | filter_strides, | ||
const Strides & | filter_dilations, | ||
const op::PadType | pad_type, | ||
CoordinateDiff & | padding_above, | ||
CoordinateDiff & | padding_below | ||
) |
Apply auto padding to padding_above and padding_below inputs if all needed informations are known.
image_shape | The shape of input image. |
filter_shape | The shape of filter input. |
filter_strides | The strides of applied padding. |
filter_dilations | The dilations of applied padding. |
pad_type | The type of padding. Auto padding is applied only for SAME_UPPER and SAME_LOWER mode. |
padding_above | The beginning of padding shape. |
end | The beginning of padding shape. |