Namespaces | Data Structures | Typedefs | Enumerations | Functions | Variables
InferenceEngine Namespace Reference

Inference Engine API. More...

Namespaces

 Builder
 Neural network builder API.
 
 CLDNNConfigParams
 GPU plugin configuration.
 
 GNAConfigParams
 GNA plugin configuration.
 
 HeteroConfigParams
 Heterogeneous plugin configuration.
 
 Metrics
 Metrics
 
 MultiDeviceConfigParams
 Multi Device plugin configuration.
 
 PluginConfigParams
 Generic plugin configuration.
 
 VPUConfigParams
 VPU plugin configuration.
 

Data Structures

class  Blob
 This class represents a universal container in the Inference Engine. More...
 
class  BlockingDesc
 This class describes blocking layouts. More...
 
class  CNNNetReader
 This is a wrapper class used to build and parse a network from the given IR. More...
 
class  CNNNetwork
 This class contains all the information about the Neural Network and the related binary information. More...
 
class  CompoundBlob
 This class represents a blob that contains other blobs. More...
 
class  Connection
 This class is the main object to describe the Inference Engine connection. More...
 
class  Context
 This class implements object. More...
 
class  Core
 This class represents Inference Engine Core entity. More...
 
class  Data
 This class represents the main Data representation node. More...
 
struct  DataConfig
 This structure describes data configuration. More...
 
class  ExecutableNetwork
 wrapper over IExecutableNetwork More...
 
class  Extension
 This class is a C++ helper to work with objects created using extensions. More...
 
class  GeneralError
 This class represents StatusCode::GENERIC_ERROR exception. More...
 
class  I420Blob
 Represents a blob that contains three planes (Y,U and V) in I420 color format. More...
 
interface  IAllocator
 Allocator concept to be used for memory management and is used as part of the Blob. More...
 
class  ICNNNetReader
 This class is the main interface to build and parse a network from a given IR. More...
 
interface  ICNNNetwork
 This is the main interface to describe the NN topology. More...
 
class  ICNNNetworkStats
 This is the interface to describe the NN topology scoring statistics. More...
 
class  IErrorListener
 This class represents a custom error listener. More...
 
class  IExecutableNetwork
 This is an interface of an executable network. More...
 
class  IExtension
 This class is the main extension interface. More...
 
class  IInferencePlugin
 This class is a main plugin interface. More...
 
class  IInferRequest
 This is an interface of asynchronous infer request. More...
 
class  ILayer
 This class is the main interface to describe the Inference Engine layer. More...
 
interface  ILayerExecImpl
 This class provides interface for the implementation with the custom execution code. More...
 
interface  ILayerImpl
 This class provides interface for extension implementations. More...
 
interface  ILayerImplFactory
 This class provides interface for extension factories. More...
 
interface  IMemoryState
 manages data for reset operations More...
 
class  INetwork
 This class is the main interface to describe the Inference Engine network. More...
 
class  INetwotkIterator
 
struct  InferenceEngineProfileInfo
 Represents basic inference profiling information per layer. More...
 
class  InferencePlugin
 This class is a C++ API wrapper for IInferencePlugin. More...
 
class  InferNotStarted
 This class represents StatusCode::INFER_NOT_STARTED exception. More...
 
class  InferRequest
 
class  InputInfo
 This class contains information about each input of the network. More...
 
class  IShapeInferExtension
 This class is the reader extension interface to provide implementation for shape propagation. More...
 
class  IShapeInferImpl
 This class provides interface for the implementation with the custom execution code. More...
 
struct  LayerConfig
 This structure describes Layer configuration. More...
 
class  LockedMemory
 This class represents locked memory for read/write memory. More...
 
class  LockedMemory< const T >
 This class is for read-only segments. More...
 
class  LockedMemory< void >
 This class is for <void*> data and allows casting to any pointers. More...
 
class  MemoryBlob
 This class implements a container object that represents a tensor in memory (host and remote/accelerated) More...
 
class  MemoryState
 C++ exception based error reporting wrapper of API class IMemoryState. More...
 
class  NetworkNodeStats
 This class implements a container which stores statistics for a layer. More...
 
class  NetworkNotLoaded
 This class represents StatusCode::NETWORK_NOT_LOADED exception. More...
 
class  NotAllocated
 This class represents StatusCode::NOT_ALLOCATED exception. More...
 
class  NotFound
 This class represents StatusCode::NOT_FOUND exception. More...
 
class  NotImplemented
 This class represents StatusCode::NOT_IMPLEMENTED exception. More...
 
class  NV12Blob
 Represents a blob that contains two planes (Y and UV) in NV12 color format. More...
 
class  OutOfBounds
 This class represents StatusCode::OUT_OF_BOUNDS exception. More...
 
class  Parameter
 This class represents an object to work with different parameters. More...
 
class  ParameterMismatch
 This class represents StatusCode::PARAMETER_MISMATCH exception. More...
 
class  PluginDispatcher
 This is a class to load a suitable plugin. More...
 
class  Port
 This class is the main object to describe the Inference Engine port. More...
 
class  PortData
 
class  PortInfo
 This class contains a pair from layerId and port index. More...
 
class  Precision
 This class holds precision value and provides precision related operations. More...
 
struct  PrecisionTrait
 Particular precision traits. More...
 
struct  PreProcessChannel
 This structure stores info about pre-processing of network inputs (scale, mean image, ...) More...
 
class  PreProcessInfo
 This class stores pre-process information for the input. More...
 
struct  PrimitiveInfo
 Structure with information about Primitive. More...
 
class  PropertyVector
 
struct  QueryNetworkResult
 Responce structure encapsulating information about supported layer. More...
 
class  RemoteBlob
 This class represents an Inference Engine abstraction to the memory allocated on the remote (non-CPU) accelerator device. More...
 
class  RemoteContext
 This class represents an Inference Engine abstraction for remote (non-CPU) accelerator device-specific execution context. Such context represents a scope on the device within which executable networks and remote memory blobs can exist, function and exchange data. More...
 
class  RequestBusy
 This class represents StatusCode::REQUEST_BUSY exception. More...
 
struct  ResponseDesc
 Represents detailed information for an error. More...
 
class  ResultNotReady
 This class represents StatusCode::RESULT_NOT_READY exception. More...
 
struct  ROI
 This structure describes ROI data. More...
 
class  ShapeInferExtension
 This class is a C++ helper to work with objects created using extensions. More...
 
class  TBlob
 Represents real host memory allocated for a Tensor/Blob per C type. More...
 
class  TensorDesc
 This class defines Tensor description. More...
 
struct  TensorInfo
 This structure describes tensor information. More...
 
class  Unexpected
 This class represents StatusCode::UNEXPECTED exception. More...
 
union  UserValue
 The method holds the user values to enable binding of data per graph node. More...
 
struct  Version
 Represents version information that describes plugins and the inference engine runtime library. More...
 

Typedefs

using gpu_handle_param = void *
 Shortcut for defining a handle parameter.
 
using BlobMap = std::map< std::string, Blob::Ptr >
 This is a convenient type for working with a map containing pairs(string, pointer to a Blob instance).
 
using SizeVector = std::vector< size_t >
 Represents tensor size. More...
 
using CNNLayerPtr = std::shared_ptr< CNNLayer >
 A smart pointer to the CNNLayer.
 
using CNNLayerWeakPtr = std::weak_ptr< CNNLayer >
 A smart weak pointer to the CNNLayer.
 
using DataPtr = std::shared_ptr< Data >
 Smart pointer to Data.
 
using CDataPtr = std::shared_ptr< const Data >
 Smart pointer to constant Data.
 
using DataWeakPtr = std::weak_ptr< Data >
 Smart weak pointer to Data.
 
using OutputsDataMap = std::map< std::string, DataPtr >
 A collection that contains string as key, and Data smart pointer as value.
 
using NetworkNodeStatsPtr = std::shared_ptr< NetworkNodeStats >
 A shared pointer to the NetworkNodeStats object.
 
using NetworkNodeStatsWeakPtr = std::weak_ptr< NetworkNodeStats >
 A smart pointer to the NetworkNodeStats object.
 
using NetworkStatsMap = std::map< std::string, NetworkNodeStatsPtr >
 A map of pairs: name of a layer and related statistics.
 
using ConstOutputsDataMap = std::map< std::string, CDataPtr >
 A collection that contains string as key, and const Data smart pointer as value.
 
using IExtensionPtr = std::shared_ptr< IExtension >
 A shared pointer to a IExtension interface.
 
using IShapeInferExtensionPtr = std::shared_ptr< IShapeInferExtension >
 A shared pointer to a IShapeInferExtension interface. More...
 
using InputsDataMap = std::map< std::string, InputInfo::Ptr >
 A collection that contains string as key, and InputInfo smart pointer as value.
 
using ConstInputsDataMap = std::map< std::string, InputInfo::CPtr >
 A collection that contains string as key, and const InputInfo smart pointer as value.
 
using GenericLayer = class CNNLayer
 Alias for CNNLayer object.
 
using idx_t = size_t
 A type of network objects indexes. More...
 
using InferenceEnginePluginPtr = InferenceEngine::details::SOPointer< IInferencePlugin >
 A C++ helper to work with objects created by the plugin. More...
 
using ParamMap = std::map< std::string, Parameter >
 An std::map object containing low-level object parameters of classes that are derived from RemoteBlob or RemoteContext.
 

Enumerations

enum  LockOp { LOCK_FOR_READ = 0, LOCK_FOR_WRITE }
 Allocator handle mapping type. More...
 
enum  Layout : uint8_t {
  ANY = 0, NCHW = 1, NHWC = 2, NCDHW = 3,
  NDHWC = 4, OIHW = 64, GOIHW = 65, OIDHW = 66,
  GOIDHW = 67, SCALAR = 95, C = 96, CHW = 128,
  HW = 192, NC = 193, CN = 194, BLOCKED = 200
}
 Layouts that the inference engine supports. More...
 
enum  ColorFormat : uint32_t {
  RAW = 0u, RGB, BGR, RGBX,
  BGRX, NV12, I420
}
 Extra information about input color format for preprocessing. More...
 
enum  StatusCode : int {
  OK = 0, GENERAL_ERROR = -1, NOT_IMPLEMENTED = -2, NETWORK_NOT_LOADED = -3,
  PARAMETER_MISMATCH = -4, NOT_FOUND = -5, OUT_OF_BOUNDS = -6, UNEXPECTED = -7,
  REQUEST_BUSY = -8, RESULT_NOT_READY = -9, NOT_ALLOCATED = -10, INFER_NOT_STARTED = -11,
  NETWORK_NOT_READ = -12
}
 This enum contains codes for all possible return values of the interface functions.
 
enum  eDIMS_AXIS : uint8_t { X_AXIS = 0, Y_AXIS, Z_AXIS }
 
enum  MeanVariant { MEAN_IMAGE, MEAN_VALUE, NONE }
 Defines available types of mean. More...
 
enum  ResizeAlgorithm { NO_RESIZE = 0, RESIZE_BILINEAR, RESIZE_AREA }
 Represents the list of supported resize algorithms.
 

Functions

template<class T >
std::shared_ptr< T > make_so_pointer (const file_name_t &name)=delete
 Creates a special shared_pointer wrapper for the given type from a specific shared module. More...
 
InferenceEngine::IAllocatorCreateDefaultAllocator () noexcept
 Creates the default implementation of the Inference Engine allocator per plugin. More...
 
template<typename T , typename std::enable_if<!std::is_pointer< T >::value &&!std::is_reference< T >::value, int >::type = 0, typename std::enable_if< std::is_base_of< Blob, T >::value, int >::type = 0>
std::shared_ptr< T > as (const Blob::Ptr &blob) noexcept
 Helper cast function to work with shared Blob objects. More...
 
template<typename T , typename std::enable_if<!std::is_pointer< T >::value &&!std::is_reference< T >::value, int >::type = 0, typename std::enable_if< std::is_base_of< Blob, T >::value, int >::type = 0>
std::shared_ptr< const T > as (const Blob::CPtr &blob) noexcept
 Helper cast function to work with shared Blob objects. More...
 
template<typename Type >
InferenceEngine::TBlob< Type >::Ptr make_shared_blob (const TensorDesc &tensorDesc)
 Creates a blob with the given tensor descriptor. More...
 
template<typename Type >
InferenceEngine::TBlob< Type >::Ptr make_shared_blob (const TensorDesc &tensorDesc, Type *ptr, size_t size=0)
 Creates a blob with the given tensor descriptor from the pointer to the pre-allocated memory. More...
 
template<typename Type >
InferenceEngine::TBlob< Type >::Ptr make_shared_blob (const TensorDesc &tensorDesc, const std::shared_ptr< InferenceEngine::IAllocator > &alloc)
 Creates a blob with the given tensor descriptor and allocator. More...
 
template<typename TypeTo >
InferenceEngine::TBlob< TypeTo >::Ptr make_shared_blob (const TBlob< TypeTo > &arg)
 Creates a copy of given TBlob instance. More...
 
template<typename T , typename... Args, typename std::enable_if< std::is_base_of< Blob, T >::value, int >::type = 0>
std::shared_ptr< T > make_shared_blob (Args &&... args)
 Creates a Blob object of the specified type. More...
 
Blob::Ptr make_shared_blob (const Blob::Ptr &inputBlob, const ROI &roi)
 Creates a blob describing given ROI object based on the given blob with pre-allocated memory. More...
 
std::ostream & operator<< (std::ostream &out, const Layout &p)
 
std::ostream & operator<< (std::ostream &out, const ColorFormat &fmt)
 
template<>
std::shared_ptr< IShapeInferExtensionmake_so_pointer (const file_name_t &name)
 Creates a special shared_pointer wrapper for the given type from a specific shared module. More...
 
template<>
std::shared_ptr< IExtensionmake_so_pointer (const file_name_t &name)
 Creates a special shared_pointer wrapper for the given type from a specific shared module. More...
 
ICNNNetReaderCreateCNNNetReader () noexcept
 Creates a CNNNetReader instance. More...
 
StatusCode CreateExtension (IExtension *&ext, ResponseDesc *resp) noexcept
 Creates the default instance of the extension. More...
 
StatusCode CreateShapeInferExtension (IShapeInferExtension *&ext, ResponseDesc *resp) noexcept
 Creates the default instance of the shape infer extension. More...
 
template<typename F >
void parallel_nt (int nthr, const F &func)
 
template<typename F >
void parallel_nt_static (int nthr, const F &func)
 
template<typename I , typename F >
void parallel_sort (I begin, I end, const F &comparator)
 
template<typename T0 , typename R , typename F >
parallel_sum (const T0 &D0, const R &input, const F &func)
 
template<typename T0 , typename T1 , typename R , typename F >
parallel_sum2d (const T0 &D0, const T1 &D1, const R &input, const F &func)
 
template<typename T0 , typename T1 , typename T2 , typename R , typename F >
parallel_sum3d (const T0 &D0, const T1 &D1, const T2 &D2, const R &input, const F &func)
 
template<typename T >
parallel_it_init (T start)
 
template<typename T , typename Q , typename R , typename... Args>
parallel_it_init (T start, Q &x, const R &X, Args &&... tuple)
 
bool parallel_it_step ()
 
template<typename Q , typename R , typename... Args>
bool parallel_it_step (Q &x, const R &X, Args &&... tuple)
 
template<typename T , typename Q >
void splitter (const T &n, const Q &team, const Q &tid, T &n_start, T &n_end)
 
template<typename T0 , typename F >
void for_1d (const int &ithr, const int &nthr, const T0 &D0, const F &func)
 
template<typename T0 , typename F >
void parallel_for (const T0 &D0, const F &func)
 
template<typename T0 , typename T1 , typename F >
void for_2d (const int &ithr, const int &nthr, const T0 &D0, const T1 &D1, const F &func)
 
template<typename T0 , typename T1 , typename F >
void parallel_for2d (const T0 &D0, const T1 &D1, const F &func)
 
template<typename T0 , typename T1 , typename T2 , typename F >
void for_3d (const int &ithr, const int &nthr, const T0 &D0, const T1 &D1, const T2 &D2, const F &func)
 
template<typename T0 , typename T1 , typename T2 , typename F >
void parallel_for3d (const T0 &D0, const T1 &D1, const T2 &D2, const F &func)
 
template<typename T0 , typename T1 , typename T2 , typename T3 , typename F >
void for_4d (const int &ithr, const int &nthr, const T0 &D0, const T1 &D1, const T2 &D2, const T3 &D3, const F &func)
 
template<typename T0 , typename T1 , typename T2 , typename T3 , typename F >
void parallel_for4d (const T0 &D0, const T1 &D1, const T2 &D2, const T3 &D3, const F &func)
 
template<typename T0 , typename T1 , typename T2 , typename T3 , typename T4 , typename F >
void for_5d (const int &ithr, const int &nthr, const T0 &D0, const T1 &D1, const T2 &D2, const T3 &D3, const T4 &D4, const F &func)
 
template<typename T0 , typename T1 , typename T2 , typename T3 , typename T4 , typename F >
void parallel_for5d (const T0 &D0, const T1 &D1, const T2 &D2, const T3 &D3, const T4 &D4, const F &func)
 
StatusCode CreatePluginEngine (IInferencePlugin *&plugin, ResponseDesc *resp) noexcept
 Creates the default instance of the interface (per plugin) More...
 
RemoteBlob::Ptr make_shared_blob (const TensorDesc &desc, RemoteContext::Ptr ctx)
 A wrapper of CreateBlob method of RemoteContext to keep consistency with plugin-specific wrappers. More...
 
std::string fileNameToString (const file_name_t &str)
 Conversion from possibly-wide character string to a single-byte chain.
 
file_name_t stringToFileName (const std::string &str)
 Conversion from single-byte character string to a possibly-wide one.
 
const VersionGetInferenceEngineVersion () noexcept
 Gets the current Inference Engine version. More...
 
template<class T >
void TopResults (unsigned int n, TBlob< T > &input, std::vector< unsigned > &output)
 Gets the top n results from a tblob. More...
 
void TopResults (unsigned int n, Blob &input, std::vector< unsigned > &output)
 Gets the top n results from a blob. More...
 
template<typename data_t >
void copyFromRGB8 (uint8_t *RGB8, size_t RGB8_size, InferenceEngine::TBlob< data_t > *blob)
 Copies a 8-bit RGB image to the blob. More...
 
void ConvertImageToInput (unsigned char *imgBufRGB8, size_t lengthbytesSize, Blob &input)
 Splits the RGB channels to either I16 Blob or float blob. More...
 
template<typename T >
void copyToFloat (float *dst, const InferenceEngine::Blob *src)
 Copies data from a certain precision to float. More...
 

Variables

struct {
   std::string   InferenceEngine::name
 Layer name.
 
   std::string   InferenceEngine::type
 Layer type.
 
   Precision   InferenceEngine::precision
 Layer precision.
 
}; 
 This is an internal common Layer parameter parsing arguments. More...
 
class {
using Ptr = std::shared_ptr< CNNLayer >
 A shared pointer to CNNLayer.
 
   std::string   InferenceEngine::name
 Layer name.
 
   std::string   InferenceEngine::type
 Layer type.
 
   Precision   InferenceEngine::precision
 Layer base operating precision.
 
   std::vector< DataPtr >   InferenceEngine::outData
 A vector of pointers to the output data elements of this layer in the di-graph (order matters)
 
   std::vector< DataWeakPtr >   InferenceEngine::insData
 A vector of weak pointers to the input data elements of this layer in the di-graph (order matters)
 
   Ptr   InferenceEngine::_fusedWith
 If suggested to fuse - a pointer to the layer which needs to be fused with this layer.
 
   UserValue   InferenceEngine::userValue
 Convenience user values to store in this object as extra data.
 
   std::string   InferenceEngine::affinity
 Layer affinity set by user.
 
   std::map< std::string, std::string >   InferenceEngine::params
 Map of pairs: (parameter name, parameter value)
 
   std::map< std::string, Blob::Ptr >   InferenceEngine::blobs
 Map of pairs: (name, weights/biases blob)
 
   std::shared_ptr< ngraph::Node >   node
 
}; 
 This is a base abstraction Layer - all DNN Layers inherit from this class. More...
 
Blob::Ptr _weights
 A pointer to a weights blob.
 
Blob::Ptr _biases
 A pointer to a biases blob.
 
PropertyVector< unsigned int > _kernel
 A convolution kernel array [X, Y, Z, ...]. More...
 
unsigned int & _kernel_x = _kernel .at(X_AXIS)
 
unsigned int & _kernel_y = _kernel .at(Y_AXIS)
 
PropertyVector< unsigned int > _padding
 A convolution paddings begin array [X, Y, Z, ...]. More...
 
unsigned int & _padding_x = _padding .at(X_AXIS)
 
unsigned int & _padding_y = _padding .at(Y_AXIS)
 
PropertyVector< unsigned int > _pads_end
 A convolution paddings end array [X, Y, Z, ...]. More...
 
PropertyVector< unsigned int > _stride
 A convolution strides array [X, Y, Z, ...]. More...
 
unsigned int & _stride_x = _stride .at(X_AXIS)
 
unsigned int & _stride_y = _stride .at(Y_AXIS)
 
PropertyVector< unsigned int > _dilation
 A convolution dilations array [X, Y, Z, ...].
 
unsigned int & _dilation_x = _dilation .at(X_AXIS)
 
unsigned int & _dilation_y = _dilation .at(Y_AXIS)
 
unsigned int _out_depth = 0u
 A number of output feature maps (size) generating the 3'rd output dimension.
 
unsigned int _group = 1u
 Number of groups.
 
std::string _auto_pad
 Auto padding type.
 
unsigned int _deformable_group = 1u
 Number of deformable groups.
 
PoolType _type = MAX
 A pooling type.
 
bool _exclude_pad = false
 A flag that indicates if padding is excluded or not.
 
eBinaryConvolutionMode _mode = xnor_popcount
 Mode of binary convolution operation.
 
unsigned int _in_depth = 0u
 A number of input feature maps (size) generating the 3'rd input dimension.
 
float _pad_value = 0.0f
 A pad value which is used to fill pad area.
 
unsigned int _out_num = 0
 A size of output.
 
unsigned int _axis = 1
 An axis on which concatenation operation is performed. More...
 
unsigned int _size = 0
 Response size.
 
unsigned int _k = 1
 K.
 
float _alpha = 0
 Alpha coefficient.
 
float _beta = 0
 Beta coefficient.
 
bool _isAcrossMaps = false
 Flag to specify normalization across feature maps (true) or across channels.
 
int axis = 1
 Axis number for a softmax operation. More...
 
float bias = 0.f
 Bias for squares sum.
 
int across_channels = 0
 Indicate that mean value is calculated across channels.
 
int normalize = 1
 Indicate that the result needs to be normalized.
 
float negative_slope = 0.0f
 Negative slope is used to takle negative inputs instead of setting them to 0.
 
float min_value = 0.0f
 A minimum value.
 
float max_value = 1.0f
 A maximum value.
 
eOperation _operation = Sum
 A type of the operation to use.
 
std::vector< float > coeff
 A vector of coefficients to scale the operands.
 
std::vector< int > dim
 A vector of dimensions to be preserved.
 
std::vector< int > offset = 0.f
 A vector of offsets for each dimension. More...
 
std::vector< int > shape
 A vector of sizes of the shape.
 
int num_axes = -1
 A number of first axises to be taken for a reshape.
 
int tiles = -1
 A number of copies to be made.
 
unsigned int _broadcast = 0
 A flag that indicates if the same value is used for all the features. If false, the value is used pixel wise.
 
std::vector< PortMapinput_port_map
 Input ports map.
 
std::vector< PortMapoutput_port_map
 Output ports map.
 
std::vector< PortMapback_edges
 Back edges map.
 
Body body
 A Tensor Iterator body.
 
CellType cellType = LSTM
 Direct type of recurrent cell (including subtypes) Description of particular cell semantics is in LSTMCell, GRUCell, RNNCell.
 
int hidden_size = 0
 Size of hidden state data. More...
 
float clip = 0.0f
 Clip data into range [-clip, clip] on input of activations. More...
 
std::vector< std::string > activations
 Activations used inside recurrent cell. More...
 
std::vector< float > activation_alpha
 Alpha parameters of activations. More...
 
std::vector< float > activation_beta
 Beta parameters of activations. More...
 
Direction direction = FWD
 Direction of iteration through sequence dimension.
 
bool _channel_shared = false
 A flag that indicates if the same negative_slope value is used for all the features. If false, the value is used pixel wise.
 
float power = 1.f
 An exponent value.
 
float scale = 1.f
 A scale factor.
 
float epsilon = 1e-3f
 A small value to add to the variance estimate to avoid division by zero.
 
float alpha = 1.f
 A scale factor of src1 matrix.
 
float beta = 1.f
 A scale factor of src3 matrix.
 
bool transpose_a = false
 A flag that indicates if the src1 matrix is to be transposed.
 
bool transpose_b = false
 A flag that indicates if the src2 matrix is to be transposed.
 
PropertyVector< unsigned int > pads_begin
 Size of padding in the beginning of each axis.
 
PropertyVector< unsigned int > pads_end
 Size of padding in the end of each axis.
 
ePadMode pad_mode = Constant
 Mode of pad operation.
 
float pad_value = 0.0f
 A pad value which is used for filling in Constant mode.
 
std::string begin_mask
 The begin_mask is a bitmask where bit i being 0 means to ignore the begin value and instead use the default value.
 
std::string end_mask
 Analogous to begin_mask.
 
std::string ellipsis_mask
 The ellipsis_mask is a bitmask where bit i being 1 means the i-th is actually an ellipsis.
 
std::string new_axis_mask
 The new_axis_mask_ is a bitmask where bit i being 1 means the i-th position creates a new 1 dimension shape.
 
std::string shrink_axis_mask
 The shrink_axis_mask is a bitmask where bit i being 1 means the i-th position shrinks the dimensionality.
 
unsigned int group = 1
 The group of output shuffled channels.
 
unsigned int block_size = 1
 The group of output shuffled channels. More...
 
std::vector< size_t > _block_shape
 Spatial dimensions blocks sizes.
 
std::vector< size_t > _pads_begin
 Size of padding in the beginning of each axis.
 
std::vector< size_t > _crops_begin
 It specifies how many elements to crop from the intermediate result across the spatial dimensions.
 
std::vector< size_t > _crops_end
 It specifies how many elements to crop from the intermediate result across the spatial dimensions.
 
bool with_right_bound = false
 Indicates whether the intervals include the right or the left bucket edge.
 
int seq_axis = 1
 The seq_axis dimension in tensor which is partially reversed.
 
int batch_axis = 0
 The batch_axis dimension in tensor along which reversal is performed.
 
unsigned int depth = 0
 A depth of representation.
 
float on_value = 1.f
 The locations represented by indices in input take value on_value.
 
float off_value = 0.f
 The locations not represented by indices in input take value off_value.
 
int levels = 1
 The number of quantization levels.
 
bool keep_dims = true
 The keep_dims dimension in tensor which is partially reversed.
 
std::string mode
 The mode could be 'max' or 'min'.
 
std::string sort
 top K values sort mode could be 'value' or 'index'
 
bool sorted
 A flag indicating whether to sort unique elements.
 
bool return_inverse
 A flag indicating whether to return indices of input data elements in the output of uniques.
 
bool return_counts
 A flag indicating whether to return a number of occurences for each unique element.
 
bool center_point_box = false
 The 'center_point_box' indicates the format of the box data.
 
bool sort_result_descending = true
 The 'sort_result_descending' indicates that result will sort descending by score through all batches and classes.
 
int flatten = 1
 flatten value
 
int grid_w = 0
 Value of grid width.
 
int grid_h = 0
 Value of grid height.
 
float stride_w = 0.f
 Value of width step between grid cells.
 
float stride_h = 0.f
 Value of height step between grid cells.
 
float min_size = 0.f
 Minimium width and height for boxes.
 
float nms_threshold = 0.7f
 Non max suppression threshold.
 
int pre_nms_topn = 1000
 Maximum number of anchors selected before nms.
 
int post_nms_topn = 1000
 Maximum number of anchors selected after nms.
 
constexpr const int MAX_DIMS_NUMBER = 12
 

Detailed Description

Inference Engine API.

Typedef Documentation

§ idx_t

using InferenceEngine::idx_t = typedef size_t

A type of network objects indexes.

Deprecated:
Use ngraph API instead.

§ InferenceEnginePluginPtr

using InferenceEngine::InferenceEnginePluginPtr = typedef InferenceEngine::details::SOPointer<IInferencePlugin>

A C++ helper to work with objects created by the plugin.

Implements different interfaces.

§ IShapeInferExtensionPtr

A shared pointer to a IShapeInferExtension interface.

Deprecated:
Migrate to IR v10 and implement shape inference in the ngraph::op::Op::validate_and_infer_types method This API will be removed in 2021.1 release.

§ SizeVector

using InferenceEngine::SizeVector = typedef std::vector<size_t>

Represents tensor size.

The order is opposite to the order in Caffe*: (w,h,n,b) where the most frequently changing element in memory is first.

Enumeration Type Documentation

§ ColorFormat

Extra information about input color format for preprocessing.

Enumerator
RAW 

Plain blob (default), no extra color processing required.

RGB 

RGB color format.

BGR 

BGR color format, default in DLDT.

RGBX 

RGBX color format with X ignored during inference.

BGRX 

BGRX color format with X ignored during inference.

NV12 

NV12 color format represented as compound Y+UV blob.

I420 

I420 color format represented as compound Y+U+V blob.

§ Layout

enum InferenceEngine::Layout : uint8_t

Layouts that the inference engine supports.

Enumerator
ANY 

"any" layout

NCHW 

NCHW layout for input / output blobs.

NHWC 

NHWC layout for input / output blobs.

NCDHW 

NCDHW layout for input / output blobs.

NDHWC 

NDHWC layout for input / output blobs.

OIHW 

NDHWC layout for operation weights.

GOIHW 

NDHWC layout for operation weights.

OIDHW 

NDHWC layout for operation weights.

GOIDHW 

NDHWC layout for operation weights.

SCALAR 

A scalar layout.

A bias layout for opearation.

CHW 

A single image layout (e.g. for mean image)

HW 

HW 2D layout.

NC 

HC 2D layout.

CN 

CN 2D layout.

BLOCKED 

A blocked layout.

§ LockOp

Allocator handle mapping type.

Enumerator
LOCK_FOR_READ 

A flag to lock data for read.

LOCK_FOR_WRITE 

A flag to lock data for write.

§ MeanVariant

Defines available types of mean.

Enumerator
MEAN_IMAGE 

mean value is specified for each input pixel

MEAN_VALUE 

mean value is specified for each input channel

NONE 

no mean value specified

Function Documentation

§ as() [1/2]

template<typename T , typename std::enable_if<!std::is_pointer< T >::value &&!std::is_reference< T >::value, int >::type = 0, typename std::enable_if< std::is_base_of< Blob, T >::value, int >::type = 0>
std::shared_ptr<T> InferenceEngine::as ( const Blob::Ptr blob)
noexcept

Helper cast function to work with shared Blob objects.

Returns
shared_ptr to the type T. Returned shared_ptr shares ownership of the object with the input Blob::Ptr

§ as() [2/2]

template<typename T , typename std::enable_if<!std::is_pointer< T >::value &&!std::is_reference< T >::value, int >::type = 0, typename std::enable_if< std::is_base_of< Blob, T >::value, int >::type = 0>
std::shared_ptr<const T> InferenceEngine::as ( const Blob::CPtr blob)
noexcept

Helper cast function to work with shared Blob objects.

Returns
shared_ptr to the type const T. Returned shared_ptr shares ownership of the object with the input Blob::Ptr

§ ConvertImageToInput()

void InferenceEngine::ConvertImageToInput ( unsigned char *  imgBufRGB8,
size_t  lengthbytesSize,
Blob input 
)
inline

Splits the RGB channels to either I16 Blob or float blob.

Deprecated:
InferenceEngine utility functions are not a part of public API. This method will be removed in 2020.4 release.

The image buffer is assumed to be packed with no support for strides.

Parameters
imgBufRGB8Packed 24bit RGB image (3 bytes per pixel: R-G-B)
lengthbytesSizeSize in bytes of the RGB image. It is equal to amount of pixels times 3 (number of channels)
inputBlob to contain the split image (to 3 channels)

§ copyFromRGB8()

template<typename data_t >
void InferenceEngine::copyFromRGB8 ( uint8_t *  RGB8,
size_t  RGB8_size,
InferenceEngine::TBlob< data_t > *  blob 
)

Copies a 8-bit RGB image to the blob.

Deprecated:
InferenceEngine utility functions are not a part of public API. This method will be removed in 2020.4 release.

Throws an exception in case of dimensions or input size mismatch

Template Parameters
data_tType of the target blob
Parameters
RGB88-bit RGB image
RGB8_sizeSize of the image
blobTarget blob to write image to

§ copyToFloat()

template<typename T >
void InferenceEngine::copyToFloat ( float *  dst,
const InferenceEngine::Blob src 
)

Copies data from a certain precision to float.

Deprecated:
InferenceEngine utility functions are not a part of public API. This method will be removed in 2020.4 release.
Parameters
dstPointer to an output float buffer, must be allocated before the call
srcSource blob to take data from

§ CreateCNNNetReader()

ICNNNetReader* InferenceEngine::CreateCNNNetReader ( )
noexcept

Creates a CNNNetReader instance.

Returns
An object that implements the ICNNNetReader interface

§ CreateDefaultAllocator()

InferenceEngine::IAllocator* InferenceEngine::CreateDefaultAllocator ( )
noexcept

Creates the default implementation of the Inference Engine allocator per plugin.

Returns
The Inference Engine IAllocator* instance

§ CreateExtension()

StatusCode InferenceEngine::CreateExtension ( IExtension *&  ext,
ResponseDesc resp 
)
noexcept

Creates the default instance of the extension.

Parameters
extExtension interface
respResponse description
Returns
Status code

§ CreatePluginEngine()

StatusCode InferenceEngine::CreatePluginEngine ( IInferencePlugin *&  plugin,
ResponseDesc resp 
)
noexcept

Creates the default instance of the interface (per plugin)

Parameters
pluginPointer to the plugin
respPointer to the response message that holds a description of an error if any occurred
Returns
Status code of the operation. InferenceEngine::OK if succeeded

§ CreateShapeInferExtension()

StatusCode InferenceEngine::CreateShapeInferExtension ( IShapeInferExtension *&  ext,
ResponseDesc resp 
)
noexcept

Creates the default instance of the shape infer extension.

Deprecated:
Migrate to IR v10 and implement shape inference in the ngraph::op::Op::validate_and_infer_types method This API will be removed in 2021.1 release.
Parameters
extShape Infer Extension interface
respResponse description
Returns
Status code

§ GetInferenceEngineVersion()

const Version* InferenceEngine::GetInferenceEngineVersion ( )
noexcept

Gets the current Inference Engine version.

Returns
The current Inference Engine version

§ make_shared_blob() [1/7]

RemoteBlob::Ptr InferenceEngine::make_shared_blob ( const TensorDesc desc,
RemoteContext::Ptr  ctx 
)
inline

A wrapper of CreateBlob method of RemoteContext to keep consistency with plugin-specific wrappers.

Parameters
descDefines the layout and dims of the blob
ctxPoniter to the plugin object derived from RemoteContext.
Returns
A pointer to plugin object that implements RemoteBlob interface.

§ make_shared_blob() [2/7]

template<typename Type >
InferenceEngine::TBlob<Type>::Ptr InferenceEngine::make_shared_blob ( const TensorDesc tensorDesc)
inline

Creates a blob with the given tensor descriptor.

Template Parameters
TypeType of the shared pointer to be created
Parameters
tensorDescTensor descriptor for Blob creation
Returns
A shared pointer to the newly created blob of the given type

§ make_shared_blob() [3/7]

template<typename Type >
InferenceEngine::TBlob<Type>::Ptr InferenceEngine::make_shared_blob ( const TensorDesc tensorDesc,
Type *  ptr,
size_t  size = 0 
)
inline

Creates a blob with the given tensor descriptor from the pointer to the pre-allocated memory.

Template Parameters
TypeType of the shared pointer to be created
Parameters
tensorDescTensorDesc for Blob creation
ptrPointer to the pre-allocated memory
sizeLength of the pre-allocated array
Returns
A shared pointer to the newly created blob of the given type

§ make_shared_blob() [4/7]

template<typename Type >
InferenceEngine::TBlob<Type>::Ptr InferenceEngine::make_shared_blob ( const TensorDesc tensorDesc,
const std::shared_ptr< InferenceEngine::IAllocator > &  alloc 
)
inline

Creates a blob with the given tensor descriptor and allocator.

Template Parameters
TypeType of the shared pointer to be created
Parameters
tensorDescTensor descriptor for Blob creation
allocShared pointer to IAllocator to use in the blob
Returns
A shared pointer to the newly created blob of the given type

§ make_shared_blob() [5/7]

template<typename TypeTo >
InferenceEngine::TBlob<TypeTo>::Ptr InferenceEngine::make_shared_blob ( const TBlob< TypeTo > &  arg)
inline

Creates a copy of given TBlob instance.

Template Parameters
TypeToType of the shared pointer to be created
Parameters
arggiven pointer to blob
Returns
A shared pointer to the newly created blob of the given type

§ make_shared_blob() [6/7]

template<typename T , typename... Args, typename std::enable_if< std::is_base_of< Blob, T >::value, int >::type = 0>
std::shared_ptr<T> InferenceEngine::make_shared_blob ( Args &&...  args)

Creates a Blob object of the specified type.

Parameters
argsConstructor arguments for the Blob object
Returns
A shared pointer to the newly created Blob object

§ make_shared_blob() [7/7]

Blob::Ptr InferenceEngine::make_shared_blob ( const Blob::Ptr inputBlob,
const ROI roi 
)

Creates a blob describing given ROI object based on the given blob with pre-allocated memory.

Parameters
inputBloboriginal blob with pre-allocated memory.
roiA ROI object inside of the original blob.
Returns
A shared pointer to the newly created blob.

§ make_so_pointer() [1/3]

template<class T >
std::shared_ptr<T> InferenceEngine::make_so_pointer ( const file_name_t &  name)
inlinedelete

Creates a special shared_pointer wrapper for the given type from a specific shared module.

Template Parameters
TAn type of object SOPointer can hold
Parameters
nameName of the shared library file
Returns
A created object
Deprecated:
Use make_so_pointer with IExtension as template argument type.
Parameters
nameName of the shared library file
Returns
shared_pointer A wrapper for the given type from a specific shared module
Parameters
nameName of the shared library file
Returns
shared_pointer A wrapper for the given type from a specific shared module

§ make_so_pointer() [2/3]

template<>
std::shared_ptr<IShapeInferExtension> InferenceEngine::make_so_pointer ( const file_name_t &  name)
inlinedelete

Creates a special shared_pointer wrapper for the given type from a specific shared module.

Deprecated:
Use make_so_pointer with IExtension as template argument type.
Parameters
nameName of the shared library file
Returns
shared_pointer A wrapper for the given type from a specific shared module

§ make_so_pointer() [3/3]

template<>
std::shared_ptr<IExtension> InferenceEngine::make_so_pointer ( const file_name_t &  name)
inlinedelete

Creates a special shared_pointer wrapper for the given type from a specific shared module.

Parameters
nameName of the shared library file
Returns
shared_pointer A wrapper for the given type from a specific shared module

§ TopResults() [1/2]

template<class T >
void InferenceEngine::TopResults ( unsigned int  n,
TBlob< T > &  input,
std::vector< unsigned > &  output 
)
inline

Gets the top n results from a tblob.

Deprecated:
InferenceEngine utility functions are not a part of public API. This method will be removed in 2020.4 release.
Parameters
nTop n count
input1D tblob that contains probabilities
outputVector of indexes for the top n places

§ TopResults() [2/2]

void InferenceEngine::TopResults ( unsigned int  n,
Blob input,
std::vector< unsigned > &  output 
)
inline

Gets the top n results from a blob.

Deprecated:
InferenceEngine utility functions are not a part of public API. This method will be removed in 2020.4 release.
Parameters
nTop n count
input1D blob that contains probabilities
outputVector of indexes for the top n places

Variable Documentation

§ @1

struct { ... }

This is an internal common Layer parameter parsing arguments.

Deprecated:
Migrate to IR v10 and work with ngraph::Function directly. The method will be removed in 2021.1

§ @3

class { ... }

This is a base abstraction Layer - all DNN Layers inherit from this class.

Deprecated:
Migrate to IR v10 and work with ngraph::Function directly. The method will be removed in 2021.1

§ _axis

unsigned int InferenceEngine::_axis = 1

An axis on which concatenation operation is performed.

An axis on which split operation is performed.

§ _kernel

PropertyVector<unsigned int> InferenceEngine::_kernel

A convolution kernel array [X, Y, Z, ...].

Pooling kernel array [X, Y, Z, ...].

§ _padding

PropertyVector<unsigned int> InferenceEngine::_padding

A convolution paddings begin array [X, Y, Z, ...].

Pooling paddings begin array [X, Y, Z, ...].

§ _pads_end

std::vector<size_t> InferenceEngine::_pads_end

A convolution paddings end array [X, Y, Z, ...].

Size of padding in the end of each axis.

Pooling paddings end array [X, Y, Z, ...].

§ _stride

PropertyVector<unsigned int> InferenceEngine::_stride

A convolution strides array [X, Y, Z, ...].

Pooling strides array [X, Y, Z, ...].

§ activation_alpha

std::vector<float> InferenceEngine::activation_alpha

Alpha parameters of activations.

Respective to activation list.

§ activation_beta

std::vector<float> InferenceEngine::activation_beta

Beta parameters of activations.

Respective to activation list.

§ activations

std::vector<std::string> InferenceEngine::activations

Activations used inside recurrent cell.

Valid values: sigmoid, tanh, relu

§ axis

int InferenceEngine::axis = 1

Axis number for a softmax operation.

The axis in Dictionary to scatter Indexes from.

The axis dimension in tensor which is top K values are picked.

Define the shape of output tensor.

The axis in tensor to shuffle channels.

The axis in Dictionary to gather Indexes from.

An axis by which iteration is performed.

An index of the axis to tile.

A number of axis to be taken for a reshape.

A vector of dimensions for cropping.

axis=0 means first input/output data blob dimension is sequence axis=1 means first input/output data blob dimension is batch

§ block_size

unsigned int InferenceEngine::block_size = 1

The group of output shuffled channels.

The group of output Space To Depth.

§ clip

float InferenceEngine::clip = 0.0f

Clip data into range [-clip, clip] on input of activations.

clip==0.0f means no clipping

§ hidden_size

int InferenceEngine::hidden_size = 0

Size of hidden state data.

In case of batch output state tensor will have shape [N, hidden_size]

§ offset

float InferenceEngine::offset = 0.f

A vector of offsets for each dimension.

An offset value.

§ precision

Precision InferenceEngine::precision

Layer precision.

Layer base operating precision.