Public Types | Public Member Functions
InferenceEngine::IInferRequest Class Referenceabstract

This is an interface of asynchronous infer request. More...

#include <ie_iinfer_request.hpp>

Inheritance diagram for InferenceEngine::IInferRequest:
Inheritance graph
[legend]
Collaboration diagram for InferenceEngine::IInferRequest:
Collaboration graph
[legend]

Public Types

enum  WaitMode : int64_t { RESULT_READY = -1, STATUS_ONLY = 0 }
 Enumeration to hold wait mode for IInferRequest. More...
 
using Ptr = std::shared_ptr< IInferRequest >
 A shared pointer to the IInferRequest object.
 
using WeakPtr = std::weak_ptr< IInferRequest >
 A smart pointer to the IInferRequest object.
 
typedef void(* CompletionCallback) (InferenceEngine::IInferRequest::Ptr context, InferenceEngine::StatusCode code)
 Completion callback definition as pointer to a function. More...
 

Public Member Functions

virtual StatusCode SetBlob (const char *name, const Blob::Ptr &data, ResponseDesc *resp) noexcept=0
 Sets input/output data to infer. More...
 
virtual StatusCode GetBlob (const char *name, Blob::Ptr &data, ResponseDesc *resp) noexcept=0
 Gets input/output data for inference. More...
 
virtual StatusCode SetBlob (const char *name, const Blob::Ptr &data, const PreProcessInfo &info, ResponseDesc *resp) noexcept=0
 Sets pre-process for input data. More...
 
virtual StatusCode GetPreProcess (const char *name, const PreProcessInfo **info, ResponseDesc *resp) const noexcept=0
 Gets pre-process for input data. More...
 
virtual StatusCode Infer (ResponseDesc *resp) noexcept=0
 Infers specified input(s) in synchronous mode. More...
 
virtual StatusCode GetPerformanceCounts (std::map< std::string, InferenceEngineProfileInfo > &perfMap, ResponseDesc *resp) const noexcept=0
 Queries performance measures per layer to get feedback of what is the most time consuming layer. More...
 
virtual InferenceEngine::StatusCode Wait (int64_t millis_timeout, ResponseDesc *resp) noexcept=0
 Waits for the result to become available. Blocks until specified millis_timeout has elapsed or the result becomes available, whichever comes first. More...
 
virtual StatusCode StartAsync (ResponseDesc *resp) noexcept=0
 Starts inference of specified input(s) in asynchronous mode. More...
 
virtual StatusCode SetCompletionCallback (CompletionCallback callback) noexcept=0
 Sets a callback function that will be called on success or failure of asynchronous request. More...
 
virtual StatusCode GetUserData (void **data, ResponseDesc *resp) noexcept=0
 Gets arbitrary data for the request and stores a pointer to a pointer to the obtained data. More...
 
virtual StatusCode SetUserData (void *data, ResponseDesc *resp) noexcept=0
 Sets arbitrary data for the request. More...
 
virtual InferenceEngine::StatusCode SetBatch (int batch_size, ResponseDesc *resp) noexcept=0
 Sets new batch size when dynamic batching is enabled in executable network that created this request. More...
 

Detailed Description

This is an interface of asynchronous infer request.

Member Typedef Documentation

§ CompletionCallback

typedef void(* InferenceEngine::IInferRequest::CompletionCallback) (InferenceEngine::IInferRequest::Ptr context, InferenceEngine::StatusCode code)

Completion callback definition as pointer to a function.

Parameters
contextPointer to request for providing context inside callback
codeCompletion result status: InferenceEngine::OK (0) for success

Member Enumeration Documentation

§ WaitMode

Enumeration to hold wait mode for IInferRequest.

Enumerator
RESULT_READY 

Wait until inference result becomes available

STATUS_ONLY 

IInferRequest doesn't block or interrupt current thread and immediately returns inference status

Member Function Documentation

§ GetBlob()

virtual StatusCode InferenceEngine::IInferRequest::GetBlob ( const char *  name,
Blob::Ptr data,
ResponseDesc resp 
)
pure virtualnoexcept

Gets input/output data for inference.

Note
Memory allocation does not happen
Parameters
nameName of input or output blob.
dataReference to input or output blob. The type of Blob must match the network input precision and size.
respOptional: pointer to an already allocated object to contain information in case of failure
Returns
Status code of the operation: InferenceEngine::OK (0) for success

§ GetPerformanceCounts()

virtual StatusCode InferenceEngine::IInferRequest::GetPerformanceCounts ( std::map< std::string, InferenceEngineProfileInfo > &  perfMap,
ResponseDesc resp 
) const
pure virtualnoexcept

Queries performance measures per layer to get feedback of what is the most time consuming layer.

Note
not all plugins provide meaningful data
Parameters
perfMapMap of layer names to profiling information for that layer
respOptional: pointer to an already allocated object to contain information in case of failure
Returns
Status code of the operation: InferenceEngine::OK (0) for success

§ GetPreProcess()

virtual StatusCode InferenceEngine::IInferRequest::GetPreProcess ( const char *  name,
const PreProcessInfo **  info,
ResponseDesc resp 
) const
pure virtualnoexcept

Gets pre-process for input data.

Parameters
nameName of input blob.
infopointer to a pointer to PreProcessInfo structure
respOptional: pointer to an already allocated object to contain information in case of failure
Returns
Status code of the operation: OK (0) for success

§ GetUserData()

virtual StatusCode InferenceEngine::IInferRequest::GetUserData ( void **  data,
ResponseDesc resp 
)
pure virtualnoexcept

Gets arbitrary data for the request and stores a pointer to a pointer to the obtained data.

Parameters
dataPointer to a pointer to the gotten arbitrary data
respOptional: a pointer to an already allocated object to contain extra information of a failure (if occurred)
Returns
Enumeration of the resulted action: InferenceEngine::OK (0) for success

§ Infer()

virtual StatusCode InferenceEngine::IInferRequest::Infer ( ResponseDesc resp)
pure virtualnoexcept

Infers specified input(s) in synchronous mode.

Note
blocks all methods of IInferRequest while request is ongoing (running or waiting in queue)
Parameters
respOptional: pointer to an already allocated object to contain information in case of failure
Returns
Status code of the operation: InferenceEngine::OK (0) for success

§ SetBatch()

virtual InferenceEngine::StatusCode InferenceEngine::IInferRequest::SetBatch ( int  batch_size,
ResponseDesc resp 
)
pure virtualnoexcept

Sets new batch size when dynamic batching is enabled in executable network that created this request.

Parameters
batch_sizenew batch size to be used by all the following inference calls for this request.
respOptional: a pointer to an already allocated object to contain extra information of a failure (if occurred)
Returns
Enumeration of the resulted action: InferenceEngine::OK (0) for success

§ SetBlob() [1/2]

virtual StatusCode InferenceEngine::IInferRequest::SetBlob ( const char *  name,
const Blob::Ptr data,
ResponseDesc resp 
)
pure virtualnoexcept

Sets input/output data to infer.

Note
Memory allocation does not happen
Parameters
nameName of input or output blob.
dataReference to input or output blob. The type of a blob must match the network input precision and size.
respOptional: pointer to an already allocated object to contain information in case of failure
Returns
Status code of the operation: InferenceEngine::OK (0) for success

§ SetBlob() [2/2]

virtual StatusCode InferenceEngine::IInferRequest::SetBlob ( const char *  name,
const Blob::Ptr data,
const PreProcessInfo info,
ResponseDesc resp 
)
pure virtualnoexcept

Sets pre-process for input data.

Parameters
nameName of input blob.
dataReference to input or output blob. The type of Blob must match the network input precision and size.
infoPreprocess info for blob.
respOptional: pointer to an already allocated object to contain information in case of failure
Returns
Status code of the operation: OK (0) for success

§ SetCompletionCallback()

virtual StatusCode InferenceEngine::IInferRequest::SetCompletionCallback ( CompletionCallback  callback)
pure virtualnoexcept

Sets a callback function that will be called on success or failure of asynchronous request.

Parameters
callbackA function to be called
Returns
Enumeration of the resulted action: InferenceEngine::OK (0) for success

§ SetUserData()

virtual StatusCode InferenceEngine::IInferRequest::SetUserData ( void *  data,
ResponseDesc resp 
)
pure virtualnoexcept

Sets arbitrary data for the request.

Parameters
dataPointer to a pointer to arbitrary data to set
respOptional: a pointer to an already allocated object to contain extra information of a failure (if occurred)
Returns
Enumeration of the resulted action: InferenceEngine::OK (0) for success

§ StartAsync()

virtual StatusCode InferenceEngine::IInferRequest::StartAsync ( ResponseDesc resp)
pure virtualnoexcept

Starts inference of specified input(s) in asynchronous mode.

Note
It returns immediately. Inference starts also immediately
Parameters
respOptional: a pointer to an already allocated object to contain extra information of a failure (if occurred)
Returns
Enumeration of the resulted action: InferenceEngine::OK (0) for success

§ Wait()

virtual InferenceEngine::StatusCode InferenceEngine::IInferRequest::Wait ( int64_t  millis_timeout,
ResponseDesc resp 
)
pure virtualnoexcept

Waits for the result to become available. Blocks until specified millis_timeout has elapsed or the result becomes available, whichever comes first.

Parameters
millis_timeoutMaximum duration in milliseconds to block for
Note
There are special cases when millis_timeout is equal some value of the WaitMode enum:
  • STATUS_ONLY - immediately returns inference status (IInferRequest::RequestStatus). It does not block or interrupt current thread
  • RESULT_READY - waits until inference result becomes available
Parameters
respOptional: a pointer to an already allocated object to contain extra information of a failure (if occurred)
Returns
Enumeration of the resulted action: InferenceEngine::OK (0) for success

The documentation for this class was generated from the following file: