This is an interface of asynchronous infer request. More...

#include <ie_iinfer_request.hpp>

Inheritance diagram for InferenceEngine::IInferRequest:

Collaboration diagram for InferenceEngine::IInferRequest:

Public Types
enum	WaitMode : int64_t { RESULT_READY = -1, STATUS_ONLY = 0 }
	Enumeration to hold wait mode for IInferRequest. More...

using	Ptr = std::shared_ptr< IInferRequest >

using	WeakPtr = std::weak_ptr< IInferRequest >

typedef void(*	CompletionCallback) (InferenceEngine::IInferRequest::Ptr context, InferenceEngine::StatusCode code)
	Completion callback definition as pointer to a function. More...

Public Member Functions
virtual StatusCode	SetBlob (const char name, const Blob::Ptr &data, ResponseDesc resp) noexcept=0
	Sets input/output data to infer. More...

virtual StatusCode	GetBlob (const char name, Blob::Ptr &data, ResponseDesc resp) noexcept=0
	Gets input/output data for inference. More...

virtual StatusCode	Infer (ResponseDesc *resp) noexcept=0
	Infers specified input(s) in synchronous mode. More...

virtual StatusCode	GetPerformanceCounts (std::map< std::string, InferenceEngineProfileInfo > &perfMap, ResponseDesc *resp) const noexcept=0
	Queries performance measures per layer to get feedback of what is the most time consuming layer. More...

virtual InferenceEngine::StatusCode	Wait (int64_t millis_timeout, ResponseDesc *resp) noexcept=0
	Waits for the result to become available. Blocks until specified millis_timeout has elapsed or the result becomes available, whichever comes first. More...

virtual StatusCode	StartAsync (ResponseDesc *resp) noexcept=0
	Starts inference of specified input(s) in asynchronous mode. More...

virtual StatusCode	SetCompletionCallback (CompletionCallback callback) noexcept=0
	Sets a callback function that will be called on success or failure of asynchronous request. More...

virtual StatusCode	GetUserData (void *data, ResponseDesc resp) noexcept=0
	Gets arbitrary data for the request and stores a pointer to a pointer to the obtained data. More...

virtual StatusCode	SetUserData (void data, ResponseDesc resp) noexcept=0
	Sets arbitrary data for the request. More...

virtual InferenceEngine::StatusCode	SetBatch (int batch_size, ResponseDesc *resp) noexcept=0
	Sets new batch size when dynamic batching is enabled in executable network that created this request. More...

Detailed Description

This is an interface of asynchronous infer request.

Member Typedef Documentation

§ CompletionCallback

typedef void(* InferenceEngine::IInferRequest::CompletionCallback) (InferenceEngine::IInferRequest::Ptr context, InferenceEngine::StatusCode code)

Completion callback definition as pointer to a function.

Parameters

context	Pointer to request for providing context inside callback
code	Completion result status: OK (0) for success

Member Enumeration Documentation

§ WaitMode

enum InferenceEngine::IInferRequest::WaitMode : int64_t

Enumeration to hold wait mode for IInferRequest.

Enumerator
RESULT_READY	Wait until inference result becomes available
STATUS_ONLY	IInferRequest doesn't block or interrupt current thread and immediately returns inference status

Member Function Documentation

§ GetBlob()

virtual StatusCode InferenceEngine::IInferRequest::GetBlob	(	const char *	name,
		Blob::Ptr &	data,
		ResponseDesc *	resp
	)

pure virtualnoexcept

Gets input/output data for inference.

Note: : Memory allocation does not happen

Parameters

name	Name of input or output blob.
data	Reference to input or output blob. The type of Blob must match the network input precision and size.
resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns: Status code of the operation: OK (0) for success

§ GetPerformanceCounts()

virtual StatusCode InferenceEngine::IInferRequest::GetPerformanceCounts	(	std::map< std::string, InferenceEngineProfileInfo > &	perfMap,
		ResponseDesc *	resp
	)		const

pure virtualnoexcept

Queries performance measures per layer to get feedback of what is the most time consuming layer.

Note: : not all plugins provide meaningful data

Parameters

perfMap	Map of layer names to profiling information for that layer
resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns: Status code of the operation: OK (0) for success

§ GetUserData()

virtual StatusCode InferenceEngine::IInferRequest::GetUserData	(	void **	data,
		ResponseDesc *	resp
	)

pure virtualnoexcept

Gets arbitrary data for the request and stores a pointer to a pointer to the obtained data.

Parameters

data	Pointer to a pointer to the gotten arbitrary data
resp	Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns: Enumeration of the resulted action: OK (0) for success

§ Infer()

virtual StatusCode InferenceEngine::IInferRequest::Infer ( ResponseDesc * resp )

pure virtualnoexcept

Infers specified input(s) in synchronous mode.

Note: blocks all methods of IInferRequest while request is ongoing (running or waiting in queue)

Parameters

resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns: Status code of the operation: OK (0) for success

§ SetBatch()

virtual InferenceEngine::StatusCode InferenceEngine::IInferRequest::SetBatch	(	int	batch_size,
		ResponseDesc *	resp
	)

pure virtualnoexcept

Sets new batch size when dynamic batching is enabled in executable network that created this request.

Parameters

batch_size	new batch size to be used by all the following inference calls for this request.
resp	Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns: Enumeration of the resulted action: OK (0) for success

§ SetBlob()

virtual StatusCode InferenceEngine::IInferRequest::SetBlob	(	const char *	name,
		const Blob::Ptr &	data,
		ResponseDesc *	resp
	)

pure virtualnoexcept

Sets input/output data to infer.

Note: : Memory allocation does not happen

Parameters

name	Name of input or output blob.
data	Reference to input or output blob. The type of a blob must match the network input precision and size.
resp	Optional: pointer to an already allocated object to contain information in case of failure

Returns: Status code of the operation: OK (0) for success

§ SetCompletionCallback()

virtual StatusCode InferenceEngine::IInferRequest::SetCompletionCallback ( CompletionCallback callback )

pure virtualnoexcept

Sets a callback function that will be called on success or failure of asynchronous request.

Parameters

callback A function to be called

Returns: Enumeration of the resulted action: OK (0) for success

§ SetUserData()

virtual StatusCode InferenceEngine::IInferRequest::SetUserData	(	void *	data,
		ResponseDesc *	resp
	)

pure virtualnoexcept

Sets arbitrary data for the request.

Parameters

data	Pointer to a pointer to arbitrary data to set
resp	Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns: Enumeration of the resulted action: OK (0) for success

§ StartAsync()

virtual StatusCode InferenceEngine::IInferRequest::StartAsync ( ResponseDesc * resp )

pure virtualnoexcept

Starts inference of specified input(s) in asynchronous mode.

Note: : It returns immediately. Inference starts also immediately

Parameters

resp	Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns: Enumeration of the resulted action: OK (0) for success

§ Wait()

virtual InferenceEngine::StatusCode InferenceEngine::IInferRequest::Wait	(	int64_t	millis_timeout,
		ResponseDesc *	resp
	)

pure virtualnoexcept

Waits for the result to become available. Blocks until specified millis_timeout has elapsed or the result becomes available, whichever comes first.

Parameters

millis_timeout Maximum duration in milliseconds to block for

Note

There are special cases when millis_timeout is equal some value of the WaitMode enum:

STATUS_ONLY - immediately returns inference status (IInferRequest::RequestStatus). It does not block or interrupt current thread
RESULT_READY - waits until inference result becomes available

Parameters

resp	Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns: Enumeration of the resulted action: OK (0) for success

The documentation for this class was generated from the following file:

tmp_docs/inference-engine/include/ie_iinfer_request.hpp

Public Types

Public Member Functions

Detailed Description

Member Typedef Documentation

§ CompletionCallback

Member Enumeration Documentation

§ WaitMode

Member Function Documentation

§ GetBlob()

§ GetPerformanceCounts()

§ GetUserData()

§ Infer()

§ SetBatch()

§ SetBlob()

§ SetCompletionCallback()

§ SetUserData()

§ StartAsync()

§ Wait()