minimum API to be implemented by plugin, which is used in InferRequestBase forwarding mechanism More...

#include <ie_infer_async_request_internal.hpp>

Inheritance diagram for InferenceEngine::AsyncInferRequestInternal:

Public Types
typedef std::shared_ptr< AsyncInferRequestInternal >	Ptr
	A shared pointer to a AsyncInferRequestInternal implementation.

Public Types inherited from InferenceEngine::IAsyncInferRequestInternal
typedef std::shared_ptr< IAsyncInferRequestInternal >	Ptr
	A shared pointer to IAsyncInferRequestInternal interface.

Public Types inherited from InferenceEngine::IInferRequestInternal
typedef std::shared_ptr< IInferRequestInternal >	Ptr
	A shared pointer to a IInferRequestInternal interface.

Public Types inherited from InferenceEngine::InferRequestInternal
typedef std::shared_ptr< InferRequestInternal >	Ptr
	A shared pointer to a InferRequestInternal implementation.

Public Member Functions
	AsyncInferRequestInternal (const InputsDataMap &networkInputs, const OutputsDataMap &networkOutputs)
	Constructs a new instance. More...

void	SetCompletionCallback (IInferRequest::CompletionCallback callback) override
	Set callback function which will be called on success or failure of asynchronous request. More...

void	GetUserData (void **data) override
	Get arbitrary data for the request. More...

void	SetUserData (void *data) override
	Set arbitrary data for the request. More...

void	SetPointerToPublicInterface (IInferRequest::Ptr ptr)
	Set weak pointer to the corresponding public interface: IInferRequest. This allow to pass it to IInferRequest::CompletionCallback. More...

void	StartAsync () override
	Start inference of specified input(s) in asynchronous mode. More...

Public Member Functions inherited from InferenceEngine::IAsyncInferRequestInternal
virtual	~IAsyncInferRequestInternal ()=default
	A virtual destructor.

virtual StatusCode	Wait (int64_t millis_timeout)=0
	Waits for the result to become available. Blocks until specified millis_timeout has elapsed or the result becomes available, whichever comes first. More...

Public Member Functions inherited from InferenceEngine::IInferRequestInternal
virtual	~IInferRequestInternal ()=default
	Destroys the object.

virtual std::map< std::string, InferenceEngineProfileInfo >	GetPerformanceCounts () const =0
	Queries performance measures per layer to get feedback of what is the most time consuming layer. Note: not all plugins may provide meaningful data. More...

Public Member Functions inherited from InferenceEngine::InferRequestInternal
	InferRequestInternal (const InputsDataMap &networkInputs, const OutputsDataMap &networkOutputs)
	Constructs a new instance. More...

virtual void	InferImpl ()=0
	The minimal infer function to be implemented by plugins. It infers specified input(s) in synchronous mode. More...

void	Infer () override
	Default common implementation for all plugins with checking input and output blobs before inference.

void	Cancel () override
	Default common implementation for all plugins.

void	SetBlob (const std::string &name, const Blob::Ptr &userBlob) override
	Given optional implementation of setting blob to avoid need for it to be implemented by plugin. More...

Blob::Ptr	GetBlob (const std::string &name) override
	Given optional implementation of getting blob to avoid need for it to be implemented by plugin. More...

void	SetBlob (const std::string &name, const Blob::Ptr &data, const PreProcessInfo &info) override
	Sets pre-process for input data. More...

const PreProcessInfo &	GetPreProcess (const std::string &name) const override
	Gets pre-process for input data. More...

void	SetBatch (int batch) override
	Sets new batch size when dynamic batching is enabled in executable network that created this request. More...

void	setPointerToExecutableNetworkInternal (std::shared_ptr< ExecutableNetworkInternal > exeNetwork)
	Sets the pointer to executable network internal. More...

virtual void	checkBlobs ()
	Checks that both inputs and outputs blob are valid. Throws an exception if they are not.

std::vector< IVariableStateInternal::Ptr >	QueryState () override
	Queries memory states. More...

Protected Member Functions
virtual void	StartAsyncImpl ()=0
	The minimal asynchronous inference function to be implemented by plugins. It starts inference of specified input(s) in asynchronous mode. More...

Protected Member Functions inherited from InferenceEngine::InferRequestInternal
void	execDataPreprocessing (InferenceEngine::BlobMap &preprocessedBlobs, bool serial=false)
	Checks and executes input data pre-processing if needed. More...

bool	findInputAndOutputBlobByName (const std::string &name, InputInfo::Ptr &foundInput, DataPtr &foundOutput) const
	Helper function to find input or output blob by name. More...

void	checkBlob (const Blob::Ptr &blob, const std::string &name, bool isInput, const SizeVector &refDims={}) const
	Check that `blob` is valid. Throws an exception if it's not. More...

bool	preProcessingRequired (const InputInfo::Ptr &info, const Blob::Ptr &userBlob, const Blob::Ptr &deviceBlob=nullptr)
	Checks whether pre-processing step is required for a given input. More...

void	addInputPreProcessingFor (const std::string &name, Blob::Ptr const &from, const Blob::Ptr &to)

Protected Attributes
IInferRequest::WeakPtr	_publicInterface
	A weak pointer to a IInferRequest interface for callback calling.

InferenceEngine::IInferRequest::CompletionCallback	_callback
	A callback.

void *	_userData
	A callback user data.

Protected Attributes inherited from InferenceEngine::InferRequestInternal
InferenceEngine::InputsDataMap	_networkInputs
	Holds information about network inputs info.

InferenceEngine::OutputsDataMap	_networkOutputs
	Holds information about network outputs data.

InferenceEngine::BlobMap	_inputs
	A map of user passed blobs for network inputs.

InferenceEngine::BlobMap	_deviceInputs
	A map of actual network inputs, in plugin specific format.

InferenceEngine::BlobMap	_outputs
	A map of user passed blobs for network outputs.

std::map< std::string, PreProcessDataPtr >	_preProcData
	A map of pre-process data per input.

int	m_curBatch
	Current batch value used in dynamic batching.

std::shared_ptr< ExecutableNetworkInternal >	_exeNetwork
	A shared pointer to ExecutableNetworkInternal interface. More...

Detailed Description

minimum API to be implemented by plugin, which is used in InferRequestBase forwarding mechanism

Constructor & Destructor Documentation

◆ AsyncInferRequestInternal()

InferenceEngine::AsyncInferRequestInternal::AsyncInferRequestInternal	(	const InputsDataMap &	networkInputs,
		const OutputsDataMap &	networkOutputs
	)

inline

Constructs a new instance.

Parameters

[in]	networkInputs	The network inputs info
[in]	networkOutputs	The network outputs data

Member Function Documentation

◆ GetUserData()

void InferenceEngine::AsyncInferRequestInternal::GetUserData ( void ** data )

inlineoverridevirtual

Get arbitrary data for the request.

Parameters

data	A pointer to a pointer to arbitrary data

Implements InferenceEngine::IAsyncInferRequestInternal.

◆ SetCompletionCallback()

void InferenceEngine::AsyncInferRequestInternal::SetCompletionCallback ( IInferRequest::CompletionCallback callback )

inlineoverridevirtual

Set callback function which will be called on success or failure of asynchronous request.

Parameters

callback - function to be called with the following description:

Implements InferenceEngine::IAsyncInferRequestInternal.

◆ SetPointerToPublicInterface()

void InferenceEngine::AsyncInferRequestInternal::SetPointerToPublicInterface ( IInferRequest::Ptr ptr )

inline

Set weak pointer to the corresponding public interface: IInferRequest. This allow to pass it to IInferRequest::CompletionCallback.

Parameters

ptr	A weak pointer to InferRequestBase

◆ SetUserData()

void InferenceEngine::AsyncInferRequestInternal::SetUserData ( void * data )

inlineoverridevirtual

Set arbitrary data for the request.

Parameters

data	A pointer to a pointer to arbitrary data

Implements InferenceEngine::IAsyncInferRequestInternal.

◆ StartAsync()

void InferenceEngine::AsyncInferRequestInternal::StartAsync ( )

inlineoverridevirtual

Start inference of specified input(s) in asynchronous mode.

Note: The method returns immediately. Inference starts also immediately.

Implements InferenceEngine::IAsyncInferRequestInternal.

◆ StartAsyncImpl()

virtual void InferenceEngine::AsyncInferRequestInternal::StartAsyncImpl ( )

protectedpure virtual

The minimal asynchronous inference function to be implemented by plugins. It starts inference of specified input(s) in asynchronous mode.

Note

The methos is used in AsyncInferRequestInternal::StartAsync which performs common steps first and calls plugin dependent implementation of this method after.
It returns immediately. Inference starts also immediately.

The documentation for this class was generated from the following file:

cpp_interfaces/impl/ie_infer_async_request_internal.hpp

Public Types

Public Member Functions

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ AsyncInferRequestInternal()

Member Function Documentation

◆ GetUserData()

◆ SetCompletionCallback()

◆ SetPointerToPublicInterface()

◆ SetUserData()

◆ StartAsync()

◆ StartAsyncImpl()