Wrapper of asynchronous inference request to support thread-safe execution. More...

#include <ie_infer_async_request_thread_safe_internal.hpp>

Inheritance diagram for InferenceEngine::AsyncInferRequestThreadSafeInternal:

Public Types
typedef std::shared_ptr< AsyncInferRequestThreadSafeInternal >	Ptr
	A shared pointer to a AsyncInferRequestThreadSafeInternal implementation.

Public Types inherited from InferenceEngine::IAsyncInferRequestInternal
typedef std::shared_ptr< IAsyncInferRequestInternal >	Ptr
	A shared pointer to IAsyncInferRequestInternal interface.

Public Types inherited from InferenceEngine::IInferRequestInternal
typedef std::shared_ptr< IInferRequestInternal >	Ptr
	A shared pointer to a IInferRequestInternal interface.

Public Member Functions
	AsyncInferRequestThreadSafeInternal ()
	Constructs a new instance.

void	StartAsync () override
	Start inference of specified input(s) in asynchronous mode. More...

void	GetUserData (void **data) override
	Get arbitrary data for the request. More...

void	SetUserData (void *data) override
	Set arbitrary data for the request. More...

void	SetCompletionCallback (IInferRequest::CompletionCallback callback) override
	Set callback function which will be called on success or failure of asynchronous request. More...

void	Infer () override
	Infers specified input(s) in synchronous mode. More...

void	GetPerformanceCounts (std::map< std::string, InferenceEngineProfileInfo > &perfMap) const override
	Queries performance measures per layer to get feedback of what is the most time consuming layer. Note: not all plugins may provide meaningful data. More...

void	SetBlob (const char *name, const Blob::Ptr &data) override
	Set input/output data to infer. More...

void	SetBlob (const char *name, const Blob::Ptr &data, const PreProcessInfo &info) override
	Sets pre-process for input data. More...

void	GetBlob (const char *name, Blob::Ptr &data) override
	Get input/output data to infer. More...

void	GetPreProcess (const char name, const PreProcessInfo *info) const override
	Gets pre-process for input data. More...

void	SetBatch (int batch) override
	Sets new batch size when dynamic batching is enabled in executable network that created this request. More...

Public Member Functions inherited from InferenceEngine::IAsyncInferRequestInternal
virtual	~IAsyncInferRequestInternal ()=default
	A virtual destructor.

virtual StatusCode	Wait (int64_t millis_timeout)=0
	Waits for the result to become available. Blocks until specified millis_timeout has elapsed or the result becomes available, whichever comes first. More...

Public Member Functions inherited from InferenceEngine::IInferRequestInternal
virtual	~IInferRequestInternal ()=default
	Destroys the object.

Protected Member Functions
virtual bool	isRequestBusy () const
	Determines if request busy. More...

virtual bool	setIsRequestBusy (bool isBusy)
	Sets the is request busy. More...

void	CheckBusy () const
	Checks whether an inference request is busy and calls ThrowBusy if `true`

virtual void	StartAsync_ThreadUnsafe ()=0
	Starts an asynchronous pipeline thread unsafe. More...

virtual void	GetUserData_ThreadUnsafe (void **data)=0
	Gets the user data thread unsafe. More...

virtual void	SetUserData_ThreadUnsafe (void *data)=0
	Sets the user data thread unsafe. More...

virtual void	SetCompletionCallback_ThreadUnsafe (IInferRequest::CompletionCallback callback)=0
	Sets the completion callback thread unsafe. More...

virtual void	Infer_ThreadUnsafe ()=0
	Performs inference of pipeline in syncronous mode. More...

virtual void	GetPerformanceCounts_ThreadUnsafe (std::map< std::string, InferenceEngineProfileInfo > &perfMap) const =0
	Gets the performance counts thread unsafe. More...

virtual void	SetBlob_ThreadUnsafe (const char *name, const Blob::Ptr &data)=0
	Sets the blob thread unsafe. More...

virtual void	SetBlob_ThreadUnsafe (const char *name, const Blob::Ptr &data, const PreProcessInfo &info)=0
	Sets the blob with preprocessing information thread unsafe. More...

virtual void	GetBlob_ThreadUnsafe (const char *name, Blob::Ptr &data)=0
	Gets the input or output blob thread unsafe. More...

virtual void	GetPreProcess_ThreadUnsafe (const char name, const PreProcessInfo *info) const =0
	Gets the preprocessing information thread unsafe. More...

virtual void	SetBatch_ThreadUnsafe (int batch)=0
	Sets the dynamic batch thread unsafe. More...

Static Protected Member Functions
static void	ThrowBusy ()
	Throws an exception that an inference request is busy.

Detailed Description

Wrapper of asynchronous inference request to support thread-safe execution.

Member Function Documentation

◆ GetBlob()

void InferenceEngine::AsyncInferRequestThreadSafeInternal::GetBlob	(	const char *	name,
		Blob::Ptr &	data
	)

inlineoverridevirtual

Get input/output data to infer.

Note: Memory allocation doesn't happen

Parameters

name	- a name of input or output blob.
data	- a reference to input or output blob. The type of Blob must correspond to the network input precision and size.

Implements InferenceEngine::IInferRequestInternal.

◆ GetBlob_ThreadUnsafe()

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::GetBlob_ThreadUnsafe	(	const char *	name,
		Blob::Ptr &	data
	)

protectedpure virtual

Gets the input or output blob thread unsafe.

Note: Used by AsyncInferRequestThreadSafeInternal::GetBlob which ensures thread-safety and calls this method after.

Parameters

[in]	name	The name of input / output data to get a blob for
	data	The data

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

◆ GetPerformanceCounts()

void InferenceEngine::AsyncInferRequestThreadSafeInternal::GetPerformanceCounts ( std::map< std::string, InferenceEngineProfileInfo > & perfMap ) const

inlineoverridevirtual

Queries performance measures per layer to get feedback of what is the most time consuming layer. Note: not all plugins may provide meaningful data.

Parameters

perfMap - a map of layer names to profiling information for that layer.

Implements InferenceEngine::IInferRequestInternal.

◆ GetPerformanceCounts_ThreadUnsafe()

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::GetPerformanceCounts_ThreadUnsafe ( std::map< std::string, InferenceEngineProfileInfo > & perfMap ) const

protectedpure virtual

Gets the performance counts thread unsafe.

Note: Used by AsyncInferRequestThreadSafeInternal::GetPerformanceCounts which ensures thread-safety and calls this method after.

Parameters

perfMap The performance map

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

◆ GetPreProcess()

void InferenceEngine::AsyncInferRequestThreadSafeInternal::GetPreProcess	(	const char *	name,
		const PreProcessInfo **	info
	)		const

inlineoverridevirtual

Gets pre-process for input data.

Parameters

name	Name of input blob.
info	pointer to a pointer to PreProcessInfo structure

Implements InferenceEngine::IInferRequestInternal.

◆ GetPreProcess_ThreadUnsafe()

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::GetPreProcess_ThreadUnsafe	(	const char *	name,
		const PreProcessInfo **	info
	)		const

protectedpure virtual

Gets the preprocessing information thread unsafe.

Note: Used by AsyncInferRequestThreadSafeInternal::GetPreProcess which ensures thread-safety and calls this method after.

Parameters

[in]	name	The name of input / output data to get a processing information for
	info	The preprocessing information

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

◆ GetUserData()

void InferenceEngine::AsyncInferRequestThreadSafeInternal::GetUserData ( void ** data )

inlineoverridevirtual

Get arbitrary data for the request.

Parameters

data	A pointer to a pointer to arbitrary data

Implements InferenceEngine::IAsyncInferRequestInternal.

◆ GetUserData_ThreadUnsafe()

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::GetUserData_ThreadUnsafe ( void ** data )

protectedpure virtual

Gets the user data thread unsafe.

Note: Used by AsyncInferRequestThreadSafeInternal::GetUserData which ensures thread-safety and calls this method after.

Parameters

data	The user data

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

◆ Infer()

void InferenceEngine::AsyncInferRequestThreadSafeInternal::Infer ( )

inlineoverridevirtual

Infers specified input(s) in synchronous mode.

Note: blocks all method of IInferRequest while request is ongoing (running or waiting in queue)

Implements InferenceEngine::IInferRequestInternal.

◆ Infer_ThreadUnsafe()

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::Infer_ThreadUnsafe ( )

protectedpure virtual

Performs inference of pipeline in syncronous mode.

Note: Used by AsyncInferRequestThreadSafeInternal::Infer which ensures thread-safety and calls this method after.

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

◆ isRequestBusy()

virtual bool InferenceEngine::AsyncInferRequestThreadSafeInternal::isRequestBusy ( ) const

inlineprotectedvirtual

Determines if request busy.

Returns: True if request busy, false otherwise.

◆ SetBatch()

void InferenceEngine::AsyncInferRequestThreadSafeInternal::SetBatch ( int batch )

inlineoverridevirtual

Sets new batch size when dynamic batching is enabled in executable network that created this request.

Parameters

batch - new batch size to be used by all the following inference calls for this request.

Implements InferenceEngine::IInferRequestInternal.

◆ SetBatch_ThreadUnsafe()

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::SetBatch_ThreadUnsafe ( int batch )

protectedpure virtual

Sets the dynamic batch thread unsafe.

Note: Used by AsyncInferRequestThreadSafeInternal::SetBatch which ensures thread-safety and calls this method after.

Parameters

[in] batch The dynamic batch value

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

◆ SetBlob() [1/2]

void InferenceEngine::AsyncInferRequestThreadSafeInternal::SetBlob	(	const char *	name,
		const Blob::Ptr &	data
	)

inlineoverridevirtual

Set input/output data to infer.

Note: Memory allocation doesn't happen

Parameters

name	- a name of input or output blob.
data	- a reference to input or output blob. The type of Blob must correspond to the network input precision and size.

Implements InferenceEngine::IInferRequestInternal.

◆ SetBlob() [2/2]

void InferenceEngine::AsyncInferRequestThreadSafeInternal::SetBlob	(	const char *	name,
		const Blob::Ptr &	data,
		const PreProcessInfo &	info
	)

inlineoverridevirtual

Sets pre-process for input data.

Parameters

name	Name of input blob.
data	- a reference to input or output blob. The type of Blob must correspond to the network input precision and size.
info	Preprocess info for blob.

Implements InferenceEngine::IInferRequestInternal.

◆ SetBlob_ThreadUnsafe() [1/2]

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::SetBlob_ThreadUnsafe	(	const char *	name,
		const Blob::Ptr &	data
	)

protectedpure virtual

Sets the blob thread unsafe.

Note: Used by AsyncInferRequestThreadSafeInternal::SetBlob which ensures thread-safety and calls this method after.

Parameters

[in]	name	The name of input / output data to set a blob to
[in]	data	The blob to set

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

◆ SetBlob_ThreadUnsafe() [2/2]

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::SetBlob_ThreadUnsafe	(	const char *	name,
		const Blob::Ptr &	data,
		const PreProcessInfo &	info
	)

protectedpure virtual

Sets the blob with preprocessing information thread unsafe.

Note: Used by AsyncInferRequestThreadSafeInternal::SetBlob which ensures thread-safety and calls this method after.

Parameters

[in]	name	The name of input / output data to set a blob to
[in]	data	The blob to set
[in]	info	The preprocessing information

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

◆ SetCompletionCallback()

void InferenceEngine::AsyncInferRequestThreadSafeInternal::SetCompletionCallback ( IInferRequest::CompletionCallback callback )

inlineoverridevirtual

Set callback function which will be called on success or failure of asynchronous request.

Parameters

callback - function to be called with the following description:

Implements InferenceEngine::IAsyncInferRequestInternal.

◆ SetCompletionCallback_ThreadUnsafe()

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::SetCompletionCallback_ThreadUnsafe ( IInferRequest::CompletionCallback callback )

protectedpure virtual

Sets the completion callback thread unsafe.

Note: Used by AsyncInferRequestThreadSafeInternal::SetCompletionCallback which ensures thread-safety and calls this method after.

Parameters

[in] callback The callback to set

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

◆ setIsRequestBusy()

virtual bool InferenceEngine::AsyncInferRequestThreadSafeInternal::setIsRequestBusy ( bool isBusy )

inlineprotectedvirtual

Sets the is request busy.

Parameters

[in] isBusy Indicates if busy

Returns: True is case of success, false otherwise.

◆ SetUserData()

void InferenceEngine::AsyncInferRequestThreadSafeInternal::SetUserData ( void * data )

inlineoverridevirtual

Set arbitrary data for the request.

Parameters

data	A pointer to a pointer to arbitrary data

Implements InferenceEngine::IAsyncInferRequestInternal.

◆ SetUserData_ThreadUnsafe()

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::SetUserData_ThreadUnsafe ( void * data )

protectedpure virtual

Sets the user data thread unsafe.

Note: Used by AsyncInferRequestThreadSafeInternal::SetUserData which ensures thread-safety and calls this method after.

Parameters

data	The user data

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

◆ StartAsync()

void InferenceEngine::AsyncInferRequestThreadSafeInternal::StartAsync ( )

inlineoverridevirtual

Start inference of specified input(s) in asynchronous mode.

Note: The method returns immediately. Inference starts also immediately.

Implements InferenceEngine::IAsyncInferRequestInternal.

◆ StartAsync_ThreadUnsafe()

virtual void InferenceEngine::AsyncInferRequestThreadSafeInternal::StartAsync_ThreadUnsafe ( )

protectedpure virtual

Starts an asynchronous pipeline thread unsafe.

Note: Used by AsyncInferRequestThreadSafeInternal::StartAsync which ensures thread-safety and calls this method after.

Implemented in InferenceEngine::AsyncInferRequestThreadSafeDefault.

The documentation for this class was generated from the following file:

cpp_interfaces/impl/ie_infer_async_request_thread_safe_internal.hpp

Public Types

Public Member Functions

Protected Member Functions

Static Protected Member Functions

Detailed Description

Member Function Documentation

◆ GetBlob()

◆ GetBlob_ThreadUnsafe()

◆ GetPerformanceCounts()

◆ GetPerformanceCounts_ThreadUnsafe()

◆ GetPreProcess()

◆ GetPreProcess_ThreadUnsafe()

◆ GetUserData()

◆ GetUserData_ThreadUnsafe()

◆ Infer()

◆ Infer_ThreadUnsafe()

◆ isRequestBusy()

◆ SetBatch()

◆ SetBatch_ThreadUnsafe()

◆ SetBlob() [1/2]

◆ SetBlob() [2/2]

◆ SetBlob_ThreadUnsafe() [1/2]

◆ SetBlob_ThreadUnsafe() [2/2]

◆ SetCompletionCallback()

◆ SetCompletionCallback_ThreadUnsafe()

◆ setIsRequestBusy()

◆ SetUserData()

◆ SetUserData_ThreadUnsafe()

◆ StartAsync()

◆ StartAsync_ThreadUnsafe()