Class Example

class AsyncInferRequestThreadSafeDefault : public IInferRequestInternal

Base class with default implementation of asynchronous multi staged inference request. To customize pipeline stages derived class should change the content of AsyncInferRequestThreadSafeDefault::_pipeline member container. It consists of pairs of tasks and executors which will run the task. The class is recommended to be used by plugins as a base class for asynchronous inference request implementation.

Example

Here is an example of asynchronous inference request implementation for some accelerator device. It uses 5 different executors to run different stages of a synchronous inference request.

Note

To synchronize derived context with stages derived class should call AsyncInferRequestThreadSafeDefault::StopAndWait() function in destructor.

Public Types

using Ptr = std::shared_ptr<AsyncInferRequestThreadSafeDefault>

A shared pointer to AsyncInferRequestThreadSafeDefault.

Public Functions

inline AsyncInferRequestThreadSafeDefault(const IInferRequestInternal::Ptr &request, const ITaskExecutor::Ptr &taskExecutor, const ITaskExecutor::Ptr &callbackExecutor)

Wraps a IInferRequestInternal::Ptr implementation and constructs a AsyncInferRequestThreadSafeDefault::_pipeline where taskExecutor is used to run IInferRequestInternal::Infer asynchronously.

Parameters

• request – [in] The synchronous request

• taskExecutor – [in] The task executor

• callbackExecutor – [in] The callback executor

inline ~AsyncInferRequestThreadSafeDefault()

Destroys the object, stops AsyncInferRequestThreadSafeDefault::_pipeline and waits for a finish.

inline StatusCode Wait(int64_t millis_timeout) override

Waits for completion of all pipeline stages If the pipeline raises an exception it will be rethrown here.

Parameters

millis_timeout – A timeout is ms to wait or special enum value of InferRequest::WaitMode

Returns

A status code