interface InferenceEngine::ITaskExecutor

Overview

Interface for Task Executor. Inference Engine uses InferenceEngine::ITaskExecutor interface to run all asynchronous internal tasks. Different implementations of task executors can be used for different purposes: More…

#include <ie_itask_executor.hpp>

template ITaskExecutor: public ov::threading::ITaskExecutor
{
    // typedefs

    typedef std::shared_ptr<ITaskExecutor> Ptr;

    // methods

    virtual void runAndWait(const std::vector<Task>& tasks);
};

// direct descendants

template IStreamsExecutor;
class ImmediateExecutor;

Inherited Members

public:
    // methods

    virtual void run(Task task) = 0;
    virtual void run_and_wait(const std::vector<Task>& tasks);

Detailed Documentation

Interface for Task Executor. Inference Engine uses InferenceEngine::ITaskExecutor interface to run all asynchronous internal tasks. Different implementations of task executors can be used for different purposes:

• To improve cache locality of memory bound CPU tasks some executors can limit task’s affinity and maximum concurrency.

• The executor with one worker thread can be used to serialize access to acceleration device.

• Immediate task executor can be used to satisfy InferenceEngine::ITaskExecutor interface restrictions but run tasks in current thread. Implementation should guaranty thread safety of all methods It is InferenceEngine::ITaskExecutor user responsibility to wait for task execution completion. The c++11 standard way to wait task completion is to use std::packaged_task or std::promise with std::future. Here is an example of how to use std::promise to wait task completion and process task’s exceptions:

  // std::promise is move only object so to satisfy copy callable constraint we use std::shared_ptr
  auto promise = std::make_shared<std::promise<void>>();
  // When the promise is created we can get std::future to wait the result
  auto future = promise->get_future();
  // Rather simple task
  InferenceEngine::Task task = [] {std::cout << "Some Output" << std::endl; };
  // Create an executor
  InferenceEngine::ITaskExecutor::Ptr taskExecutor = std::make_shared<InferenceEngine::CPUStreamsExecutor>();
  if (taskExecutor == nullptr) {
      // ProcessError(e);
      return;
  }
  // We capture the task and the promise. When the task is executed in the task executor context
  // we munually call std::promise::set_value() method
  taskExecutor->run([task, promise] {
      std::exception_ptr currentException;
      try {
          task();
      } catch(...) {
          // If there is some exceptions store the pointer to current exception
          currentException = std::current_exception();
      }
  
      if (nullptr == currentException) {
          promise->set_value();  //  <-- If there is no problems just call std::promise::set_value()
      } else {
          promise->set_exception(currentException);    //  <-- If there is an exception forward it to std::future object
      }
  });
  // To wait the task completion we call std::future::wait method
  future.wait();  //  The current thread will be blocked here and wait when std::promise::set_value()
                  //  or std::promise::set_exception() method will be called.
  
  // If the future store the exception it will be rethrown in std::future::get method
  try {
      future.get();
  } catch(std::exception& /\*e\*/) {
      // ProcessError(e);
  }

Typedefs

typedef std::shared_ptr<ITaskExecutor> Ptr

A shared pointer to ITaskExecutor interface

Methods

virtual void runAndWait(const std::vector<Task>& tasks)

Execute all of the tasks and waits for its completion. Default runAndWait() method implementation uses run() pure virtual method and higher level synchronization primitives from STL. The task is wrapped into std::packaged_task which returns std::future. std::packaged_task will call the task and signal to std::future that the task is finished or the exception is thrown from task Than std::future is used to wait for task execution completion and task exception extraction.

runAndWait() does not copy or capture tasks!

Parameters:

tasks

A vector of tasks to execute