interface ov::threading::ExecutorManager

Overview

Interface for tasks execution manager. This is global point for getting task executor objects by string id. It’s necessary in multiple asynchronous requests for having unique executors to avoid oversubscription. E.g. There 2 task executors for CPU device: one - in FPGA, another - in OneDNN. Parallel execution both of them leads to not optimal CPU usage. More efficient to run the corresponding tasks one by one via single executor. More…

#include <executor_manager.hpp>

template ExecutorManager
{
    // methods

    virtual std::shared_ptr<ov::threading::ITaskExecutor> get_executor(const std::string& id) = 0;
    virtual std::shared_ptr<ov::threading::IStreamsExecutor> get_idle_cpu_streams_executor(const ov::threading::IStreamsExecutor::Config& config) = 0;
    virtual void set_property(const ov::AnyMap& properties) = 0;
    virtual ov::Any get_property(const std::string& name) const = 0;
};

Detailed Documentation

Interface for tasks execution manager. This is global point for getting task executor objects by string id. It’s necessary in multiple asynchronous requests for having unique executors to avoid oversubscription. E.g. There 2 task executors for CPU device: one - in FPGA, another - in OneDNN. Parallel execution both of them leads to not optimal CPU usage. More efficient to run the corresponding tasks one by one via single executor.

Methods

virtual std::shared_ptr<ov::threading::ITaskExecutor> get_executor(const std::string& id) = 0

Returns executor by unique identificator.

Parameters:

id	An unique identificator of device (Usually string representation of TargetDevice)

Returns:

A shared pointer to existing or newly ITaskExecutor

virtual std::shared_ptr<ov::threading::IStreamsExecutor> get_idle_cpu_streams_executor(const ov::threading::IStreamsExecutor::Config& config) = 0

Returns idle cpu streams executor.

Parameters:

config

Streams executor config

Returns:

pointer to streams executor config

virtual void set_property(const ov::AnyMap& properties) = 0

Allows to configure executor manager.

Parameters:

properties

map with configuration

virtual ov::Any get_property(const std::string& name) const = 0

Returns configuration.

Parameters:

name	property name

Returns:

Property value