interface InferenceEngine::IStreamsExecutor

Overview

Interface for Streams Task Executor. This executor groups worker threads into so-called streams. More…

#include <ie_istreams_executor.hpp>

template IStreamsExecutor:
    public InferenceEngine::ITaskExecutor,
    public ov::threading::IStreamsExecutor
{
    // typedefs

    typedef std::shared_ptr<IStreamsExecutor> Ptr;

    // structs

    struct Config;

    // methods

    virtual int GetStreamId() = 0;
    virtual int GetNumaNodeId() = 0;
    virtual void Execute(Task task) = 0;
    virtual int get_stream_id();
    virtual int get_numa_node_id();
    virtual void execute(Task task);
};

// direct descendants

class CPUStreamsExecutor;
class TBBStreamsExecutor;

Inherited Members

public:
    // typedefs

    typedef std::shared_ptr<ITaskExecutor> Ptr;

    // enums

    enum ThreadBindingType;

    // structs

    struct Config;

    // methods

    virtual void run(Task task) = 0;
    virtual void run_and_wait(const std::vector<Task>& tasks);
    virtual void runAndWait(const std::vector<Task>& tasks);
    virtual int get_stream_id() = 0;
    virtual int get_numa_node_id() = 0;
    virtual void execute(Task task) = 0;

Detailed Documentation

Interface for Streams Task Executor. This executor groups worker threads into so-called streams.

The executor executes all parallel tasks using threads from one stream. With proper pinning settings it should reduce cache misses for memory bound workloads.

On NUMA hosts GetNumaNodeId() method can be used to define the NUMA node of current stream

Typedefs

typedef std::shared_ptr<IStreamsExecutor> Ptr

A shared pointer to IStreamsExecutor interface

Methods

virtual int GetStreamId() = 0

Return the index of current stream.

Returns:

An index of current stream. Or throw exceptions if called not from stream thread

virtual int GetNumaNodeId() = 0

Return the id of current NUMA Node.

Returns:

ID of current NUMA Node, or throws exceptions if called not from stream thread

virtual void Execute(Task task) = 0

Execute the task in the current thread using streams executor configuration and constraints.

Parameters:

task	A task to start

virtual int get_stream_id()

Return the index of current stream.

Returns:

An index of current stream. Or throw exceptions if called not from stream thread

virtual int get_numa_node_id()

Return the id of current NUMA Node.

Returns:

ID of current NUMA Node, or throws exceptions if called not from stream thread

virtual void execute(Task task)

Execute the task in the current thread using streams executor configuration and constraints.

Parameters:

task	A task to start