Struct ov::threading::IStreamsExecutor::Config

struct Config

Defines IStreamsExecutor configuration.

Public Functions

void set_property(const ov::AnyMap &properties)

Sets configuration.

Parameters

properties – map of properties

void set_property(const std::string &key, const ov::Any &value)

Sets configuration.

Parameters

• key – property name

• value – property value

ov::Any get_property(const std::string &key) const

Return configuration value.

Parameters

key – configuration key

Returns

configuration value wrapped into ov::Any

inline Config(std::string name = "StreamsExecutor", int streams = 1, int threadsPerStream = 0, ThreadBindingType threadBindingType = ThreadBindingType::NONE, int threadBindingStep = 1, int threadBindingOffset = 0, int threads = 0, PreferredCoreType threadPreferredCoreType = PreferredCoreType::ANY, std::vector<std::vector<int>> streamsInfoTable = {}, bool cpuReservation = false)

A constructor with arguments.

Parameters

• name – [in] The executor name

• streams – [in] Number of streams.

• threadsPerStream – [in] Number of threads per stream that executes ov_parallel calls.

• threadBindingType – [in]

• threadBindingStep – [in]

• threadBindingOffset – [in]

• threads – [in]

• threadPreferBigCores – [in]

void update_executor_config(int stream_nums, int threads_per_stream, PreferredCoreType core_type, bool cpu_pinning)

Modify _streams_info_table and related configuration according to user-specified parameters, bind threads to cpu cores if cpu_pinning is true.

Parameters

• stream_nums – Number of streams specified by user

• threads_per_stream – Number of threads per stream specified by user

• core_type – Cpu type (Big/Little/Any) specified by user

• cpu_pinning – Whether to bind the threads to cpu cores

void set_config_zero_stream()

Set _streams_info_table and _cpu_reservation in cpu streams executor config when nstreams = 0, that is, only create one thread with TBB.

Public Members

std::string _name

Used by ITT to name executor threads.

int _streams = 1

Number of streams.

int _threadsPerStream = 0

Number of threads per stream that executes ov_parallel calls.

ThreadBindingType _threadBindingType = ThreadBindingType::NONE

Thread binding to hardware resource type. No binding by default

int _threadBindingStep = 1

In case of CORES binding offset type thread binded to cores with defined step

int _threadBindingOffset = 0

In case of CORES binding offset type thread binded to cores starting from offset

int _threads = 0

Number of threads distributed between streams. Reserved. Should not be used.

int _big_core_streams = 0

Number of streams in Performance-core(big core)

int _small_core_streams = 0

Number of streams in Efficient-core(small core)

int _threads_per_stream_big = 0

Threads per stream in big cores.

int _threads_per_stream_small = 0

Threads per stream in small cores.

int _small_core_offset = 0

Calculate small core start offset when binding cpu cores.

bool _enable_hyper_thread = true

enable hyper thread

enum ov::threading::IStreamsExecutor::Config::PreferredCoreType _threadPreferredCoreType = PreferredCoreType::ANY

In case of HYBRID_AWARE hints the TBB to affinitize.

Public Static Functions

static Config make_default_multi_threaded(const Config &initial, const bool fp_intesive = true)

Create appropriate multithreaded configuration filing unconfigured values from initial configuration using hardware properties.

Parameters

• initial – Inital configuration

• fp_intesive – additional hint for the the (Hybrid) core-types selection logic whether the executor should be configured for floating point intensive work (as opposite to int8 intensive)

Returns

configured values