Group System configuration utilities

group ov_dev_api_system_conf

API to get information about the system, core processor capabilities.

Functions

bool check_open_mp_env_vars(bool include_omp_num_threads = true)

Checks whether OpenMP environment variables are defined.

Parameters:

include_omp_num_threads – [in] Indicates if the omp number threads is included

Returns:

True if any OpenMP environment variable is defined, false otherwise

std::vector<int> get_available_numa_nodes()

Returns available CPU NUMA nodes (on Linux, and Windows [only with TBB], single node is assumed on all other OSes)

Returns:

NUMA nodes

std::vector<int> get_available_cores_types()

Returns available CPU cores types (on Linux, and Windows) and ONLY with TBB, single core type is assumed otherwise.

Returns:

Vector of core types

int get_number_of_cpu_cores(bool big_cores_only = false)

Returns number of CPU physical cores on Linux/Windows (which is considered to be more performance friendly for servers) (on other OSes it simply relies on the original parallel API of choice, which usually uses the logical cores). call function with ‘false’ to get #phys cores of all types call function with ‘true’ to get #phys ‘Big’ cores number of ‘Little’ = ‘all’ - ‘Big’.

Parameters:

big_cores_only – [in] Additionally limits the number of reported cores to the ‘Big’ cores only.

Returns:

Number of physical CPU cores.

int get_number_of_logical_cpu_cores(bool big_cores_only = false)

Returns number of CPU logical cores on Linux/Windows (on other OSes it simply relies on the original parallel API of choice, which uses the ‘all’ logical cores). call function with ‘false’ to get #logical cores of all types call function with ‘true’ to get #logical ‘Big’ cores number of ‘Little’ = ‘all’ - ‘Big’.

Parameters:

big_cores_only – [in] Additionally limits the number of reported cores to the ‘Big’ cores only.

Returns:

Number of logical CPU cores.

int get_number_of_blocked_cores()

Returns number of blocked CPU cores. Please note that this is a temporary interface for performance optimization on a specific platform. May be removed in future release.

Returns:

Number of blocked CPU cores.

bool with_cpu_x86_sse42()

Checks whether CPU supports SSE 4.2 capability.

Returns:

True is SSE 4.2 instructions are available, false otherwise

bool with_cpu_neon_fp16()

Checks whether CPU supports ARM NEON FP16 capability.

Returns:

True is ARM NEON FP16 instructions are available, false otherwise

bool with_cpu_sve()

Checks whether CPU supports ARM SVE capability.

Returns:

True if ARM SVE instructions are available, false otherwise

bool with_cpu_x86_avx()

Checks whether CPU supports AVX capability.

Returns:

True is AVX instructions are available, false otherwise

bool with_cpu_x86_avx2()

Checks whether CPU supports AVX2 capability.

Returns:

True is AVX2 instructions are available, false otherwise

bool with_cpu_x86_avx2_vnni()

Checks whether CPU supports AVX2_VNNI capability.

Returns:

True is AVX2_VNNI instructions are available, false otherwise

bool with_cpu_x86_avx512f()

Checks whether CPU supports AVX 512 capability.

Returns:

True is AVX512F (foundation) instructions are available, false otherwise

bool with_cpu_x86_avx512_core()

Checks whether CPU supports AVX 512 capability.

Returns:

True is AVX512F, AVX512BW, AVX512DQ instructions are available, false otherwise

bool with_cpu_x86_avx512_core_vnni()

Checks whether CPU supports AVX 512 VNNI capability.

Returns:

True is AVX512F, AVX512BW, AVX512DQ, AVX512_VNNI instructions are available, false otherwise

bool with_cpu_x86_bfloat16()

Checks whether CPU supports BFloat16 capability.

Returns:

True is tAVX512_BF16 instructions are available, false otherwise

bool with_cpu_x86_avx512_core_fp16()

Checks whether CPU supports fp16 capability.

Returns:

True is tAVX512_FP16 instructions are available, false otherwise

bool with_cpu_x86_avx512_core_amx_int8()

Checks whether CPU supports AMX int8 capability.

Returns:

True is tAMX_INT8 instructions are available, false otherwise

bool with_cpu_x86_avx512_core_amx_bf16()

Checks whether CPU supports AMX bf16 capability.

Returns:

True is tAMX_BF16 instructions are available, false otherwise

bool with_cpu_x86_avx512_core_amx_fp16()

Checks whether CPU supports AMX fp16 capability.

Returns:

True is tAMX_FP16 instructions are available, false otherwise

bool with_cpu_x86_avx512_core_amx()

Checks whether CPU supports AMX capability.

Returns:

True is tAMX_INT8 or tAMX_BF16 instructions are available, false otherwise

bool is_cpu_map_available()

Checks whether cpu_mapping Available.

Returns:

True is CPU mapping is available, false otherwise

int get_num_numa_nodes()

Get number of numa nodes.

Returns:

Number of numa nodes

int get_num_sockets()

Get number of sockets.

Returns:

Number of sockets

std::vector<std::vector<int>> get_proc_type_table()

Returns a table of number of processor types on Linux/Windows.

1. Processor table of one socket CPU desktop ALL_PROC | MAIN_CORE_PROC | EFFICIENT_CORE_PROC | HYPER_THREADING_PROC 32 8 16 8 // Total number of one socket

Returns:

A table about number of CPU cores of different types defined with ColumnOfProcessorTypeTable The following are two example of processor type table.

1. Processor table of two socket CPUs XEON server ALL_PROC | MAIN_CORE_PROC | EFFICIENT_CORE_PROC | HYPER_THREADING_PROC 96 48 0 48 // Total number of two sockets 48 24 0 24 // Number of socket one 48 24 0 24 // Number of socket two

int get_current_socket_id()

Returns the socket ID in cpu mapping table of the currently running thread.

Returns:

socket ID in cpu mapping

std::vector<std::vector<int>> get_org_proc_type_table()

Returns a table of original number of processor types without filtering other plugins occupying CPU resources. The difference from get_proc_type_table: This is used to get the configuration of current machine. For example, GPU plugin occupies all Pcores, there is only one type core in proc_type_table from get_proc_type_table(). If user wants to get the real configuration of this machine which should be got from get_org_proc_type_table.

Returns:

A table about number of CPU cores of different types defined with ColumnOfProcessorTypeTable

void reserve_available_cpus(const std::vector<std::vector<int>> streams_info_table, std::vector<std::vector<int>> &stream_processors, const int cpu_status = NOT_USED)

Get and reserve available cpu ids.

Parameters:

• streams_info_table – [in] streams information table.

• stream_processors – [in] processors grouped in stream which is used in core binding in cpu streams executor

• cpu_status – [in] set cpu status

void set_cpu_used(const std::vector<int> &cpu_ids, const int used)

Set CPU_MAP_USED_FLAG of cpu_mapping.

Parameters:

• cpu_ids – [in] cpus in cpu_mapping.

• used – [in] update CPU_MAP_USED_FLAG of cpu_mapping with this flag bit

int get_socket_by_numa_node(int numa_node_id)

Get socket id by current numa node id.

Parameters:

numa_node_id – [in] numa node id

Returns:

socket id

int get_org_socket_id(int socket_id)

Get original socket id by current socket id, the input socket id is recalculated after filtering (like numactl), while the original socket id is the original id before filtering.

Parameters:

socket_id – [in] socket id

Returns:

socket id

int get_org_numa_id(int numa_node_id)

Get original numa node id by current numa node id, the input numa node id is recalculated after filtering (like numactl), while the original numa node id is the original id before filtering.

Parameters:

numa_node_id – [in] numa node id

Returns:

numa node id