System configuration utilities¶

Overview¶

API to get information about the system, core processor capabilities. More…

// global functions

bool InferenceEngine::checkOpenMpEnvVars(bool includeOMPNumThreads = true);
std::vector<int> InferenceEngine::getAvailableNUMANodes();
std::vector<int> InferenceEngine::getAvailableCoresTypes();
int InferenceEngine::getNumberOfCPUCores(bool bigCoresOnly = false);
int InferenceEngine::getNumberOfLogicalCPUCores(bool bigCoresOnly = false);
bool InferenceEngine::with_cpu_x86_sse42();
bool InferenceEngine::with_cpu_x86_avx();
bool InferenceEngine::with_cpu_x86_avx2();
bool InferenceEngine::with_cpu_x86_avx512f();
bool InferenceEngine::with_cpu_x86_avx512_core();
bool InferenceEngine::with_cpu_x86_avx512_core_vnni();
bool InferenceEngine::with_cpu_x86_bfloat16();
bool InferenceEngine::with_cpu_x86_avx512_core_amx_int8();
bool InferenceEngine::with_cpu_x86_avx512_core_amx_bf16();
bool InferenceEngine::with_cpu_x86_avx512_core_amx();

Detailed Documentation¶

API to get information about the system, core processor capabilities.

Global Functions¶

bool InferenceEngine::checkOpenMpEnvVars(bool includeOMPNumThreads = true)

Checks whether OpenMP environment variables are defined.

Parameters:

includeOMPNumThreads

Indicates if the omp number threads is included

Returns:

True if any OpenMP environment variable is defined, false otherwise

std::vector<int> InferenceEngine::getAvailableNUMANodes()

Returns available CPU NUMA nodes (on Linux, and Windows [only with TBB], single node is assumed on all other OSes)

Returns:

NUMA nodes

std::vector<int> InferenceEngine::getAvailableCoresTypes()

Returns available CPU cores types (on Linux, and Windows) and ONLY with TBB, single core type is assumed otherwise.

Returns:

Vector of core types

int InferenceEngine::getNumberOfCPUCores(bool bigCoresOnly = false)

Returns number of CPU physical cores on Linux/Windows (which is considered to be more performance friendly for servers) (on other OSes it simply relies on the original parallel API of choice, which usually uses the logical cores). call function with ‘false’ to get #phys cores of all types call function with ‘true’ to get #phys ‘Big’ cores number of ‘Little’ = ‘all’ - ‘Big’.

Parameters:

bigCoresOnly

Additionally limits the number of reported cores to the ‘Big’ cores only.

Returns:

Number of physical CPU cores.

int InferenceEngine::getNumberOfLogicalCPUCores(bool bigCoresOnly = false)

Returns number of CPU logical cores on Linux/Windows (on other OSes it simply relies on the original parallel API of choice, which uses the ‘all’ logical cores). call function with ‘false’ to get #logical cores of all types call function with ‘true’ to get #logical ‘Big’ cores number of ‘Little’ = ‘all’ - ‘Big’.

Parameters:

bigCoresOnly

Additionally limits the number of reported cores to the ‘Big’ cores only.

Returns:

Number of logical CPU cores.

bool InferenceEngine::with_cpu_x86_sse42()

Checks whether CPU supports SSE 4.2 capability.

Returns:

True is SSE 4.2 instructions are available, false otherwise

bool InferenceEngine::with_cpu_x86_avx()

Checks whether CPU supports AVX capability.

Returns:

True is AVX instructions are available, false otherwise

bool InferenceEngine::with_cpu_x86_avx2()

Checks whether CPU supports AVX2 capability.

Returns:

True is AVX2 instructions are available, false otherwise

bool InferenceEngine::with_cpu_x86_avx512f()

Checks whether CPU supports AVX 512 capability.

Returns:

True is AVX512F (foundation) instructions are available, false otherwise

bool InferenceEngine::with_cpu_x86_avx512_core()

Checks whether CPU supports AVX 512 capability.

Returns:

True is AVX512F, AVX512BW, AVX512DQ instructions are available, false otherwise

bool InferenceEngine::with_cpu_x86_avx512_core_vnni()

Checks whether CPU supports AVX 512 VNNI capability.

Returns:

True is AVX512F, AVX512BW, AVX512DQ, AVX512_VNNI instructions are available, false otherwise

bool InferenceEngine::with_cpu_x86_bfloat16()

Checks whether CPU supports BFloat16 capability.

Returns:

True is tAVX512_BF16 instructions are available, false otherwise

bool InferenceEngine::with_cpu_x86_avx512_core_amx_int8()

Checks whether CPU supports AMX int8 capability.

Returns:

True is tAMX_INT8 instructions are available, false otherwise

bool InferenceEngine::with_cpu_x86_avx512_core_amx_bf16()

Checks whether CPU supports AMX bf16 capability.

Returns:

True is tAMX_BF16 instructions are available, false otherwise

bool InferenceEngine::with_cpu_x86_avx512_core_amx()

Checks whether CPU supports AMX capability.

Returns:

True is tAMX_INT8 or tAMX_BF16 instructions are available, false otherwise