Plugin base classes

Overview

A set of base and helper classes to implement a plugin class. More…

// templates

template ov::ICore;

// global variables

static constexpr Property<std::vector<PropertyName>, PropertyMutability::RO> ov::internal::caching_properties {"CACHING_PROPERTIES"};
static constexpr Property<bool, PropertyMutability::RW> ov::internal::exclusive_async_requests {"EXCLUSIVE_ASYNC_REQUESTS"};
static constexpr Property<std::string, PropertyMutability::WO> ov::internal::config_device_id {"CONFIG_DEVICE_ID"};
static constexpr Property<ov::threading::IStreamsExecutor::ThreadBindingType, PropertyMutability::RW> ov::internal::cpu_bind_thread {     "CPU_BIND_THREAD"};
static constexpr Property<size_t, PropertyMutability::RW> ov::internal::threads_per_stream {"THREADS_PER_STREAM"};

// macros

#define OV_CREATE_PLUGIN
#define OV_DEFINE_PLUGIN_CREATE_FUNCTION(PluginType, version, ...)

Detailed Documentation

A set of base and helper classes to implement a plugin class.

Global Variables

static constexpr Property<std::vector<PropertyName>, PropertyMutability::RO> ov::internal::caching_properties {"CACHING_PROPERTIES"}

Read-only property to get a std::vector<PropertyName> of properties which should affect the hash calculation for model cache.

static constexpr Property<bool, PropertyMutability::RW> ov::internal::exclusive_async_requests {"EXCLUSIVE_ASYNC_REQUESTS"}

Allow to create exclusive_async_requests with one executor.

static constexpr Property<std::string, PropertyMutability::WO> ov::internal::config_device_id {"CONFIG_DEVICE_ID"}

the property for setting of required device for which config to be updated values: device id starts from “0” - first device, “1” - second device, etc note: plugin may have different devices naming convention

static constexpr Property<ov::threading::IStreamsExecutor::ThreadBindingType, PropertyMutability::RW> ov::internal::cpu_bind_thread {     "CPU_BIND_THREAD"}

The name for setting CPU affinity per thread option.

It is passed to Core::get_property() PluginConfigParams::NO (no pinning for CPU inference threads) PluginConfigParams::YES, which is default on the conventional CPUs (pinning threads to cores, best for static benchmarks),

the following options are implemented only for the TBB as a threading option ov::threading::IStreamsExecutor::ThreadBindingType::NUMA (pinning threads to NUMA nodes, best for real-life, contented cases) on the Windows and MacOS* this option behaves as YES ov::threading::IStreamsExecutor::ThreadBindingType::HYBRID_AWARE (let the runtime to do pinning to the cores types, e.g. prefer the “big” cores for latency tasks) on the hybrid CPUs this option is default

Also, the settings are ignored, if the OpenVINO compiled with OpenMP and any affinity-related OpenMP’s environment variable is set (as affinity is configured explicitly)

static constexpr Property<size_t, PropertyMutability::RW> ov::internal::threads_per_stream {"THREADS_PER_STREAM"}

Limit #threads that are used by IStreamsExecutor to execute parallel_for calls.

Macros

#define OV_CREATE_PLUGIN

Defines a name of a function creating plugin instance.

#define OV_DEFINE_PLUGIN_CREATE_FUNCTION(PluginType, version, ...)

Defines the exported OV_CREATE_PLUGIN function which is used to create a plugin instance.