Group Intel NPU LevelZero interoperability#

group ov_runtime_level_zero_npu_cpp_api

Set of C++ classes and properties to work with Remote API for Intel NPU LevelZero plugin.

Enums

enum class MemType#

Enum to define the type of the shared memory buffer.

Values:

enumerator L0_INTERNAL_BUF#

Internal Level Zero buffer type allocated by plugin.

enumerator SHARED_BUF#

Shared buffer.

enum class TensorType#

Enum to define the type of the tensor.

Values:

enumerator INPUT#

Tensor is only used as input.

enumerator OUTPUT#

Tensor is only used as output.

enumerator BINDED#

Tensor could be used as input and output.

Variables

static constexpr Property<MemType> mem_type = {"MEM_TYPE"}#

This key identifies type of internal shared memory in a shared memory tensor parameter map.

static constexpr Property<npu_handle_param> mem_handle = {"MEM_HANDLE"}#

This key identifies memory handle in a shared memory tensor parameter map.

static constexpr Property<npu_handle_param> l0_context = {"L0_CONTEXT"}#

This key identifies LevelZero context handle in a shared context parameter map.

static constexpr Property<TensorType> tensor_type = {"TENSOR_TYPE"}#

This key sets the type of the internal Level Zero buffer allocated by the plugin in a shared memory tensor parameter map.

class ZeroBufferTensor : public ov::RemoteTensor#
#include <level_zero.hpp>

This class represents an abstraction for NPU plugin remote tensor which can be shared with user-supplied LevelZero buffer. The plugin object derived from this class can be obtained with ZeroContext::create_tensor() call.

Note

User can obtain Level Zero buffer handle from this class.

Public Functions

inline void *get()#

Returns the underlying LevelZero memory object handle.

Returns:

underlying void* memory object handle

Public Static Functions

static inline void type_check(const Tensor &tensor)#

Checks that type defined runtime parameters are presented in remote object.

Parameters:

tensor – a tensor to check

class ZeroContext : public ov::RemoteContext#
#include <level_zero.hpp>

This class represents an abstraction for NPU plugin remote context which is shared with LevelZero context object. The plugin object derived from this class can be obtained either with CompiledModel::get_context() or Core::create_context() calls.

Public Functions

inline ZeroContext(Core &core)#

Constructs context object from user-supplied LevelZero context handle.

Parameters:

core – A reference to OpenVINO Runtime Core object

inline void *get()#

Returns the underlying LevelZero context handle.

Returns:

void*

inline ZeroBufferTensor create_tensor(const element::Type type, const Shape &shape, void *buffer)#

This function is used to obtain remote tensor object from user-supplied NT handle object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • buffer – A void* object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline ZeroBufferTensor create_tensor(const element::Type type, const Shape &shape, int fd)#

This function is used to obtain remote tensor object from user-supplied DMA-BUF System Heap object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • fd – A int object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline ZeroBufferTensor create_l0_host_tensor(const element::Type type, const Shape &shape, const TensorType tensor_type = TensorType::BINDED)#

This function is used to obtain remote tensor object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • tensor_type – Type of the tensor to be shared, input, output or binded

Returns:

A remote tensor instance

RemoteTensor create_tensor(const element::Type &type, const Shape &shape, const AnyMap &params = {})#

Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.

Parameters:

  • type – Defines the element type of the tensor.

  • shape – Defines the shape of the tensor.

  • params – Map of the low-level tensor object parameters.

Returns:

Pointer to a plugin object that implements the RemoteTensor interface.