Group Intel GPU OpenCL interoperability#

group ov_runtime_ocl_gpu_cpp_api

Set of C++ classes and properties to work with Remote API for Intel GPU OpenCL plugin.

Typedefs

using gpu_handle_param = void*#

Shortcut for defining a handle parameter.

Enums

enum class ContextType#

Enum to define the type of the shared context.

Values:

enumerator OCL#

Pure OpenCL context.

enumerator VA_SHARED#

Context shared with a video decoding device.

enum class SharedMemType#

Enum to define the type of the shared memory buffer.

Values:

enumerator OCL_BUFFER#

Shared OpenCL buffer blob.

enumerator OCL_IMAGE2D#

Shared OpenCL 2D image blob.

enumerator USM_USER_BUFFER#

Shared USM pointer allocated by user.

enumerator USM_HOST_BUFFER#

Shared USM pointer type with host allocation type allocated by plugin.

enumerator USM_DEVICE_BUFFER#

Shared USM pointer type with device allocation type allocated by plugin.

enumerator VA_SURFACE#

Shared video decoder surface or D3D 2D texture blob.

enumerator DX_BUFFER#

Shared D3D buffer blob.

Variables

static constexpr Property<ContextType> context_type = {"CONTEXT_TYPE"}#

Shared device context type: can be either pure OpenCL (OCL) or shared video decoder (VA_SHARED) context.

static constexpr Property<gpu_handle_param> ocl_context = {"OCL_CONTEXT"}#

This key identifies OpenCL context handle in a shared context or shared memory blob parameter map.

static constexpr Property<int> ocl_context_device_id = {"OCL_CONTEXT_DEVICE_ID"}#

This key identifies ID of device in OpenCL context if multiple devices are present in the context.

static constexpr Property<int> tile_id = {"TILE_ID"}#

In case of multi-tile system, this key identifies tile within given context.

static constexpr Property<gpu_handle_param> ocl_queue = {"OCL_QUEUE"}#

This key identifies OpenCL queue handle in a shared context.

static constexpr Property<gpu_handle_param> va_device = {"VA_DEVICE"}#

This key identifies video acceleration device/display handle in a shared context or shared memory blob parameter map.

static constexpr Property<SharedMemType> shared_mem_type = {"SHARED_MEM_TYPE"}#

This key identifies type of internal shared memory in a shared memory blob parameter map.

static constexpr Property<gpu_handle_param> mem_handle = {"MEM_HANDLE"}#

This key identifies OpenCL memory handle in a shared memory blob parameter map.

static constexpr Property<gpu_handle_param> dev_object_handle = {"DEV_OBJECT_HANDLE"}#

This key identifies video decoder surface handle in a shared memory blob parameter map.

static constexpr Property<uint32_t> va_plane = {"VA_PLANE"}#

This key identifies video decoder surface plane in a shared memory blob parameter map.

class D3DBufferTensor : public ov::intel_gpu::ocl::ClBufferTensor
#include <dx.hpp>

This class represents an abstraction for GPU plugin remote tensor which is shared with Direct3D 11 buffer. The plugin object derived from this class can be obtained with D3DContext::create_tensor() call.

Note

User can also obtain OpenCL buffer handle from this class.

Public Functions

inline operator ID3D11Buffer*()

ID3D11Buffer conversion operator for the D3DContext object.

Returns:

Pointer to underlying ID3D11Buffer interface

Public Static Functions

static inline void type_check(const Tensor &tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters:

tensor – a tensor to check

class D3DSurface2DTensor : public ov::intel_gpu::ocl::ClImage2DTensor
#include <dx.hpp>

This class represents an abstraction for GPU plugin remote tensor which is shared with Direct3D 11 2D texture. The plugin object derived from this class can be obtained with D3DContext::create_tensor() call.

Note

User can also obtain OpenCL 2D image handle from this class.

Public Functions

inline operator ID3D11Texture2D*()

ID3D11Texture2D conversion operator for the D3DContext object.

Returns:

Pointer to underlying ID3D11Texture2D interface

inline uint32_t plane()

Returns plane ID of underlying video decoder surface, or 0 if no video surface was shared.

Returns:

Plane ID

Public Static Functions

static inline void type_check(const Tensor &remote_tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters:

remote_tensor – remote tensor to check

class D3DContext : public ov::intel_gpu::ocl::ClContext
#include <dx.hpp>

This class represents an abstraction for GPU plugin remote context which is shared with Direct3D 11 device. The plugin object derived from this class can be obtained either with CompiledModel::get_context() or Core::create_context() calls.

Note

User can also obtain OpenCL context handle from this class.

Public Functions

inline operator ID3D11Device*()

ID3D11Device conversion operator for the D3DContext object.

Returns:

Pointer to underlying ID3D11Device interface

inline D3DContext(Core &core, ID3D11Device *device, int target_tile_id = -1)

Constructs D3DContext remote context object from ID3D11Device.

Parameters:

  • core – OpenVINO Runtime Core object instance

  • device – A pointer to ID3D11Device to be used to create a remote context

  • target_tile_id – Desired tile id within given context for multi-tile system. Default value (-1) means that root device should be used

inline std::pair<D3DSurface2DTensor, D3DSurface2DTensor> create_tensor_nv12(const size_t height, const size_t width, ID3D11Texture2D *nv12_surf)

This function is used to obtain a NV12 tensor from NV12 DXGI video decoder output. The resulting tensor contains two remote tensors for Y and UV planes of the surface.

Parameters:

  • height – Height of Y plane

  • width – Width of Y plane

  • nv12_surf – A ID3D11Texture2D instance to create NV12 tensor from

Returns:

A pair of remote tensors for each plane

inline D3DBufferTensor create_tensor(const element::Type type, const Shape &shape, ID3D11Buffer *buffer)

This function is used to obtain remote tensor object from ID3D11Buffer.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • buffer – A pointer to ID3D11Buffer instance to create remote tensor based on

Returns:

A remote tensor instance

inline D3DSurface2DTensor create_tensor(const element::Type type, const Shape &shape, ID3D11Texture2D *surface, uint32_t plane = 0)

This function is used to obtain remote tensor object from ID3D11Texture2D.

Note

The underlying ID3D11Texture2D can also be a plane of output surface of DXGI video decoder

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • surface – Pointer to ID3D11Texture2D interface of the objects that owns NV12 texture

  • plane – ID of the plane to be shared (0 or 1)

Returns:

D3DSurface2DTensor tensor

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl_mem buffer)

This function is used to obtain remote tensor object from user-supplied cl_mem object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • buffer – A cl_mem object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl::Buffer &buffer)

This function is used to obtain remote tensor object from user-supplied cl::Buffer object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • buffer – A cl::Buffer object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline ClImage2DTensor create_tensor(const element::Type type, const Shape &shape, const cl::Image2D &image)

This function is used to obtain remote tensor object from user-supplied cl::Image2D object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • image – A cl::Image2D object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline USMTensor create_tensor(const element::Type type, const Shape &shape, void *usm_ptr)

This function is used to obtain remote tensor object from user-supplied USM pointer.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • usm_ptr – A USM pointer that should be wrapped by a remote tensor

Returns:

A remote tensor instance

Public Static Functions

static inline void type_check(const RemoteContext &remote_context)

Checks that type defined runtime parameters are presented in remote object.

Parameters:

remote_context – A remote context to check

class ClBufferTensor : public ov::RemoteTensor
#include <ocl.hpp>

This class represents an abstraction for GPU plugin remote tensor which can be shared with user-supplied OpenCL buffer. The plugin object derived from this class can be obtained with ClContext::create_tensor() call.

Note

User can obtain OpenCL buffer handle from this class.

Subclassed by ov::intel_gpu::ocl::D3DBufferTensor

Public Functions

inline cl_mem get()

Returns the underlying OpenCL memory object handle.

Returns:

underlying OpenCL memory object handle

inline operator cl_mem()

OpenCL memory handle conversion operator.

Returns:

cl_mem

inline operator cl::Buffer()

Standard Khronos cl::Buffer wrapper conversion operator.

Returns:

cl::Buffer object

Public Static Functions

static inline void type_check(const Tensor &tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters:

tensor – a tensor to check

class ClImage2DTensor : public ov::RemoteTensor
#include <ocl.hpp>

This class represents an abstraction for GPU plugin remote tensor which can be shared with user-supplied OpenCL 2D Image. The plugin object derived from this class can be obtained with ClContext::create_tensor() call.

Note

User can obtain OpenCL image handle from this class.

Subclassed by ov::intel_gpu::ocl::D3DSurface2DTensor, ov::intel_gpu::ocl::VASurfaceTensor

Public Functions

inline cl_mem get()

Returns the underlying OpenCL memory object handle.

Returns:

underlying OpenCL memory object handle

inline operator cl_mem()

OpenCL memory handle conversion operator.

Returns:

cl_mem

inline operator cl::Image2D()

Standard Khronos cl::Image2D wrapper conversion operator for the ClContext object.

Returns:

cl::Image2D object

Public Static Functions

static inline void type_check(const Tensor &tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters:

tensor – a tensor to check

class USMTensor : public ov::RemoteTensor
#include <ocl.hpp>

This class represents an abstraction for GPU plugin remote tensor which can be shared with user-supplied USM device pointer. The plugin object derived from this class can be obtained with ClContext::create_tensor() call.

Note

User can obtain USM pointer from this class.

Public Functions

inline void *get()

Returns the underlying USM pointer.

Returns:

underlying USM pointer

Public Static Functions

static inline void type_check(const Tensor &tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters:

tensor – a tensor to check

class ClContext : public ov::RemoteContext
#include <ocl.hpp>

This class represents an abstraction for GPU plugin remote context which is shared with OpenCL context object. The plugin object derived from this class can be obtained either with CompiledModel::get_context() or Core::create_context() calls.

Subclassed by ov::intel_gpu::ocl::D3DContext, ov::intel_gpu::ocl::VAContext

Public Functions

inline ClContext(Core &core, cl_context ctx, int ctx_device_id = 0)

Constructs context object from user-supplied OpenCL context handle.

Parameters:

  • core – A reference to OpenVINO Runtime Core object

  • ctx – A OpenCL context to be used to create shared remote context

  • ctx_device_id – An ID of device to be used from ctx

inline ClContext(Core &core, cl_command_queue queue)

Constructs context object from user-supplied OpenCL context handle.

Note

Only latency mode is supported for such context sharing case.

Parameters:

  • core – A reference to OpenVINO Runtime Core object

  • queue – An OpenCL queue to be used to create shared remote context. Queue will be reused inside the plugin.

inline cl_context get()

Returns the underlying OpenCL context handle.

Returns:

cl_context

inline operator cl_context()

OpenCL context handle conversion operator for the ClContext object.

Returns:

cl_context

inline operator cl::Context()

Standard Khronos cl::Context wrapper conversion operator for the ClContext object.

Returns:

cl::Context object

inline std::pair<ClImage2DTensor, ClImage2DTensor> create_tensor_nv12(const cl::Image2D &nv12_image_plane_y, const cl::Image2D &nv12_image_plane_uv)

This function is used to construct a NV12 compound tensor object from two cl::Image2D wrapper objects. The resulting compound contains two remote tensors for Y and UV planes of the surface.

Parameters:

  • nv12_image_plane_y – cl::Image2D object containing Y plane data.

  • nv12_image_plane_uv – cl::Image2D object containing UV plane data.

Returns:

A pair of remote tensors for each plane

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl_mem buffer)

This function is used to obtain remote tensor object from user-supplied cl_mem object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • buffer – A cl_mem object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl::Buffer &buffer)

This function is used to obtain remote tensor object from user-supplied cl::Buffer object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • buffer – A cl::Buffer object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline ClImage2DTensor create_tensor(const element::Type type, const Shape &shape, const cl::Image2D &image)

This function is used to obtain remote tensor object from user-supplied cl::Image2D object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • image – A cl::Image2D object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline USMTensor create_tensor(const element::Type type, const Shape &shape, void *usm_ptr)

This function is used to obtain remote tensor object from user-supplied USM pointer.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • usm_ptr – A USM pointer that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline USMTensor create_usm_host_tensor(const element::Type type, const Shape &shape)

This function is used to allocate USM tensor with host allocation type.

Parameters:
Returns:

A remote tensor instance

inline USMTensor create_usm_device_tensor(const element::Type type, const Shape &shape)

This function is used to allocate USM tensor with device allocation type.

Parameters:
Returns:

A remote tensor instance

RemoteTensor create_tensor(const element::Type &type, const Shape &shape, const AnyMap &params = {})

Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.

Parameters:

  • type – Defines the element type of the tensor.

  • shape – Defines the shape of the tensor.

  • params – Map of the low-level tensor object parameters.

Returns:

Pointer to a plugin object that implements the RemoteTensor interface.

Public Static Functions

static inline void type_check(const RemoteContext &remote_context)

Checks that type defined runtime parameters are presented in remote object.

Parameters:

remote_context – A remote context to check

class VASurfaceTensor : public ov::intel_gpu::ocl::ClImage2DTensor
#include <va.hpp>

This class represents an abstraction for GPU plugin remote tensor which is shared with VA output surface. The plugin object derived from this class can be obtained with VAContext::create_tensor() call.

Note

User can also obtain OpenCL 2D image handle from this class.

Public Functions

inline operator VASurfaceID()

VASurfaceID conversion operator for the VASurfaceTensor object.

Returns:

VASurfaceID handle

inline uint32_t plane()

Returns plane ID of underlying video decoder surface.

Returns:

Plane ID

Public Static Functions

static inline void type_check(const Tensor &tensor)

Checks that type defined runtime parameters are presented in remote object.

Parameters:

tensor – a tensor to check

class VAContext : public ov::intel_gpu::ocl::ClContext
#include <va.hpp>

This class represents an abstraction for GPU plugin remote context which is shared with VA display object. The plugin object derived from this class can be obtained either with CompiledModel::get_context() or Core::create_context() calls.

Note

User can also obtain OpenCL context handle from this class.

Public Functions

inline operator VADisplay()

VADisplay conversion operator for the VAContext object.

Returns:

Underlying VADisplay object handle

inline VAContext(Core &core, VADisplay device, int target_tile_id = -1)

Constructs remote context object from VA display handle.

Parameters:

  • core – OpenVINO Runtime Core object

  • device – A VADisplay to create remote context from

  • target_tile_id – Desired tile id within given context for multi-tile system. Default value (-1) means that root device should be used

inline std::pair<VASurfaceTensor, VASurfaceTensor> create_tensor_nv12(const size_t height, const size_t width, const VASurfaceID nv12_surf)

This function is used to obtain a NV12 tensor from NV12 VA decoder output. The resulting tensor contains two remote tensors for Y and UV planes of the surface.

Parameters:

  • height – A height of Y plane

  • width – A width of Y plane

  • nv12_surf – NV12 VASurfaceID to create NV12 from

Returns:

A pair of remote tensors for each plane

inline VASurfaceTensor create_tensor(const element::Type type, const Shape &shape, const VASurfaceID surface, const uint32_t plane = 0)

This function is used to create remote tensor from VA surface handle.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • surface – A VASurfaceID to create remote tensor from

  • plane – An index of a plane inside VASurfaceID to create tensor from

Returns:

A remote tensor wrapping VASurfaceID

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl_mem buffer)

This function is used to obtain remote tensor object from user-supplied cl_mem object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • buffer – A cl_mem object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline ClBufferTensor create_tensor(const element::Type type, const Shape &shape, const cl::Buffer &buffer)

This function is used to obtain remote tensor object from user-supplied cl::Buffer object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • buffer – A cl::Buffer object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline ClImage2DTensor create_tensor(const element::Type type, const Shape &shape, const cl::Image2D &image)

This function is used to obtain remote tensor object from user-supplied cl::Image2D object.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • image – A cl::Image2D object that should be wrapped by a remote tensor

Returns:

A remote tensor instance

inline USMTensor create_tensor(const element::Type type, const Shape &shape, void *usm_ptr)

This function is used to obtain remote tensor object from user-supplied USM pointer.

Parameters:

  • type – Tensor element type

  • shape – Tensor shape

  • usm_ptr – A USM pointer that should be wrapped by a remote tensor

Returns:

A remote tensor instance

RemoteTensor create_tensor(const element::Type &type, const Shape &shape, const AnyMap &params = {})

Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.

Parameters:

  • type – Defines the element type of the tensor.

  • shape – Defines the shape of the tensor.

  • params – Map of the low-level tensor object parameters.

Returns:

Pointer to a plugin object that implements the RemoteTensor interface.

Public Static Functions

static inline void type_check(const RemoteContext &remote_context)

Checks that type defined runtime parameters are presented in remote object.

Parameters:

remote_context – A remote context to check