The GPU plugin implementation of the
RemoteBlob interfaces supports GPU pipeline developers who need video memory sharing and interoperability with existing native APIs such as OpenCL*, Microsoft DirectX*, or VAAPI*. Using these interfaces allows to avoid any memory copy overhead when plugging the OpenVINO™ inference into an existing GPU pipeline. It also enables OpenCL kernels participating in the pipeline to become native buffer consumers or producers of the OpenVINO™ inference. Since the GPU plugin works on top of the clDNN library, the functionality above is also implemented using OpenCL and its sharing extensions provided by Intel®.
There are two interoperability scenarios that are supported for the Remote Blob API:
Class and function declarations for the API are defined in the following files:
The most common way to enable the interaction of your application with the Remote Blob API is to use user-side utility classes and functions that consume or produce native handles directly.
GPU plugin classes that implement the
RemoteContext interface are responsible for context sharing. Obtaining a pointer to a context object is the first step of sharing pipeline objects. The context object of the GPU plugin directly wraps OpenCL context, setting a scope for sharing
RemoteBlob objects. To create such objects within user context, explicitly provide the context to the plugin using the
make_shared_context() overloaded function. Depending on the platform, the function accepts the
cl_context handle, the pointer to the
ID3D11Device interface, or the
VADisplay handle, and returns a smart pointer to the
RemoteContext plugin object.
If you do not provide any user context, the plugin uses its default internal context. The plugin attempts to use the same internal context object as long as plugin options are kept the same. Therefore, all ExecutableNetwork objects created during this time share the same context. Once the plugin options are changed, the internal context is replaced by the new one.
To request the current default context of the plugin, call the
GetDefaultContext() method of the core engine. To request the internal context of the given
ExecutableNetwork, use the
The classes that implement the
RemoteBlob interface both are wrappers for native API memory handles (which can be obtained from them at any moment) and act just like regular OpenVINO™
Once you obtain the context, you can use it to compile a new
ExecutableNetwork or create
RemoteBlob objects. For network compilation, use a dedicated flavor of
LoadNetwork(), which accepts the context as an additional parameter.
To create a shared blob from a native memory handle, use
make_shared_blob() overloaded functions that can accept the
cl_mem handles, and either
ID3D11Texture2D pointers or the
VASurfaceID handle. All
make_shared_blob() flavors return a smart pointer to the
Blob object, which can be directly passed to the
SetBlob()method of an inference request object.
To support the direct consumption of a hardware video decoder output, plugin accepts two-plane video surfaces as arguments for the
make_shared_blob_nv12() function, which creates an
NV12Blob object and returns a smart pointer to it, which is cast to
To ensure that the plugin generates the correct execution graph for the NV12 dual-plane input, set the
CLDNNConfigParams::KEY_CLDNN_NV12_TWO_INPUTS plugin configuration flag to
The high-level wrappers above bring a direct dependency on native APIs to the user program. If you want to avoid the dependency, you still can directly use the
getParams() methods. On this level, native handles are re-interpreted as void pointers and all arguments are passed using
std::map containers that are filled with
std::string, InferenceEngine::Parameter pairs. Two types of map entries are possible: descriptor and container. The first map entry is a descriptor, which sets the expected structure and possible parameter values of the map.
Parameter Map Entries
|Key Name||Description and Possible Parameter Values|
|Describes the type of the shared context in a map. Can be |
|Contains the OpenCL context handle.|
|Contains the native video decoding device handle. Can be |
|Describes the type of the shared memory buffer in a map. Can be |
|Contains the OpenCL memory handle.|
|Contains the native video decoder surface handle.|
|Contains the NV12 video decoder surface plane index. Can be |
Refer to the sections below to see pseudo-code of usage examples.
NOTE: For low-level parameter usage examples, see the source code of user-side wrappers from the include files mentioned above.
This example uses the OpenCL context obtained from an executable network object.