Class ov::RemoteContext

class RemoteContext

This class represents an abstraction

for remote (non-CPU) accelerator device-specific inference context. Such context represents a scope on the device within which compiled models and remote memory tensors can exist, function, and exchange data.

Subclassed by ov::intel_gpu::ocl::ClContext, ov::intel_npu::level_zero::ZeroContext

Public Functions

RemoteContext() = default

Default constructor.

RemoteContext(const RemoteContext &other) = default

Default copy constructor.

Parameters:

other – Another RemoteContext object.

RemoteContext &operator=(const RemoteContext &other) = default

Default copy assignment operator.

Parameters:

other – Another RemoteContext object.

Returns:

Reference to the current object.

RemoteContext(RemoteContext &&other) = default

Default move constructor.

Parameters:

other – Another RemoteContext object.

RemoteContext &operator=(RemoteContext &&other) = default

Default move assignment operator.

Parameters:

other – Another RemoteContext object.

Returns:

Reference to the current object.

operator bool() const noexcept

Checks if current RemoteContext object is initialized.

Returns:

true if current RemoteContext object is initialized, false - otherwise

~RemoteContext()

Destructor that preserves unloading order of implementation object and reference to the library.

template<typename T>
inline bool is() const noexcept

Checks if the RemoteContext object can be cast to the type T.

Template Parameters:

T – Type to be checked. Must represent a class derived from RemoteContext.

Returns:

True if this object can be dynamically cast to the type T*; false, otherwise.

template<typename T>
inline const T as() const

Casts this RemoteContext object to the type T.

Template Parameters:

T – Type to cast to. Must represent a class derived from RemoteContext.

Returns:

T Object.

RemoteTensor create_tensor(const element::Type &type, const Shape &shape, const AnyMap &params = {})

Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.

Parameters:

• type – Defines the element type of the tensor.

• shape – Defines the shape of the tensor.

• params – Map of the low-level tensor object parameters.

Returns:

Pointer to a plugin object that implements the RemoteTensor interface.

AnyMap get_params() const

Returns a map of device-specific parameters required for low-level operations with the underlying object. Parameters include device/context handles, access flags, etc. Content of the returned map depends on a remote execution context that is currently set on the device (working scenario). Abstract method.

Returns:

A map of name/parameter elements.

Tensor create_host_tensor(const element::Type type, const Shape &shape)

This method is used to create a host tensor object friendly for the device in current context. For example, GPU context may allocate USM host memory (if corresponding extension is available), which could be more efficient than regular host memory.

Parameters:

• type – Tensor element type.

• shape – Tensor shape.

Returns:

A tensor instance with device friendly memory.

Public Static Functions

static void type_check(const RemoteContext &remote_context, const std::map<std::string, std::vector<std::string>> &type_info = {})

Internal method: checks remote type.

Parameters:

• remote_context – Remote context which type is checked.

• type_info – Map with remote object runtime info.

Throws:

Exception – if type check with the specified parameters failed.