class ov::RemoteContext

Overview

This class represents an abstraction

for remote (non-CPU) accelerator device-specific inference context. Such context represents a scope on the device within which compiled models and remote memory tensors can exist, function, and exchange data. More…

#include <remote_context.hpp>

class RemoteContext
{
public:
    // construction

    RemoteContext();
    RemoteContext(const RemoteContext& other);
    RemoteContext(RemoteContext&& other);

    // methods

    RemoteContext& operator = (const RemoteContext& other);
    RemoteContext& operator = (RemoteContext&& other);

    template <typename T>
    bool is() const;

    template <typename T>
    const T as() const;

    RemoteTensor create_tensor(
        const element::Type& type,
        const Shape& shape,
        const AnyMap& params = {}
        );

    AnyMap get_params() const;
    Tensor create_host_tensor(const element::Type type, const Shape& shape);

    static void type_check(
        const RemoteContext& remote_context,
        const std::map<std::string, std::vector<std::string>>& type_info = {}
        );
};

// direct descendants

class ClContext;

Detailed Documentation

This class represents an abstraction

for remote (non-CPU) accelerator device-specific inference context. Such context represents a scope on the device within which compiled models and remote memory tensors can exist, function, and exchange data.

Construction

RemoteContext()

Default constructor.

RemoteContext(const RemoteContext& other)

Default copy constructor.

Parameters:

other

Another RemoteContext object.

RemoteContext(RemoteContext&& other)

Default move constructor.

Parameters:

other

Another RemoteContext object.

Methods

RemoteContext& operator = (const RemoteContext& other)

Default copy assignment operator.

Parameters:

other

Another RemoteContext object.

Returns:

Reference to the current object.

RemoteContext& operator = (RemoteContext&& other)

Default move assignment operator.

Parameters:

other

Another RemoteContext object.

Returns:

Reference to the current object.

template <typename T>
bool is() const

Checks if the RemoteContext object can be cast to the type T.

Parameters:

T	Type to be checked. Must represent a class derived from RemoteContext.

Returns:

True if this object can be dynamically cast to the type T*; false, otherwise.

template <typename T>
const T as() const

Casts this RemoteContext object to the type T.

Parameters:

T	Type to cast to. Must represent a class derived from RemoteContext.

Returns:

T Object.

RemoteTensor create_tensor(
    const element::Type& type,
    const Shape& shape,
    const AnyMap& params = {}
    )

Allocates memory tensor in device memory or wraps user-supplied memory handle using the specified tensor description and low-level device-specific parameters. Returns a pointer to the object that implements the RemoteTensor interface.

Parameters:

type	Defines the element type of the tensor.
shape	Defines the shape of the tensor.
params	Map of the low-level tensor object parameters.

Returns:

Pointer to a plugin object that implements the RemoteTensor interface.

AnyMap get_params() const

Returns a map of device-specific parameters required for low-level operations with the underlying object. Parameters include device/context handles, access flags, etc. Content of the returned map depends on a remote execution context that is currently set on the device (working scenario). Abstract method.

Returns:

A map of name/parameter elements.

Tensor create_host_tensor(const element::Type type, const Shape& shape)

This method is used to create a host tensor object friendly for the device in current context. For example, GPU context may allocate USM host memory (if corresponding extension is available), which could be more efficient than regular host memory.

Parameters:

type	Tensor element type.
shape	Tensor shape.

Returns:

A tensor instance with device friendly memory.

static void type_check(
    const RemoteContext& remote_context,
    const std::map<std::string, std::vector<std::string>>& type_info = {}
    )

Internal method: checks remote type.

Parameters:

remote_context	Remote context which type is checked.
type_info	Map with remote object runtime info.
Exception	if type check with the specified parameters failed.