class ov::TensorRemoteBlob

Overview

Create InferenceEngine::RemoteBlob from the Tensor. More…

#include <remote_utils.hpp>

class TensorRemoteBlob:
    public InferenceEngine::RemoteBlob,
    public ov::legacy_convert::TensorHolder
{
public:
    // construction

    TensorRemoteBlob(
        const ov::SoPtr<ITensor>& tensor,
        InferenceEngine::TensorDesc desc
        );

    // methods

    std::shared_ptr<ov::IRemoteTensor> cast_tensor() const;
    virtual AnyMap getParams() const;
    virtual std::string getDeviceName() const;
    virtual std::shared_ptr<InferenceEngine::RemoteContext> getContext() const;
    virtual void allocate();
    virtual bool deallocate();
    virtual InferenceEngine::LockedMemory<void> rwmap();
    virtual InferenceEngine::LockedMemory<const void> rmap() const;
    virtual InferenceEngine::LockedMemory<void> wmap();
    virtual const std::shared_ptr<InferenceEngine::IAllocator>& getAllocator() const;
    virtual void \* getHandle() const;
};

Inherited Members

public:
    // typedefs

    typedef std::shared_ptr<Blob> Ptr;
    typedef std::shared_ptr<const Blob> CPtr;
    typedef std::shared_ptr<MemoryBlob> Ptr;
    typedef std::shared_ptr<const MemoryBlob> CPtr;
    typedef std::shared_ptr<RemoteBlob> Ptr;
    typedef std::shared_ptr<const RemoteBlob> CPtr;

    // methods

    static Ptr CreateFromData(const DataPtr& data);

    template <
        typename T,
        typename std::enable_if<!std::is_pointer<T>::value&&!std::is_reference<T>::value, int>::type = 0,
        typename std::enable_if<std::is_base_of<Blob, T>::value, int>::type = 0
        >
    bool is();

    template <
        typename T,
        typename std::enable_if<!std::is_pointer<T>::value&&!std::is_reference<T>::value, int>::type = 0,
        typename std::enable_if<std::is_base_of<Blob, T>::value, int>::type = 0
        >
    bool is() const;

    template <
        typename T,
        typename std::enable_if<!std::is_pointer<T>::value&&!std::is_reference<T>::value, int>::type = 0,
        typename std::enable_if<std::is_base_of<Blob, T>::value&&!std::is_same<RemoteBlob, T>::value, int>::type = 0
        >
    T \* as();

    template <
        typename T,
        typename std::enable_if<!std::is_pointer<T>::value&&!std::is_reference<T>::value, int>::type = 0,
        typename std::enable_if<std::is_base_of<Blob, T>::value&&!std::is_same<RemoteBlob, T>::value, int>::type = 0
        >
    const T \* as() const;

    virtual const TensorDesc& getTensorDesc() const;
    virtual TensorDesc& getTensorDesc();
    virtual size_t size() const;
    virtual size_t byteSize() const;
    virtual size_t element_size() const = 0;
    virtual void allocate() = 0;
    virtual bool deallocate() = 0;
    virtual void setShape(const SizeVector& dims);
    virtual Blob::Ptr createROI(const ROI& roi) const;

    virtual Blob::Ptr createROI(
        const std::vector<std::size_t>& begin,
        const std::vector<std::size_t>& end
        ) const;

    virtual const TensorDesc& getTensorDesc() const;
    virtual TensorDesc& getTensorDesc();
    virtual size_t size() const;
    virtual size_t byteSize() const;
    virtual size_t element_size() const;
    virtual void allocate() = 0;
    virtual bool deallocate() = 0;
    virtual LockedMemory<void> rwmap() = 0;
    virtual LockedMemory<const void> rmap() const = 0;
    virtual LockedMemory<void> wmap() = 0;
    virtual ParamMap getParams() const = 0;
    virtual std::string getDeviceName() const = 0;
    virtual std::shared_ptr<RemoteContext> getContext() const = 0;
    const ov::SoPtr<ov::ITensor>& get_tensor() const;

Detailed Documentation

Create InferenceEngine::RemoteBlob from the Tensor.

Methods

virtual AnyMap getParams() const

Returns a map of device-specific parameters required for low-level operations with underlying object. Parameters include device/context/surface/buffer handles, access flags, etc. Contents of the map returned depend on remote execution context that is currently set on the device (working scenario). Abstract method.

Returns:

A map of name/parameter elements.

virtual std::string getDeviceName() const

Returns name of the device on which underlying object is allocated. Abstract method.

Returns:

A device name string in the same format as that in plugin metric.

virtual std::shared_ptr<InferenceEngine::RemoteContext> getContext() const

Returns device context which underlying object belongs to. Abstract method.

Returns:

Pointer to plugin-specific context class object, which is derived from RemoteContext. Dynamic casting should be used if it is necessary to retrieve a pointer to original class.

virtual void allocate()

Allocates memory to store the data.

Abstract method.

virtual bool deallocate()

Releases previously allocated data.

Abstract method.

Returns:

True if deallocation happens successfully, false otherwise.

virtual InferenceEngine::LockedMemory<void> rwmap()

Gets read/write access to the memory in virtual space of the process. The function returns object which retains mapped memory. The memory been addressed in the MemoryBlob in general case can be allocated on remote device. This function maps remote memory to the memory in the virtual process space and after destruction of the LockedMemory will upload changed content to the accelerator.

To avoid extra copy of data, you can use rmap() and wmap() functions.

In case of memory originally allocated on the host, this function returns LockedMemory which will transparently refer to original memory address. No extra copy will happen

In general case, pointer received from that LockedMemory becomes invalid just after destruction of LockedMemory instance. Keep Locked memory alive while you need to address memory in the process on the host.

Abstract method.

Returns:

A LockedMemory object

virtual InferenceEngine::LockedMemory<const void> rmap() const

Gets read only access to the memory in virtual space of the process. The function returns object which retains mapped memory.

The memory been addressed in the MemoryBlob in general case can be allocated on remote device. This function copies remote memory to the memory in the virtual process space and after destruction of the LockedMemory it will not upload host memory back, because it is expected that content is not changed.

To have an ability change content, you can use rwmap() and wmap() functions.

In case of memory originally allocated on the host, this function returns LockedMemory which will transparently refer to original memory address. No extra copy will happen

In general case, pointer received from that LockedMemory becomes invalid just after destruction of LockedMemory instance. Keep Locked memory alive while you need to address memory in the process on the host.

Abstract method.

Returns:

A LockedMemory object

virtual InferenceEngine::LockedMemory<void> wmap()

Gets “write only direction” access to the memory in virtual space of the process. The function returns object which retains memory to be uploaded on device.

The memory been addressed in the MemoryBlob in general case can be allocated on remote device. This function does not copy of the content from the device to the memory in the virtual process space, the content of the memory just after calling of this function is not specified. After destruction of the LockedMemory, content will be upload host memory. In the same time there is no abilities to restrict reading from the memory, you need to care of reading from memory got by wmap(), it might have sense in some cases like filling of content and before uploading to device

To access data stored in the blob, you can use rwmap() and rmap() functions.

In case of memory originally allocated on the host, this function returns LockedMemory which will transparently refer to original memory address. No extra copy will happen

In general case, pointer received from that LockedMemory becomes invalid just after destruction of LockedMemory instance. Keep Locked memory alive while you need to address memory in the process on the host.

Abstract method.

Returns:

A LockedMemory object

virtual const std::shared_ptr<InferenceEngine::IAllocator>& getAllocator() const

Gets the allocator for allocator-based blobs.

Returns:

The allocator for allocator-based blobs or if there is none then a nullptr.

virtual void \* getHandle() const

Gets the handle to allocated memory.

Returns:

The handle to allocated memory for allocator-based blobs or if there is none then a nullptr.