class InferenceEngine::IInferRequest

Overview

This is an interface of asynchronous infer request. More…

#include <ie_iinfer_request.hpp>

class IInferRequest: public std::enable_shared_from_this< IInferRequest >
{
public:
    // typedefs

    typedef std::shared_ptr<IInferRequest> Ptr;
    typedef std::weak_ptr<IInferRequest> WeakPtr;

    typedef void(\* CompletionCallback)(
        InferenceEngine::IInferRequest::Ptr context,
        InferenceEngine::StatusCode code
        );

    // enums

    enum WaitMode;

    // methods

    virtual StatusCode SetBlob(
        const char \* name,
        const Blob::Ptr& data,
        ResponseDesc \* resp
        ) = 0;

    virtual StatusCode GetBlob(
        const char \* name,
        Blob::Ptr& data,
        ResponseDesc \* resp
        ) = 0;

    virtual StatusCode SetBlob(
        const char \* name,
        const Blob::Ptr& data,
        const PreProcessInfo& info,
        ResponseDesc \* resp
        ) = 0;

    virtual StatusCode GetPreProcess(
        const char \* name,
        const PreProcessInfo \*\* info,
        ResponseDesc \* resp
        ) const = 0;

    virtual StatusCode Infer(ResponseDesc \* resp) = 0;
    virtual StatusCode Cancel(ResponseDesc \* resp) = 0;

    virtual StatusCode GetPerformanceCounts(
        std::map<std::string, InferenceEngineProfileInfo>& perfMap,
        ResponseDesc \* resp
        ) const = 0;

    virtual InferenceEngine::StatusCode Wait(
        int64_t millis_timeout,
        ResponseDesc \* resp
        ) = 0;

    virtual StatusCode StartAsync(ResponseDesc \* resp) = 0;
    virtual StatusCode SetCompletionCallback(CompletionCallback callback) = 0;
    virtual StatusCode GetUserData(void \*\* data, ResponseDesc \* resp) = 0;
    virtual StatusCode SetUserData(void \* data, ResponseDesc \* resp) = 0;

    virtual InferenceEngine::StatusCode SetBatch(
        int batch_size,
        ResponseDesc \* resp
        ) = 0;

protected:
};

Detailed Documentation

This is an interface of asynchronous infer request.

Deprecated Use InferenceEngine::InferRequest C++ wrapper

Typedefs

typedef std::shared_ptr<IInferRequest> Ptr

A shared pointer to the IInferRequest object.

typedef std::weak_ptr<IInferRequest> WeakPtr

A smart pointer to the IInferRequest object.

typedef void(\* CompletionCallback)(
    InferenceEngine::IInferRequest::Ptr context,
    InferenceEngine::StatusCode code
    )

Completion callback definition as pointer to a function.

Parameters:

context

Pointer to request for providing context inside callback

code

Completion result status: InferenceEngine::OK (0) for success

Methods

virtual StatusCode SetBlob(
    const char \* name,
    const Blob::Ptr& data,
    ResponseDesc \* resp
    ) = 0

Sets input/output data to infer.

Memory allocation does not happen

Parameters:

name

Name of input or output blob.

data

Reference to input or output blob. The type of a blob must match the network input precision and size.

resp

Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: InferenceEngine::OK (0) for success

virtual StatusCode GetBlob(
    const char \* name,
    Blob::Ptr& data,
    ResponseDesc \* resp
    ) = 0

Gets input/output data for inference.

Memory allocation does not happen

Parameters:

name

Name of input or output blob.

data

Reference to input or output blob. The type of Blob must match the network input precision and size.

resp

Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: InferenceEngine::OK (0) for success

virtual StatusCode SetBlob(
    const char \* name,
    const Blob::Ptr& data,
    const PreProcessInfo& info,
    ResponseDesc \* resp
    ) = 0

Sets pre-process for input data.

Parameters:

name

Name of input blob.

data

Reference to input or output blob. The type of Blob must match the network input precision and size.

info

Preprocess info for blob.

resp

Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: OK (0) for success

virtual StatusCode GetPreProcess(
    const char \* name,
    const PreProcessInfo \*\* info,
    ResponseDesc \* resp
    ) const = 0

Gets pre-process for input data.

Parameters:

name

Name of input blob.

info

pointer to a pointer to PreProcessInfo structure

resp

Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: OK (0) for success

virtual StatusCode Infer(ResponseDesc \* resp) = 0

Infers specified input(s) in synchronous mode.

blocks all methods of IInferRequest while request is ongoing (running or waiting in queue)

Parameters:

resp

Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: InferenceEngine::OK (0) for success

virtual StatusCode Cancel(ResponseDesc \* resp) = 0

Cancels current async inference request.

Parameters:

resp

Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: InferenceEngine::OK (0) for success

virtual StatusCode GetPerformanceCounts(
    std::map<std::string, InferenceEngineProfileInfo>& perfMap,
    ResponseDesc \* resp
    ) const = 0

Queries performance measures per layer to get feedback of what is the most time consuming layer.

not all plugins provide meaningful data

Parameters:

perfMap

Map of layer names to profiling information for that layer

resp

Optional: pointer to an already allocated object to contain information in case of failure

Returns:

Status code of the operation: InferenceEngine::OK (0) for success

virtual InferenceEngine::StatusCode Wait(
    int64_t millis_timeout,
    ResponseDesc \* resp
    ) = 0

Waits for the result to become available. Blocks until specified millis_timeout has elapsed or the result becomes available, whichever comes first.

There are special cases when millis_timeout is equal some value of the WaitMode enum:

  • STATUS_ONLY - immediately returns inference status (IInferRequest::RequestStatus). It does not block or interrupt current thread

  • RESULT_READY - waits until inference result becomes available

Parameters:

millis_timeout

Maximum duration in milliseconds to block for

resp

Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success

virtual StatusCode StartAsync(ResponseDesc \* resp) = 0

Starts inference of specified input(s) in asynchronous mode.

It returns immediately. Inference starts also immediately

Parameters:

resp

Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success

virtual StatusCode SetCompletionCallback(CompletionCallback callback) = 0

Sets a callback function that will be called on success or failure of asynchronous request.

Parameters:

callback

A function to be called

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success

virtual StatusCode GetUserData(void \*\* data, ResponseDesc \* resp) = 0

Gets arbitrary data for the request and stores a pointer to a pointer to the obtained data.

Parameters:

data

Pointer to a pointer to the gotten arbitrary data

resp

Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success

virtual StatusCode SetUserData(void \* data, ResponseDesc \* resp) = 0

Sets arbitrary data for the request.

Parameters:

data

Pointer to a pointer to arbitrary data to set

resp

Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success

virtual InferenceEngine::StatusCode SetBatch(
    int batch_size,
    ResponseDesc \* resp
    ) = 0

Sets new batch size when dynamic batching is enabled in executable network that created this request.

Parameters:

batch_size

new batch size to be used by all the following inference calls for this request.

resp

Optional: a pointer to an already allocated object to contain extra information of a failure (if occurred)

Returns:

Enumeration of the resulted action: InferenceEngine::OK (0) for success