InferRequest

Overview

Set of functions responsible for dedicated inference for certain ExecutableNetwork. More…

// global functions

void ie_infer_request_free(ie_infer_request_t** infer_request);

IEStatusCode ie_infer_request_get_blob(
    ie_infer_request_t* infer_request,
    const char* name,
    ie_blob_t** blob
    );

IEStatusCode ie_infer_request_set_blob(
    ie_infer_request_t* infer_request,
    const char* name,
    const ie_blob_t* blob
    );

IEStatusCode ie_infer_request_infer(ie_infer_request_t* infer_request);
IEStatusCode ie_infer_request_infer_async(ie_infer_request_t* infer_request);

IEStatusCode ie_infer_set_completion_callback(
    ie_infer_request_t* infer_request,
    ie_complete_call_back_t* callback
    );

IEStatusCode ie_infer_request_wait(
    ie_infer_request_t* infer_request,
    const int64_t timeout
    );

IEStatusCode ie_infer_request_set_batch(
    ie_infer_request_t* infer_request,
    const size_t size
    );

Detailed Documentation

Set of functions responsible for dedicated inference for certain ExecutableNetwork.

Global Functions

void ie_infer_request_free(ie_infer_request_t** infer_request)

Releases memory occupied by ie_infer_request_t instance.

Parameters:

infer_request

A pointer to the ie_infer_request_t to free memory.

IEStatusCode ie_infer_request_get_blob(
    ie_infer_request_t* infer_request,
    const char* name,
    ie_blob_t** blob
    )

Gets input/output data for inference.

Parameters:

infer_request

A pointer to ie_infer_request_t instance.

name

Name of input or output blob.

blob

A pointer to input or output blob. The type of Blob must match the network input precision and size.

Returns:

Status code of the operation: OK(0) for success.

IEStatusCode ie_infer_request_set_blob(
    ie_infer_request_t* infer_request,
    const char* name,
    const ie_blob_t* blob
    )

Sets input/output data to inference.

Parameters:

infer_request

A pointer to ie_infer_request_t instance.

name

Name of input or output blob.

blob

Reference to input or output blob. The type of a blob must match the network input precision and size.

Returns:

Status code of the operation: OK(0) for success.

IEStatusCode ie_infer_request_infer(ie_infer_request_t* infer_request)

Starts synchronous inference of the infer request and fill outputs.

Parameters:

infer_request

A pointer to ie_infer_request_t instance.

Returns:

Status code of the operation: OK(0) for success.

IEStatusCode ie_infer_request_infer_async(ie_infer_request_t* infer_request)

Starts asynchronous inference of the infer request and fill outputs.

Parameters:

infer_request

A pointer to ie_infer_request_t instance.

Returns:

Status code of the operation: OK(0) for success.

IEStatusCode ie_infer_set_completion_callback(
    ie_infer_request_t* infer_request,
    ie_complete_call_back_t* callback
    )

Sets a callback function that will be called on success or failure of asynchronous request.

Parameters:

infer_request

A pointer to ie_infer_request_t instance.

callback

A function to be called.

Returns:

Status code of the operation: OK(0) for success.

IEStatusCode ie_infer_request_wait(
    ie_infer_request_t* infer_request,
    const int64_t timeout
    )

Waits for the result to become available. Blocks until specified timeout elapses or the result becomes available, whichever comes first.

There are special cases when timeout is equal some value of the WaitMode enum:

  • 0 - Immediately returns the inference status. It does not block or interrupt execution.

  • -1 - waits until inference result becomes available

Parameters:

infer_request

A pointer to ie_infer_request_t instance.

timeout

Maximum duration in milliseconds to block for

Returns:

Status code of the operation: OK(0) for success.

IEStatusCode ie_infer_request_set_batch(
    ie_infer_request_t* infer_request,
    const size_t size
    )

Sets new batch size for certain infer request when dynamic batching is enabled in executable network that created this request.

Parameters:

infer_request

A pointer to ie_infer_request_t instance.

size

New batch size to be used by all the following inference calls for this request.

Returns:

Status code of the operation: OK(0) for success.