interface InferenceEngine::IInferRequestInternal¶
Overview¶
An internal API of synchronous inference request to be implemented by plugin, which is used in InferRequestBase forwarding mechanism. More…
#include <ie_iinfer_request_internal.hpp>
template IInferRequestInternal: public std::enable_shared_from_this< IInferRequestInternal >
{
// typedefs
typedef std::shared_ptr<IInferRequestInternal> Ptr;
typedef std::function<void(std::exception_ptr)> Callback;
// construction
IInferRequestInternal();
IInferRequestInternal(
const InputsDataMap& networkInputs,
const OutputsDataMap& networkOutputs
);
// methods
virtual void Infer();
virtual void InferImpl();
virtual void Cancel();
virtual std::map<std::string, InferenceEngineProfileInfo> GetPerformanceCounts() const;
virtual void SetBlob(const std::string& name, const Blob::Ptr& data);
virtual Blob::Ptr GetBlob(const std::string& name);
virtual void SetBlob(
const std::string& name,
const Blob::Ptr& data,
const PreProcessInfo& info
);
virtual const PreProcessInfo& GetPreProcess(const std::string& name) const;
virtual void SetBatch(int batch);
virtual std::vector<std::shared_ptr<IVariableStateInternal>> QueryState();
virtual void StartAsync();
virtual void StartAsyncImpl();
virtual StatusCode Wait(int64_t millis_timeout);
virtual void SetCallback(Callback callback);
void checkBlob(
const Blob::Ptr& blob,
const std::string& name,
bool isInput,
const SizeVector& refDims = {}
) const;
virtual void checkBlobs();
void setPointerToExecutableNetworkInternal(const std::shared_ptr<IExecutableNetworkInternal>& exeNetwork);
void* GetUserData();
void SetUserData(void* userData);
protected:
};
// direct descendants
class AsyncInferRequestThreadSafeDefault;
Detailed Documentation¶
An internal API of synchronous inference request to be implemented by plugin, which is used in InferRequestBase forwarding mechanism.
Typedefs¶
typedef std::shared_ptr<IInferRequestInternal> Ptr
A shared pointer to a IInferRequestInternal interface.
typedef std::function<void(std::exception_ptr)> Callback
Alias for callback type.
Construction¶
IInferRequestInternal(
const InputsDataMap& networkInputs,
const OutputsDataMap& networkOutputs
)
Constructs a new instance.
Parameters:
networkInputs |
The network inputs info |
networkOutputs |
The network outputs data |
Methods¶
virtual void Infer()
Infers specified input(s) in synchronous mode.
blocks all method of InferRequest while request is ongoing (running or waiting in queue)
virtual void InferImpl()
The minimal infer function to be implemented by plugins. It infers specified input(s) in synchronous mode.
This method is used in IInferRequestInternal::Infer, which calls the common code first and after uses this plugin dependent implementation.
Blocks all method of InferRequest while request is ongoing (running or waiting in queue)
virtual void Cancel()
Cancel current inference request execution.
virtual std::map<std::string, InferenceEngineProfileInfo> GetPerformanceCounts() const
Queries performance measures per layer to get feedback of what is the most time consuming layer. Note: not all plugins may provide meaningful data.
Returns:
a map of layer names to profiling information for that layer.
virtual void SetBlob(const std::string& name, const Blob::Ptr& data)
Set input/output data to infer.
Memory allocation doesn’t happen
Parameters:
name |
|
data |
|
virtual Blob::Ptr GetBlob(const std::string& name)
Get input/output data to infer.
Memory allocation doesn’t happen
Parameters:
name |
|
data |
|
virtual void SetBlob(
const std::string& name,
const Blob::Ptr& data,
const PreProcessInfo& info
)
Sets pre-process for input data.
Parameters:
name |
Name of input blob. |
data |
|
info |
Preprocess info for blob. |
virtual const PreProcessInfo& GetPreProcess(const std::string& name) const
Gets pre-process for input data.
Parameters:
name |
Name of input blob. |
info |
pointer to a pointer to PreProcessInfo structure |
virtual void SetBatch(int batch)
Sets new batch size when dynamic batching is enabled in executable network that created this request.
Parameters:
batch |
|
virtual std::vector<std::shared_ptr<IVariableStateInternal>> QueryState()
Queries memory states.
Returns:
Returns memory states
virtual void StartAsync()
Start inference of specified input(s) in asynchronous mode.
The method returns immediately. Inference starts also immediately.
virtual void StartAsyncImpl()
The minimal asynchronous inference function to be implemented by plugins. It starts inference of specified input(s) in asynchronous mode.
The methos is used in AsyncInferRequestInternal::StartAsync which performs common steps first and calls plugin dependent implementation of this method after.
It returns immediately. Inference starts also immediately.
virtual StatusCode Wait(int64_t millis_timeout)
Waits for the result to become available. Blocks until specified millis_timeout has elapsed or the result becomes available, whichever comes first.
There are special cases when millis_timeout is equal some value of WaitMode enum:
STATUS_ONLY - immediately returns request status (InferRequest::StatusCode). It doesn’t block or interrupt current thread.
RESULT_READY - waits until inference result becomes available
Parameters:
millis_timeout |
|
Returns:
A status code
virtual void SetCallback(Callback callback)
Set callback function which will be called on success or failure of asynchronous request.
Parameters:
callback |
|
void checkBlob(
const Blob::Ptr& blob,
const std::string& name,
bool isInput,
const SizeVector& refDims = {}
) const
Check that blob
is valid. Throws an exception if it’s not.
Parameters:
blob |
The blob to check |
name |
The name of input or output depending of if the |
isInput |
Indicates if |
refDims |
The reference dims, empty if not specified |
virtual void checkBlobs()
Check that all of the blobs is valid. Throws an exception if it’s not.
void setPointerToExecutableNetworkInternal(const std::shared_ptr<IExecutableNetworkInternal>& exeNetwork)
Sets the pointer to executable network internal.
Needed to correctly handle ownership between objects.
Parameters:
exeNetwork |
The executable network |
void* GetUserData()
Gets the pointer to userData.
Returns:
Pointer to user data
void SetUserData(void* userData)
Sets the pointer to userData.
Parameters:
Pointer |
to user data |