NOTE: It is a preview version of the Inference Engine Python* API for evaluation purpose only. Module structure and API itself may be changed in future releases.
This API provides a simplified interface for Inference Engine functionality that allows to:
Currently the Inference Engine Python* API is supported on Ubuntu* 16.04, Microsoft Windows* 10 and CentOS* 7.3 OSes. Supported Python* versions:
To configure the environment for the Inference Engine Python* API, run:
source <INSTALL_DIR>/bin/setupvars.sh .
call <INSTALL_DIR>\deployment_tools\inference_engine\python_api\setenv.bat
The script automatically detects latest installed Python* version and configures required environment if the version is supported. If you want to use certain version of Python*, set the environment variable PYTHONPATH=<INSTALL_DIR>/deployment_tools/inference_engine/python_api/<desired_python_version>
after running the environment configuration script.
This class stores main information about the layer and allow to modify some layer parameters
name
- Name of the layertype
- Layer typeprecision
- Layer base operating precision. Provides getter and setter interfaces.layout
- Returns the layout of shape of the layer.shape
- Return the list of the shape of the layer.parents
- Returns a list, which contains names of layers preceding this layer.children
- Returns a list, which contains names of layers following this layer.affinity
- Layer affinity set by user or a default affinity set by the IEPlugin.set_initial_affinity()
method. The affinity attribute provides getter and setter interfaces, so the layer affinity can be modified directly. For example: To correctly set affinity for the network, you must first initialize and properly configure the HETERO plugin. set_config({"TARGET_FALLBACK": "HETERO:FPGA,GPU"})
function configures the plugin fallback devices and their order. plugin.set_initial_affinity(net)
function sets affinity parameter of model layers according to its support on specified devices.
After default affinity is set by the plugin, override the default values by setting affinity manually how it's described in example above
To understand how default and non-default affinities are set:
net.layers
function right after model loading and check that layer affinity parameter is empty.plugin.set_default_affinity(net)
.net.layers
and check layer affinity parameters to see how plugin set a default affinitynet.layers
again and check layer affinity parameters to see how it was changed after manual affinity settingPlease refer to affinity_setting_demo.py
to see the full usage pipeline.
weights
- Dictionary with layer weights, biases or custom blobs if anyparams
- Layer specific parameters. Provides getter and setter interfaces to get and modify layer parameters. Please note that some modifications can be ignored and overwriten by target plugin (e.g. modification of convolution kernel size will be reflected in layer parameters but finally the plugin will ignore it and will use initial kernel size)This class contains the information about the network model read from IR and allows you to manipulate with some model parameters such as layers affinity and output layers.
__init__(model: str, weights: str)
.xml
file of the IR.bin
file of the IRname
- Name of the loaded networkinputs
- A dictionary that maps input layer names to InputInfo objects. For example, to get a shape of the input layer:outputs
- A dictionary that maps output layer names to OutputInfo objects For example, to get a shape of the output layer:batch_size
- Batch size of the network. Provides getter and setter interfaces to get and modify the network batch size. For example:layers
- Return dictionary that maps network layer names to IENetLayer
objects containing layer properties in topological order. For example, to list all network layers:stats
- Returns LayersStatsMap
object containing dictionary that maps network layer names to calibration statistics represented by LayerStats
objects. LayersStatsMap
class inherited from built-in python dict
and overrides default update()
method to allow to set or modify layers calibration statistics. from_ir(model: str, weights: str)
Note: The function is deprecated. Please use IENetwork()
class constructor to create valid instance of IENetwork
* Description: The class method serves to read the model from the `.xml` and `.bin` files of the IR. * Parameters: * model - Path to `.xml` file of the IR * weights - Path to `.bin` file of the IR * Return value: An instance of the `IENetwork` class * Usage example:
add_outputs(outputs)
:Description:
The method serves to mark any intermediate layer as output layer to retrieve the inference results from the specified layers.
outputs
- List of layer names to be set as model outputs. In case of setting one layer as output, string with one layer can be provided.Return value:
None
Note
The last layers (nodes without successors in graph representation of the model) are set as output by default. In the case above, prob
layer is a default output and conv5_1/dwise
, conv2_1/expand
are user-defined outputs.
reshape(input_shapes: dict)
:Description:
The method reshapes the network to change spatial dimensions, batch size, or any dimension.
Note:
Before using this method, make sure that the target shape is applicable for the network Changing the network shape to an arbitrary value may lead to unpredictable behaviour.
input_shapes
- The dictionary that maps input layer names to tuples with the target shapeReturn value:
None
serialize(path_to_xml, path_to_bin)
:Description:
The method serializes the network and stores it in files.
path_to_xml
- path to a file, where a serialized model will be stored.path_to_bin
- path to a file, where serialized weights will be stored.Return value:
None
Layer calibration statistic container
__init__(min: tuple = (), max: tuple = ())
This class contains the information about the network input layers
precision
- Precision of the input data provided by user. Provides setter and getter interfaces to get and modify input layer precision.
List of applicable precisions: FP32 FP16, I32, I16, I8, U32, U16
Note: Support of any calculation precision depends on the target plugin
layout
- Layout of the input data provided by user. Provides setter and getter interfaces to get and modify input layer layout.
List of applicable layouts: NCHW, NHWC, OIHW, C, CHW, HW, NC, CN, BLOCKED
shape
- input layer data shapeThis class contains the information about the network input layers
precision
- Precision of the output data. Provides setter and getter interfaces to get and modify output layer precision.layout
- Layout of the output data provided by usershape
- Input layer data shapeThis class is the main plugin interface and serves to initialize and configure the plugin.
__init__(device: str, plugin_dirs=None)
device
- Target device name. Supported devices: CPU, GPU, FPGA, MYRIAD, HETEROplugin_dirs
- List of paths to plugin directoriesdevice
- a name of the device that was specified to initialize IEPluginversion
- a version of the pluginload(network: IENetwork, num_requests: int=1, config=None)
Description:
Loads a network that was read from the IR to the plugin and creates an executable network from a network object. You can create as many networks as you need and use them simultaneously (up to the limitation of the hardware resources).
network
- A valid IENetwork
instancenum_requests
- A positive integer value of infer requests to be created. Number of infer requests may be limited by device capabilities.config
- A dictionary of plugin configuration keys and their valuesReturn value:
None
set_initial_affinity(net: IENetwork)
Description:
Sets initial affinity for model layers according to the HETERO plugin logic. Applicable only if IEPlugin was initialized for HETERO device.
net
- A valid instance of IENetworkReturn value:
None
Usage example:
See affinity
attribute of the IENetLayer
class.
add_cpu_extension(extension_path: str)
Description:
Loads extensions library to the plugin. Applicable only for CPU device and HETERO device with CPU
extension_path
- A full path to CPU extensions libraryReturn value:
None
set_config(config: dict)
Description:
Sets a configuration for the plugin. Refer to SetConfig()
in Inference Engine C++ documentation for acceptable keys and values list.
config
- A dictionary of keys and values of acceptable configuration parametersReturn value:
None
Usage examples:
See set_affinity
method of the IENetwork
class.
get_supported_layers(net: IENetwork)
Description:
Returns the set of layers supported by the plugin. Please note that in case of CPU plugin support of a layer may depends on extension loaded by add_cpu_extenstion()
method
net
- A valid instance of IENetworkReturn value:
Set of layers supported by the plugin
Usage example:
See affinity
attribute of the IENetLayer
class.
This class represents a network instance loaded to plugin and ready for inference.
There is no explicit class constructor. To make a valid instance of ExecutableNetwork
, use load()
method of the IEPlugin
class.
requests
- A tuple of InferRequest instancesinfer(inputs=None)
Description:
Starts synchronous inference for the first infer request of the executable network and returns output data. Wraps infer()
method of the InferRequest
class
inputs
- A dictionary that maps input layer names to numpy.ndarray
objects of proper shape with input data for the layerReturn value:
A dictionary that maps output layer names to numpy.ndarray
objects with output data of the layer
For illustration of input data preparation, please see samples (for example, classification_sample.py
).
start_async(request_id, inputs=None)
Description:
Starts asynchronous inference for specified infer request. Wraps async_infer()
method of the InferRequest
class
request_id
- Index of infer request to start inferenceinputs
- A dictionary that maps input layer names to numpy.ndarray
objects of proper shape with input data for the layerReturn value:
A handler of specified infer request, which is an instance of the InferRequest
class.
For more details about infer requests processing, see classification_sample_async.py
(simplified case) and object_detection_demo_ssd_async.py
(real asynchronous use case) samples.
This class provides an interface to infer requests of ExecutableNetwork
and serves to handle infer requests execution and to set and get output data.
There is no explicit class constructor. To make a valid InferRequest
instance, use load()
method of the IEPlugin
class with specified number of requests to get ExecutableNetwork
instance which stores infer requests.
inputs
- A dictionary that maps input layer names to numpy.ndarray
objects of proper shape with input data for the layeroutputs
- A dictionary that maps output layer names to numpy.ndarray
objects with output data of the layerIt is not recommended to run inference directly on InferRequest
instance. To run inference, please use simplified methods infer()
and start_async()
of ExecutableNetwork
.
infer(inputs=None)
Description:
Starts synchronous inference of the infer request and fill outputs array
inputs
- A dictionary that maps input layer names to numpy.ndarray
objects of proper shape with input data for the layerReturn value:
None
async_infer(inputs=None)
Description:
Starts asynchronous inference of the infer request and fill outputs array
inputs
- A dictionary that maps input layer names to numpy.ndarray
objects of proper shape with input data for the layerReturn value:
None
wait(timeout=-1)
Description:
Waits for the result to become available. Blocks until specified timeout elapses or the result becomes available, whichever comes first.
Note:
There are special values of the timeout parameter:
timeout
- Time to wait in milliseconds or special (0, -1) cases described above. If not specified, timeout
value is set to -1 by default.Usage example:
See async_infer()
method of the the InferRequest
class.
get_perf_counts()
Description:
Queries performance measures per layer to get feedback of what is the most time consuming layer. .
Note:
Performance counters data and format depends on the plugin
Parameters:
None
set_batch(size)
Description: Sets new batch size for certain infer request when dynamic batching is enabled in executable network that created this request.
Note: Support of dynamic batch size depends on the target plugin.
batch
- new batch size to be used by all the following inference calls for this request.dynamic_batch_demo.py
to see the full usage example.