Using Shape Inference

Shape Inference feature enables resizing network before loading it to a plugin. It makes possible to specify differently-sized input upon reading the model by the Inference Engine without going back to the Model Optimizer. The feature is exposed to replace InferenceEngine::ICNNNetwork::SetBatchSize as well, as setting batch is a special case of setting the whole input shape.

Usage

The primary method of the feature is InferenceEngine::CNNNetwork::reshape. It gets new input shapes and propagates it from input to output for all intermediates layers of the given network. The method takes InferenceEngine::ICNNNetwork::InputShapes - a map of pairs: name of input data and its dimension.

The algorithm for resizing network is the following:

1) Collect the map of input names and shapes from Intermediate Representation (IR) using helper method InferenceEngine::CNNNetwork::getInputShapes

2) Set new input shapes

3) Call reshape

Here is a code example:

// ------------- 0. Read IR and image ----------------------------------------------
CNNNetReader network_reader;
network_reader.ReadNetwork("path/to/IR/xml");
CNNNetwork network = network_reader.getNetwork();
cv::Mat image = cv::imread("path/to/image");
// ---------------------------------------------------------------------------------
// ------------- 1. Collect the map of input names and shapes from IR---------------
auto input_shapes = network.getInputShapes();
// ---------------------------------------------------------------------------------
// ------------- 2. Set new input shapes -------------------------------------------
std::string input_name;
SizeVector input_shape;
std::tie(input_name, input_shape) = *input_shapes.begin(); // let's consider first input only
input_shape[0] = batch_size; // set batch size to the first input dimension
input_shape[2] = image.rows; // changes input height to the image one
input_shape[3] = image.cols; // changes input width to the image one
input_shapes[input_name] = input_shape;
// ---------------------------------------------------------------------------------
// ------------- 3. Call reshape ---------------------------------------------------
network.reshape(input_shapes);
// ---------------------------------------------------------------------------------
...
// ------------- 4. Loading model to the device ------------------------------------
InferenceEngine::Core core;
std::string device = "CPU";
ExecutableNetwork executable_network = core.LoadNetwork(network, device);
// ---------------------------------------------------------------------------------

Shape Inference feature is used in Smart classroom sample.

Extensibility

Custom Shape Inference functions are registered via calling InferenceEngine::ICNNNetwork::AddExtension with implemented InferenceEngine::IShapeInferExtension - holder of the custom implementations. Holder requires to implement 2 key methods:

Supported Layer Types

Limitations

Shape Inference is a preview feature with a set of limitations: