NOTE: It is a preview version of the Inference Engine Deep Neural Network Builder API for evaluation purpose only. Module structure and API itself may be changed in future releases.
This API extends the Inference Engine functionality that allows to create and modify topologies in the source code.
InferenceEngine::Builder::Network
allows to create and modify graphs. This class does not modify the original graph if it is used for graph modification. Instead, it creates a copy of the original graph and works with a copied object. Also the use of this class allows to avoid invalid graphs because it checks:
If a graph contains custom layers and shape inference is required, you should add Functions for shape inference to the Network builder from custom Context.
Network builder contains the following methods for graph modification:
addLayer(...)
allows to add new layer builders to the network builder. This method creates a copy of the original layer builder, puts the copy to the network builder, and returns the ID of the layer builder which was added to the network builder.removeLayer(...)
allows to remove layer builder from the network builder by ID.connect(...)
allows to connect two layer builders using layer builder IDs and port indexes.disconnect(...)
allows to remove connection from the network builder.getLayer(...)
allows to get the layer builder from the network builder by ID.getLayerConnections(...)
allows to get all connections for a layer builder by ID.getLayers()
allows to get all layer builders.build()
allows to generate Inference Engine Network. This method validates each layer builder and graph structure and creates INetwork.The function convertToICNNNetwork(...)
converts INetwork to CNNNetwork.
InferenceEngine::Builder::Layer
class creates and modifies layers. This class allows you to modify all layer parameters, add new constant data, change type and name of the layer, and create a valid layer object.
Each default Inference Engine layer has a special builder added in order to simplify the process of layer creation. These builders hide all unnecessary methods for the specific layer and add new methods.
Below you can see the list of builders for default layers:
InferenceEngine::Builder::ArgMax
InferenceEngine::Builder::BatchNormalization
InferenceEngine::Builder::Clamp
InferenceEngine::Builder::Concat
InferenceEngine::Builder::Const
InferenceEngine::Builder::Convolution
InferenceEngine::Builder::Crop
InferenceEngine::Builder::CTCGreedyDecoder
InferenceEngine::Builder::Deconvolution
InferenceEngine::Builder::DetectionOutput
InferenceEngine::Builder::Eltwise
InferenceEngine::Builder::ELU
InferenceEngine::Builder::FullyConnected
InferenceEngine::Builder::GEN
InferenceEngine::Builder::Input
InferenceEngine::Builder::Memory
InferenceEngine::Builder::MVN
InferenceEngine::Builder::Norm
InferenceEngine::Builder::Normalize
InferenceEngine::Builder::Output
InferenceEngine::Builder::Permute
InferenceEngine::Builder::Pooling
InferenceEngine::Builder::Power
InferenceEngine::Builder::PReLU
InferenceEngine::Builder::PriorBoxClustered
InferenceEngine::Builder::PriorBox
InferenceEngine::Builder::Proposal
InferenceEngine::Builder::PSROIPooling
InferenceEngine::Builder::RegionYolo
InferenceEngine::Builder::ReLU6
InferenceEngine::Builder::ReLU
InferenceEngine::Builder::ReorgYolo
InferenceEngine::Builder::Reshape
InferenceEngine::Builder::ROIPooling
InferenceEngine::Builder::ScaleShift
InferenceEngine::Builder::Sigmoid
InferenceEngine::Builder::SimplerNMS
InferenceEngine::Builder::SoftMax
InferenceEngine::Builder::Split
InferenceEngine::Builder::TanH
InferenceEngine::Builder::Tile
The Inference Engine Deep Neural Network Builder API does not support the TensorIterator layer.
To use the DNN Builder API, include ie_builders.hpp
header which includes all Inference Engine builders.
After that, all builders will be available to use.
The DNN Builder can be created in different ways:
// Get network from the reader InferenceEngine::CNNNetwork cnnNetwork = networkReader.getNetwork();
// Create DNN builder with a name InferenceEngine::Builder::Network graph1("Example1"); // Create DNN builder from CNNNetwork InferenceEngine::Builder::Network graph2(cnnNetwork);
// Build a network InferenceEngine::INetwork::Ptr iNetwork = graph2.build(); // Create DNN builder from INetwork InferenceEngine::Builder::Network graph3(*iNetwork);
// Create an Inference Engine context InferenceEngine::Context customContext; // Add shape infer extension customContext.addExtension(customShapeInferExtension);
// Create DNN builder with custom context (all other examples also allow to create graph with custom context) InferenceEngine::Builder::Network graph4(customContext, *iNetwork);
You can modify a graph with the DNN Builder:
// Create DNN builder with a name InferenceEngine::Builder::Network graph("Example1");
// Add new layers
// Add an input layer builder in place idx_t inputLayerId = graph.addLayer(Builder::InputLayer("in").setPort(Port({1, 3, 22, 22})));
// Add a ReLU layer builder in place with a negative slope 0.1 and connect it with output port 0 of the Input layer builder // In this example, layerId is equal to new Input layer builder ID, port index is not set, because 0 is a default value ({layerId} == {layerId, 0}) idx_t relu1Id = graph.addLayer({{inputLayerId}}, Builder::ReLULayer("relu1").setNegativeSlope(0.1f));
// Add a ScaleShift layer builder in place InferenceEngine::Blob::Ptr blobWithScaleShiftBiases = make_shared_blob<float>(TensorDesc(Precision::FP32, {3}, Layout::C)); blobWithScaleShiftBiases->allocate(); auto *data = blobWithScaleShiftBiases->buffer().as< float *>(); data[0] = 1; data[1] = 2; data[2] = 3; idx_t scaleShiftId = graph.addLayer(Builder::ScaleShiftLayer("scaleShift1").setBiases(blobWithScaleShiftBiases));
// Connect ScaleShift layer in place with relu1 graph.connect({relu1Id}, {scaleShiftId}); // Also port indexes could be defined (0 is default value) builder.connect({layerId, outPortIdx}, {scaleShiftId, inPortIdx});
// Create a ReLU layer builder in place with a negative slope 0.2 using generic layer builder and connect it with scaleShift idx_t relu2Id = graph.addLayer({{scaleShiftId}}, Builder::Layer("ReLU", "relu2").setParameters({{"negative_slope", 0.2f}}).setOutputPorts({Port()}).setInputPorts({Port()}));
// All branches in the graph should end with the Output layer. The following line creates the Output layer idx_t outId = graph.addLayer({{relu2Id, 0}}, Builder::OutputLayer("out"));
// Build a network InferenceEngine::INetwork::Ptr finalNetwork = graph.build(); std::shared_ptr<InferenceEngine::ICNNNetwork> cnnNetwork = InferenceEngine::Builder::convertToICNNNetwork(finalNetwork);
// Remove the relu2 layer from the topology std::vector<InferenceEngine::Connection> connections = graph.getLayerConnections(relu2Id); for (const auto& connection : connections) { graph.disconnect(connection); } graph.removeLayer(relu2Id);
// Connect scaleShift1 and out graph.connect({scaleShiftId}, {outId}); // Build a network without relu2 InferenceEngine::INetwork::Ptr changedNetwork = graph.build();