Model Representation in OpenVINO™ Runtime#

Classes and methods use to work with models#

ov_input.get_element_type()

ov_input->get_element_type();

    partial_shape = node.output(0).get_partial_shape() # get zero output partial shape
    if not partial_shape.is_dynamic: # or partial_shape.is_static
        static_shape = partial_shape.get_shape()

    ov::Shape static_shape;
    ov::PartialShape partial_shape = node->output(0).get_partial_shape(); // get zero output partial shape
    if (!partial_shape.is_dynamic() /* or partial_shape.is_static() */) {
        static_shape = partial_shape.get_shape();
    }

Build a Model in OpenVINO™ Runtime#

You can create a model from source by constructing it with operations from an available opsetX operation set. Each operation set integrates a list of pre-compiled operations that work for this purpose. In other words, opsetX defines a set of operations for building a model. To create an ov::Model instance from opset15 operations, include the following libraries:

import openvino as ov

#include <openvino/core/model.hpp>
#include <openvino/opsets/opset8.hpp>

Then, you can create a model, as shown in these two examples:

def create_simple_model():
    # This example shows how to create ov::Function
    #
    # Parameter--->Multiply--->Add--->Result
    #    Constant---'          /
    #              Constant---'
    data = ops.parameter([3, 1, 2], ov.Type.f32)
    mul_constant = ops.constant([1.5], ov.Type.f32)
    mul = ops.multiply(data, mul_constant)
    add_constant = ops.constant([0.5], ov.Type.f32)
    add = ops.add(mul, add_constant)
    res = ops.result(add)
    return ov.Model([res], [data], "model")

std::shared_ptr<ov::Model> create_simple_model() {
    // This example shows how to create ov::Model
    //
    // Parameter--->Multiply--->Add--->Result
    //    Constant---'          /
    //              Constant---'

    // Create opset8::Parameter operation with static shape
    auto data = std::make_shared<ov::opset8::Parameter>(ov::element::f32, ov::Shape{3, 1, 2});

    auto mul_constant = ov::opset8::Constant::create(ov::element::f32, ov::Shape{1}, {1.5});
    auto mul = std::make_shared<ov::opset8::Multiply>(data, mul_constant);

    auto add_constant = ov::opset8::Constant::create(ov::element::f32, ov::Shape{1}, {0.5});
    auto add = std::make_shared<ov::opset8::Add>(mul, add_constant);

    // Create opset8::Result operation
    auto res = std::make_shared<ov::opset8::Result>(mul);

    // Create OpenVINO function
    return std::make_shared<ov::Model>(ov::ResultVector{res}, ov::ParameterVector{data});
}

def create_advanced_model():
    # Advanced example with multi output operation
    #
    # Parameter->Split---0-->Result
    #               | `--1-->Relu-->Result
    #               `----2-->Result
    data = ops.parameter(ov.Shape([1, 3, 64, 64]), ov.Type.f32)
    # Create Constant for axis value
    axis_const = ops.constant(1, dtype=ov.Type.i64)

    # Create opset12::Split operation that splits input to three slices across 1st dimension
    split = ops.split(data, axis_const, 3)

    # Create opset12::Relu operation that takes 1st Split output as input
    relu = ops.relu(split.output(1))

    # Results operations will be created automatically based on provided OutputVector
    return ov.Model([split.output(0), relu.output(0), split.output(2)], [data], "model")

std::shared_ptr<ov::Model> create_advanced_model() {
    // Advanced example with multi output operation
    //
    // Parameter->Split---0-->Result
    //               | `--1-->Relu-->Result
    //               `----2-->Result

    auto data = std::make_shared<ov::opset8::Parameter>(ov::element::f32, ov::Shape{1, 3, 64, 64});

    // Create Constant for axis value
    auto axis_const = ov::opset8::Constant::create(ov::element::i64, ov::Shape{} /*scalar shape*/, {1});

    // Create opset8::Split operation that splits input to three slices across 1st dimension
    auto split = std::make_shared<ov::opset8::Split>(data, axis_const, 3);

    // Create opset8::Relu operation that takes 1st Split output as input
    auto relu = std::make_shared<ov::opset8::Relu>(split->output(1) /*specify explicit output*/);

    // Results operations will be created automatically based on provided OutputVector
    return std::make_shared<ov::Model>(ov::OutputVector{split->output(0), relu, split->output(2)},
                                       ov::ParameterVector{data});
}

Debug a Model#

Here is a list of features that can help you debug models in OpenVINO:

• DENABLE_OPENVINO_DEBUG=ON - used to rebuild the OpenVINO Runtime library, to receive additional messages about applied model modifications

• ov::pass::VisualizeTree - used to save ov::Model to xDot format. It can be parametrized via environment variables:

  • OV_VISUALIZE_TREE_OUTPUT_SHAPES=1 - visualize shapes

  • OV_VISUALIZE_TREE_OUTPUT_TYPES=1 - visualize types

  • OV_VISUALIZE_TREE_MIN_MAX_DENORMAL=1 - pretty denormal values

  • OV_VISUALIZE_TREE_RUNTIME_INFO=1 - print runtime information

  • OV_VISUALIZE_TREE_IO=1 - print I/O ports

  • OV_VISUALIZE_TREE_MEMBERS_NAME=1 - print member names

• ov::serialize - used to save a model to IR:

  Python

  def serialize_example(m : ov.Model):
      ov.serialize(m, xml_path='model.xml', bin_path='model.bin')
  

  C++

  void serialize_example(const std::shared_ptr<ov::Model>& model) {
      ov::serialize(model, "/path/to/file/model.xml", "/path/to/file/model.bin");
  }
  

  Note

  While save_model is the preferred method for saving a model to IR, it is recommended to use serialize for debugging purposes. It requires less time and computational resources, as it does not apply weight compression.

• xDot format to image conversion - used to visualize model graphs:

  Python

  def visualize_example(m : ov.Model):
      # Need import:
      # * import openvino.runtime.passes as passes
      pass_manager = passes.Manager()
      pass_manager.register_pass(passes.VisualizeTree(file_name='image.svg'))
      pass_manager.run_passes(m)
  

  C++

  void visualize_example(const std::shared_ptr<ov::Model>& m) {
      // Need include:
      // * openvino/pass/manager.hpp
      // * openvino/pass/visualize_tree.hpp
      ov::pass::Manager manager;
  
      // Serialize ov::Model to before.svg file before transformation
      manager.register_pass<ov::pass::VisualizeTree>("image.svg");
  
      manager.run_passes(m);
  }
  

Additional Resources#

• Available Operation Sets.

• OpenVINO™ Runtime Extensibility Developer Guide.

• Transformations Developer Guide.