Converting a TensorFlow Model

This page provides general instructions on how to run model conversion from a TensorFlow format to the OpenVINO IR format. The instructions are different depending on whether your model was created with TensorFlow v1.X or TensorFlow v2.X.

Note

TensorFlow models can be loaded by openvino.Core.read_model or openvino.Core.compile_model methods by OpenVINO runtime API without preparing OpenVINO IR first. Refer to the inference example for more details. Using openvino.convert_model is still recommended if model load latency matters for the inference application.

Note

Examples below that convert TensorFlow models from a file, do not require any version of TensorFlow to be installed on the system, except in cases when the tensorflow module is imported explicitly.

Converting TensorFlow 2 Models

TensorFlow 2.X officially supports two model formats: SavedModel and Keras H5 (or HDF5). Below are the instructions on how to convert each of them.

SavedModel Format

A model in the SavedModel format consists of a directory with a saved_model.pb file and two subfolders: variables and assets inside. To convert a model, run conversion with the directory as the model argument:

import openvino as ov
ov_model = ov.convert_model('path_to_saved_model_dir')

ovc path_to_saved_model_dir

Keras H5 Format

If you have a model in the HDF5 format, load the model using TensorFlow 2 and serialize it in the SavedModel format. Here is an example of how to do it:

import tensorflow as tf
model = tf.keras.models.load_model('model.h5')
tf.saved_model.save(model,'model')

Converting a Keras H5 model with a custom layer to the SavedModel format requires special considerations. For example, the model with a custom layer CustomLayer from custom_layer.py is converted as follows:

import tensorflow as tf
from custom_layer import CustomLayer
model = tf.keras.models.load_model('model.h5', custom_objects={'CustomLayer': CustomLayer})
tf.saved_model.save(model,'model')

Then follow the above instructions for the SavedModel format.

Note

Avoid using any workarounds or hacks to resave TensorFlow 2 models into TensorFlow 1 formats.

Converting TensorFlow 1 Models

Converting Frozen Model Format

To convert a TensorFlow model, run model conversion with the path to the input model *.pb* file:

import openvino as ov
ov_model = ov.convert_model('your_model_file.pb')

ovc your_model_file.pb

Converting Non-Frozen Model Formats

There are three ways to store non-frozen TensorFlow models.

1. SavedModel format. In this case, a model consists of a special directory with a .pb file and several subfolders: variables, assets, and assets.extra. For more information about the SavedModel directory, refer to the README file in the TensorFlow repository. To convert such TensorFlow model, run the conversion similarly to other model formats and pass a path to the directory as a model argument:

import openvino as ov
ov_model = ov.convert_model('path_to_saved_model_dir')

ovc path_to_saved_model_dir

2. Checkpoint. In this case, a model consists of two files: inference_graph.pb (or inference_graph.pbtxt) and checkpoint_file.ckpt. If you do not have an inference graph file, refer to the Freezing Custom Models in Python section. To convert the model with the inference graph in .pb format, provide paths to both files as an argument for ovc or openvino.convert_model:

import openvino as ov
ov_model = ov.convert_model(['path_to_inference_graph.pb', 'path_to_checkpoint_file.ckpt'])

ovc path_to_inference_graph.pb path_to_checkpoint_file.ckpt

To convert the model with the inference graph in the .pbtxt format, specify the path to .pbtxt file instead of the .pb file. The conversion API automatically detects the format of the provided file, there is no need to specify the model file format explicitly when calling ovc or openvino.convert_model in all examples in this document.

3. MetaGraph. In this case, a model consists of three or four files stored in the same directory: model_name.meta, model_name.index, model_name.data-00000-of-00001 (the numbers may vary), and checkpoint (optional). To convert such a TensorFlow model, run the conversion providing a path to .meta file as an argument:

import openvino as ov
ov_model = ov.convert_model('path_to_meta_graph.meta')

ovc path_to_meta_graph.meta

Freezing Custom Models in Python

When a model is defined in Python code, you must create an inference graph file. Graphs are usually built in a form that allows model training. That means all trainable parameters are represented as variables in the graph. To be able to use such a graph with the model conversion API, it should be frozen first before passing to the openvino.convert_model function:

import tensorflow as tf
from tensorflow.python.framework import graph_io
frozen = tf.compat.v1.graph_util.convert_variables_to_constants(sess, sess.graph_def, ["name_of_the_output_node"])

import openvino as ov
ov_model = ov.convert_model(frozen)

Where:

  • sess is the instance of the TensorFlow Session object where the network topology is defined.

  • ["name_of_the_output_node"] is the list of output node names in the graph; frozen graph will include only those nodes from the original sess.graph_def that are directly or indirectly used to compute given output nodes. The 'name_of_the_output_node' is an example of a possible output node name. You should derive the names based on your own graph.

Converting TensorFlow Models from Memory Using Python API

Model conversion API supports passing TensorFlow/TensorFlow2 models directly from memory.

  • tf.keras.Model

    import openvino as ov
model = tf.keras.applications.ResNet50(weights="imagenet")
ov_model = ov.convert_model(model)

  • tf.keras.layers.Layer. Requires saving model to TensorFlow saved_model file format and then loading to openvino.convert_model. Saving to the file and then restoring is required due to a known bug in openvino.convert_model that ignores model signature.

    import tensorflow_hub as hub
import openvino as ov

model = hub.KerasLayer("https://tfhub.dev/google/imagenet/mobilenet_v1_100_224/classification/5")
model.build([None, 224, 224, 3])
model.save('mobilenet_v1_100_224')  # use a temporary directory

ov_model = ov.convert_model('mobilenet_v1_100_224')

  • tf.Module. Requires setting shapes in input parameter.

    import tensorflow as tf
import openvino as ov

class MyModule(tf.Module):
   def __init__(self, name=None):
      super().__init__(name=name)
      self.constant1 = tf.constant(5.0, name="var1")
      self.constant2 = tf.constant(1.0, name="var2")
   def __call__(self, x):
      return self.constant1 * x + self.constant2

model = MyModule(name="simple_module")
ov_model = ov.convert_model(model, input=[-1])

Note

There is a known bug in openvino.convert_model on using tf.Variable nodes in the model graph. The results of the conversion of such models are unpredictable. It is recommended to save a model with tf.Variable into TensorFlow Saved Model format and load it with openvino.convert_model.

  • tf.compat.v1.Graph

    with tf.compat.v1.Session() as sess:
   inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1')
   inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2')
   output = tf.nn.relu(inp1 + inp2, name='Relu')
   tf.compat.v1.global_variables_initializer()
   model = sess.graph

import openvino as ov
ov_model = ov.convert_model(model)

  • tf.compat.v1.GraphDef

    with tf.compat.v1.Session() as sess:
   inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1')
   inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2')
   output = tf.nn.relu(inp1 + inp2, name='Relu')
   tf.compat.v1.global_variables_initializer()
   model = sess.graph_def

import openvino as ov
ov_model = ov.convert_model(model)

  • tf.function

    @tf.function(
   input_signature=[tf.TensorSpec(shape=[1, 2, 3], dtype=tf.float32),
                    tf.TensorSpec(shape=[1, 2, 3], dtype=tf.float32)])
def func(x, y):
   return tf.nn.sigmoid(tf.nn.relu(x + y))

import openvino as ov
ov_model = ov.convert_model(func)

  • tf.compat.v1.session

    with tf.compat.v1.Session() as sess:
   inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1')
   inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2')
   output = tf.nn.relu(inp1 + inp2, name='Relu')
   tf.compat.v1.global_variables_initializer()

   import openvino as ov
   ov_model = ov.convert_model(sess)

  • tf.train.checkpoint

    model = tf.keras.Model(...)
checkpoint = tf.train.Checkpoint(model)
save_path = checkpoint.save(save_directory)
# ...
checkpoint.restore(save_path)

import openvino as ov
ov_model = ov.convert_model(checkpoint)

Supported TensorFlow and TensorFlow 2 Keras Layers

For the list of supported standard layers, refer to the Supported Operations page.

Summary

In this document, you learned:

  • Basic information about how the model conversion API works with TensorFlow models.

  • Which TensorFlow models are supported.

  • How to freeze a TensorFlow model.