Convert to OpenVINO IR#
OpenVINO IR is the proprietary model format used by OpenVINO, typically obtained by converting models of supported frameworks:
The
convert_model()
method:This is the only method applicable to PyTorch models.
List of supported formats:
Python objects:
torch.nn.Module
torch.jit.ScriptModule
torch.jit.ScriptFunction
torch.export.ExportedProgram
import torchvision import openvino as ov model = torchvision.models.resnet50(weights='DEFAULT') ov_model = ov.convert_model(model) compiled_model = ov.compile_model(ov_model, "AUTO")
For more details on conversion, refer to the guide and an example tutorial on this topic.
The
convert_model()
method:When you use the
convert_model()
method, you have more control and you can specify additional adjustments forov.Model
. Theread_model()
andcompile_model()
methods are easier to use, however, they do not have such capabilities. Withov.Model
you can choose to optimize, compile and run inference on it or serialize it into a file for subsequent use.List of supported formats:
Files:
SavedModel -
<SAVED_MODEL_DIRECTORY>
or<INPUT_MODEL>.pb
Checkpoint -
<INFERENCE_GRAPH>.pb
or<INFERENCE_GRAPH>.pbtxt
MetaGraph -
<INPUT_META_GRAPH>.meta
Python objects:
tf.keras.Model
tf.keras.layers.Layer
tf.Module
tf.compat.v1.Graph
tf.compat.v1.GraphDef
tf.function
tf.compat.v1.session
tf.train.checkpoint
import openvino as ov ov_model = ov.convert_model("saved_model.pb") compiled_model = ov.compile_model(ov_model, "AUTO")
For more details on conversion, refer to the guide and an example tutorial on this topic.
The
read_model()
andcompile_model()
methods:List of supported formats:
Files:
SavedModel -
<SAVED_MODEL_DIRECTORY>
or<INPUT_MODEL>.pb
Checkpoint -
<INFERENCE_GRAPH>.pb
or<INFERENCE_GRAPH>.pbtxt
MetaGraph -
<INPUT_META_GRAPH>.meta
import openvino as ov core = ov.Core() ov_model = core.read_model("saved_model.pb") compiled_model = ov.compile_model(ov_model, "AUTO")
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
The
compile_model()
method:List of supported formats:
Files:
SavedModel -
<SAVED_MODEL_DIRECTORY>
or<INPUT_MODEL>.pb
Checkpoint -
<INFERENCE_GRAPH>.pb
or<INFERENCE_GRAPH>.pbtxt
MetaGraph -
<INPUT_META_GRAPH>.meta
ov::CompiledModel compiled_model = core.compile_model("saved_model.pb", "AUTO");
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
The
compile_model()
method:List of supported formats:
Files:
SavedModel -
<SAVED_MODEL_DIRECTORY>
or<INPUT_MODEL>.pb
Checkpoint -
<INFERENCE_GRAPH>.pb
or<INFERENCE_GRAPH>.pbtxt
MetaGraph -
<INPUT_META_GRAPH>.meta
ov_compiled_model_t* compiled_model = NULL; ov_core_compile_model_from_file(core, "saved_model.pb", "AUTO", 0, &compiled_model);
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
You can use ovc
command-line tool to convert a model to IR. The obtained IR can
then be read by read_model()
and inferred.
ovc <INPUT_MODEL>.pb
For details on the conversion, refer to the article.
The
convert_model()
method:When you use the
convert_model()
method, you have more control and you can specify additional adjustments forov.Model
. Theread_model()
andcompile_model()
methods are easier to use, however, they do not have such capabilities. Withov.Model
you can choose to optimize, compile and run inference on it or serialize it into a file for subsequent use.List of supported formats:
Files:
<INPUT_MODEL>.tflite
import openvino as ov ov_model = ov.convert_model("<INPUT_MODEL>.tflite") compiled_model = ov.compile_model(ov_model, "AUTO")
For more details on conversion, refer to the guide and an example tutorial on this topic.
The
read_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.tflite
import openvino as ov core = ov.Core() ov_model = core.read_model("<INPUT_MODEL>.tflite") compiled_model = ov.compile_model(ov_model, "AUTO")
The
compile_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.tflite
import openvino as ov compiled_model = ov.compile_model("<INPUT_MODEL>.tflite", "AUTO")
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
The
compile_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.tflite
ov::CompiledModel compiled_model = core.compile_model("<INPUT_MODEL>.tflite", "AUTO");
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
The
compile_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.tflite
ov_compiled_model_t* compiled_model = NULL; ov_core_compile_model_from_file(core, "<INPUT_MODEL>.tflite", "AUTO", 0, &compiled_model);
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
The
convert_model()
method:You can use
mo
command-line tool to convert a model to IR. The obtained IR can then be read byread_model()
and inferred.List of supported formats:
Files:
<INPUT_MODEL>.tflite
ovc <INPUT_MODEL>.tflite
For details on the conversion, refer to the article.
The
convert_model()
method:When you use the
convert_model()
method, you have more control and you can specify additional adjustments forov.Model
. Theread_model()
andcompile_model()
methods are easier to use, however, they do not have such capabilities. Withov.Model
you can choose to optimize, compile and run inference on it or serialize it into a file for subsequent use.List of supported formats:
Files:
<INPUT_MODEL>.onnx
import openvino as ov ov_model = ov.convert_model("<INPUT_MODEL>.onnx") compiled_model = ov.compile_model(ov_model, "AUTO")
For more details on conversion, refer to the guide and an example tutorial on this topic.
The
read_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.onnx
import openvino as ov core = ov.Core() ov_model = core.read_model("<INPUT_MODEL>.onnx") compiled_model = ov.compile_model(ov_model, "AUTO")
The
compile_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.onnx
import openvino as ov compiled_model = ov.compile_model("<INPUT_MODEL>.onnx", "AUTO")
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
The
compile_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.onnx
ov::CompiledModel compiled_model = core.compile_model("<INPUT_MODEL>.onnx", "AUTO");
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
The
compile_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.onnx
ov_compiled_model_t* compiled_model = NULL; ov_core_compile_model_from_file(core, "<INPUT_MODEL>.onnx", "AUTO", 0, &compiled_model);
For details on the conversion, refer to the article
The
convert_model()
method:You can use
mo
command-line tool to convert a model to IR. The obtained IR can then be read byread_model()
and inferred.List of supported formats:
Files:
<INPUT_MODEL>.onnx
ovc <INPUT_MODEL>.onnx
For details on the conversion, refer to the article
The
convert_model()
method:When you use the
convert_model()
method, you have more control and you can specify additional adjustments forov.Model
. Theread_model()
andcompile_model()
methods are easier to use, however, they do not have such capabilities. Withov.Model
you can choose to optimize, compile and run inference on it or serialize it into a file for subsequent use.List of supported formats:
Files:
<INPUT_MODEL>.pdmodel
Python objects:
paddle.hapi.model.Model
paddle.fluid.dygraph.layers.Layer
paddle.fluid.executor.Executor
import openvino as ov ov_model = ov.convert_model("<INPUT_MODEL>.pdmodel") compiled_model = ov.compile_model(ov_model, "AUTO")
For more details on conversion, refer to the guide and an example tutorial on this topic.
The
read_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.pdmodel
import openvino as ov core = ov.Core() ov_model = core.read_model("<INPUT_MODEL>.pdmodel") compiled_model = ov.compile_model(ov_model, "AUTO")
The
compile_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.pdmodel
import openvino as ov compiled_model = ov.compile_model("<INPUT_MODEL>.pdmodel", "AUTO")
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
The
compile_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.pdmodel
ov::CompiledModel compiled_model = core.compile_model("<INPUT_MODEL>.pdmodel", "AUTO");
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
The
compile_model()
method:List of supported formats:
Files:
<INPUT_MODEL>.pdmodel
ov_compiled_model_t* compiled_model = NULL; ov_core_compile_model_from_file(core, "<INPUT_MODEL>.pdmodel", "AUTO", 0, &compiled_model);
For a guide on how to run inference, see how to Integrate OpenVINO™ with Your Application.
The
convert_model()
method:You can use
mo
command-line tool to convert a model to IR. The obtained IR can then be read byread_model()
and inferred.List of supported formats:
Files:
<INPUT_MODEL>.pdmodel
ovc <INPUT_MODEL>.pdmodel
For details on the conversion, refer to the article.
These are basic examples, for detailed conversion instructions, see the individual guides on PyTorch, ONNX, TensorFlow, TensorFlow Lite, and PaddlePaddle.
Refer to the list of all supported conversion options in Conversion Parameters.
IR Conversion Benefits#
save_model
and then load it with read_model
as
needed. This should improve the time it takes the model to make the first inference as it
avoids the conversion step.Here is an example of how to benefit from OpenVINO IR, saving a model once and running it multiple times:
# Run once
import openvino as ov
import tensorflow as tf
# 1. Convert model created with TF code
model = tf.keras.applications.resnet50.ResNet50(weights="imagenet")
ov_model = ov.convert_model(model)
# 2. Save model as OpenVINO IR
ov.save_model(ov_model, 'model.xml', compress_to_fp16=True) # enabled by default
# Repeat as needed
import openvino as ov
# 3. Load model from file
core = ov.Core()
ov_model = core.read_model("model.xml")
# 4. Compile model from memory
compiled_model = ov.compile_model(ov_model)