logo
  • Get Started
  • Documentation
  • Tutorials
  • API Reference
  • Model Zoo
  • Resources
English Chinese

OpenVINO 2022.1 introduces a new version of OpenVINO API (API 2.0). For more information on the changes and transition steps, see the transition guide

Converting and Preparing Models

  • Convert model with Model Optimizer
    • Setting Input Shapes
    • Cutting Off Parts of a Model
    • Embedding Preprocessing Computation
    • Compression of a Model to FP16
    • Converting a TensorFlow* Model
    • Converting an ONNX Model
    • Converting a PyTorch* Model
    • Converting a PaddlePaddle* Model
    • Converting an MXNet* Model
    • Converting a Caffe* Model
    • Converting a Kaldi* Model
    • Model Conversion Tutorials
      • Convert TensorFlow Attention OCR Model
      • Convert TensorFlow BERT Model
      • Convert TensorFlow CRNN Model
      • Convert TensorFlow DeepSpeech Model
      • Convert TensorFlow EfficientDet Models
      • Convert TensorFlow FaceNet Models
      • Convert TensorFlow GNMT Model
      • Convert TensorFlow Language Model on One Billion Word Benchmark
      • Convert TensorFlow Neural Collaborative Filtering Model
      • Convert TensorFlow Object Detection API Models
      • Converting TensorFlow RetinaNet Model
      • Convert TensorFlow Slim Image Classification Model Library Models
      • Convert TensorFlow Wide and Deep Family Models
      • Convert TensorFlow XLNet Model
      • Convert TensorFlow YOLO Models
      • Convert ONNX* Faster R-CNN Model
      • Convert ONNX* GPT-2 Model
      • Convert ONNX* Mask R-CNN Model
      • Convert PyTorch* BERT-NER Model
      • Convert PyTorch Cascade RCNN R-101 Model
      • Convert PyTorch* F3Net Model
      • Convert PyTorch* QuartzNet Model
      • Convert PyTorch* RCAN Model
      • Convert PyTorch* RNN-T Model
      • Convert PyTorch* YOLACT Model
      • Convert MXNet GluonCV* Models
      • Convert MXNet Style Transfer Model
      • Convert Kaldi* ASpIRE Chain Time Delay Neural Network (TDNN) Model
    • Model Optimizer Frequently Asked Questions
  • Model Downloader and other automation tools

Deploying Inference

  • Performing inference with OpenVINO Runtime
    • Integrate OpenVINO™ with Your Application
      • Model Representation in OpenVINO™ Runtime
      • OpenVINO™ Inference Request
      • OpenVINO™ Python API exclusives
    • Changing input shapes
      • Troubleshooting Reshape Errors
    • Working with devices
      • Query device properties, configuration
      • CPU device
      • GPU device
        • Remote Tensor API of GPU Plugin
      • VPU devices
        • MYRIAD device
        • HDDL device
      • GNA device
      • Arm® CPU device
    • Optimize Preprocessing
      • Preprocessing API - details
      • Layout API overview
      • Use Case - Integrate and Save Preprocessing Steps Into IR
    • Dynamic Shapes
      • When Dynamic Shapes API is Not Applicable
    • Automatic device selection
      • Debugging Auto-Device Plugin
    • Running on multiple devices simultaneously
    • Heterogeneous execution
    • High-level Performance Hints
    • Automatic Batching
    • Stateful models
  • Transition to OpenVINO™ API 2.0
    • Installation & Deployment
    • Inference Pipeline
    • Configuring Devices
    • Preprocessing
    • Model Creation in Runtime
  • Deploy with OpenVINO
    • Deployment Manager
    • Local distribution
  • Compile Tool

Tuning for Performance

  • Introduction to Performance Optimization
  • Getting Performance Numbers
  • Model Optimization Guide
    • Optimizing models post-training
      • Quantizing Model
        • DefaultQuantization Method
      • Quantizing Model with Accuracy Control
        • AccuracyAwareQuantization Method
      • Quantization Best Practices
        • Saturation Issue
      • API Reference
      • Command-line Interface
        • Simplified Mode
        • Configuration File Description
      • Examples
        • API Examples
          • Quantizatiing Image Classification Model
          • Quantizatiing Object Detection Model with Accuracy Control
          • Quantizatiing Cascaded Model
          • Quantizatiing Semantic Segmentation Model
          • Quantizatiing 3D Segmentation Model
          • Quantizatiing for GNA Device
        • Command-line Example
      • Post-training Optimization Tool Frequently Asked Questions
    • Neural Network Compression Framework
    • (Experimental) Protecting Model
  • Runtime Inference Optimizations
    • General Optimizations
    • Optimizing for the Latency
      • Model Caching Overview
    • Optimizing for Throughput
    • Using Advanced Throughput Options: Streams and Batching
    • Further Low-Level Implementation Details
  • Tuning Utilities
    • Deep Learning accuracy validation framework
      • Adapters
      • Annotation Converters
      • Custom Evaluators for Accuracy Checker
      • Data Readers
      • How to configure Caffe launcher
      • How to configure G-API launcher
      • How to configure MXNet launcher
      • How to configure ONNX Runtime launcher
      • How to configure OpenCV launcher
      • How to configure OpenVINO™ launcher
      • How to configure PaddlePaddle launcher
      • How to configure PyTorch launcher
      • How to configure TensorFlow 2.0 launcher
      • How to configure TensorFlow Lite launcher
      • How to configure TensorFlow launcher
      • How to use predefined configuration files
      • Metrics
      • Postprocessors
      • Preprocessors
      • Sample
    • Dataset Preparation Guide
    • Cross Check Tool
  • Performance Benchmarks
    • Intel® Distribution of OpenVINO™ toolkit Benchmark Results
      • Performance Information Frequently Asked Questions
      • Download Performance Data Spreadsheet in MS Excel* Format
      • Model Accuracy for INT8 and FP32 Precision
    • OpenVINO™ Model Server Benchmark Results

Graphical Web Interface for OpenVINO™ toolkit

  • OpenVINO™ Deep Learning Workbench Overview
  • Installation
    • Prerequisites
    • Run the DL Workbench Locally
      • Advanced DL Workbench Configurations
      • Work with Docker Container
    • Run the DL Workbench in the Intel® DevCloud for the Edge
  • Get Started
    • Import Model
    • Create Project
    • Educational Resources about DL Workbench
      • DL Workbench Key Concepts
  • Tutorials
    • Object Detection Model (YOLOv4)
    • Object Detection Model (SSD_mobilenet)
    • Classification Model (mobilenet)
    • Classification Model (squeezenet)
    • Instance Segmentation Model (mask R-cnn)
    • Semantic Segmentation Model (deeplab)
    • Style Transfer Model (fast-nst-onnx)
    • NLP Model (BERT)
  • User Guide
    • Obtain Models
      • Import Open Model Zoo Models
      • Import Original Model
        • Import Original Model Recommendations
    • Obtain Datasets
      • Dataset Types
        • Cut Datasets
    • Select Environment
      • Work with Remote Targets
        • Profile on Remote Machine
        • Set Up Remote Target
        • Register Remote Target in DL Workbench
        • Manipulate Remote Machines
    • Optimize Model Performance
    • Explore Inference Configurations
      • Run Inference
      • View Inference Results
      • Compare Performance between Two Versions of a Model
      • Visualize Model
    • Visualize Model Output
    • Create Accuracy Report
      • Accuracy Configuration
      • Set Accuracy Configuration
      • Interpret Accuracy Report Results
    • Create Deployment Package
      • Deploy and Integrate Performance Criteria into Application
    • Export Project
    • Learn OpenVINO in DL Workbench
      • Learn Model Inference with OpenVINO™ API in JupyterLab* Environment
    • Restore DL Workbench State
    • Run DL Workbench Securely
      • Enable Authentication in DL Workbench
      • Configure Transport Layer Security (TLS)
  • Troubleshooting
    • Troubleshooting for DL Workbench in the Intel® DevCloud for the Edge

Media Processing and Computer Vision Libraries

  • Intel® Deep Learning Streamer
  • Introduction to OpenCV Graph API (G-API)
    • Graph API Kernel API
    • Implementing a Face Beautification Algorithm
    • Building a Face Analytics Pipeline
  • OpenCV* Developer Guide
  • OpenCL™ Developer Guide

Add-Ons

  • OpenVINO™ Model Server
    • Quickstart Guide
    • Architecture
    • Model Repository
    • Starting the Server
      • Single-Model Mode
      • Multiple-Model mode with a Config File
      • Model Server in Docker Containers
      • Bare Metal and Virtual Hosts
      • Model Server Parameters
      • Using Cloud Storage as a Model Repository
      • Using AI Accelerators
      • Model Version Policy
      • Batch, Shape and Layout
      • Online Configuration Updates
      • Security Considerations
    • API Reference Guide
      • gRPC API
      • RESTful API
    • Clients
    • Directed Acyclic Graph (DAG) Scheduler
      • Demultiplexing in DAG
      • Custom Node Development Guide
    • Support for Binary Input Data
      • Convert TensorFlow Models to accept binary inputs
    • Model Cache
    • CPU Extensions
    • Dynamic Input Parameters
      • Dynamic batch size with OpenVINO™ Model Server Demultiplexer
      • Dynamic Batch Size with Automatic Model Reloading
      • Dynamic Shape with Automatic Model Reloading
      • Dynamic Shape with a Custom Node
      • Dynamic Shape with Binary Inputs
      • Dynamic Shape with dynamic IR/ONNX Model
    • Serving Stateful Models
    • Custom Model Loader
    • Performance tuning
    • Deploy Model Server in Kubernetes
      • Helm Deployment
      • Kubernetes Operator
      • OpenShift Operator
    • Demos
      • Age and Gender Recognition via REST API
      • Horizontal Text Detection in Real-Time
      • Optical Character Recognition with Directed Acyclic Graph
      • Face Detection Demo
      • Face Blur Pipeline Demo with OVMS
      • Single Face Analysis Pipeline Demo
      • Multi Faces Analysis Pipeline Demo
      • Model Ensemble Pipeline Demo
      • Image Classification Demos
        • Image Classification Demo (Python)
        • Image Classification Demo (C++)
        • Image Classification Demo (Go)
      • Prediction Example with an ONNX Model
      • Person, vehicle, bike detection with multiple data sources
      • Vehicle Analysis Pipeline Demo
      • Real Time Stream Analysis Demo
      • BERT Question Answering Demo
      • Speech Recognition on Kaldi Model
      • Benchmark Client
        • Benchmark Client (Python)
        • Benchmark Client (C++)
    • Troubleshooting
  • OpenVINO™ Security Add-on

OpenVINO Extensibility

  • OpenVINO Extensibility Mechanism
    • Custom OpenVINO™ Operations
    • Frontend Extensions
    • How to Implement Custom GPU Operations
    • How to Implement Custom Layers for VPU (Intel® Neural Compute Stick 2)
    • Model Optimizer Extensibility
      • Extending Model Optimizer with Caffe* Python Layers
  • Overview of Transformations API
    • OpenVINO Model Pass
    • OpenVINO Matcher Pass
    • OpenVINO Graph Rewrite Pass
  • OpenVINO Plugin Developer Guide
    • Implement Plugin Functionality
    • Implement Executable Network Functionality
    • Implement Synchronous Inference Request
    • Implement Asynchronous Inference Request
    • Build Plugin Using CMake*
    • Plugin Testing
    • Advanced Topics
      • Quantized networks compute and restrictions
      • OpenVINO™ Low Precision Transformations
        • Attributes
          • AvgPoolPrecisionPreserved
          • IntervalsAlignment
          • PerTensorQuantization
          • PrecisionPreserved
          • Precisions
          • QuantizationAlignment
        • Step 1. Prerequisites transformations
        • Step 2. Markup transformations
        • Step 3. Main transformations
        • Step 4. Cleanup transformations
    • Plugin API Reference
      • Inference Engine Plugin API
        • Asynchronous Inference Request base classes
          • AsyncInferRequestThreadSafeDefault
        • Blob creation and memory utilities
        • Error handling and debug helpers
          • DescriptionBuffer
        • Executable Network base classes
          • IExecutableNetworkInternal
          • ExecutableNetworkThreadSafeDefault
        • Execution graph utilities
          • ExecGraphInfoSerialization
          • ExecutionNode
        • FP16 to FP32 precision utilities
          • PrecisionUtils
        • File utilities
          • FileUtils
        • ITT profiling utilities
          • openvino
          • ScopedTask
          • TaskChain
        • Inference Request base classes
          • IInferRequestInternal
        • Plugin base classes
          • PluginConfigInternalParams
          • ICore
          • IInferencePlugin
        • Preprocessing API
        • System configuration utilities
        • Threading utilities
          • ExecutorManager
          • IStreamsExecutor
          • ITaskExecutor
          • CPUStreamsExecutor
          • ImmediateExecutor
        • Variable state base classes
          • IVariableStateInternal
        • XML helper utilities
          • XMLParseUtils
          • parse_result
      • Inference Engine Transformation API
        • Common optimization passes
          • ngraph
          • AddFakeQuantizeFusion
          • AddOldApiMapToParameters
          • AddTransformation
          • AlignQuantizationIntervals
          • AlignQuantizationParameters
          • AvgPoolPrecisionPreservedAttribute
          • AvgPoolTransformation
          • BatchToSpaceFusion
          • BidirectionalGRUSequenceDecomposition
          • BidirectionalLSTMSequenceDecomposition
          • BidirectionalRNNSequenceDecomposition
          • BidirectionalSequenceDecomposition
          • BinarizeWeights
          • BroadcastConstRangeReplacement
          • BroadcastElementwiseFusion
          • ClampFusion
          • ClampTransformation
          • CompressFloatConstants
          • CompressFloatConstantsImpl
          • ConcatReduceFusion
          • ConcatTransformation
          • ConvStridesPropagation
          • ConvToBinaryConv
          • ConvertBatchToSpace
          • ConvertCompressedOnlyToLegacy
          • ConvertDeformableConv8To1
          • ConvertDetectionOutput1ToDetectionOutput8
          • ConvertDetectionOutput8ToDetectionOutput1
          • ConvertGRUSequenceMatcher
          • ConvertGRUSequenceToTensorIterator
          • ConvertGather0D
          • ConvertGather1ToGather7
          • ConvertGather7ToGather1
          • ConvertGather7ToGather8
          • ConvertGather8ToGather7
          • ConvertInterpolate1ToInterpolate4
          • ConvertLSTMSequenceMatcher
          • ConvertLSTMSequenceToTensorIterator
          • ConvertMVN1ToMVN6
          • ConvertMaxPool1ToMaxPool8
          • ConvertMaxPool8ToMaxPool1
          • ConvertNmsGatherPathToUnsigned
          • ConvertPadToGroupConvolution
          • ConvertPriorBox8To0
          • ConvertQuantizeDequantize
          • ConvertRNNSequenceMatcher
          • ConvertRNNSequenceToTensorIterator
          • ConvertScatterElementsToScatter
          • ConvertSoftMax1ToSoftMax8
          • ConvertSoftMax8ToSoftMax1
          • ConvertSpaceToBatch
          • ConvertSubtractConstant
          • ConvertTensorIteratorToGRUSequence
          • ConvertTensorIteratorToLSTMSequence
          • ConvertTensorIteratorToRNNSequence
          • ConvolutionBackpropDataTransformation
          • ConvolutionTransformation
          • CreateAttribute
          • CreatePrecisionsDependentAttribute
          • DepthToSpaceFusion
          • DepthToSpaceTransformation
          • DilatedConvolutionConverter
          • DisableDecompressionConvertConstantFolding
          • DisableRandomUniformConstantFolding
          • DivideFusion
          • DivisionByZeroFP16Resolver
          • DropoutWithRandomUniformReplacer
          • EinsumDecomposition
          • EliminateConcat
          • EliminateConvert
          • EliminateConvertNonZero
          • EliminateEltwise
          • EliminateGatherUnsqueeze
          • EliminatePad
          • EliminateSplit
          • EliminateSqueeze
          • EliminateTranspose
          • EliminateUnsqueezeGather
          • EltwiseBaseTransformation
          • EnableDecompressionConvertConstantFolding
          • FakeQuantizeDecomposition
          • FakeQuantizeDecompositionTransformation
          • FakeQuantizeMulFusion
          • FakeQuantizeReshapeFusion
          • FakeQuantizeTransformation
          • FixRtInfo
          • FoldConvertTransformation
          • FoldFakeQuantizeTransformation
          • FuseConvertTransformation
          • FuseMultiplyToFakeQuantizeTransformation
          • FuseSubtractToFakeQuantizeTransformation
          • GRUCellDecomposition
          • GatherNegativeConstIndicesNormalize
          • GatherNopElimination
          • Gelu7Downgrade
          • GeluFusion
          • GeluFusionWithErfOne
          • GeluFusionWithErfThree
          • GeluFusionWithErfTwo
          • GroupConvolutionTransformation
          • GroupedGatherElimination
          • GroupedStridedSliceOptimizer
          • HSigmoidDecomposition
          • HSigmoidFusion
          • HSigmoidFusionWithClampDiv
          • HSigmoidFusionWithClampMul
          • HSigmoidFusionWithReluDiv
          • HSigmoidFusionWithReluMul
          • HSigmoidFusionWithoutRelu
          • HSwishDecomposition
          • HSwishFusion
          • HSwishFusionWithClamp
          • HSwishFusionWithHSigmoid
          • HSwishFusionWithReluDiv
          • HSwishFusionWithReluMul
          • InitConstMask
          • InitMasks
          • InitNodeInfo
          • InterpolateSequenceFusion
          • InterpolateTransformation
          • IntervalsAlignmentAttribute
          • IntervalsAlignmentSharedValue
          • LSTMCellDecomposition
          • LayerTransformation
          • LeakyReluFusion
          • LinOpSequenceFusion
          • LogSoftmaxDecomposition
          • MVN6Decomposition
          • MVNFusion
          • MVNFusionWithConstantsInside
          • MVNFusionWithoutConstants
          • MVNTransformation
          • MarkPrecisionSensitiveDivides
          • MarkPrecisionSensitiveSubgraphs
          • MarkupAvgPoolPrecisionPreserved
          • MarkupCanBeQuantized
          • MarkupPerTensorQuantization
          • MarkupPrecisions
          • MatMulMultiplyFusion
          • MatMulTransformation
          • MaxPoolTransformation
          • MimicSetBatchSize
          • MishFusion
          • MulFakeQuantizeFusion
          • MultiplyConvolutionFusion
          • MultiplyToGroupConvolutionTransformation
          • MultiplyTransformation
          • NearestNeighborUpsamplingFusion
          • NormalizeL2Decomposition
          • NormalizeL2Fusion
          • NormalizeL2Transformation
          • PReluTransformation
          • PadFusionAvgPool
          • PadFusionConvolution
          • PadFusionConvolutionBackpropData
          • PadFusionGroupConvolution
          • PadFusionGroupConvolutionBackpropData
          • PadTransformation
          • PerTensorQuantizationAttribute
          • PrecisionPreservedAttribute
          • PrecisionsAttribute
          • PropagateMasks
          • PropagatePrecisions
          • PropagateSharedValue
          • PropagateThroughPrecisionPreserved
          • PropagateToInput
          • Proposal1Scales
          • Pruning
          • PullReshapeThroughDequantization
          • PullSqueezeThroughEltwise
          • PullTransposeThroughDequantization
          • QuantizationAlignmentAttribute
          • RNNCellDecomposition
          • RandomUniformFusion
          • ReduceBaseTransformation
          • ReduceL1Decomposition
          • ReduceL2Decomposition
          • ReduceMaxTransformation
          • ReduceMeanTransformation
          • ReduceMinTransformation
          • ReduceSumTransformation
          • ReluFakeQuantizeFusion
          • ReluTransformation
          • RemoveConcatZeroDimInput
          • ReplaceConcatReduceByMinOrMax
          • ReshapeAMatMul
          • ReshapeSequenceFusion
          • ReshapeTo1D
          • ReshapeTransformation
          • ResolveNameCollisions
          • ReverseInputChannelsFusion
          • SetBatchSize
          • SharedShapeOf
          • SharedSqueeze
          • SharedStridedSliceEraser
          • ShrinkWeights
          • ShuffleChannelsFusion
          • ShuffleChannelsTransformation
          • SkipGatherBeforeTransposeAndReshape
          • SliceToStridedSlice
          • SoftPlusDecomposition
          • SoftPlusFusion
          • SoftPlusToMishFusion
          • SoftmaxDecomposition
          • SoftmaxFusion
          • SpaceToBatchFusion
          • SplitConcatPairToInterpolateFusion
          • SplitSqueezeConcatFusion
          • SplitTransformation
          • SqueezeStridedSlice
          • SqueezeTransformation
          • StridedSliceOptimization
          • StridedSliceSqueeze
          • StridedSliceTransformation
          • StridesOptimization
          • SubtractFusion
          • SubtractTransformation
          • SupportedNodesStridesPropagation
          • SwishFusion
          • SwishFusionWithBeta
          • SwishFusionWithSigmoid
          • SwishFusionWithSigmoidWithBeta
          • SwishFusionWithoutBeta
          • TransformationContext
          • TransparentBaseTransformation
          • TransposeConvert
          • TransposeEltwise
          • TransposeFQReduction
          • TransposeFuse
          • TransposeReduction
          • TransposeReshapeEliminationForMatmul
          • TransposeSinking
          • TransposeToReshape
          • TransposeTransformation
          • UnrollIf
          • UnrollTensorIterator
          • UnsqueezeTransformation
          • UnsupportedNodesStridesPropagation
          • UpdateSharedPrecisionPreserved
          • UselessStridedSliceEraser
          • VariadicSplitTransformation
          • WeightableLayerTransformation
          • WeightsDequantizeToFakeQuantize
          • WrapInterpolateIntoTransposes
        • Conversion from opset2 to opset1
        • Conversion from opset3 to opset2
        • Runtime information
          • Decompression
          • DisableFP16Compression
          • FusedNames
          • Mask
          • NonconvertibleDivide
          • OldApiMapElementType
          • OldApiMapOrder

Use OpenVINO™ Toolkit Securely

  • Introduction to OpenVINO™ Security
  • Deep Learning Workbench Security
  • Using Encrypted Models with OpenVINO™
  • OpenVINO™ Security Add-on
On this page
.pdf .zip
Edit this page

Tuning Utilities¶

Tuning Utilities

  • Deep Learning accuracy validation framework
  • Dataset Preparation Guide
  • Cross Check Tool
Prev Next

© Copyright 2021, Intel®.

Created using Sphinx 3.2.1.