Interactive Tutorials (Python)

This collection of Python tutorials are written for running on Jupyter notebooks. The tutorials provide an introduction to the OpenVINO™ toolkit and explain how to use the Python API and tools for optimized deep learning inference. You can run the code one section at a time to see how to integrate your application with OpenVINO libraries.

Notebooks with and buttons can be run without installing anything. Once you have found the tutorial of your interest, just click the button next to its name and the Jupyter notebook will start it in a new tab of a browser.

Note

Binder and Google Colab are free online services with limited resources. For the best performance and more control, you should run the notebooks locally. Follow the Installation Guide in order to get information on how to run and manage the notebooks on your machine.

More examples along with additonal details regarding OpenVINO Notebooks are available in OpenVINO™ Notebooks Github Repository.

The Jupyter notebooks are categorized into following classes:

• First steps with OpenVINO

• Convert & Optimize

• Model Demos

• Model Training

• Live Demos

• Recommended Tutorials

First steps with OpenVINO

Brief tutorials that demonstrate how to use Python API for inference in OpenVINO.

Notebook	Description	Preview
001-hello-world	Classify an image with OpenVINO.
002-openvino-api	Learn the OpenVINO Python API.
003-hello-segmentation	Semantic segmentation with OpenVINO.
004-hello-detection	Text detection with OpenVINO.

Convert & Optimize

Tutorials that explain how to optimize and quantize models with OpenVINO tools.

Notebook	Description	Preview
101-tensorflow-classification-to-openvino	Convert TensorFlow models to OpenVINO IR.
102-pytorch-to-openvino	Convert PyTorch models to OpenVINO IR.
103-paddle-onnx-to-openvino	Convert PaddlePaddle models to OpenVINO IR.
121-convert-to-openvino	Learn OpenVINO model conversion API

Explore more notebooks here.

Notebook	Description
102-pytorch-onnx-to-openvino	Convert PyTorch models to OpenVINO IR.
104-model-tools	Download, convert and benchmark models from Open Model Zoo.
105-language-quantize-bert	Optimize and quantize a pre-trained BERT model.
106-auto-device	Demonstrates how to use AUTO Device.
107-speech-recognition-quantization	Optimize and quantize a pre-trained Data2Vec speech model.
107-speech-recognition-quantization	Optimize and quantize a pre-trained Wav2Vec2 speech model.
108-gpu-device	Working with GPUs in OpenVINO™
109-latency-tricks	Performance tricks for latency mode in OpenVINO™.
109-throughput-tricks	Performance tricks for throughput mode in OpenVINO™.
110-ct-segmentation-quantize	Live inference of a kidney segmentation model and benchmark CT-scan data with OpenVINO.
110-ct-segmentation-quantize	Quantize a kidney segmentation model and show live inference.
111-yolov5-quantization-migration	Migrate YOLOv5 POT API based quantization pipeline on Neural Network Compression Framework (NNCF).
112-pytorch-post-training-quantization-nncf	Use Neural Network Compression Framework (NNCF) to quantize PyTorch model in post-training mode (without model fine-tuning).
113-image-classification-quantization	Quantize MobileNet image classification.
115-async-api	Use asynchronous execution to improve data pipelining.
116-sparsity-optimization	Improve performance of sparse Transformer models.
117-model-server	Improve performance of sparse Transformer models.
118-optimize-preprocessing	Improve performance of image preprocessing step.
119-tflite-to-openvino	Convert TensorFlow Lite models to OpenVINO IR.
120-tensorflow-object-detection-to-openvino	Convert TensorFlow Object Detection models to OpenVINO IR
122-speech-recognition-quantization-wav2vec2	Quantize Speech Recognition Models with accuracy control using NNCF PTQ API.
122-yolov8-quantization-with-accuracy-control	Convert and Optimize YOLOv8 with OpenVINO™.

Model Demos

Demos that demonstrate inference on a particular model.

Notebook	Description	Preview
205-vision-background-removal	Remove and replace the background in an image using salient object detection.
209-handwritten-ocr	OCR for handwritten simplified Chinese and Japanese.	的人不一了是他有为在责新中任自之我们
211-speech-to-text	Run inference on speech-to-text recognition model.
215-image-inpainting	Fill missing pixels with image in-painting.
218-vehicle-detection-and-recognition	Use pre-trained models to detect and recognize vehicles and their attributes with OpenVINO.

Explore more notebooks below.

Notebook	Description	Preview
201-vision-monodepth	Monocular depth estimation with images and video.
202-vision-superresolution-image	Upscale raw images with a super resolution model.	→
202-vision-superresolution-video	Turn 360p into 1080p video using a super resolution model.	→
203-meter-reader	PaddlePaddle pre-trained models to read industrial meter’s value.
204-segmenter-semantic-segmentation	Semantic segmentation with OpenVINO™ using Segmenter.
206-vision-paddlegan-anime	Turn an image into anime using a GAN.	→
207-vision-paddlegan-superresolution	Upscale small images with superresolution using a PaddleGAN model.	→
208-optical-character-recognition	Annotate text on images using text recognition resnet.
212-pyannote-speaker-diarization	Run inference on speaker diarization pipeline.
210-slowfast-video-recognition	Video Recognition using SlowFast and OpenVINO™
213-question-answering	Answer your questions basing on a context.
214-grammar-correction	Grammatical error correction with OpenVINO.
216-attention-center	The attention center model with OpenVINO™
217-vision-deblur	Deblur images with DeblurGAN-v2.
219-knowledge-graphs-conve	Optimize the knowledge graph embeddings model (ConvE) with OpenVINO.
220-cross-lingual-books-alignment	Cross-lingual Books Alignment With Transformers and OpenVINO™
221-machine-translation	Real-time translation from English to German.
222-vision-image-colorization	Use pre-trained models to colorize black & white images using OpenVINO.
223-text-prediction	Use pre-trained models to perform text prediction on an input sequence.
224-3D-segmentation-point-clouds	Process point cloud data and run 3D Part Segmentation with OpenVINO.
225-stable-diffusion-text-to-image	Text-to-image generation with Stable Diffusion method.
226-yolov7-optimization	Optimize YOLOv7, using NNCF PTQ API.
227-whisper-subtitles-generation	Generate subtitles for video with OpenAI Whisper and OpenVINO.
228-clip-zero-shot-convert	Zero-shot Image Classification with OpenAI CLIP and OpenVINO™
228-clip-zero-shot-quantize	Post-Training Quantization of OpenAI CLIP model with NNCF
229-distilbert-sequence-classification	Sequence classification with OpenVINO.
230-yolov8-optimization	Optimize YOLOv8, using NNCF PTQ API.
231-instruct-pix2pix-image-editing	Image editing with InstructPix2Pix.
232-clip-language-saliency-map	Language-visual saliency with CLIP and OpenVINO™.
233-blip-visual-language-processing	Visual question answering and image captioning using BLIP and OpenVINO™.
234-encodec-audio-compression	Audio compression with EnCodec and OpenVINO™.
235-controlnet-stable-diffusion	A text-to-image generation with ControlNet Conditioning and OpenVINO™.
236-stable-diffusion-v2	Text-to-image generation and Infinite Zoom with Stable Diffusion v2 and OpenVINO™.
236-stable-diffusion-v2	Stable Diffusion v2.1 using Optimum-Intel OpenVINO and multiple Intel Hardware.
236-stable-diffusion-v2	Stable Diffusion v2.1 using Optimum-Intel OpenVINO.
236-stable-diffusion-v2	Stable Diffusion Text-to-Image Demo.
236-stable-diffusion-v2	Text-to-image generation with Stable Diffusion v2 and OpenVINO™.
237-segment-anything	Prompt based object segmentation mask generation, using Segment Anything and OpenVINO™.
238-deep-floyd-if	Text-to-image generation with DeepFloyd IF and OpenVINO™.
239-image-bind	Binding multimodal data, using ImageBind and OpenVINO™.
240-dolly-2-instruction-following	Instruction following using Databricks Dolly 2.0 and OpenVINO™.
241-riffusion-text-to-music	Text-to-Music generation using Riffusion and OpenVINO™.
242-freevc-voice-conversion	High-Quality Text-Free One-Shot Voice Conversion with FreeVC and OpenVINO™
243-tflite-selfie-segmentation	Selfie Segmentation using TFLite and OpenVINO™.
244-named-entity-recognition	Named entity recognition with OpenVINO™.
245-typo-detector	English Typo Detection in sentences with OpenVINO™.
246-depth-estimation-videpth	Monocular Visual-Inertial Depth Estimation with OpenVINO™.
247-code-language-id	Identify the programming language used in an arbitrary code snippet.
248-stable-diffusion-xl	Image generation with Stable Diffusion XL and OpenVINO™.
249-oneformer-segmentation	Universal segmentation with OneFormer and OpenVINO™.
250-music-generation	Controllable Music Generation with MusicGen and OpenVINO™.
251-tiny-sd-image-generation	Image Generation with Tiny-SD and OpenVINO™.
252-fastcomposer-image-generation	Image generation with FastComposer and OpenVINO™.
253-zeroscope-text2video	Text-to video synthesis with ZeroScope and OpenVINO™.	A panda eating bamboo on a rock.

Model Training

Tutorials that include code to train neural networks.

Notebook	Description	Preview
301-tensorflow-training-openvino	Train a flower classification model from TensorFlow, then convert to OpenVINO IR.
302-pytorch-quantization-aware-training	Use Neural Network Compression Framework (NNCF) to quantize PyTorch model.
305-tensorflow-quantization-aware-training	Use Neural Network Compression Framework (NNCF) to quantize TensorFlow model.

Live Demos

Live inference demos that run on a webcam or video files.

Notebook	Description	Preview
401-object-detection-webcam	Object detection with a webcam or video file.
402-pose-estimation-webcam	Human pose estimation with a webcam or video file.
403-action-recognition-webcam	Human action recognition with a webcam or video file.
404-style-transfer-webcam	Style transfer with a webcam or video file.
405-paddle-ocr-webcam	OCR with a webcam or video file.
406-3D-pose-estimation-webcam	3D display of human pose estimation with a webcam or video file.
407-person-tracking-webcam	Person tracking with a webcam or video file.

Recommended Tutorials

The following tutorials are guaranteed to provide a great experience with inference in OpenVINO:

Notebook		Preview
YOLOv8 - Optimization	Optimize YOLOv8, using NNCF PTQ API.
SAM - Segment Anything Model	Prompt based object segmentation mask generation, using Segment Anything and OpenVINO™.
ControlNet - Stable-Diffusion	A text-to-image generation with ControlNet Conditioning and OpenVINO™.
Stable Diffusion v2	Text-to-image generation and Infinite Zoom with Stable Diffusion v2 and OpenVINO™.
Whisper - Subtitles generation	Generate subtitles for video with OpenAI Whisper and OpenVINO.
CLIP - zero-shot-image-classification	Perform Zero-shot image classification with CLIP and OpenVINO.
BLIP - Visual-language-processing	Visual question answering and image captioning using BLIP and OpenVINO™.
Instruct pix2pix - Image-editing	Image editing with InstructPix2Pix.
DeepFloyd IF - Text-to-Image generation	Text-to-image generation with DeepFloyd IF and OpenVINO™.
ImageBind	Binding multimodal data, using ImageBind and OpenVINO™.
Dolly v2	Instruction following using Databricks Dolly 2.0 and OpenVINO™.
Stable Diffusion XL	Image generation with Stable Diffusion XL and OpenVINO™.
MusicGen	Controllable Music Generation with MusicGen and OpenVINO™.
Tiny SD	Image Generation with Tiny-SD and OpenVINO™.

Note

If there are any issues while running the notebooks, refer to the Troubleshooting and FAQ sections in the Installation Guide or start a GitHub discussion.

Additional Resources

• OpenVINO™ Notebooks - Github Repository

• Binder documentation

• Google Colab