OpenVINO Model Server Demos¶
OpenVINO Model Server demos show how to use the model server and its features. Source code for the demos below is available on GitHub. See the demos along with steps to reproduce:
Age and Gender Recognition via REST API - run predictions on JPEG images using an age and gender recognition model via the REST API.
Horizontal Text Detection in Real-Time - run predictions on a camera stream using a horizontal text detection model via the gRPC API.
Age, Gender and Emotion Recognition with Pipelined Models - run predictions on JPEG images using a pipeline of age-gender recognition and emotion recognition models via the gRPC API.
Optical Character Recognition Pipeline - run predictions on JPEG images using a pipeline of text recognition and text detection models, with a custom node for processing intermediate results, via the gRPC API.
Simple Face Detection - run predictions on JPEG images using a face detection model via the gRPC API.
Age, Gender and Emotion Recognition with Pipelined Models (full) - run predictions on JPEG images using a pipeline of age-gender recognition and emotion recognition models via the gRPC API.
Image Classification with ONNX Models - run predictions on JPEG images using a ResNet-50 classification model from ONNX Model Zoo via the gRPC API.
Preparing TensorFlow Model For Serving - download and convert a ResNet-50 TensorFlow model to the OpenVINO IR format accepted by OpenVINO Model Server.
Vehicle Analysis Pipeline - detect vehicles and recognize their attributes using a pipeline of vehicle detection and vehicle attribute recognition models, with a custom node for processing intermediate results, via the gRPC API.
Natural Language Processing with BERT - provide a knowledge source and a query a BERT model for question answering use case via gRPC API.
Run Predictions with a C++ application - build a C++ client application in a Docker container and send predictions via the gRPC API.
Run Predictions with a Go application - build a Go client application in a Docker container and send predictions via the gRPC API.
ovmsclient Python samples - a set of samples that use the
ovmsclient
Python package for predictions, getting model status and model metadata via both the gRPC and REST APIs.
Additional demos that show how to handle dynamic inputs:
dynamic batch size with a demuliplexer - create a simple pipeline that splits data of any batch size and performs inference on each element in the batch separately.
dynamic batch size with automatic model reloading - configure the model server to reload the model every time it receives a request with batch size other than what is currently set.
dynamic shape with automatic model reloading - configure the model server to reload a model every time the model receives a request with data in a shape other than what is currently set.
dynamic input shape with a custom node - create a simple pipeline by pairing a model with a custom node that performs data preprocessing and provides the model with data in an acceptable shape.
dynamic input shape with binary input format - send data in binary format (i.e. JPEG or PNG encoded), so the model server adjusts the input on data decoding.