This example demonstrates an approach to create interactive applications for video processing. It shows the basic architecture for building model pipelines supporting model placement on different devices and simultaneous parallel or sequential execution using OpenVINO library in Python. In particular, this demo uses 3 models to build a pipeline able to detect faces on videos, their keypoints (aka "landmarks"), and recognize persons using the provided faces database (the gallery). The following pretrained models can be used:
face-detection-retail-0004
and face-detection-adas-0001
, to detect faces and predict their bounding boxes;landmarks-regression-retail-0009
, to predict face keypoints;face-reidentification-retail-0095
, to recognize persons.For more information about the pre-trained models, refer to the model documentation.
The application is invoked from command line. It reads the specified input video stream frame-by-frame, be it a camera device or a video file, and performs independent analysis of each frame. In order to make predictions the application deploys 3 models on the specified devices using OpenVINO library and runs them in asynchronous manner. An input frame is processed by the face detection model to predict face bounding boxes. Then, face keypoints are predicted by the corresponding model. The final step in frame processing is done by the face recognition model, which uses keypoints found to align the faces and the face gallery to match faces found on a video frame with the ones in the gallery. Then, the processing results are visualized and displayed on the screen or written to the output file.
To recognize faces the application uses a face database, or a gallery. The gallery is a folder with images of persons. Each image in the gallery can be of arbitrary size and should contain one or more frontally-oriented faces with decent quality. There are allowed multiple images of the same person, but the naming format in that case should be specific - {id}-{num_of_instance}.jpg
. For example, there could be images Paul-0.jpg
, Paul-1.jpg
etc. and they all will be treated as images of the same person. In case when there is one image per person, you can use format {id}.jpg
(e.g. Paul.jpg
). The application can use face detector during the gallery building, that is controlled by --run_detector
flag. This allows gallery images to contain more than one face image and not to be tightly cropped. In that mode the user will be asked if he wants to add a specific image to the images gallery (and it leads to automatic dumping images to the same folder on disk). If it is, then the user should specify the name for the image in the open window and press Enter
. If it's not, then press Escape
. The user may add multiple images of the same person by setting the same name in the open window. However, the resulting gallery needs to be checked more thoroughly, since a face detector can fail and produce poor crops.
Image file name is used as a person name during the visualization. Use the following name convention: person_N_name.png
or person_N_name.jpg
.
The demo depends on:
To install all the required Python modules you can use:
Running the application with the -h
option or without any arguments yields the following message:
Example of a valid command line to run the application:
Linux (sh
, bash
, ...) (assuming OpenVINO installed in /opt/intel/openvino
):
Windows (cmd
, powershell
) (assuming OpenVINO installed in C:/Intel/openvino
):
Notice that the custom networks should be converted to the Inference Engine format (*.xml + *bin) first. To do this use the Model Optimizer tool.
The demo uses OpenCV window to display the resulting video frame and detections. If specified, it also writes output to a file. It outputs logs to the terminal.