Multi Camera Multi Target Python* Demo¶
This demo demonstrates how to run Multi Camera Multi Target (e.g. person or vehicle) demo using OpenVINO™.
How It Works¶
The demo expects the following models in the Intermediate Representation (IR) format:
object detection model or object instance segmentation model
object re-identification model
As input, the demo application takes:
paths to one or several video files
numerical indexes of web cameras
The demo workflow is the following:
The demo application reads tuples of frames from web cameras/videos one by one. For each frame in tuple it runs object detector and then for each detected object it extracts embeddings using re-identification model.
All embeddings are passed to tracker which assigns an ID to each object.
The demo visualizes the resulting bounding boxes and unique object IDs assigned during tracking.
NOTE: By default, Open Model Zoo demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the demo application or reconvert your model using the Model Optimizer tool with the
--reverse_input_channelsargument specified. For more information about the argument, refer to When to Reverse Input Channels section of [Embedding Preprocessing Computation](@ref openvino_docs_MO_DG_Additional_Optimization_Use_Cases).
Preparing to Run¶
For demo input image or video files, refer to the section Media Files Available for Demos in the Open Model Zoo Demos Overview.
The list of models supported by the demo is in <omz_dir>/demos/multi_camera_multi_target_tracking_demo/python/models.lst file.
This file can be used as a parameter for Model Downloader and Converter to download and, if necessary, convert models to OpenVINO IR format (*.xml + *.bin).
An example of using the Model Downloader:
omz_downloader --list models.lst
An example of using the Model Converter:
omz_converter --list models.lst
Supported Models¶
instance-segmentation-security-0002
instance-segmentation-security-0091
instance-segmentation-security-0228
instance-segmentation-security-1039
instance-segmentation-security-1040
person-detection-retail-0013
person-reidentification-retail-0277
person-reidentification-retail-0286
person-reidentification-retail-0287
person-reidentification-retail-0288
vehicle-reid-0001
NOTE: Refer to the tables Intel’s Pre-Trained Models Device Support and Public Pre-Trained Models Device Support for the details on models inference support at different devices.
Running¶
Command Line Arguments¶
Run the application with the -h option to see the following usage message:
usage: multi_camera_multi_target_tracking_demo.py [-h] -i INPUT [INPUT ...]
[--loop] [--config CONFIG]
[--detections DETECTIONS]
[-m M_DETECTOR]
[--t_detector T_DETECTOR]
[--m_segmentation M_SEGMENTATION]
[--t_segmentation T_SEGMENTATION]
--m_reid M_REID
[--output_video OUTPUT_VIDEO]
[--history_file HISTORY_FILE]
[--save_detections SAVE_DETECTIONS]
[--no_show] [-d DEVICE]
[-u UTILIZATION_MONITORS]
Multi camera multi object tracking live demo script
optional arguments:
-h, --help show this help message and exit
-i INPUT [INPUT ...], --input INPUT [INPUT ...]
Required. Input sources (indexes of cameras or paths
to video files)
--loop Optional. Enable reading the input in a loop
--config CONFIG Configuration file
--detections DETECTIONS
JSON file with bounding boxes
-m M_DETECTOR, --m_detector M_DETECTOR
Path to the object detection model
--t_detector T_DETECTOR
Threshold for the object detection model
--m_segmentation M_SEGMENTATION
Path to the object instance segmentation model
--t_segmentation T_SEGMENTATION
Threshold for object instance segmentation model
--m_reid M_REID Required. Path to the object re-identification model
--output_video OUTPUT_VIDEO
Optional. Path to output video
--history_file HISTORY_FILE
Optional. Path to file in JSON format to save results
of the demo
--save_detections SAVE_DETECTIONS
Optional. Path to file in JSON format to save bounding
boxes
--no_show Optional. Don't show output
-d DEVICE, --device DEVICE
-u UTILIZATION_MONITORS, --utilization_monitors UTILIZATION_MONITORS
Optional. List of monitors to show initially.
Minimum command examples to run the demo for person tracking (for vehicle tracking the commands are the same with appropriate vehicle detection/re-identification models):
# videos
python multi_camera_multi_target_tracking_demo.py \
-i <path_to_video>/video_1.avi <path_to_video>/video_2.avi \
--m_detector <path_to_model>/person-detection-retail-0013.xml \
--m_reid <path_to_model>/person-reidentification-retail-0277.xml \
--config configs/person.py
# videos with instance segmentation model
python multi_camera_multi_target_tracking_demo.py \
-i <path_to_video>/video_1.avi <path_to_video>/video_2.avi \
--m_segmentation <path_to_model>/instance-segmentation-security-0228.xml \
--m_reid <path_to_model>/person-reidentification-retail-0277.xml \
--config configs/person.py
# webcam
python multi_camera_multi_target_tracking_demo.py \
-i 0 1 \
--m_detector <path_to_model>/person-detection-retail-0013.xml \
--m_reid <path_to_model>/person-reidentification-retail-0277.xml \
--config configs/person.py
The demo can use a JSON file with detections instead of an object detector. The structure of this file should be as follows:
[
[ # Source#0
{
"frame_id": 0,
"boxes": [[x0, y0, x1, y1], [x0, y0, x1, y1], ...], # N bounding boxes
"scores": [score0, score1, ...], # N scores
},
{
"frame_id": 1,
"boxes": [[x0, y0, x1, y1], [x0, y0, x1, y1], ...],
"scores": [score0, score1, ...],
},
...
],
[ # Source#1
{
"frame_id": 0,
"boxes": [[x0, y0, x1, y1], [x0, y0, x1, y1], ...], # N bounding boxes
"scores": [score0, score1, ...], # N scores
},
{
"frame_id": 1,
"boxes": [[x0, y0, x1, y1], [x0, y0, x1, y1], ...],
"scores": [score0, score1, ...],
},
...
],
...
]
Such file with detections can be saved from the demo. Specify the argument --save_detections with path to an output file.
Demo Output¶
The demo displays bounding boxes of tracked objects and unique IDs of those objects. The demo reports
FPS: average rate of video frame processing (frames per second).
Latency: average time required to process one frame (from reading the frame to displaying the results). You can use both of these metrics to measure application-level performance.
To save output video with the result please use the option --output_video,
to change configuration parameters please open the configs/person.py (or configs/vehicle.py for vehicle tracking demo) file and edit it.
Visualization can be controlled using the following keys:
space- pause or next frameenter- resume videoesc- exit
Also demo can dump resulting tracks to a json file. To specify the file use the
--history_file argument.
Quality Measuring¶
The demo provides tools for measure quality of the multi camera multi target tracker:
Evaluation MOT metrics
Visualize the demo results from a history file
For MOT metrics evaluation we use py-motmetrics module. It is necessary to have ground truth annotation file for the evaluation. Supported format of the ground truth annotation can be obtained via the annotation tool CVAT. The annotation must includes the following labels and attributes:
[
{
"name": "person",
"id": 0,
"attributes": [
{
"id": 0,
"name": "id",
"type": "text",
"mutable": false,
"values": [
" "
]
}
]
}
]
To run evaluation MOT metrics use the following command:
python run_evaluate.py \
--history_file <path_to_file>/file.json \
--gt_files \
<path_to_file>/source_0.xml \
<path_to_file>/source_1.xml \
Number of ground truth files depends on the number of used video sources.
For the visualization of the demo results please use the next command:
python run_history_visualize.py \
-i <path_to_video>/video_1.avi <path_to_video>/video_2.avi \
--history_file <path_to_file>/file.json \
This a minimum arguments set for the script. To show all available arguments run the command with -h option:
usage: run_history_visualize.py [-h] [-i I [I ...]] --history_file
HISTORY_FILE [--output_video OUTPUT_VIDEO]
[--gt_files GT_FILES [GT_FILES ...]]
[--timeline TIMELINE] [--match_gt_ids]
[--merge_outputs]
Multi camera multi target tracking visualization demo script
optional arguments:
-h, --help show this help message and exit
-i I [I ...] Input videos
--history_file HISTORY_FILE
File with tracker history
--output_video OUTPUT_VIDEO
Output video file
--gt_files GT_FILES [GT_FILES ...]
Files with ground truth annotation
--timeline TIMELINE Plot and save timeline
--match_gt_ids Match GT ids to ids from history
--merge_outputs Merge GT and history tracks into one frame
Ground truth files have the same format that was described in the MOT metrics evaluation part.
Process Analysis¶
Two options are available during the demo execution:
Visualize distances between embeddings that are criterion for matching tracks.
Save and visualize embeddings (via
tensorboard).
By default these options are disabled.
To enable the first one please set in configuration file for analyzer parameter
enable to True.
For the second one, install TensorBoard (for example, with pip install tensorboard).
Then, for embeddings specify parameter save_path
that is a directory where data related to embeddings will be saved
(if it is an empty string the option is disabled). There is paramater use_images in embeddings.
If it is True an image with object will be drawn for every embedding instead of point.