In this sample we will go through typical steps required to evaluate DL topologies.
We will try to evaluate SampLeNet topology as an example.
1. Download and extract dataset¶
In this sample we will use toy dataset which we refer to as sample dataset, which contains 10K images of 10 different classes (classification problem), which is actually CIFAR10 dataset converted to PNG (image conversion will be done automatically in evaluation process)
You can download original CIFAR10 dataset from official website.
Extract downloaded dataset to sample directory
tar xvf cifar-10-python.tar.gz -C sample
2. Evaluate sample topology¶
Typically you need to write a configuration file describing evaluation process of your topology. There is already a prepared config file for evaluating SampLeNet using the OpenVINO framework at
<omz_dir>/tools/accuracy_checker/sample/sample_config.yml, so please read it carefully before using.
Then run Accuracy Checker with the following command:
accuracy_check -c sample/sample_config.yml -m data/test_models -s sample
-c path to evaluation config,
-m directory where models are stored,
-s directory where source data (datasets).
If everything worked correctly, you should be able to get
Now try edit config, to run SampLeNet on other device or framework (e.g., Caffe, MXNet or OpenCV), or go directly to your topology!
Additional useful resources¶
large collection of configuration files for models from Open Model Zoo. You can find instruction how to use predefined configuration here
<omz_dir>/tools/accuracy_checker/sample/opencv_sample_config.ymlfor running SampleNet via OpenCV launcher.
<omz_dir>/tools/accuracy_checker/sample/sample_blob_config.ymlfor running SampleNet using compiled executable network blob. > NOTE : Not all OpenVINO devices support compiled blob execution.