In this sample we will go through typical steps required to evaluate DL topologies.
We will try to evaluate SampLeNet topology as an example.
1. Download and extract dataset¶
In this sample we will use toy dataset which we refer to as sample dataset, which contains 10K images of 10 different classes (classification problem), which is actually CIFAR10 dataset converted to PNG (image conversion will be done automatically in evaluation process)
You can download original CIFAR10 dataset from official website.
Extract downloaded dataset to sample directory
tar xvf cifar-10-python.tar.gz -C sample
2. Evaluate sample topology¶
Typically you need to write a configuration file describing evaluation process of your topology. There is already a config file for evaluating SampLeNet using OpenVINO framework, read it carefully. It runs Caffe model using Model Optimizer which requires installed Caffe. If you have not opportunity to use Caffe, please replace
model: SampleNet.xml weights: SampleNet.bin
accuracy_check -c sample/sample_config.yml -m data/test_models -s sample
-c path to evaluation config,
-m directory where models are stored,
-s directory where source data (datasets).
If everything worked correctly, you should be able to get
Now try edit config, to run SampLeNet on other device or framework (e.g. Caffe, MXNet or OpenCV), or go directly to your topology!