Introduction

This version of Deep Learning Workbench is a feature preview release. As a result, this documentation and corresponding functionality within the application are subject to change and can contain errata or be inaccurate. Installing, hosting and using this application is at your own risk.

Deep Learning Workbench is a web-based graphical environment that allows users to visualize a simulation of performance of deep learning models and datasets on various Intel® architecture configurations (CPU, GPU, VPU). In addition, users can automatically fine-tune the performance of an OpenVINO™ model by reducing the precision of certain model layers (quantization) from FP32 to INT8. Additional tuning algorithms will be supported in future releases.

To get started, follow the Installation Guide. Then go to https://127.0.0.1:5665 in your web browser. Google Chrome* 72 or higher is recommended.

To learn more about precision tuning for optimized performance on Intel® architecture, refer to the Calibration tool documentation.

General Workflow

To start a new project, select Get Started from the home screen or the New button next to Projects after an initial inference is run.
Create a new project configuration through a 3-step wizard. The steps are:

1. Import and select a modelthrough one of the following options:
   • Import a new model from your local file system or
   • Import a model from the Open Model Zoo, an open source online repository of models or
   • Select a previously imported model
2. Import/Generate and select a dataset through one of the following options:
   • Import a sample dataset or
   • Generate a random noise dataset or
   • Select a previously imported dataset
3. Run the initial inference

Profile a configuration with the following steps:

1. Select a target architecture to perform a baseline inference
2. Experiment with model optimization and inference options
3. Analyze the inference results
4. Apply the optimal configuration to you application

Pages Overview

The Workbench is a web application running on a Linux* server that provides an intuitive graphic interface for performance profiling and tuning of deep learning models.

When you go to the DL Workbench URL (by default - http://127.0.0.1:5665), you see a Get Started page inviting you to start a new project.

Once you import and configure a model and dataset, you can perform inference experiments to identify the model performance and optimal parameters to achieve the maximum performance on Intel® hardware.

Configuration Tab

Use this tab to create and manage a profile configuration for initial inference of your model that will be used as a reference for further optimization. Configuration includes:

• Your pretrained model.
• A dataset to use for running inference.
• Inference settings such as target device, API type, number of runs and number of batches.

Core Use Cases Overview

Workbench supports several advanced profiling scenarios:

• Parallel asynchronous inference requests (aka streams for CPU) and batch tuning for a target device
• Profiling INT8 calibration of the model
• Winograd Algorithmic Tuning

Table of Contents

• Get Started
  • Install DL Workbench
    • Install from the Intel® Distribution of OpenVINO™ Toolkit Package
    • Install from Dockerfile
    • Install from Docker* Hub
  • Working with Models and Sample Datasets
    • Import and Select Models
    • Import, Generate and Select Datasets
    • Run Baseline Inference
• Measure and Interpret Model Performance
  • Run Single Inference
  • Run Range of Inferences
  • View Inference Results
    • Visualize Model
• Tune Models for Enhanced Performance
  • INT8 Quantization
  • Winograd Algorithmic Tuning
  • Compare Performance between Two Versions of Models
• Accuracy Measurements
  • Configure Accuracy Settings
  • Measure Accuracy
• Deploy and Integrate Performance Criteria into Application
• Advanced
  • Restart Docker Container
  • Enter Docker Container with DL Workbench
  • Mount Folder to Docker Container
• Troubleshooting