Introduction to Deep Learning Workbench

This version of the Deep Learning Workbench (DL Workbench) is a feature preview release. This documentation and corresponding functionality within the application are subject to change and can contain errata or be inaccurate. Installing, hosting and using this application is at your own risk.

DL Workbench is a web-based graphical environment that enables you to visualize, fine-tune, and compare performance of deep learning models on various Intel® architecture configurations, such as CPU, Intel® Processor Graphics (GPU), Intel® Movidius™ Neural Compute Stick 2 (NCS 2), and Intel® Vision Accelerator Design with Intel® Movidius™ VPUs.

The intuitive web-based interface of the DL Workbench enables you to easily use various OpenVINO™ toolkit components:

• Model Downloader to download models from the Intel® Open Model Zoo with pretrained models for a range of different tasks
• Model Optimizer to transform models into the Intermediate Representation (IR) format
• Post-Training Optimization toolkit to calibrate a model and then execute it in the INT8 precision
• Accuracy Checker to determine the accuracy of a model
• Benchmark Tool to estimate deep learning inference performance on supported devices

DL Workbench is available to be installed and run locally and also in the Intel® DevCloud for the Edge:

• Running the DL Workbench on your local system enables you to profile your neural network on your own hardware configuration, as well as connect to targets in your local network and profile on them remotely. You have access to an extended feature list including accuracy measurements and Winograd algorithmic tuning. You also do not compete for resources with other Intel® DevCloud for the Edge users. As a result, your experiments are conducted faster.
  To get started, follow the Installation Guide. DL Workbench uses authentication tokens to access the application. A token is generated automatically and displayed in the console output when you run the container for the first time.
• Running the DL Workbench in the Intel® DevCloud for the Edge enables you to profile your neural network on various Intel® hardware configurations hosted in the cloud environment without any hardware setup at your end and integrate the optimized model in the friendly environment of JupyterLab*. You can also choose this option if you just want to get familiar with the DL Workbench and explore its features.
  To get started, follow the instructions in Run DL Workbench in the Intel® DevCloud for the Edge.

General Workflow

To start a new configuration, select Get Started on the home page:

Create a new configuration on the Configurations page. A configuration includes:

• Pretrained model
• Validation dataset to run inference on
• Target device

Once you import and configure a model and dataset, you can experiment with the model to identify the model performance and optimal parameters to achieve the maximum performance on Intel® hardware:

1. Calibrate the model in INT8 precision
   
2. Find the best combination of inference parameters: number of streams and batches
3. Analyze inference results and compare them across different configurations
4. Implement an optimal configuration into your application
   

Core Use Cases

DL Workbench supports several advanced profiling scenarios:

• Parallel asynchronous inference requests (streams for CPUs) and batch tuning for a target device
• Profiling INT8 calibration of the model
• Winograd algorithmic tuning

Table of Contents

• DL Workbench Installation Guide
  • Install from Docker* Hub
    • Install on Linux*
    • Install on Windows*
    • Install on macOS*
  • Install from the Intel® Distribution of OpenVINO™ Toolkit Package
  • Run DL Workbench in the Intel® DevCloud for the Edge
  • Enter DL Workbench
• DL Workbench Get Started Guide
  • Select Models
    • Import Models
    • Import Frozen TensorFlow* SSD MobileNet v2 COCO Tutorial
    • Import MXNet* MobileNet v2 Tutorial
    • Import ONNX* MobileNet v2 Tutorial
  • Select Datasets
    • Import Datasets
    • Generate Datasets
    • Dataset Types
    • Download and Cut Datasets
  • Select Environment
  • Run Baseline Inference
  • Educational Resources about DL Workbench
• DL Workbench Developer Guide
  • Measure and Interpret Model Performance
    • Run Single Inference
    • Run Group Inference
    • View Inference Results
      • Visualize Model
      • Visualize Model Output
  • Tune Models for Enhanced Performance
    • INT8 Calibration
    • Winograd Algorithmic Tuning
    • Compare Performance between Two Versions of Models
  • Accuracy Measurements
    • Configure Accuracy Settings
    • Measure Accuracy
  • Work with Remote Targets
    • Profile on Remote Machine
    • Set Up Remote Target
    • Register Remote Target in DL Workbench
    • Manipulate Remote Machines
  • Build Your Application with Deployment Package
  • Deploy and Integrate Performance Criteria into Application
  • Work with Docker Container
  • Persist Database State
• DL Workbench Security Guide
  • Configure Transport Layer Security (TLS)
• Troubleshooting
  • Troubleshooting for DL Workbench in the Intel® DevCloud for the Edge