# Neural Network Compression Framework¶

This document describes the Neural Network Compression Framework (NNCF) which is distributed as a separate tool but is highly aligned with OpenVINO in terms of the supported optimization features and models. It is open-sourced and available on GitHub.

## Introduction¶

Neural Network Compression Framework (NNCF) is aimed at optimizing Deep Neural Network (DNN) by applying optimization methods, such as quantization, pruning, etc., to the original framework model. It provides in-training optimization capabilities which means that optimization methods require model fine-tuning or even re-training. The diagram below shows the model optimization workflow using NNCF.

### Features¶

• Support optimization of PyTorch and TensorFlow 2.x models.

• Support of various optimization algorithms, applied during a model fine-tuning process to achieve a better performance-accuracy trade-off:

Compression algorithm

PyTorch

TensorFlow 2.x

8- bit quantization

Supported

Supported

Filter pruning

Supported

Supported

Sparsity

Supported

Supported

Mixed-precision quantization

Supported

Not supported

Binarization

Supported

Not supported

• Stacking of optimization methods. For example: 8-bit quaNtization + Filter Pruning.

• Support for Accuracy-Aware model training pipelines via the Adaptive Compression Level Training and Early Exit Training.

• Automatic, configurable model graph transformation to obtain the compressed model.

Note

Limited support for TensorFlow models. Only the models created, using Sequential or Keras Functional API, are supported.

• GPU-accelerated layers for the faster compressed model fine-tuning.

• Distributed training support.

• Configuration file examples for each supported compression algorithm.

• Exporting PyTorch compressed models to ONNX* checkpoints and TensorFlow compressed models to SavedModel or Frozen Graph format, ready to use with OpenVINO toolkit.

• Git patches for prominent third-party repositories (huggingface-transformers) demonstrating the process of integrating NNCF into custom training pipelines

## Get started¶

### Installation¶

NNCF provides the packages available for installation through the PyPI repository. To install the latest version via pip manager run the following command:

pip install nncf

### Usage examples¶

NNCF provides various examples and tutorials that demonstrate usage of optimization methods.