t2t-vit-14

Use Case and High-Level Description

The t2t-vit-14 model is a variant of the Tokens-To-Token Vision Transformer(T2T-ViT) pre-trained on ImageNet dataset for image classification task. T2T-ViT progressively tokenize the image to tokens and has an efficient backbone. T2T-ViT consists of two main components: 1) a layer-wise “Tokens-to-Token module” to model the local structure information of the image and reduce the length of tokens progressively; 2) an efficient “T2T-ViT backbone” to draw the global attention relation on tokens from the T2T module. The model has 14 transformer layers in T2T-ViT backbone with 384 hidden dimensions.

More details provided in the paper and repository.

Specification

Metric

Value

Type

Classification

GFlops

9.5451

MParams

21.5498

Source framework

PyTorch*

Accuracy

Metric

Value

Top 1

81.44%

Top 5

95.66%

Input

Original Model

Image, name: image, shape: 1, 3, 224, 224, format: B, C, H, W, where:

  • B - batch size

  • C - number of channels

  • H - image height

  • W - image width

Expected color order: RGB. Mean values - [123.675, 116.28, 103.53], scale values - [58.395, 57.12, 57.375].

Converted Model

Image, name: image, shape: 1, 3, 224, 224, format: B, C, H, W, where:

  • B - batch size

  • C - number of channels

  • H - image height

  • W - image width

Expected color order: BGR.

Output

Original Model

Object classifier according to ImageNet classes, name: probs, shape: 1, 1000, output data format is B, C, where:

  • B - batch size

  • C - vector of probabilities for all dataset classes in logits format

Converted Model

Object classifier according to ImageNet classes, name: probs, shape: 1, 1000, output data format is B, C, where:

  • B - batch size

  • C - vector of probabilities for all dataset classes in logits format

Download a Model and Convert it into OpenVINO™ IR Format

You can download models and if necessary convert them into OpenVINO™ IR format using the Model Downloader and other automation tools as shown in the examples below.

An example of using the Model Downloader:

omz_downloader --name <model_name>

An example of using the Model Converter:

omz_converter --name <model_name>

Demo usage

The model can be used in the following demos provided by the Open Model Zoo to show its capabilities: