Annotation Converters¶
Annotation converter is a function which converts annotation file to suitable for metric evaluation format. Each annotation converter expects specific annotation file format or data structure, which depends on original dataset. If converter for your data format is not supported by Accuracy Checker, you can provide your own annotation converter. Each annotation converter has parameters available for configuration.
Process of conversion can be implemented in two ways:
via configuration file
via command line
Describing Annotation Conversion in Configuration File¶
Annotation conversion can be provided in dataset
section of your configuration file to convert annotation in-place before every evaluation. Each conversion configuration should contain converter
field filled with a selected converter name and provide converter specific parameters (more details in supported converters section). All paths can be prefixed via command line with -s, --source
argument.
You can additionally use optional parameters like:
subsample_size
- Dataset subsample size. You can specify the number of ground truth objects or dataset ratio in percentage. Please, be careful to use this option, some datasets does not support subsampling. You can also specifysubsample_seed
if you want to generate subsample with specific random seed.annotation
- path to store converted annotation pickle file. You can use this parameter if you need to reuse converted annotation to avoid subsequent conversions.dataset_meta
- path to store meta information about converted annotation if it is provided.analyze_dataset
- flag which allow to get statistics about converted dataset. Supported annotations:ClassificationAnnotation
,DetectionAnnotation
,MultiLabelRecognitionAnnotation
,RegressionAnnotation
. Default value is False.
Example of usage:
annotation_conversion:
# Converter name which will be called for conversion.
converter: sample
# Converter specific parameters, can be different depend on converter realization.
data_dir: sample/sample_dataset
# (Optional) subsample generation. Can be also used with prepared annotation file.
subsample_size: 1000
# (Optional) paths to store annotation files for following usage. In the next evaluation these files will be directly used instead running conversion.
annotation: sample_dataset.pickle
dataset_meta: sample_dataset.json
Conversing Process via Command Line¶
The command line for annotation conversion looks like:
convert_annotation <converter_name> <converter_specific parameters>
All converter specific options should have format --<parameter_name> <parameter_value>
You may refer to -h, --help
to full list of command line options. Some optional arguments are:
-o, --output_dir
- directory to save converted annotation and meta info.-a, --annotation_name
- annotation file name.-m, --meta_name
- meta info file name.
Supported Converters¶
Accuracy Checker supports following list of annotation converters and specific for them parameters:
cifar
- converts CIFAR classification dataset toClassificationAnnotation
data_batch_file
- path to pickle file which contain dataset batch (e.g. test_batch)has_background
- allows to add background label to original labels (Optional, default value is False).convert_images
- allows to convert images from pickle file to user specified directory (default value is False).converted_images_dir
- path to converted images location.num_classes
- the number of classes in the dataset - 10 or 100 (Optional, default 10)dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.
mnist
- convert MNIST dataset for handwritten digit recognition toClaassificationAnnotation
. Dataset can be downloaded here.labels_file
- binary file which contains labels.images_file
- binary file which contains images.convert_images
- allows to convert images from data file to user specified directory (default value is False).converted_images_dir
- path to converted images location if enabledconvert_images
.dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding). Optional, more details in Customizing dataset meta section.
mnist_csv
- convert MNIST dataset for handwritten digit recognition stored in csv format toClassificationAnnotation
.annotation_file
- path to dataset file in csv format.convert_images
- allows to convert images from annotation file to user specified directory (default value is False).converted_images_dir
- path to converted images location if enabledconvert_images
.dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding). Optional, more details in Customizing dataset meta section.
fashion_mnist
- convert Fashion-MNIST dataset toClassificationAnnotation
.annotation_file
- path to labels file in binary format.data_file
- path to images file in binary format.convert_images
- allows to convert images from data file to user specified directory (default value is False).converted_images_dir
- path to converted images location if enabledconvert_images
.dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding). Optional, more details in Customizing dataset meta section.
imagenet
- convert ImageNet dataset for image classification task toClassificationAnnotation
.annotation_file
- path to annotation in txt format.labels_file
- path to file with word description of labels (synset_words).has_background
- allows to add background label to original labels and convert dataset for 1001 classes instead 1000 (default value is False).dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.
voc_detection
- converts Pascal VOC annotation for detection task toDetectionAnnotation
.imageset_file
- path to file with validation image list.annotations_dir
- path to directory with annotation files.images_dir
- path to directory with images related to devkit root (default JPEGImages).has_background
- allows convert dataset with/without adding background_label. Accepted values are True or False. (default is True)dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.
voc_segmentation
- converts Pascal VOC annotation for semantic segmentation task toSegmentationAnnotation
.imageset_file
- path to file with validation image list.images_dir
- path to directory with images related to devkit root (default JPEGImages).mask_dir
- path to directory with ground truth segmentation masks related to devkit root (default SegmentationClass).dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section. Note : Since OpenVINO 2020.4 the converter behaviour changed.data_source
parameter of dataset should contains directory for images only, if you have segmentation mask in separated location, please usesegmentation_masks_source
for specifying gt masks location.
mscoco_detection
- converts MS COCO dataset for object detection task toDetectionAnnotation
.annotation_file
- path to annotation file in json format.has_background
- allows convert dataset with/without adding background_label. Accepted values are True or False. (default is False).use_full_label_map
- allows to use original label map (with 91 object categories) from paper instead public available(80 categories).sort_annotations
- allows to save annotations in a specific order: ascending order of image id or ascending order of image size.sort_key
- key by which annotations will be sorted(supported keys areimage_id
andimage_size
, default isimage_id
).dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.convert_COCO_to_VOC_labels
- allows to convert COCO labels to Pacsal VOC labels. Optional, default is False.
mscoco_segmentation
- converts MS COCO dataset for object instance segmentation task toCocoInstanceSegmentationAnnotation
.annotation_file
- path to annotation file in json format.has_background
- allows convert dataset with/without adding background_label. Accepted values are True or False. (default is False).use_full_label_map
- allows to use original label map (with 91 object categories) from paper instead public available(80 categories).sort_annotations
- allows to save annotations in a specific order: ascending order of image id or ascending order of image size.sort_key
- key by which annotations will be sorted (supported keys areimage_id
andimage_size
, default isimage_id
).dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.semantic_only
- converts MS COCO dataset annotation toSegmentationAnnotation
. (Optional, default value is False)masks_dir
- path to store segmentation masks insemantic_only
modeconvert_COCO_to_VOC_labels
- allows to convert COCO labels to Pacsal VOC labels. Optional, default is False.
mscoco_mask_rcnn
- converts MS COCO dataset toContainerAnnotation
withDetectionAnnotation
andCocoInstanceSegmentationAnnotation
nameddetection_annotation
andsegmentation_annotation
respectively.annotation_file
- path to annotation file in json format.has_background
- allows convert dataset with/without adding background_label. Accepted values are True or False. (default is False).use_full_label_map
- allows to use original label map (with 91 object categories) from paper instead public available(80 categories).sort_annotations
- allows to save annotations in a specific order: ascending order of image id or ascending order of image size.sort_key
- key by which annotations will be sorted (supported keys areimage_id
andimage_size
, default isimage_id
).dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.convert_COCO_to_VOC_labels
- allows to convert COCO labels to Pacsal VOC labels. Optional, default is False.
mscoco_keypoints
- converts MS COCO dataset for keypoints localization task toPoseEstimationAnnotation
.annotation_file
- path to annotation file in json format.sort_annotations
- allows to save annotations in a specific order: ascending order of image id or ascending order of image size.sort_key
- key by which annotations will be sorted (supported keys areimage_id
andimage_size
, default isimage_id
).remove_empty_images
- allows excluding/inclusing images without objects from/to the dataset..dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.
wider
- converts from Wider Face dataset toDetectionAnnotation
.annotation_file
- path to txt file, which contains ground truth data in WiderFace dataset format.label_start
- specifies face label index in label map. Default value is 1. You can provide another value, if you want to use this dataset for separate label validation, in case when your network predicts other class for faces.dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.
detection_opencv_storage
- converts detection annotation stored in Detection OpenCV storage format toDetectionAnnotation
.annotation_file
- path to annotation in xml format.image_names_file
- path to txt file, which contains image name list for dataset.label_start
- specifies label index start in label map. Default value is 1. You can provide another value, if you want to use this dataset for separate label validation.background_label
- specifies which index will be used for background label. You can not provide this parameter if your dataset has not background label.dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.
cityscapes
- converts CityScapes Dataset toSegmentationAnnotation
.dataset_root_dir
- path to dataset root.images_subfolder
- path from dataset root to directory with validation images (Optional, defaultimgsFine/leftImg8bit/val
).masks_subfolder
- path from dataset root to directory with ground truth masks (Optional,gtFine/val
).masks_suffix
- suffix for mask file names (Optional, default_gtFine_labelTrainIds
).images_suffix
- suffix for image file names (Optional, default_leftImg8bit
).use_full_label_map
- allows to use full label map with 33 classes instead train label map with 18 classes (Optional, defaultFalse
).dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.
mapillary_20
- converts Mapillary dataset contained 20 classes toSegmentationAnnotation
.data_dir
- path to dataset root folder. Relative paths to images and masks directory determine asimgs
andmasks
respectively. In way when images and masks are located in non default directories, you can use parameters described below.images_dir
- path to images folder.mask_dir
- path to ground truth mask folder.images_subfolder
- sub-directory for images(Optional, defaultimgs
)mask_subfolder
- sub-directory for ground truth mask(Optional, defaultmasks
)
mapillary_vistas
- converts Mapillary Vistas dataset contained 20 classes toSegmentationAnnotation
.data_dir
- path to dataset root folder. Relative paths to images and masks directory determine asimages
andlabels
respectively. In way when images and masks are located in non default directories, you can use parameters described below.images_dir
- path to images folder.mask_dir
- path to ground truth mask folder.images_subfolder
- sub-directory for images(Optional, defaultimages
)mask_subfolder
- sub-directory for ground truth mask(Optional, defaultlabels
)
vgg_face
- converts VGG Face 2 dataset for facial landmarks regression task toFacialLandmarksAnnotation
.landmarks_csv_file
- path to csv file with coordinates of landmarks points.bbox_csv_file
- path to cvs file which contains bounding box coordinates for faces (optional parameter).
lfw
- converts Labeled Faces in the Wild dataset for face reidentification toReidentificationClassificationAnnotation
.pairs_file
- path to file with annotation positive and negative pairs.train_file
- path to file with annotation positive and negative pairs used for network train (optional parameter).landmarks_file
- path to file with facial landmarks coordinates for annotation images (optional parameter).extension
- images extension(optional, default -jpg
).
face_recognition_bin
- converts preprocessed face recognition dataset stored in binary format toReidentificationClassificationAnnotation
.bin_file
- file with dataset. Example of datasets can be found here.images_dir
- directory for saving converted images (Optional, used only ifconvert_images
enabled, if not provided<dataset_root>/converted_images
will be used)convert_images
- allows decode and save images.
mars
- converts MARS person reidentification dataset toReidentificationAnnotation
.data_dir
- path to data directory, where gallery (bbox_test
) andquery
subdirectories are located.
market1501_reid
- converts Market1501 person reidentification dataset toReidentificationAnnotation
.data_dir
- path to data directory, where gallery (bounding_box_test
) andquery
subdirectories are located.
veri776_reid
- converts VeRi776 vehicle reidentification dataset toReidentificationAnnotation
.data_dir
- path to data directory, where gallery (image_test
) andimage_query
subdirectories are located.
image_processing
- converts dataset for common single image processing tasks (e.g. image denoising, style transferring) toImageProcessingAnnotation
. This converter is suitable for tasks where model output produced on specific input image should be compared with target image.data_dir
- path to folder, where images in low and high resolution are located.input_suffix
- input file name’s suffix (defaultin
).target_suffix
- target ground truth file name’s suffix (defaultout
).recursive
- enables acquiring of dataset files fromdata_dir
subcatalogs (default False).annotation_loader
- which library will be used for ground truth image reading. Supported:opencv
,pillow
(Optional. Default value is pillow). Note, color space of image depends on loader (OpenCV uses BGR, Pillow uses RGB for image reading).
parametric_image_processing
- converts dataset for image processing which required variable conditions for getting result, toImageProcessingAnnotation. Parameters provided as float value in reference image name using
_as delimeter. *
input_dir- directory with input images. *
reference_dir- directory with reference images. *
annotation_loader- which library will be used for ground truth image reading. Supported:
opencv,
pillow(Optional. Default value is pillow). Note, color space of image depends on loader (OpenCV uses BGR, Pillow uses RGB for image reading). *
param_scale- multiplayer for parameters (Optional, default
0.001). *
super_resolution- converts dataset for single image super resolution task to
SuperResolutionAnnotation. *
data_dir- path to folder, where images in low and high resolution are located. *
lr_dir- path to directory, where images in low resolution are located. *
hr_dir- path to directory, where images in high resolution are located. **Note:** inside converted annotation, path to directory is not stored, only file name, please use
additional_data_sourcefor providing prefix. *
upsampled_dir- path to directory, where upsampled images are located, if 2 streams used. *
ignore_suffixes- matched low resolution, high resolution image located in different directories without usage suffixes, using numeric ids (Optional, default False). *
lr_suffix- low resolution file name's suffix (default lr). *
hr_suffix- high resolution file name's suffix (default hr). *
annotation_loader- which library will be used for ground truth image reading. Supported:
opencv,
pillow,
pillow_rgb<tt>(for explicit data conversion to RGB format),dicom,
skimage. (Optional. Default value is pillow). Note, color space of image depends on loader (OpenCV uses BGR, Pillow uses RGB for image reading). *
two_streams- enable 2 input streams where usually first for original image and second for upsampled image. (Optional, default False). *
upsample_suffix- upsample images file name's suffix (default upsample). *
super_resolution_dir_based- converts dataset for single image super resolution task to
SuperResolutionAnnotationwhich have directory based structure (high resolution and low resolution images located on separated directories and matches by name or numeric id). The main difference between this converter and
super_resolutionin data organization.
super_resolutionconverter should be used if all high and low resolution images located in the same dir and have difference in suffixes. *
annotation_loader- which library will be used for ground truth image reading. Supported:
opencv,
pillow,
pillow_rgb<tt>(for explicit data conversion to RGB format),dicom,
skimage. (Optional. Default value is pillow). Note, color space of image depends on loader (OpenCV uses BGR, Pillow uses RGB for image reading). *
two_streams- enable 2 input streams where usually first for original image and second for upsampled image. (Optional, default False). *
images_dir- path to dataset root, where directories with low and high resolutions are located. *
lr_dir- path to directory, where images in low resolution are located (Optional, default
<images_dir>/LR). *
hr_dir- path to directory, where images in high resolution are located (Optional, default
<images_dir>/HR). **Note:** inside converted annotation, path to directory is not stored, only file name, please use
additional_data_sourcefor providing prefix. *
upsampled_dir- path to directory, where upsampled images are located, if 2 streams used (Optional, default
<images_dir>/upsample). *
relaxed_names- allow to use more relaxed search of high resolution or/and upsampled images matching only numeric ids. Optional, by default full name matching required. *
hr_prefixed- allow to use partial name matching when low resolution filename is a part of high resolution filename. Not applicable when
relaxed_namesis set. Optional, by default full name matching required. *
multi_frame_super_resolution- converts dataset for super resolution task with multiple input frames usage. *
data_dir- path to folder, where images in low and high resolution are located. *
lr_suffix- low resolution file name's suffix (default lr). *
hr_suffix- high resolution file name's suffix (default hr). *
annotation_loader- which library will be used for ground truth image reading. Supported:
opencv,
pillow(Optional. Default value is pillow). Note, color space of image depends on loader (OpenCV uses BGR, Pillow uses RGB for image reading). *
number_input_frames- the number of input frames per inference. *
reference_frame- the id of frame in sample frame sequence used for matching with high resolution. You can define number of frame or choose one of predefined:
first<tt>(first frame used as reference),middle(
num_frames/ 2),
last<tt>(last frame in sequence). *multi_target_super_resolution- converts dataset for single image super resolution task with multiple target resolutions to
ContainerAnnotationwith
SuperResolutionAnnotationrepresentations for each target resolution. *
data_dir- path to dataset root, where directories with low and high resolutions are located. *
lr_path- path to low resolution images directory relative to
data_dir. *
hr_mapping- dictionary which represent mapping between target resolution and directory with images. Keys are also used as keys for
ContainerAnnotation. All paths should be relative to
data_dir. *
icdar_detection- converts ICDAR13 and ICDAR15 datasets for text detection challenge to
TextDetectionAnnotation. *
data_dir- path to folder with annotations on txt format. *
word_spotting- if it is true then transcriptions that have lengths less than 3 symbols or transcriptions containing non-alphanumeric symbols will be marked as difficult. *
icdar13_recognition- converts ICDAR13 dataset for text recognition task to
CharacterRecognitionAnnotation. *
annotation_file- path to annotation file in txt format. *
delimeter- delimeter between image and text for recognition. Supported values -
spaceand
tabfor space and tabular separator respectively. *
lmdb_text_recognition_database- converter for text recognition dataset in a form of LMDB database. *
lower_case- parameter describing if ground truth text should be converted to lower case. *
unicode_character_recognition- converts [Kondate](`http://web.tuat.ac.jp/~nakagawa/database/en/kondate_about.html <http://web.tuat.ac.jp/~nakagawa/database/en/kondate_about.html>`__) dataset and [Nakayosi](`http://web.tuat.ac.jp/~nakagawa/database/en/about_nakayosi.html <http://web.tuat.ac.jp/~nakagawa/database/en/about_nakayosi.html>`__) for handwritten Japanese text recognition task , and [SCUT-EPT](`https://github.com/HCIILAB/SCUT-EPT_Dataset_Release <https://github.com/HCIILAB/SCUT-EPT_Dataset_Release>`__) for handwritten simplified Chinese text recognition task to
CharacterRecognitionAnnotation. *
annotation_file- path to annotation file in txt format. *
decoding_char_file - path to decoding_char_file, consisting of all supported characters separated by‘ in txt format.
brats
- converts BraTS dataset format toBrainTumorSegmentationAnnotation
format. Also can be used to convert other nifti-based datasets.data_dir
- dataset root directory, which contain subdirectories with validation data (imagesTr
) and ground truth labels (labelsTr
). Optionally you can provide relative path for these subdirectories (if they have different location) usingimage_folder
andmask_folder
parameters respectively.mask_channels_first
- allows read gt mask nifti files and transpose in order where channels first (Optional, defaultFalse
)labels_file
- path to file, which contains labels (optional, if omitted no labels will be shown)relaxed_names
- allows to use more relaxed search of labels matching only numeric ids. Optional, by default full name matching required.multi_frame
- allows to convert annotation of 3D images as sequence of 2D frames (optional, defaultFalse
)frame_separator
- string separator between file name and frame number inmulti_frame
(optional, default#
)frame_axis
- number of frame axis in 3D Image (optional, default-1
, last axis)as_regression
- allows dataset conversion asNiftiRegressionAnnotation
annotation (optional, defaultFalse
)
k_space_mri
- convertsk-spaced MRI
dataset format toImageRepresentationAnnotation
format. MRI datasets, for exampleCalgary-Campinas
, provides data in Fourier images form (so called k-space images). Converter performs dataset annotation and preprocessing of ground truth images and model input.data_dir
- path to dataset rootimage_folder
- path to source k-space files directory, relativelydata_dir
(optional, defaultimages
)reconstructed_folder
- path to reconstructed images directory, relativelydata_dir
(optional, defaultreconstructed
)masked_folder
- path to masked k-space files directory, relativelydata_dir
(optional, defaultmasked
)mask_file
- k-space mask filenamestats_file
- k-space normalization factors filenameskip_dumps
- allows dataset annotation without preprocessing
movie_lens_converter
- converts Movie Lens Datasets format toHitRatioAnnotation
format.rating_file
- path to file which contains movieId with top score for each userID (for example ml-1m-test-ratings.csv)negative_file
- path to file which contains negative examples.users_max_number
- the number of users which will be used for validation (Optional, it gives opportunity to cut list of users. If argument is not provided, full list of users will be used.).
brats_numpy
- converts Brain Tumor Segmentation dataset toBrainTumorSegmentationAnnotation
. This converter works with Numpy representation of BraTS dataset.data_dir
- path to dataset root directory.ids_file
- path to file, which contains names of images in datasetlabels_file
- path to file, which contains labels (optional, if omitted no labels will be shown)data_suffix
- suffix for files with data (default_data_cropped
)label_suffix
- suffix for files with groundtruth data (default_label_cropped
)boxes_file
- path to file with brain boxes (optional). Set this option with including postprocessorsegmentation-prediction-resample
(see Postprocessors).
wmt
- converts WMT dataset for Machine Translation task toMachineTranslationAnnotation
.input_file
- path to file which contains input sentences tokens for translation.reference_file
- path to file with reference for translation.
common_semantic_segmentation
- converts general format of datasets for semantic segmentation task toSegmentationAnnotation
. The converter expects following dataset structure:images and GT masks are located in separated directories (e.g.
<dataset_root>/images
for images and<dataset_root>/masks
for masks respectively)images and GT masks has common part in names and can have difference in prefix and postfix (e.g. image name is image0001.jpeg, mask for it is gt0001.png are acceptable. In this case base_part - 0001, image_prefix - image, image_postfix - .jpeg, mask_prefix - gt, mask_postfix - .png)
images_dir
- path to directory with images.masks_dir
- path to directory with GT masks.image_prefix
- prefix part for image file names. (Optional, default is empty).image_postfix
- postfix part for image file names (optional, default is.png
).mask_prefix
- prefix part for mask file names. (Optional, default is empty).image_postfix
- postfix part for mask file names (optional, default is.png
).mask_loader
- the way how GT mask should be loaded. Supported methods:pillow
,opencv
,nifti
,numpy
,scipy
.dataset_meta_file
- path to json file with prepared dataset meta info. It should containlabel_map
key with dictionary in format class_id: class_name and optionallysegmentation_colors
(if your dataset uses color encoding). Segmentation colors is a list of channel-wise values for each class. (e.g. if your dataset has 3 classes in BGR colors, segmentation colors for it will looks like:[[255, 0, 0], [0, 255, 0], [0, 0, 255]]
). (Optional, you can provide self-created file asdataset_meta
in your config). Note: since OpenVINO 2020.4 converter behaviour changed.data_source
parameter of dataset should contain the directory for images only, if you have segmentation mask in separated location, please usesegmentation_masks_source
for specifying gt masks location.
background_matting
- converts general format of datasets for background matting task toBackgroundMattingAnnotation
. The converter expects following dataset structure:images and GT masks are located in separated directories (e.g.
<dataset_root>/images
for images and<dataset_root>/masks
for masks respectively)images and GT masks has common part in names and can have difference in prefix and postfix (e.g. image name is image0001.jpeg, mask for it is gt0001.png are acceptable. In this case base_part - 0001, image_prefix - image, image_postfix - .jpeg, mask_prefix - gt, mask_postfix - .png)
images_dir
- path to directory with images.masks_dir
- path to directory with GT masks.image_prefix
- prefix part for image file names. (Optional, default is empty).image_postfix
- postfix part for image file names (optional, default is.png
).mask_prefix
- prefix part for mask file names. (Optional, default is empty).image_postfix
- postfix part for mask file names (optional, default is.png
).
camvid
- converts CamVid dataset with 12 classes toSegmentationAnnotation
. Dataset can be found in the following repositoryannotation_file
- file in txt format which contains list of validation pairs (<path_to_image>
<path_to_annotation>
separated by space)dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.
camvid_32
- converts CamVid dataset with 32 classes toSegmentationAnnotation
. Dataset can be found here.labels_dir
- directory with labeled ground truth images.images_dir
- directory with input data.val_subset_ratio
- ratio of subset, which should be used for validation. It is the float value in (0, 1] range for definition subset size as<total_dataset_size> * <subset_ratio>
. Optional, default 1 (it means full dataset used for validation).dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.
image_retrieval
- converts dataset for image retrieval task toReidentificationAnnotation
. Dataset should have following structure:the dataset root directory contains 2 subdirectory named
gallery
andqueries
for gallery images and query images respectively.Every of these subdirectories should contains text file with list of pairs:
<path_to_image>
<image_ID>
(image_path and image_ID should be separated by space), where<path_to_image>
is path to the image related dataset root,<image_ID>
is the number which represent image id in the gallery.
data_dir
- path to dataset root directory.gallery_annotation_file
- file with gallery images and IDs concordance in txt format (Optional, default value is<data_dir>/gallery/list.txt
)queries_annotation_file
- file with queries images and IDs concordance in txt format (Optional, default value is<data_dir>/queries/list.txt
)
cvat_object_detection
- converts CVAT XML annotation version 1.1 format for images toDetectionAnnotation
.annotation_file
- path to xml file in appropriate format.has_background
- allows prepend original labels with special class represented background and convert dataset for n+1 classes instead n (default value is True).dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.
cvat_attributes_recognition
- converts CVAT XML annotation version 1.1 format for images toClassificationAnnotation
orContainerAnnotation
withClassificationAnnotation
as value type and attribute names as keys (in multiple attributes case). Used bbox attributes as annotation classes.annotation_file
- path to xml file in appropriate format.label
- the dataset label which will be used for attributes collection (e.g. if your dataset contains 2 labels:face
andperson
and you want recognise attributes for face, you should useface
as value for this parameter).
cvat_age_gender
- converts CVAT XML annotation version 1.1 format for images which represent dataset for age gender recognition toContainerAnnotation
withClassificationAnnotation
for gender recognition,ClassificationAnnotation
for age classification andRegeressionAnnotation
for age regression. The identifiers for representations following:gender_annotation
,age_class_annotation
,age_regression_annotation
.annotation_file
- path to xml file in appropriate format.
cvat_facial_landmarks
- converts CVAT XML annotation version 1.1 format for images toFacialLandmarksAnnotation
.annotation_file
- path to xml file in appropriate format.
cvat_pose_estimation
- converts CVAT XML annotation version 1.1 format for images toPoseEstimationAnnotation
.annotation_file
- path to xml file in appropriate format.
cvat_text_recognition
- converts CVAT XML annotation version 1.1 format for images toCharacterRecognitionAnnotation
.annotation_file
- path to xml file in appropriate format.
cvat_binary_multilabel_attributes_recognition
- converts CVAT XML annotation version 1.1 format for images toMultiLabelRecognitionAnnotation
. Used bbox attributes as annotation classes. Each attribute field should containsT
orF
values for attribute existence/non-existence on the image respectively.annotation_file
- path to xml file in appropriate format.label
- the dataset label which will be used for attributes collection (e.g. if your dataset contains 2 labels:face
andperson
and you want recognise attributes for face, you should useface
as value for this parameter).
cvat_person_detection_action_recognition
converts dataset with CVAT XML annotation version 1.1 for person detection and action recognition task toContainerAnnotation
withDetectionAnnotation
for person detection quality estimation namedperson_annotation
andActionDetectionAnnotation
for action recognition namedaction_annotation
.annotation_file
- path to xml file with ground truth.use_case
- use case, which determines the dataset label map. Supported range actions:common_3_actions
(seating, standing, raising hand)common_6_actions
(seating, writing, raising hand, standing, turned around, lie on the desk)teacher
(standing, writing, demonstrating)raising_hand
(seating, raising hand)
lpr_txt
- converts annotation for license plate recognition task in txt format toCharacterRecognitionAnnotation
.annotation_file
- path to txt annotation.decoding_dictionary
- path to file containing dictionary for output decoding.
squad_emb
- converts the Stanford Question Answering Dataset (SQuAD) toQuestion Answering Embedding Annotation
. Note: This converter not only converts data to metric specific format but also tokenize and encodes input for BERT.testing_file
- path to testing file.vocab_file
- path to model co vocabulary file.max_seq_length
- maximum total input sequence length after word-piece tokenization (Optional, default value is 128).max_query_length
- maximum number of tokens for the question (Optional, default value is 64).lower_case
- allows switching tokens to lower case register. It is useful for working with uncased models (Optional, default value is False)
squad
- converts the Stanford Question Answering Dataset (SQuAD) toQuestion Answering Annotation
. Note: This converter not only converts data to metric specific format but also tokenize and encodes input for BERT.testing_file
- path to testing file.vocab_file
- path to model co vocabulary file.max_seq_length
- maximum total input sequence length after word-piece tokenization (Optional, default value is 128).max_query_length
- maximum number of tokens for the question (Optional, default value is 64).doc_stride
-stride size between chunks for splitting up long document (Optional, default value is 128).lower_case
- allows switching tokens to lower case register. It is useful for working with uncased models (Optional, default value is False)
squad_bidaf
- converts the Stanford Question Answering Dataset (SQuAD) toQuestionAnsweringBiDAFAnnotation
. Note: This converter not only converts data to metric specific format but also tokenize and encodes input for BiDAF using nltk.word_tokenize.testing_file
- path to testing file.
xnli
- converts The Cross-lingual Natural Language Inference Corpus (XNLI) toTextClassificationAnnotattion
. Note: This converter not only converts data to metric specific format but also tokenize and encodes input for BERT.annotation_file
- path to dataset annotation file in tsv format.vocab_file
- path to model vocabulary file for WordPiece tokinezation (Optional in case, when another tokenization approach used).sentence_piece_model_file
- model used for SentencePiece tokenization (Optional in case, when another tokenization approach used).max_seq_length
- maximum total input sequence length after word-piece tokenization (Optional, default value is 128).lower_case
- allows switching tokens to lower case register. It is useful for working with uncased models (Optional, default value is False).language_filter
- comma-separated list of used in annotation language tags for selecting records for specific languages only. (Optional, if not used full annotation will be converted).
mnli
- converts The Multi-Genre Natural Language Inference Corpus (MNLI) toTextClassificationAnnotattion
. Note: This converter not only converts data to metric specific format but also tokenize and encodes input for BERT.annotation_file
- path to dataset annotation file in tsv format.vocab_file
- path to model vocabulary file for WordPiece tokinezation. (Optional, can be not provided in case, when another tokenization approach used.)sentence_piece_model_file
- model used for SentencePiece tokenization (Optional in case, when another tokenization approach used).max_seq_length
- maximum total input sequence length after tokenization (Optional, default value is 128).lower_case
- allows switching tokens to lower case register. It is useful for working with uncased models (Optional, default value is False).
mrpc
- converts The Microsoft Research Paraphrase Corpus (MRPC) toTextClassificationAnnotattion
. Note: This converter not only converts data to metric specific format but also tokenize and encodes input for BERT.annotation_file
- path to dataset annotation file in tsv format.vocab_file
- path to model vocabulary file for WordPiece tokenization. (Optional, can be not provided in case, when another tokenization approach used.)sentence_piece_model_file
- model used for SentencePiece tokenization (Optional in case, when another tokenization approach used).max_seq_length
- maximum total input sequence length after tokenization (Optional, default value is 128).lower_case
- allows switching tokens to lower case register. It is useful for working with uncased models (Optional, default value is False).
cola
- converts The Corpus of Linguistic Acceptability (CoLA) toTextClassificationAnnotattion
. Note: This converter not only converts data to metric specific format but also tokenize and encodes input for BERT.annotation_file
- path to dataset annotation file in tsv format.vocab_file
- path to model vocabulary file for WordPiece tokinezation. (Optional, can be not provided in case, when another tokenization approach used.)sentence_piece_model_file
- model used for SentencePiece tokenization (Optional in case, when another tokenization approach used).max_seq_length
- maximum total input sequence length after tokenization (Optional, default value is 128).lower_case
- allows switching tokens to lower case register. It is useful for working with uncased models (Optional, default value is False).
cola
- converts The Corpus of Linguistic Acceptability (CoLA) toTextClassificationAnnotattion
. Note: This converter not only converts data to metric specific format but also tokenize and encodes input for BERT.annotation_file
- path to dataset annotation file in tsv format.vocab_file
- path to model vocabulary file for WordPiece tokenization. (Optional, can be not provided in case, when another tokenization approach used.)sentence_piece_model_file
- model used for SentencePiece tokenization (Optional in case, when another tokenization approach used).max_seq_length
- maximum total input sequence length after tokenization (Optional, default value is 128).lower_case
- allows switching tokens to lower case register. It is useful for working with uncased models (Optional, default value is False).
imdb
- converts IMDB sentiment dataset toTextClassificationAnnotattion
. Note: This converter not only converts data to metric specific format but also tokenize and encodes input for BERT.annotation_file
- path to dataset annotation file in tsv format.vocab_file
- path to model vocabulary file for WordPiece tokinezation. (Optional, can be not provided in case, when another tokenization approach used.)sentence_piece_model_file
- model used for SentencePiece tokenization (Optional in case, when another tokenization approach used).max_seq_length
- maximum total input sequence length after tokenization (Optional, default value is 128).lower_case
- allows switching tokens to lower case register. It is useful for working with uncased models (Optional, default value is False).
bert_xnli_tf_record
- converts The Cross-lingual Natural Language Inference Corpus (XNLI) stored in tf records format. This converter usage requires TensorFlow installation. Please make sure that TensorFlow installed before conversion.annotattion_file
- path to annotation file in tf records format.
cmu_panoptic_keypoints
- converts CMU Panoptic dataset toPoseEstimation3dAnnotation
format.data_dir
- dataset root directory, which contain subdirectories with validation scenes data.
clip_action_recognition
- converts annotation video-based action recognition datasets. Before conversion validation set should be preprocessed using approach described here.annotation_file
- path to annotation file in json format.data_dir
- path to directory with prepared data (e. g. data/kinetics/frames_data).clips_per_video
- number of clips per video (Optional, default 3).clip_duration
- clip duration (Optional, default 16)temporal_stride
- temporal stride for frames selection (Optional, default 2).numpy_input
- allows usage numpy files instead images. It can be useful if data required difficult preprocessing steps (e.g. conversion to optical flow) (Optional, defaultFalse
)subset
- dataset split:train
,validation
ortest
(Optional, defaultvalidation
).dataset_meta_file
- path to json file with dataset meta (e.g. label_map, color_encoding).Optional, more details in Customizing dataset meta section.num_samples
- select first n samples from dataset (Optional, if not provided full subset of samples will be used).
continuous_clip_action_recognition
- converts annotation of video-based MS-ASL dataset toClassificationAnnotation
.annotation_file
- path to annotation file in txt format.data_dir
- dataset root directory, which contains subdirectories with extracted video frames.out_fps
- output frame rate of generated video clips.clip_length
- number of frames of generated video clips.img_prefix
- prefix for used images. (Optional, default -img_
).
redweb
- converts ReDWeb dataset for monocular relative depth perception toDepthEstimationAnnotation
data_dir
- the dataset root directory, whereimgs
- directory with RGB images andRD
- directory with relative depth maps are located (Optional, if you want to provideannotation_file
)annotation_file
- the file in txt format which contains pairs of image and depth map files. (Optional, if not provided full content ofdata_dir
will be considered as dataset.)
nyu_depth_v2
- converts NYU Depth Dataset V2 for depth estimation toDepthEstimationAnnotation
. This converter accept preprocessed data stored in HDF5 format, which can be downloaded from this pagedata_dir
- directory with HDF5 files. (Optional, can be omitted if you already have converted images and depth maps).images_dir
- directory for images. Ifallow_convert_data
is True, the directory will be used for saving converted images, otherwise used for data reading. (Optional, can be not provided in conversion case, default value<data_dir>/converted/images
).depth_map_dir
- directory for reference depth maps, stored in numpy format. Ifallow_convert_data
is True, the directory will be used for saving converted depth maps, otherwise used for data reading. (Optional, can be not provided in conversion case, default value<data_dir>/converted/depth
). Please, note, you need to specify path to directory with depth maps withadditional_data_source
parameter in your config during evaluation.allow_convert_data
- allows to convert data from HDF5 format (Optional, default False).
inpainting
- converts images toImageInpaintingAnnotation
.images_dir
- path to images directory.masks_dir
- path to mask dataset to be used for inpainting (Optional).
aflw2000_3d
- converts AFLW2000-3D dataset for 3d facial landmarks regression task toFacialLandmarks3DAnnotation
.data_dir
- directory, where input images and annotation files in MATLAB format stored.
style_transfer
- converts images toStyleTransferAnnotation
.images_dir
- path to images directory.
ade20k
- converts ADE20K dataset toSegmentationAnnotation
.images_dir
- path to directory with images (e.g.ADEChallengeData2016/images/validation
).annotations_dir
- path to directory with annotations (e.g.ADEChallengeData2016/annotations/validation
).object_categories_file
- path to file with labels (e.g.ADEChallengeData2016/objectInfo150.txt
).num_classes
- number of used classes.
criteo_kaggle_dac
- converts Criteo datasets toClassificationAnnotation
.testing_file
- path to preprocessed Criteo file (e.g.criteo/terabyte/terabyte_preprocessed,npz
).batch
- batch size expected by modelsubsample_size
- number of batches in test-only dataset, If provided, total number of records is batch * subsample_sizevalidation
- if provided, only second half of dataset converted to annotations, according to dataset definitionpreprocessed_dir
- path to store preprocessed batch files (e.g.criteo/terabyte/preprocessed
).separator
- symbol used to separate feature identifiers from batch data filename.save_preprocessed_features
- allow to save preprocessed input features intopreprocessed_dir
(Optional, default True).
features_regression
- converts dataset stored in format of directories with preprocessed input numeric data (features) in text files and reference data in the same format toFeatureRegressionAnnotation
. This approach allows comparing output of model from different frameworks (e.g. OpenVINO converted model and source framework realisation).input_dir
- directory with input data files.reference_dir
- directory with reference data. Note: inside converted annotation, path to directory is not stored, only file name, please useadditional_data_source
for providing prefix.input_suffix
- suffix for input files (usually file extension). Optional, default.txt
.reference_suffix
- suffix for reference files (usually file extension). Optional, default.txt
.use_bin_data
- this flag specifies that input data in binary format, optional, defaultFalse
bin_data_dtype
- data type for reading binary data.
multi_feature_regression
- converts dataset stored in format of directories with preprocessed input numeric data (features) in dictionary format, where keys are layer names and values - features and reference data in the same format toFeatureRegressionAnnotation
. This approach allows comparing output of model from different frameworks (e.g. OpenVINO converted model and source framework realisation). Please note, that input and reference should be stored as dict-like objects in npy files.data_dir
- directory with input and reference files.input_suffix
- suffix for input files (usually file extension). Optional, defaultin.npy
.reference_suffix
- suffix for reference files (usually file extension). Optional, defaultout.npy
.prefix
- prefix for input files selection (Optional, ignored if not provided).
librispeech
- converts librispeech dataset toCharachterRecognitionAnnotation
.data_dir
- path to dataset directory, which contains converted wav files.annotation_file
- path to file which describe the data which should be used in evaluation (audio_filepath
,text
,duration
). Optional, used only for data filtering and sorting audio samples by duration.use_numpy
- allows to use preprocessed data stored in npy-files instead of audio (Optional, default False).top_n
- numeric value for getting only the n shortest samples Note: applicable only withannotation_file
providing.max_duration
- maximum clip duration to include into annotation. Default 0, means no duration checking.
criteo
- converts Criteo datasets toClassificationAnnotation
.testing_file
- Path to testing file, terabyte_preprocessed.npz (Criteo Terabyte) or day_6_processed.npz (Criteo Kaggle Dac)batch
- Model batch.subsample_size
- Subsample size in batchesvalidation
- Allows to use half of dataset for validation purposesblock
- Make batch-oriented annotationsseparator
- Separator between input identifier and file identifierpreprocessed_dir
- Preprocessed dataset locationdense_features
- Name of model dense features inputsparse_features
- Name of model sparse features input. For multiple inputs use comma-separated list in form<name>:<index>
lso_features
- Name of lS_o-like features input
im2latex_formula_recognition
- converts im2latex-like datasets toCharacterRecognitionAnnotation
. Example of the datasetimages_dir
- path to input images (rendered or scanned formulas)formula_file
- path to file containing one formula per linesplit_file
- path to file containingimg_name
and corresponding formulaindex
informula_file
separated by tab per linevocab_file
- file containing vocabulary to cast token class indices into human-readable tokens
dna_sequence
- converts dataset for DNA sequencing toDNASequenceAnnotation
.chunks_file
- npy file with input chunks.ref_file
- npy file with reference sequence.num_chunks
- subset size for usage in validation, if not provided the whole dataset will be used.alphabet
- alphabet for sequence decoding (Optional, default [“N”, “A”, “C”, “G”, “T”]).
place_recognition
- converts dataset for image based localization task toPlaceRecognitionAnnotation
subset_file
- matlab file contains info about subset used in validation.
mpii
- converts MPII Human Pose Estimation dataset toPoseEstimationAnnotation
.annotation_file
- json-file with annotation.headboxes_file
- numpy file with boxes contained head coordinates for each image.gt_pos_file
- numpy file with ground truth keypoints, optional, if not provided, default keypoints from annotation will be used.joints_visibility_file
- numpy file with ground truth keypoints visibility level, optional, if not provided, default visibility level from annotation will be used.
cluttered_mnist
- converts MNIST dataset from spatial transformer network example toClassificationAnnotation
.data_file
- npz file with dataset.split
- dataset split:train
- for training subset,valid
- for train-validation subset,test
- for testing subset (Optional, default test).convert_images
- allows convert images from raw data stored in npz and save them into provided directory (Optional, default True).images_dir
- directory for saving converted images (Optional, if not provided, the images will be saved into converted_images directory in the same location, where data_file is stored)
antispoofing
- converts dataset for antispoofing classification task toClassificationAnnotation
data_dir
- path to root folder of the datasetannotation_file
- path to json file containing annotations to the dataset ({index: {path:”…”, labels:[…], bbox:[…] (optional), …})label_id
- number of label in the annotation file representing spoof/real labelsdataset_meta_file
- path to json file with dataset meta (e.g. label_map)
sound_classification
- converts dataset for sound classification toClassificationAnnotation
. The dataset should be represented by directory with input wav files and annotation in 2 column csv format, where first column is audio file name and second is label id from dataset.annotation_file
- csv file with selected subset for evaluation, file structure described above.audio_dir
- directory with input data, (optional, required only if you want check file existence during annotation conversion).
ade20k_image_translation
- converts ADE20K dataset toImageProcessingAnnotation
according toreference_file
.annotations_dir
- path to directory with annotations (e.g.ADEChallengeData2016/annotations
).reference_file
- path to file with pairs key (validation): value (train).
salient_object_detection
- converts dataset for salient object detection toSalientRegionAnnotation
. The dataset should have following structure:images have numeric ids like names and
jpg
extension (e.g. image/0.jpg, image/1.jpg, image/2.jpg, …).salience map located in separated directory, have the same ids like images and
png
extension (e.g. mask/0.png, mask/1.png, mask/2.png).
images_dir
- directory with input images.masks_dir
- directory with reference salience maps.annotation_file
- txt file with selected image ids.
wflw
- converts WFLW dataset for facial landmarks regression task toFacialLandmarksAnnotation
.annotation_file
- path to txt file with ground truth data in WFLW dataset format.images_dir
- path to dataset images, used only for content existence check (optional parameter).
common_object_detection
- converts object detection dataset toDetectionAnnotation
. Dataset should be stored in following format:labels_map defined as text file, where defined labels line by line.
annotations for each image stored in separated text file. Box is represented by space separated info: <label_id> <x_min> <y_min> <x_max> <y_max>.
name of annotation file the same like image name (or additional file with file mapping should be defined).
annotation_dir
- path to directory with annotation files.images_dir
- path to directory with images (Optional, used only for content check step).labels_file
- path to file with labels.pairs_file
- path to file where described image and annotation file pairs (Optional, if not provided list will be created according to annotation_dir content).has_background
- flag that background label should be added to label_map (Optional, default False).add_background_to_label_id
- flag that label_ids defined in annotation should be shifted ifhas_background
enabled.
see_in_the_dark
- converts See-in-the-Dark dataset described in the paper toImageProcessingAnnotation
.annotation_file
- path to image pairs file in txt format.
conll_ner
- converts CONLL 2003 dataset for Named Entity Recognition toBERTNamedEntityRecognitionAnnotation
.annotation_file
- annotation file in txt formavocab_file
- vocab file for word piece tokenization.lower_case
- converts all tokens to lower case during tokenization (Optional, defaultFalse
).max_length
- maximal input sequence length (Optional, default 128).pad_input
- allow padding for input sequence if input less thatmax_length
(Optional, defaultTrue
).include_special_token_lables
- allow extension original dataset labels with special token labels ([CLS,
[SEP]]) (Optional, default
False). *
tacotron2_data_converter- converts input data for custom tacotron2 pipeline. *
annotation_file- tsv file with location input data and reference. *
noise_suppression_dataset- converts dataset for audio denoising to
NoiseSuppressionAnnotation*
annotation_file- txt file with file pairs
<clean_signal> <noisy_signal>. *
vimeo90k- converts Vimeo-90K dataset for a systematic evaluation of video processing algorithms to
SuperResolutionAnnotation. *
annotation_file- path to text file with list of dataset setuplets included in test. *
add_flow- allows annotation of flow data (optional, default
False). *
kaldi_asr_data- converts preprocessed Kaldi\* features dataset to
CharacterRecognitionAnnotation. *
annotation_file- file with gt transcription table. *
data_dir- directory with ark files. *
features_subset_file- file with list testing ark files, Optional, if not provided, all found in
data_dirfiles will be used. *
ivectors- include ivectors features to input, Optional, default
False. *
kaldi_feat_regression- converts preprocessed Kaldi\* features to
RegressionAnnotation. *
data_dir- directory with input ark files. *
features_subset_file- file with list testing ark files, Optional, if not provided, all found in
data_dirfiles will be used. *
ivectors- include ivectors features to input, Optional, default
False. *
ref_data_dir- directory with reference ark files (Optional, if not provided
data_dirwill be used instead). *
vectors_mode- allow usage each vector in utterance as independent data. *
ref_file_suffix- suffix for search reference files (Optional, default
_kaldi_score). *
electricity- converts Electricity dataset to
TimeSeriesForecastingAnnotation. *
data_path_file- Path to dataset file in .csv format. *
num_encoder_steps- The maximum number of historical timestamps that model use. *
yolo_labeling- converts object detection dataset with annotation in YOLO labeling format to
DetectionAnnotation. *
annotations_dir- path to directory with annotation files in txt format. *
images_dir- path to directory with images (Optional). *
labels_file- path to file with labels in txt format (Optional). *
images_suffix- suffix for image file names (Optional, default:
.jpg). *
label_me_detection- converts dataset obtained using [LabelMe](`http://labelme.csail.mit.edu/Release3.0/ <http://labelme.csail.mit.edu/Release3.0/>`__) Annotation Tool to
DetectionAnnotation. *
annotations_dir- path to directory with annotation files in xml format. *
dataset_meta_file- path to json file with dataset meta (e.g. label_map, color_encoding). More details in [Customizing dataset meta](#customizing-dataset-meta) section. *
images_dir- path to directory with images (Optional). *
has_background- allows convert dataset with/without adding background_label (Optional, default: False). *
label_me_segmentation- converts dataset obtained using [LabelMe](`http://labelme.csail.mit.edu/Release3.0/ <http://labelme.csail.mit.edu/Release3.0/>`__) Annotation Tool to
SegmentationAnnotation. *
annotations_dir- path to directory with annotation files in xml format. *
dataset_meta_file- path to json file with dataset meta (e.g. label_map, color_encoding). More details in [Customizing dataset meta](#customizing-dataset-meta) section. *
images_dir- path to directory with images (Optional). *
masks_dir- path to directory with ground truth segmentation masks (Optional). *
cls_dataset_folder- converts generic classification dataset with [DatasetFolder](`https://pytorch.org/vision/stable/datasets.html <https://pytorch.org/vision/stable/datasets.html>`__#datasetfolder) format to
ClassificationAnnotation. *
data_dir` - directory with input images in following structure:data_dir/class_a/xxx.ext data_dir/class_a/xxy.ext data_dir/class_b/[...]/xxz.ext ... data_dir/class_y/123.ext data_dir/class_z/nsdf3.ext data_dir/class_z/[...]/asd932_.ext
open_images_detection
- converts Open Images dataset for object detection task toDetectionAnnotation
.bbox_csv_file
- path to cvs file which contains bounding box coordinates.labels_file
- path to file with class labels in csv format.images_dir
- path to images folder (Optional).label_start
- specifies label index start in label map. You can provide another value, if you want to use this dataset for separate label validation (Optional, default value is 1).
Customizing Dataset Meta¶
There are situations when we need to customize some default dataset parameters (e.g. replace original dataset label map with own.) You are able to overload parameters such as label_map
, segmentation_colors
, background_label
using dataset_meta_file
argument. Dataset meta file is JSON file, which can contain the following parameters:
label_map
is dictionary where<CLASS_ID>
is key and<CLASS_NAME>
- value.labels
is the list of strings, which represent class names (order is matter, the index of class name used as class id). Can be used insteadlabel_map
.background_label
- id of background label in the dataset.segmentation_colors
(if your dataset for semantic segmentation task uses color encoding). Segmentation colors is a list of channel-wise values for each class. (e.g. if your dataset has 3 classes in BGR colors, segmentation colors for it will looks like:[[255, 0, 0], [0, 255, 0], [0, 0, 255]]
). Example of dataset_meta.json content:{ "label_map": {"0": "background", "1": "cat", "2": "dog"}, "background_label": "0", "segmentation_colors": [[0, 0, 0], [255, 0, 0], [0, 0, 255]] }