Cut Datasets¶

Original datasets are considerably big in size. If you want to save your time when loading original datasets into the DL Workbench, cut them as described in the following sections.

ImageNet Dataset¶

Cut ImageNet Dataset¶

1. Save the script to cut datatsets to the following directory:

• Linux*, macOS*: /home/<user>/Work. Replace <user> with your username.

• Windows* : C:\Work

Note

Replace <user> with your username. Run the following command in a terminal for Linux, macOS and in the Windows PowerShell* for Windows.

python /home/<user>/Work/cut_dataset.py \
--source_archive_dir=/home/<user>/Work/imagenet.zip \
--output_size=20 \
--output_archive_dir=/home/<user>/Work/subsets \
--dataset_type=imagenet \
--first_image=10

python C:\\Work\\cut_dataset.py
--source_archive_dir=C:\\Work\\imagenet.zip 
--output_size=20
--output_archive_dir=C:\\Work\\subsets 
--dataset_type=imagenet
--first_image=10


This command runs the script with the following arguments:

Parameter

Explanation

--source_archive_dir

--output_size=20

Number of images to be left in a smaller dataset

--output_archive_dir

Full directory to the smaller dataset, excluding the name

--dataset_type

Type of the source dataset

--first_image

Optional . The index of the image to start cutting from. Specify if you want to split your dataset into training and validation subsets. The default value is 0.

Pascal Visual Object Classes (VOC) Dataset¶

Cut Pascal VOC Dataset¶

1. Save the script to cut datatsets to the following directory:

• Linux*, macOS*: /home/<user>/Work. Replace <user> with your username.

• Windows* : C:\Work

Note

Replace <user> with your username. Run the following command in a terminal for Linux, macOS and in the Windows PowerShell* for Windows.

python /home/<user>/Work/cut_dataset.py \
--source_archive_dir=/home/<user>/Work/voc.tar.gz \
--output_size=20 \
--output_archive_dir=/home/<user>/Work/subsets \
--dataset_type=voc \
--first_image=10

python C:\\Work\\cut_dataset.py 
--source_archive_dir=C:\\Work\\voc.tar.gz
--output_size=20 
--output_archive_dir=C:\\Work\\subsets
--dataset_type=voc 
--first_image=10


This command runs the script with the following arguments:

Parameter

Explanation

--source_archive_dir

--output_size=20

Number of images to be left in a smaller dataset

--output_archive_dir

Full directory to the smaller dataset, excluding the name

--dataset_type

Type of the source dataset

--first_image

Optional . The index of the image to start cutting from. Specify if you want to split your dataset into training and validation subsets. The default value is 0.

Common Objects in Context (COCO) Dataset¶

Cut COCO Dataset¶

1. Save the script to cut datatsets to the following directory:

• Linux*, macOS*: /home/<user>/Work. Replace <user> with your username.

Note

Replace <user> with your username.

* Windows* : C:\Work

Note

Replace <user> with your username. Run the following command in a terminal for Linux, macOS and in the Windows PowerShell* for Windows.

python /home/<user>/Work/cut_dataset.py \
--source_images_archive_dir=/home/<user>/Work/coco_images.zip \
--source_annotations_archive_dir=/home/<user>/Work/coco_annotations_.zip \
--output_size=20 \
--output_archive_dir=/home/<user>/Work/subsets \
--first_image=10

python C:\\Work\\cut_dataset.py
--source_images_archive_dir=C:\\Work\\coco_images.zip 
--source_annotations_archive_dir=C:\\Work\\coco_annotations_.zip
--output_size=20 
--output_archive_dir=C:\\Work\\subsets
--first_image=10


This command runs the script with the following arguments:

Parameter

Explanation

--source_images_archive_dir

--source_annotations_archive_dir

--output_size

Number of images to be left in a smaller dataset

--output_archive_dir

Full directory to the smaller dataset excluding the name

--dataset_type

Type of the source dataset

--first_image`

Optional . The number of the image to start cutting from. Specify if you want to split your dataset into training and validation subsets. The default value is 0.