Need help with ImageNet-Datasets-Downloader?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

mf1024
335 Stars 106 Forks 40 Commits 17 Opened issues

Description

ImageNet dataset downloader. Creates a custom dataset by specifying the required number of classes and images in a class.

Services available

!
?

Need anything else?

Contributors list

# 220,880
Python
36 commits
# 98,802
rgbd
visual-...
Common ...
ml
1 commit
# 537,032
PHP
Shell
Python
1 commit

ImageNet Downloader

This is ImageNet dataset downloader. You can create new datasets from subsets of ImageNet by specifying how many classes you need and how many images per class you need. This is achieved by using image urls provided by ImageNet API.

In this blog post I wrote in a bit more detail how and why I wrote the tool. Also, I did a little analysis of the current state of the ImageNet URLs in the post.

This software is written in Python 3

Usage

The following command will randomly select 100 of ImageNet classes with at least 200 images in them and start downloading:

python ./downloader.py \
    -data_root /data_root_folder/imagenet \
    -number_of_classes 100 \
    -images_per_class 200

The following command will download 500 images from each of selected class:

python ./downloader.py 
    -data_root /data_root_folder/imagenet \
    -use_class_list True \
    -class_list n09858165 n01539573 n03405111 \
    -images_per_class 500 
You can find class list in this csv where I list every class that appear in the ImageNet with number of total urls and total flickr urls it that class.

Multiprocessing workers

I've implementet parallel request processing and I've added multiprocessing_workers parameter which by default is 8. You can turn it higher, but I havent yet tested the limits of flickr allowed bandwith myself, so use it with care and you will have to find the limits yourself if you want to go for the maximum speed.

You can do something like this:

python ./downloader.py \
    -data_root /data_root_folder/imagenet \
    -number_of_classes 1000 \
    -images_per_class 500 \
    -multiprocessing_workers 24

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.