Need help with pytorch-cifar-models?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

junyuseu
287 Stars 89 Forks 18 Commits 6 Opened issues

Description

3.41% and 17.11% error on CIFAR-10 and CIFAR-100

Services available

!
?

Need anything else?

Contributors list

# 160,157
Python
Shell
cifar10
pytorch
14 commits
# 150,032
C++
TeX
Jupyter...
pytorch
3 commits

Experiments on CIFAR datasets with PyTorch

Introduction

Reimplement state-of-the-art CNN models in cifar dataset with PyTorch, now including:

1.ResNet

2.PreActResNet

3.WideResNet

4.ResNeXt

5.DenseNet

other results will be added later.

Requirements:software

Requirements for PyTorch

Requirements:hardware

For most experiments, one or two K40(~11G of memory) gpus is enough cause PyTorch is very memory efficient. However, to train DenseNet on cifar(10 or 100), you need at least 4 K40 gpus.

Usage

  1. Clone this repository
git clone https://github.com/junyuseu/pytorch-cifar-models.git

In this project, the network structure is defined in the models folder, the script

gen_mean_std.py
is used to calculate the mean and standard deviation value of the dataset.
  1. Edit main.py and run.sh

In the

main.py
, you can specify the network you want to train(for example):
model = resnet20_cifar(num_classes=10)
...
fdir = 'result/resnet20_cifar10'

Then, you need specify some parameter for training in

run.sh
. For resnet20:
CUDA_VISIBLE_DEVICES=0 python main.py --epoch 160 --batch-size 128 --lr 0.1 --momentum 0.9 --wd 1e-4 -ct 10
  1. Train
nohup sh run.sh > resnet20_cifar10.log &

After training, the training log will be recorded in the .log file, the best model(on the test set) will be stored in the fdir.

Note:For first training, cifar10 or cifar100 dataset will be downloaded, so make sure your comuter is online. Otherwise, download the datasets and decompress them and put them in the

data
folder.
  1. Test
CUDA_VISIBLE_DEVICES=0 python main.py -e --resume=fdir/model_best.pth.tar
  1. CIFAR100

The default setting in the code is for cifar10, to train with cifar100, you need specify it explicitly in the code.

model = resnet20_cifar(num_classes=100)

Note: you should also change fdir In the run.sh, you should set

-ct 100

Results

Note:The results as follow are got by only one single experiment.

We got comparable or even better results than the original papers, the experiment settings are totally follow the original ones

ResNet

layers

#params error(%)
20 0.27M 8.33
32 0.46M 7.36
44 0.66M 6.77
56 0.85M 6.73
110 1.7M 6.13
1202 19.4M -

PreActResNet

dataset

network baseline unit pre-activation unit
CIFAR-10 ResNet-110 6.13 6.13
CIFAR-10 ResNet-164 5.84 5.35
CIFAR-10 ResNet-1001 11.27 5.13
CIFAR-100 ResNet-164 24.99 24.50
CIFAR-100 ResNet-1001 31.73 24.03

WideResNet

depth-k

#params CIFAR-10 CIFAR-100
20-10 26.8M 4.27 19.73
26-10 36.5M 3.89 19.51

ResNeXt

network

#params CIFAR-10 CIFAR-100
ResNeXt-29,1x64d 4.9M 4.51 22.09
ResNeXt-29,8x64d 34.4M 3.78 17.44
ResNeXt-29,16x64d 68.1M 3.69 17.11

DenseNet

network

depth #params CIFAR-10 CIFAR-100
DenseNet-BC(k=12) 100 0.8M 4.69 22.19
DenseNet-BC(k=24) 250 15.3M 3.44 17.17
DenseNet-BC(k=40) 190 25.6M 3.41 17.33

References:

[1] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016.

[2] K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In ECCV, 2016.

[3] S. Zagoruyko and N. Komodakis. Wide residual networks. In BMVC, 2016.

[4] S. Xie, G. Ross, P. Dollar, Z. Tu and K. He Aggregated residual transformations for deep neural networks. In CVPR, 2017

[5] H. Gao, Z. Liu, L. Maaten and K. Weinberger. Densely connected convolutional networks. In CVPR, 2017

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.