Need help with pytorch-prunes?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

138 Stars 17 Forks MIT License 11 Commits 1 Opened issues


Code for

Services available


Need anything else?

Contributors list

# 397,521
11 commits

A Closer Look at Structured Pruning for Neural Network Compression

Code used to reproduce experiments in

To prune, we fill our networks with custom

, which are manipulated using
in There will certainly be a better way to do this, but we leave this as an exercise to someone who can code much better than we can.


This is best done in a clean conda environment:

conda create -n prunes python=3.6
conda activate prunes
conda install pytorch torchvision -c pytorch

Repository layout

: contains all of the code for training large models from scratch and for training pruned models from scratch
: contains the code for pruning trained models
: contains useful pruning functions and any functions we used commonly

CIFAR Experiments

First, you will need some initial models.

To train a WRN-40-2:

python --net='res' --depth=40 --width=2.0 --data_loc= --save_file='res'

The default arguments of are suitable for training WRNs. The following trains a DenseNet-BC-100 (k=12) with its default hyperparameters:

python --net='dense' --depth=100 --data_loc= --save_file='dense' --no_epochs 300 -b 64 --epoch_step '[150,225]' --weight_decay 0.0001 --lr_decay_ratio 0.1

These will automatically save checkpoints to the



Once training is finished, we can prune our networks using (defaults are set to WRN pruning, so extra arguments are needed for DenseNets)
``` python --net='res' --dataloc= --basemodel='res' --savefile='resfisher' python --net='res' --dataloc= --l1prune=True --basemodel='res' --savefile='res_l1'

python --net='dense' --depth 100 --dataloc= --basemodel='dense' --savefile='densefisher' --learningrate 1e-3 --weightdecay 1e-4 --batchsize 64 --noepochs 2600 python --net='dense' --depth 100 --dataloc= --l1prune=True --basemodel='dense' --savefile='densel1' --learningrate 1e-3 --weightdecay 1e-4 --batchsize 64 --no_epochs 2600

Note that the default is to perform Fisher pruning, so you don't need to pass a flag to use it.  
Once finished, we can train the pruned models from scratch, e.g.:  

python --dataloc= --net='res' --basefile='resfisherprunes' --deploy --mask=1 --savefile='resfisherprunesscratch' ```

Each model can then be evaluated using:

python --deploy --eval --data_loc= --net='res' --mask=1 --base_file='res_fisher__prunes'

Training Reduced models

This can be done by varying the input arguments to To reduce depth or width of a WRN, change the corresponding option:

python --net='res' --depth= --width= --data_loc= --save_file='res_reduced'

To add bottlenecks, use the following:

python --net='res' --depth=40 --width=2.0 --data_loc= --save_file='res_bottle' --bottle --bottle_mult 

With DenseNets you can modify the

, or use
--bottle --bottle_mult 
as above.


Jack Turner wrote the L1 stuff, and some other stuff for that matter.

Code has been liberally borrowed from many a repo, including, but not limited to:

Citing this work

If you would like to cite this work, please use the following bibtex entry:

  title={A Closer Look at Structured Pruning for Neural Network Compression},
  author={Crowley, Elliot J and Turner, Jack and Storkey, Amos and O'Boyle, Michael},
  journal={arXiv preprint arXiv:1810.04622},

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.