Need help with magic_init?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

philkr
133 Stars 50 Forks Other 11 Commits 2 Opened issues

Services available

!
?

Need anything else?

Contributors list

# 10,510
C++
Jupyter...
MATLAB
ffmpeg
10 commits
# 508
Shell
C++
MATLAB
acceler...
1 commit

Data-dependent initialization of convolutional neural networks

Created by Philipp Krähenbühl.

Introduction

This code implements the initialization presented in our arXiv tech report, which is under submission at ICLR 2016.

This is a reimplementation and currently work in progress. Use at your own risk.

License

This code is released under the BSD License (refer to the LICENSE file for details).

Citing

If you find our initialization useful in your research, please consider citing:

@article{krahenbuhl2015data,
  title={Data-dependent Initializations of Convolutional Neural Networks},
  author={Kr{\"a}henb{\"u}hl, Philipp and Doersch, Carl and Donahue, Jeff and Darrell, Trevor},
  journal={arXiv preprint arXiv:1511.06856},
  year={2015}
}

Setup

Checkout the project and create a symlink to caffe in the

magic_init
directory:
Shell
ln -s path/to/caffe/python/caffe caffe

Examples

Here is a quick example on how to initialize alexnet:

bash
python magic_init.py path/to/alexnet/deploy.prototxt path/to/output.caffemodel -d "path/to/some/images/*.png" -q -nit 10 -cs
Here
-d
flag allows you to initialize the network using your own images. Feel free to use imagenet, Pascal, COCO or whatever you have at hand, it shouldn't make a big difference. The
-q
(queit) flag suppresses all the caffe logging,
-nit
controls the number of batches used (while
-bs
controls the batch size). Finally
-cs
rescales the gradients accross layers. This rescaling currently works best for feed-forward networks, and might not work too well for DAG structured networks (we are working on that).

To run the k-means initialization use:

bash
python magic_init.py path/to/alexnet/deploy.prototxt path/to/output.caffemodel -d "path/to/some/images/*.png" -q -nit 10 -cs -t kmeans

Finally,

python magic_init.py -h
should provide you with more help.

Pro tips

If you're numpy implementation is based on openblas, try disabeling threading

export OPENBLAS_NUM_THREADS=1
, it can improve the runtime performance a bit.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.