Need help with TensorNet?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

Bihaqo
126 Stars 35 Forks 29 Commits 3 Opened issues

Services available

!
?

Need anything else?

Contributors list

No Data

TensorNet

This is a MATLAB and Theano+Lasagne implementation of the Tensor Train layer (TT-layer) of a neural network. For a TensorFlow implementation see a separate repository.

In short, the TT-layer acts as a fully-connected layer but is much more compact and allows to use lots of hidden units without slowing down the learning and inference.
For the additional information see the following paper:

Tensorizing Neural Networks
Alexander Novikov, Dmitry Podoprikhin, Anton Osokin, Dmitry Vetrov; In Advances in Neural Information Processing Systems 28 (NIPS-2015) [arXiv].

Please cite it if you write a scientific paper using this code.
In BiBTeX format:

latex
@incollection{novikov15tensornet,
  author    = {Novikov, Alexander and Podoprikhin, Dmitry and Osokin, Anton and Vetrov, Dmitry},
  title     = {Tensorizing Neural Networks},
  booktitle = {Advances in Neural Information Processing Systems 28 (NIPS)},
  year      = {2015},
}

Installation

MATLAB version

Install the TT-Toolbox (just download it and run

setup.m
to add everything important into the MATLAB path).

Install the MatConvNet framework (preferably with the GPU support). TensorNet works with MatConvNet 1.0-beta11 (April 2015) and higher (the latest tested version is 1.0-beta14).
Add the

mataconvnet_path/examples
folder to the MATLAB path to be able to use the
cnn_train
function.

Copy this repository and add the

src/matlab
folder into the MATLAB path.

Now you can test TensorNet using the command

matlab
vl_test_ttlayers

To test GPU support (if you have compiled MatConvNet in GPU mode) use:

matlab
vl_test_ttlayers(1)

Theano+Lasagne version

Install fresh version of Theano and Lasagne.

Copy this repository and add the

src/python
folder into the Python path.

Pretrained models

MNIST shapes

In this experiment we compared how shapes and ranks influence the performance of the TT-layer using the MNIST dataset (see figure 1 and section 6.1 of the original paper for the details). Download models in the MatConvNet format (.mat file, 2.9 Mb) and preprocessed MNIST dataset (.mat file, 132 Mb).

You will find a cell array of models with metadata, the first and the last epochs of training included for each model. Example of usage (computing the validation error):

matlab
imdb = load('imdb.mat');
load('mnist_shapes.mat');
% Choose (for example) the 5-th model whose shape equal 4 x 8 x 8 x 4.
net = models{5}.lastEpoch.net;
% Remove the softmax layer (unnecessary during the validation).
net.layers(end) = [];
valIdx = find(imdb.images.set == 3);
res = vl_simplenn(net, imdb.images.data(:, :, :, valIdx));
scores = squeeze(res(end).x);
[bestScore, best] = max(scores);
acc = mean(best == imdb.images.labels(valIdx));
fprintf('Accuracy is %f\n', acc);

Reproducing experiments

Right now just one basic example on the MNIST dataset is available (more experiments from the paper are coming soon). To try it out, navigate to the

experiments/mnist
folder and type the following command in the MATLAB prompt:
matlab
[net_tt, info_tt] = cnn_mnist_tt('expDir', 'data/mnist-tt');

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.