keras-gp

by alshedivat

alshedivat / keras-gp

Keras + Gaussian Processes: Learning scalable deep and recurrent kernels.

217 Stars 52 Forks Last release: over 3 years ago (0.3.2) MIT License 66 Commits 3 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Gaussian Processes for Keras

Build Status Coverage Status license

KGP extends Keras with Gaussian Process (GP) layers. It allows one to build flexible GP models with kernels structured with deep and recurrent networks built with Keras. The structured part of the model (the neural net) runs on Theano or Tensorflow. The GP layers use a custom backend based on GPML 4.0 library, and builds on KISS-GP and extensions. The models can be trained in stages or jointly, using full-batch or semi-stochastic optimization approaches (see our paper). For additional resources and tutorials on Deep Kernel Learning and KISS-GP see https://people.orie.cornell.edu/andrew/code/

KGP is compatible with: Python 2.7-3.5.

In particular, this package implements the method described in our paper:
Learning Scalable Deep Kernels with Recurrent Structure
Maruan Al-Shedivat, Andrew Gordon Wilson, Yunus Saatchi, Zhiting Hu, Eric P. Xing
Journal of Machine Learning Research, 2017.

Getting started

KGP allows to build models in the same fashion as Keras, using the functional API. For example, a simple GP-RNN model can be built and compiled in just a few lines of code:

from keras.layers import Input, SimpleRNN
from keras.optimizers import Adam

from kgp.layers import GP from kgp.models import Model from kgp.losses import gen_gp_loss

input_shape = (10, 2) # 10 time steps, 2 dimensions batch_size = 32 nb_train_samples = 512 gp_hypers = {'lik': -2.0, 'cov': [[-0.7], [0.0]]}

Build the model

inputs = Input(shape=input_shape) rnn = SimpleRNN(32)(inputs) gp = GP(gp_hypers, batch_size=batch_size, nb_train_samples=nb_train_samples) outputs = [gp(rnn)] model = Model(inputs=inputs, outputs=outputs)

Compile the model

loss = [gen_gp_loss(gp) for gp in model.output_layers] model.compile(optimizer=Adam(1e-2), loss=loss)

Note that KGP models support arbitrary off-the-shelf optimizers from Keras.

Further resources: - A quick tutorial that walks you through the key components of the library. - A few more examples.

Installation

KGP depends on Keras and requires either Theano or TensorFlow being installed. The GPML backend requires either MATLAB or Octave and a corresponding Python interface package: Oct2Py for Octave or the MATLAB engine for Python. Generally, MATLAB backend seems to provide faster runtime. However, if you compile the latest version of Octave with JIT and OpenBLAS support, the overhead gets reduced to minimum.

If you are using Octave, you will need the

statistics
package. You can install the package using Octave-Forge:
bash
$ octave --eval "pkg install -forge -verbose io"
$ octave --eval "pkg install -forge -verbose statistics"

The requirements can be installed via pip as follows (use

sudo
if necessary):
$ pip install -r requirements.txt

To install the package, clone the repository and run

setup.py
as follows:
$ git clone --recursive https://github.com/alshedivat/kgp
$ cd kgp
$ python setup.py develop [--user]

The

--user
flag (optional) will install the package for a given user only.

Note: Recursive clone is required to get GPML library as a submodule. If you already have a copy of GPML, you can set

GPML_PATH
environment variable to point to your GPML folder instead.

Contribution

Contributions and especially bug reports are more than welcome.

Citation

@article{alshedivat2017srk,
  title={Learning scalable deep kernels with recurrent structure},
  author={Al-Shedivat, Maruan and Wilson, Andrew Gordon and Saatchi, Yunus and Hu, Zhiting and Xing, Eric P},
  journal={Journal of Machine Learning Research},
  volume={18},
  number={1},
  year={2017},
}

License

For questions about the code and licensing details, please contact Maruan Al-Shedivat and Andrew Gordon Wilson.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.