TensorFlow implementation of Fast Weights
The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:
This repo is a TensorFlow implementation of
Using Fast Weights to Attend to the Recent Past Jimmy Ba, Geoffrey Hinton, Volodymyr Mnih, Joel Z. Leibo, Catalin Ionescu NIPS 2016, https://arxiv.org/abs/1610.06258
Specifically, we follow the experiments in
Sec 4.1 Associative retrievaland try to reproduce the results in Table 1 and Figure 2. The fast weights model can achieve 100% accuracy (0% error rate) on R=50 setting in ~30K iterations.
Running result as follows:
Fast Weights(with layernorm):
Fast Weights(without layernorm):
Both trained on GTX 980 Ti, with TensorFlow 0.11rc1.
Setting on R=50, using ADAM optimizer with default parameters.
LSTMbaseline model in similar ways.
Fan Wu ([email protected])