Pixel RNN

July 12, 2016 ยท View on GitHub

Theano implementation of the Diagonal BiLSTM and Pixel CNN models from Pixel Recurrent Neural Networks (van der Oord et al.) on binarized MNIST.

Not yet implemented:

  • Residual connections
  • 256-way softmax output

The Diagonal BiLSTM gets a dev-set NLL of 82.12 after 100K iterations (at which point it still hasn't fully converged).

Samples from the Diagonal BiLSTM after 100K iterations:

Diagonal BiLSTM samples

Ground-truth binarized MNIST samples:

Ground-truth samples