deepmat

Project Website: None

Github Link:

Description

= Generative Stochastic Network =

A simple implementation of GSN according to (Bengio et al., 2013)
= Convolutional Neural Network =

A naive implementation (purely using Matlab)
Pooling: max (Jonathan Masci’s code) and average
Not for serious use!
= Restricted Boltzmann Machine & Deep Belief Networks =

Binary/Gaussian Visible Units + Binary Hidden Units
Enhanced Gradient, Adaptive Learning Rate
Adadelta for RBM
Contrastive Divergence
(Fast) Persistent Contrastive Divergence
Parallel Tempering
DBN: Up-down Learning Algorithm
= Deep Boltzmann Machine =

Binary/Gaussian Visible Units + Binary Hidden Units
(Persistent) Contrastive Divergence
Enhanced Gradient, Adaptive Learning Rate
Two-stage Pretraining Algorithm (example)
Centering Trick (fixed center variables only)
= Denoising Autoencoder (Tied Weights) =

Binary/Gaussian Visible Units + Binary(Sigmoid)/Gaussian Hidden Units
tanh/sigm/relu nonlinearities
Shallow: sparsity, contractive, soft-sparsity (log-cosh) regularization
Deep: stochastic backprop
Adagrad, Adadelta
= Multi-layer Perceptron =

Stochastic Backpropagation, Dropout
tanh/sigm/relu nonlinearities
Adagrad, Adadelta
Balanced minibatches using crossvalind()