We look at how to use Bayesian Inference on Neural Networks. Specifically on variational approximate inference. Often when we train a model, we have a cost function constructed from, say taking the difference of network output and the desired output in the training set. The result of the optimisation is a minimised cost function at a single set of model parameters. Bayesian inference aims to get a distribution of the model parameters instead (generally obtaining a posterior distribution).

Here we show how to fit data to a distribution with neural networks as well as inferring a distribution for unobserved variables. Consider some random variable x with a few observations generated by a distribution $p(x)$. Often we would like to assume there are latent variables $z$ that generates $x$ according to some conditional probability $p(x|z)$.

“Deep learning” describes the use of multilayer computational architectures, typically of Neural Networks, to capture a hierarchy of features embedded within the data. Some of these techniques have achieved super human performance in recent years on pattern recognition tasks. This post documents my experiments on implementing a multilayer recurrent neural network for learning musical patterns and generation. Code written in Python is hosted here.

This post documents the inner workings of a statistical model (Restricted Boltzmann Machine) for modelling the joint probability of multiple random variables (R.V.s). In many real life scanarios, getting to grips with a statistical model can often give us an upper hand. In some cases we are given some data and have to work out a statistical model for it. RBMs does exactly that for multiple R.V.s. The challenge is how do we configure an RBM, which behavior depends on its parameters, to best describe the statistics of the data. Project code written in Python can be found here