(1.5 hours to learn)
Boltzmann machines are a kind of probabilistic neural network used in density modeling. They can be viewed as an MRF with only pairwise connections between units, and where the units are typically binary-valued. Restricted Boltzmann machines (RBMs) are a widely used special case.
This concept has the prerequisites:
- Know the definition of a Boltzmann machine (i.e. what distribution it represents)
- Be able to (approximately) sample from a Boltzmann machine using Gibbs sampling.
- Derive the fact that the model correlations must match the data correlations at the maximum likelihood solution.
- Why can it be beneficial to add hidden units to the network?
- Be aware of the analogies between Boltzmann machine updates and Hopfield network updates.
Core resources (read/watch one of the following)
→ Information Theory, Inference, and Learning Algorithms
A graudate-level textbook on machine learning and information theory.
→ Coursera: Neural Networks for Machine Learning (2012)
An online course by Geoff Hinton, who invented many of the core ideas behind neural nets and deep learning.