(1.5 hours to learn)
Boltzmann machines are a kind of probabilistic neural network used in density modeling. They can be viewed as an MRF with only pairwise connections between units, and where the units are typically binary-valued. Restricted Boltzmann machines (RBMs) are a widely used special case.
This concept has the prerequisites:
- Hopfield networks (Boltzmann machines are a probabilistic version of Hopfield networks.)
- maximum likelihood (Boltzmann machines are trained using maximum likelihood.)
- gradient (The gradient is needed for the maximum likelihood update.)
- Gibbs sampling (Gibbs sampling can be used to approximately sample from the equilibrium distribution.)
- Know the definition of a Boltzmann machine (i.e. what distribution it represents)
- Be able to (approximately) sample from a Boltzmann machine using Gibbs sampling.
- Derive the fact that the model correlations must match the data correlations at the maximum likelihood solution.
- Why can it be beneficial to add hidden units to the network?
- Be aware of the analogies between Boltzmann machine updates and Hopfield network updates.
Core resources (read/watch one of the following)
→ Information Theory, Inference, and Learning Algorithms
A graudate-level textbook on machine learning and information theory.
→ Coursera: Neural Networks for Machine Learning (2012)
An online course by Geoff Hinton, who invented many of the core ideas behind neural nets and deep learning.
- Restricted Boltzmann machines (RBMs) are a special case of Boltzmann machines often used in practice.
- The model distribution can also be approximated using the mean field approximation .
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation