Boltzmann machines

(1.5 hours to learn)

Summary

Boltzmann machines are a kind of probabilistic neural network used in density modeling. They can be viewed as an MRF with only pairwise connections between units, and where the units are typically binary-valued. Restricted Boltzmann machines (RBMs) are a widely used special case.

Context

This concept has the prerequisites:

Goals

  • Know the definition of a Boltzmann machine (i.e. what distribution it represents)
  • Be able to (approximately) sample from a Boltzmann machine using Gibbs sampling.
  • Derive the fact that the model correlations must match the data correlations at the maximum likelihood solution.
  • Why can it be beneficial to add hidden units to the network?
  • Be aware of the analogies between Boltzmann machine updates and Hopfield network updates.

Core resources (read/watch one of the following)

-Free-

Information Theory, Inference, and Learning Algorithms
A graudate-level textbook on machine learning and information theory.
Author: David MacKay
Coursera: Neural Networks for Machine Learning (2012)
An online course by Geoff Hinton, who invented many of the core ideas behind neural nets and deep learning.
Author: Geoffrey E. Hinton

See also