Gibbs sampling

(45 minutes to learn)


Gibbs sampling is a Markov Chain Monte Carlo (MCMC) algorithm where each random variable is iteratively resampled from its conditional distribution given the remaining variables. It's a simple and often highly effective approach for performing posterior inference in probabilistic models.


This concept has the prerequisites:

Core resources (read/watch one of the following)


Coursera: Probabilistic Graphical Models (2013)
An online course on probabilistic graphical models.
Author: Daphne Koller
Additional dependencies:
  • Bayesian networks
Other notes:
  • Click on "Preview" to see the videos.
Bayesian Reasoning and Machine Learning
A textbook for a graudate machine learning course.
Author: David Barber
Additional dependencies:
  • Bayesian networks
  • multivariate Gaussian distribution


Supplemental resources (the following are optional, but you may find them useful)


Information Theory, Inference, and Learning Algorithms
A graudate-level textbook on machine learning and information theory.
Author: David MacKay
Additional dependencies:
  • Metropolis-Hastings algorithm
Machine learning summer school: Markov chain Monte Carlo (2009)
A video tutorial on MCMC methods.
Location: 69:40 to 77:34
Author: Iain Murray


See also