Gibbs sampling

(45 minutes to learn)

Summary

Gibbs sampling is a Markov Chain Monte Carlo (MCMC) algorithm where each random variable is iteratively resampled from its conditional distribution given the remaining variables. It's a simple and often highly effective approach for performing posterior inference in probabilistic models.

Context

This concept has the prerequisites:

Core resources (read/watch one of the following)

-Free-

Coursera: Probabilistic Graphical Models (2013)
An online course on probabilistic graphical models.
Author: Daphne Koller
Additional dependencies:
  • Bayesian networks
Other notes:
  • Click on "Preview" to see the videos.
Bayesian Reasoning and Machine Learning
A textbook for a graudate machine learning course.
Author: David Barber
Additional dependencies:
  • Bayesian networks
  • multivariate Gaussian distribution

-Paid-

Supplemental resources (the following are optional, but you may find them useful)

-Free-

Information Theory, Inference, and Learning Algorithms
A graudate-level textbook on machine learning and information theory.
Author: David MacKay
Additional dependencies:
  • Metropolis-Hastings algorithm
Machine learning summer school: Markov chain Monte Carlo (2009)
A video tutorial on MCMC methods.
Location: 69:40 to 77:34
Author: Iain Murray

-Paid-

See also