collapsed Gibbs sampling

Summary

MCMC samplers can often be improved by marginalizing out a subset of the variables in closed form and performing MCMC over the remaining variables. This is more statistically efficient since each particle can cover a larger part of the distribution, and it can also improve mixing by allowing larger jumps.

Context

This concept has the prerequisites:

Goals

  • Be able to derive the update rules for collapsed Gibbs sampling
  • Be aware of the motivations in terms of:
    • greater statistical efficiency (from the Rao-Blackwell theorem)
    • faster mixing

Core resources (we're sorry, we haven't finished tracking down resources for this concept yet)

Supplemental resources (the following are optional, but you may find them useful)

-Free-

Machine learning summer school: Markov chain Monte Carlo (2009)
A video tutorial on MCMC methods.
Location: Part 2, 16:47 to 21:37
Author: Iain Murray

-Paid-

See also

-No Additional Notes-