mean field approximation

(1 hours to learn)

Summary

In variational inference algorithms, we try to approximate an intractable distribution with a tractable one. Mean field is probably the most common example. The approximating distribution is factorized into independent terms corresponding to different variables or groups of variables. Variational Bayes and variational Bayes EM are important applications of mean field to Bayesian parameter estimation.

Context

This concept has the prerequisites:

Core resources (read/watch one of the following)

-Paid-

Supplemental resources (the following are optional, but you may find them useful)

-Free-

Graphical models, exponential families, and variational inference (2008)
An in-depth review of exact and approximate inference methods for graphical models.
Authors: Martin J. Wainwright,Michael I. Jordan
Additional dependencies:
  • Gaussian MRFs
  • convex optimization

-Paid-

See also

  • Mean field updates in an Ising model have a similar form to updates in a Hopfield network