variational Bayes
(1.8 hours to learn)
Summary
Bayesian parameter estimation often results in an intractable posterior over model parameters. Variational Bayes is an application of variational inference (in particular, mean field) to approximating the marginals over parameters as well as the marginal likelihood.
Context
This concept has the prerequisites:
- Bayesian parameter estimation (Variational Bayes is a way of approximating Bayesian parameter estimation.)
- Bayesian model comparison (Variational Bayes is a way of approximating Bayesian model comparison.)
- mean field approximation (Variational Bayes is an application of the mean field approximation.)
Core resources (read/watch one of the following)
-Paid-
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location:
Section 21.5, pages 742-749
Additional dependencies:
- Bayesian parameter estimation: Gaussian distribution
- Bayesian linear regression
Supplemental resources (the following are optional, but you may find them useful)
-Paid-
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location:
Sections 10.1.3-10.1.4, pages 470-474
Additional dependencies:
- Bayesian parameter estimation: Gaussian distribution
See also
- Some examples of variational Bayes: Variational Bayes is especially appropriate for fitting exponential families .
- Other approximations to marginal likelihood include: