Bayesian parameter estimation: multivariate Gaussians
(1 hours to learn)
Using the Bayesian framework, we can infer the posterior over the mean vector of a multivariate Gaussian, the covariance matrix, or both. Since multivariate Gaussians are widely used in probabilistic modeling, the computations that go into this are common motifs in Bayesian machine learning more generally.
This concept has the prerequisites:
- Derive the conjugate priors for the multivariate distribution in three cases:
- unknown mean, but known covariance
- known mean, but unknown covariance
- unknown mean and unknown covariance
- Derive the posterior distributions for each of these cases.
Core resources (read/watch one of the following)
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location: Sections 4.6-4.6.2, pgs. 127-131
Supplemental resources (the following are optional, but you may find them useful)
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location: Section 2.3.6, pgs. 97-102
- These techniques are used in various models, including:Wishart process allows us to model dependencies between different Wishart-distributed random variables.
- When there's not enough data to estimate a full covariance matrix, here are some related models with more structure: