Bayesian parameter estimation: Gaussian distribution
(1.6 hours to learn)
Summary
Using the Bayesian framework, we can infer the mean parameter of a Gaussian distribution, the scale parameter, or both. Since Gaussians are widely used in probabilistic modeling, the computations that go into this are common motifs in Bayesian machine learning more generally.
Context
This concept has the prerequisites:
- Bayesian parameter estimation
- Gaussian distribution
- gamma distribution (The inverse gamma distribution is the conjugate prior for the scale parameter.)
- Student-t distribution (The predictive distribution, with the scale parameter integrated out, is a student-t distribution.)
Goals
- Derive the conjugate priors for three cases:
- unknown mean, but known variance
- known mean, but unknown variance
- unknown mean and unknown variance
- Derive the posterior distribution and the posterior predictive distribution for each of these cases.
Core resources (read/watch one of the following)
-Free-
→ Information Theory, Inference, and Learning Algorithms
A graudate-level textbook on machine learning and information theory.
-Paid-
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location:
Sections 2.3.6-2.3.7 (except for the part about multivariate Gaussians), pgs. 97-105
See also
-No Additional Notes-