Bayesian linear regression
(2 hours to learn)
Summary
By interpreting linear regression as a Bayesian model, we can automatically infer the prior variance and the noise variance, and make calibrated predictions. Bayesian linear regression is a useful component in fancier probabilistic models.
Context
This concept has the prerequisites:
- linear regression as maximum likelihood (Bayesian linear regression is based on the probabilistic interpretation of linear regression.)
- Bayesian parameter estimation (This is an example of Bayesian parameter estimation.)
- Bayesian parameter estimation: multivariate Gaussians (Bayesian linear regression uses some of the same computations as Bayesian inference for multivariate Gaussians.)
- computations on multivariate Gaussians (Bayesian linear regression requires conditioning and marginalizing multivariate Gaussians.)
Goals
- Know the form of the Bayesian linear regression model
- Visualize the prior, evidence, and posterior
- Derive the predictive distribution
- Visualize the posterior predictive distribution
- Be able to infer the variance parameters (with the evidence approximation or a conjugate prior)
Core resources (read/watch one of the following)
-Paid-
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location:
Section 3.3, pages 152-161
Additional dependencies:
- the evidence approximation
Supplemental resources (the following are optional, but you may find them useful)
-Paid-
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location:
Sections 7.6-7.6.2, pages 231-234
See also
- Gaussian process regression is a nonparametric analogue of Bayesian linear regression which uses kernels.