# Bayesian linear regression

(2 hours to learn)

## Summary

By interpreting linear regression as a Bayesian model, we can automatically infer the prior variance and the noise variance, and make calibrated predictions. Bayesian linear regression is a useful component in fancier probabilistic models.

## Context

This concept has the prerequisites:

- linear regression as maximum likelihood (Bayesian linear regression is based on the probabilistic interpretation of linear regression.)
- Bayesian parameter estimation (This is an example of Bayesian parameter estimation.)
- Bayesian parameter estimation: multivariate Gaussians (Bayesian linear regression uses some of the same computations as Bayesian inference for multivariate Gaussians.)
- computations on multivariate Gaussians (Bayesian linear regression requires conditioning and marginalizing multivariate Gaussians.)

## Goals

- Know the form of the Bayesian linear regression model

- Visualize the prior, evidence, and posterior

- Derive the predictive distribution

- Visualize the posterior predictive distribution

- Be able to infer the variance parameters (with the evidence approximation or a conjugate prior)

## Core resources (read/watch one of the following)

## -Paid-

→ Pattern Recognition and Machine Learning

A textbook for a graduate machine learning course, with a focus on Bayesian methods.

Location:
Section 3.3, pages 152-161

Additional dependencies:

- the evidence approximation

## Supplemental resources (the following are optional, but you may find them useful)

## -Paid-

→ Machine Learning: a Probabilistic Perspective

A very comprehensive graudate-level machine learning textbook.

Location:
Sections 7.6-7.6.2, pages 231-234

## See also

- Gaussian process regression is a nonparametric analogue of Bayesian linear regression which uses kernels.