# bias-variance decomposition

(55 minutes to learn)

## Summary

The bias-variance decomposition (often referred to as the bias-variance tradeoff) is a frequentist analysis of the generalization capability of an estimator, i.e. a learning algorithm.

## Context

This concept has the prerequisites:

- linear regression (The bias-variance decomposition holds exactly in the case of linear regression.)
- linear regression: closed-form solution (The bias-variance decomposition can be derived from the closed-form solution to linear regression.)
- generalization (The bias-variance decomposition is a way of analyzing generalization error.)

## Core resources (read/watch one of the following)

## -Paid-

→ Pattern Recognition and Machine Learning

A textbook for a graduate machine learning course, with a focus on Bayesian methods.

Location:
Section 3.2

## Supplemental resources (the following are optional, but you may find them useful)

## -Free-

→ Mathematical Monk: Machine Learning (2011)

Online videos on machine learning.

→ The Elements of Statistical Learning

A graudate-level statistical learning textbook with a focus on frequentist methods.

## -Paid-

→ Machine Learning: a Probabilistic Perspective

A very comprehensive graudate-level machine learning textbook.

Location:
Section 6.4.4

Other notes:

- not self-contained but provides some nice examples

## See also

- The variance term depends on the effective number of parameters of a model.
- Statistical learning theory gives a way of analyzing bias and variance of estimators more generally.
- Cross-validation is a simple and general technique for managing the bias-variance tradeoff. [ (go to concept)](cross_validation)