(55 minutes to learn)
The bias-variance decomposition (often referred to as the bias-variance tradeoff) is a frequentist analysis of the generalization capability of an estimator, i.e. a learning algorithm.
This concept has the prerequisites:
- linear regression (The bias-variance decomposition holds exactly in the case of linear regression.)
- linear regression: closed-form solution (The bias-variance decomposition can be derived from the closed-form solution to linear regression.)
- generalization (The bias-variance decomposition is a way of analyzing generalization error.)
Core resources (read/watch one of the following)
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location: Section 3.2
Supplemental resources (the following are optional, but you may find them useful)
→ Mathematical Monk: Machine Learning (2011)
Online videos on machine learning.
→ The Elements of Statistical Learning
A graudate-level statistical learning textbook with a focus on frequentist methods.
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location: Section 6.4.4
- not self-contained but provides some nice examples
- The variance term depends on the effective number of parameters of a model.
- Statistical learning theory gives a way of analyzing bias and variance of estimators more generally.
- Cross-validation is a simple and general technique for managing the bias-variance tradeoff. [ (go to concept)](cross_validation)
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation