loss function
(1.5 hours to learn)
Summary
A loss function or cost function is a function that maps the outcome of a decision to a real-valued cost associated with that outcome. Loss functions are common in machine learning, information theory, statistics, and mathematical optimization, and help guide decision making under uncertainty.
Context
-this concept has no prerequisites-
Core resources (read/watch one of the following)
-Free-
→ Part II Decision Theory Lecture Notes
Location:
pages 40-45
Other notes:
- working though the exercises is very helpful but not essential
Supplemental resources (the following are optional, but you may find them useful)
-Free-
→ Wikipedia
Location:
Article: Loss Function
Other notes:
- read the "Introduction" and "Use in Statistics" sections
-Paid-
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location:
Section 1.5, "Decision Theory," pages 38-39
See also
- In Bayesian decision theory we perform inference to minimize the [posterior expected loss](posterior_expected_loss) using various loss functions. Some important results are:
- minimizing a zero-one loss function yields a [Maximum A Posteriori (MAP) parameter estimation](map_parameter_estimation)
- minimizing the squared error loss function yields the posterior mean parameter estimation
- minimizing the L1 loss function function yields a posterior median parameter estimation