(1.4 hours to learn)
The covariance of two random variables is a measure of their relatedness. It is closely related to the correlation coefficient, but is more commonly used in probability theory because it has nice mathematical properties.
This concept has the prerequisites:
- expectation and variance (Covariance is a generalization of variance.)
- multivariate distributions (Covariance is defined in terms of a multivariate PMF or PDF.)
- independent random variables (Independent random variables have zero covariance.)
- Know the definitions of covariance and correlation
- Write the covariance in terms of the moments of the distribution
- Know the Cauchy-Schwartz inequality for covariance (which bounds the covariance in terms of the individual variances)
- Be able to compute the variance of a linear combination of random variables in terms of their variances and covariances
- Know that independent random variables have zero covariance
- Show that variance of a sum of independent random variables is a sum of the variances
Core resources (read/watch one of the following)
→ Mathematical Monk: Probability Primer (2011)
Online videos on probability theory.
- This uses the measure theoretic notion of probability, but should still be accessible without that background. Refer to Lecture 1.S for unfamiliar terms.
→ Probability and Statistics
An introductory textbook on probability theory and statistics.
Location: Section 4.6, "Covariance and correlation," pages 214-221
→ Mathematical Statistics and Data Analysis
An undergraduate statistics textbook.
Location: Section 4.3, "Covariance and correlation," pages 138-146
→ A First Course in Probability
An introductory probability textbook.
Location: Section 7.4, "Covariance, variance of sums, and correlations," pages 355-364
→ An Introduction to Probability Theory and its Applications
A classic introductory probability textbook.
- Section 9.5, "Covariance
- variance of a sum," pages 215-219
- Covariance matrices are a way of representing all of the variances and covariances between a set of random variables.
- Multivariate Gaussians are a widely used "bell-shaped" distribution, parameterized in terms of expectation and covariance.
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation