covariance

(1.4 hours to learn)

Summary

The covariance of two random variables is a measure of their relatedness. It is closely related to the correlation coefficient, but is more commonly used in probability theory because it has nice mathematical properties.

Context

This concept has the prerequisites:

Goals

  • Know the definitions of covariance and correlation
  • Write the covariance in terms of the moments of the distribution
  • Know the Cauchy-Schwartz inequality for covariance (which bounds the covariance in terms of the individual variances)
  • Be able to compute the variance of a linear combination of random variables in terms of their variances and covariances
  • Know that independent random variables have zero covariance
  • Show that variance of a sum of independent random variables is a sum of the variances

Core resources (read/watch one of the following)

-Free-

Mathematical Monk: Probability Primer (2011)
Online videos on probability theory.
Other notes:
  • This uses the measure theoretic notion of probability, but should still be accessible without that background. Refer to Lecture 1.S for unfamiliar terms.

-Paid-

See also

  • Covariance matrices are a way of representing all of the variances and covariances between a set of random variables.
  • Multivariate Gaussians are a widely used "bell-shaped" distribution, parameterized in terms of expectation and covariance.