# covariance

(1.4 hours to learn)

## Summary

The covariance of two random variables is a measure of their relatedness. It is closely related to the correlation coefficient, but is more commonly used in probability theory because it has nice mathematical properties.

## Context

This concept has the prerequisites:

- expectation and variance (Covariance is a generalization of variance.)
- multivariate distributions (Covariance is defined in terms of a multivariate PMF or PDF.)
- independent random variables (Independent random variables have zero covariance.)

## Goals

- Know the definitions of covariance and correlation

- Write the covariance in terms of the moments of the distribution

- Know the Cauchy-Schwartz inequality for covariance (which bounds the covariance in terms of the individual variances)

- Be able to compute the variance of a linear combination of random variables in terms of their variances and covariances

- Know that independent random variables have zero covariance

- Show that variance of a sum of independent random variables is a sum of the variances

## Core resources (read/watch one of the following)

## -Free-

→ Mathematical Monk: Probability Primer (2011)

Online videos on probability theory.

Other notes:

- This uses the measure theoretic notion of probability, but should still be accessible without that background. Refer to Lecture 1.S for unfamiliar terms.

## -Paid-

→ Probability and Statistics

An introductory textbook on probability theory and statistics.

Location:
Section 4.6, "Covariance and correlation," pages 214-221

→ Mathematical Statistics and Data Analysis

An undergraduate statistics textbook.

Location:
Section 4.3, "Covariance and correlation," pages 138-146

→ A First Course in Probability

An introductory probability textbook.

Location:
Section 7.4, "Covariance, variance of sums, and correlations," pages 355-364

→ An Introduction to Probability Theory and its Applications

A classic introductory probability textbook.

- Section 9.5, "Covariance
- variance of a sum," pages 215-219

## See also

- Covariance matrices are a way of representing all of the variances and covariances between a set of random variables.
- Multivariate Gaussians are a widely used "bell-shaped" distribution, parameterized in terms of expectation and covariance.