expectation and variance
(3.7 hours to learn)
Summary
The expectation of a random variable is the value that it takes "on average," and the variance is a measure of how much the random variable deviates from that value "on average." Expectation and variance have several convenient properties that often allow one to abstract away the underlying PDFs or PMFs.
Context
This concept has the prerequisites:
- random variables (Expected value is a property of random variables.)
Core resources (read/watch one of the following)
-Free-
→ Sets, Counting, and Probability
→ Mathematical Monk: Probability Primer (2011)
Online videos on probability theory.
Other notes:
- This uses the measure theoretic notion of probability, but should still be accessible without that background. Refer to Lecture 1.S for unfamiliar terms.
-Paid-
→ A First Course in Probability
An introductory probability textbook.
- Section 4.3, "Expected value," pages 140-143
- Section 4.4, "Expectation of a function of a random varible," pages 144-148
- Section 4.5, "Variance," pages 148-150
- Section 5.2, "Expectation and variance of continuous random variables," pages 209-214
- Section 7.1, "Introduction," pages 327-328
- Section 7.2, "Expectation of sums of random variables," not counting 7.2.1 and 7.2.2, pages 328-342
Other notes:
- Don't worry about the proofs as far as this node is concerned. See also Proposition 4.1 of Section 7.4, which says that expectations multiply for independent random variables.
→ Probability and Statistics
An introductory textbook on probability theory and statistics.
- Section 4.1, "The expectation of a random variable," pages 181-188
- Section 4.2, "Properties of expectations," pages 189-196
- Section 4.3, "Variance," pages 197-202
Other notes:
- Don't worry about the proofs as far as this node is concerned.
→ Mathematical Statistics and Data Analysis
An undergraduate statistics textbook.
- Section 4.1, "The expected value of a random variable," pages 116-130
- Section 4.2, "Variance and standard deviation," pages 130-137
→ An Introduction to Probability Theory and its Applications
A classic introductory probability textbook.
- Section 9.2, "Expectations," pages 207-209
- Section 9.3, "Examples and applications," pages 209-213
- Section 9.4, "The variance," pages 213-215
Other notes:
- This reference only talks about discrete random variables. All the theorems hold for continuous variables as well. For the definitions, just replace sums with integrals.
See also
- Some other statistics which are based on expected value:
- variance, which reflects how much a random variable typically deviates from its expected value
- moment generating functions, a mathematical representation convenient for analyzing sums of random variables (go to concept)
- Martingales are a kind of sequence of random variables whose expected value at the next time step is the current value.
- Monte Carlo techniques are a way of estimating expectations by sampling from a distribution. (go to concept)