Central Limit Theorem
(1.4 hours to learn)
Summary
The Central Limit Theorem states that the sum of a large number of independent, identically distributed random variables is approximately Gaussian. It can be used to approximate the probability that a sum of independent random variables lies within some range, even if the distributions are otherwise hard to work with. This theorem is one of the reasons that Gaussian distributions are so ubiquitous in statistics and probabilistic modeling.
Context
This concept has the prerequisites:
- Gaussian distribution (The statement of the theorem involves Gaussian distributions.)
- independent random variables (The theorem is a statement about independent random variables.)
- expectation and variance (The statement of the theorem involves expected value and variances.)
- moment generating functions (Moment generating functions are used to prove the theorem.)
Goals
- Know the statement of the central limit theorem
- Be able to use it to estimate the distribution of a sum of i.i.d. random variables
- Prove the theorem
Core resources (read/watch one of the following)
-Paid-
→ Probability and Statistics
An introductory textbook on probability theory and statistics.
Location:
Section 5.7, "The Central Limit Theorem," pages 282-290
→ A First Course in Probability
An introductory probability textbook.
Location:
Section 8.3, "The Central Limit Theorem," pages 434-443
Supplemental resources (the following are optional, but you may find them useful)
-Free-
→ Mathematical Monk: Probability Primer (2011)
Online videos on probability theory.
→ Stats.Stackexchange Question: What intuitive explanation is there for the central limit theorem?
-Paid-
→ Mathematical Statistics and Data Analysis
An undergraduate statistics textbook.
Location:
Section 5.3, "Convergence in distribution and the Central Limit Theorem," pages 181-188
See also
- Proof of the theorem
- The theorem can be generalized to: Other results about sums of series: