Gaussian distribution
(2 hours to learn)
Summary
The Gaussian (or normal) distribution has a bell shape, and is one of the most common in all of statistics. The Central Limit Theorem shows that sums of large numbers of independent, identically distributed random variables are well approximated by a Gaussian distribution. The parameter estimates in a statistical model are also asymptotically Gaussian. Gaussians are widely used in probabilistic modeling for these reasons, together with the fact that Gaussian distributions can be efficiently manipulated using the techniques of linear algebra.
Context
This concept has the prerequisites:
- random variables
- expectation and variance (The Gaussian distribution is parameterized in terms of expected value and variance.)
Core resources (read/watch one of the following)
-Paid-
→ Probability and Statistics
An introductory textbook on probability theory and statistics.
Location:
Section 5.6, "The normal distribution," pages 268-280
Other notes:
- The parts about computing the normalizing constant and the MGF are optional.
→ A First Course in Probability
An introductory probability textbook.
Location:
Section 5.4, "Normal random variables," pages 218-230
Other notes:
- The part about computing the normalizing constant is optional.
Supplemental resources (the following are optional, but you may find them useful)
-Paid-
→ Mathematical Statistics and Data Analysis
An undergraduate statistics textbook.
Location:
Section 2.2.3, "The normal distribution," pages 54-58
See also
- Multivariate Gaussians are a generalization of Gaussian distributions to multiple variables.
- We see Gaussian distributions in a wide variety of situations:
- By the central limit theorem , sums of large numbers of independent random variables are approximately Gaussian
- In statistical inference, the variance of an estimator often approaches a Gaussian as more data points are observed
- Gaussians are widely used in computer science because they allow computationally efficient methods for accounting for dependencies between different variables.
- Probability distributions related to the Gaussian distribution:
- The gamma distribution is often used to construct a [prior for the variance parameter](bayesian_parameter_estimation_gaussian) .
- The sum of squares of Gaussian random variables follows a chi-squared distribtuion .
- How to sample from a Gaussian distribution