(2 hours to learn)
The Gaussian (or normal) distribution has a bell shape, and is one of the most common in all of statistics. The Central Limit Theorem shows that sums of large numbers of independent, identically distributed random variables are well approximated by a Gaussian distribution. The parameter estimates in a statistical model are also asymptotically Gaussian. Gaussians are widely used in probabilistic modeling for these reasons, together with the fact that Gaussian distributions can be efficiently manipulated using the techniques of linear algebra.
This concept has the prerequisites:
Core resources (read/watch one of the following)
→ Probability and Statistics
An introductory textbook on probability theory and statistics.
Location: Section 5.6, "The normal distribution," pages 268-280
- The parts about computing the normalizing constant and the MGF are optional.
→ A First Course in Probability
An introductory probability textbook.
Location: Section 5.4, "Normal random variables," pages 218-230
- The part about computing the normalizing constant is optional.
Supplemental resources (the following are optional, but you may find them useful)
→ Mathematical Statistics and Data Analysis
An undergraduate statistics textbook.
Location: Section 2.2.3, "The normal distribution," pages 54-58
- Multivariate Gaussians are a generalization of Gaussian distributions to multiple variables.
- We see Gaussian distributions in a wide variety of situations:
- By the central limit theorem , sums of large numbers of independent random variables are approximately Gaussian
- In statistical inference, the variance of an estimator often approaches a Gaussian as more data points are observed
- Gaussians are widely used in computer science because they allow computationally efficient methods for accounting for dependencies between different variables.
- Probability distributions related to the Gaussian distribution:
- The gamma distribution is often used to construct a [prior for the variance parameter](bayesian_parameter_estimation_gaussian) .
- The sum of squares of Gaussian random variables follows a chi-squared distribtuion .
- How to sample from a Gaussian distribution
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation