Gaussian distribution

(2 hours to learn)

Summary

The Gaussian (or normal) distribution has a bell shape, and is one of the most common in all of statistics. The Central Limit Theorem shows that sums of large numbers of independent, identically distributed random variables are well approximated by a Gaussian distribution. The parameter estimates in a statistical model are also asymptotically Gaussian. Gaussians are widely used in probabilistic modeling for these reasons, together with the fact that Gaussian distributions can be efficiently manipulated using the techniques of linear algebra.

Context

This concept has the prerequisites:

Core resources (read/watch one of the following)

-Paid-

Supplemental resources (the following are optional, but you may find them useful)

-Paid-

See also

  • Multivariate Gaussians are a generalization of Gaussian distributions to multiple variables.
  • We see Gaussian distributions in a wide variety of situations:
    • By the central limit theorem , sums of large numbers of independent random variables are approximately Gaussian
    • In statistical inference, the variance of an estimator often approaches a Gaussian as more data points are observed
    Gaussian distributions have weak, tails, in that they are unlikely to take extreme values; many phenomena are better modeled by heavy-tailed distributions .
  • Gaussians are widely used in computer science because they allow computationally efficient methods for accounting for dependencies between different variables.
  • Probability distributions related to the Gaussian distribution:
    • The gamma distribution is often used to construct a [prior for the variance parameter](bayesian_parameter_estimation_gaussian) .
    • The sum of squares of Gaussian random variables follows a chi-squared distribtuion .
    Information form is an alternative parameterization for Gaussian distributions.
  • How to sample from a Gaussian distribution