multivariate Gaussian distribution
(1.6 hours to learn)
Summary
The multivariate Gaussian distribution is a generalization of the Gaussian distribution to higher dimensions. The parameters of an n-dimension multivariate Gaussian distribution are an n-dimensional mean vector and an n-by-n dimensional covariance matrix.
Context
This concept has the prerequisites:
- Gaussian distribution
- covariance (The parameterization of a multivariate Gaussian includes a covariance matrix.)
- covariance matrices (The parameterization of a multivariate Gaussian includes a covariance matrix.)
- determinant (The determinant of the covariance matrix is part of the normalizing constant.)
- matrix inverse (The inverse of the covariance matrix is part of the PDF.)
Core resources (read/watch one of the following)
-Free-
→ Stanford's Machine Learning lecture notes
Lecture notes for Stanford's machine learning course, aimed at graduate and advanced undergraduate students.
Location:
Lecture 2, "Generative learning algorithms," Section 1.1, "The multivariate normal distribution"
-Paid-
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location:
Section 2.3, "The Gaussian distribution," not including subsections, pages 78-85
Other notes:
- Don't worry about the discussion of eigenvalues and eigenvectors if you're not familiar with those.
Supplemental resources (the following are optional, but you may find them useful)
-Free-
→ Wikipedia
-Paid-
→ Probabilistic Graphical Models: Principles and Techniques
A very comprehensive textbook for a graduate-level course on probabilistic AI.
Location:
Section 7.1.1, "Basic parameterization," pages 247-249
See also
- Some reasons that the multivariate Gaussian distribution is often seen in nature:
- the central limit theorem says that sums of large numbers of independent random variables are approximately Gaussian
- the variance of many statistical estimators approaches a Gaussian as more data points are observed
- The parameters of a multivariate Gaussian can be estimated from data using: Some statistical models which use multivariate Gaussians:
- mixture of Gaussians models, which are used to cluster data points
- Bayesian linear regression
- linear dynamical systems , which model how a system changes over time