multivariate Gaussian distribution
(1.6 hours to learn)
The multivariate Gaussian distribution is a generalization of the Gaussian distribution to higher dimensions. The parameters of an n-dimension multivariate Gaussian distribution are an n-dimensional mean vector and an n-by-n dimensional covariance matrix.
This concept has the prerequisites:
- Gaussian distribution
- covariance (The parameterization of a multivariate Gaussian includes a covariance matrix.)
- covariance matrices (The parameterization of a multivariate Gaussian includes a covariance matrix.)
- determinant (The determinant of the covariance matrix is part of the normalizing constant.)
- matrix inverse (The inverse of the covariance matrix is part of the PDF.)
Core resources (read/watch one of the following)
→ Stanford's Machine Learning lecture notes
Lecture notes for Stanford's machine learning course, aimed at graduate and advanced undergraduate students.
Location: Lecture 2, "Generative learning algorithms," Section 1.1, "The multivariate normal distribution"
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location: Section 2.3, "The Gaussian distribution," not including subsections, pages 78-85
- Don't worry about the discussion of eigenvalues and eigenvectors if you're not familiar with those.
Supplemental resources (the following are optional, but you may find them useful)
→ Probabilistic Graphical Models: Principles and Techniques
A very comprehensive textbook for a graduate-level course on probabilistic AI.
Location: Section 7.1.1, "Basic parameterization," pages 247-249
- Some reasons that the multivariate Gaussian distribution is often seen in nature:
- the central limit theorem says that sums of large numbers of independent random variables are approximately Gaussian
- the variance of many statistical estimators approaches a Gaussian as more data points are observed
- The parameters of a multivariate Gaussian can be estimated from data using:
- mixture of Gaussians models, which are used to cluster data points
- Bayesian linear regression
- linear dynamical systems , which model how a system changes over time
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation