Gaussian discriminant analysis

(1.5 hours to learn)

Summary

Gaussian discriminant analysis (GDA) is a generative model for classification where the distribution of each class is modeled as a multivariate Gaussian.

Context

This concept has the prerequisites:

Goals

  • Derive the form of the decision boundary in the two-class case when the within-class covariance matrices are shared. In particular, show that it is always a hyperplane.
  • What do the decision regions look like when there are more than two classes?
  • Show that the decision boundary is quadratic when the covariance matrices are not shared between classes.
  • Optional: show that the decision boundary in the two-class case is equivalent to performing classification with linear least squares .

Core resources (read/watch one of the following)

-Free-

The Elements of Statistical Learning
A graudate-level statistical learning textbook with a focus on frequentist methods.
Authors: Trevor Hastie,Robert Tibshirani,Jerome Friedman

-Paid-

Supplemental resources (the following are optional, but you may find them useful)

-Free-

Stanford's Machine Learning lecture notes
Lecture notes for Stanford's machine learning course, aimed at graduate and advanced undergraduate students.
Author: Andrew Y. Ng

See also