generalized linear models

(2.1 hours to learn)

Summary

Generalized linear models are a class of linear models which unify several widely used models, including linear regression and logistic regression. The distribution over each output is assumed to be an exponential family distribution whose natural parameters are a linear function of the inputs.

Context

This concept has the prerequisites:

Goals

  • Understand the basic assumptions behind a generalized linear model.
    • In particular, the natural parameters are assumed to be a linear function of the inputs. What role does this play in the maximum likelihood update rule?
  • Show that some other model (e.g. linear regression or logistic regression) is a special case of generalized linear models.
  • Derive the gradient ascent update rule for GLMs.

Core resources (read/watch one of the following)

-Free-

Stanford's Machine Learning lecture notes
Lecture notes for Stanford's machine learning course, aimed at graduate and advanced undergraduate students.
Author: Andrew Y. Ng
Other notes:
  • Section 5 explains logistic regression.

Supplemental resources (the following are optional, but you may find them useful)

-Paid-

See also