generalized linear models
(2.1 hours to learn)
Generalized linear models are a class of linear models which unify several widely used models, including linear regression and logistic regression. The distribution over each output is assumed to be an exponential family distribution whose natural parameters are a linear function of the inputs.
This concept has the prerequisites:
- exponential families (Generalized linear models are defined in terms of exponential family distributions.)
- maximum likelihood (Generalized linear models can be fit using maximum likelihood.)
- optimization problems (Maximum likelihood is an optimization problem.)
- linear regression as maximum likelihood (Linear regression is an instructive example of a generalized linear model.)
- gradient descent (The gradient ascent update rule for generalized linear models has a convenient form.)
- Understand the basic assumptions behind a generalized linear model.
- In particular, the natural parameters are assumed to be a linear function of the inputs. What role does this play in the maximum likelihood update rule?
- Show that some other model (e.g. linear regression or logistic regression) is a special case of generalized linear models.
- Derive the gradient ascent update rule for GLMs.
Core resources (read/watch one of the following)
→ Stanford's Machine Learning lecture notes
Lecture notes for Stanford's machine learning course, aimed at graduate and advanced undergraduate students.
Location: Lecture 1, "Supervised learning, discriminative algorithms," part 9, "Constructing GLMs," pages 24-30
- Section 5 explains logistic regression.
Supplemental resources (the following are optional, but you may find them useful)
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location: Section 4.3.6, "Canonical link functions," pages 212-213
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location: Section 9.3, "Generalized linear models (GLMs)," pages 290-293
- Important example of generalized linear models include:Exponential family PCA is a class of matrix factorization models based on the same ideas as generalized linear models.
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation