generalized linear models
(2.1 hours to learn)
Generalized linear models are a class of linear models which unify several widely used models, including linear regression and logistic regression. The distribution over each output is assumed to be an exponential family distribution whose natural parameters are a linear function of the inputs.
This concept has the prerequisites:
- Understand the basic assumptions behind a generalized linear model.
- In particular, the natural parameters are assumed to be a linear function of the inputs. What role does this play in the maximum likelihood update rule?
- Show that some other model (e.g. linear regression or logistic regression) is a special case of generalized linear models.
- Derive the gradient ascent update rule for GLMs.
Core resources (read/watch one of the following)
→ Stanford's Machine Learning lecture notes
Lecture notes for Stanford's machine learning course, aimed at graduate and advanced undergraduate students.
Location: Lecture 1, "Supervised learning, discriminative algorithms," part 9, "Constructing GLMs," pages 24-30
- Section 5 explains logistic regression.
Supplemental resources (the following are optional, but you may find them useful)
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location: Section 4.3.6, "Canonical link functions," pages 212-213
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location: Section 9.3, "Generalized linear models (GLMs)," pages 290-293
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation