probit regression
(30 minutes to learn)
Summary
Probit regression is a discriminative model for classification. In this model, the binary targets are generated by sampling latent Gaussian variables whose means are linear in the inputs, and passing them through a threshold.
Context
This concept has the prerequisites:
- binary linear classifiers (Probit regression is a binary linear classifier.)
- logistic regression (Probit regression is a variant on logistic regression.)
- probit function (Probit regression is defined in terms of the probit function.)
- gradient descent (Probit regression can be fit using gradient descent.)
Goals
- Know what the probit regression model is
- Be able to derive the gradient descent update rules
- Understand the relationship with logistic regression:
- the two have similar activation functions
- however, probit regression is more sensitive to outliers
- Interpret the model in terms of a latent Gaussian variable and a threshold
Core resources (read/watch one of the following)
-Paid-
→ Machine Learning: a Probabilistic Perspective
A very comprehensive graudate-level machine learning textbook.
Location:
Section 9.4, "Probit regression," pages 293-295
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location:
Section 4.3.5, "Probit regression," pages 210-212
See also
- Probit regression is less robust to outliers than [logistic regression](logistic_regression) .
- Probit functions are convenient in probabilistic models because they allow for sampling algorithms which fit the latent Gaussian variables.