binary linear classifiers
(45 minutes to learn)
Summary
A linear classifier makes a classification decision for a given observation based on the value of a linear combination of the observation's features. In a ``binary'' linear classifier, the observation is classified into one of two possible classes using a linear boundary in the input feature space.
Context
This concept has the prerequisites:
- linear regression (Linear regression provides useful intuitions for thinking about binary linear classification.)
Core resources (read/watch one of the following)
-Free-
→ Optimization Models and Applications
Supplemental resources (the following are optional, but you may find them useful)
-Paid-
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location:
Section 4-intro - 4.1.1, pages 179-182
See also
- Here are some commonly used binary classification algorithms:
- perceptron
- logistic regression
- support vector machines (SVMs)
- Gaussian discriminant analysis
- naive Bayes
- real-valued
- categorical
- ordinal (i.e. only the ranking matters)
- Some general techniques for improving generalization include:
- regularization , where overly complex solutions are penalized
- model selection
- feature selection