naive Bayes
(55 minutes to learn)
Summary
Naive Bayes is a modeling assumption used in classification, where we assume the observed data are conditionally independent given their class assignments. Despite its name, the standard naive Bayes model does not use Bayesian inference, but rather, a maximum likelihood estimation.
Context
This concept has the prerequisites:
- binary linear classifiers (Naive Bayes is a linear classifier.)
- maximum likelihood (Naive Bayes is fit using maximum likelihood.)
- optimization problems (Finding the maximum likelihood solution requires solving an optimization problem.)
Core resources (read/watch one of the following)
-Free-
→ Coursera: Machine Learning
An online machine learning course aimed at advanced undergraduates.
Location:
Lecture "The naive Bayes classifier"
Other notes:
- Click on "Preview" to see the videos.
→ Stanford's Machine Learning lecture notes
Lecture notes for Stanford's machine learning course, aimed at graduate and advanced undergraduate students.
Location:
Section 2 (Naive Bayes) of Part IV notes (Generative Learning algorithms) in Lecture Notes 2
→ The Elements of Statistical Learning
A graudate-level statistical learning textbook with a focus on frequentist methods.
→ Bayesian Reasoning and Machine Learning
A textbook for a graudate machine learning course.
Supplemental resources (the following are optional, but you may find them useful)
-Free-
→ Mathematical Monk Tutorials
-Paid-
→ Pattern Recognition and Machine Learning
A textbook for a graduate machine learning course, with a focus on Bayesian methods.
Location:
page 380-381
See also
- Naive Bayes is a generative model .
- Rather than performing a maximum likelihood estimation of the Naive Bayes model, we can use Bayesian inference to obtain posterior beliefs for the classes given the observed data.
- Here are some other commonly used classification algorithms: