# AdaBoost

(1.1 hours to learn)

## Summary

AdaBoost is an example of a boosting algorithm, where the goal is to take a "weak classifier" (one which performs slightly above chance) and make it into a "strong classifier" (one which performs well on the training set). It is widely used in data mining, especially in conjunction with decision trees, because of its simplicity and effectiveness.

## Context

-this concept has no prerequisites-

## Goals

- Understand the steps in the AdaBoost algorithm

- Be aware of the underlying motivation, namely taking a "weak classifier" which performs slightly above chance, and producing a "strong classifier," which classifies the whole training set correctly.

## Core resources (read/watch one of the following)

## -Free-

→ The Elements of Statistical Learning

A graudate-level statistical learning textbook with a focus on frequentist methods.

Other notes:

- Read the introductory chapters if you're not familiar with the basic machine learning setup.

→ Coursera: Machine Learning

An online machine learning course aimed at advanced undergraduates.

Other notes:

- Watch the Week One lectures if you're not familiar with the basic machine learning setup.
- Click on "Preview" to see the videos.

## -Paid-

→ Artificial Intelligence: a Modern Approach

A textbook giving a broad overview of all of AI.

Location:
Section 18.4, "Ensemble learning," pages 664-668

Other notes:

- Read 18.1-18.2 if you're not familiar with the basic machine learning setup, and skim 18.3 to learn about decision trees.

## Supplemental resources (the following are optional, but you may find them useful)

## -Paid-

→ Pattern Recognition and Machine Learning

A textbook for a graduate machine learning course, with a focus on Bayesian methods.

Location:
Section 14.3, "Boosting," not including 14.3.1, pages 657-659

## See also

- AdaBoost can be interpreted as a greedy algorithm to minimize exponential loss .
- Decision trees are often used as the base classifier.