# fitting logistic regression with iterative reweighted least squares

(30 minutes to learn)

## Summary

One way of fitting logistic regression is using Newton's method. This winds up having an intuitive form, where each update takes the form of a linear regression problem and the data points are all assigned weights depending how far they are from the decision boundary.

## Context

This concept has the prerequisites:

- logistic regression
- linear regression: closed-form solution (The IRLS updates have the same form as the linear regression solution, but with reweighted data points.)

## Core resources (read/watch one of the following)

## -Paid-

→ Pattern Recognition and Machine Learning

A textbook for a graduate machine learning course, with a focus on Bayesian methods.

Location:
Section 4.3.3, pages 207-208

→ Machine Learning: a Probabilistic Perspective

A very comprehensive graudate-level machine learning textbook.

Location:
Sections 8.3.3-8.3.4, pages 249-251

## Supplemental resources (the following are optional, but you may find them useful)

## -Free-

→ Stanford's Machine Learning lecture notes

Lecture notes for Stanford's machine learning course, aimed at graduate and advanced undergraduate students.

Location:
Chapter 1, section 7, pages 20-21

## See also

- While Newton's method is very fast for small scale problems, it doesn't scale very well.
- For large numbers of features, consider quasi-Newton methods .
- For large numbers of data points, consider stochastic gradient descent .