# Bayesian information criterion

(1.2 hours to learn)

## Summary

The Bayesian information criterion (BIC) is a rough approximation to the marginal likelihood, based on the asymptotic behavior of the Laplace approximation as more data is observed.

## Context

This concept has the prerequisites:

- Bayesian model comparison (The BIC is an approximation to Bayesian model comparison.)
- the Laplace approximation (The Laplace approximation is a way of justifying the BIC.)

## Goals

- Know the formula for the BIC

- Derive the formula in terms of the Laplace approximation

## Core resources (read/watch one of the following)

## -Free-

→ The Bayesian Information Criterion

## Supplemental resources (the following are optional, but you may find them useful)

## -Paid-

→ Pattern Recognition and Machine Learning

A textbook for a graduate machine learning course, with a focus on Bayesian methods.

Location:
Section 4.4.1, pages 216-217

→ Machine Learning: a Probabilistic Perspective

A very comprehensive graudate-level machine learning textbook.

Location:
Sections 5.3.2.4 (pages 161-162) and 8.4.2 (pages 255-256)

## See also

- Akaike Information Criterion (AIC) is a different model selection criterion with different theoretical underpinnings, and practically, AIC does not penalize the number of parameters as severely as BIC
- Mathematical justification of the BIC.