# statistical hypothesis testing

(2.4 hours to learn)

## Summary

Statistical hypothesis testing is a method for deciding what conclusions can be drawn from data. A central question is determining whether an outcome is statistically significant, or unlikely to have arisen by chance.

## Context

This concept has the prerequisites:

- probability
- random variables (Many hypothesis tests concern parameters of distributions.)
- Gaussian distribution (Some canonical examples of statistical tests involve the Gaussian distribution.)
- cumulative distribution function (P-values are determined using the CDF of the test statistic.)

## Goals

- Understand what is required of a hypothesis test in the Neyman-Pearson paradigm

- Know basic terminology, including:
- null hypothesis and alternative hypothesis
- test statistic
- type 1 and type 2 errros
- the power function of a test
- the level of a test
- simple and composite hypotheses

- Know what is meant by a p-value and how to compute it from the test statistic and its distribution
- In particular, understand why it doesn't give the probability that a hypothesis is true

- Understand why statistical significance doesn't imply that a difference is large in magnitude

## Core resources (read/watch one of the following)

## -Paid-

→ Probability and Statistics

An introductory textbook on probability theory and statistics.

Location:
Section 9.1, "Problems of testing hypotheses," pages 530-547

→ All of Statistics

A very concise introductory statistics textbook.

- Chapter 10 introduction, pages 149-152
- Section 10.2, "p-values," pages 156-159

Other notes:

- Skim Section 10.1 to learn about the Wald test.

→ Mathematical Statistics and Data Analysis

An undergraduate statistics textbook.

Location:
Section 9.2, "The Neyman-Pearson paradigm," pages 331-336

## See also

- Some examples of statistical hypothesis testing: Often, instead of hypothesis testing, it is preferable to compute a confidence interval .