statistical hypothesis testing
(2.4 hours to learn)
Statistical hypothesis testing is a method for deciding what conclusions can be drawn from data. A central question is determining whether an outcome is statistically significant, or unlikely to have arisen by chance.
This concept has the prerequisites:
- random variables (Many hypothesis tests concern parameters of distributions.)
- Gaussian distribution (Some canonical examples of statistical tests involve the Gaussian distribution.)
- cumulative distribution function (P-values are determined using the CDF of the test statistic.)
- Understand what is required of a hypothesis test in the Neyman-Pearson paradigm
- Know basic terminology, including:
- null hypothesis and alternative hypothesis
- test statistic
- type 1 and type 2 errros
- the power function of a test
- the level of a test
- simple and composite hypotheses
- Know what is meant by a p-value and how to compute it from the test statistic and its distribution
- In particular, understand why it doesn't give the probability that a hypothesis is true
- Understand why statistical significance doesn't imply that a difference is large in magnitude
Core resources (read/watch one of the following)
→ Probability and Statistics
An introductory textbook on probability theory and statistics.
Location: Section 9.1, "Problems of testing hypotheses," pages 530-547
→ All of Statistics
A very concise introductory statistics textbook.
- Chapter 10 introduction, pages 149-152
- Section 10.2, "p-values," pages 156-159
- Skim Section 10.1 to learn about the Wald test.
→ Mathematical Statistics and Data Analysis
An undergraduate statistics textbook.
Location: Section 9.2, "The Neyman-Pearson paradigm," pages 331-336
- Some examples of statistical hypothesis testing:confidence interval .
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation