Beyond Two Groups
You've seen t-tests for comparing two groups. Inevitably, you'll want to compare three or more. You might be wondering: can't I just run a t-test between every pair of groups?
You can. You absolutely should not. Run a t-test between every pair of, say, five groups, and you've run ten tests. Multiple comparisons problem. False positives compound rapidly. With ten tests at α = 0.05, you have roughly a 40% chance of at least one false positive even when no real differences exist.
ANOVA — Analysis of Variance — lets you ask "are any of these groups different from each other?" with a single test, while controlling Type 1 error appropriately.
ANOVA Tells You 'Some Group Differs' — Not Which One
The alternative hypothesis in ANOVA is: "at least one group mean is significantly different from the others." When you reject the null, you know something is different — not what. You need post hoc tests for that, and they're covered at the end of this chapter.