What is the Tukey-Kramer Test?
The Tukey-Kramer test is a post hoc test that analysts use after a one-way ANOVA. It compares all possible pairs of group means to determine which are significantly different. The test adjusts for multiple comparisons to control the overall error rate, making it a preferred method when testing more than two groups. It’s especially useful when your ANOVA shows a statistically significant result, but you want to know which specific groups differ from each other.
This test extends Tukey’s Honestly Significant Difference (HSD) test to allow for unequal sample sizes across groups. The Tukey-Kramer test shares the same assumptions as one-way ANOVA.
The Tukey-Kramer test compares the difference between each pair of means to a critical value that accounts for the number of groups and comparisons. The formula adjusts the confidence intervals and p-values to maintain the familywise error rate at the desired level (usually 0.05). It uses the studentized range distribution to determine significance thresholds.
Use the Tukey-Kramer Test when you:
- Have three or more groups.
- You’ve already run a one-way ANOVA and found a significant difference.
- Want to perform all pairwise comparisons of means.
Example
For example, consider a one-way ANOVA analysis that produces a statistically significant result for four groups. These four groups produce six comparison between all of them. Which ones are different?
The graph below shows Tukey-Kramer Test adjusted confidence intervals for the pairwise differences between group means. It provides a quick visual summary of which comparisons are statistically significant. If a confidence interval crosses zero, it indicates that the difference between those two group means is not significant.
In this chart, only the interval for D – B does not include zero, indicating a statistically significant difference between those two groups. The other group comparisons are not significant.
The Tukey-Kramer method adjusted these CIs by widening them appropriately to control the familywise error rate across all comparisons. This visual approach complements adjusted p-values by also showing the magnitude and direction of the differences, offering deeper insight than adjusted p-values alone.
This test allows the researcher to draw specific conclusions while keeping the overall Type I error rate under control.
« Back to Glossary Index