Fishers exact test determines whether a statistically significant association exists between two categorical variables.
For example, does a relationship exist between gender (Male/Female) and voting Yes or No on a referendum?
Typically, you’ll display data for Fisher’s exact test in a two-way contingency table. Frequently, this analysis assesses 2X2 contingency tables, but there are extensions for two-way tables with any number of rows and columns.
In this post, learn about Fisher’s exact test, when to use it, and how to interpret it using an example. I include a calculator so you can apply what you learn.
When to Use Fishers Exact Test vs Chi Square
When reading the description above, you might have thought that Fishers exact test sounds like the Chi-Square Test of Independence. And you’re right! They both serve the same purpose—assessing a relationship between categorical variables.
However, differences in the underlying methodology affect when you should use each method.
The Chi-Square Test of Independence is a more traditional hypothesis test that uses a test statistic (chi-square) and its sampling distribution to calculate the p-value. However, the chi-square sampling distribution only approximates the correct distribution, providing better p-values as the cell values in the table increase. Consequently, chi-square p-values are invalid when you have small cell counts. Learn more about the Chi-Square Test of Independence with an Example.
On the other hand, Fisher’s exact test doesn’t use the chi-square statistic and sampling distribution. Instead, it calculates the number of all possible contingency tables with the same row and column totals (i.e., marginal distributions) as the observed table. Then it calculates the probability for the p-value by finding the proportion of possible tables that are more extreme than the observed table. Technically, Fisher’s exact test is appropriate for all sample sizes. However, the number of possible tables grows at an exponential rate and soon becomes unwieldy. Hence, statisticians use this test for smaller sample sizes.
Chi-square is generally best for larger samples and Fisher’s is better for smaller samples. Here are the guidelines for when to use Fisher’s exact test:
- Cell counts are smaller than 20
- A cell has an expected value 5 or less.
- The column or row marginal values are extremely uneven.
Learn more about Test Statistics, Sampling Distributions, and Interpreting P-Values.
How to Interpret Fishers Exact Test
Let’s work through the voting by gender example. Fisher’s exact test will determine whether a statistically significant relationship exists between gender and voting.
As with any hypothesis test, this analysis has a null and alternative hypothesis. For our example, the hypotheses are the following:
- Null (H0): There is no association between gender and voting. They are independent.
- Alternative (HA): A relationship between gender and voting exists in the population.
When your p-value is below your significance level (e.g., 0.05), reject the null hypothesis. The sample data is strong enough to conclude that a relationship between the categorical variables exists in the population. Knowing the value of one variable provides information about the value of the other variable.
Related posts: Null Hypothesis and Significance Level.
Example Interpretation
The following contingency table displays our example data for Fisher’s exact test.
| Male | Female | |
| Yes | 4 | 9 |
| No | 10 | 3 |
In the table, it appears that females are more likely to vote Yes, while males are more likely to vote No on the referendum issue. However, the apparent relationship in the sample data might be random sampling error rather than a real correlation. Let’s perform the analysis!
The cell counts are too small for the chi-square analysis. Consequently, we’ll use Fisher’s exact test to determine whether this relationship is statistically significant.
We’ll use a Fisher’s exact test calculator to obtain the p-value.
Enter the following values for each letter field in the calculator and choose two-tailed in Test type:
- A: 4
- B: 9
- C: 10
- D: 3
The calculator calculates a p-value of 0.047 for the Fisher’s exact test, which is less than our significance level of 0.05. Our results are statistically significant. We can reject the null and conclude that a relationship exists between gender and voting choice.
Comments and Questions