Fisher’s exact test determines whether a statistically significant association exists between two categorical variables. You can also use it for a 2-sample proportion test when you have a small sample size.
For example, does a relationship exist between gender (Male/Female) and voting Yes or No on a referendum?
Typically, you’ll display data for Fisher’s exact test in a two-way contingency table. Frequently, this analysis assesses 2X2 contingency tables, but there are extensions for two-way tables with any number of rows and columns.
In this post, learn about Fisher’s exact test, when to use it, and how to interpret it using an example. I include a calculator so you can apply what you learn.
When to Use Fishers Exact Test vs Chi Square
When reading the description above, you might have thought that Fisher’s exact test sounds like the Chi-Square Test of Independence. And you’re right! They both serve the same purpose—assessing a relationship between categorical variables.
However, differences in the underlying methodology affect when you should use each method.
The Chi-Square Test of Independence is a more traditional hypothesis test that uses a test statistic (chi-square) and its sampling distribution to calculate the p-value. However, the chi-square sampling distribution only approximates the correct distribution, providing better p-values as the cell values in the table increase. Consequently, chi-square p-values are invalid when you have small cell counts. Learn more about the Chi-Square Test of Independence with an Example.
On the other hand, Fisher’s exact test doesn’t use the chi-square statistic and sampling distribution. Instead, it calculates the number of all possible contingency tables with the same row and column totals (i.e., marginal distributions) as the observed table. Then it calculates the probability for the p-value by finding the proportion of possible tables that are more extreme than the observed table. Technically, Fisher’s exact test is appropriate for all sample sizes. However, the number of possible tables grows at an exponential rate and soon becomes unwieldy. Hence, statisticians use this test for smaller sample sizes.
Chi-square is generally best for larger samples and Fisher’s is better for smaller samples. Here are the guidelines for when to use Fisher’s exact test:
- Cell counts are smaller than 20
- A cell has an expected value 5 or less.
- The column or row marginal values are extremely uneven.
Learn more about Test Statistics, Sampling Distributions, and Interpreting P-Values.
How to Interpret Fisher’s Exact Test
Let’s work through the voting by gender example. Fisher’s exact test will determine whether a statistically significant relationship exists between gender and voting.
As with any hypothesis test, this analysis has a null and alternative hypothesis. For our example, the hypotheses are the following:
- Null (H0): There is no association between gender and voting. They are independent.
- Alternative (HA): A relationship between gender and voting exists in the population.
When your p-value is below your significance level (e.g., 0.05), reject the null hypothesis. The sample data is strong enough to conclude that a relationship between the categorical variables exists in the population. Knowing the value of one variable provides information about the value of the other variable.
Related posts: Null Hypothesis and Significance Level.
Example Interpretation
The following contingency table displays our example data for Fisher’s exact test.
| Male | Female | |
| Yes | 4 | 9 |
| No | 10 | 3 |
In the table, it appears that females are more likely to vote Yes, while males are more likely to vote No on the referendum issue. However, the apparent relationship in the sample data might be random sampling error rather than a real correlation. Let’s perform the analysis!
The cell counts are too small for the chi-square analysis. Consequently, we’ll use Fisher’s exact test to determine whether this relationship is statistically significant.
We’ll use a Fisher’s exact test calculator to obtain the p-value.
Enter the following values for each letter field in the calculator and choose two-tailed in Test type:
- A: 4
- B: 9
- C: 10
- D: 3
The calculator calculates a p-value of 0.047 for the Fisher’s exact test, which is less than our significance level of 0.05. Our results are statistically significant. We can reject the null and conclude that a relationship exists between gender and voting choice.
Share this:
Reader Interactions
Comments
Comments and Questions
Hi Jim,
Thanks for the explanation and example.
However, Using the formula P = [(a+b)! * (c+d)! * (a+c)! * (b+d)!] / [a! * b! * c! * d! * N!], I calculated the p-value for the given example (a=4, b=9, c=10, d=3) to be 0.021174, but not 0.047 in the article. Did I miss any steps? Thank you.
Hi Hui,
I’m not 100% sure but here’s a possibility. I’ve performed a two-tailed test. Is it possible that you performed a one-tailed test? The result for one-tail is close but not equal to your answer.
The value 0.021174 closely resembles the one-tailed result (0.024). In Fisher’s exact test, a two-tailed p-value is often double the one-tailed p-value or slightly adjusted based on the combined probability of tables as extreme or more so than the observed one. If you calculated a single probability without summing over the relevant tables, you would indeed obtain a lower result closer to 0.021.
That’s my best guessed based on you’ve provided.
Jim, Relative to your example, how would you write a one tailed hypothesis stating that males were more like to vote no.
And, please show us the step by step computations for the Fisher Exact Test that test the hypothesis.
Thank you
Mike
Hi Mike,
For starters, I show how to write the null and alternative hypotheses for the example in the post. Note that the test assesses whether a relationship exists or not. So, that’s how you write the hypotheses. The test is not specifically testing whether males are more likely to vote no. However, when you reject the null, you can conclude the relationship between gender and vote exists. Then by looking at the table you can understand that males are more likely to vote no. However, strictly speaking, the test determines whether the relationship exists but not the precise nature of it. So, again, look at the post for the correct way to write the hypotheses for the example.
Here’s how to calculate Fisher’s exact test.
The probability of observing the specific arrangement of counts in the table under the null hypothesis is calculated using the following formula:
P = [(a+b)! * (c+d)! * (a+c)! * (b+d)!] / [a! * b! * c! * d! * N!]
Where:
– “(a+b)!” is the factorial of the sum of a and b, representing the number of ways to arrange a items from a + b items.
– “(c+d)!” is the factorial of the sum of c and d, representing the number of ways to arrange c items from c + d items.
– “(a+c)!” is the factorial of the sum of a and c, representing the number of ways to choose a and c together.
– “(b+d)!” is the factorial of the sum of b and d, representing the number of ways to choose b and d together.
– “a!”, “b!”, “c!”, and “d!” are the factorials of the individual counts in the table.
– “N!” is the factorial of the total number of observations.
This formula calculates the probability of seeing the specific counts in your table, assuming the null hypothesis is true.
Hi Jim, I am wondering what will happen if in a 5×3 or a 8X2 table, expected frequency is lower than 5 in some cells. Should I then move to doing Fisher’s exact instead of Chi square? Should I do the exact in SPSS instead of Stata?
Hi Shambhu,
Yes, you should use Fisher’s exact test. Unless one places a limit on the size of the table for Fisher’s exact test, you can use either software depending on your comfort level. It is computationally intensive.
Hi Jim – how do we interpret the odds ratio of the output?
Hi,
I’ve written a post about odds ratios and how to interpret them. Click to read more. If you have questions after reading that, don’t hesitate to ask!
i am working on a project where we are subjecting the bottle for sterlisation using steam.We are producing the bottle in 3 different machines A,C and B.While performing sterlisation process A and C machine bottle fail and B machine bottle with stands. So i am trying to perform few hypothesis testing
1) Comparison of A machine bottle for thickness and height before and after sterlisation
2)Comparisson of A machine bottle with B machine bottle for thickness and height
Please suggest which hypotesis test has to be done as i feel T test will help .But my manager is insisting on Fisher test which i didnt understand..
Got it. Thank you so much for the help.
Hi, Can I use a fisher test on samples that have an ordinal variable? If not what do you suggest? The samples are really small.
Hi Sharyl,
Yes! You can use Fisher’s test with ordinal data. The basic Fisher’s test works with a 2X2 table. However, there are extensions for larger tables. Just be sure that you’re using an extension appropriate for your table size (check your software). Be aware that while very small sample sizes are valid for this test, they do reduce statistical power, meaning that it can be hard to obtain significant results.
What if there are 60 samples, can i use Fisher Exact test?
If i need to find association between a categorical variable (like marital status) and a continuous variable (like initial social functioning score) which test is best?
Hi, if you have two groups, Married and Single, and a continuous outcome variable, consider using a 2-sample t-test, also known as an independent samples t-test.
Hello Jim,
Thank you for the explanation. I performed Fisher’s exact test in SPSS, on a 5 x 6 table and sample of 75. However, I got a feedback that “cannot be computed because there is insufficient memory”. What could be the issue? Thanks
Hi. I performed Fisher’s exact test in SPSS, on a sample of 141 diabetic patients who have diabetic foot and I wanted to explore if there is an association between smoking levels on the rows side (nonsmoker, smoker, exsmoker) and diabetic foot prognosis on the columns side (No amputation, Amputation). I had the data in a 3 by 2 table, and I got SPSS output table showing the following numbers:
first a value of 1.791 on the left side adjacent to “Fisher’s exact test” sentence and I wonder what does it represent? and how is it calculated.
second: a P value in the same row (0.454).
third: a value named “The standardized statistic is -1.042.” in the notes below the table, and I wonder what does it mean? and how is it calculated?
Interestingly, on the same sample of 141 diabetic patients, when I wanted to explore the association between diabetes mellitus type on the rows side (type 1, type 2) and diabetic foot prognosis on the columns side (No amputation, Amputation), I got SPSS output table showing empty cell adjacent to “Fisher’s exact test” sentence on the left side. and a P value in the same row (0.720) and the term “The standardized statistic is -0.339.” in the notes below the table mean. I wish if I could upload an image showing these results better than words.
Hi Katib,
The main statistic you’re interested in for both tests are the p-values. Unfortunately, both p-values are not significant. There’s insufficient evidence in your sample to conclude that a relationship between the variables exists in the population.
Unfortunately, I’m not familiar with with the standardized statistic in SPSS. However, given that the results are not significant, there’s no point trying to interpret the other statistics. All you can conclude is that you fail to reject the null. There is no detectable relationship.
Fisher’s Exact test is best for small samples where you have cells in your table that have expected counts that are less than 5. If your expected counts all exceed 5, consider using the chi-square test results instead of Fisher’s exact test. Because your sample size is 141, you might not need to use Fisher’s exact test. I’m not saying that’ll change your results notably, but it’s a consideration.