Margin of Error: Formula and Interpreting

What is the Margin of Error?

The margin of error (MOE) for a survey tells you how near you can expect the survey results to be to the correct population value. For example, a survey indicates that 72% of respondents favor Brand A over Brand B with a 3% margin of error. In this case, the actual population percentage that prefers Brand A likely falls within the range of 72% ± 3%, or 69 – 75%.

The margin of error calculates a distance from the survey’s value in which the actual population value is likely to occur. It assesses the precision of a survey’s estimates. A smaller margin of error suggests that the survey’s results will tend to be close to the correct values. Conversely, larger MOEs indicate that the survey’s estimates can be further away from the population values.

Surveys frequently use random samples to estimate population percentages. These population values, also known as parameters, are almost always unknown because it is infeasible to measure entire populations. Instead, surveys use samples to estimate them. Unfortunately, these estimates seldom equal the correct value exactly due to random sampling error. The MOE is an inferential statistics procedure related to confidence intervals that evaluates the uncertainty involved when using random samples to estimate a population’s characteristics.

In this post, learn about the margin of error formula and how to calculate and interpret them.

Related posts: Populations and Samples, Parameters vs. Statistics, and Populations, Parameters, and Samples in Inferential Statistics

Confidence Levels

Like confidence intervals, the margin of error has a confidence level. Different random samples drawn from the same population are likely to produce slightly different estimates. If you draw many random samples and calculate a margin of error around each estimate, a percentage of those ranges will contain the population value.

Confidence levels are the long-run percentage of times that the population value falls within the MOE. Standard confidence levels are 90, 95, and 99%. 95% is the most common. For surveys that use a 95% confidence level, you’d expect 95% of their results to have margins of error that contain the population value.

For the image below, imagine that you draw 20 random samples and administer the same survey to all of them. Then you calculate the margin of error for an item on all 20 surveys. If you use a 95% confidence level, you’d expect that 19 out of the 20 intervals will contain the correct population value.

Related posts: Confidence Intervals and Sample Estimates Are Always Wrong (to Some Extent)!

Maximum Margin of Error for a Survey

In a survey, the size of the margin of error varies depending on the percentage. Surveys frequently cite its maximum MOE. That’s the value you’ll see in the media when they report a survey’s margin of error. Knowing this value is beneficial because the margins of error for all percentages in a survey will not exceed this value.

The largest margin of error for a given sample size occurs when the percentage is 50%. For example, 50% of the respondents prefer Candidate Smith. As the percentage for an item moves away from 50%, either higher or lower, the MOE decreases.

Let’s see this phenomenon in action graphically! The graph below displays the 95% margin of error for a sample size of 1,000. The vertical axis represents the MOE, while the horizontal axis indicates percentages in the survey. The highest margin of error occurs at the red line where the survey result is 50%, and it diminishes in both directions from that peak.

In the margin of error formula section, I’ll show you how to find the maximum and other MOEs.

Download the Excel file I made to create this graph: Margin Of Error. In this file, you can change the sample size to see how it affects the MOE for all percentages.

Margin of Error Formula

Surveys frequently use proportions and percentages in their results. For example, 92% agree with a particular decision. Consequently, the margin of error formula for surveys relates to percentages. When you add and subtract the MOE from the survey proportion, you obtain a confidence interval of the proportion.

In statistics, most margin of error/confidence interval calculations involve multiplying a critical value by a standard error. In the following formula, the square root is the standard error of the proportion, and you multiply it by a critical Z-value. Learn more about standard errors.

The margin of error formula is the following:

Where:

• The Z-value is the critical Z-value that corresponds to your confidence level.
• p is the sample proportion or percentage.
• n is the sample size.

You set the confidence level by choosing the critical Z-value. Below are the Z-values for the standard confidence levels.

Confidence Level	Z-value
90%	1.64
95%	1.96
99%	2.58

Related posts: Critical Values, Z-scores, and Z-table

Finding the Maximum Margin of Error Formula

To find the maximum MOE that surveys typically report, we need to set p to 0.5 and use the Z-value for our confidence level. 95% is the standard confidence level. Consequently, I’ll use the corresponding Z-value of 1.96. Because we’re setting the values of these two inputs (Z and p), the maximum margin of error formula becomes a direct transformation of the survey’s sample size, as shown below.

For any survey, all we need to enter into the formula is the sample size to find its maximum margin of error. Each sample size has its own maximum value. For example, if our sample size is 1000, our maximum MOE is the following:

This maximum MOE corresponds to the peak in the chart above.

Finding the Margin of Error for Other Percentages

It’s easy to use the margin of error formula for other percentages. Simply enter the percentage as a proportion (i.e., 75% = 0.75), sample size, and Z-score into the formula.

For example, a survey item finds that 75% of the respondents agree with the governor’s policy. 1000 individuals responded to this survey. To find the 95% margin of error for this percentage, we do the following:

We can be 95% confident that the population percentage for this survey item is within 75% ± 2.684%, or 72.316 – 77.684%.

Additionally, notice that the MOE for this specific percentage (75%) is 2.684% versus the maximum MOE of 3.099% for 50% when you have a sample size of 1,000. You can find both of these points on the earlier chart.

Caveats and Cautions for MOEs

The margin of error only accounts for random sampling error. It cannot factor in wording biases, non-representative samples, difficulties that exclude particular types of respondents, etc.

Additionally, these MOEs are for individual percentages and not the differences between responses. For example, suppose 51% of respondents prefer Candidate A and 48% prefer Candidate B. If the margin of error is 4%, the media will often state that it’s a statistical dead heat or tie because the difference between the two candidates (3%) is smaller than the margin of error (4%). Learn more about the Challenges of Political Polling.

While that is true, it’s an overly simplistic method because it doesn’t indicate how large the difference between two percentages must be to be statistically significant. To evaluate the differences between two percentages, you need to calculate a different type of interval—the confidence interval of the difference between two proportions. I use that type of test in my post about the effectiveness of flu vaccines.

However, when you only know the reported margin of error for individual percentages, you might still be able to draw some conclusions. If the difference between the two percentages is twice the MOE, it is statistically significant. Additionally, differences slightly smaller than 2 * MOE can also be significant, but you’d need to perform the correct test to know for sure.

I write about a similar phenomenon when comparing group means using confidence intervals of the differences rather than the individual means.