A confidence level is the percentage reflecting how confident you are in the method that calculated a confidence interval (CI). It tells you how frequently the procedure produces intervals that contain the true population value if you repeated the study many times.
Common confidence levels include 90%, 95%, and 99%, with 95% being the most frequently used in practice. The confidence level is set by the analyst before calculating the interval and reflects how cautious the analysis should be.
For example, a 95% confidence level means that if you were to draw many random samples from the same population and compute a confidence interval for each one, about 95% of those intervals would contain the true population value.
Higher confidence levels produce wider confidence intervals to ensure they contain the population value more often.
For example, twenty different researchers conduct separate studies to estimate the average number of steps taken per day in the same population. Each study uses a random sample and calculates its CI using a 95% confidence level. If all assumptions are met, we expect that about 19 out of the 20 intervals (95%) will contain the true population mean. One interval might miss the true value, but researchers can be confident that the method produces accurate results most of the time.
