Typically, quality improvement analysts use control charts to assess business processes and don’t have hypothesis tests in mind. Do you know how control charts provide tremendous benefits in other settings and with hypothesis testing? Spoilers—control charts check an assumption that we often forget about for hypothesis tests!
Before we get to using control charts with hypothesis tests, bear with me while I quickly explain their standard usage in statistical process control (SPC) and quality improvement initiatives.
Control charts plot process data and help you identify common cause and special cause variation. These graphs can determine whether the process is stable and if variability is a problem. If variability is problematic, control charts can determine whether the variability is intrinsic to the process or related to specific sources. By identifying the different sources of variation, you can keep your process stable without over-correction. Control charts guide your remedial actions.
When control charts determine that a process is stable, you can perform additional analyses to draw conclusions about the process. However, an unstable process is unpredictable, and you can’t draw reliable conclusions about its behavior. Any conclusions that you draw today might not be correct tomorrow.
Control charts are linked to business processes, but I’ll make the case that these plots provide tremendous benefits for processes and hypothesis testing that fall outside the realm of quality improvement. I’ll show a real example where control charts gave me a clear answer that would’ve been hard to find otherwise.
Control Charts can Assess Non-Business Processes
The trick to seeing how control charts work in a wide variety of settings is to enlarge your notion of processes to include non-business processes. After all, instability and variability are problems in many other environments. For instance:
- Teaching is the process of transferring knowledge that is measured by testing.
- People with diabetes have a process for maintaining blood sugar at a stable level.
- I had a process for causing research participants to experience impacts of 6 times their body weight.
These processes can be unstable or stable, have some natural variability, and might have special causes of variability. Assessing these issues can help you improve the processes. Just like business processes, if your data aren’t stable, conclusions that you draw using hypothesis tests are unreliable.
Let’s start by showing how control charts can provide crucial information in a non-business process.
The third bullet point above recounts a study that I was a part of. Our study had middle school participants jumping off 24-inch steps 30 times on alternating school days. The research goal was to determine whether these impacts would cause their bone density to increase. We defined the treatment as impacts of six times their body weight. Unfortunately, not all subjects experienced impacts of this magnitude initially.
While the mean of the impacts was above six times the body weight, and a hypothesis test confirmed this, we knew this was not good enough. All subjects should achieve the target impact force.
Using a Control Chart in My Research Study
To devise a solution, I conducted a pilot study and plotted the data from this process on an Xbar-S control chart.
To interpret this chart, you start by looking at the S chart on the bottom, which displays the variability of each subject’s landing impacts. There are no points outside of the control limits. Consequently, this graph indicates that each subject has their own consistent landing style. This variability is in control.
The Xbar chart on the top shows that the overall mean (6.141) is greater than our target. Unfortunately, data points fall outside of the control limits, which indicate that this process is out of control. Different subjects have dramatically different average landing impacts.
Taken all together, the interpretation of the control chart indicates that some participants have large impacts consistently while others have lower impacts consistently. However, this variability is not intrinsic to the process (common cause variation) but assignable to differences between the participants (special cause variation).
This information guided the corrective measures that we implemented. Had the variability been inherent in the process, we probably would have built higher steps. However, because we could attribute the variability to the subjects, we decided to teach the subjects how to land and have a nurse watch all jumping sessions to provide feedback on the spot. This combination lessened the variability enough so that all impacts were greater than six body weights.
Success! Even though this was not a business process, a control chart provided invaluable information.
Use Control Charts to Test Assumptions for Hypothesis Tests
Controls charts verify the assumption that a process is stable. We don’t usually think of applying this assumption to hypothesis tests. However, data for a hypothesis test must also be stable otherwise the conclusions aren’t reliable.
To illustrate this point, suppose we need to compare test scores between two groups. You can download the CSV data file: 2TControlCharts. To compare the means, we’ll perform a 2-sample t-test. The results are below.
Related post: How T-Tests Work
The statistical output shows that group A has a higher mean than group B. Furthermore, the p-value of 0.000 indicates that this difference is statistically significant. Group B’s standard deviation is slightly higher, but this test does not assume they are equal. If you perform normality tests on the samples, you’ll find that both groups are normally distributed. Although, our sample sizes are large enough that we don’t have to worry about this assumption. It all looks good, right?
The I-MR charts tells us another story!
The I-MR chart for group A indicates that these scores are in control. However, group B has many points that are out-of-control. Group B is unstable, and you can see the negative trend. It is not valid to draw conclusions from the unstable group even though the data satisfies all of the other assumptions. The difference between the two groups is not constant and depends on when you take your measurements.
The I-MR chart for group B displays just one of many types of problems that control charts can detect. Control charts are a valuable addition to your toolbox because other methods can miss these problems.
Using the Different Types of Control Charts
This blog post highlights only the tip of the iceberg for the capabilities of control charts. There are different kinds of control charts you can use based on your data and whether you have subgroups in your data.
For the examples in this post, the Xbar-S and I-MR charts both assess the mean but look at different forms of variability. Additionally, the Xbar-S chart assesses data that are in subgroups while the I-MR chart does not.
There are other types of control charts for other kinds of data. For example, if you are assessing:
- Proportions, consider using the P Chart before performing a 1 Proportion or 2 Proportion hypothesis test.
- Count data, consider using the U chart before conducting a 1-Sample or 2-Sample Poisson Rate hypothesis test.
Learn more about control charts even if you’re not working in the field of quality improvement. They can be tremendously helpful when you’re analyzing data and performing hypothesis tests!