Nonlinear regression is an extremely flexible analysis that can fit most any curve that is present in your data. R-squared seems like a very intuitive way to assess the goodness-of-fit for a regression model. Unfortunately, the two just don’t go together. R-squared is invalid for nonlinear regression.
Some statistical software calculates R-squared for these models even though it is statistically incorrect. Consequently, it’s important that you understand why you should not trust R-squared for models that are not linear. In this post, I highlight research that shows you how assessing R-squared for nonlinear regression causes serious problems and leads you astray.
Why Is R-squared Valid for Only Linear Regression?
In my post about how to interpret R-squared, I explain how R-squared is the following proportion:
Furthermore, the variances always add up in a particular way:
Explained variance + Error variance = Total variance.
This arrangement produces an R-squared that is always between 0 – 100%.
That all makes sense, right? For linear models, this works out as you expect.
However, this math works out correctly only for linear regression models. In nonlinear regression, these underlying assumptions are incorrect. Explained variance + Error variance DO NOT add up to the total variance! The result is that R-squared isn’t necessarily between 0 and 100%. There are other problems with it as well.
This problem completely undermines R-squared in the context of nonlinear regression.
Keep in mind that I’m referring specifically to nonlinear models. R-squared is valid for linear models that use polynomials to model curvature. If you’re not clear about the difference between these two types of models, read my post to learn how to distinguish between linear and nonlinear regression.
Specific Problems of Using R-squared with Nonlinear Regression
The general mathematic framework for R-squared doesn’t work out correctly if the regression model is not linear. Despite this issue, most statistical software still calculates R-squared for nonlinear models. This questionable practice can cause problems for you. Let’s see the ramifications!
Spiess and Neumeyer* performed a simulation study to look at the effect of using R-squared to assess the goodness-of-fit for models that are not linear. Their study ran thousands of simulations and found that R-squared leads you to draw false conclusions about which nonlinear models are best.
If you use R-squared for nonlinear models, their study indicates you will experience the following problems:
- R-squared is consistently high for both excellent and appalling models.
- R-squared will not rise for better models all of the time.
- If you use R-squared to pick the best model, it leads to the proper model only 28-43% of the time.
If you take all of these together, R-squared can’t differentiate between good and bad nonlinear models. It just doesn’t work. The authors go on to disparage the continuing practice of statistical software to calculate R-squared for nonlinear regression:
In the field of biochemical and pharmacological literature there is a reasonably high occurrence in the use of R2 as the basis of arguing against or in favor of a certain model. . . . Additionally, almost all of the commercially available statistical software packages calculate R2 values for nonlinear fits, which is bound to unintentionally corroborate its frequent use. . . . As a result from this work, we would like to advocate that R2 should not be reported or demanded in pharmacological and biochemical literature when discussing nonlinear data analysis.
If your statistical software calculates R-squared for nonlinear models, don’t trust it!
There are other goodness-of-fit measures you can use for regression models that are not linear. For instance, you can use the standard error of the regression. For this statistic, smaller values represent better models.
Spiess, Andrej-Nikolai, Natalie Neumeyer. An evaluation of R2 as an inadequate measure for nonlinear models in pharmacological and biochemical research: a Monte Carlo approach. BMC Pharmacology. 2010; 10: 6.