The F-test of overall significance indicates whether your linear regression model provides a better fit to the data than a model that contains no independent variables. In this post, I look at how the F-test of overall significance fits in with other regression statistics, such as R-squared. R-squared tells you how well your model fits the data, and the F-test is related to it.

An F-test is a type of statistical test that is very flexible. You can use them in a wide variety of settings. F-tests can evaluate multiple model terms simultaneously, which allows them to compare the fits of different linear models. In contrast, t-tests can evaluate just one term at a time.

Read my blog post about how F-tests work in ANOVA.

To calculate the F-test of overall significance, your statistical software just needs to include the proper terms in the two models that it compares. The overall F-test compares the model that you specify to the model with no independent variables. This type of model is also known as an intercept-only model.

The F-test for overall significance has the following two hypotheses:

- The null hypothesis states that the model with no independent variables fits the data as well as your model.
- The alternative hypothesis says that your model fits the data better than the intercept-only model.

In statistical output, you can find the overall F-test in the ANOVA table. An example is below.

## Interpreting the Overall F-test of Significance

Compare the p-value for the F-test to your significance level. If the p-value is less than the significance level, your sample data provide sufficient evidence to conclude that your regression model fits the data better than the model with no independent variables.

This finding is good news because it means that the independent variables in your model improve the fit!

Generally speaking, if none of your independent variables are statistically significant, the overall F-test is also not statistically significant. Occasionally, the tests can produce conflicting results. This disagreement can occur because the F-test of overall significance assesses all of the coefficients jointly whereas the t-test for each coefficient examines them individually. For example, the overall F-test can find that the coefficients are significant *jointly *while the t-tests can fail to find significance *individually*.

These conflicting test results can be hard to understand, but think about it this way. The F-test sums the predictive power of all independent variables and determines that it is unlikely that *all* of the coefficients equal zero. However, it’s possible that each variable isn’t predictive enough on its own to be statistically significant. In other words, your sample provides sufficient evidence to conclude that your model is significant, but not enough to conclude that any individual variable is significant.

**Related post**: How to Interpret Regression Coefficients and their P-values.

## Additional Ways to Interpret the F-test of Overall Significance

If you have a statistically significant overall F-test, you can draw several other conclusions.

For the model with no independent variables, the intercept-only model, all of the model’s predictions equal the mean of the dependent variable. Consequently, if the overall F-test is statistically significant, your model’s predictions are an improvement over using the mean.

R-squared measures the strength of the relationship between your model and the dependent variable. However, it is not a formal test for the relationship. The F-test of overall significance is the hypothesis test for this relationship. If the overall F-test is significant, you can conclude that R-squared does not equal zero, and the correlation between the model and dependent variable is statistically significant.

It’s fabulous if your regression model is statistically significant! However, check your residual plots to determine whether the results are trustworthy! And, learn how to choose the correct regression model!

If you’re learning regression and like the approach I use in my blog, check out my eBook!

**Note: I wrote a different version of this post that appeared elsewhere. I’ve completely rewritten and updated it for my blog site.**

Aujah Thompson says

If my p value is larger than alpha, what does my f stat value have to be to also suggest failing to reject the null? My p value is larger than my alpha but my professor said what’s the f value. I have the f values from the ANOVA but I don’t understand enough to interpret the values.

Jim Frost says

Hi Aujah,

The p-value is based on the F-value. Typically, you don’t interpret the F-value directly. Your statistical software takes the F-value, factors in the degrees of freedom, and then uses that information to calculate the probability, which it presents as the p-value.

So, I am a little puzzled as to why your professor wants the F-value. By itself, it is fairly meaningless. It’s only after further calculations to produce the p-value does can you interpret it. I write about how the F-test works in ANOVA. It’s the same test but in a different context. Reading my other article will help you understand the relationship between the F-value and p-value. And, show you what I mean by incorporating degrees of freedom and calculating the probability.

At any rate, just focus on the p-value for the F-test.

Ali says

Hi Jim,

I am struggling to understand the relationship between the factors that can influence the F-Statistic. For example, if I were to double a sample size from 1000 to 2000, how would this influence the F-Statistic and why?

Jim Frost says

Hi Ali,

To understand what to expect, you have to know whether the relationships that your model describe actually exist in the population. If those relationships truly exist, then as you increase sample size, you’d expect the F-value to increase and its p-value to become smaller (more significant). The reason why is that because these relationships exist, the increased sample size gives the hypothesis test more power to detect them.

However, if those relationships do not exist, then F-values should decrease and the p-values increase (less significant). The reason why for this case is because those relationships do not exist, increase the power to detect them does not help. There’s nothing to detect.

Natasha says

Hi Jim,

I have a p value less than 0.05 and an F value of 84.42. Does this mean that I accept the null hypothesis or the alternative hypothesis?

Thanks, Tash

Jim Frost says

Hi Tash,

In a hypothesis test, you have two choices:

1) Fail to reject the null

2) Reject the null

Assuming your significance level is 0.05, you can reject the null because your p-value is less than it. Your model explains more of the variance around the dependent variable than just using the mean of the dependent variable. That’s a good sign!

And, a quick clarification, you never “accept” a hypothesis. I know that sounds strange, but read my post about failing to reject the null hypothesis for more information.

Russ Ziegler says

Good evening Jim, I am working on an RD design and am having problems interpreting the data. I was wondering if you could help? One question that I have is how small can an R squared value be and still be ok? For example:

Residual standard error: 0.01852 on 3968 degrees of freedom

Multiple R-squared: 0.0122, Adjusted R-squared: 0.0112

F-statistic: 12.25 on 4 and 3968 DF, p-value: 6.702e-10

The p-value on my F-statistic shows that it is significant but the R-squared value is only .0122? Is that telling me that my model does not fit the data well?

Jim Frost says

Hi Russ,

Technically, your F-test of overall significance indicates that your R-squared is significantly different from zero. However, your model barely explains any of the dependent variable’s variance. Were any of your IVs significant?

Steven T says

Hi Jim, thank you for this post. I’ve run into a problem that doesn’t seem easily searchable. I have two regression models with the same dependent variable. They each have 3 independent variables, 1 of which differs between the models while the other 2 are the same. The 1 variable that differs between the two models is a continuous variable. I am trying to determine if either model is significantly more predictive of the dependent variable than the other. Do you know how I might to do that?

Jim Frost says

Hi Steven, because they have the same dependent variable, you can use the standard error of the regression (S) to compare how well each model fits using the units of the DV. For more information, read my post about the standard error of the regression.

Marko says

Hi, I need some help, please. I have got F value in ANOVA (I did spss regression) = 67,662. How can I interpretate this result ? Because I got Sig in Anova = .000, my R square for multiple regression is .936. Altough in table coefficients on variable has Sig .0001, beta .5, and I got one more with Sig .000 and beta=.6.

Thank You.

Best regards,

MR

Jim Frost says

Hi Marko,

Typically, you don’t interpret the F-value directly, but instead the p-value associated with it. For the F-test, your p-value of 0.000 indicates the model as a whole is statistically significant. Additionally, it looks like your independent variables are also significant. The R-squared is also high. It looks like good results overall. However, check the residual plots to be sure there aren’t any problems.

Grace says

Hi Jim,

Thank you so much for your valuable post, is this F -test only suitable for linear regression, or it also can be suitable for nonlinear regression? I use Tablecurve 2.0 to fit the data, it also give a output like this for nonlinear model?

Thank you very much.

Grace

Oladayo says

Hi Jim,

Please how do I interprete a multiple regression model whose result yielded: multicollinearity absence, positive autocorrelation and heteroscedasticity presence.

Here’s the model:Y =76.891X1+ 0.006X2- 521308.877

Jim Frost says

Hi Oladayo,

I recommend you read my post about OLS assumptions to see how the autocorrelation and heteroscedasticity affect your results.

For help about interpreting the equation, read my post about how to interpret regression coefficients. Although, beware of the effects of violating the assumptions.

Dam says

Hi Jim,

thank you very much for your excellent posts. I have a question. I am doing some multiple regression analysis for my thesis to investigate the impact of CSR on ROE with some other control variables. If the F test is significant (p less than 0.05) but the coefficient of CSR is not significant (p larger than 0.05) can I conclude that the null hypothesis for this predictor is accepted ? I mean CSR doesn’t impact ROE.

Many thanks for your attention.

Best regards

Dam

Fieze says

Hi Jim,

How can I determine if the difference between two independent sample regression model from the same population are significant when I create two subsamples male and female.

Jim Frost says

Hi Fieze,

Read this post about comparing regression models and use gender as the indicator variable as I describe. Then, check the significance of the gender variable.

Best of luck with your analysis!

Sawan says

Hi Jim! Thank you for the above answer. I have a little different question. Since F-statistic is obtained from F-distribution which in turn is ratio of two ch-squared variable with the respective degree of freedoms.

Some nomenclature:

TSS = Total sum of squares = variances of y from its mean

RSS = Residual sum of squares = summation of residuals

p= number of predictors

n= number of observations

F =((TSS-RSS)/n)/(RSS/(n-p-1))

E[RSS/(n-p-1)] = sigma^2 (i.e variance of error term). [This i know]

How to obtain that under the null hypothesis i.e when all regression coefficients are zero

we get E[(TSS-RSS)/n] = sigma^2 ?

Thank you!

Chau says

Thank you for your clear and helpful answer. It helped me a lot !

chau says

Thanks so much for the article.

I wonder what’s the difference between F-test for variance and F-test in regression.

F-test in variance: https://www.youtube.com/watch?v=g9pGHRs-cxc

Is that they’re the same kind (F-test), but a bit modified for each use-case? Thanks

Jim Frost says

Hi Chau,

The F-test is a very flexible type of hypothesis test. This test assesses whether the variances in the numerator and denominator are equal. By changing the variances that you include in the numerator and denominator, you change the purpose of the test. As the video shows, you can simply test to determine whether the variances of two populations are different. You put your variance estimate for one population in the numerator and the other variance estimate in the denominator.

However, if you change the numerator and denominator, you’re assessing different things. The F-test of overall significance in regression analysis uses the following variances in the numerator/denominator:

F = Variance the model explains / Error (Unexplained) variance

So, it’s the same test but you change the variances that you use. In turn, that changes the meaning of the results. I discuss how to interpret the F-test of overall significance in regression in the post so I won’t restate it here.

For yet another application of the F-test, see how ANOVA uses the F-test to determine whether the mean of three or more groups are different. It’s still an F-test but is uses different variances in the numerator and denominator.

Thanks for the great question!

Mani says

Hey Sir, Appreciate your effort .I have one question that I m analyzing cross sectional data related to determinants of child nutrition.By using OLS technique, it will not give consistent estimate because of endogenous variable.So I have to use 2SLS .But when I first stage regression by regressing endogenous variable on all other independent variable and my instrument ..The F value increased as 4540.17 with n=12834 ?Can you explain what is the problem?

Jim Frost says

HI Mani,

You might not have a problem. You have an extremely large sample size which gives the F-test a great deal of statistical power and explain the large F-value.

Wing says

Hi Jim,

Thank you for the post. It is very clear and helpful! Quick question, if the F-test indicates no significance of the model, does it mean the model is completely useless? Can I still look at the Intercept or residuals to say something about the data?

Jim Frost says

Hi, thanks! I’m glad you found it to be helpful!

Unfortunately, there’s not usually much you can say when the overall model is not significant. You can assess the residuals to double-check that you’re not missing curvature that might be present in your data. If that’s the case, you can fit the curvature and possibly obtain a significant model.

Occasionally, the overall F-test can be insignificant but an IV in the model can still be significant. That disagreement doesn’t happen often, but when it does, you can still say that the IV is statistically significant and draw conclusions about that relationship.

While I’m sure that you were hoping for a significant model, insignificant results can be helpful too. It helps stop you from jumping to conclusions. I write about this in my most recent post about how high p-values can be meaningful. I write about it in the hypothesis testing context (e.g., 2-sample t-tests and the like) but it also applies to other tests, such as the F-test of overall significance for linear models.

Balbir Singh says

Hey Jim, Have you published your book on regression analysis? if yes please provide me the link.

Jim Frost says

Hi Balbir,

Not quite yet but it’s oh so close! Just a week or two from now! I have one small hitch at the end.

Fokhrul Islam says

Hey Jim, I am new in using regression equation. can I test 1% to 5% significance level using this “F test”

Jim Frost says

Hi Fokhrul,

Yes, you can use either of those standard significance levels. 5% is more common!

rema says

Hi, this is helpful but I still struggle in interpreting the results using STATA and how to implement it correctly, any advice?

Jim Frost says

Hi Rema, if you’re refering to difficulties with regression analysis in general, you’re in luck, I’ll be publishing my ebook on regression analysis in March 2019. I think that’ll be very helpful for you. I cover all aspects of it in a very intuitive manner.

Anne-Lise says

Hi Jim,

Thanks so much for the great posts, really helping me study for my stats exam!

I’m having a hard time grasping what exactly the F-test is testing. I read that the null hypothesis is that all group means are equal when using ANOVA, and here it says that it is that the model with no independent variables fits the data better than one with independent variables. Are these two things equivalent?

Both interpretations were used in my lecture simultaneously and not really differentiated so I am a bit confused. Would appreciate some clarification, thanks a lot!

Jim Frost says

Hi Anne-Lise,

You’re very welcome! ðŸ™‚

The F-test is a very flexible test. In its most general sense, the F-test takes a ratio of two variances and tests whether the ratio equals 1. A ratio of 1 indicates that the two sets of variances are equal. A ratio greater than one suggests that the numerator is greater than the denominator. Statistical significance indicates that you can reject the null hypothesis that the ratio equals 1.

By changing the variances that are in the numerator and the denominator, you change what an F-test assesses. And, that’s leading to your confusing because there are different possible uses for F-tests depending on how it’s set up.

For an overall F-test, the ratio contains the variance accounted for by your model in the numerator and the variance accounted for by a model with no predictors in the denominator. If the ratio is significantly greater than one, you know that the numerator is significantly greater than the denominator. Hence, based on what is included in the numerator and denominator in the overall F-test, you know that your model (numerator) provides a fit that is significantly better than one without any predictors (denominator).

However, the F-test is set up differently in the ANOVA context by including different variances in the numerator and denominator, which changes your interpretation of the results. I’ve written about how the F-test works in ANOVA. As you read through that post, notice how the numerator/denominator are different than what I describe above for the overall F-test.

To answer you question, are these two things equivalent? They both use the same underlying test, but the variances in the numerator/denominator are different. In one sense, the interpretation is the same–statistical significance indicates that the ratio is greater than one, which, in turn, indicates that the numerator is greater than the denominator. However, interpreting numerator > denominator understandably depends on the variances that you include in them.

I hope this helps!

San San Ei says

Thank You so much completely reply!!

San San Ei says

Hi, Sir

Thank you so much for your reply!!

The overall F-test is compared with their critical value ( critical F-statistic) al alpha value 0.01, 0.05 and 0.1 level, not use p-value. When I test the model by using 0.05 and 0.1 level, it is significant. But, when using the alpha value 0.01 level, the overall model is not significant ( all model is rejected ) . So, how should I interpret for 0.01 level according to theorical terms. I ever see the lover significance level, the more the data must diverge from the null hypothesis to be significant.

Thank you..

Jim Frost says

Hi, whether you compare the F-value to the critical value or the p-value to the significance level, the results are the same. It’s a different way of doing the same thing. An F-value that equals the critical value is equivalent to the p-value equaling the significance level.

You pick one significance level before performing the analysis and then determine significance. If you compare your results to lower and lower significance levels, you’ll always find that it is not significant at some significance level. You never truly obtain a p-value of 0 (software might round to zero, it’s not actually zero.)

So, pick one level, and then determine significance using only that level, and that’s that. Don’t use more than one significance level.

For your last question, yes, lower significance levels require stronger evidence that the null hypothesis is not true. For statistical significance, you’ll need a larger divergence between the observed value and the null hypothesis value. You can read more about it in my post about the significance level and p-values.

San San Ei says

Hi Jim,

If the overall regression model is not significant at alpha level 0.01, how should I interpret about this?

Thank you so much!!

Jim Frost says

Hi,

If you set your significance level at 0.01 and your p-value for the overall F-test is greater than that value, you fail to conclude that your model fits the data better than a model with no predictors. However, I should point out that a significance level of 0.05 is more common. Although, there can be good reasons for using a lower significance level like you are. A lower significance level requires more evidence that your model is a good fit.

I hope this helps!

Lore says

Jim,

Thank you so much for your reply! You are great, and thanks to your blog I might have a not only a chance to understand a little more statistics, but also to get better marks for my module!

Have a lovely Christmas!

Regards,

Lore

Jim Frost says

Hi Lore,

I’m so happy to hear that it has been helpful! ðŸ™‚

Merry Christmas to you as well!

Lore says

Hi Jim,

Thank you so much for replying to my message. I would like to apologise for the quality of the output, and I must say that you’ve been so good in trying to make sense of that mess. I have no idea what sort of software was used, I would say R or STATA, however the bits I’ve posted here were copied from a regression table- hence the poor formatting.

Jim Frost says

Hi Lore,

I had to edit the extensive output out of your comment because it was so long. But, yes, the overall F-test indicates whether your model is better than a model with no predictors. And the t-tests for the individual variables indicate whether specific variables are significant. For more information about that aspect, read my post about regression coefficients and p-values.

Lore lory says

Hi Jim,

Thank you for your post/page and I cannot tell you how much I appreciate that someone like you is trying to make it easier for those like me!

I find myself thrown into the deep end, as I started a phd in political science with a background in law. I’ve never studied statistics and we’ve been literally bombarded with projects and term papers.

My lecturer isn’t the best at explaining, and lots of us are dreading the deadlines. On top of

that we have to score an overall B to pass, and here I am asking basic questions. Oh dear!

Please spare a minute to look at this:

R2 . 033 . 049

Adjusted R2 .031 .043

Residual Std. Error 2.266 (df = 489) 2.253 (df = 487)

F Statistic 16.866*** (df = 1; 489) 8.297*** (df = 3; 487)

*P<.05

**P<.01

***P<.001

Am I correct in saying that the results indicated the the F test shows that the independent variable are significant in both models?

R2 .008 .105 .134

Adjusted R2 .006 .099 .121

Residual Std. Error .928 (df = 497) .884 (df = 490) .873 (df = 486)

F Statistic 3.930* (df = 1; 497) 19.096*** (df = 3; 490) 10.698*** (df = 7; 486)

*P<.05

**P<.01

***P<.001

Would the same apply in this case?

Thank you so much for this.

Regards.

Jim Frost says

Hi Lore,

That does not sound like a fun situation to be stuck in! Sorry about that!

It looks like those are overall F-tests that are possibly looking at models possibly in a stepwise regression, or at least multiple models. It seems to be adding variables. Although, it’s not completely clear from the output. However, assuming these are overall F-tests, it does not refer to the significance of individual independent variables. Instead, it’s testing whether the model as a whole is better than a model with no independent variables. It’s also testing whether you can reject the null hypothesis that R-squared equals zero. This post talks in more depth about how to interpret these results.

It appears like all the models are significant. Although, again, I’m not entirely clear what is going on. The top set of output appears to assess two models, one with 1 IV and another with 3 IVs. The bottom set appears to assess three models: 1 IV, 3 IVs, and 7 IVs. For the models with 1 IV, the overall F-test does indicate that the single IV is significant. But, you can’t tell about individual IVs with models that have more than 1 IV.

What software produced this output? I don’t like how it doesn’t calculate the exact p-value! That’s poor practice in statistics. (That’s not directed at you, I’m just surprised that the software would not provide the exact p-values!)

I hope this helps!

Mohammad says

I would like to express my sincere gratitude for your efforts in making statistics more easily

I have a multiple regression model, and I have values of F test for 6 models and they are range between 17.85 and 20.90 and the Prob > F for all of them is zero, and have 5 independent variables have statistical significant effects on Dependent variable, but the last independent variable is insignificant. and the adjusted R square range between 0.48 to 0.52 . could you please help in interpreting my findings ? and clear my doubt about the insignificant independent variable ( the last one ), whereas F test is significant .

Jim Frost says

Hi Mohammad,

The overall F-test of significance just tells you whether your model predicts the outcome variable better than using the mean. Your models all do this. In other words, each of your models is a significant improvement over just using the mean value of the outcome variable. The overall F-test isn’t meant to tell you anything more. It doesn’t help you determine which independent variables are significant and should be included. So, it’s not surprising to have a significant overall F-test but an insignificant variable (or even more than one).

Regarding the model with the insignificant independent variable, you’ll have to use a mix of statistics and theory to determine whether to leave that variable in the model. Because its p-value is not significant, you’d often remove that variable from the model. Including insignificant variables in the model can reduce the precision of the model. However, if theory and subject-area knowledge suggest that it’s an important variable that should not be left out, it’s ok to keep it, just indicate the reasoning behind that in any report/paper.

As for adjusted R-squared, I’ll refer you to my post about it: adjusted R-squared.

I hope this helps!

Jackson says

Hi Jim,

I have values for F-statistics ranging from 39.39 to 69.81 for 6 different models with their respective p-values all <0.0001. What would you make of such information?

Jim Frost says

Hi Jackson,

Typically, you don’t interpret the F-values directly. Instead, you can use the p-values. Because your p-values are less than all common levels of significance, your models are statistically significant. This post tells you what a statistically significant model means.

Yamila says

Hi Jim,

When running the linear model in RStudio I get this results.

Coefficients:

Estimate Std. Error t value Pr(>|t|)

(Intercept) 11063.069 7305.303 1.514 0.1353

mpg -114.743 79.172 -1.449 0.1526

rep78 710.879 322.117 2.207 0.0312 *

headroom -725.636 416.660 -1.742 0.0868 .

trunk 70.113 103.633 0.677 0.5013

weight 4.034 1.532 2.634 0.0108 *

length -84.390 43.982 -1.919 0.0599 .

turn -207.480 131.497 -1.578 0.1200

displacement 16.630 8.995 1.849 0.0695 .

gear_ratio 1642.587 1061.150 1.548 0.1270

—

Signif. codes: 0 â€˜***â€™ 0.001 â€˜**â€™ 0.01 â€˜*â€™ 0.05 â€˜.â€™ 0.1 â€˜ â€™ 1

Residual standard error: 2178 on 59 degrees of freedom

(5 observations deleted due to missingness)

Multiple R-squared: 0.5149, Adjusted R-squared: 0.4409

F-statistic: 6.958 on 9 and 59 DF, p-value: 9.131e-07

p-value: 9.131e-07 is not in the threshold right? but my R-Square and F-Stats have a good high result. I am having a hard time interpretating this result. Could you help me?

Jim Frost says

Hi Yamila,

Your p-value is written in scientific notation. You need to move the decimal point 7 places to the left. Your p-value is actually 0.0000009131. That’s extremely low and your model is statistically significant.

Hifza says

When i run the regression i took 1 dependent and 2 dependent variable.. After run the regression my results are F =8.385337 and F Significance=0.106549 and Rsquare=0.893450 and p value=0.0027062 so plz tell me according to this results what is the interpretation of R-square and model significance as per probability of F test …

I mean what is the actual conclusion of the interpretation plz guide me…..

Jim Frost says

Hi Hifza,

Usually you don’t need to interpret the F-value itself. Your statistical software uses the F-value to calculate the p-value, which is what you should focus on. (I don’t know what the F significance value refers to.)

Because your p-value for the overall F-test (0.002) is less than the typical significance level of 0.05, we can conclude that your model explains the variability of the dependent variable around its mean better than using the mean itself. As for how it relates to the R-squared, you have sufficient evidence to reject the null hypothesis that your model’s R-squared equals zero. By most standards, you have a nice and high R-squared value. Fortunately, it all suggests that you have a good model–at least according to the statistics. However, be sure to check the residual plots as well!

Usually, you also look at the p-values for the specific independent variables to see which ones are significant on their own.

I hope this helps. Best of luck with your analysis!

Honey Shandilya says

Hello sir,my regression model has the r-square value is 0.392 and p value is 0.00029 ,which is good,but none of my independent variables have good coefficient or are not significant.So , what should i conclude from this.Help me sir.

Jim Frost says

Hi,

If that p-value is for the overall F-test, it suggests that your model is statistically significant. It is unusual to see such a low p-value for the overall F-test and not have any of the independent variables also be significant. It’s hard to know for sure, but it’s possible that your model has multicollinearity (correlated independent variables). This problem can make significant variables appear to be insignificant. However, multicollinearity does not affect R-squared and the overall F-test–which might explain what that p-value is still significant while the others are not.

To learn if this problem affects your model, read my post about multicollinearity.

I hope this helps!

Emily says

Thank you for this! I have a question regarding this. Can I use the F test to pick which model fits my data the best? I have tried several different models, and the F test for some of them is significant. Since I have multiple significant models, how do I choose which one to use? Do I choose the one with the lowest significant F test? The p-values for the individuals vary between the models, so picking which model to use really affects the results of my analysis.

Jim Frost says

Hi Emily,

Determining the best model for your data can be complicated. It will involving looking at more than one statistics–such as the F-test. I’ve written a blog post about how to choose the best model, which goes over a variety of things you should check. That post should answer a lot of your questions. But, don’t hesitate to ask if you have more!

Also, if the p-values and coefficients change dramatically depending on the variables that you include in the model, your model might have multicollinearity (correlated independent variables). This issue can make identifying the best model more difficult. You might want to read about it in my post about multicollinearity.

I hope this helps!

Emmanuel Nkant says

Merci beaucoup, tres concit, a real pleasure…to follow your teaching

Jim Frost says

Thank you, Emmanuel!

Olusola says

pls give a lecture on the Wald Chi square compared to F-value or f-test

CORDELIA says

Hello Jim,

I enjoy every bit of your lectures here. Please does it imply, when F-test is statistically insignificant?

Jim Frost says

Hi Cordelia,

I’m glad that you find these to be helpful!

If your F-test of overall significance is NOT significant, then you fail to reject the null hypothesis. For this test, you can interpret it in several equivalent ways.

You can say that you have insufficient evidence to conclude that your model explains the variation in the dependent variable any better than just using the mean of the dependent variable.

Or, you can say that you have insufficient evidence to conclude that the R-squared is significantly greater than zero.

In short, your model is not explaining the variability in dependent variable to a statistically significant degree.

I hope this helps!

Dawn says

Jim,

Thank you for a great post! I have a question. If my F-value was found to not be significant (p=.069), do I still interpret the individual t-values in the coefficients table? If I do, then there are two variables that are significant (p<.05). I have been searching and having difficulty finding an answer to this!

Thank You

Dawn

Jim Frost says

Hi Dawn, yes, the results of the different tests can disagree. Despite the insignificant F-test, you can still conclude that your two variables are statistically significant. I’d guess that either you’re leaving insignificant variables in the model and/or those two variables are close to the significance level.

Nara says

Hi Jim,

if the overall regression model and none of predictor is not significant, how should I interpret the F-value and R-squared?

Jim Frost says

Hi Nara, unfortunately, when the overall F-test is not significant and none of the predictors are significant, you really have no evidence of any relationships between your model and the response variable.

In terms of how to interpret the F-value, that’s the test statistic for F-tests. The test uses this statistic to calculate the p-value. The F-value is the ratio of two variances. For this type of test, the ratio is: Variance explained by your model / Variance explained by the intercept-only model. As the F-value increases for this test, it indicates that your model is doing better compared to the intercept-only model. When the F-value reaches a critical value, you can reject the null hypothesis. I’ve written about how the F-test works in one-way ANOVA. That post shows how F-values are converted to P-values. That’s a different use of the F-test, but the ideas are very much the same. You just change the variances that are included in the ratio.

As for the R-squared. Because your F-test of overall significance is not statistically significant, it means you have insufficient evidence to conclude that your R-squared is greater than zero. The R-squared value in your analysis might not equal zero, but that’s probably just due to chance correlations rather than a true explanation of the population variance.

Kate says

Hi Jim,

I’m trying to interpret the results of a general linear model I have run. I have two factors – treatment and date (where the same experiment was repeated on different dates). Both give a significant p value but one has a much higher F value (136 compared to 8). Does that mean the factor with the higher F value is having a greater effect?

Jim Frost says

Hi Kate, that’s a great question. In a nutshell, no, the higher F-value doesn’t indicate a greater effect. I write about how to identify the most important variables in your model. I talk about it in the regression context, but you can apply some of the principles to ANOVA as well. I think that post will help you with this issue.

Aasia says

Hi Jim. i want to know in ANOVA table of regression analysis if p value is significant, still is there any limit for F value? what if it comes as big as 300 0r 450 etc

Jim Frost says

Hi Aasia, an F-value is the ratio of two variances. Theoretically, there is no limit to the F value. In terms of the explained variance, the better your model is compared to the intercept only model, the higher the F-value. However, for a specific model with a given number of degrees of freedom in the numerator and denominator, higher F-values occur less frequently.

I hope this helps! Thanks for the great question!

Duc-Anh Luong says

Hi Jim,

Thank you so much for your interesting and easily understandable post. However, I have a question when we have the conflict between overall F-test and significant t-test for each predictor. What should we do if the t-test for some of prediction is non-significant? Should we remove this predictors and fit the model again?

Many thanks,

Duc Anh

Jim Frost says

Hi Duc-Anh,

Frequently you do remove an independent variable from a model if it is not statistically significant. There are some exceptions to this rule. If you believe that theoretical considerations suggest that the variable should be in the model despite an insignificant p-value, you could consider leaving it in. Additionally, if it is a variable that you are specifically testing in an experiment, you would leave it in to demonstrate the test results.

But, yes, frequently you would consider removing the predictor from the model if it is not statistically significant. Your dataset provides insufficient evidence to conclude that there is a relationship between that predictor and the response.

One more point, be sure to check the residual plots. There might be a curvilinear relationship.

I hope this helps,

Jim

Duc-Anh Luong says

Hi Jim,

Thank you so much for your reply. In case we keep one or more predictors that are not statistically significant based on some except rule you mentioned in the previous comment, how can we interpret the results now?

Best regards,

Duc Anh

Jim Frost says

Hi again Duc Anh,

It depends on why you leave the predictor in the model. If you’re leaving it in the model because it’s the specific term you are testing for your experiment, then you state that you have insufficient evidence to conclude that there is a relationship between this variable and the response.

However, if you’re leaving the variable in for theoretical reasons, that’s what you should state. The variable wasn’t statistically significant but theory/other studies suggest it belongs in the model. You might even investigate possible reasons for why it is not significant, such as a small sample size, noisy data, a fluky sample, etc. Even though you suspect the variable belongs in the model, your sample still provides insufficient evidence to conclude that the relationship exists. You really have to make sure you have a good strong reason for this approach and state clearly why you are doing so.

I hope this helps,

Jim

Duc-Anh Luong says

Hi Jim,

Thank you so much for your very specific response. I think that it is very true when we interpret the model parameter. How’s about when we use the model with one or more non-statistically significant variables to make prediction? Sorry for my stupid questions!

Best regards,

Duc Anh

Jim Frost says

Hi Duc Anh,

I was referring to the case where you leave a predictor in the model when it is not significant. If you’re using the model to make predictions, you have the additional consideration of the precision of the predictions. Leaving an insignificant predictor in the model might reduce the precision.

What you want to do is to compare the predicted R-squared and width of the prediction intervals between the model with the insignificant predictors and the model with only significant predictors. Read my post about using regression to make predictions for more information!

And, there really is no such thing as a stupid question! ðŸ™‚

Jim