• Skip to secondary menu
  • Skip to main content
  • Skip to primary sidebar
  • My Store
  • Glossary
  • Home
  • About Me
  • Contact Me

Statistics By Jim

Making statistics intuitive

  • Graphs
  • Basics
  • Hypothesis Testing
  • Regression
  • ANOVA
  • Probability
  • Time Series
  • Fun

conceptual

Statistical Hypothesis Testing Overview

By Jim Frost 50 Comments

In this blog post, I explain why you need to use statistical hypothesis testing and help you navigate the essential terminology. Hypothesis testing is a crucial procedure to perform when you want to make inferences about a population using a random sample. These inferences include estimating population properties such as the mean, differences between means, proportions, and the relationships between variables.

This post provides an overview of statistical hypothesis testing. If you need to perform hypothesis tests, consider getting my book, Hypothesis Testing: An Intuitive Guide.

[Read more…] about Statistical Hypothesis Testing Overview

Filed Under: Hypothesis Testing Tagged With: conceptual

Understanding Interaction Effects in Statistics

By Jim Frost 466 Comments

What are Interaction Effects?

An interaction effect occurs when the effect of one variable depends on the value of another variable. Interaction effects are common in regression models, ANOVA, and designed experiments. In this post, I explain interaction effects, the interaction effect test, how to interpret interaction models, and describe the problems you can face if you don’t include them in your model. [Read more…] about Understanding Interaction Effects in Statistics

Filed Under: Regression Tagged With: analysis example, conceptual, graphs, interpreting results

When Should I Use Regression Analysis?

By Jim Frost 172 Comments

Use regression analysis to describe the relationships between a set of independent variables and the dependent variable. Regression analysis produces a regression equation where the coefficients represent the relationship between each independent variable and the dependent variable. You can also use the equation to make predictions.

As a statistician, I should probably tell you that I love all statistical analyses equally—like parents with their kids. But, shhh, I have secret! Regression analysis is my favorite because it provides tremendous flexibility, which makes it useful in so many different circumstances. In fact, I’ve described regression analysis as taking correlation to the next level!

In this blog post, I explain the capabilities of regression analysis, the types of relationships it can assess, how it controls the variables, and generally why I love it! You’ll learn when you should consider using regression analysis. [Read more…] about When Should I Use Regression Analysis?

Filed Under: Regression Tagged With: conceptual

Degrees of Freedom in Statistics

By Jim Frost 91 Comments

What are Degrees of Freedom?

The degrees of freedom (DF) in statistics indicate the number of independent values that can vary in an analysis without breaking any constraints. It is an essential idea that appears in many contexts throughout statistics including hypothesis tests, probability distributions, and linear regression. Learn how this fundamental concept affects the power and precision of your analysis!

In this post, I bring this concept to life in an intuitive manner. You’ll learn the degrees of freedom definition and know how to find degrees of freedom for various analyses, such as linear regression, t-tests, and chi-square. I’ll start by defining degrees of freedom and providing the formula. However, I’ll quickly move on to practical examples in the context of various statistical analyses because they make this concept easier to understand.
[Read more…] about Degrees of Freedom in Statistics

Filed Under: Hypothesis Testing Tagged With: conceptual

Why Are There No P Values in Nonlinear Regression?

By Jim Frost 29 Comments

Nonlinear regression analysis cannot calculate P values for the independent variables in your model. Why not? And, what do you use instead? Those are the topics of this blog post. [Read more…] about Why Are There No P Values in Nonlinear Regression?

Filed Under: Regression Tagged With: conceptual

Five Regression Analysis Tips to Avoid Common Problems

By Jim Frost 18 Comments

Image of lightbulb to represent the regression tips in this article.Regression is a very powerful statistical analysis. It allows you to isolate and understand the effects of individual variables, model curvature and interactions, and make predictions. Regression analysis offers high flexibility but presents a variety of potential pitfalls. Great power requires great responsibility!

In this post, I offer five tips that will not only help you avoid common problems but also make the modeling process easier. I’ll close by showing you the difference between the modeling process that a top analyst uses versus the procedure of a less rigorous analyst. [Read more…] about Five Regression Analysis Tips to Avoid Common Problems

Filed Under: Regression Tagged With: conceptual

What is the Relationship Between the Reproducibility of Experimental Results and P Values?

By Jim Frost 8 Comments

The ability to reproduce experimental results should be related to P values. After all, both of these statistical concepts have similar foundations.

  • P values help you separate the signal of population level effects from the noise in sample data.
  • Reproducible results support the notion that the findings can be generalized to the population rather than applying only to a specific sample.

So, P values are related to reproducibility in theory. But, does this relationship exist in the real world? In this blog post, I present the findings of an exciting study that answers this question! [Read more…] about What is the Relationship Between the Reproducibility of Experimental Results and P Values?

Filed Under: Hypothesis Testing Tagged With: conceptual

Understand Precision in Predictive Analytics to Avoid Costly Mistakes

By Jim Frost 9 Comments

Precision in predictive analytics refers to how close the model’s predictions are to the observed values. The more precise the model, the closer the data points are to the predictions. When you have an imprecise model, the observations tend to be further away from the predictions, thereby reducing the usefulness of the predictions. If you have a model that is not sufficiently precise, you risk making costly mistakes! [Read more…] about Understand Precision in Predictive Analytics to Avoid Costly Mistakes

Filed Under: Regression Tagged With: analysis example, conceptual, graphs, interpreting results

Heteroscedasticity in Regression Analysis

By Jim Frost 62 Comments

Heteroscedasticity means unequal scatter. In regression analysis, we talk about heteroscedasticity in the context of the residuals or error term. Specifically, heteroscedasticity is a systematic change in the spread of the residuals over the range of measured values. Heteroscedasticity is a problem because ordinary least squares (OLS) regression assumes that all residuals are drawn from a population that has a constant variance (homoscedasticity).

To satisfy the regression assumptions and be able to trust the results, the residuals should have a constant variance. In this blog post, I show you how to identify heteroscedasticity, explain what produces it, the problems it causes, and work through an example to show you several solutions. [Read more…] about Heteroscedasticity in Regression Analysis

Filed Under: Regression Tagged With: assumptions, conceptual, graphs

How to Choose Between Linear and Nonlinear Regression

By Jim Frost 32 Comments

As you fit regression models, you might need to make a choice between linear and nonlinear regression models. The field of statistics can be weird. Despite their names, both forms of regression can fit curvature in your data. So, how do you choose? In this blog post, I show you how to choose between linear and nonlinear regression models. [Read more…] about How to Choose Between Linear and Nonlinear Regression

Filed Under: Regression Tagged With: analysis example, assumptions, choosing analysis, conceptual, interpreting results

Statistics, Old Love Letters, and Changing Times

By Jim Frost 2 Comments

Have you ever seen your present reflected in an object from the past? This summer I’ve discovered glimpses of my daily life working with statistical software in words written more than 70 years ago. Bear with me because this blog post takes the scenic route to arrive at modern statistics. [Read more…] about Statistics, Old Love Letters, and Changing Times

Filed Under: Fun Tagged With: conceptual

Why Are P Values Misinterpreted So Frequently?

By Jim Frost 4 Comments

P values are commonly misinterpreted. It’s a very slippery concept that requires a lot of background knowledge to understand. Not surprisingly, I’ve received many questions about P values in statistical hypothesis testing over the years. However, one question stands out. Why are P value misinterpretations so prevalent? I answer that question in this blog post, and help you avoid making the same mistakes. [Read more…] about Why Are P Values Misinterpreted So Frequently?

Filed Under: Hypothesis Testing Tagged With: conceptual

Model Specification: Choosing the Best Regression Model

By Jim Frost 60 Comments

Model specification is the process of determining which independent variables to include and exclude from a regression equation. How do you choose the best regression model? The world is complicated and trying to explain it with a small sample doesn’t help. In this post, I’ll show you how to decide on the model. I’ll cover statistical methods, difficulties that can arise, and provide practical suggestions for selecting your model. Often, the variable selection process is a mixture of statistics, theory, and practical knowledge. [Read more…] about Model Specification: Choosing the Best Regression Model

Filed Under: Regression Tagged With: conceptual

Comparing Regression Lines with Hypothesis Tests

By Jim Frost 69 Comments


How do you compare regression lines statistically? Imagine you are studying the relationship between height and weight and want to determine whether this relationship differs between basketball players and non-basketball players. You can graph the two regression lines to see if they look different. However, you should perform hypothesis tests to determine whether the visible differences are statistically significant. In this blog post, I show you how to determine whether the differences between coefficients and constants in different regression models are statistically significant. [Read more…] about Comparing Regression Lines with Hypothesis Tests

Filed Under: Regression Tagged With: analysis example, conceptual, interpreting results

Identifying the Most Important Independent Variables in Regression Models

By Jim Frost 72 Comments


You’ve settled on a regression model that contains independent variables that are statistically significant. By interpreting the statistical results, you can understand how changes in the independent variables are related to shifts in the dependent variable. At this point, it’s natural to wonder, “Which independent variable is the most important?” [Read more…] about Identifying the Most Important Independent Variables in Regression Models

Filed Under: Regression Tagged With: analysis example, conceptual, interpreting results

Confidence Intervals vs Prediction Intervals vs Tolerance Intervals

By Jim Frost 36 Comments

Intervals are estimation methods in statistics that use sample data to produce ranges of values that are likely to contain the population value of interest. In contrast, point estimates are single value estimates of a population value. Of the different types of statistical intervals, confidence intervals are the most well-known. However, certain kinds of analyses and situations call for other types of ranges that provide different information. [Read more…] about Confidence Intervals vs Prediction Intervals vs Tolerance Intervals

Filed Under: Hypothesis Testing Tagged With: choosing analysis, conceptual

As a Statistician, Can I Say Age is Just a Number?

By Jim Frost 1 Comment

My last birthday wasn’t one of those difficult ages that end with a zero. Thank goodness! However, the passage of another year got me thinking. At that point, I told myself that age is just a number. Can you do a mental double-take? I think I did one. Can a statistician say that age is just a number? After all, it’s through numbers that statisticians understand the world and how it works. [Read more…] about As a Statistician, Can I Say Age is Just a Number?

Filed Under: Fun Tagged With: conceptual

Using Data Mining to Select Regression Models Can Create Serious Problems

By Jim Frost 13 Comments


Data mining and regression seem to go together naturally. I’ve described regression as a seductive analysis because it is so tempting and so easy to add more variables in the pursuit of a larger R-squared. In this post, I’ll begin by illustrating the problems that data mining creates. To do this, I’ll show how data mining with regression analysis can take randomly generated data and produce a misleading model that appears to have significant variables and a good R-squared. Then, I’ll explain how data mining creates these deceptive results and how to avoid them. [Read more…] about Using Data Mining to Select Regression Models Can Create Serious Problems

Filed Under: Regression Tagged With: analysis example, conceptual, interpreting results

Five Reasons Why Your R-squared can be Too High

By Jim Frost 19 Comments

When your regression model has a high R-squared, you assume it’s a good thing. You want a high R-squared, right? However, as I’ll show in this post, a high R-squared can occasionally indicate that there is a problem with your model. I’ll explain five reasons why your R-squared can be too high and how to determine whether one of them affects your regression model. [Read more…] about Five Reasons Why Your R-squared can be Too High

Filed Under: Regression Tagged With: conceptual

Five P Value Tips to Avoid Being Fooled by False Positives and other Misleading Hypothesis Test Results

By Jim Frost 14 Comments

Despite the popular notion to the contrary, understanding the results of your statistical hypothesis test is not as simple as determining only whether your P value is less than your significance level. In this post, I present additional considerations that help you assess and minimize the possibility of being fooled by false positives and other misleading results. [Read more…] about Five P Value Tips to Avoid Being Fooled by False Positives and other Misleading Hypothesis Test Results

Filed Under: Hypothesis Testing Tagged With: conceptual

  • « Go to Previous Page
  • Go to page 1
  • Interim pages omitted …
  • Go to page 5
  • Go to page 6
  • Go to page 7
  • Go to page 8
  • Go to page 9
  • Go to Next Page »

Primary Sidebar

Meet Jim

I’ll help you intuitively understand statistics by focusing on concepts and using plain English so you can concentrate on understanding your results.

Read More...

Buy My Introduction to Statistics eBook!

New! Buy My Hypothesis Testing eBook!

Buy My Regression eBook!

Subscribe by Email

Enter your email address to receive notifications of new posts by email.

    I won't send you spam. Unsubscribe at any time.

    Follow Me

    • FacebookFacebook
    • RSS FeedRSS Feed
    • TwitterTwitter
    • Popular
    • Latest
    Popular
    • How To Interpret R-squared in Regression Analysis
    • How to Interpret P-values and Coefficients in Regression Analysis
    • Measures of Central Tendency: Mean, Median, and Mode
    • Normal Distribution in Statistics
    • Multicollinearity in Regression Analysis: Problems, Detection, and Solutions
    • How to Interpret the F-test of Overall Significance in Regression Analysis
    • Understanding Interaction Effects in Statistics
    Latest
    • Sampling Methods: Different Types in Research
    • Beta Distribution: Uses, Parameters & Examples
    • Geometric Distribution: Uses, Calculator & Formula
    • What is Power in Statistics?
    • Conditional Distribution: Definition & Finding
    • Marginal Distribution: Definition & Finding
    • Content Validity: Definition, Examples & Measuring

    Recent Comments

    • Chris Anderson on Guide to Data Types and How to Graph Them in Statistics
    • James on Introduction to Bootstrapping in Statistics with an Example
    • Khursheed Ahmad on Sampling Methods: Different Types in Research
    • Jim Frost on Interpreting Correlation Coefficients
    • Jim Frost on Interpreting Correlation Coefficients

    Copyright © 2022 · Jim Frost · Privacy Policy