• Skip to secondary menu
  • Skip to main content
  • Skip to primary sidebar
  • My Store
  • Glossary
  • Home
  • About Me
  • Contact Me

Statistics By Jim

Making statistics intuitive

  • Graphs
  • Basics
  • Hypothesis Testing
  • Regression
  • ANOVA
  • Probability
  • Time Series
  • Fun

Blog

Model Specification: Choosing the Best Regression Model

By Jim Frost 60 Comments

Model specification is the process of determining which independent variables to include and exclude from a regression equation. How do you choose the best regression model? The world is complicated and trying to explain it with a small sample doesn’t help. In this post, I’ll show you how to decide on the model. I’ll cover statistical methods, difficulties that can arise, and provide practical suggestions for selecting your model. Often, the variable selection process is a mixture of statistics, theory, and practical knowledge. [Read more…] about Model Specification: Choosing the Best Regression Model

Filed Under: Regression Tagged With: conceptual

Comparing Regression Lines with Hypothesis Tests

By Jim Frost 71 Comments


How do you compare regression lines statistically? Imagine you are studying the relationship between height and weight and want to determine whether this relationship differs between basketball players and non-basketball players. You can graph the two regression lines to see if they look different. However, you should perform hypothesis tests to determine whether the visible differences are statistically significant. In this blog post, I show you how to determine whether the differences between coefficients and constants in different regression models are statistically significant. [Read more…] about Comparing Regression Lines with Hypothesis Tests

Filed Under: Regression Tagged With: analysis example, conceptual, interpreting results

Identifying the Most Important Independent Variables in Regression Models

By Jim Frost 73 Comments


You’ve settled on a regression model that contains independent variables that are statistically significant. By interpreting the statistical results, you can understand how changes in the independent variables are related to shifts in the dependent variable. At this point, it’s natural to wonder, “Which independent variable is the most important?” [Read more…] about Identifying the Most Important Independent Variables in Regression Models

Filed Under: Regression Tagged With: analysis example, conceptual, interpreting results

Confidence Intervals vs Prediction Intervals vs Tolerance Intervals

By Jim Frost 36 Comments

Intervals are estimation methods in statistics that use sample data to produce ranges of values that are likely to contain the population value of interest. In contrast, point estimates are single value estimates of a population value. Of the different types of statistical intervals, confidence intervals are the most well-known. However, certain kinds of analyses and situations call for other types of ranges that provide different information. [Read more…] about Confidence Intervals vs Prediction Intervals vs Tolerance Intervals

Filed Under: Hypothesis Testing Tagged With: choosing analysis, conceptual

As a Statistician, Can I Say Age is Just a Number?

By Jim Frost 1 Comment

My last birthday wasn’t one of those difficult ages that end with a zero. Thank goodness! However, the passage of another year got me thinking. At that point, I told myself that age is just a number. Can you do a mental double-take? I think I did one. Can a statistician say that age is just a number? After all, it’s through numbers that statisticians understand the world and how it works. [Read more…] about As a Statistician, Can I Say Age is Just a Number?

Filed Under: Fun Tagged With: conceptual

Using Data Mining to Select Regression Models Can Create Serious Problems

By Jim Frost 13 Comments


Data mining and regression seem to go together naturally. I’ve described regression as a seductive analysis because it is so tempting and so easy to add more variables in the pursuit of a larger R-squared. In this post, I’ll begin by illustrating the problems that data mining creates. To do this, I’ll show how data mining with regression analysis can take randomly generated data and produce a misleading model that appears to have significant variables and a good R-squared. Then, I’ll explain how data mining creates these deceptive results and how to avoid them. [Read more…] about Using Data Mining to Select Regression Models Can Create Serious Problems

Filed Under: Regression Tagged With: analysis example, conceptual, interpreting results

Five Reasons Why Your R-squared can be Too High

By Jim Frost 19 Comments

When your regression model has a high R-squared, you assume it’s a good thing. You want a high R-squared, right? However, as I’ll show in this post, a high R-squared can occasionally indicate that there is a problem with your model. I’ll explain five reasons why your R-squared can be too high and how to determine whether one of them affects your regression model. [Read more…] about Five Reasons Why Your R-squared can be Too High

Filed Under: Regression Tagged With: conceptual

Five P Value Tips to Avoid Being Fooled by False Positives and other Misleading Hypothesis Test Results

By Jim Frost 14 Comments

Despite the popular notion to the contrary, understanding the results of your statistical hypothesis test is not as simple as determining only whether your P value is less than your significance level. In this post, I present additional considerations that help you assess and minimize the possibility of being fooled by false positives and other misleading results. [Read more…] about Five P Value Tips to Avoid Being Fooled by False Positives and other Misleading Hypothesis Test Results

Filed Under: Hypothesis Testing Tagged With: conceptual

Overfitting Regression Models: Problems, Detection, and Avoidance

By Jim Frost 58 Comments

Overfitting a model is a condition where a statistical model begins to describe the random error in the data rather than the relationships between variables. This problem occurs when the model is too complex. In regression analysis, overfitting can produce misleading R-squared values, regression coefficients, and p-values. In this post, I explain how overfitting models is a problem and how you can identify and avoid it. [Read more…] about Overfitting Regression Models: Problems, Detection, and Avoidance

Filed Under: Regression Tagged With: conceptual

World Travel, Rough Roads, and Manually Adjusting Graph Scales!

By Jim Frost 2 Comments

As my family and I were being rattled around in a four-wheel drive vehicle in the remote Osa Peninsula in Costa Rica, it struck me that traveling to exotic locations is just like manually adjusting the scales on graphs! That’s probably not what you were expecting, but let me explain! Unlike most of my statistical blog posts, this one gets a bit philosophical! [Read more…] about World Travel, Rough Roads, and Manually Adjusting Graph Scales!

Filed Under: Fun Tagged With: conceptual, graphs

Guide to Stepwise Regression and Best Subsets Regression

By Jim Frost 13 Comments


Automatic variable selection procedures are algorithms that pick the variables to include in your regression model. Stepwise regression and Best Subsets regression are two of the more common variable selection methods. In this post, I compare how these methods work and which one provides better results. [Read more…] about Guide to Stepwise Regression and Best Subsets Regression

Filed Under: Regression Tagged With: analysis example, interpreting results

Goodness-of-Fit Tests for Discrete Distributions

By Jim Frost 23 Comments

Discrete probability distributions are based on discrete variables, which have a finite or countable number of values. In this post, I show you how to perform goodness-of-fit tests to determine how well your data fit various discrete probability distributions. [Read more…] about Goodness-of-Fit Tests for Discrete Distributions

Filed Under: Hypothesis Testing Tagged With: analysis example, distributions, interpreting results

How to Interpret Regression Models that have Significant Variables but a Low R-squared

By Jim Frost 84 Comments


Does your regression model have a low R-squared? That seems like a problem—but it might not be. Learn what a low R-squared does and does not mean for your model. [Read more…] about How to Interpret Regression Models that have Significant Variables but a Low R-squared

Filed Under: Regression Tagged With: conceptual, graphs

How High Does R-squared Need to Be?

By Jim Frost 11 Comments

How high does R-squared need to be in regression analysis? That seems to be an eternal question. [Read more…] about How High Does R-squared Need to Be?

Filed Under: Regression Tagged With: conceptual

Examples of Hypothesis Tests: Busting Myths about the Battle of the Sexes

By Jim Frost 8 Comments

In my house, we love the Mythbusters TV show on the Discovery Channel. The Mythbusters conduct scientific investigations in their quest to test myths and urban legends. In the process, the show provides some fun examples of when and how you should use statistical hypothesis tests to analyze data. [Read more…] about Examples of Hypothesis Tests: Busting Myths about the Battle of the Sexes

Filed Under: Hypothesis Testing Tagged With: analysis example, interpreting results

Making Predictions with Regression Analysis

By Jim Frost 35 Comments

If you were able to make predictions about something important to you, you’d probably love that, right? It’s even better if you know that your predictions are sound. In this post, I show how to use regression analysis to make predictions and determine whether they are both unbiased and precise. [Read more…] about Making Predictions with Regression Analysis

Filed Under: Regression Tagged With: analysis example, assumptions, interpreting results

How to Identify the Distribution of Your Data

By Jim Frost 98 Comments

You’re probably familiar with data that follow the normal distribution. The normal distribution is that nice, familiar bell-shaped curve. Unfortunately, not all data are normally distributed or as intuitive to understand. You can picture the symmetric normal distribution, but what about the Weibull or Gamma distributions? This uncertainty might leave you feeling unsettled. In this post, I show you how to identify the probability distribution of your data. [Read more…] about How to Identify the Distribution of Your Data

Filed Under: Hypothesis Testing Tagged With: distributions, graphs

Curve Fitting using Linear and Nonlinear Regression

By Jim Frost 42 Comments


In regression analysis, curve fitting is the process of specifying the model that provides the best fit to the specific curves in your dataset. Curved relationships between variables are not as straightforward to fit and interpret as linear relationships. [Read more…] about Curve Fitting using Linear and Nonlinear Regression

Filed Under: Regression Tagged With: analysis example, assumptions, conceptual, interpreting results

Interpreting P values

By Jim Frost 96 Comments

P values determine whether your hypothesis test results are statistically significant. Statistics use them all over the place. You’ll find P values in t-tests, distribution tests, ANOVA, and regression analysis. P values have become so important that they’ve taken on a life of their own. They can determine which studies are published, which projects receive funding, and which university faculty members become tenured!

Ironically, despite being so influential, P values are misinterpreted very frequently. What is the correct interpretation of P values? What do P values really mean? That’s the topic of this post! [Read more…] about Interpreting P values

Filed Under: Hypothesis Testing Tagged With: conceptual, interpreting results

How To Interpret R-squared in Regression Analysis

By Jim Frost 125 Comments

R-squared is a goodness-of-fit measure for linear regression models. This statistic indicates the percentage of the variance in the dependent variable that the independent variables explain collectively. R-squared measures the strength of the relationship between your model and the dependent variable on a convenient 0 – 100% scale. [Read more…] about How To Interpret R-squared in Regression Analysis

Filed Under: Regression Tagged With: conceptual, interpreting results

  • « Go to Previous Page
  • Go to page 1
  • Interim pages omitted …
  • Go to page 9
  • Go to page 10
  • Go to page 11
  • Go to page 12
  • Go to page 13
  • Go to Next Page »

Primary Sidebar

Meet Jim

I’ll help you intuitively understand statistics by focusing on concepts and using plain English so you can concentrate on understanding your results.

Read More...

Buy My Introduction to Statistics Book!

Cover of my Introduction to Statistics: An Intuitive Guide ebook.

Buy My Hypothesis Testing Book!

Cover image of my Hypothesis Testing: An Intuitive Guide ebook.

Buy My Regression Book!

Cover for my ebook, Regression Analysis: An Intuitive Guide for Using and Interpreting Linear Models.

Subscribe by Email

Enter your email address to receive notifications of new posts by email.

    I won't send you spam. Unsubscribe at any time.

    Follow Me

    • FacebookFacebook
    • RSS FeedRSS Feed
    • TwitterTwitter
    • Popular
    • Latest
    Popular
    • How To Interpret R-squared in Regression Analysis
    • How to Interpret P-values and Coefficients in Regression Analysis
    • Measures of Central Tendency: Mean, Median, and Mode
    • Normal Distribution in Statistics
    • Multicollinearity in Regression Analysis: Problems, Detection, and Solutions
    • How to Interpret the F-test of Overall Significance in Regression Analysis
    • Understanding Interaction Effects in Statistics
    Latest
    • Cronbach’s Alpha: Definition, Calculations & Example
    • Cohens D: Definition, Using & Examples
    • Statistical Inference: Definition, Methods & Example
    • T Distribution: Definition & Uses
    • Representative Sample: Definition, Uses & Methods
    • Difference Between Standard Deviation and Standard Error
    • How to Find the P value: Process and Calculations

    Recent Comments

    • Lia on Multivariate ANOVA (MANOVA) Benefits and When to Use It
    • neaw on Weibull Distribution: Uses, Parameters & Examples
    • Jim Frost on Choosing the Correct Type of Regression Analysis
    • 1991 on Choosing the Correct Type of Regression Analysis
    • Eero on Making Predictions with Regression Analysis

    Copyright © 2022 · Jim Frost · Privacy Policy