Making statistics intuitive
Weighted least squares (WLS) is a type of linear regression that assigns different weights to each data point when fitting the model. Instead of minimizing the simple residual sum of squares as ordinary least squares (OLS) does, WLS minimizes the weighted (wi) sum of squared residuals as the summation symbol below indicates:
Multinomial logistic regression statistically models the probabilities of at least three categorical outcomes that do not have a natural order. This technique uses a linear combination of independent variables to explore correlations with outcome likelihoods and to predict outcomes using specific input conditions. This analysis is also known as nominal logistic regression. [Read more…] about Multinomial Logistic Regression: Overview & Example
Logistic regression statistically models the probabilities of categorical outcomes, which can be binary (two possible values) or have more than two categories. These models use a linear combination of independent variables to help you understand how they correlate with the likelihood of the outcomes and predict them based on specific conditions you enter into the model. [Read more…] about Logistic Regression Overview with Example
Poisson regression statistically models events that you count within a specified observation space. Frequently, analysts define the observation space using time, but it can also relate to a volume, area, or item. These models allow you to understand the independent variables that affect the counts and predict them given specific conditions you enter into the model. [Read more…] about Poisson Regression Analysis Overview with Example
The residual sum of squares (RSS) measures the difference between your observed data and the model’s predictions. It is the portion of variability your regression model does not explain, also known as the model’s error. Use RSS to evaluate how well your model fits the data. [Read more…] about Residual Sum of Squares (RSS) Explained
Omitted variable bias (OVB) occurs when a regression model excludes a relevant variable. The absence of these critical variables can skew the estimated relationships between variables in the model, potentially leading to erroneous interpretations. This bias can exaggerate, mask, or entirely flip the direction of the estimated relationship between an independent and dependent variable. [Read more…] about Omitted Variable Bias: Definition, Avoiding & Example
A parsimonious model in statistics is one that uses relatively few independent variables to obtain a good fit to the data. [Read more…] about What is a Parsimonious Model? Benefits and Selecting
The sum of squares (SS) is a statistic that measures the variability of a dataset’s observations around the mean. It’s the cumulative total of each data point’s squared difference from the mean. [Read more…] about Sum of Squares: Definition, Formula & Types
The root mean square error (RMSE) measures the average difference between a statistical model’s predicted values and the actual values. Mathematically, it is the standard deviation of the residuals. Residuals represent the distance between the regression line and the data points. [Read more…] about Root Mean Square Error (RMSE)
An ordinary least squares regression line represents the relationship between variables in a scatterplot. The procedure fits the line to the data points in a way that minimizes the sum of the squared vertical distances between the line and the points. It is also known as a line of best fit or a trend line. [Read more…] about Ordinary Least Squares Regression: Definition, Formulas & Example
A linear regression equation describes the relationship between the independent variables (IVs) and the dependent variable (DV). It can also predict new values of the DV for the IV values you specify. [Read more…] about Linear Regression Equation Explained
Linear regression models the relationships between at least one explanatory variable and an outcome variable. This flexible analysis allows you to separate the effects of complicated research questions, allowing you to isolate each variable’s role. Additionally, linear models can fit curvature and interaction effects. [Read more…] about Linear Regression Explained with Examples
Mean squared error (MSE) measures the amount of error in statistical models. It assesses the average squared difference between the observed and predicted values. When a model has no error, the MSE equals zero. As model error increases, its value increases. The mean squared error is also known as the mean squared deviation (MSD). [Read more…] about Mean Squared Error (MSE)
Orthogonality is a mathematical property that is beneficial for statistical models. It’s particularly helpful when performing factorial analysis of designed experiments. [Read more…] about Orthogonal: Models, Definition & Finding
Independent variables and dependent variables are the two fundamental types of variables in statistical modeling and experimental designs. Analysts use these methods to understand the relationships between the variables and estimate effect sizes. What effect does one variable have on another?
In this post, learn the definitions of independent and dependent variables, how to identify each type, how they differ between different types of studies, and see examples of them in use. [Read more…] about Independent and Dependent Variables: Differences & Examples
Historians rank the U.S. Presidents from best to worse using all the historical knowledge at their disposal. Frequently, groups, such as C-Span, ask these historians to rank the Presidents and average the results together to help reduce bias. The idea is to produce a set of rankings that incorporates a broad range of historians, a vast array of information, and a historical perspective. These rankings include informed assessments of each President’s effectiveness, leadership, moral authority, administrative skills, economic management, vision, and so on. [Read more…] about Understanding Historians’ Rankings of U.S. Presidents using Regression Models
Proxy variables are easily measurable variables that analysts include in a model in place of a variable that cannot be measured or is difficult to measure. Proxy variables can be something that is not of any great interest itself, but has a close correlation with the variable of interest. [Read more…] about Proxy Variables: The Good Twin of Confounding Variables
Variance Inflation Factors (VIFs) measure the correlation among independent variables in least squares regression models. Statisticians refer to this type of correlation as multicollinearity. Excessive multicollinearity can cause problems for regression models.
In this post, I focus on VIFs and how they detect multicollinearity, why they’re better than pairwise correlations, how to calculate VIFs yourself, and interpreting VIFs. If you need a refresher about the types of problems that multicollinearity causes and how to fix them, read my post: Multicollinearity: Problems, Detection, and Solutions. [Read more…] about Variance Inflation Factors (VIFs)
Excel can perform various statistical analyses, including regression analysis. It is a great option because nearly everyone can access Excel. This post is an excellent introduction to performing and interpreting regression analysis, even if Excel isn’t your primary statistical software package.
[Read more…] about How to Perform Regression Analysis using Excel
I’m thrilled to announce the release of my first book! Regression Analysis: An Intuitive Guide for Using and Interpreting Linear Models.
If you like the clear writing style I use on this website, you’ll love this book! The end of the post displays the entire table of contents! [Read more…] about New eBook Release! Regression Analysis: An Intuitive Guide