Weighted Least Squares (WLS) Explained

What is Weighted Least Squares (WLS)?

Weighted least squares (WLS) is a type of linear regression that assigns different weights to each data point when fitting the model. Instead of minimizing the simple residual sum of squares as ordinary least squares (OLS) does, WLS minimizes the weighted (w_i) sum of squared residuals as the summation symbol below indicates:

Ordinary least squares (OLS) regression is the standard linear regression method. It assigns equal weight to all data points, which is usually a good thing.

However, there are cases where you legitimately need to weight observations differently using weighted least squares (WLS). The most common reason is to address heteroscedasticity, which occurs when the residuals don’t have a constant variance. However, there are several other crucial cases when you need to use WLS. In short, analysts use weighted least squares when treating all observations equally would distort the model.

Fortunately, weighted least squares regression uses the same mechanics as ordinary regression—the intercept, coefficients, t-tests, and R-squared all carry over. However, it adjusts the influence of each data point. Observations with greater precision or importance contribute more to the model’s estimates.

In this post, learn about conditions that weighted least squares can help with and how to choose the weights.

When to Use Weighted Least Squares

Ordinary least squares (OLS) can produce unreliable results when some observations contribute more noise than others. Weighted least squares (WLS) improves model performance in several common situations:

• Heteroscedasticity: When the variance of the residuals changes across levels of a predictor.
• Unequal measurement precision: When instruments measure some observations more precisely than others.
• Disproportionate stratified sampling: When researchers over- or under-sampled certain subgroups in a survey design.
• Sample statistics with varying reliability: When each data point is a summary (like a mean) with its own standard error.

In each of these cases, assigning appropriate weights to the observations in a weighted least squares regression model can correct for the imbalance and lead to more accurate and trustworthy results.

Let’s investigate these various use cases!

Learn more about the Ordinary Least Squares Assumptions.

Correcting Heteroscedasticity

One of the most common reasons for using weighted least squares is to correct for heteroscedasticity, which exists when the variance of the residuals increases or decreases with an independent variable. In these cases, ordinary least squares (OLS) estimates remain unbiased, but the standard errors, p-values, and confidence intervals become unreliable. Weighted least squares improves efficiency by giving more weight to observations with lower residual variance.

Frequently, you’ll see the problematic non-constant residual variance show up in the residual plots for a linear regression model. It commonly produces a characteristic fan shape, as shown below:

In practice, the true structure of the weights is usually unknown, so we need to estimate it. This process often begins by fitting an OLS regression and assessing the residuals. However, individual residuals are too noisy to use directly for weighting. Instead, we use the squared residual for each observation as a rough estimate of the error variance at that point. Alternatively, the absolute residual can serve as an estimate of the standard deviation, which is often more robust in the presence of outliers.

Next, we model how the residual variance or standard deviation changes across observations, typically as a function of one or more predictors. The resulting variance or standard deviation function allows us to assign weights using the rule wᵢ = 1 / σ̂ᵢ²​ for the weighted least squares regression model.

Alternatively, you can try using the inverse of a predictor variable (especially one that spans several orders of magnitude) as a shortcut when it reasonably approximates the variance trend.

Learn more about Heteroscedasticity in Regression Analysis where I use WLS to resolve the residual problem shown above.

Precision Weighting

Weighted least squares regression is also useful when some measurements are more precise than others. In experimental data, researchers might know the level of uncertainty for each measurement.

If you know the measurement precision for each observation, you can weight them by the inverse of their measurement variance, giving more influence to trustworthy values.

Survey Design Weights

Weighted least squares is also valuable when analyzing data from complex surveys. Many large-scale surveys employ stratified sampling with disproportionate selection or cluster sampling, which can result in unequal probabilities of inclusion. If the analysts don’t account for these design features, regression estimates can be biased or unrepresentative.

Survey organizations often provide sampling weights to correct for these unequal probabilities. Weighted least squares regression allows you to incorporate those weights directly into the regression model, preserving the integrity of the survey design and ensuring that the results reflect the target population.

Learn more about Stratified Sampling and Cluster Sampling.

Regressing on Sample Statistics with Unequal Variability

Weighted least squares is also useful when each observation in your dataset is a sample statistic, such as a group mean. In these cases, the precision of each observation varies depending on the group’s internal variability and sample size. Ordinary least squares treats all group means equally, but weighted least squares improves accuracy by giving more weight to the more precise means.

This situation commonly arises in meta-analysis, ecological regressions, and any context where the data summarize groups rather than individuals.

If each observation represents a sample mean, you can calculate weights using the standard error (SE) of the mean:

• If you know the standard deviation (SDᵢ) and sample size (nᵢ) for each group, compute the standard error:
  SEᵢ = SDᵢ / √nᵢ
• Then compute the weight:
  wᵢ = 1 / SEᵢ²

For this situation, weighted least squares uses the standard error of the mean for the weight instead of the standard deviation because the standard error reflects the precision of the group mean. It accounts for both the variability within the group and the size of the group, making it a more appropriate basis for weighting.

Learn more in-depth about the Standard Error vs. Standard Deviation.

Example

Suppose you’re modeling average test scores from different schools based on average class size. Each school provides its mean score, standard deviation, and sample size.

School	Mean Score (Yᵢ)	Class Size (Xᵢ)	SDᵢ	nᵢ	SEᵢ = SDᵢ / √nᵢ	wᵢ = 1 / SEᵢ²
A	78.5	24	12.0	30	2.19	0.208
B	81.2	21	8.0	40	1.26	0.630
C	75.0	30	15.0	25	3.00	0.111

In this example:

• School B provides the most precise estimate (smallest SE), so it receives the largest weight.
• School C has the least precise mean (largest SE), so it contributes less to the regression.

Using these weights in a weighted least squares regression of Mean Score on Class Size results in a model that better reflects the varying reliability of the observations.

Weighted Least Squares Example

Imagine your study is modeling household electricity use (kWh) based on square footage. After fitting an ordinary least squares (OLS) model, the residual plot shows that variability increases with house size. Statisticians refer to this pattern as heteroscedasticity and it violates a key OLS assumption of homoscedasticity. It can lead to unreliable standard errors.

To address this, divide the data into size bands—ranges of square footage—and estimate the variance of the residuals within each band. Then assign each observation a weight equal to the inverse of the estimated variance for its size band. In other words, smaller houses with more consistent electricity usage (lower residual variance) receive higher weights.

Fitting a weighted least squares regression using these weights yields a more efficient model. The slope estimate becomes more precise, and the resulting confidence intervals and p-values are more trustworthy than those from the OLS model.