• Skip to secondary menu
  • Skip to main content
  • Skip to primary sidebar
  • My Store
  • Glossary
  • Home
  • About Me
  • Contact Me

Statistics By Jim

Making statistics intuitive

  • Graphs
  • Basics
  • Hypothesis Testing
  • Regression
  • ANOVA
  • Probability
  • Time Series
  • Fun

Predictive Validity: Definition, Assessing & Examples

By Jim Frost Leave a Comment

What is Predictive Validity?

Predictive validity is the degree to which a test score or construct scale predicts a criterion variable measuring a future outcome, behavior, or performance. Evaluating predictive validity involves assessing the correlation between the pre-test score and the subsequent criterion outcome.

Researchers frequently assess predictive validity in psychology, job performance, and academic settings.

Woman with crystal ball who needs to assess predictive validity.
No, not this type of prediction!

Examples of predictive validity include the following:

  • Psychological personality inventory that predicts future behaviors.
  • Survey of risk factors that predicts chances of getting a disease.
  • Pre-hire test assessment that predicts job performance after a year.
  • SAT and GRE scores that predict future grade point averages (GPA).

In all these predictive validity examples, you have a preliminary assessment followed by a completely independent assessment that uses a different measurement system (e.g., SAT scores → GPAs).

Additionally, each example focuses on an extremely specific criterion the study defines. For example, the researchers might correlate SAT scores to college freshman GPA. For job performance, HR needs to devise a meaningful performance measure they can correlate with the pre-hire test.

The designers of the initial assessment typically intend to use its predictive ability to improve decision-making about things such as health choices, hiring decisions, and student selection. Consequently, evaluating its effectiveness is essential. For example, if you’re developing a survey of heart disease risk factors to help individuals determine and reduce their risk, it’s crucial that the instrument actually predicts heart disease outcomes.

If the assessment instrument has low predictive validity, it can lead people to make the wrong decisions.

Predictive validity is one of three subtypes of criterion validity. Criterion validity evaluates how well a test score correlates with an outcome measured in the past (retrospective), present (concurrent), or future (predictive).

Learn more about Validity in Research: Types and Examples, Criterion Validity, and Concurrent Validity.

Evaluating Predictive Validity

Evaluating predictive validity involves a pre-test that investigators measure before the outcome occurs. The outcome itself is measured at a later point using a different assessment process than the pre-test. Then analysts find the correlation between the two measures.

A test has predictive validity when a large correlation exists between test scores and the criterion variable. In a general sense, higher absolute correlations are better. A near zero correlation indicates that the assessment instrument does not predict the outcome—it has poor predictive validity.

The correlation can be either highly positive or negative. Other sources indicate only highly positive correlations are acceptable, but highly negative correlations are also predictive. However, positive correlations are more common.

Learn more about Interpreting Correlation Coefficients.

The question you’re probably asking is, how strong of a correlation does it require?

Subject-area knowledge provides the answer. Compare the correlation magnitude to similar studies and incorporate contextual information to determine whether it represents satisfactory predictive validity. In the next section, I cover an example using SAT scores showing how to do that.

Be aware that many social science and psychological predictive validity assessments produce low correlations—frequently less than r < 0.5, accounting for only 25% of the variance (R2). Predicting human behavior is difficult! Consequently, a single test often produces lackluster results when it tries to predict something as complex and multifaceted as job performance or Freshman GPA.

Finally, consider how a selection process might restrict a sample’s range of test scores. Suppose a selection process chooses people only with SAT or pre-hire test scores greater than a specific value. In that case, your sample will be more homogenous than the general population. Range restriction reduces the correlation with the criterion, making predictive validity appear worse than it actually is.

Predictive Validity Example—SAT Scores

Let’s assess the predictive validity of using SAT scores to predict first-year of college GPAs. The SAT is the pre-test, and freshman GPAs are the criterion variable. College admissions use SAT scores as part of the selection process.

Studies consistently find SAT scores by themselves have an r ≈ 0.50 correlation with first year of college GPAs. They account for only 25% of the variability in GPAs. However, researchers consider SATs as having high predictive validity because of the context.

Consider that SATs are taken on a single day during high school and attempt to predict first-year GPAs in an entirely different college setting. This single test taken at one point in time predicts Freshman GPAs as effectively as four years of cumulative high school GPAs!

Reference

Validity of the SAT® for Predicting First-Year Grades and Retention to the Second Year (collegeboard.org)

Share this:

  • Tweet

Related

Filed Under: Basics Tagged With: conceptual, experimental design

Reader Interactions

Comments and Questions Cancel reply

Primary Sidebar

Meet Jim

I’ll help you intuitively understand statistics by focusing on concepts and using plain English so you can concentrate on understanding your results.

Read More...

Buy My Introduction to Statistics Book!

Cover of my Introduction to Statistics: An Intuitive Guide ebook.

Buy My Hypothesis Testing Book!

Cover image of my Hypothesis Testing: An Intuitive Guide ebook.

Buy My Regression Book!

Cover for my ebook, Regression Analysis: An Intuitive Guide for Using and Interpreting Linear Models.

Subscribe by Email

Enter your email address to receive notifications of new posts by email.

    I won't send you spam. Unsubscribe at any time.

    Follow Me

    • FacebookFacebook
    • RSS FeedRSS Feed
    • TwitterTwitter

    Top Posts

    • How to Interpret P-values and Coefficients in Regression Analysis
    • How To Interpret R-squared in Regression Analysis
    • Multicollinearity in Regression Analysis: Problems, Detection, and Solutions
    • Mean, Median, and Mode: Measures of Central Tendency
    • How to Find the P value: Process and Calculations
    • How to do t-Tests in Excel
    • Z-table
    • One-Tailed and Two-Tailed Hypothesis Tests Explained
    • Choosing the Correct Type of Regression Analysis
    • How to Interpret the F-test of Overall Significance in Regression Analysis

    Recent Posts

    • Slope Intercept Form of Linear Equations: A Guide
    • Population vs Sample: Uses and Examples
    • How to Calculate a Percentage
    • Control Chart: Uses, Example, and Types
    • Monte Carlo Simulation: Make Better Decisions
    • Principal Component Analysis Guide & Example

    Recent Comments

    • Jim Frost on Monte Carlo Simulation: Make Better Decisions
    • Gilberto on Monte Carlo Simulation: Make Better Decisions
    • Sultan Mahmood on Linear Regression Equation Explained
    • Sanjay Kumar P on What is the Mean and How to Find It: Definition & Formula
    • Dave on Control Variables: Definition, Uses & Examples

    Copyright © 2023 · Jim Frost · Privacy Policy