• Skip to secondary menu
  • Skip to main content
  • Skip to primary sidebar
  • My Store
  • Glossary
  • Home
  • About Me
  • Contact Me

Statistics By Jim

Making statistics intuitive

  • Graphs
  • Basics
  • Hypothesis Testing
  • Regression
  • ANOVA
  • Probability
  • Time Series
  • Fun

Probability Definition and Fundamentals

By Jim Frost 10 Comments

What is Probability?

The definition of probability is the likelihood of an event happening. Probability theory analyzes the chances of events occurring. You can think of probabilities as being the following:

  • The long-term proportion of times an event occurs during a random process.
  • The propensity for a particular outcome to occur.

Common terms for describing probabilities include likelihood, chances, and odds.

Photograph of a roulette wheel to illustrate probability theory.For example, we’re all familiar with flipping a coin and that the chances of getting a “heads” is 0.5. We can apply that to a single coin flip or consider it to be the long-term proportion of flipping coins many times. We’d expect 50% of all coin flips to produce heads, and there is a 50% chance the next coin flip will be heads.

Probability values range from 0 to 1. Zero indicates that the event cannot happen while one represents an event that is guaranteed to happen. Values between 0 and 1 denote uncertainty over whether the event will occur. As the likelihood increases, the event becomes more likely. The middle value of 0.5 signifies that the event is equally likely to happen or not. In a coin flip, the probability of heads occurring equals the likelihood of it not occurring (tails).

In this post, I describe real-world uses for probabilities, show how to calculate them, and provide an overview of the two probability theory branches.

Real Life Examples of Using Probability Theory

What are the chances of that occurring?! Have you ever asked yourself that after an unusual occurrence? You can use probabilities in many facets of your personal life. What are the chances of winning the lottery or being in a car accident? Are you more likely to be hit by lightening or winning the lottery? Does wearing a seatbelt change the probability of being injured? How likely is it that you’ll become pregnant?

Risks are the chances of bad events happening, and modeling them is crucial for planning. Actuarial sciences and financial analysts need to understand the likelihood associated with risks to plan for them. Governments use probabilities to know how likely adverse events are to occur and to plan accordingly. How often do catastrophic floods or hurricanes happen in a particular area? What is the likelihood of flood water exceeding a particular level?

Manufacturers need to understand the probability of their products’ failure over time to avoid unhappy customers and determine their warranties’ lengths. Have you ever had a warranty expire just before a product failed? That’s no coincidence! Famously, you can use probability theory to help you win games of chance. Unfortunately for gamblers, casinos use probabilities to ensure they’ll make profits. The house always wins in the long run!

Statistical hypothesis testing uses probabilities to help you evaluate hypotheses relevant to your study. P-values are a well-known type of likelihood, and they allow you to determine whether your results are statistically significant. Is the likelihood of contracting the flu lower if you are vaccinated? Probabilities are an integral part of experiments and statistical analyses.

How to Find Probabilities

For this post, I’ll show you how to calculate simple probabilities to help you understand the fundamentals. Other posts cover more complex cases. For now, we’ll look at independent random events where the occurrence of an event, or lack thereof, does not affect future probabilities. For example, the outcome of one coin toss does not affect the outcome of future coin flips.

At its most basic, a probability of an event occurring equals the following:

Probability (Event) = {\displaystyle \frac {{\text{Number of ways event occurs}}}{{\text{Total number of outcomes}}}}

The numerator equals the number of ways an event can occur. We define what counts as an event based on our interests. For example, we can choose to consider heads in a coin toss or drawing a king from a deck of cards as events. If we define an event as rolling a 1 or 6 on a die, there are two ways an event occurs.

The denominator represents the number of possible outcomes. The subject matter defines this value. For example, coin tosses can have only two results, heads or tails. There are 52 cards in a standard deck of cards. Each outcome is mutually exclusive from the others.

The law of large numbers states that as the number of trials (i.e., coin flips, rolls of the die, drawing cards, etc.) increases, the observed proportion will converge on the expected probability.

Here are some of the other methods for finding probabilities:

  • Using Contingency Tables to Calculate Probabilities
  • Using the Multiplication Rule to Calculate Probabilities
  • Calculating Conditional Probabilities
  • Using Permutations to Calculate Probabilities
  • Using Combinations to Calculate Probabilities
  • Odds and Odds Ratios
  • Relative Risk

You can even use Pascal’s triangle to find the number of combinations!

Calculating Probability

Photo of a coin toss to represent probabilities of heads occurring.

Let’s start simple with a coin toss and define heads as the single outcome that counts as an event. There is only one way an event can occur and there are two possible outcomes.

P(H) = 1/2 = 0.5.

I wrote that using standard notation and it indicates that the likelihood of heads equals 0.5.

Photo of a die for examples of probabilities of different outcomes.Now, let’s calculate the probabilities for rolling a die. We’ll find the likelihood of rolling a 6, a 1 or a 6, and rolling an even number. Notice how each example changes the number of outcomes that count as an event in the numerator. For a standard die, there are always six potential outcomes. Consequently, the denominator is always 6.

  • P(6) = 1/6 = 0.167
  • P(1 or 6) = 2/6 = 0.33
  • P(Even) = 3/6 = 0.50

King of hearts card to represent probabilities of drawing cards from a deck of cards.Finally, we’ll calculate likelihoods for a randomized, full deck of cards. What’s are the chances of drawing any card with a heart (H), any king (K), and a king of hearts (KH)? In a full deck, there are 52 cards, as indicated in the denominator.

  • P(H) = 13/52 = 0.25
  • P(K) = 4/52 = 0.077
  • P(KH) = 1/52 = 0.019

However, these chances only apply to the first draw from a full deck. Any card we remove affects the likelihood of the next card. Drawing successive cards from a deck are not independent events like coin tosses and dice rolls.

Related posts: Using the Binomial Distribution to Calculate Probabilities

Two Branches of Probability Theory

The previous probability calculations are fairly simple and occur under very controlled settings. Unfortunately, real-world applications for them are often not so nice and neat as flipping coins! Some questions can be rather complex and yield surprising results, such as the Monty Hall Problem and the Birthday Problem.

While I won’t cover how to calculate more complex cases in this post, I want you to know about two broad branches.

Objectivists/Frequentists

Objectivists numerically calculate likelihoods for objective conditions. Frequentist probability is the most common form you’ll run into and it forms the basis for statistical hypothesis tests. In this branch, the likelihood of a random event defines the relative frequency of their occurrence in experiments if you were to repeat an experiment many times. In other words, probabilities are long-run frequencies of outcomes.

Frequentist methodologies provide guidance for applying mathematical probability theory to real-world situations. They offer distinct guidance in the construction and design of practical experiments and evaluating competing hypotheses. Objectivists consider probabilities to be long-run proportion that you can calculate only by using repeated observations in experiments.

Related post: Relative Frequencies and Their Distributions

Subjectivists/Bayesians

Subjectivists incorporate beliefs into their probabilities. The most common form is Bayesian probabilities. This branch includes expert opinions with experimental data to produce likelihoods. Ideally, the expert opinions contain all known information about the subject matter. When combined with experimental data, the process creates a posterior probability distribution. This distribution defines the chances for a particular outcome. Subjectivists are more flexible about what they consider a probability. For example, they can use non-experimental data to calculate probabilities for a singular event, such as the outcome of an election.

Unsurprisingly, there are tradeoffs between these approaches. Objectivists do not rely on opinion but their results can exclude relevant known information. On the other hand, subjectivists incorporate a degree of belief, but their analyses can include different types of information that affect the outcome. Frequentist and Bayesian approaches are the broad divisions in statistics for testing hypotheses by incorporating probabilities. Each methodology has its ardent supporters.

Venn diagrams are an effective way to present complex probabilities.

Share this:

  • Tweet

Related

Filed Under: Probability Tagged With: conceptual

Reader Interactions

Comments

  1. Kenneth Mazhindu says

    April 15, 2021 at 4:06 am

    Thank you so much for your elaborate explanation of statistical concepts. I always find joy in going through your publications.

    Reply
    • Jim Frost says

      April 15, 2021 at 4:50 pm

      Hi Kenneth, thanks so much! I’m glad it’s been helpful!

      Reply
  2. Charles says

    February 4, 2021 at 7:03 am

    I like the simplicity in your explanation and looking forward to getting one on survival analysis of a time to event nature

    Reply
    • Jim Frost says

      February 5, 2021 at 10:34 pm

      Hi Charles, thanks so much! I do have one post about survival analysis. It was written by a guest author.

      Reply
  3. Peter says

    February 2, 2021 at 7:59 am

    Thanks

    Reply
  4. Rai says

    February 2, 2021 at 3:26 am

    I always look forward to your articles. I like how you clearly explain the concepts. Looking forward to part 2

    Reply
  5. Ali says

    February 1, 2021 at 2:07 am

    Thank you and waiting for PART 2 of the concept

    Reply
  6. Niyungeko Antoine says

    February 1, 2021 at 12:39 am

    Thanks.

    Reply
  7. Dr. Manish Shrivastava says

    January 31, 2021 at 11:56 pm

    Thank you, a great piece of clear concept

    Reply
  8. Khursheed Ahmad says

    January 31, 2021 at 11:32 pm

    Thnks a lot sir ____was a valuable piece to clear basics # Probability .
    # will b waiting fr next post relating to probability .

    Reply

Comments and Questions Cancel reply

Primary Sidebar

Meet Jim

I’ll help you intuitively understand statistics by focusing on concepts and using plain English so you can concentrate on understanding your results.

Read More...

Buy My Introduction to Statistics Book!

Cover of my Introduction to Statistics: An Intuitive Guide ebook.

Buy My Hypothesis Testing Book!

Cover image of my Hypothesis Testing: An Intuitive Guide ebook.

Buy My Regression Book!

Cover for my ebook, Regression Analysis: An Intuitive Guide for Using and Interpreting Linear Models.

Subscribe by Email

Enter your email address to receive notifications of new posts by email.

    I won't send you spam. Unsubscribe at any time.

    Follow Me

    • FacebookFacebook
    • RSS FeedRSS Feed
    • TwitterTwitter

    Top Posts

    • How to Interpret P-values and Coefficients in Regression Analysis
    • How To Interpret R-squared in Regression Analysis
    • Multicollinearity in Regression Analysis: Problems, Detection, and Solutions
    • Mean, Median, and Mode: Measures of Central Tendency
    • How to Find the P value: Process and Calculations
    • How to do t-Tests in Excel
    • Z-table
    • Choosing the Correct Type of Regression Analysis
    • One-Tailed and Two-Tailed Hypothesis Tests Explained
    • How to Interpret the F-test of Overall Significance in Regression Analysis

    Recent Posts

    • Slope Intercept Form of Linear Equations: A Guide
    • Population vs Sample: Uses and Examples
    • How to Calculate a Percentage
    • Control Chart: Uses, Example, and Types
    • Monte Carlo Simulation: Make Better Decisions
    • Principal Component Analysis Guide & Example

    Recent Comments

    • Jim Frost on Monte Carlo Simulation: Make Better Decisions
    • Gilberto on Monte Carlo Simulation: Make Better Decisions
    • Sultan Mahmood on Linear Regression Equation Explained
    • Sanjay Kumar P on What is the Mean and How to Find It: Definition & Formula
    • Dave on Control Variables: Definition, Uses & Examples

    Copyright © 2023 · Jim Frost · Privacy Policy