Probability Definition and Fundamentals

What is Probability?

The definition of probability is the likelihood of an event happening. Probability theory analyzes the chances of events occurring. You can think of probabilities as being the following:

• The long-term proportion of times an event occurs during a random process.
• The propensity for a particular outcome to occur.

Common terms for describing probabilities include likelihood, chances, and odds.

For example, we’re all familiar with flipping a coin and that the chances of getting a “heads” is 0.5. We can apply that to a single coin flip or consider it to be the long-term proportion of flipping coins many times. We’d expect 50% of all coin flips to produce heads, and there is a 50% chance the next coin flip will be heads.

Probability values range from 0 to 1. Zero indicates that the event cannot happen while one represents an event that is guaranteed to happen. Values between 0 and 1 denote uncertainty over whether the event will occur. As the likelihood increases, the event becomes more likely. The middle value of 0.5 signifies that the event is equally likely to happen or not. In a coin flip, the probability of heads occurring equals the likelihood of it not occurring (tails).

In this post, I describe real-world uses for probabilities, show how to calculate them, and provide an overview of the two probability theory branches.

Real Life Examples of Using Probability Theory

What are the chances of that occurring?! Have you ever asked yourself that after an unusual occurrence? You can use probabilities in many facets of your personal life. What are the chances of winning the lottery or being in a car accident? Are you more likely to be hit by lightening or winning the lottery? Does wearing a seatbelt change the probability of being injured? How likely is it that you’ll become pregnant?

Risks are the chances of bad events happening, and modeling them is crucial for planning. Actuarial sciences and financial analysts need to understand the likelihood associated with risks to plan for them. Governments use probabilities to know how likely adverse events are to occur and to plan accordingly. How often do catastrophic floods or hurricanes happen in a particular area? What is the likelihood of flood water exceeding a particular level?

Manufacturers need to understand the probability of their products’ failure over time to avoid unhappy customers and determine their warranties’ lengths. Have you ever had a warranty expire just before a product failed? That’s no coincidence! Famously, you can use probability theory to help you win games of chance. Unfortunately for gamblers, casinos use probabilities to ensure they’ll make profits. The house always wins in the long run!

Statistical hypothesis testing uses probabilities to help you evaluate hypotheses relevant to your study. P-values are a well-known type of likelihood, and they allow you to determine whether your results are statistically significant. Is the likelihood of contracting the flu lower if you are vaccinated? Probabilities are an integral part of experiments and statistical analyses.

How to Find Probabilities

For this post, I’ll show you how to calculate simple probabilities to help you understand the fundamentals. Other posts cover more complex cases. For now, we’ll look at independent random events where the occurrence of an event, or lack thereof, does not affect future probabilities. For example, the outcome of one coin toss does not affect the outcome of future coin flips. Learn more about Independent Events.

At its most basic, a probability of an event occurring equals the following:

The numerator equals the number of ways an event can occur. We define what counts as an event based on our interests. For example, we can choose to consider heads in a coin toss or drawing a king from a deck of cards as events. If we define an event as rolling a 1 or 6 on a die, there are two ways an event occurs.

The denominator represents the number of possible outcomes. The subject matter defines this value. For example, coin tosses can have only two results, heads or tails. There are 52 cards in a standard deck of cards. Each outcome is mutually exclusive from the others.

The law of large numbers states that as the number of trials (i.e., coin flips, rolls of the die, drawing cards, etc.) increases, the observed proportion will converge on the expected probability.

Here are some of the other methods for finding probabilities:

• Using Contingency Tables to Calculate Probabilities
• Using the Multiplication Rule to Calculate Probabilities
• Calculating Conditional Probabilities
• Calculating Joint Probabilities
• Using Permutations to Calculate Probabilities
• Using Combinations to Calculate Probabilities
• Odds and Odds Ratios
• Relative Risk

You can even use Pascal’s triangle to find the number of combinations!

Calculating Probability

Let’s start simple with a coin toss and define heads as the single outcome that counts as an event. There is only one way an event can occur and there are two possible outcomes.

P(H) = 1/2 = 0.5.

I wrote that using standard notation and it indicates that the likelihood of heads equals 0.5.

Now, let’s calculate the probabilities for rolling a die. We’ll find the likelihood of rolling a 6, a 1 or a 6, and rolling an even number. Notice how each example changes the number of outcomes that count as an event in the numerator. For a standard die, there are always six potential outcomes. Consequently, the denominator is always 6.

• P(6) = 1/6 = 0.167
• P(1 or 6) = 2/6 = 0.33
• P(Even) = 3/6 = 0.50

Finally, we’ll calculate likelihoods for a randomized, full deck of cards. What’s are the chances of drawing any card with a heart (H), any king (K), and a king of hearts (KH)? In a full deck, there are 52 cards, as indicated in the denominator.

• P(H) = 13/52 = 0.25
• P(K) = 4/52 = 0.077
• P(KH) = 1/52 = 0.019

However, these chances only apply to the first draw from a full deck. Any card we remove affects the likelihood of the next card. Drawing successive cards from a deck are not independent events like coin tosses and dice rolls.

Related posts: Using the Binomial Distribution to Calculate Probabilities

Two Branches of Probability Theory

The previous probability calculations are fairly simple and occur under very controlled settings. Unfortunately, real-world applications for them are often not so nice and neat as flipping coins! Some questions can be rather complex and yield surprising results, such as the Monty Hall Problem and the Birthday Problem.

While I won’t cover how to calculate more complex cases in this post, I want you to know about two broad branches.

Objectivists/Frequentists

Objectivists numerically calculate likelihoods for objective conditions. Frequentist probability is the most common form you’ll run into and it forms the basis for statistical hypothesis tests. In this branch, the likelihood of a random event defines the relative frequency of their occurrence in experiments if you were to repeat an experiment many times. In other words, probabilities are long-run frequencies of outcomes.

Frequentist methodologies provide guidance for applying mathematical probability theory to real-world situations. They offer distinct guidance in the construction and design of practical experiments and evaluating competing hypotheses. Objectivists consider probabilities to be long-run proportion that you can calculate only by using repeated observations in experiments.

Related post: Relative Frequencies and Their Distributions

Subjectivists/Bayesians

Subjectivists incorporate beliefs into their probabilities. The most common form is Bayesian probabilities. This branch includes expert opinions with experimental data to produce likelihoods. Ideally, the expert opinions contain all known information about the subject matter. When combined with experimental data, the process creates a posterior probability distribution. This distribution defines the chances for a particular outcome. Subjectivists are more flexible about what they consider a probability. For example, they can use non-experimental data to calculate probabilities for a singular event, such as the outcome of an election.

Unsurprisingly, there are tradeoffs between these approaches. Objectivists do not rely on opinion but their results can exclude relevant known information. On the other hand, subjectivists incorporate a degree of belief, but their analyses can include different types of information that affect the outcome. Frequentist and Bayesian approaches are the broad divisions in statistics for testing hypotheses by incorporating probabilities. Each methodology has its ardent supporters.

Venn diagrams are an effective way to present complex probabilities.