Probability Fundamentals

Probability theory analyzes the likelihood of events occurring. You can think of probabilities as being the following:

• The long-term proportion of times an event occurs during a random process.
• The propensity for a particular outcome to occur.

Common terms for describing probabilities include likelihood, chances, and odds.

For example, we’re all familiar with flipping a coin and that the probability of getting a “heads” is 0.5. We can apply that to a single coin flip or consider it to be the long-term proportion of flipping coins many times. We’d expect 50% of all coin flips to produce heads, and there is a 50% chance the next coin flip will be heads.

Probability values range from 0 to 1. Zero indicates that the event cannot happen while one represents an event that is guaranteed to happen. Values between 0 and 1 denote uncertainty over whether the event will occur. As the probability increases, the event becomes more likely. The middle value of 0.5 signifies that the event is equally likely to happen or not. In a coin flip, the probability of heads occurring equals the likelihood of it not occurring (tails).

In this post, I describe real-world uses for probabilities, show how to calculate probabilities, and provide an overview of the two probability theory branches.

Real Life Examples of Using Probability Theory

What are the chances of that occurring?! Have you ever asked yourself that after an unusual occurrence? You can use probabilities in many facets of your personal life. What are the chances of winning the lottery or being in a car accident? Are you more likely to be hit by lightening or winning the lottery? Does wearing a seatbelt change the probability of being injured? How likely is it that you’ll become pregnant?

Risks are the probabilities of bad events happening, and modeling them is crucial for planning. Actuarial sciences and financial analysts need to understand the likelihood associated with risks to plan for them. Governments use probabilities to know how likely adverse events are to occur and to plan accordingly. How often do catastrophic floods or hurricanes happen in a particular area? What is the probability of flood water exceeding a particular level?

Manufacturers need to understand the probability of their products’ failure over time to avoid unhappy customers and determine their warranties’ lengths. Have you ever had a warranty expire just before a product failed? That’s no coincidence! Famously, you can use probability theory to help you win games of chance. Unfortunately for gamblers, casinos use probabilities to ensure they’ll make profits. The house always wins in the long run!

Statistical hypothesis testing uses probabilities to help you evaluate hypotheses relevant to your study. P-values are a well-known type of probability, and they allow you to determine whether your results are statistically significant. Is the likelihood of contracting the flu lower if you are vaccinated? Probabilities are an integral part of experiments and statistical analyses.

How to Calculate Probabilities

For this post, I’ll show you how to calculate simple probabilities to help you understand the fundamentals. Later posts will cover more complex cases. For now, we’ll look at independent random events where the occurrence of an event, or lack thereof, does not affect future probabilities. For example, the outcome of one coin toss does not affect the outcome of future coin flips.

At its most basic, a probability of an event occurring equals the following:

The numerator equals the number of ways an event can occur. We define what counts as an event based on our interests. For example, we can choose to consider heads in a coin toss or drawing a king from a deck of cards as events. If we define an event as rolling a 1 or 6 on a die, there are two ways an event occurs.

The denominator represents the number of possible outcomes. The subject matter defines this value. For example, coin tosses can have only two results, heads or tails. There are 52 cards in a standard deck of cards. Each outcome is mutually exclusive from the others.

Related post: Using Contingency Tables to Calculate Probabilities

Example Probability Calculations

Let’s start simple with a coin toss and define heads as the single outcome that counts as an event. There is only one way an event can occur and there are two possible outcomes.

P(H) = 1/2 = 0.5.

I wrote that using standard notation and it indicates that the probability of heads equals 0.5.

Now, let’s calculate the probabilities for rolling a die. We’ll find the likelihood of rolling a 6, a 1 or a 6, and rolling an even number. Notice how each example changes the number of outcomes that count as an event in the numerator. For a standard die, there are always six potential outcomes. Consequently, the denominator is always 6.

• P(6) = 1/6 = 0.167
• P(1 or 6) = 2/6 = 0.33
• P(Even) = 3/6 = 0.50

Finally, we’ll calculate probabilities for a randomized, full deck of cards. What’s are the chances of drawing any card with a heart (H), any king (K), and a king of hearts (KH)? In a full deck, there are 52 cards, as indicated in the denominator.

• P(H) = 13/52 = 0.25
• P(K) = 4/52 = 0.077
• P(KH) = 1/52 = 0.019

However, these probabilities only apply to the first draw from a full deck. Any card we remove affects the likelihood of the next card. Drawing successive cards from a deck are not independent events like coin tosses and dice rolls.

Related post: Using the Binomial Distribution to Calculate Probabilities

Two Branches of Probability Theory

The previous probability calculations are fairly simple and occur under very controlled settings. Unfortunately, real-world applications for probabilities are often not so nice and neat as flipping coins! Some probably questions can be rather complex and yield surprising results, such as the Monty Hall Problem and the Birthday Problem.

While I won’t cover how to calculate more complex probabilities in this post, I want you to know about two broad branches of Probability Theory.

Objectivists/Frequentists

Objectivists numerically calculate probabilities for objective conditions. Frequentist probability is the most common form you’ll run into and it forms the basis for statistical hypothesis tests. In this branch, the likelihood of a random event defines the relative frequency of their occurrence in experiments if you were to repeat an experiment many times. In other words, probabilities are long-run frequencies of outcomes.

Frequentist methodologies provide guidance for applying mathematical probability theory to real-world situations. They offer distinct guidance in the construction and design of practical experiments and evaluating competing hypotheses. Objectivists consider probabilities to be long-run proportion that you can calculate only by using repeated observations in experiments.

Subjectivists/Bayesians

Subjectivists incorporate beliefs into their probabilities. The most common form is Bayesian probabilities. This branch includes expert opinions with experimental data to produce probabilities. Ideally, the expert opinions contain all known information about the subject matter. When combined with experimental data, the process creates a posterior probability distribution. This distribution defines the probability for a particular outcome. Subjectivists are more flexible about what they consider a probability. For example, they can use non-experimental data to calculate probabilities for a singular event, such as the outcome of an election.

Unsurprisingly, there are tradeoffs between these approaches. Objectivists do not rely on opinion but their results can exclude relevant known information. On the other hand, subjectivists incorporate a degree of belief, but their analyses can include different types of information that affect the outcome. Frequentist and Bayesian approaches are the broad divisions in statistics for testing hypotheses by incorporating probabilities. Each methodology has its ardent supporters.