The Probability Problem
The Idea
Probabilities aren’t always probabilities.
Probabilities are deeply embedded in our everyday lives. From the weather to business results, sports outcomes, elections or personal life decisions, all the way down to the fundamental layer of reality (quantum mechanics) — probabilities govern much of how the world unfolds in front of us. In other words, our world is intrinsically probabilistic; nothing is certain, a lot can happen, nothing has to.
And yet, while we colloquially speak of probabilities or likelihoods all the time, we commonly misunderstand their true nature and draw ill-informed conclusions.
Probabilities can be looked at from different angles (maths, statistics, physics, economics, philosophy, ...) but the following three major distinctions are essential to understand if one wants to not be fooled by probability statements and make better decisions.
Probabilities based on objective facts about the world
The probability of rolling a three with a 6-sided die
The probability of flipping heads with a 2-sided coin
The probability of pulling the Queen of Spades from a deck with 52 cards
...
In these cases, there exist current real world features determining an objective likelihood one can calculate if full information about these features is available (i.e. that the die is fair and has six sides).
Probabilities based on objective historic events
The probability of the stock market gaining more than 10% in any given year
The probability of a tech startup surviving its first five years
The probability of Bayern Munich winning the German Bundesliga in any given year
...
In these cases, there don’t exist current real world features, but reliable historic real world frequency data that we can use as a proxy for probabilities.
Note that in these cases, there exists no objective current probability that one can calculate, but one can extrapolate from past into future. Additionally, the more additional information one can gather about the current state (i.e. the state of the economy or the fitness of the sports team), the better one can adjust their probability estimations.
“Probabilities” based on subjective assumptions and personal beliefs
The probability of you making a good career at a particular company
The probability of a business strategy yielding a profit
The probability of a specific political situation leading to war
…
In these cases, there neither exist current real world features, nor reliable historic real world frequency data one can use a proxy. Instead, one forms opinions based on observations, personal experience, assumptions and subjective beliefs.
Here, additional information about the state of the environment will also lead to adjusted assumptions and thus different probability estimates.
While the nature of probabilities in the first case can be defined straightforwardly, in cases two and three, the lines are a lot more blurry and using the term “probability” can mislead us to assume that we have objective knowledge about the nature of reality that we in fact don’t.
In maths and statistics, probability science quickly becomes highly complex, nuanced and often counter-intuitive, especially when conditional probabilities come into play.
In philosophy and the foundations of physics, there exist a debate of whether probabilities truly exist at all at a fundamental level or if they always just point to our lack of information in different contexts.
Regardless of the true mystery of probabilities, however, understanding the conceptual distinctions between the three cases above helps us make better decisions in uncertain situations and guards us against misguided predictions.
“Life is a school of probability.”
— Walter Bagehot
“Human history is highly nonlinear and unpredictable.”
— Michael Shermer
“People don't realize that we cannot forecast the future. What we can do is have probabilities of what causes what, but that's as far as we go.”
— Alan Greenspan
“Probability fractions arise from our knowledge and from our ignorance.”
— Ian Hacking
Go Deep
🎥 Watch this math-based introduction to types of probabilities.
🎥 Watch this long World Science Festival event on the counter-intuitive science of probabilities.
📝 For an overview of even more types of probabilities check out this article.
📝 Check out the Wikipedia entries on probability and probability theory to get a sense for the vastness of this topic across fields.
Go Beyond
A few further resources you might like if you find above idea interesting:
📚 Nassim Taleb’s The Black Swan
📚 Nassim Taleb’s Fooled By Randomness
📝 MindVault: The Plausibility Bias
📝 MindVault: The Black Swan Theory
📝 MindVault: Chaos Theory
📝 MindVault: Base Rate Error
📝 MindVault: Bayesian Thinking
↓
If you find these weekly ideas useful, consider sharing the MindVault Letter with those who enjoy deep thoughts like you do.