Think like a Bayesian

Picture this. You’re a married professional man who has just returned home from a four day work trip. As you unbutton and drop your shirt into the laundry hamper, you notice another unfamiliar looking shirt in there. You pause to pick it up and see it’s an Arrow brand shirt, not one you own. What’s going through your mind? Chances are you’re thinking “Is my wife cheating on me?” to be continued…

Traditional statistics or “Frequentism”

Ronald Aylmer Fisher. The name might not ring too many bells but if you’ve taken any basic statistics course, then you’re familiar with his work. He’s responsible for a lot of the statistical methods widely used today, including the core idea of testing for “statistical significance”, meaning whether a particular relationship (e.g. number of cars on the road and level of pollution in ppm) is likely to have occurred by chance or because a true relationship exists between the factors being considered.

This (the most commonly used) brand of statistics has been called ‘Fisherian’ or lately, ‘frequentism’ meaning that the probability (prediction) of any event (e.g. whether voters will pick BJP over RJD in the Bihar state elections) is a function of it’s relative frequency in a large number of trials.

Put another way, the level of uncertainty in a statistical problem results exclusively from collecting data among just a sample of the population rather than the whole population. So, by polling a sample of the population of Bihar on their preference, the likely winner can be estimated, along with a margin for error based on the size of the sampling poll.

No, this is not a post on statistics theory (I’m not qualified for that), but how this method of analytical thinking has drawbacks in many spheres of life, including investing, from Nate Silver’s book ‘The signal and the noise

The frequentist approach toward statistics seeks to wash its hands of the reason that predictions most often go wrong: human error (bias). It views uncertainty as something intrinsic to the experiment rather than something intrinsic in our ability to understand the real world. This method implies that as you collect more data, the error will eventually approach zero. Thus an analyst looking at daily movements in the NIFTY would see that in 15 years (over 4,100 trading days), the index has dropped by over 6% on only 17 days (just 0.41% of the time) thus assigning a low probability to a forecast about a sharp fall in the market.

Nate’s biggest objection to Fisher’s brand of statistics is how such analysis discourages researchers from introducing their own context or plausibility. Enter Bayesian reasoning.

Probabilistic or Bayesian Reasoning

“If a man will begin with certainties, he shall end in doubts; but if he will be content to begin with doubts, he shall end in certainties.”
Francis Bacon

Source: Scientific American
Source: Scientific American (scene from Big Bang Theory)

In a nutshell: Bayes’s theorem simply says that Initial belief plus new evidence = New and Improved belief

The Calm Investor | Bayes's Theorem

Back to where we left you, standing at the laundry hamper lost in thought…

The (uncomfortable) hypothesis you’re trying to prove or disprove is that your wife is cheating on you.

Bayes’s theorem can help you answer this sort of question, by estimating three quantities:

  1. The likelihood (probability) of the shirt appearing in your hamper as a condition of the hypothesis being true, i.e. your wife is cheating on you. If she’s cheating on you, it’s easy to imagine how the shirt got in the hamper. But then, if she is in fact cheating, wouldn’t she be more careful about such a thing? Let’s assign a probability of 75% (reasonably high) to the shirt appearing conditional on her cheating on you
  2. The likelihood of the shirt appearing in your hamper as a condition of the hypothesis being false, i.e. no cheating.  What reasons other than her cheating on you might explain the presence of the shirt? She bought it as a gift for you but decided to wash it first. One of her male friends who you know (and trust) from out of town stayed over and forgot his shirt. Your laundry service mixed someone’s shirt in your lot. As unlikely as these are, you collectively quantify this probability at 10%
  3. Most importantlywhat is the likelihood you would assign to your wife cheating on you, before you found the shirt? This is called a prior in Bayes’s language. Given you have a normal healthy marriage, you could choose to fall back on empirical studies that suggest roughly 5% of spouses cheat. That is your prior in this case (note that you can assign it at any value provided you truly take only factors before the shirt into account)

Bayes’s theorem can be used to now estimate the probability that your wife is cheating on you, given you found the shirt in the hamper.

Belief: Prior Probability (x) that your wife is cheating on you: 5%

New evidence: A mysterious shirt is found in the hamper

y: Probability that the shirt appeared given she’s cheating on you: 75%

z: Probability that the shirt appeared given she’s not cheating on you: 10%

Revised Belief: Posterior probability (New x’) that your wife is cheating on you

New x’ = x * y / [(x * y) + z * (1 – x)] = (0.05 * 0.75) / [(0.05 * 0.75) + 0.10 * (1 – 0.05)] = 28%

The revised probability that your wife is cheating on you is now 28%, up from the previous 5% but hardly close to certainty.  The frequentists way of thinking would be say that in 75% (3 out of 4) of cases, another man’s shirt found in your hamper indicates a cheating wife but taking a more uncertain view of the world means, in spite of the seemingly incriminating piece of evidence, it might be prudent to not assume the worst, just yet.

What this means

  • The key is your prior belief of the world. This involves starting with a hypothesis based on all the evidence at your disposal. Think of this as a foundation of common sense which would protect you from holding beliefs like toads can predict earthquakes
  • If you start with a certain view of the world, no amount of new evidence will change your mind (try the above calculations with x at 0 or 100%), but if you can leave your mind open to the possibility that you could be wrong, you stand to be able to refine your views based on new evidence
  • You’re constantly learning. Once you adopt a Bayesian approach to thinking in probabilistic terms, every new piece of evidence enriches your understanding of the world without making you susceptible to recency bias where you base you decisions on what happened most recently

Extend this to the markets and your investment decision-making. If your prior view is that equities of fundamentally strong companies will increase in value over the long term, then short term declines will temper your expectations but not distort them.

Further reading:

Frequentism and Bayesianism: A practical introduction

Statpac: Statistical Significance

How Bayes’ rules can make you a better thinker

Scientific American – Bayes’s theorem: What’s the big deal?

2 thoughts on “Think like a Bayesian

  • August 5, 2020 at 11:58 pm

    Amazing writing! Kept me engaged till the end.

  • May 14, 2021 at 11:15 am

    Amazing article, just have one doubt, shouldn’t z(Probability that the shirt appeared given she’s not cheating on you) and y(Probability that the shirt appeared given she’s cheating on you) sum up to be a hundred percent?

Leave a Reply

Your email address will not be published. Required fields are marked *