Survivorship Bias OR How a mathematician helped win a war
In 1942, in the middle of the bloodiest wars ever seen, the United States military, brought together some of the brightest minds borrowed from the top universities around the country and called it, the Applied Mathematics Panel.
“Here is how it worked: Somewhere inside the vast machinery of war a commander would stumble into a problem. That commander would send a request to the head of the Panel who would then assign the task to the group he thought would best be able to resolve the issue. Scientists in that group would then travel to Washington and meet with top military personnel and advisors and explain to them how they might go about solving the problem. It was like calling technical support, except you called a computational genius who then invented a new way of understanding the world through math in an effort to win a global conflict for control of the planet.”[Source]
One of the members of this group was the Hungarian-born son of a Jewish baker. Abraham Wald had fled to the US in 1938, as the Nazi threat in Europe was reaching it’s pinnacle. The group within the AMP that Wald worked with specialized in air combat and the latest problem he worked on was keeping airplanes in the air.
“In some years of World War II, the chances of a member of a bomber crew making it through a tour of duty were about the same as calling heads in a coin toss and winning. As a member of a World War II bomber crew, you flew for hours above an entire nation that was hoping to murder you while you were suspended in the air, huge, visible from far away, and vulnerable from every direction above and below as bullets and flak streamed out to puncture you. “Ghosts already,” that’s how historian Kevin Wilson described World War II airmen. They expected to die because it always felt like the chances of surviving the next bombing run were about the same as running shirtless across a football field swarming with angry hornets and making it unharmed to the other side. You might make it across once, but if you kept running back and forth, eventually your luck would run out. Any advantage the mathematicians could provide, even a very small one, would make a big difference day after day, mission after mission.”
Given the limitation on how much weight a bomber could carry, military engineers wanted to know “the best places on the planes to add protection (extra armour)”. The military commanders already had a plan in mind. They had done extensive analysis of bombers that had returned from enemy territory, and meticulously tracked where the planes had taken the most damage. Their analysis found bullet holes tended to be clustered around the wings, the tail gunner, and down the center of the body. And this is where they wanted to add extra protection.
Would you agree? Did Wald and his team agree?
They didn’t. They proclaimed that adding extra armour to those three areas would do nothing to improve how many planes survived. That by focusing on the planes that survived, the military was falling prey to, you guessed it, survivorship bias
“The mistake, which Wald saw instantly, was that the holes showed where the planes were strongest. The holes showed where a bomber could be shot and still survive the flight home. After all, here they were, holes and all. It was the planes that weren’t there that needed extra protection, and they had needed it in places that these planes had not. The holes in the surviving planes actually revealed the locations that needed the least additional armour. Look at where the survivors are unharmed, he said, and that’s where these bombers are most vulnerable; that’s where the planes that didn’t make it back were hit.“
Wald then worked out how much individual damage each individual part of a plane could take before it fell out of the sky. And therefore where the planes needed extra reinforcement. Those calculations are still in use today.
Survivorship Bias in everyday life or the cult of the super…
The AMP anecdote makes for fascinating reading. It’s not hard to imagine a Hollywood production based on a group of academic (and goodlooking) geniuses using raw intellect to overpower the enemy. What is not apparent is that survivorship bias is not limited to military strategy. It pervades our daily lives, far more than we realize.
The Super Company
Dec 2009. This company just finishes the year having shipped close to 38 Million units. 1 out of every 5 units sold on the planet is made by them. In 2007, it shipped 12 Million units, 70% year-on-year growth. The stock is at $70, down from pre-2008 credit crisis highs of $138, but still offering 33% annual returns from 2006 to 2010. In 2007, it became the most valuable company in it’s home country. Later that year, a book chronicling its rapid rise, as the “world’s foremost mobile business tool” is published. It details learning for other companies who want to dominate their respective spaces.
The Super CEO / Sports Coach / Entrepreneur
From Jack Welch’s book “Jack: Straight from the Gut” in a chapter on “Building Self Confidence” on how having lost a close game of ice hockey, Jack reacted by flinging his stick across the ice. His mom burst into the locker room to admonish him on learning on how to lose better to be a winner. Further, he writes about how she was instrumental in building his confidence and therefore setting him up for success:
“Perhaps the greatest single gift she gave me was self-confidence. It’s what I’ve looked for and tried to build in every executive who has ever worked with me. Confidence gives you courage and extends your reach. It lets you take greater risks and achieve far more than you ever thought possible. Building self-confidence in others is a huge part of leadership. It comes from providing opportunities and challenges for people to do things they never imagined they could do — rewarding them after each success in every way possible.”
The Super Investor
The investor who bought Page Industries at 300 to see it go to 15,000 (50x return). (S)he also bought Hawkins Cookers at 25 and held it till it got to 4,500 (180x). Not one or two but a list of such picks that appreciated handsomely. You wouldn’t blame any budding investor who seeks this person out at dinner parties, to get a nugget or two. We’d all want to listen to their CNBC interview on the impact on stocks on demonetization. Maybe even buy their book on “How to get rich in the stock market”.
Wait, what about those who didn’t “survive”?
Did the “Super Company” make a series of brilliant moves that put it in pole position? Would other companies making those same moves, at that time, have prospered?
2010, the company buys a small software product company to revamp it’s operating system even as competing platforms start to make inroads. It sees its market share decline for the first time. 2011, it’s latest product launches flop, cuts 10% of its workforce. 2012 – 2013, it’s slide into irrelevance accelerates with more failed product launches. Market share drops below 3%. Stock price down to $10. 2014 onwards, multiple Op eds and case studies on why RIM (Blackberry) failed are published.
Are the traits and habits exhibited by the “Super CEO” the cause of his company’s performance and therefore his legendary status? Might there be others who exhibited those same traits, came from similar backgrounds, but didn’t make it beyond mediocre track records?
If you’ve read any books centered on individuals who attained “success” in their respective fields, the typical (auto)biography has two aspects; one is a narrative of the individual from their early years and two, is the set of behaviours or central tenets that enabled this individual to “succeed” so readers can adopt those habits to get on their own path to success. Essentially, I did this and I succeeded, therefore here’s what you need to succeed. At last count, Amazon lists over 1 Million titles under ‘Biographies and Memoirs’. Of these about 100,000 are by artists / writers, 18,000 by sports persons / coaches and 60,000 by business professionals.
If there was a standard template to achieve success, then would we need so many different versions? And if there isn’t a standard template, and there are millions who did all the things these books say but didn’t make it, then what does emulating Elon Musk’s habits offer?
How did investors other than the “Super Investor”, who used similar criteria to pick their (different) portfolios do? i.e. Can the “Super Investor’s” returns be replicated by identifying the criteria used and picking a different set of stocks?
There are 23.3 Million demat accounts in India. Let’s say I pick one of the 3,000-odd listed stocks and send a strong ‘BUY’ to 11.65 Million (50%) and a strong ‘SELL’ to the remaining. I send this email monthly. If I repeat the process with the 50% for whom my previous recommendation was useful, after 12 months, 5,688 people will have no choice but to believe I am a particularly gifted investor who was able to correctly predict stock market moves for 12 consecutive months. Put another way, from the universe of investors, pure chance will turn up a small set who get several consecutive stock picks right and will inevitably believe themselves to be particularly gifted.
“Survivorship bias flash-freezes your brain into a state of ignorance from which you believe success is more common than it truly is and therefore you leap to the conclusion that it also must be easier to obtain. You develop a completely inaccurate assessment of reality thanks to a prejudice that grants the tiny number of survivors the privilege of representing the much larger group to which they originally belonged.”
Our brains are pattern-seeking machines, forever trying to neatly assign causes to effects. Daniel Kahnemann said “If you group successes together and look for what makes them similar, the only real answer will be luck.” So, next time you listen to / about a success, wonder about those who might’ve done those same things but didn’t make it to the podium.
How Blackberry fell – New Yorker
Blackberry Planet: The story of RIM and the little device that took the world by storm – link