Survivorship bias: why you can’t copy someone else’s success story

You can find them at the front of any bookstore year in and year out: bestselling publications that analyse the secrets to success among entrepreneurs, athletes, and artists. They are considered to be guides for success. Yet, these books consistently bring the most fame and fortune to their authors. The majority of readers fail, even though they faithfully follow Steve Jobs, Dirk Nowitzki, and Lady Gaga’s instructions. But why?

The recipes for success themselves are flawed. The authors of the books are liable to survivorship bias: By only taking successful protagonists into account, their analyses are distorted. All those failed entrepreneurs, athletes, and artists no one has ever heard of used these same supposed “secrets of success” – only without ever achieving success.

Survivorship bias is among the best-researched cognitive distortions in psychology. Because success in everyday life enjoys greater visibility, people systematically overestimate their own chances of success. Examples can be found in every sphere of life.

The term survivorship bias was first used by Allied engineers during the Second World War.

In their analysis of how they were better able to arm their airplanes against enemy attacks, they noticed that their improvements made no impact, even though they strengthened the areas in which the returned planes showed the most bullet holes. It was only when they reinforced the armour plating where returning planes showed no hits that the number of pilots returning home increased. This is because shots at these exact spots had a fatal effect: if the planes were hit, they crashed. Planes that didn’t crash and returned consequently showed hardly any hits.

The survivor bias had initially narrowed the view of the engineers to the survivors, the planes that had returned. Only when they took all the airplanes into consideration, including those that had crashed, did they find the weak spots.

To find examples of how our focus on the successful distorts our overall thinking, we don’t have to go all the way back to the Second World War.

Investment companies regularly remove unsuccessful funds from their portfolio so that their poor results are not incorporated into the balance of the total portfolio. To investors, the performance of the fund, thus, appears better at first glance than it actually would be if the loss-making fund was also included in the statistics.

Start-up investors are regularly influenced by survivorship bias. Month by month, billions in exits are reported in the media. Are you thinking of investing in the next Google, Amazon or Apple? Don’t get your hopes up, as the majority of start-ups fail. Nine out of ten start-ups go bankrupt – only no one talks about them. The risk that you would lose your money in the start-up world is high, even if everything you have ever read about it seems to indicate otherwise.

In medicine, survivorship bias plays an important role in the assessment of survival rates, for example, in cancer patients. If the patients pass away immediately after their diagnosis, they are often not even recorded in a study sample, which positively distorts the chances of survival.

In marketing too, the risk of falling into the trap of survivorship bias is real. Here are some situational examples where a reasoning error can result.

On the Internet you may have already seen one of the many success stories that read a little like this: “Thanks to this email template we increased our sales by 200% in one month”. Be aware that the (many?) consultants and agencies that don’t achieve any significant success with the same template don’t publish an article about it. Is it even worth looking at them then? Yes, do get inspired, but then consider the information within its context and also check which self-interests the author is pursuing by publishing their “secrets”. It is rare for someone to share information for purely altruistic reasons. If you would like to borrow a strategy, be sure to adapt it to your target audience, your product, and your industry.

Having a close relationship with customers is always a good idea. Staying in contact means hearing about any deficiencies in your product quickly, and possibly getting a chance to persuade unhappy customers to give your brand another chance. However, surveys are only partially suitable for capturing a realistic picture of how customers feel. This is because normally only a fraction of customers participate in surveys, with the most (positively or negatively) worked-up ones being the most likely to speak up.

If your customer feedback is overwhelmingly negative, you shouldn’t blindly change course to appease the critics. Similarly, if the feedback is overwhelmingly positive, don’t let it lull you into a false sense of security. Instead, reflect critically on the sample and validity of the survey. In order to obtain the most comprehensive picture, it makes sense to include former customers in the survey sample. This is because you must consider that your current customers are in a sense also “survivors”; they are satisfied enough with your product that they have been buying it up until now.

“Amazon succeeded with this strategy…” “This approach has been working for Facebook for years…” Copying the marketing strategies of successful companies often misses the mark. Why is that?

Correlations are not causalities. Just because orange-coloured buttons convert to bestsellers for Company A, doesn’t mean this will work for your company. Just because Bill Gates and Steve Jobs dropped out of college, doesn’t mean that a university degree is anywhere near an obstacle to leading a company to global success. The statistics argue the opposite: Those who drop out of college fail as company founders.

If you would like to adapt existing strategies, try to find models with the most similar basic conditions to yours. What is expedient in the real estate sector may have no relevance for the online drink trade or a software provider. Then test the success strategy you believe to be the best and optimise it using your own practical results.

Due to evolution, the human brain is programmed to detect deviations. That is why we are fascinated by stories of successful outsiders. It is why success stories remain in our memory more so than other types of news. This is because the majority of us do not belong to the successful elite.

Because success stories sell well, media coverage focuses on this kind of content and strengthens our tendency to notice the successful first.

Not least, the proclivity toward the successful provides people with a feeling of control. We want to believe that there is a recipe for success. If we become aware of survivorship bias, we must also come to the painful realisation that the fast route to success doesn’t exist, and that fortune, chance, and many individual factors play a crucial role.

Survivorship bias is a cognitive distortion that you should always take into consideration for marketing campaigns. Whether you are getting inspiration from well-known brands or analysing your own data, survivorship bias can cloud your judgment. It’s not inevitable, however, and studies and analyses do not lose their authority as a result. If you are aware of the reasoning error, you can free yourself from survivorship bias in your marketing by keeping the following in mind:

• Question success stories: Does causality play a role here at all? Could it just be an exception to the rule? How probable is it that you would get the same result if you were to reproduce the approach?
• Expand your focus: Check in advance that your own A/B tests, surveys, and inquiries not only consider “survivors”, but the entire database of results and examples.
• Ask “why?” more often: Marketers are interested in the ‘how’. How have others managed to do it? How can we achieve our goals? By thoroughly picking through their own results and those of others, they will likely be more successful than by blindly copying supposed success recipes.