During the Second World War the US military needed to reduce the number of planes they were losing to enemy fire. One method they tried was to study the returning aircraft, note where they had taken the most bullets, and add heavy armour to those sections. So far, so seemingly rational.
But then a brilliant statistician named Abraham Wald made a startling observation. The planes being inspected for damage were, by definition, the ones that had made it back. Whatever damage they had sustained was clearly survivable.
Wald realised that missing from the analysis were the planes that were not present for inspection because they had been shot down. He saw that the key to protecting the aircraft was to look not at where the bullet holes were on the surviving planes, but where they were not.
The phenomenon Wald overcame is known as survivorship bias, and it occurs when we draw conclusions based only on what is visible, while inadvertently overlooking what is missing.
Survivorship bias can help explain why people of a certain vintage sometimes claim music, movies, books and even advertisements were better in their day. This is often because only the best stuff survives in our memory. The dross is usually filtered out.
The same applies when people assert that products were designed to a much higher standard in the past, that they were built to last back then. This can seem to be true because classic designs endure. But it is misleading, as the vast majority of products do not survive. Machines that break down are thrown away. Only the best are left behind.
Great buildings can also create the misleading impression they are representative of their age. We protect them and restore them, like the beautiful Grade II Listed building, the Storey, where Hotfoot Design is based. It is easy to forget about all the lesser buildings that have been torn down and replaced.
But my favourite example of survivorship bias is probably a business owner who once explained to me that there was no point in updating his website as he did not get many visitors. We eventually did persuade him to invest in a new site with some compelling content, and unsurprisingly those missing customers magically appeared.
This article originally appeared in the Lancaster Guardian, where I am a guest columnist.
