Every July a panel of seventy wine connoisseurs gathers in sun-drenched Sacramento to judge and award prizes at the California State Fair in a blind-tasting competition.
It is a prestigious event with a history stretching back to 1850. For winemakers a gold, silver or bronze medal offers the opportunity to gain status, publicity and – perhaps most importantly – sales.
But the owner of one small vineyard in California, armed with some knowledge of statistics, had real doubts about whether anyone really can consistently judge wine. And so Robert Hodgson decided to conduct a test (or lay a trap, depending on how you look at these things).
With the agreement of the head judge, Hodgson entered 100 bottles of wine into the competition, and secretly ran each of the wines past the judging panel not once but three separate times. He did this for four consecutive years. And then he published the results.
Just as Hodgson predicted many of the judges awarded wildly different scores to exactly the same wine. Even judges that were consistent with their scores in one year were usually terrible the next.
The problem was not unique to the judges at the California State Fair. Hodgson looked at how wines performed at blind-tasting competitions across America, and found a gold award winner in one was no more likely than any other wine to win in another.
The whole thing was seemingly random, with the only real winners being the competition organisers who charge a hefty fee for each entry.
Hodgson’s findings have been widely publicised, but little has changed. When a wine wins an award, however dubious the judging process might be, you can bet it will be proudly displayed on the bottle.
And that is because an award logo is one of the visual cues we look for to help validate whether we are making the right purchase decision.
Faced with a wall of wine bottles we might still be left with a decision to make even after narrowing our choice by region, grape and price.
If the brand is unknown the shiny medal on the neck of the bottle might be the thing that swings it. And so the show goes on.
This article originally appeared in the Lancaster Guardian where I write a column.
