For chocolate lovers, the headlines seemed like sweet, sweet vindication. Most were along these lines: “Eating Chocolate May Help You Win Nobel Prize” and “The Secret to Genius: It Might Be More Chocolate” and (my favorite) “Scientific fact: Nobel prize winners eat more chocolate.”
What could be better than knowing that with each bite of chocolate, whether it be $2 bar from Hershey’s or a $9 one from Marie Belle, you may be making yourself smarter?
The headlines (and their accompanying articles) were about a paper published last week in the prestigious New England Journal of Medicine. In it, Dr. Franz Messerli, a professor of clinical medicine at New York’s Columbia University, declared the results of a study he had conducted on chocolate consumption and the “cognitive function” of the populations of various countries. He said he had found “a surprisingly powerful correlation between chocolate intake per capita and the number of Nobel laureates in various countries.”
In other words, countries whose populations devoured the highest amounts of chocolate were most likely to produce Nobel laureates.
An Onion-like spoof?
When I read a MedPage Today summary of this study last week, I burst into laughter. The study had to be an Onion-like spoof, I thought. If not, it was a great example of how misleading the findings from a correlation study can be.
The study turned out to be both. As Messerli told those reporters who bothered to contact him before writing their stories, he intended his paper to be a satirical example of the dangers of such studies.
But even without talking with Messerli, reporters should have caught the strong clues in the paper that his tongue was firmly planted in his cheek as he wrote it. Take, for example, this paragraph:
The only possible outlier in Figure 1 seems to be Sweden. Given its per capita chocolate consumption of 6.4 kg per year, we would predict that Sweden should have produced a total of about 14 Nobel laureates, yet we observe 32. Considering that in this instance the observed number exceeds the expected number by a factor of more than 2, one cannot quite escape the notion that either the Nobel Committee in Stockholm has some inherent patriotic bias when assessing the candidates for these awards or, perhaps, that the Swedes are particularly sensitive to chocolate, and even minuscule amounts greatly enhance their cognition.
And then there’s this statement, in which Messerli offers an amusing hypothesis for his finding:
That receiving the Nobel Prize would in itself increase chocolate intake countrywide seems unlikely, although perhaps celebratory events associated with this unique honor may trigger a widespread but most likely transient increase.
A cautionary example
Reuters reporter Frederick Joelving figured out what Messerli was up to — and gave him a call. The Columbia University professor quickly admitted that his study’s finding was “absurd” and that he intended his paper to be an example of how legitimate data can lead to very illegitimate scientific conclusions
Messerli … came up with the idea for the study after seeing a study that linked flavonoids, a type of antioxidants present in cocoa and wine, to better scores on cognitive tests.
He began with industry data on chocolate intake in 23 countries and a list from Wikipedia ranking countries according to the number of Nobel laureates per capita.
“I started plotting this in a hotel room in Kathmandu, because I had nothing else to do, and I could not believe my eyes,” he told Reuters Health. All the countries lined up neatly on a graph, with higher chocolate intake tied to more laureates.
The link was so strong it made a joke out of a statistic that virtually all studies in medical journals hinge on — the so-called p-value. Technically, this is the probability that a given result would be at least as “extreme” as the observation assuming, in this case, that there is no correlation.
The p-value Messerli calculated was 0.0001.
“This means that the odds of this being due to chance are less than one in 10,000,” he said. “As physician scientists we live and die by p-values, and here we have a p-value of a magnitude that is incredible, and unless you teach me otherwise it’s a complete nonsense correlation.
“So,” he added, “how good are p-values at giving us certainty? That is really some of the concern here.”
It’s not the first time scientists have found correlations that seem to defy all logic — and indeed may. The number of storks across Europe has been linked to birth rates, for instance, and sunspots have been tied to suicides in men.
By chance alone, these freak results are destined to find their way into mainstream medical journals as well.
“Scientists look at hundreds and hundreds of different things, and every once in a while they will find two things that are surprisingly correlated with each other, and then they will say, ‘Look at those very strong correlations and how important that is,'” [Nobel Prize-winning physicist Eric] Cornell told Reuters Health … “But what they don’t do is tell you about all the different things that aren’t correlated.”
As a result, Cornell said, “you are going to very much underestimate the randomness of what you got.”
The other possibility is that the link is real, but meaningless.
“National chocolate consumption is correlated with a country’s wealth and high-quality research is correlated with a country’s wealth,” said Cornell. “So therefore chocolate is going to be correlated with high-quality research, but there is no causal connection there.”
Messerli’s paper is a cautionary tale for all health reporters — and consumers. Correlations are not the same as cause-and-effect.
And, unfortunately, eating chocolate is not going to make any of us smarter.