March 2013

The article, “Chocolate Consumption, Cognitive Function, and Nobel Laureates” recently published in the New England Journal of Medicine (NEJM), demonstrates that even published research may contain errors.  The author, Franz H. Messerli, examines the correlation between countries’ chocolate consumption per capita and Nobel laureates per 10 million people. The linear relationship is strong, as shown in the article’s chart reproduced below:

Based on this result, the author concludes:

“However, since chocolate consumption has been documented to improve cognitive function, it seems most likely that in a dose-dependent way, chocolate intake provides abundant fertile ground needed for the sprouting of Nobel laureates.”

Stripping away the plant growing imagery from the quotation above, Dr. Messerli is quite simply asserting that chocolate consumption contributes to higher Nobel laureate counts per capita across countries. Below are the top five reasons why Dr. Messerli’s conclusions aren’t supported by his analysis:

  1. Improper order of cause and result. Dr. Messerli measures chocolate consumption at the present day; however, almost every Nobel prize included in his counts occur well before (sometimes by over a century) the point in time chocolate consumption was measured. While present day consumption might be indicative of past consumption levels (i.e. those that existed while the Nobel laureates were developing cognitively), the analysis should not ignore the possibility that consumption patterns have changed in the last century.
  2. Unlikely link between cause and result. By Dr. Messerli’s own admission, flavanols (the key ingredient in chocolate) improve cognition mostly within elderly patients, particularly in the form of reducing the risk of dementia. Cognitive improvements required for Nobel prizes are different than protecting an older mind from the effects of aging. More important would be findings that high-performing thinkers become still more capable with increased chocolate consumption.
  3. Failure to consider more likely explanations. Other factors almost certainly affect both chocolate consumption and Nobel laureate counts. For example, wealth as measured by GDP is demonstrably linked to both chocolate consumption and the number of Nobel laureates and more likely explains both higher chocolate consumption and Nobel laureate counts. This explanation. without further analysis (e.g.. multiple linear regression) that separates the consumption effect from the wealth effect, makes the reported effect of consumption basically meaningless.
  4. Failure to demonstrate that subjects received the “treatment” (i.e. increased chocolate consumption). Dr. Messerli does not measure consumption levels of the Nobel laureates themselves, and instead assumes that the laureates on average consume chocolate consistent with the consumption levels of their countries of birth. Without knowing whether the laureates consumed any chocolate, we cannot be sure the substance has any effect at all. This treatment assumption is further problematic because several laureates moved away from their country of birth at a young age and were therefore not even exposed to their countries’ average level of chocolate consumption.
  5. Omission of several countries, including those with relationships in opposition to his conclusion. Dr. Messerli only focuses on a narrow sample of countries (likely highly populous countries or those for which data on chocolate consumption is available). By excluding countries with unreported (and thereby likely low) consumption levels, Dr. Messerli is stacking the deck in favor of his desired conclusion. Take Israel as just one example: it has extremely high Nobel laureates per 10 million people (13.2), but its chocolate consumption is much lower (3 kilograms per capita per year) than most Western countries. This one counterexample by itself doesn’t disprove much, but it does illustrate what may be lacking in Dr. Messerli’s analysis.

Fulcrum Inquiry performs statistical analyses in litigation.