Writing online for the Atlantic magazine, Dr. Richard Gunderman, a radiologist and professor of medical education and medical ethics at Indiana University, explains why we should always read reports on medical research findings with a skeptical eye.
“Physicians, scientists, and the general public should be cautious about accepting many research reports at face value,” he writes. “The mere fact that biomedical researchers can find a statistically significant relationship between good health and a particular drug, nutritional supplement, dietary modification, or medical device does not in fact establish that it is healthful. Depending who is analyzing the statistics and how, numbers can lie, and in some cases, they can lure us to perdition.”
Gunderman gives the example of vitamin E supplements, once loudly touted as a protector against heart disease and cancer due primarily to epidemiological findings in a few early studies. Later, better-designed studies (randomized clinical trials) found, however, that vitamin E supplements can actually do more harm than good, increasing the risk of heart failure and prostate cancer.
Research results are often wrong because of something called false positive findings, Gunderman explains. “Simply put,” he explains, “some research models make it more likely for reported research results to be false than true, in part because a great deal of research merely amplifies preexisting biases.”
“Anyone making health and lifestyle decisions based on the scientific literature or reports of its findings in the popular press needs to understand these pitfalls,” he adds.
Key reasons for bias
Gunderman describes some of those pitfalls. One is the “wide latitude” researchers can use to design their studies and describe the outcomes. They can pick an endpoint, for example, that may or may not have significance in terms of lowering the risk of a disease. (A drug may lower cholesterol, for example, but that doesn’t mean it lowers the risk of heart attack and stroke.)
Another major pitfall is a system that offers powerful economic and career incentives to researchers who produce studies with positive rather than negative or neutral results.
“Major awards are virtually never presented to researchers for negative results,” Gunderman writes. “Egaz Moniz received the Nobel Prize in Medicine for developing a form of frontal lobotomy, but the researchers who later showed its poor benefit/risk ratio were not similarly recognized.”
An example of how positive bias infiltrates study results can be found in research reports on the effectiveness of antidepressant medications, writes Gunderman:
One analysis of articles in the scientific literature concluded that the effectiveness and benefit/risk ratio of the most popular class of antidepressants had been greatly exaggerated. For example, of 74 studies registered with the Food and Drug Administration, 37 that showed positive results were published in journals, while 22 that showed negative results were not. Moreover, 11 studies that showed negative results were published in a way that suggested a positive result. Overall, 94 percent of published studies indicated a positive result, when only 51 percent were actually positive.
‘Positive results are good business’
“Simply put,” writes Gunderman, “positive results are good business. There is money to be made every time a new drug or device is brought to market. It draws public interest and makes for good news copy. And it is something that most consumers and patients are hungry for. Wouldn’t it be great if we could lower our blood pressure, narrow our waistlines, increase our energy levels, elevate our moods, and prolong our lives simply by taking some new pill or making use of some new medical advice?”
But there is no “shortcut” or “magic pill or device that will make up for our bad choices, promote our health, and prolong our lives,” he adds. “… In the final analysis, living wisely matters most. Where health is concerned, this means having meaningful work in life (whether paid or not), eating moderately from a wide variety of foods, exercising in moderation, getting plenty of sleep, and avoiding excess. The latest research findings offer no substitute for living well.”
You can read Gunderman’s article on the Atlantic magazine website.