UCare generously supports MinnPost’s Second Opinion coverage; learn why.

Read medical research findings skeptically, doctor advises

CC/Flickr/Frederic Poirot
Studies touting the benefits of vitamin E supplements were later contradicted by later, better-designed studies that found the supplements can actually increase the risk of heart failure and prostate cancer.

Writing online for the Atlantic magazine, Dr. Richard Gunderman, a radiologist and professor of medical education and medical ethics at Indiana University, explains why we should always read reports on medical research findings with a skeptical eye.

“Physicians, scientists, and the general public should be cautious about accepting many research reports at face value,” he writes. “The mere fact that biomedical researchers can find a statistically significant relationship between good health and a particular drug, nutritional supplement, dietary modification, or medical device does not in fact establish that it is healthful. Depending who is analyzing the statistics and how, numbers can lie, and in some cases, they can lure us to perdition.”

Gunderman gives the example of vitamin E supplements, once loudly touted as a protector against heart disease and cancer due primarily to epidemiological findings in a few early studies. Later, better-designed studies (randomized clinical trials) found, however, that vitamin E supplements can actually do more harm than good, increasing the risk of heart failure and prostate cancer.

Research results are often wrong because of something called false positive findings, Gunderman explains. “Simply put,” he explains, “some research models make it more likely for reported research results to be false than true, in part because a great deal of research merely amplifies preexisting biases.”

“Anyone making health and lifestyle decisions based on the scientific literature or reports of its findings in the popular press needs to understand these pitfalls,” he adds.

Key reasons for bias

Gunderman describes some of those pitfalls. One is the “wide latitude” researchers can use to design their studies and describe the outcomes. They can pick an endpoint, for example, that may or may not have significance in terms of lowering the risk of a disease. (A drug may lower cholesterol, for example, but that doesn’t mean it lowers the risk of heart attack and stroke.)

Another major pitfall is a system that offers powerful economic and career incentives to researchers who produce studies with positive rather than negative or neutral results.

“Major awards are virtually never presented to researchers for negative results,” Gunderman writes. “Egaz Moniz received the Nobel Prize in Medicine for developing a form of frontal lobotomy, but the researchers who later showed its poor benefit/risk ratio were not similarly recognized.”

An example of how positive bias infiltrates study results can be found in research reports on the effectiveness of antidepressant medications, writes Gunderman:

One analysis of articles in the scientific literature concluded that the effectiveness and benefit/risk ratio of the most popular class of antidepressants had been greatly exaggerated. For example, of 74 studies registered with the Food and Drug Administration, 37 that showed positive results were published in journals, while 22 that showed negative results were not. Moreover, 11 studies that showed negative results were published in a way that suggested a positive result. Overall, 94 percent of published studies indicated a positive result, when only 51 percent were actually positive.

‘Positive results are good business’

“Simply put,” writes Gunderman, “positive results are good business. There is money to be made every time a new drug or device is brought to market. It draws public interest and makes for good news copy. And it is something that most consumers and patients are hungry for. Wouldn’t it be great if we could lower our blood pressure, narrow our waistlines, increase our energy levels, elevate our moods, and prolong our lives simply by taking some new pill or making use of some new medical advice?”

But there is no “shortcut” or “magic pill or device that will make up for our bad choices, promote our health, and prolong our lives,” he adds. “… In the final analysis, living wisely matters most. Where health is concerned, this means having meaningful work in life (whether paid or not), eating moderately from a wide variety of foods, exercising in moderation, getting plenty of sleep, and avoiding excess. The latest research findings offer no substitute for living well.”

You can read Gunderman’s article on the Atlantic magazine website.

Comments (3)

  1. Submitted by Paul Brandon on 03/25/2013 - 09:44 am.

    Good points!

    One might add the ‘file drawer effect’:
    the preference of journals for publishing positive rather than negative outcomes.

  2. Anonymous Submitted by Anonymous on 03/25/2013 - 12:12 pm.

    It’s not just the popular press that you should be wary about. Count up the retracted studies published in major medical journals. Pharma basically owns the journals, along with the continuing medical education programs, not to mention their direct payments to doctors. It’s not just the press that can’t be trusted – many physicians cannot be trusted because they put too much trust in their own professional journals and educational processes, without being skeptical of the money chasing certain outcomes.

  3. Submitted by Todd Hintz on 03/26/2013 - 04:21 pm.

    Business Aims

    Gee, yet another company (or in this case a whole industry), that’s bent on not doing what’s best for their customers, but rather padding their bottom line. Forgive my cynicism, but this just strikes me as par for the course when it comes to business ethics in America.

    I can’t really blame the doctors that much as they’re like the rest of us: pressed for time. So they rely on their trade journals to provide them with accurate information. Taking bribes from the pharmaceutical companies though is another ball of wax and should be completely banned. Science and logic should steer doctors towards the best treatment plan, not which company gives the best golf outing.

Leave a Reply