Although all studies have their flaws, randomized clinical trials are generally considered the gold standard when it comes to determining the effectiveness of medical treatments and interventions.
Sometimes such studies are halted early because the results are so encouraging it’s considered unethical to continue to allow people to remain in the placebo (untreated) arm of the study.
A new review, however, now suggests that truncating those clinical trials may not be such a good idea. After analyzing 91 halted studies, an international team of researchers has found that many of the positive treatment effects reported in such studies turn out, under closer examination, to be exaggerated. The analysis was published earlier this week in the Journal of the American Medical Association (JAMA).
“We found that studies that stopped earlier than planned overestimated the treatment effect by an average of 30 percent,” said Victor Montori, MD, an endocrinologist at the Mayo Clinic in Rochester and one of the authors of the study, in a phone interview.
That’s for treatments that are later found to have none of the relative-risk-reducing effects that showed up in the truncated clinical trial. When a treatment is later found to actually reduce relative risk by 20 percent, said Montori, the truncated clinical trial is likely to have doubled the estimate of benefit — to 40 percent.
(Yes, this is the same Dr. Montori who co-authored the study reported here on Monday regarding the link between researchers’ views on the diabetes drug rosiglitazone and financial conflicts of interest. He’s had a busy couple of weeks.)
Strong incentives
Many people are happy when a clinical trial ends early because of positive findings — the trial’s researchers, their funding sources, the pharmaceutical companies that make the drug or device, physicians (who now have a “proven” treatment to offer their patients), scientific journals (which sell lots of issues of the journal with the study’s results) and, yes, even the media (which get to report upbeat medical news).
“Everybody loves these big studies that stop early,” Montori said. “Everybody benefits — except the patient who has to make the decision about risk and benefits with these overestimated benefit estimates.”
In the JAMA study, Montori and his co-authors urge researchers to use more restraint in stopping clinical trials early and to bring the trials to a halt only after a truly sufficient amount of data has been accumulated.
They also urge physicians — and the rest of us — to consider the very real likelihood that those exciting positive results from a clinical trial stopped early are exaggerations of the treatment’s true effects.
