Meta-analyses — studies that combine and analyze the results of several studies on a topic — are among the most influential papers published in medical journals. They’re cited more than any other type of study, and are relied on heavily when medical groups determine their best practices.
In other words, they strongly influence the decisions your doctor makes about your medical tests, procedures and treatments.
A new study has found, however, that meta-analyses leave out a crucial bit of information: financial conflicts of interest in the studies they analyze.
Hidden in a footnote
As reported Tuesday in the Journal of the American Medical Association (JAMA), researchers reviewed 29 meta-analyses of drug treatments published in top-tier medical journals (including JAMA, the New England Journal of Medicine, and the Annals of Internal Medicine) during a 10-month period in 2009. These meta-analyses included data from 509 randomized controlled trials.
The researchers found that none of the meta-analyses reported the financial conflicts of interest (ties to the pharmaceutical industry) of the authors of the original randomized controlled trials and only two provided information on conflicts of interest regarding the trials’ funding sources.
And you’d have to search those two meta-analyses very carefully to find that information about the funding sources. Both listed the conflicts not in the text of the analyses, but among tables — one in a table that appeared after the study’s references list and the other in a table footnote.
A major gap
Yet many of the clinical trials included in the 29 meta-analyses had a direct link to the pharmaceutical industry. Some 70 percent (219/318) of the trials that reported funding sources were industry funded. And about 70 percent (91/132) of those that reported the financial disclosures of their authors had at least one author with financial ties to drug companies.
“The results of the present study highlight a major gap in the reporting of [conflicts of interest] and suggest that, without a formal reporting policy, [conflicts of interest] from [randomized controlled trials] are unlikely to be reported when results are synthesized in meta-analyses,” wrote the JAMA study’s authors.
Indeed, in 7 of the 29 meta-analyses, 100 percent of the trials being analyzed reported at least one conflict of interest (pharmaceutical funding or author-industry financial ties), yet only one of those meta-analyses made mention of that fact.
Not surprising
“It’s an interesting study,” said Glen Spielmans, an associate professor of psychology at Metropolitan State University, whose own research has uncovered ways in which drug-industry sponsorship influences study outcomes, in an e-mail exchange with MinnPost. “Actually, I was not surprised by its findings. Both industry sponsorship and author conflict of interest are associated with favorable results for products in clinical trials.”
“Most journals ask for authors to report conflicts of interest,” he added, “although authors do not always disclose their conflicts when asked. But it’s actually not clear that disclosure of conflicts of interest actually reduces any bias in research. Don’t get me wrong, I am in favor of disclosure of conflicts of interest, but I don’t think that disclosure really ends up solving many problems in research.”
Bias creeps into industry-funded research studies in many different and subtle ways, as the authors of the JAMA study point out:
[Conflicts of interest] may influence the framing of research questions, study design, data analysis, interpretation of findings, whether to publish results, and what results are reported. Results from positive trials and from favorable analyses are more likely to be published than results unfavorable to sponsors. Compared with nonindustry-funded trials, pharmaceutical industry-funded studies more often yield results or conclusions in support of the sponsor’s drug, and authors’ relationships with drug manufacturers have been linked to favorable assessments of drug efficacy and safety.
Recommendations
The authors of the JAMA study recommend that meta-analysts be required, at a minimum, to report the funding sources of the trials they evaluate. (Right now, they’re not required to do that.)
But they also propose that meta-analysts include an evaluation of potential study bias due to drug-industry funding and author ties to that industry.
That’s going to mean a lot more work for the meta-analysts — but it may be necessary if health consumers and their physicians are going to be able to trust the results of these big studies.
“Given that [conflicts of interest] are prominent in bio-medical research and have been empirically linked to bias, it is the responsibility of authors of meta-analyses to transparently document for readers their efforts to evaluate the likely influence of [conflicts of interest] on meta-analysis outcomes,” write the authors.
Spielmans agrees that more transparency is needed. “Conflicts of interest are very common in the medical world, and I think it’s reasonable for consumers of medical science — both readers of medical journals and patients — to expect practitioners and researchers to disclose any relevant conflicts of interest,” he said.
[Full disclosure: I teach writing and editing at Metropolitan State University.]
Does anyone else recall how even openly disclosed support from tobacco companies totally tainted any findings.
Is what is good for the goose also good for the gander?
Man, if I worked for Eli Lilly or some other pharmaceutical company, I’d have paid to publish this discovery. They could have got out ahead of it. How sad that instead this news had to tie up researchers’ time.