Mindfulness-based therapies are trendy these days — in the medical literature as well as in the popular press.
Recent studies have suggested that such therapies can help people with certain psychiatric conditions, including depression, anxiety, post-traumatic stress disorder and eating disorders. Mindfulness therapies have also been reported to improve the quality of life for people with various medical conditions, including cancer, heart disease, diabetes and arthritis.
One of the most recent studies on the topic, published in March in the Journal of the American Medical Association (JAMA), reported that a mindfulness-based therapy is effective in reducing chronic low back pain.
Marketers have been … ahem … mindful of these developments. As New York Times reporter David Gelles (and author of “Mindful Work”) noted recently, it seems “like everybody is peddling mindfulness.” He offered these examples:
The Golden State Warriors, the Seattle Seahawks and the Boston Red Sox are now practicing mindfulness in the locker room. After Google began teaching the practice to its employees, stuffy companies like McKinsey and BlackRock started doing the same.
Consumer offerings are prolific, too. There are more than two dozen mindfulness apps for smartphones, some offering $400 lifetime subscriptions. The Great Courses has two mindfulness packages, each with a couple of dozen DVDs for $250. For an enterprising contemplative, it’s never been easier to make a buck.
A hidden bias
But what about all the mindfulness health claims? Just how robust is the science in support of them?
A new study, published earlier this month in the journal PLOS One, suggests we might want to curb our enthusiasm about these therapies — at least, until we have a more valid view of all the evidence. For the study, led by researchers at Montreal’s McGill University, found that the overwhelmingly favorable results published to date about mindfulness-based therapies are influenced by a phenomenon known among scientists as reporting bias.
“The proportion of mindfulness-based therapy trials with statistically significant results may overstate what would occur in practice,” the McGill researchers conclude.
Reporting bias occurs when statistically significant “positive” outcomes get published, while non-significant, or “negative” outcomes, don’t. There are many reasons why this happens. Sometimes, for example, researchers believe that negative results are simply not interesting enough for a major journal to publish, so they simply “file” them away. Other times, commercial funders of a study (such as a pharmaceutical company) may choose to not make the negative findings public.
Whatever the reasons, reporting bias has become a growing concern.
Focusing on the gold standard
For the PLOS One study, researchers analyzed 124 published randomized controlled trials (RCTs) of mindfulness-based therapies. (RCTs are considered the “gold standard” of clinical trials.) The trials involved either mindfulness-based stress reduction (which combines mindfulness meditation, body awareness, and sometimes gentle yoga exercises) or mindfulness-based cognitive therapy (which expands traditional cognitive behavioral therapy to include meditation and other mindfulness techniques).
Half of the trials were conducted in North America, a third in Europe and the rest in other regions of the world. All but 3 percent of the studies were published after 2000, and 80 percent were published in 2010 or later.
The researchers found that 108, or 87 percent, of the 124 trials were presented at publication as having one or more positive outcomes. Only three of the trials were presented as unequivocally negative — “without any alternative interpretations or caveats to mitigate the negative results and suggest that the treatment might still be an effective treatment,” explain the McGill researchers.
In all, 109 (88 percent) of the clinical trials concluded that mindfulness-based therapy was effective — a result that was 1.6 times greater than the number that the McGill researchers say would have been expected.
Also troubling was the finding that only 21 (17 percent) of the 124 studies were registered before they started collecting data (as required by the International Committee of Medical Journal Editors), and none of these adequately specified in the registry — at the start of the study — what the primary outcome of the clinical trial was.
In addition, 13 (62 percent) of the registered trials remained unpublished 30 months after they were completed (the time frame within which the National Institutes of Health expects studies to be published), and all of the studies that did make their way into a journal with 30 months had positive results.
“It may be the case that selective outcome reporting as well as ‘data dredging’ and selective reporting of analyses may play an important roles in the proportion of positive studies that we found among [mindfulness-based therapy] RCTs in the present study,” write the McGill researchers. “If one assumes that there is some effect of [mindfulness-based therapy] on mental health outcomes, albeit a smaller effect than reported in published studies, the ability to selectively publish from multiple outcome options or multiple analyses could easily lead to exaggerated effect estimates and a rate of positive trial reports that exceeds plausibility, as we found in our study.”
Unlikely to do harm
Does the finding that studies on mindfulness-base therapies may be plagued with reporting bias necessarily mean we should discount their results altogether?
As Alan Cassels, a drug policy researcher affiliated with Canada’s University of Victoria, noted in a commentary on the study for the Minnesota-based website HealthNewsReview, “it’s surely not worth tossing the baby out with the bathwater.”
“Probably the truest thing that you can say about mindfulness or meditation is that it is unlikely to ever be harmful — a sentiment you could apply to almost no health care intervention in the modern world,” he adds.
Brett Thombs, a clinical psychologist who was the senior author of the PLOS One study, appears to agree.
“I don’t believe that mindfulness training is completely ineffective or is harmful,” he told Cassels. “I do believe — and I am supported by the evidence that we are publishing — that we don’t have a very good idea of how effective it is.”
FMI: PLOS One is an open-access journal, so you can read the entire study on the journal’s website.