Nonprofit, nonpartisan journalism. Supported by readers.

Donate

UCare generously supports MinnPost’s Second Opinion coverage; learn why.

We don’t really know how effective mindfulness therapies are, researchers say

REUTERS/Edgard Garrido
A new study, published earlier this month in the journal PLOS One, suggests we might want to curb our enthusiasm about mindfulness therapies.

Mindfulness-based therapies are trendy these days — in the medical literature as well as in the popular press.

Recent studies have suggested that such therapies can help people with certain psychiatric conditions, including depression, anxiety, post-traumatic stress disorder and eating disorders. Mindfulness therapies have also been reported to improve the quality of life for people with various medical conditions, including cancer, heart disease, diabetes and arthritis.

One of the most recent studies on the topic, published in March in the Journal of the American Medical Association (JAMA), reported that a mindfulness-based therapy is effective in reducing chronic low back pain.

Marketers have been … ahem … mindful of these developments. As New York Times reporter David Gelles (and author of “Mindful Work”noted recently, it seems “like everybody is peddling mindfulness.” He offered these examples:

The Golden State Warriors, the Seattle Seahawks and the Boston Red Sox are now practicing mindfulness in the locker room. After Google began teaching the practice to its employees, stuffy companies like McKinsey and BlackRock started doing the same.

Consumer offerings are prolific, too. There are more than two dozen mindfulness apps for smartphones, some offering $400 lifetime subscriptions. The Great Courses has two mindfulness packages, each with a couple of dozen DVDs for $250. For an enterprising contemplative, it’s never been easier to make a buck.

A hidden bias

But what about all the mindfulness health claims? Just how robust is the science in support of them?

A new study, published earlier this month in the journal PLOS One, suggests we might want to curb our enthusiasm about these therapies — at least, until we have a more valid view of all the evidence. For the study, led by researchers at Montreal’s McGill University, found that the overwhelmingly favorable results published to date about mindfulness-based therapies are influenced by a phenomenon known among scientists as reporting bias.

“The proportion of mindfulness-based therapy trials with statistically significant results may overstate what would occur in practice,” the McGill researchers conclude. 

Reporting bias occurs when statistically significant “positive” outcomes get published, while non-significant, or “negative” outcomes, don’t. There are many reasons why this happens. Sometimes, for example, researchers believe that negative results are simply not interesting enough for a major journal to publish, so they simply “file” them away. Other times, commercial funders of a study (such as a pharmaceutical company) may choose to not make the negative findings public. 

Whatever the reasons, reporting bias has become a growing concern. 

Focusing on the gold standard

For the PLOS One study, researchers analyzed 124 published randomized controlled trials (RCTs) of mindfulness-based therapies. (RCTs are considered the “gold standard” of clinical trials.) The trials involved either mindfulness-based stress reduction (which combines mindfulness meditation, body awareness, and sometimes gentle yoga exercises) or mindfulness-based cognitive therapy (which expands traditional cognitive behavioral therapy to include meditation and other mindfulness techniques).

Half of the trials were conducted in North America, a third in Europe and the rest in other regions of the world. All but 3 percent of the studies were published after 2000, and 80 percent were published in 2010 or later.

The researchers found that 108, or 87 percent, of the 124 trials were presented at publication as having one or more positive outcomes. Only three of the trials were presented as unequivocally negative — “without any alternative interpretations or caveats to mitigate the negative results and suggest that the treatment might still be an effective treatment,” explain the McGill researchers. 

In all, 109 (88 percent) of the clinical trials concluded that mindfulness-based therapy was effective — a result that was 1.6 times greater than the number that the McGill researchers say would have been expected.

Also troubling was the finding that only 21 (17 percent) of the 124 studies were registered before they started collecting data (as required by the International Committee of Medical Journal Editors), and none of these adequately specified in the registry — at the start of the study — what the primary outcome of the clinical trial was.

In addition, 13 (62 percent) of the registered trials remained unpublished 30 months after they were completed (the time frame within which the National Institutes of Health expects studies to be published), and all of the studies that did make their way into a journal with 30 months had positive results. 

“It may be the case that selective outcome reporting as well as ‘data dredging’ and selective reporting of analyses may play an important roles in the proportion of positive studies that we found among [mindfulness-based therapy] RCTs in the present study,” write the McGill researchers. “If one assumes that there is some effect of [mindfulness-based therapy] on mental health outcomes, albeit a smaller effect than reported in published studies, the ability to selectively publish from multiple outcome options or multiple analyses could easily lead to exaggerated effect estimates and a rate of positive trial reports that exceeds plausibility, as we found in our study.”

Unlikely to do harm

Does the finding that studies on mindfulness-base therapies may be plagued with reporting bias necessarily mean we should discount their results altogether?

As Alan Cassels, a drug policy researcher affiliated with Canada’s University of Victoria, noted in a commentary on the study for the Minnesota-based website HealthNewsReview, “it’s surely not worth tossing the baby out with the bathwater.”

“Probably the truest thing that you can say about mindfulness or meditation is that it is unlikely to ever be harmful — a sentiment you could apply to almost no health care intervention in the modern world,” he adds.

Brett Thombs, a clinical psychologist who was the senior author of the PLOS One study, appears to agree.

“I don’t believe that mindfulness training is completely ineffective or is harmful,” he told Cassels. “I do believe — and I am supported by the evidence that we are publishing — that we don’t have a very good idea of how effective it is.”

FMI:  PLOS One is an open-access journal, so you can read the entire study on the journal’s website.

You can also learn about all our free newsletter options.

Comments (5)

  1. Submitted by Steven Lancaster on 04/19/2016 - 10:43 am.

    My experience

    I just had an article accepted in the journal “Mindfulness” which showed mixed results for the effect of a single session of mindfulness (versus traditional relaxation) on anxiety. I was very impressed with how willing they were to consider an article that was not all positive for the treatment which is the name of that journal.

    One thing to remember about publication bias is that can happen both from researchers not trying to publish and journals not willing to publish null results (those studies which “didnt work”). Additionally, conducting research is very consuming, both in terms of time and emotions. Spending 2-3 years on an idea and then realizing that your results are off from what you expected is very demoralizing. I certainly grant that these findings can benefit the field, but finding the momentum and time to write up something that personally demoralized you and might not get accepted anyway can be very challenging. Just something to consider on the human side of things.

  2. Submitted by Curt Nelson on 04/19/2016 - 03:49 pm.

    unlikely to do harm?

    Just because things like this don’t cause physical harm doesn’t mean they aren’t harmful. The harm from alternative therapies comes from the time and money wasted on them that might have been spent on proven therapies, and from the erosion of trust in science they engender.

  3. Submitted by Bill Gleason on 04/19/2016 - 05:00 pm.

    Unlikely to do harm?

    Mr. Nelson is on the money.

    Homeopathic remedies that contain 99.99999…% water and a few molecules – if you are lucky – of the therapeutic agent are unlikely to do harm either.

    Pathetic.

  4. Submitted by Paul Brandon on 04/19/2016 - 08:00 pm.

    The real gold standard

    for a test of ‘mindfulness’ would be much more ambitious.
    The problem is that ‘mindfulness’ is a vague category label for a wide variety of therapies, most of which are not scripted.
    To properly evaluate it would mean first identifying all of the different operations that are used in ‘mindfulness’ treatments.
    Then, you would need matched groups for each of these operations. Each group would have to be given all ‘mindfulness’ operations -except- a specified one. That would identify which operations are necessary for ‘mindfulness’ to be effective.
    I suspect that we would end up with a garden variety CBT (cognitive behavior therapy) treatment.
    I doubt that this sort of study will be done in our lifetime — the ‘mindfulness’ fad will have long passed.

  5. Submitted by Bill Kahn on 04/19/2016 - 11:51 pm.

    “Couldn’t hurt” is the punchline to an old joke in answer to a question from a doctor who has been instructed by an audience member to give an actor who has just dropped dead on stage, an enema: “Will it help?”

    Reporting bias has probably screwed up more science than we will ever know, but mindfulness therapies undoubtedly work better than enemas, except perhaps for cases of constipation.

Leave a Reply