If health reporting has gotten even a little bit better in recent years, some credit may go to Gary Schwitzer and his Minnesota-based Health News Review website.
For the past nine years, Schwitzer and his team of more than three dozen experts from around the country have been reviewing and grading daily health news reporting by major U.S. news organizations. The website’s purpose is straightforward: “To improve the public dialogue about health care by helping consumers critically analyze claims about health care interventions.” That allows consumers to take the information to their physicians, to make wiser decisions about their health and medical care.
No health journalist wants to see his or her reporting receive a low score on the site. It can be a cringe-worthy moment.
But Health News Review is less about shaming health reporters than helping them do their job better. Reading through the 10 elements on which the website’s reviewers base their scores (such as “Does the story adequately explain and quantify the harms of the [medical] intervention?” and “Does the story compare the new approach with existing alternatives?”) offers reporters — and consumers — a lesson in how to evaluate health-related news.
And Schwitzer knows what he’s talking about. His CV includes more than a decade as a medical news reporter and head of the medical news unit at CNN, a stint as the founding editor-in-chief of the Mayo Clinic’s consumer health website, and several years developing shared decision-making videos for Dartmouth College’s Informed Medical Decisions Foundation (the original funder of Health News Review).
From 2001-2010, Schweitzer was also a tenured professor at the University of Minnesota, teaching health journalism and media ethics. He gave up that job to devote more time to Health News Review and activities related to it, such as speaking to and training journalists around the country.
Last month, Health News Review was relaunched with a new website design and other features, thanks to a two-year, $1.3 million grant from the Laura and John Arnold Foundation. The website also has a new institutional home at the University of Minnesota’s School of Public Health.
MinnPost recently talked with Schwitzer about the quality of health news reporting today and the “red flags” that consumers need to look for when reading any health-related news story. An edited version of that interview follows.
MinnPost: Of the 2,000 or so news stories you or one of your experts have reviewed on your site, only a little more than a third have received either a 5- or a 4-star rating, and the average score is only about 3 stars. What does that say about the state of health reporting today?
Gary Schwitzer: I do think we’re seeing some of the best healthcare journalism that this country has ever seen. But between those peaks of excellence — often in-depth, investigative, data-driven stories — the valleys are becoming wider and deeper. I call it the daily drumbeat of dreck, and I think it overwhelms any of the good that’s done by those occasional peaks of excellence. It’s the dreck down in the valley that are the types of stories we review — the stories that [are published] day after day in a volume that we can’t keep up with.
MP: What are the most common failings of those stories?
GS: You see it in the numbers. Nearly 70 percent of stories fail to adequately quantify benefits, fail to adequately quantify harms and fail to adequately discuss costs. What that says to me is that almost 70 percent of the time — in a sample of 2,000, in the eyes of usually three independent reviewers — we are making things look terrific, risk-free and without a price tag. So you tell me what kind of an informed consumer population we’re fostering with that kind of a daily drumbeat of media messages.
I often say to journalists, “I know you don’t roll out of bed in the morning, put your pants on and say, ‘How can I go in and screw up today? How can I go in and hurt people?’ But the harm, even though it is not malicious and is not intentional, is nonetheless very real.” … With our media messages, we are arguably poisoning the well 60 to 70 percent of the time.
MP: It’s not just individuals who become influenced by those media messages. It’s policy makers, too.
GS: Absolutely.
MP: One of the reporting failings you write and talk about frequently is disease mongering. Can you explain what that is and how it harms?
GS: That’s the most wide-ranging of any of our 10 criteria. It is the biggest gray area and requires the most judgment on the part of our reviewers. It is the medicalization of normal states of health. It can range from the ridiculous to the serious. [It’s what turns] dandruff or a little stage fright when we have to give a speech into a disease that should be medicated. [It also] treats risk factors for serious diseases as if they are diseases unto themselves. …
Should we really have a population running around listening to these “know your numbers” campaigns? Do we really want to have people focused on their LDL number, or do we want to have them learn to analyze the evidence and think about outcomes we really care about, like heart attacks and death? Do we really want men to be obsessed about whether their PSA level has just gone from a 3 to a 4, or do we want them to understand that there isn’t good evidence that treating the early prostate cancers found with the PSA test results in better outcomes than a watchful waiting approach?
It’s actually easier to get caught up in the weeds of the silly end of the spectrum because there’s just so much of it.
MP: Is there any significant difference in how print media — and I’m including online media in that category — and broadcast media cover health-related stories?
GS: [Broadcast media] is largely driven by artificial time limits of 90 seconds. Or less. An old friend of mine who works for a major CBS- owned and operated station wrote to me and said, “Gary, it’s now down to 72 seconds.” Think about that. You’re supposed to address our 10 criteria in 72 seconds? You can’t even read one of them in 72 seconds.
But it’s as variable as the individuals in any given news organization, and I try to look at it from a glass-half-full perspective. What we’ve seen — with [reporters] who really care and who know what they don’t know and who go out and seek training and seek personal improvement — is the difference that even one person can make in any organization and certainly in any one news organization. But we’ve also seen the harm that any one person can do in any one news organization. It’s not just the classic cases of fabrication and fraud that have taken place through the years in major news organizations, but good people doing shoddy journalism.
We have our own version of the Rolling Stone rape story every day in healthcare journalism. When you look at the excellent, scathing report on [the Rolling Stone article] in [the Columbia Journalism Review], some of the exact same themes are what we see practiced every day in healthcare journalism.
MP: Can you give an example?
GS: Here are quotes from the CJR report: “The failure encompassed reporting, editing, editorial supervision and fact-checking,” to which I say check, check and check. “Failing to state where important information had come from. … There is a tension in magazine and narrative editing between crafting a readable story — a story that flows — and providing clear attribution of quotations and facts. … The editors invested Rolling Stone’s reputation in a single source. … The problem of confirmation bias — the tendency of people to be trapped by pre-existing assumptions and to select facts that support their own views while overlooking contradictory ones.”
Let me give you the side note. Look at some of the mammography [screening] coverage. You’ll see confirmation bias in those stories.
MP: It can be hard for a journalist to persuade editors that some of this stuff isn’t what it seems.
GS: I know that. Increasingly, we’re trying to offer ourselves to people in that situation, and it’s been somewhat comforting through the years to have reporters tell us that they will bring our reviews — ones for which they got slammed by us — to their editor and say, “See? Look at what they dinged us for. It was exactly the stuff that you cut out of my story.”
MP: What should the public be most skeptical of when reading or watching a health-related news report? Are there any red flags?
GS: Yeah, there are. After talking about this in interviews for years I finally wrote them down:
- If it sounds too good to be true, it almost always is.
- If you ever see a healthcare claim that only discusses the potential benefits, proceed with caution. If harms aren’t discussed, it’s an incomplete discussion.
- If costs aren’t discussed, just learn from our experience: 70 percent of the 2,000 stories that we’ve looked at didn’t adequately discuss costs. At some point journalists have to take some degree of the responsibility for the fact that we spend a greater percentage of the GDP on healthcare than any other country on the face of the Earth — and we don’t have the outcomes to show for it.
- If you ever hear somebody talk about a simple screening test, run for the hills because there is no such thing as a simple screening test. Our decisions to seek screening should be among the most complex decisions that we face because, depending on whatever happens as a result of that test, you may kick off a whole cascade of issues that you probably are uninformed about. You need to have a balanced consideration of them — and a balanced discussion with your physician about harms and benefits.
- In health care, more is not always better. Newer is not always better. Less can be more.
MP: I would think, too, it’s important to make sure that any study has been published and peer reviewed. And to point out that if the study has been done only in animals, there’s a good chance it will not turn out to be relevant in humans.
GS: Yeah. We’ve just started [reviewing] news releases, and just this morning published a review of one out of Cedars-Sinai [Hospital in Los Angeles]. It was headlined “Stem Cell Injection May Soon Reverse Vision Loss Caused by Age-related Macular Degeneration.” The study was in rats. Do you want to put money on that crystal ball projection? What does “soon” mean? They allowed a researcher to get away with this quote in the news release: “Our institute is close to a time when we can offer adult stem cells as a promising source for personalized therapies for this and other human diseases.” “Soon” and “close to a time” when you’re talking about a couple of rats? … News releases can mislead overworked and underpaid healthcare journalists who have to meet a quota.
MP: Anything else people should be skeptical about?
GS: Messages based on observational studies. Observational studies cannot prove a causal link. They can show a very strong statistical association, but you can’t make them what they’re not. So when you use a causal verb to describe an observational study — when you say it “boosts risk” or it “offers protection” — you’re being inaccurate. It doesn’t. You cannot say that. Just because the rooster crows every morning when the sun comes up does not mean that the rooster crowing made the sun come up. We see that false crowing of research results all the time.
This is what creates these editorial ping pong matches where we leave readers with their heads spinning because one day coffee is protective and the next day coffee is the leading cause of stroke or whatever.
MP: But observational studies do have their place. Some research cannot be done in a clinical trial, so we have to look at observational studies and make our best judgment, right?
GS: Absolutely. We wouldn’t know what we know about smoking if it had not been for observational studies. It’s not that they don’t have value. It’s when we make them what they cannot be that we mislead.
MP: Are you hopeful going forward? Are you seeing any positive trends in the way health-related news is being reported?
GS: Yes. I would have to be crazy to still be doing this if I didn’t think that there was hope. It’s very tough to measure improvement with what we do. One of the things that make it so difficult is that there is such turnover on the [health] beat. So it is a constantly shifting landscape. …
But we have many more good role models and examples to look at. Many of them, not coincidentally, are a result of foundation-supported work, like Pro Publica and Kaiser Health News. And our work would have died without foundation support.
MS: So you’re optimistic.
GS: Yes. Some editorial decision-makers are seeing the light. They’re realizing that this is not a beat you can shove in a corner [and have your reporters] crank things out with single sources and no time to do any research, no time to do anything more than read the abstract of a study — if that. We need to shine a light on those folks because we need a lot more of them.
