A ‘simple blood test’ can predict Alzheimer’s? Not so fast

Have researchers developed a “simple blood test” that can predict with 87 percent accuracy whether someone is going to develop Alzheimer’s disease within the next year or so?

It’s certainly understandable why you might have come to that conclusion this week. Many media outlets began proclaiming the existence of such a blood test on Tuesday, after the publication of a new research paper in the journal Alzheimer’s and Dementia.

The authors of that paper said that after studying the blood of 1,148 elderly people — 476 with clinically diagnosed Alzheimer’s disease, 220 with a mild (and controversial) form of dementia known as mild cognitive impairment (MCI) and 452 with no signs of dementia — they had found proteins that could identify which people had Alzheimer’s and which had MCI.

Furthermore, they said, 10 of the proteins were strongly associated with predicting which patients with MCI would go on to develop Alzheimer’s disease over a period of approximately 12 months — a prediction that had an accuracy of 87 percent, the researchers claimed.

That sounds impressive. … But is it?

Digging deeper into the numbers

Not really, explains John Gever, deputy managing editor of MedPage Today, in one of his organization’s “HypeWatch” columns.  That’s because the positive predictive value of the 10-protein test — the proportion of people it identified through a positive test result who truly had Alzheimer’s disease — was much, much lower.

So low, in fact, to be meaningless.

Here’s Gever’s explanation:

Only about 10% of patients [with] MCI convert to clinical dementia per year. With nearly 30% of positive results false [the specificity of the 10-protein test was 71%] as well as 15% of negative results false, most of the positive results in such a group will be false.

Yes, it’s time once again for a tutorial in positive predictive values. If we have 100 MCI patients and a 10% conversion rate, then 10 of them will develop dementia. These are the true positives. There will be 90 true negatives — the ones who don’t convert.

But with a specificity of 71%, the test will falsely identify 29% of the 90 true negatives, or 26, as positive. Meanwhile, with a false negative rate of 15%, only nine (rounding up from 8.5) of the 10 true positives will be correctly identified.

That’s 26 false positive results against nine correctly positive. That’s useless in a clinical setting. In fact, it’s worse than useless, since the false-negative results will expose patients to unnecessary clinical visits and treatments, and generate anxiety for them and their families.

Or, as an equally critical analysis of the study on the United Kingdom’s NHS Choices website (published by that country’s National Health Service) puts it: “This means that the combined test will only identify people correctly in around half of all tests, and potentially less, similar to the predictive ability of flipping a coin [emphasis added].”

Effective treatment needed first

But even if the test were 100% accurate, it wouldn’t be of much help to patients and clinicians “in the real world of geriatric medicine,” writes Gever.

People who test positive could “be watched more closely for onset of disabling symptoms and perhaps treated more aggressively with symptomatic medications such as cholinesterase inhibitors,” he says. “But without a disease-modifying therapy, it won’t transform the experience of patients.”

After all, there is currently no effective treatment for Alzheimer’s disease.

You can read Gever’s critique of the study on the MedPage Today website. You’ll find the NHS’ critique on the organization’s “NHS Choices” website — a great resource, by the way, for anyone who wants behind-the-headlines examinations of new studies.

The study itself can be downloaded and read in full online. The research was funded by several public entities, including the U.K.’s Medical Research Council and National Institute of Health Research. The researchers who conducted the study come from academic and commercial laboratories, including a lab at the British life sciences company Proteome Sciences, which has a commercial interest in biomarker testing, and one at the pharmaceutical company GlaxoSmithKline, which is working on developing drugs for the treatment of Alzheimer’s.

Those commercial interests — as well as the understandable desire of many in the media to publish something positive about a devastating disease that affects 5 million Americans and their families — may explain this week’s media hype.