Nonprofit, nonpartisan journalism. Supported by readers.


UCare generously supports MinnPost’s Second Opinion coverage; learn why.

Poor psychological science is rarely challenged in courtrooms, study finds

Photo by Bill Oxford on Unsplash
Legal challenges to psychological evidence occurs in only one in 20 court cases.
The quality of the psychological scientific evidence used to win over juries and judges in U.S. courtrooms varies widely and is often unreliable, according to a study published in the journal Psychological Science in the Public Interest.

Specifically, the study found that 60 percent of the psychological tests and assessment tools used in courtrooms are not favorably rated by experts and thus scientifically suspect.

Yet, legal challenges to psychological evidence occur in only one in 20 court cases, and such challenges succeed only about a third of the time, the study also found.

These findings are troubling. Psychological tests, tools and assessments are used in a wide range of legal cases, whether it be judging parental fitness for custody purposes, assessing the validity of eye-witness accounts, deciding eligibility for disability benefits, or assessing the competence of criminal defendants to stand trial.

“Most dramatically, intelligence tests have become all but dispositive in determining whether a person should be sentenced to death under the Supreme Court’s case law exempting people with intellectual disability from the death penalty,” the study’s authors write.

“One might think that, given the stakes involved, the validity of such tests would always be carefully examined,” they add. “That is not, however, always what happens.”

“Although some psychological assessments used in court have strong scientific validity, many do not,” says Tess M.S. Neal, the study’s lead author and a professor of social and behavioral sciences at the University of Arizona, in a released statement. “Unfortunately, the courts do not appear to be calibrated to the strength of the psychological assessment evidence.”

A study in two parts

For the study, Neal and her colleagues examined 364 psychological assessment tools that psychologists who serve as forensic experts have reported that they use in legal cases. They then analyzed those tools from both a scientific and legal perspective.

The researchers found that although 90 percent (326) of the psychological tools have undergone scientific testing, only 67 percent had been reviewed in the field’s most prominent journals and manuals. And of those, only 40 percent had received generally favorable reviews.

Almost a quarter of the tools were determined by the reviewers to be unreliable.

Neal and her colleagues then looked closely at 372 state and federal court cases (from the years 2016-2018) in which 30 different psychological assessment tools were used as evidence. Despite 60 percent of those tools failing to have the support of the scientific community, their admissibility in court was challenged only 19 times.

That meant that in almost 95 percent of the cases, questionable scientific evidence went unchallenged.

And only a third of those challenges were successful.

“Attorneys rarely challenge psychological expert assessment evidence, and when they do, judges often fail to exercise the scrutiny required by law,” Neal and her colleagues point out.

Possible explanations 

Several reasons may explain why this situation occurs in courtrooms. One has to do with lack of expertise among lawyers. As the researchers note, “Lawyers are generally not trained in how to analyze the validity of a psychological tool. Rather, they are likely to defer to what experts tell them.”

The issue of legal precedent is another impediment. “If lawyers and experts have always used a particular tool without challenge, then a new challenge is not likely forthcoming,” the researchers explain.

Psychological testing is also big business, “and some test publishers — some of them million- and billion-dollar companies publicly traded on the stock exchange — look to maximize profit,” write Neal and her colleagues.

The publishers “sell thousands of psychological assessment tools, many of which are revised and republished with updated version over time,” they add. “Many of these tools are sold for hundreds of dollars, and usually there are recurring per-use costs for items such as answer sheets, record forms, or online administration and scoring programs.”

Closer scrutiny needed

In their paper, Neal and her colleagues provide specific recommendations for how professionals and the public can make sure that psychological evidence presented in court is reliable.

“We suggest that before using a psychological test in a legal setting, psychologists ensure its psychometric and context-relevant validation studies have survived scientific peer review through an academic journal, ideally before publication in a manual,” said Neal in an interview with Rick Nauert, an editor for the online news site Psych Central.

“For lawyers and judges, the methods of psychologist expert witnesses can and should be scrutinized, and we give specific suggestions for how to do so,” she added.

FMI: The study can be read in full on the Psychological Science in the Public Interest website.

Comments (4)

  1. Submitted by James Hamilton on 02/21/2020 - 12:56 pm.

    Those who conducted this study appear to be under the common delusion that scientific witnesses, psychological or otherwise, are called to assist a judge or jury in making an objective assessment. Nothing could be further from the truth.

    In my more than 30 years in the practice of law, I never saw an expert witness called for any purpose other than to buttress a party’s case. The standards used by the courts to weigh the admissibility of such testimony is not intended to ensure the accuracy of the theory on which testimony might be predicated or the application of that theory to the case at hand. At best, it weeds out the most far-fetched expert testimony. Even that is highly debatable.

  2. Submitted by Paul Udstrand on 02/23/2020 - 09:04 am.

    Back in the late 80’s and up till the mid to late 90’s we had a rash of psychological quackery sweep through the courts promoted satanic ritual abuse, and multiple personality disorders, and recovered memories. Literally hundreds of people ended up behind bars until REAL experts demolished these courtroom quacks. Almost everything these quacks were claiming in courts for years was literally contrary to what science at the time actually knew and knows about human memory, personality, and psychology yet it took years to organize a response. The whole concept of “dissociaton” has no lost most of the legitimacy it ran the table with during the SRA witch hunts of the 90s.

    The lesson here is that the adversarial system our legal system relies on is severely flawed. Expert testimony is supposed to be subject to Daubert Hearings that establish legitimate credentials but since the judges making those decisions rarely understand the science it’s always up to the legal team to produce it’s own experts or counter experts. There but for the grace of god go those who fall victim to trendy science and quack diagnosis.

  3. Submitted by Ray Schoch on 02/24/2020 - 08:39 am.

    “Psychological science” is a phrase that – to a degree, at least – is oxymoronic. Actual science produces, at least in most cases, reliably replicable results. Were that the case in psychology, disputes over interpretations of behavior or motivations would be few and far between. We don’t live in that world.

    Psychology has advanced to the point where some degree of predictability has become the expected norm, but it remains at least partially an art form, relying on the insights and hunches of individual therapists and other practitioners. You can call that experience, or talent, or even magic, but I don’t think it qualifies as science.

Leave a Reply