UCare generously supports MinnPost’s Second Opinion coverage; learn why.

A fascinating tale about U’s MMPI controversy

You can now read online Maura Lerner’s intriguing behind-the-scenes look at the current controversy regarding the 70-year-old MMPI (Minnesota Multiphasic Personality Inventory) test, which ran in last Sunday’s dead-tree version of the Star Tribune.

The MMPI is, as Lerner notes, “the most widely used personality test in the world, assessing the emotional stability of millions of people,” and is “routinely used to screen candidates for highly sensitive jobs — pilots, police, nuclear powerplant operators — and in charged legal situations, such as child custody cases.”

The test has also been a long-term moneymaker for the University of Minnesota, bringing the school about $1 million a year in royalties, reports Lerner.

And where there’s money, controversy always seems to follow.

The current MMPI controversy concerns recent changes to the test’s questionnaire — and possible conflicts-of-interest at the University of Minnesota Press, which issued the grants for the test’s redesign. Writes Lerner:

[In 2007] a formal university audit uncovered numerous problems in the University Press test division, particularly the way it awarded $300,000 a year in MMPI research grants. The audit disclosed that 83 percent of the money went to projects involving the two members of its advisory board and that one of them had reviewed three of his own grant proposals.

An inquiry led by the U’s Vice President Tim Mulcahy concluded last year that no ethical wrongdoing had occurred and that the controversy was really about “professional differences of opinion,” Lerner reports. But the panel also cautioned that “considering the worldwide impacts of the MMPI … the University can ill afford to ignore the warning signs raised by the current controversy.”

It’s hard to know who’s right and who’s wrong in this controversy, but Lerner’s telling of the tale makes for a very good read.

Muddled methodology
One of the historical tidbits I found so interesting in Lerner’s article was the detail about how unscientifically by today’s standards (OK — by today’s purported standards) the original developers of the MMPI questionnaire went about choosing their “normal” control group. They compared test answers from patients in hospitals for the mentally ill with answers from people working in and visiting those hospitals.

Who’s to say those workers and visitors were mentally healthy?

As I read that, I was reminded of a broadcast I listened to last week from BBC Radio 4’s “Mind Changers” series, which has been examining how the science of psychology developed during the 20th century. (Unfortunately, the series, which I’ve only just stumbled across, isn’t available by podcast; you’ll have to listen to it online.)

This particular broadcast was about the controversy surrounding the Hawthorne Effect — the widely accepted idea that when people are being observed (such as during a psychological experiment), the fact that they know they are being observed will change their behavior.

Here’s the kicker, though: The source of the Hawthorne Effect, a study conducted during the late 1920s and early 1930s at the Western Electric Hawthorne Factory near Chicago, involved only five women — and when two of those women got tired of being so closely observed and asked to be taken out of the study (or were asked by the study’s researchers — that part is unclear), two new women were brought in and plugged into the departing women’s places in the study, as if they were interchangeable! Very sloppy methodology.

The study had other problems as well, including, amusingly, the bias of the people who sat listening to the chatter of these women day after day, year after year. Often, as an expert in the BBC broadcast notes, the comments from these observers tell us more about them than about the women in the study. For example, after one young factory worker began going out dancing at night, a disapproving observer wrote: “A restlessness has grown out of this change in social habits, and this restlessness has reduced operator 4’s output considerably.”

I suspect she was one of the women who left the study.

You can also learn about all our free newsletter options.

Comments (1)

  1. Submitted by Howard Miller on 08/13/2009 - 06:10 pm.

    The conflict-of-interest problems with the MMPI advisory board members is an ethical disaster that can not help but to undermine faith in the research data supporting the test’s uses. The U is usually sharper on such matters. Something needs fixing though.

    The Hawthorne effect – that people’s behavior changes under scrutiny – is likely robust enough to withstand a complete trashing of the research that first lead to it’s publication.

Leave a Reply