On Tuesday, federal officials released the results of the 2011 National Assessment of Educational Progress, standardized tests administered by the U.S. Department of Education. Sometimes referred to as “the nation’s report card,” the NAEPs generate big headlines because they are the only standardized tests that allow for an apples-to-apples comparison of performance by students throughout the country.
Perhaps you saw some of those headlines earlier this week and wondered how Minnesota could both still be well above the national average and lagging? The explanation is instructional — both in terms of what’s going on in schools and in terms of how poorly news media often report on achievement data.
Every year for the last two decades, the NAEPs have assessed a national cross-section of students and broken down the results by state, race, income level and other categories. This year, fourth and eighth graders were tested in reading, math, writing and science. Next year, 12th graders will take the tests.
Overall, Minnesota did post higher scores than all but two other states. The average “scale scores” posted by our students is 249, while students in Massachusetts and New Hampshire posted averages of 253 and 252, respectively. Two other states had outcomes similar enough to ours to essentially tie.
Broken down, though, top billing starts to lose its luster. Only 12 percent of Minnesota kids’ scores qualified as “advanced” and 41 percent were “proficient.” Another 35 percent were rated “basic” and the remaining 12 percent below basic.
At the next level, things get even less shiny. NAEP data ranks states, but also clearly points out when the difference between one score and another is not big enough to be statistically significant. Not unlike the error rate accompanying polling data, this disclaimer means that if the same groups of kids were to take the same tests on another day, the results might reverse themselves.
Few bright spots
Interestingly, on lots of the NAEP measures, upwards of 30 states post scores that are not statistically distinct from the rest. So if you punch interactive charts and maps that show different outcomes, you typically see a few brightly colored states and a vast sea of statistically indistinct, lusterless beige.
Translation: In many cases, when we’re in the top few, really, we’re only performing better, in any meaningful way, than a handful of states. And on many indicators we’re incrementally behind states traditionally thought of as academic backwaters, like Mississippi, Louisiana and Texas.
The bullet on that middle-American mass, according to the New York Times’ summary: “Elementary and middle school students have improved greatly in math, but their reading skills have stagnated over the last two decades, federal officials said on Tuesday.”
And Minnesota, it’s fair to say, is ahead of most states in math overall. Among black and white fourth-graders, we scored higher than all states except New Hampshire and Massachusetts. And we were one of three states that trailed New Hampshire and Massachusetts among black and white eighth-graders.
Only one state — Texas — posted statistically significant better scores in math among eighth-grade African Americans. Texas, Massachusetts and New Hampshire did better with fourth-grade African Americans, while eight states did worse, statistically.
Minnesota’s outlier appearances aren’t all brag-worthy. In both reading and math in both grades surveyed, for instance, the only place with a larger gap between scores posted by African-American pupils and white ones is Washington, D.C.
Just 10 states had gaps that were statistically meaningful and smaller than Minnesota’s 30 points in fourth grade reading; six had real, smaller gaps in eighth grade reading. D.C. was the only place with a larger gap in eighth grade math.
To put that gap into perspective: African-American students in fourth and eighth grades are more than three grade levels behind white students in math, according to the education advocacy group MinnCAN. African-American students in fourth and eighth grades are more than two grade levels behind whites in reading and low-income students in both grades are more than two levels behind their wealthier peers.
And if you wade into that vast sea of beige, there are some sobering items: In crude numbers, African-American fourth-graders in Alabama did better than in Minnesota.
So what’s to be made of this? As founding director of the University of Minnesota’s College Readiness Consortium, Kent Pekel is as steeped in NAEP’s fine points as anyone hereabouts, and his main takeaway is that instead of worrying about where we are right now, we should be troubled by where we’ll be soon.
The Minnesota students doing the least well are students of color, he explained, and they are doing less well — making slower gain than and backsliding faster than — their peers in other states.
“In reading, we are improving less rapidly than other states,” he said. “That’s also true of math, but we’re further ahead right now.
“One wonders if Minnesota students’ high but relatively flat performance in math portends problems to come,” he added.
Not only are other states making faster progress, the student populations that are growing rapidly are students of color. If our not just statistically significant but eye-popping racial and socioeconomic gaps persist, we’ll lag further and further.
The danger here, of course, is that an array of statistics such as these can be used to rationalize just about any ideological approach or strategy you can imagine.
Last year, with education reform atop the Legislature’s agenda, NAEP scores were used by state Republicans who wanted Minnesota to adopt a package of reforms former Florida Gov. Jeb Bush credited for outsize gains on the tests made by some groups in his state.
Parsed more objectively, the statistics showed that narrow groups in single grades had made particular gains — not that political footballs such as vouchers and privatization had a direct impact.
Better, in Pekel’s opinion, to mine the data not for numbers that support a particular approach but to see what has been possible elsewhere.
“Whatever we decide to do, we have to commit to a bipartisan state policy and stick to it for a long time,” he said.
“If you can’t get to a critical mass that has looked at the policy in enough detail and a tough level, we’re going to be right back here in two or three years.”