Skip to Content

Support MinnPost

MinnPost's education reporting is made possible by a grant from the Bush Foundation.

How to make sense of the latest Minnesota student test results

The shutdown may have delayed it by a month, but this morning’s release of the results of the 2011 Minnesota Comprehensive Assessment, or MCA, was as inevitable as death and taxes. Equally inevitable: wailing and gnashing over the interpretation of the scores.

They don’t call them high-stakes tests for nothing.

I hadn’t thought to write about the MCAs this year because, frankly, after years of writing articles suggesting that the tests take up precious instructional time, don’t measure actual learning and provide little of use to educators, I have lost interest in the topic.  

But then late yesterday morning my phone started to ring, and quickly the afternoon was lost as one anxious-sounding policy wonk after another volunteered to help frame my analysis. Each predicted sky-is-falling headlines. I gave up, got out my No. 2 pencil and started taking notes.

Incremental gains (with many asterisks)
The bottom line: By and large, Minnesota students are pretty much where they were last year, and the year before, and the year before. They are making incremental gains, with wealthier kids posting enviable scores and their poorer brethren struggling.

Beyond that there are enough wrinkles affecting this year’s scores [PowerPoint] to use up an entire carton of asterisks.

Topping them all, of course, is the fact that as contemplated by the federal law that spawned the tests, the much-despised No Child Left Behind Act, we have hit the point where even our best schools are in danger of being labeled failures.

Reading scores in all grades rose by 1.6 percent to 74 percent proficient. Same headline as 2010: Progress, everyone from state Education Commissioner Brenda Cassellius to Education Minnesota President Tom Dooher agreed, but nowhere near the order that closes gaps.

More rigorous math test
Math is where the first wrinkles appear. Among third- through eighth-graders, there was an 8.7 percent drop in proficiency, which means 56 percent of kids passed the tests.  

But this does not mean math proficiency has fallen. Rather, it’s a reflection of the fact that last year students in those grades took a new, harder version of the MCAs pegged to a much more rigorous performance standard established in 2007. (Reading tests are still pegged to 2003 standards.)

Yes, it takes four years — and an ocean of tax dollars that might otherwise fund actual learning — for a private test developer to come up with an assessment that measures whether students meet the standard. Which makes it all the more unpleasant that in the past, the standards often haven’t had much to do with learning.

And yes, education administrators are prone to rationalizing poor test scores as the result of rigor. But Minnesota 11th-graders are still taking the old tests, the MCA IIs. Statewide, they posted an impressive 5.6 percent gain.

So, in terms of math, the upshot is that we don’t know how Minnesota students are doing compared to last year …

... or the year before. In case you forgot last year’s asterisks, 2010 was the first year Minnesota used a so-called growth-model test, which compared individual students’ performance from one year to the next.

Because their purpose was to identify bad schools, not gaps in kids’ skills, prior to that the MCAs measured year-over-year performance by entire grades. So we could declare an individual teacher or principal a failure, but not identify what would help their pupils.

Teensy but steady gains from 2006-10
What we do know is that teensy but steady gains were posted between 2006 and 2010.

And we know some things about the characteristics common to many of the schools that show up as outliers: High-poverty schools with high proficiency rates or bigger-than-normal gains from one year to another. But that’s another story.

In theory, then, next year will be the first in which we’ll be able to tell whether a schoolchild has learned a year’s worth of math in a year, and whether that math is at grade level.

Except, of course, that Cassellius has formally asked her federal counterpart, Arne Duncan, for a waiver from NCLB. In her preliminary request for flexibility, she asked that Minnesota be given a three-year hiatus from having to demonstrate Adequate Yearly Progress.

The waiver would keep cumbersome, costly penalties attached to failing to make AYP from kicking in. In great schools, NCLB foes argue, those sanctions take resources away from programs that are working. In struggling schools, they’re downright punitive.

Minnesota will still hold itself accountable for student performance, Cassellius assured the U.S. Secretary of Education. But it would like to do so by using tests that supply meaningful intelligence about what’s going on in the classroom.

Little agreement on what should follow NCLB
Duncan is offering waivers because, while policymakers on both sides of the aisle agree that NCLB’s high-stakes testing has been a resounding failure, there is zero agreement what to do about it. The secretary is much fonder than most educators would like of continuing to administer standardized tests — just better ones.

Lacking consensus within his caucus, Lakeville Republican John Kline, who heads the powerful U.S. House of Representatives Committee on Education and the Workforce, would like to replace the law piecemeal.

So which Minnesota schools are making the NCLB grade? I’m back where I was yesterday morning when the first alarm bell rang: I don’t care, and I’m hard pressed to suggest that you should, either.

Perhaps the banner headline of the day ought to trumpet the fact that Minnesota is one of a dwindling number of states not wracked by a scandal involving administrators alleged to have doctored students’ tests.

Get MinnPost's top stories in your inbox

Comments (15)

"So which Minnesota schools are making the NCLB grade? ...I don’t care, and I’m hard pressed to suggest that you should, either."

Shorter Hawkins: Excuse, spin, obfuscations and don't ask about exceptions to them.

According to the script, I, as a selfish uncaring conservative should take this opportunity to crow about what a good idea it was for my wife and I to sell our house to pay for private educations for our kids.

...but honestly and sincerely, when I think about what leftist social experimentation and moral degradation has done (and is doing) to American families, and what leftists so willingly join the teacher's unions in doing to our once world-class public education system; all of the wasted potential; all of the kids hitting the streets functionally illiterate, I get so depressed I just want to throw up.

Beth,

Do Minnesota children take any Nationally normed tests such as the Stanford Achievement Test or the Iowa Test of Basic Skills? Is there some non-MCA baseline against which the state attempts to measure children?

When I used to teach in Texas, the kids took the Texas Assessment of Knoweldge and Skills (TAKS)test as well as the Stanford test. Only the TAKS counted toward AYP and state rewards/sanctions. We would see yearly gains, sometimes extraordinary gains, on the TAKS but the Stanford results remained flat. There was a lot of test prep devoted to TAKS but little to the Stanford. I always thought the TAKS results were suspect based on this discrepency.

Joe;

They do, but not all the same other tests. As in all states, a cross-section takes the National Assessment of Educational Progress, aka "The Nation's Report Card." Education scholars often use these numbers when comparing one state to another. In addition, many Minnesota kids take growth-model tests that are paid for by their home districts that show fall-to-spring gains. These are known by a variety of acronyms; in Twin Cities schools the most common are the MAPs and the CALTs.

As you observed in Texas, there's lots of cramming for MCAs, lots of teaching to the test and a flurry of "please feed your child a good breakfast" backpack notes. The growth-model tests, by contrast, get virtually no prep because teachers often will actually use the data to see how kids are faring and where they need help.

Bonus growth-model test factoid for serious assessment geeks: One of the main architects of these exams, which the Dayton administration would like to use in some form to replace the current system, is Bloomington Public Schools' own Jim Angermeyer.

Just as a “growth” model in economics is ultimately self-defeating – unless we very quickly find another very similar planet nearby, but uninhabited, so that we can use all ITS natural resources after we’ve consumed all of our own – an “adequate yearly progress” model for the taking of statewide tests (which don’t really measure very much that’s useful beyond an ability to take standardized tests) is ultimately self-defeating, and that’s assuming that the tests are worth taking in the first place, an assumption I don’t make.

If Mr. Swift’s privately-educated children perform well on the test when they take it the first time, especially if they perform as well as we’d expect the offspring of a thoughtful and tolerant conservative such as Mr. Swift to do, well… where do you go when you’re already at the top? Once they’ve reached “proficiency,” then what? “Mastery?” “Superlative?” “Genius?” Is it like college degrees? If a 5th-grader is a math whiz, can already do trig and calculus, and scores in the “nth” percentile the first time s/he takes a relevant math test, how can s/he improve on “knowing the subject thoroughly” in standardized-test-land?

I do look forward in the near future, however, to a piece that gets at “…And we know some things about the characteristics common to many of the schools that show up as outliers: High-poverty schools with high proficiency rates or bigger-than-normal gains from one year to another. But that’s another story.” I’d like to see what that “other story” is.

Meanwhile, we can safely ignore the usual meaningless drivel from Mr. Swift about our world-class education system being destroyed by morally-bankrupt leftists. It appears, even from the rather limited scope of this article, that – with the notable exception of those schools about which there’s “another story,” the same parameters that have fairly accurately predicted academic success for the past half century and more continue to be in operation. Kids from affluent families generally score well. Kids from poor families generally do not.

Mr. Swift didn’t need to sell his house, if, in fact, he did so, and as someone who attended both an exclusive private school as well as public schools, I’ll suggest that the quality of education acquired is largely dependent upon the quality of the student.

The relevant axiom is: Them that has, gets. Them that don’t, don’t.

Maybe it is time to re-think what we, as Minnesotans, agree to in providing a public education. It seems to me that the academic achievement of a child is influenced by three factors; their natural ability, their teachers and materials, and their familial support. Clearly the state cannot change a child's natural ability, but we can test to assume an achievable norm for average ability. We do provide the teachers and materials, and we analyze this process to excess.

But lastly, what is the larger group agreeing to in order to neutralize the effects of income and class disparities? Do we mean that it is the responsibility of all to provide every child with a home life that structures regular homework time, provides tutoring as necessary, provides regular meals and role models? Are we all responsible for providing an adult in the home who speaks with an expanded vocabulary and will correctly define words upon request? Are we all responsible for communicating, through support and recognition, the lifelong benefits of an education?

If we knew what it was we were responsible for, maybe it would easier to evaluate a public education as a public good.

"Mr. Swift didn’t need to sell his house, if, in fact, he did so"

Gee Ray, if only you were around when we were working out the finances ten years ago we might still be living in Highland Park...hope my wife doesn't find out!

Personally, I'm sick of hearing defenders of the status quo whine about "teaching to the test". If a student can't solve math problems, he can't solve solve math problems; if a student can't read, she can't frickin' read!

Pick your assessment method; BST, MCA I & II, Profiles of Learning, % of college freshman requiring remedial course work; students in MN's largest 2 public school districts have failed 'em all.

Well, there is one exception.

June Maker's "multiple intelligence" tests did fill SPPS Capital Hill magnet school with the "diverse" student body the district was aiming for...most of whom promptly tanked the "gifted and talented" charter's BST and MCA scores.

I'm just happy that they are going to a four-year standard to measure HS graduation, instead of whatever weirdness they did before. Hopefully it will mean something and not be a measure of social promotion. The employers sure will not be fooled with incompetence.

WHAT RAY SAID!

MCA-II reading results for Saint Paul Public Schools surpassed state gains and broke a four-year trend of small changes with a 4% increase for SPPS students. This means that approximately 700 more students are reading at grade level. At the same time, the state saw a 2.3% increase in the percent proficient in reading.

ELL students in Saint Paul do better than ELL students statewide.

White students in Saint Paul do better than white students statewide. Swiftty didn't have to move his kids to get a good education.

The achievement gap has narrowed slightly, but is still shameful.

In Math, Saint Paul ELL, white, and hispanic all outperformed the state as a whole. Swifty appears to have screwed his kids out of a good education.

Math
In math, SPPS EL, Caucasian and Hispanic/Latino student groups outperformed the state (EL SPPS 29.8% versus MN 25.7%; Hispanic/Latino SPPS 32.3% versus MN 32.0%; Caucasian SPPS 67.7% versus MN 63.3%).

All of the following elementary schools, including CApital Hill that Swifty disparaged, outperformed the state in math.

Capitol Hill Magnet 83%
Chelsea Heights 62%
Hayden Heights 60%
Highland Park 63%
Horace Man 73%
L'Etoile du Nord 77%
Randolph Heights 75%
St. Anthony Park 81%

Long way to go. These tests really are counterproductive. But if you are going to use them to bash teachers, bash kids, and bash schools, at least us some basic facts.

One last thing. St. Paul's preK and all day kindergarten referendum started about 5 years ago, and those cohorts are now filtering through the testable grades. St. Paul's gains are certainly aided by these enhanced groups of students. The referendum is up next year.

tsk...tsk.

Here's a basic fact: Alec is reciting the figures for 3rd grade tests...the grade 11 numbers tell a different story.

Thanks for the post, Beth. I'm pretty much in agreement with what you wrote. The whole point of NCLB from Day One was to have nearly every public school declared a "failure," so that we could then privatize education and give everyone vouchers. This was a feature, not a bug, so I could never figure out why Ted Kennedy and a bunch of supposedly progressive Democrats supported NCLB. It was always a trojan horse.

That being said, I believe some form of standardized testing is still needed, but with moderation in all things and with the focus on individual student growth. And yes, I believe the collective results of these tests can also be used to assess individual teachers--especially if we do rolling three-year averages. This is called "value-added" data. It works best for identifying teachers in the top 15 percent and the bottom 15 percent. It's less effective in identifying the 70 percent who are somewhere in the middle. But if we could retain the top 15 percent of our teachers and get rid of the 15 percent who are consistently under-performing, that would be a huge improvement over our current status quo.

Value-added data works well for some subjects It obviously doesn't work well for say, the gym or art teacher. But the idea that all employees must be evaluated under the identical rubrics is stupid---most profession or work place evaluate different kinds of work with different kinds of evaluations.

Harvard University Press just published a new book called "Value-Added Data in Education: What Every Educator Should Know." I know. I know. It doesn't sound riveting. But it's actually a surprisingly easy and interesting read---it was designed to explain value-added data to teachers and other non-statisticians.

Minneapolis Public Schools have one of the best and oldest banks of value-added data in the state. The data is weighted demographically. District officials and principals can look at it and for example, see which of their teachers are consistently the most successful in teaching African-American boys how to read--and which of their teachers are consistently the least successful.

Of course under the current contract, the district can't use that data to make any hiring or lay-off decisions. It's just evaluations for evaluations sake and therefore it's not "punitive."

My question is.....punitive for who? What about the "punitive" effects for families whose kids are stuck in a classroom with an ineffective teacher who, year after year after year, has been in the bottom 20 percent of his/her teaching peers when it comes to teaching reading or math skills?

Out of curiosity, how did private schools do compared to public in the same areas? I know the data is available--I saw it. With a quick glance, I didn't see any striking differences from public, but I didn't spend any significant time on it, and I don't have time to directly compare the data (comparing apples to apples might be hard since the economic disparities in private schools are probably much less significant than in public schools).

While I agree with the writer that standardized testing isn't exactly coffee shop talk, that doesn't mean it shouldn't be.

It is true, however, that the data-based nature of the conversation can get a bit too wonky for regular moms and dads to be interested in-- here's the takeaway: the Minnesota Dept of Education is ramping up the rigor in the area of mathematics, and despite a considerable level of focus on math that is part of the NCLB measurements (to the exclusion of other core subjects)-- MN school children have a lot to learn.

Discover more on how you might present the data to people of your district by visiting: http://hakesfordistrict621.com/. MDE features a wealth of statistical information in its data download section that will make your own analysis possible.

Trying to improve education with the mind set implicit in the current system is like moving deck chairs on the Titanic.

To get a wider perspective it may be helpful to listen to Tom Friedman's recent talk at the Humphrey Center: http://bit.ly/mWCxnR

Or check out the PISA http://bit.ly/qRiyc6 or http://bit.ly/oWudyU