Nonprofit, nonpartisan journalism. Supported by readers.

Donate

Community Voices features opinion pieces from a wide variety of authors and perspectives. (Submission Guidelines)

Who questions the test questioners?

scantron
Too much standardized testing operates behind a veil of secrecy.

For a public service, too much standardized testing operates behind a veil of secrecy.

We have a more polite term for it — “test security” — but what it amounts to is secrecy.

diedrich
Michael Diedrich

When I was teaching, sealed MCA testing packets would arrive in boxes at our school. At the beginning of each test, I would have students unseal the packets, take the test, and re-seal the packets before collecting them into a box. Then they went back to the testing company (American Institutes for Research, a nonprofit) for scoring and analysis. Even though my school was being held “accountable” for how students did on these tests, the only people who ever saw the tests in their final form were the testing company and the students themselves.

Couldn’t I have snuck a peek at the questions? Sure, if I wanted to violate testing procedures. Check out the 2012-13 procedures manual [PDF] for MCA testing. Those administering tests are specifically told not to, “View test items for any reason except as allowed in the administration of an assessment,” nor were we supposed to, “‘Look over the shoulder’ to read test items when monitoring students taking a test.” Also on the “naughty” list: “Offer an opinion to a student, class or other staff member that a question is ‘bad’ or doesn’t have a correct answer.” (Apparently singing the praises of a question is totally fine.)

In other words, too much testing operates in a black box, with limited opportunities for oversight. Even when some efforts at oversight exist, they often don’t prove sufficient.

As the Atlanta Journal-Constitution has found, several states have administered and continue to administer questions of dubious quality. These include, for example, questions without right answers and questions that tend to be answered incorrectly by otherwise high-scoring students while being answered correctly by otherwise low-scoring students. While Minnesota does operate Item Review panels to advise AIR on question quality, we have to take it on faith that they’re responding appropriately to our concerns and screening out questions with weird statistics.

Concerns about bias

We also have reason to be concerned about cultural and class bias on our tests. I’m not talking here of active racism, but rather of the passive bias that tends to come with privilege. Minnesota does operate Bias Review advisory panels (which you can register for here), but again we have to take it on trust that AIR is responding appropriately to those concerns.

Minnesota appears to be in better shape than many other states, though we’re still far from ideal. We’re also fortunate to have a progressive administration that at least brings some reasonable skepticism about the appropiate scope and purpose of testing. The demise of the GRAD tests for graduation in the last legislative session was a good start; I’m hoping to see further critical evaluations of our testing mindset and approach.

It’s important to remember, though, that no matter how lousy tests can be proven to be, and no matter how bad they are for kids, there will always be some with an incentive to argue for more testing. I’ll discuss two groups: testing-as-ends and testing-as-means.

Testing as ends, testing as means

The testing-as-ends interests are the ones who directly benefit from a large amount of testing. Primarily, I’m referring to the testing companies who collect big checks from the state for developing and scoring the tests, and who often collect plenty of not-so-small checks from districts for “aligned” curricula (i.e. test prep curricula). While some may show restraint, all of them have an incentive to advocate for more testing.

The testing-as-means crowd are the ones who think that a large number of tests, administered regularly and changed every few years to keep everyone off their game, are a great way to slam the school system. Never mind that 70 percent of variation in test scores is driven by out-of-school factors. What matters to these folks is that there always be a reason to slam the people working in the public schools.

The best way to counter these forces is by having a clear, limited definition of what we want testing to accomplish. There are valid reasons to support a baseline of standardized assessment. Disaggregation of test scores by student group helps us identify areas where we as a society (including but not limited to our school system) have equity gaps.

Red flag or trigger for punishments?

Used as a red flag instead of a trigger for punishments, a small amount of testing that identifies schools and districts in need of more support can be a useful tool for promoting equity. We need to be vigilant that this testing be used for building awareness and support. If it is linked to dire consequences, it will lead to a narrowing of curriculum, developmentally inappropriate instruction, and increased pressure to cheat.

We also need to increase public scrutiny of testing companies at a national level. I believe most of the people working for them do so for the right reasons, but we need to demand more sunlight on the final products and how they’re being used.

Michael Diedrich, who taught English for two years as a Teach for America corps member, is a master of public policy student at the University of Minnesota’s Humphrey School of Public Affairs, where he is pursuing a concentration in education policy. He is also an Education Fellow at Minnesota 2020, on whose website this article originally appeared.

WANT TO ADD YOUR VOICE?

If you’re interested in joining the discussion, add your voice to the Comment section below — or consider writing a letter or a longer-form Community Voices commentary. (For more information about Community Voices, email Susan Albright at salbright@minnpost.com.)

Comments (7)

  1. Submitted by Barbara Skoglund on 10/14/2013 - 11:27 am.

    What is author’s testing background?

    I see the author is a teach for america person. Are you a psychometrician? Do you have any degree in education? In educational measurement or applied statistics?

    The development of educational test do not live in a black box – you may not have the professional expertise to be involved. There is no possible way to develop valid tests with the participation of any and everyone interested.

    I’d much rather take it on faith that MDE item review and bias review panels and the psychometricians involved in test development than I take it on faith that a month of summer fun makes an effective teacher.

    How test results are used are more a function of political pressure from the Legislature. Don’t blame test developers who repeatedly report what tests do and do not indicate. Tests are designed to measure a variety of things. Sadly the Minnesota Legislature turned tests designed to review overall state performace into tests to measure teacher and school effectiveness. Psychometricians didn’t do that, elected representatives did.

    • Submitted by Raj Maddali on 10/14/2013 - 07:38 pm.

      A TFA teacher…..

      Now Mr Diedrich i would have rather have you as a teacher than the over qualified, uncaring, biased, vengeful and spiteful English teacher my kid had in high school. If Minnpost permits me i will even publish her name.

      Sure according to the Union you probably are evil and incompetent, because you are not a psychometrician (whatever that means). But compared to the hell on earth teacher you sound like a caring and thoughtful individual who cares about the student. Now that English teacher copped an attitude on the first day of classes and was openly biased towards certain students.

      Have u seen what such hide bound, non caring (no parent feedback mechanism so who cares right), probably does to the enthusiasm of a kid who wants to study. Its eons worse than any lack of formal education in teaching that u do not posses.

  2. Submitted by Jim Bartholomew on 10/14/2013 - 02:14 pm.

    Can we move on?!

    MN’s state standards in reading, math and science – and the aligned exams (MCAs) have been created and vetted in partnership with MN teachers, high ed representatives and community and employer representatives.

    And congratulations to MN, our standards and exams have set higher expectations for students than most other states.

    Not sure what Mr. Diedrich’s goal is, but all his commentary does is include a bunch of contradictory messages.

    * We want people to be able to look at the test questions, but we need security so people don’t cheat.
    * Tests can be biased, but MN has a good bias review process.
    * Tests are good, but only if they aren’t used to spur changes where student performance is persistently low.

    As ususal, Mr. Diedrich would have us stuck talking about testing – not what we’re going to celebrate because its working, or what we’re going to do differently because it isn’t working.

    If excellence and equity is what Mr. Diedrich is looking for, then he should focus on what policies and practices our schools can adopt to better serve our students.

    The MCAs serve the critical role of telling us how well our students are meeting expectations, and where to start looking for success, and where to improve.

Leave a Reply