Given how much lip service accountability, performance and outcomes get in American education these days, it’s remarkable how little we actually know about how well a given school is doing. A school can do great things with struggling students and still end up labeled a failure, or post high test results year after year without really challenging kids who started out proficient.
A better measure for evaluating a school’s ability to close the achievement gap is how much learning kids get in a year, most reformers believe. If a student starts an academic year several grades behind and makes two years’ progress, her school has knocked one out of the park, whether she’s deemed proficient or not. Similarly, a school that consistently fails to deliver a year’s learning each year might need shuttering.
A Minneapolis nonprofit that works to increase the quality of local charters, Charter School Partners, has come up with what it believes is a better way to crunch data about student performance, a graphic tool it’s calling the Minnesota Academic Performance for Charter Schools, or MAP.
The MAP is, quite literally, a map. Or two interactive maps, to be precise, one charting charter schools’ performance in math and the other in reading. One axis notes the percentage of students who test proficient, while the other shows the percent of growth.
Diamonds representing schools are scattered across the center, which is divided into four quadrants: higher-proficiency/higher-performance; lower-proficiency/lower-growth; lower proficiency/higher-growth and higher proficiency/lower-growth.
Colors indicate relative wealth, poverty
High-poverty schools are green; wealthier ones blue. Users scroll over the diamonds to see which schools they correspond to and what the data is for each. (Mapping and data freaks can find a YouTube tutorial on the tool here.)
The result is a different portrait of school performance from the one offered by Minnesota’s historically one-dimensional school report cards. At Global Academy in Columbia Heights, for example, 92 percent of students are impoverished and 71 percent are learning English.
Only 66 percent of students are proficient readers below the state average of 72 percent. But 56 percent made exceptional growth last year, versus 35 percent statewide, according to state test data.
As is often the case when students are learning English, math scores are higher: 88 percent are proficient, significantly higher than the state average of 66 percent. Meanwhile, 58 percent made exceptional growth, versus the state rate of 34 percent.
The MAP doesn’t say what, but clearly someone is doing something right at the 2-year-old program. If they keep doing it, the gap will close. And if someone can identify just what’s going right, it might help other schools accelerate learning.
Lower left quadrant houses some abysmal performers
Conversely, the lower left quadrant is home to some abysmal performers. At the 8-year-old BlueSky Charter School just 15 percent of students are poor, 11 percent are proficient at math and 12 percent outpaced average growth rates. Reading proficiency is 47 percent, but growth rates are less than 29 percent.
The Minnesota Department of Education recently told BlueSky it had 30 days to change curriculum and graduation standards or face fines of up to $18,000 per day until fixes are made.
There are some lower-left schools that merit disclaimers: They may be new, or in a couple of cases, work only with special-ed students, according to Charter School Partners’ Brian Sweeny, director of business excellence and advocacy. Similarly, some of the schools in the lower-right quadrant would be hard-pressed to show exceptional growth because they have long served highly proficient student bodies, he noted.
Data drawn from state statistics
The data underlying the MAP are drawn from state statistics and the much-reviled Minnesota Comprehensive Assessments. The 2009-2010 school year was the first time the state collected the kind of information that allowed this type of analysis.
Until recently, all the MCAs measured was the percentage of kids in a classroom who were proficient in math or reading. This year, Minnesota began measuring how much individual kids learned from one year to the next, and reporting how many showed low or high growth, regardless of proficiency.
Charter School Partners hopes to use the MAP to identify good practices that can be replicated in other schools, Sweeny said. But more than that, the organization hopes it will be helpful to parents trying to decide whether to enroll their child in a particular program.
If a school is a good fit for reasons that have nothing to do with academics, and is delivering high growth, parents might not need to be scared of low overall proficiency. Similarly, they should probably walk away from an established program with bad numbers.
All the caveats, disclaimers, colors of the rainbow, and such, point up the fallacy of trying to “judge” schools using reams of numbers. The point, I believe, is NOT to figure out which schools are the best, because that can’t really be done without going to each school, spending tons of time there, talking to everyone,and getting a true understanding of exactly what is going on there.
The real point of these “studies” is to sow confusion and to continue the misguided myth that charter schools can be made to compete, educationally, with regular public schools. But they can’t – on average they are twice as bad as regular public schools. According to figures from the Minnesota Department of Education, charters are seven time more likely to be failing than regular public schools. And that’s a fact you won’t hear from self-interested fog machines like Charter School Partners – which by the way is connected to Joel Kramer, the publisher of this website.
I will leave it to others to respond to the merits of Rob’s argument about high-performance charter schools. Re his assertion about me, I have a daughter-in-law who works for Charter School Partners. I do not tell Beth Hawkins what to write about, and I did not know about this article until I read it on our site.
My point is that the discussion of education reform, and charters, in particular, is skewed by the liberal elite who push failed reforms. Most are in on it. Many make good money doing it. It is led by the organizations like the Minneapolis Foundation, which funds all these supposedly liberal causes. There is an elite group think going on with charters that is deluded and hypocritical, as well as dangerous to our educational system.
Despite the focus on “numbers” and “what works” we almost NEVER hear about the true track record on charters. Why is that?
Yes, there are a few successful charters. But as Diane Ravitch has written, there is no pattern, nothing extensible. Which is why that for every student who does better at a charter, two do worse. That is a fact. They are also failing at a rate seven times greater than regular public schools in Minnesota. That is a fact, too.
And despite the claim that charters exist to lessen achievement gaps, in Minnesota charters have particularly HURT poor and minority students.
I want to chime in and note that Joel Kramer truly does not get involved in the decisions I make about what to cover. We have never had a conversation about his thoughts on charter schools, or for that matter most education issues. For the most part, I personally find the charter vs. Mainline public schools debate to be stale, particularly because so many of both types of schools are failing massive numbers of kids, who don’t care what their governance structure is. I thought CSP’s maps merited a blog post because they might help identify elements that good schools use to accellerate learning. I’d love to see the same analysis applied to every school in Minnesota. My guess is we’d be in for some big surprises.
I appreciate your point of view, Beth. I didn’t mean to imply that Joel Kramer called you up and asked you to write this post. Quite the opposite – my point is the the whole discourse is skewed by an education reform establishment that publishes endless surveys and studies, all ignoring the overall trend. While you may find the debate of charters vs. regular public schools “stale,” the fact is that more charters = worse public education, so it’s relevant.
As a parent of two young children enrolled in a charter school, I can speak to the “when it works, it works great” aspect of our experience, as our school occupies the high proficiency/achievement quadrant. What’s problematic is the lack of consistency of outcomes among charters, and the glaring failures of more than a few. And much the same is true of Minneapolis public schools as well, which have produced a failure rate that has shocked the state and the nation.
I appreciate Beth’s comment about governance… parents care about results, as do students. We chose to go the charter school route not because we love charters, but because the good public schools available to us are turning people away. At kindergarten open houses, they pretty much tell you you don’t stand a chance unless a sibling is there already. And sadly, the same is true of successful charters. We were extremely fortunate to get in.
For the second year in a row, the state withheld promised funds for schools, a 30% shortfall for our school at least. The dysfunction in our nation’s educational system is as mind-blowing as the lack of funding, and parents will continue to turn to charter schools in order to feel some sense of control over the outcome of their children’s futures. We have, and it’s working for us.
And honestly, if I hear “liberal elite” one more time in relation to the failures of schools, I’m going to scream. The failure of our schools falls on all of us. We fund the largest prison system in the world with 20 times the money we invest in schools. So we can’t be appalled by the results… or even surprised.
Just to be clear, when I say “liberal elite,” I’m talking about the liberal and Democratic sell-outs who have cavalierly sold out our education system, and much of the rest of the country, to the privatizers, corporatists, and plutocrats. Not the “liberal elite” who right wingers complain about bringing socialism to our country. There’s a difference.
Rob–I’d love your take on one question I had about the maps: Typically, we assume poverty correlates with low achievement. But that doesn’t hold so true on CSP’s graphs.
If you’ll pardon the old retired teacher guy…
Schools neither fail nor succeed, whether they’re public, private, parochial, charter or Martian.
Most of this (and thousands of preceding) discussion about education misses much of the point because it begins with a very flawed assumption that somehow schools “do” something. They do not.
Principals and other administrators “do” something. Sometimes badly, sometimes well, but they’re being paid, essentially, to supervise and set the “tone” or “ambience” inside a school. They also evaluate teachers, even though they often have no relevant teaching experience themselves.
Teachers and guidance counselors and other professional staff, but especially teachers, also “do” something. They have subject matter to present to students, skills whose further development they want to foster and encourage through exploration and practice, and evaluations to conduct, moment-by-moment as well as through written and other performance-based testing.
And of course, all of this is pointless without the student, who must also “do” something. It’s the job of the student to learn the academic concepts and material being presented, to demonstrate that learning by various means, and to explore and practice new skills as part of the instructional program of which s/he is a part.
If I may take issue with Ms. Hawkins…
“…If a student starts an academic year several grades behind and makes two years’ progress, her school has knocked one out of the park, whether she’s deemed proficient or not. Similarly, a school that consistently fails to deliver a year’s learning each year might need shuttering” is an exercise in misdirection.
I’d argue that, in the first instance, “the school” has done nothing beyond provide the opportunity and the means, and the STUDENT has “knocked one out of the park.” I was fortunate to have had a few students like that – a couple of them recent immigrants to the United States – in my 30 classroom years. They make a difference in both their teachers’ days and those of their classmates. Be that as it may, my point is that it’s the student who has accomplished something, not the school. Similarly, though at the other end of the spectrum, a student who is in the same place at the end of the year as s/he was at the beginning does not reflect a failure on the part of the school, which might be the very same school as the first example. The student stuck in the same place at the end of the year reflects a failure on the part of that child.
You could – and there are some instances when I might join in – place the blame for that failure somewhere other than squarely on the shoulders of that student. The single most reliable indicator of a child’s academic success in this society, as measured over and over again over the past half century and more, is the socioeconomic status of that child’s parents. That doesn’t mean poverty-stricken kids never rise to the occasion – quite a few do. Nor does it guarantee academic success to the offspring of the affluent. I had quite a few sluggards in class whose parents garnered several times my income. On the whole, however, socioeconomic factors weigh heavily on educational outcome, and simply cannot be ignored if what we’re trying to do is genuinely understand the appalling achievement gap that continues to grow in our schools.
Diane Ravitch has written eloquently about this on more than one occasion, most recently (that I’ve seen) in the New York Review of Books, where her latest article is directed squarely at what she calls “The Myth of Charter Schools.” While not overtly hostile to all charter schools – it’s a concept she used to promote – Ravitch has made a distinguished career of studying the history of education in the United States, and charter schools have been around long enough that a substantial database is accumulating around the results. Some charter schools, like some public schools, are great places to be a kid and to learn. Some charter schools, like some public schools, are God-awful places to send a child for any reason, including learning. Most of them, like most of the public schools regularly being crucified in the media, fall somewhere in between those extremes.
In short, charter schools are not significantly better than “matched” public schools that serve similar student populations. Another point Ravitch makes is that our economic competitors in the world, whose students our own are falling farther and farther behind, do not generally have superior (i.e., more elaborate and expensive) facilities. What they DO have are students who are not bombarded with advertising, societies that treat teachers with respect, administrators who have extensive teaching experience, parents who pay attention to the education of their children, and a culture that values academic success.
Well, in our case, it was a republican governor, hardly a liberal elite, who defunded schools of a good chunk of their operating budgets— not once but twice, and probably for the foreseeable future. But the deliverables for schools remain the same, whether they get all their money or only 70%. It’s a losing proposition, no matter who is behind it.
My point is that most kids and parents just don’t care how good education gets done, or who does it. And to point the finger at one group over another absolves a lot of guilty parties. We all need to get behind fixing education in this country, by any means necessary— public, for profit, charter, or a mixture of all of the above. And we need to fund them as robustly as we do our prisons. In the meantime, the results of the current state of affairs speak for themselves.
Beth – there could be many answers to your question. First, there is the question of the validity of the testing regimen and the statistics themselves.
Even if you concede that the statistics are correct (though I don’t), there could be many other factors in success, such as a cultural homogeneity of a community, support from outside groups like churches, philanthropies, etc.
There might also be other government programs which support the children there in early childhood development. There could also be a difference in the socio/economic or educational attainment makeup of the parents. In short, graphs like that show very little, and they especially miss the larger picture concerning charter schools.
Just to make two other important points:
1) Educational achievement has been increasing since the 1970s, when true liberal reforms were implemented. These gains slowed, but didn’t stop, since the advent of free market reforms in the 1980s. Achievement gaps have also decreased, following the same pattern.
2) As but one example of not trusting the high-stakes testing done for the past decade or so, look at New York State. Remember all those reports of increasing achievement and decreasing gaps concerning the so-called successful charter schools in NY? Turns out they were rigged – the tests had changed, making them easier.
The research does show that only a small percentage of charter schools outperform comparable district schools, but Mr. Levine appears to have confused “correlation” with “causation.” To make the claim that more charter schools worsen public education is comparable to making the claim that more teachers worsen public education. Neither is a factual statement and neither could ever be inferred from the research.
I support charter schools, not failing ones, but successful ones that are showing outstanding results and closing the achievement gap. The problem is not that we have charter schools, but rather that we don’t have enough high quality charter schools. Don’t throw the baby out with the bath water…more charter schools are “beating the odds” than traditional district schools in MN and around the country. The reason they are able to do so is because of their flexibility, their autonomy, and their mission-driven focus. In other words, being a charter is what allows them to be successful.
BETH,
Saint Paul has been doing these value added tests for two years now, called the NWEA-MAP tests. They are an improvement over just the MCA. The kids, your specific kids, take the test in the fall, winter, and spring and measure growth. These are an improvement, but it is still too easy to use them in a punitive way. Really reform has to include an unflinching look at student data to improve instruction and student outcomes. This will never happen when the data game is so punitive.
In addition, even the value added tests have little use in judging a teacher. For example, I had two classes take the MAP last year. One class achieved over 150% of the score increases that were expected. My other class was the most challenging I’ve ever had and they increased on 95% of expected. So did I do an exceptional job or was a failure? When we looked at the same kids reading scores, my high achievers gained 200% on the reading test, so I actually underperformed the reading teachers. However, my failure class only achieved 20% on their reading tests, so they overachieved. I was the same teacher teaching the same class using the same curriculum. Was I a rousing success or an abysmal failure?
Sue Wollan says:
“Mr. Levine appears to have confused “correlation” with “causation.”
That is a truly ridiculous statement. I didn’t say anything about correlation or causation. I said that the most definitive studies have shown that charters, on the whole, do worse than regular public schools. If charters do worse, on average, than regular public schools, then replacing regular public schools with charters will worsen public education. It is a simple equation.
You also say, “I support charter schools, not failing ones.” That’s like saying I support lottery tickets, but not the losing ones. It makes no sense.
Even though some charter schools may have just as many students from families in poverty, they have one thing public schools don’t. All of the students chose to attend the school and supposedly want to be there, what ever their income. That is a critical element lacking in public schools. Many kids in public schools are simply putting in their time until they don’t have to be there. In general that is not the the case with charter schools. Public schools get the whole bottom tier of kids whoose parents have failed to ready them for learning and care very little about it for a host of very tragic and stubborn reasons. This is significant.
Student and school math and reading performance may seem plausible to measure, but the measures we need are not individual student or teacher or school measures. What we should be measuring are system measures. The reason is that performance of a student (or teacher, or principal) is 95% attributable to the system and 5% to the individual. Education does not have a teacher or student or school problem, they have a system problem. The system is comprised of all those in the education system (parents, teachers, administrators, students) and other elements like structure, technology, work design, social status, resoources, system conditions, management thinking, etc.
The argument that individual performance can be separated from the system is a flaw. Trying to make meaningul changes to improve student performance based on individual measurments, no matter how well intentioned, will result in dysfunctional behavior by the people affected by the measurements–in this case members of the schools’ staff–to improve test scores at the expense of real learning.
For example, the decrease in student scores in NYC after more stringent tests were instituted demonstrated that under the old tests people were gaming the system to increase scores.
Until we can move from thinking in individual terms to thinking in system terms we are on a fools errand.
Ray Schoch says that “Some charter schools, like some public schools, are great places to be a kid and to learn. Some charter schools, like some public schools, are God-awful places to send a child for any reason, including learning.” That pretty much sums up the state of education: there are great traditional public schools, charter schools, alternative schools and private schools; and there are also schools of each type that aren’t places we’d want to send our own children. Condemning all of one kind of schools because there are bad schools of that kind doesn’t move us toward higher quality. Instead, we should be focusing on learning what does increase student achievement and finding ways to create more schools that incorporate what works. I’d like to see the argument move from an “us against them” mentality about charters vs. other schools, and instead start focusing on ensuring our students have a variety of options for high quality schools. Let’s give charters, as well as other types of public schools, a chance to ensure our students have the knowledge and skill they need to be confident citizens capable of succeeding in their work, personal and family lives.
I agree with Joel that our present evaluation system focuses on individual student performance, or an aggregate of individual student performance. So those who want the “best” school are inclined to cluster together in a high ranking school. However, I think many people in the education system would say that the clustering of students by performance is fostered by demographics and does not optimize public education as a public good. A measure of how well a school performs given a diverse student population would allow parents who wish to contribute to an optimal public school system (and understand that their children can in fact be successful in these schools) a means of making am informed school selection.
Susan Doherty says:
Saying that, overall, charters do a worse job of educating children than regular public schools is not “condemning all of one kind of school.” As I said somewhere in this thread there are some good charter schools. But a higher percentage of charter schools do poorly than do regular public schools.
Then Susan says:
That is the kind of cognitive dissonance I see over and over in this education debate. Let’s do what works! the education deformers say. Well, the logical extension of that is, let’s NOT do what doesn’t work.
Charter supports have some sort of built in denial system that lets them think they want to do what improves public education, all the while pushing a proven failure.