Like other school leaders, Eric Mahmoud, the president of Harvest Network of Schools, got the results of this year’s Minnesota Comprehensive Assessments (MCAs) about a month ago.
They showed significant drops in achievement at Harvest Network’s three schools.
“As a matter of fact, it was hard to get out of bed,” Mahmoud said. “But when I did get up I knew it was time to take a deep dive into the numbers.”
During the six years that the news media and state officials have been highlighting positive outliers, Harvest — where 99 percent of students are black and 95 percent live in poverty — has been among the Twin Cities’ top 10 high-poverty, high-performing schools.
The ability to reach those students has made Harvest a national model, with a book published on its methods and plenty of media attention — including an entire episode of Soledad O’Brien’s CNN series “Black in America,” which screened at the Convention Center. Harvest’s success also earned Mahmoud a place in the National Alliance for Public Charter Schools Hall of Fame.
Yet this year, overall math proficiency at Harvest Prep, Best Academy and the Mastery School is down 20 percentage points, to 53.3 percent. Reading scores dropped 9 points, to 33 percent. For the first time, the school made the top-10 list only in math.
To be sure, the three schools still outperform the state math average for black students and Minneapolis Public Schools averages in reading and math both for African American and impoverished students. But Mahmoud and his colleagues say the dropoff is unacceptable.
“We had an explosion here,” Mahmoud said. “We are doing the forensics to try to figure it out.”
Seven hypotheses
Harvest is hardly alone in that effort. Overall, reading scores are up a single point over last year, for a passing rate of 60 percent. Math scores dipped 2 percent, leaving 60 percent of students proficient.
Gaps persist largely unchanged between white students and children of color, impoverished students, those with disabilities and kids learning English. In some subgroups and subjects, passage rates barely make it into the double-digits.
And Harvest is not the only historically high-flyer that, while still ahead of the pack, saw its test scores fall back toward earth. At seven of the top 10 high poverty schools, fewer than half of students read at grade level. Only three of the top 10 math odds-beaters had passing rates of more than 60 percent.
(Here we pause for the Kramer Disclaimer: Two of the schools on those lists are part of the Hiawatha Academies network, which is led by Eli Kramer, son of MinnPost founders Joel and Laurie Kramer. None of them was consulted during the preparation of this report.)
Doubtless few school leaders are celebrating this year, but Mahmoud had particular reason to pore over the subsets of data the MCAs delivered. In an effort to improve the skills that will help Harvest graduates complete college, Mahmoud has pushed his staff to learn complex teaching methods that drive critical thinking and other higher order learning.
After he dusted himself off, Mahmoud did two things. The first was to develop a list of seven hypotheses as to what had happened. The second was to ask Dave Heistad, the dean of Minnesota’s assessment and evaluation community, to analyze the schools’ scores. He also asked Heistad to drill down to figure out what was going on in the classrooms where gains were made.
Now the Director of Research, Evaluation and Assessment for Bloomington Public Schools, Heistad was involved in the design of many of the state’s best data collection and analysis tools.
According to Heistad’s report, at least two of Mahmoud’s hypotheses likely were significant factors in Harvest’s 2015 losses: a dramatic increase in new special ed students and the elimination of the network’s longer school year.
The demographic shift was probably the single largest factor, according to Heistad. Last year, the percentage of all Harvest Network students in special ed shot from 8 percent to 19 percent. That and an increase in the number of English-language learners (to one in five) brings the schools in line demographically with Minneapolis Public Schools. It also makes the coming years a test of whether the strategies that propelled Harvest onto the list of odds-beating programs can keep them there.
But last year, it meant trying to reach more than 100 new students whose disabilities had already been diagnosed, but possibly undertreated, quickly.
“That’s a good thing,” said Mahmoud. “Those are the students we want. What we didn’t anticipate is that they would have to take the same test as everyone else.”
This year Minnesota began requiring virtually all special ed students to take the regular test without modifications. And every single one of them had to do so online, which presented challenges for a population used to adapting paper tests for kids with learning disabilities.
A ‘concierge service’ to stay in touch
Add to that another problem Mahmoud is happy to have: a high number of students new to the schools. The three schools enrolled 400 new students this year, 200 of them in newly added seats, bringing the percentage of new students up dramatically. In both third and fourth grades in the original Harvest program, for instance, the number of new students was nearly triple what it was last year.
Not surprisingly, scores were much higher among students who had attended the network two or more years. Just 28 percent of first-year third graders passed the math test and 13 percent reading, for instance. Meanwhile, all students with Harvest two or more years passed math and 68 percent reading.
Struck by the difference in performance between new students and those who have stable attendance, Mahmoud started a “concierge service” to stay in touch with families to make sure they come back. This fall’s predicted retention rate from last spring: 85 percent.
Unstable housing is the number one obstacle, so 12 Harvest employees have made contact with 100 families each at least three times this summer. If a family’s circumstance included, say, a move from Minneapolis to Brooklyn Center, the school makes sure they know it sends a bus to the northern suburb.
Fighting the summer slide
The other big issue the data pointed to for both Mahmoud and Heistad was the impact of losing summer school. Because the state cut the funding that paid for the extended year, last year was the first in seven that Harvest had to cut its school year — by five weeks.
Comparing Harvest’s spring 2014 assessment data to last fall’s, Heistad identified a pronounced dip. It’s typical to expect kids on average to experience a “summer slide” of about 1 percentage point.
Because they do not spend their summers traveling, attending camps and other enrichment programs the slide is more pronounced for impoverished kids. Research is divided on how summer after summer of slumps compound, but the bottom line is it’s harder to make up.
Harvest’s students saw fall-over-spring dips of 3 percent to 12 percent. To put that in perspective, Mahmoud noted that they expect growth of 7 percent to 9 percent per academic year.
Heistad was out of town this week but the assessment czar who preceded him, Jim Angermeyr, now retired, said students frequently make up summer losses by the following spring’s assessment season. But the loss of several weeks of programming has to be significant, he said.
This year’s legislature reinstated extended year funding, but—inexplicably, because high-performing charters are the schools most likely to have longer school years—now reimburses charters one-fourth the funding available to mainline district schools.
Mahmoud said he has already moved to plug the time gap. “If we have to cut whatever, we have to have that time,” he said. “Fortunately we had the data early enough we could react to it.”
As a consequence, when Harvest students start school later this month they will have a school day that’s two hours longer. And those that arrive with behavior challenges and learning disabilities will see a beefed-up special ed staff.
New standards also create challenges
The last big issue is one named by school leaders all over the country who are facing flat or declining assessment data. New, more rigorous standards for what students should know are challenging teachers.
As he was this time last year, Mahmoud is convinced the new standards—designed to require critical thinking, problem-solving and other complex skills—are imperative. Experience with intellectual challenge is crucial for making sure that the schools’ alums persist through college.
Harvest’s teachers are coming back early again this year. Among the tasks they will be asked to do is to work with a new tool school leaders developed that will help them identify the best approaches.
“It was tough to present the news,” Mahmoud said. “Our teachers work their behinds off. You want to give nothing but good news to inspire them.”
He also asked Heistad to crunch the numbers for individual classrooms, not to punish teachers but to identify those who were very successful. So when he presented the grim overall results to his staff, he was able to call out the school’s stars.
“We don’t have to go to New York,” he told them. “Right here, with new students, with more special ed students, with summer slide, here are your peers who are turning it around.”
Can they turn it around in every classroom? They don’t have a choice, Mahmoud said.
Hmmm…
Not to pile on to already-bad news, but this sounds pretty much like a case of a much-lauded charter school finding itself in difficulty when it, and its students, have to live in the same academic world as “regular” public schools. Dealing with special-ed students has been a long-standing problem with public schools all over the country, the vast majority of which have never had the option of saying, in effect, “Well, no, you don’t fit our profile. Sorry.” The extended summer vacation, a holdover from the days when agriculture was the primary employer and industry throughout the country, ought to be done away with nation-wide. That Harvest was able to game the system in such a way that its kids had an extra 5 or 6 weeks of instruction surely accounts for a significant portion of the example Harvest purported to set for all those “ineffective” teachers in regular public schools. And, of course, funding is a perpetual issue for all schools in a society as anti-intellectual as this one.
I wish Harvest all the success in the world, but it’s kind of refreshing to see that, when confronted with the same kinds of challenges as “regular” public schools, Harvest’s results appear to be pretty “regular” themselves. If Mahmoud and his colleagues are able to successfully overcome the achievement issues confronting them now, I hope they’ll live up to their obligation to share their methods and techniques with staff and administrators in the MPS and other school systems. That is, after all, the rationale for charter schools in the first place – to serve as laboratories for new and/or innovative approaches and techniques which, if successful, can and will be exported to their more conventional public school neighbors.
Great comment
Ray
Their results, while lower, still are better than public schools. That shows if you had vouchers to let parents decide where their children go and have a say in what/how their kids learn, things can get better. It would make all schools compete for students and that would make for better education. What we are doing now is not working!! We are spending billions and test scores are down, drop out rates are up and basic skills (math, reading, writing) are getting worse by the year. At what point will folks just admit our current system is broken, especially in the inner cities. Give parents a chance to pick their child’s school and see if results change.
We already have competition in Minnesota
in the form of open enrollment. Thanks to local control, school districts differ sharply, and Minnesota parents can choose.
As far as spending “billions” is concerned, well, yes, quality costs money, and so does solving problems. If you’re appalled at how much the public schools spend per pupil, take a look at what some of the local private schools (the ones not subsidized by churches) charge for tuition.
Blake starts at $14,000 a year for nursery school and goes up to $23,000 for high school. Breck starts at $26,000 for first grade and goes up to $27,000 for high school. Neither one takes special ed students, the students with limited English come from affluent families, and the parents are strongly motivated to provide the best education for their children.
Test scores are not meaningless, but the whole No Child Left Behind scheme was a “bipartisan” effort promoted by George W. Bush and Ted Kennedy, neither of whom ever attended a public school in his life.
Do you know what the real measure of a school’s success is? Whether the students are intellectually curious, whether they read for pleasure, whether they have at least one interest that isn’t sports or pop culture, whether they have a b.s. detector, whether they have participated in an extracurricular activity that will last a lifetime, whether they can express themselves coherently in speech and writing, and whether they can find a vocational path that suits them. Work on those values, and the test scores will take care of themselves.
Instead of tearing down the public schools, let’s look at the features that rich people want for their own children: small classes, a challenging curriculum that provides broad knowledge, individual attention, teachers with at least an M.A. in their subject (not in education), the arts, foreign languages, well-equipped science labs, and well-maintained buildings and grounds.
The parents in Scarsdale, NY, don’t want their children subjected to high-stakes testing. Why should poor children be subjected to it?
Suppose we treated children from poor families not as lower life forms to be disciplined harshly, kept below their potential, and taught just enough to be fit for the fast food industry, but as budding human beings with unique talents who can do surprising things if we take the time to find out what makes them tick. (As a college professor, I was often amazed at how unpromising, lackadaisical students could become superstars in at least one subject if they found something that really interested them.)
Suppose we got rid of the layers of administrators and hired classroom teachers for those salaries.
Suppose we got rid of the varsity sports teams and involved every student in some kind of physical activity, even if it was just walking or riding a bike to school.
Suppose we returned kindergarten to what it was meant to be, not a year of worksheets and tests, but a year in which the teacher nurtures the intellectual, physical, and cultural skills that the children will need when they learn to read in first grade. (Don’t worry. Some European countries don’t start reading instruction until age seven.)
Suppose all elementary school classes were limited to fifteen pupils, as in the best private schools, Suppose every child had the opportunity to study literature, art, music, history, geography, political systems, the natural sciences, foreign languages and cultures, and other studies that help young people become informed citizens who can use their leisure in productive ways.
Suppose each teen was guided into the vocational path that he or she was most suited for, even if it meant putting the rich kid in an apprenticeship program and sending the poor kid to Harvard.
Suppose our mass media followed the admonition on the bumper sticker, “Stop making stupid people famous.” Instead of endless coverage of empty-headed people doing boring things, what about smart people doing interesting, exciting, and noble things?
Suppose the politicians who advocate privatization of anything and everything didn’t have friends and major contributors standing by and waiting to take advantage of outsourcing of government functions.
You’ve stated a real mouthful of common sense and insight !!
I especially appreciate your iteration through a number of characteristics of an intellectually ALIVE person in the suggestion you make of what the “real measure of a school’s success” is. This is not some pie-in-the-sky ideal, it is the entirely natural outcome of an effective education. How so many of our kids are being cheated of their life’s potential !!
Eric Mahmoud is right on the money in his concern that students must graduate with “critical thinking, problem-solving and other complex skills”. I admire this man for his incredible determination and sustained effort regardless of the numerous headwinds this column identifies. I hope his teachers hang in there with him.
Suppose our children learning how to learn (not what to learn) was a priority. Suppose the billions spent actually increased graduation rates. Suppose the billions spent produced more engineers so we don’t have to import them. My biggest suppose is we actually get the quality that costs billions. High school students in America are rated 26-35 in the world at basic skills. Suppose that was better.
Countless middle-aged engineers unemployed or
working only on short-term contracts is a phenomenon that suggests that there is not actually a shortage of engineers, only a shortage of employers willing to pay for them.
Why should young Americans major in computer science or engineering if they have seen their parents’ generation of IT professionals and engineers being thrown on the trash heap?
Lessons Learned
The breakdown of the data should remind all of us to look hard at the scores that indicate that a school is doing better than expected. It is essential to see if the kids being educated and taught are the same across all schools and if other factors like time of instruction, length of enrollment, native language, etc are comparable. IF they’re not the same than it is unclear whether the apparent superiority of instructional methods is true or not.
I Agree
My kids are in Robbinsdale which is a challenging district. We often lose Parents/Students to Wayzata because they see the better test scores as proof that it is a “better district”. When in reality they just do not have nearly the same challenges to overcome. (ie 10% vs 40% poverty)
I also find it frustrating when Conservatives try to compare Status Quo Publics directly against Charters/ Magnets, all the while ignoring the demographic and parental differences.
On the other hand, the union folks tend to seek excuses in the results rather than aggressively changing their culture / systems for the good of the unluckiest kids.
For your amusement…
http://www.brainyquote.com/quotes/keywords/statistics.html
District schools have received summer $ for years
Mr. Schoch, district public schools have received $ for summer programs for years, and continue to receive them. Last year MDE convinced a court that charters should not receive funds for summer programs – despite the fact that for years charters had been receiving the $, as had district schools.
District public schools are receiving thousands of dollars more than charter public schools.
Fortunately both Harvest Prep and MPS have been working together to increase the # of students who can attend a school like Harvest – which has had a major waiting list.
As for measuring a school’s success – unquestionably it’s far more than test scores. One of the contributions of the charter world has been the Hope Survey. This measures whether students are learning to set and work toward goals, and whether they feel they can accomplish their goals. More info here:
https://www.hopesurvey.org/
Our Center also advocates the best of parent, survey and graduate surveys, which should made public. We think the percentage of students involved in some form of community service, and what they are doing, also is important. Those are a few examples of assessing what’s happening in a school, beyond test scores and graduation rates.
test scores
It is a mistake to lump ELL, special ed, and other scores together and average them. Special education scores are tremendously sensitive to the composition of the students. Statewide only 25% of the students in the special education category meet the proficiency cutoff. Comparing a school with 20% special ed composed of many high degree of difficulty students to a school with 5% special ed with low degree of learning difficulty makes no sense. Whether we like it or not, the challenge for many special education student in taking the highly demanding MCA-III reading test can be very daunting, especially if accomplished without accommodation.
Rolling ELL students into the mix is equally problematic. Some ELL students come from families where the parents have English; some come from refugee camps where they have had no school at all, from families with no English and in some cases, no written native language. Putting an MCA-III common core test in front of a child who has a couple years of English in school and no language support at home, is likewise a daunting task.
ELL student performance on the MCA-III in Minnesota is at the 15% proficiency rate. In a school that is adding ELL students at a high rate, you can be adding 50 kids a year who are years away from being able to score proficient on the MCA-II, even if the school is doing a knockout job of teaching English. But the MDE insists on dumping all these scores, without interpretation into the media all at once so that the media can compare school A with 15%ELL, most of whom are new to country and 20% special ed, and compare them to a school where the parents are all college graduates, and then announce that the one school is obviously better than the other.
Teaching a bright kid with dyslexia is different from teaching a kid with oppositional defiance disorder, is different from teaching a child who needs three hours to learn what another child needs five minutes to learn, is different from teaching the child of a doctor or lawyer who goes to math camp in the summer and gets read to for an hour. It’s great that we’re finally getting some recognition from the charter movement that the degree of challenge makes a difference.
We need to provide charters and traditional publics with the resources to be successful with all children. If that means summer programs, professional staff development, or whatever the heck it takes. Let’s just stop pretending that it is the charter model that makes the difference: its what goes on in the classroom, the quality of the teacher, the leadership in the building, the additional staff support, the collaboration model, the professional staff development and all the rest.
Where to start
Ray,
“That Harvest was able to game the system in such a way that its kids had an extra 5 or 6 weeks of instruction”. How is this gaming the system? I am pretty sure the normal districts and Teachers could do this if they really wanted to put the needs of the unlucky kids before their own.
Regarding even playing field, please remember that Mpls schools gets a lot more money per student than a local charter. They get the local tax revenue.
Now why again would a charter happily collaborate with a union led near monopoly that will not even sell them their old building that the district can no longer keep full. I always thought it was odd that the districts could tie up those tax payer funded buildings for their own gain.
Joe,
Though the charters have the advantage of less bureaucracy, fewer work rules that put the child second and other benefits. One of their biggest advantages is that parents of the kids who go there care about their child’s education. The Mpls public gets what is left.
Karen,
The expensive Privates also accomplish incredible academic results. Not so for the Publics that have nearly the same funding. (though student body makes most of the difference)
Also, if you know engineers and computer personnel who are unemployed in MN right now. You know some folks who did not stay current and honing their skills. Or they simply want too much money for their realistic capabilities.