It’s common at the moment to draw a distinction between academic abilities and so-called soft skills. One, the conventional wisdom has it, is measured by tests, while the other reveals itself in life experiences.
Which attributes, news stories and Internet threads ask, count for more? Good academic scores can get young people into college, research increasingly shows, but soft skills are what enable them to earn a degree.
The either-or dichotomy is even starker when the talk concerns education policy. Can persistence, character and grit be taught? Will they be if test scores remain a part of the picture?
Over the last 20 years, the nonprofit Project SUCCESS has built relationships with tens of thousands of middle- and high-school students in St. Paul and Minneapolis. It does things that we assume defy quantifying, in terms of results, like giving theater tickets to kids’ families.
Impact of enrichment
Along the way, the organization has kept track of which students participate in its programs, which also include student musical productions, college tours and Boundary Waters canoe trips. And it has collected testimony on the motivational impact of those experiences.
On evaluations, 80 percent of alumni say the program motivated them to be more self-directed, to set goals and track progress on their after-school plans. Even more students put the program ahead of teachers and counselors in terms of helping them plan for their futures.
That’s nice — but does it translate into higher achievement? It does. And the higher the “dosage,” or exposure to enrichment, the bigger the effect.
Recently Project SUCCESS was handed numbers linking its work to better in-school outcomes. Overall, data shows that participation in its programs can result in a 10 percent increase in attendance, grade-point average and on-time graduation.
The leaders of the organization are rightfully proud. Eric Moore, director of research, evaluation and assessment at Minneapolis Public Schools (MPS), is elated. To him, the data illuminate the impact an enrichment program can have on classroom performance.
“School is constrained in terms of time,” says Moore. “But out-of-school partners can extend time and can load that time with grit and persistence.”
Assessing what works
His analysis of Project SUCCESS’ data — which also reveal places where there is no immediate correlation with academic achievement — suggests that it matters how and when outside organizations go about their work. The information should help school leaders evaluate which programs are the best fit.
A separate effort being undertaken by the organization, which launched its own database a few weeks ago, will yield information about which kids are opting in and what would enable it to increase the “dosage” for those kids.
With an overall aim of getting kids to dream about their futures, set goals to realize those dreams and break the journey down to discrete steps, Project SUCCESS operates in 16 schools in St. Paul and Minneapolis, including all seven mainline MPS high schools.
In Minneapolis, students first encounter the group in middle school language arts classes. Mentors visit once a month to facilitate goal-setting sessions. Over the next seven years, from grades 6-12, students have the opportunity to increase their exposure to the program.
Students and families are offered free tickets to plays at local theaters, as well as opportunities to stage productions, spend a week up north canoeing or tour colleges here and elsewhere. The program works hard to keep mentors and students together for as many years as possible.
Not an ‘either/or,’ but an ‘and’
In Moore’s view, the program’s outcomes numbers jibe with mounting evidence that academic achievement and soft skills aren’t an “either/or,” but an “and.” Psychology and brain science back him up.
The formula as Moore sees it: Persistence is the ability to overcome an obstacle. It requires motivation and effort, as well as powers of analysis and critical thinking.
People who persist and succeed because they did so come to see themselves as changeable — capable of learning from mistakes and mastering hard skills. So a student’s grades are a better proxy for this skillset than proficiency as measured on annual assessments.
Grade-point changes
African-American and Hispanic students who go on one of Project SUCCESS’ college tours see an average increase of 0.07 grade points, according to Moore. High schoolers who go every year, then, could see a bump of 0.28.
From grades 9-12, black and Latino participants who go on a college tour and attend a musical see an increase of 0.15 grade points. Because theater attendance doesn’t seem to have the same cumulative potential as the tours, the overall bump is a likely 0.34.
Students of color were 8 percent more likely to attend high school, which translates to approximately two additional school days a year. Participants who attend a college tour are 2 percent more likely to graduate on time.
In Moore’s view, the places where there does not appear to be a correlation between exposure to Project SUCCESS and academic outcomes make perfect sense.
For instance, using a growth model the analyses do not show any statistical difference in middle schoolers’ performance on standardized math and reading assessments or their attendance. The goal-setting process, Moore points out, is by its nature long-term.
Plus, the other area where exposure did not correlate to better academics was among students who only were exposed to Project SUCCESS’ mentors in language arts classes.
Here Moore sees what he anticipates is the effect of something called stereotype threat— the phenomena wherein people live out negative expectations of their group. The brains of people experiencing the accompanying stress secrete a hormone that blocks access to memory.
The in-class work with the mentors may be what gets students to opt-in to Project SUCCESS’ other programs, while the fact that those experiences take place outside the regular school day frees them to engage, he says.
Seeking ways to reach more students
More than 12,000 students a month participate in the organization’s in-class activities, and an even higher number of individuals, including parents, attend plays. Participation is much lower for college tours and BWCA trips.
So the question for the program is how to encourage more students to opt-in to the activities that will increase their dosage. In addition to asking Moore to analyze its data along with MPS’ numbers, Project SUCCESS has built its own database.
The effort should help identify which groups need more outreach and what kind. Some families may not understand why a college tour is important, for instance, or why they might allow their child to spend a week in the north woods.
They also hope to learn what propels students to seek more interactions and whether those students also increase their participation with other MPS partner organizations.
And if the district must turn to outside organizations for help lengthening students’ days, something that appears crucial to closing racial and socioeconomic achievement gaps, Moore is anxious to mine the data for indicators that will help select programs with capacity like Project SUCCESS.
It’s just one piece, Moore acknowledges, but it shows “qualitative analysis can support the understanding of human experience,” as he puts it.
“We have to give students opportunities for success,” he says. “When you talk to students [about Project SUCCESS] they talk about field trips and about engaging with others about the play or about the project.”
Project SUCCESS’ 2014-2015 college tours begin next week and continue throughout the school year. Most are offered at no cost to students.
