The Impact of Re-thinking Research and Practice

By Steven Holochwost, WolfBrown

Thanks to the generosity of The Pierre and Tana Matisse Foundation, we’ve created New Victory SPARK, or “Schools with the Performing Arts Reach Kids,” an innovative and robust multi-year arts program specifically designed for schools underserved in the arts. With the esteemed research firm WolfBrown, we’re also measuring and analyzing the “intrinsic impact” of this program. The following piece is the third in a four-part story about our initial findings.

Increasingly, arts and cultural organizations are asked whether they contribute to the greater good. Answering that question is rarely simple, particularly at a time when public and private funders alike press organizations to prove that something they did (e.g., changing concert format, working with seniors or running programs in juvenile detention centers) actually caused the change that they would like to claim (a more diverse audience, fewer doctor visits or lowered rates of recidivism).

In the case of the SPARK program, we were looking to make the case that young people who participated became different from their peers: that time spent in the world of theater could cause stronger inter- and intra-personal skills. Like many evaluators, we turned to the existing research literature to find out how others have measured the growth in these hard-to-capture domains. This method of working from past research to inform new studies has many advantages: measures taken from the research literature often reflect years of conceptualization, testing and refinement. So, drawing on past research, we decided to use a measure called Reading the Mind in the Eyes which assesses children’s knowledge of other people’s emotions by asking them to look at photos of the upper portion of faces and naming the emotion they detect there. Since becoming available twenty years ago, this measure has been used in over 500 published studies, including those examining the effects of theater education.

But the measure behaved in unexpected ways. We found that children participating in The New Victory’s programming—over 90% of whom were young people of color—struggled to identify the emotions in the photos—the great majority of which portrayed adult Caucasian faces. Moreover, when young people selected an incorrect option, it often reflected a hostile emotion (e.g., anger). This was a moment when the tables turned: it was time for practice to inform research. The more diverse youth in SPARK classrooms had a message for research: to assess children’s ability to read emotion expressions validly, our photos had to represent the people whom SPARK students “read” and react to every day. By putting out a call to its diverse population of theater artists, New Victory staff helped to develop a revised measure that included people from a wide array of ages, cultures, and backgrounds.

We have just begun to collect data with this new tool. We may have still more to learn on our way to valid measures. But the experience opened all of our eyes—researchers, staff and teaching artists—to the ways in which research tools reflect our assumptions, including whose faces are “universal”. It was investing in sustained work in new neighborhoods, with young people of color who have not been the usual subjects of arts education research, that made this clear.