Editor’s Note: Author Jeff Kosovich, at the University of Virginia, works with mentor Chris Hulleman, Carnegie Fellow and a research associate professor in the department of educational leadership, policy, and foundations at the University of Virginia, to create practical and useful interventions to increase students’ productive persistence. As part of the Carnegie Alpha Lab Research Network, this program of research involves rapid prototyping and testing of innovations to examine how changing students’ motivation beliefs towards their classes (e.g., perceptions of confidence and value) can improve academic outcomes. 

Recent research findings show that social psychological interventions hold remarkable power to improve student success. To most people, the idea that a short, in-class writing activity can change a student’s academic trajectory is difficult to believe. Yet, findings that underscore the strength of psychological interventions serve to highlight the importance of the work of Carnegie’s Alpha Lab Network.

We began this work with the knowledge that mathematics is an emotionally-charged subject for many students. For example, there is a vast body of research that specifically focuses on math anxiety. Typically, students have encountered 12 or 13 years of instruction by the time they reach college. Those students who have had negative experiences with math often express two related attitudes: “Math is useless in my life,” and “I can’t do math.” Unfortunately, students who report negative attitudes towards math also display less persistence and poorer performance in school.

The Carnegie Alpha Lab work explores how to help students see the value and usefulness of math to their lives. Previous studies by Chris Hulleman and his colleagues found that asking students to write an essay reflecting on the usefulness of math to their future life goals can increase grades and motivation in psychology as well as science. In both studies, students who participated in the utility value intervention (writing about usefulness) benefitted relative to a comparison group who just summarized what they’d learned in class.

A secondary focus of our work in the Alpha Labs is on altering students’ beliefs about their ability to learn math. Research on ability beliefs has found that people who think their abilities or skills are innate (i.e., they have a fixed mindset) also tend to avoid challenges and experiences related to those abilities or skills; they believe that if you aren’t good at math, nothing you do will change that fact. Counter to this belief, people who believe skills and abilities can be changed with effort and effective learning strategies (i.e., they have a growth mindset), tend to seek challenge and persist in the face of adversity. They also tend to demonstrate better performance.

This past spring, we used random assignment to test a mindset intervention (i.e., learning that strategic effort and practice can improve skills and abilities)  followed several weeks later by a utility value intervention (i.e., writing about usefulness) that was delivered online. We have now conducted two rounds of iterative testing in all. In the intervention, averaged across all sections and groups of students, there were no positive or negative effects. As a result, we decided to explore the data further.

Based on preliminary analyses, the growth mindset intervention resulted in lower course dropout rates (but not pass rates) in the intervention group to (6.8 percent) compared to the control group (11.6 percent) among students taking intermediate algebra. We also checked if the intervention had selective effects on students of different races or ethnicities. In these exploratory analyses, we found that the mindset intervention resulted in higher pass rates (73 percent) than the control group (57 percent) among Hispanic/Latino students, but not among other groups. This is a notable finding because 31 percent of students at the college identify as Hispanic or Latino (only 35% of students identify as Caucasian).

In the first pilot study, there were no effects of the utility value intervention. Although these findings present limited benefits, it’s important to note that more than half of the students either did not start, or did not finish the utility value activity and that only one in ten students participated in all of the study’s activities.

During Spring 2014 we discovered that students were becoming frustrated with the number of questionnaires and that some teachers were uninformed about intervention requirements. Our second pilot study was revised based upon this information. As a result, we selected fewer classrooms and focused only on the utility value intervention. Students were randomly assigned to either a control activity, which involved summarizing what they were learning in class, or a utility value activity, which involved writing about the usefulness of their class material to their everyday life, future career, and hobbies and interests.

Compared to students in the control condition, students who completed the utility value activity reported higher perceptions of utility value and interest. Based on these results, as well as those found in spring, it appears that the utility value and mindset interventions can benefit these new college students in some cases.

The power of an improvement research design is evidenced by our story. Unlike traditional studies that may not uncover implementation problems for years, we did so in a single semester. Going forward, it’s important to determine how to integrate the intervention into practice so that we can ensure better student engagement. In research situated in real-world settings (as opposed to the laboratory), it is critical that we collaborate with faculty at the college to ensure that the intervention is implemented as planned. In the cases where we do not find positive results, we are conducting interviews with students and consulting with our partners about potential obstacles for successful intervention.

An additional benefit of this Alpha Lab work has been the researcher-practitioner collaboration. Working with practitioners helps us get a better sense of the support needed to introduce these interventions at scale, and also help us move out of the world of theory to real world situations. However, it is the opportunity to influence student outcomes that may be the more rewarding effects from our work. These interventions provide us with the opportunity to bring potentially life-changing psychological interventions for students into this college setting. By decreasing course drop rates, increasing math interest, and increasing pass rates, we are providing developmental math students with an opportunity to overcome past academic barriers and move forward in their academic and life pursuits.