Editors’ Note: This is the first blog post in our series featuring reflections on and learnings from the 2016 Carnegie Foundation Summit on Improvement in Education. Each post provides recaps of sessions you may have missed and further insight into presentations you may want to revisit.
When we reflect on improvement work some time after the fact, hindsight can make it seem like a straightforward, orderly, and highly logical process: we identify a specific problem, we study its causes, we make and test some type of change, and we arrive at a set of measured improvements.
But the tidiness of a seemingly-complete driver diagram, like the one for the Carnegie Math Pathways reproduced below, belies its dynamic character and all the hard and complex work that goes on beneath the surface to create it. In his keynote address at the Summit on Improvement in Education, Carnegie Foundation President Anthony S. Bryk pulled back the curtain to give the audience a glimpse of what it looks like to struggle towards a common understanding of a problem.
In keeping with this year’s theme of “Living Improvement,” Bryk’s talk included an in-depth exploration of how the Foundation and its partners engaged in the difficult process of defining a working theory of improvement during the early days of the Carnegie Math Pathways work. The team that initiated this networked improvement community (NIC) started out with a compelling problem (the unconscionably high failure rates of developmental math students in community colleges) and a clear aim (to increase the percentage of community college students achieving college math credit). Having a readily discernable and unambiguous ambition like this is an advantage that not every NIC has; new NICs can sometimes struggle even to agree on what the specific problem is that they want to address.
They learned that students can often feel that they do not belong in the learning environment, they think of themselves as someone who is not a "math person," and they question the relevance or value of what they are being asked to learn.
Even so, the Carnegie Math Pathways group had to map out the root causes of the developmental math problem and decide on the levers, or drivers, that would contribute to accomplishing the aim—not an easy task, particularly with a problem so large and complex in scope. During this early phase of the work, a diverse initiation team (comprised of academic thought partners and community college leaders and faculty) identified a key concern among college faculty and administrative leaders about students’ motivation and engagement with college. The team began by looking at what colleges were already doing. They encountered “solutionitis” at work. “This is a very natural human tendency,” said Bryk, “when confronting a perplexing educational problem. We feel a need to do something to help more of these students succeed, so we jump on solutions.” But as the team found out during their research of existing student success programs, this approach can lead to a dizzying array of programs and services that guess at possible fixes—better career planning support, financial incentives, improved orientation materials, and more—but that are not based on a clear understanding of what, precisely, is holding students back.
So the network initiation team went to the source: the community college students themselves. They learned that students can often feel that they do not belong in the learning environment, they think of themselves as someone who is not a “math person,” and they question the relevance or value of what they are being asked to learn. The initiation team incorporated these ideas into the Carnegie Math Pathways driver diagram as a primary driver called “Productive Persistence,” making it a priority in working towards the aim; it was especially salient for the group not only because of its relevance to students, but also because it was a component of the problem that faculty could have a direct impact on.
These student accounts aligned with extant qualitative case studies showing that students who eventually failed to complete developmental math courses usually showed signs of disengagement in the first few weeks of the course. This led to the inclusion of “Starting Strong” as a secondary driver, which faculty members played a key role in refining. Much of the faculty members’ reflections centered on what they themselves did early on in their courses, how they met students and established relationships with them, the routines they put in place in their opening classes to create an engaging community of learners. This professional insight—what Bryk called “the wisdom of practice”—brought a whole set of new change ideas clearly into view.
Bryk went on to detail the exploration of one specific change idea, the introduction of a direct intervention to shift students’ thinking about learning math from a fixed to a growth mindset. This involved a series of Plan, Do, Study, Act (PDSA) cycles (including two short randomized control trials) that, in less than six months, moved this from a novel idea tried in one classroom with a few students to a network-wide improvement. Bryk’s talk highlighted the complex series of events, learnings, and “aha moments” that led to the development of this single change idea—nested within the Carnegie Math Pathway’s more comprehensive theory of improvement—to illustrate the realities of engaging in improvement science in NICs.
Bryk’s keynote address comes at a time when the improvement community is growing in size and maturity. This year’s Summit on Improvement in Education brought together more than one thousand attendees from across the country and around the world. Representatives from nonprofit organizations, K-12 schools and districts, universities, philanthropic organizations, government entities, and corporations attended sessions ranging from “An Introduction to Improvement Science” to “Leveraging Technology to Accelerate Improvement in NICs.” More and more people and organizations are moving from learning about the basics of improvement science to beginning to apply it to solve problems in their local contexts, as evidenced by the over 100 session speakers who shared their ongoing improvement efforts.
As more teams engage in this work, many will inevitably face challenges similar to those the Foundation has encountered and worked through. Many will struggle, as the Foundation did, with the difficult and time-consuming work of coming to a common understanding of both the problems they are trying to solve and the manner in which complex systems contribute to those problems.
Engaging in this work "means being able to confront uncertainties and doubts, trusting that if we persist, we will learn, insights will emerge, and the path toward genuine improvement will start to clear."
Bryk emphasized that improvement leaders must be able to maintain a “paradoxical mindset.” One has to deliberately seek out diverse points of view and live with the uncertainty this may create. It means accepting and being able to say, “We don’t know yet,” but not being paralyzed by this. It means believing in what we are trying to accomplish and how we are going about it and yet also maintaining a critical perspective about the correctness of each next step we may take. This means living the improvement mantra “definitely incomplete and possibly wrong.”
This mindset “is the antithesis of strategic planning as typically executed,” said Bryk. “Under a strategic planning mindset, you presume to know exactly where you are going and that you can rationally backmap all the steps necessary to get there.” A “finished” driver diagram can seem to be a product of this kind of thinking. But, explained Bryk, “in complex improvement settings, both the precise goal and the productive steps toward it only fully emerge in the act of learning by doing.” The work of the Carnegie Math Pathways network initiation team shows this quite clearly. Engaging in this work “means being able to confront uncertainties and doubts, trusting that if we persist, we will learn, insights will emerge, and the path toward genuine improvement will start to clear.”