A concrete way to learn how a Networked Improvement Community (NIC) might organize and carry out a better program of educational R&D is to build one. In this spirit, the Carnegie Foundation in partnership with several other colleagues and institutions, is now initiating a prototype NIC aimed at addressing the extraordinary failure rates in developmental mathematics in community colleges.

The aim of this NIC is to double the proportion of students who in a one-year course sequence achieve college credit and are mathematically prepared to succeed in subsequent academic pursuits. Our first effort in this regard is to launch the Carnegie Statway Network. This network is redesigning traditional developmental mathematics by creating a one-year pathway to and through statistics that integrates necessary mathematics learning along the way.

Carnegie President Anthony Bryk and colleagues delved into this work during a recent presentation at the annual meeting of the American Educational Research Association. He emphasized the Foundation’s commitment to an approach to educational research and development that joins practitioners, researchers and developers in purposeful collective action to address a problem of practice, in this case developmental math. Bryk said this network organizational approach can surface and test new insights and enable more fluid exchanges across contexts and traditional institutional boundaries—thus holding potential to enhance designing for scale.

“We are committed to principles of openness and transparency,” he said. “Openness of all of the resources we are building and drawing on. Transparency in sharing what we are doing, why we are doing it and what we are learning along the way—both successes and failures.”

Carnegie is drawing on Englebart’s 1992 work on high performing  learning organizations, where networked improvement communities organize and apply diverse expertise to solve complex problems. Englebart’s Multilevel Model for Learning for Improvement characterizes the work of organizations in terms of three broad domains of activity. For Carnegie, A-level work is the front-line teaching and learning work of classrooms. B-level activity describes within-organization efforts that are designed to improve the on-the-ground work (like the work of institutional research units in community college) and C-level activity is inter-institutional engagement in concurrent development. This model affords mechanisms for testing the validity of local knowledge and adjusting local understanding of the true nature of a problem.

Specifically, the Carnegie network involves the community college faculty in participating institutions who teach and implement Statway and other math pathways along with Carnegie’s improvement specialists and researchers. Together, they test changes with hypothesized benefits, warrant those changes with empirical evidence, provide for local adaptations, and over time contribute to the modification of the pathway. The NIC also includes deans, institutional researchers and others who address the institutional requirements; thinking partners who are those individuals with technical and substantive expertise; Carnegie staff provide ongoing technical, analytic and organizational support as a hub for the network.

NICs engage in disciplined inquiry. These inquiries are organized around the four core questions of improvement science—Carnegie’s approach to R&D: What are we trying to accomplish? How do we understand the problem and system in which they are embedded? What change might we introduce? How will we know that the changes are improvements?

Measurement is vital. Anchoring the NIC around a common core of interventions, participants conduct multiple small tests of change, also known as rapid prototyping. As a professional community, we study the impact of those changes, learn from them and adjust as needed.  We are paying close attention to variability in performance and the multiple factors that may contribute to it. For example, we expect that Statway effects will vary depending on specific characteristics of students, faculty and the contexts in which they both work. Given that, instead of asking whether an intervention works (e.g., “Is A better than B?” “Is C better than nothing?”), in the NIC, we ask, “what works, when, for whom and under what conditions.” It is not good enough to know that Statway can be made to work in a few places—the point of an improvement oriented approach to education R & D is to achieve effective implementation across local contexts, reliably and at scale.

The design of this work is practical and nimble and adapted from practices pioneered by the Institute for Healthcare Improvement. The basic idea is straightforward: establish baseline results, intervene, measure outcomes, keep doing it. It’s called a PDSA cycle — Plan, Do, Study, Act. For example, in the study phase, measurement is conducted in the web of daily activity: a 60 second student survey (simply asking how they’re doing); a three-minute teacher report (asking how the lesson went and what they might change and why); or cull informal queries and comments. The idea is to test fast, fail fast and early, learn and improve.

With 30 colleges involved and an as yet unknown number of students, we are looking at collecting data from multiple data streams and will feed this continuous stream of information to quick problem solving teams with one week revision and testing targets. With a year of lessons and different start dates among our college network, we aim to exploit and study the natural variation in the outcomes of implementations and feed that knowledge back into the design to increase its effectiveness in real time. The bottom line is to learn how to put usable knowledge to work as part of the design / development process to support increasing student success.

The big question, of course, is can we initiate and sustain a networked community that accelerates improvement? The whole enterprise is itself a learning through doing experiment. We are indeed doing improvement research on ourselves. Stay tuned.

Carnegie Ideas Gaining Traction
The notice from the U.S. Department of Education’s Institute of Education Sciences for five-year funding contracts for each of ten Regional Education Laboratories contains language familiar to Carnegie.  The solicitation reads: “The purpose is to enter into contracts with entities to establish a networked system … .”  Further, after expanding on the expectations of the labs’ mission to build the research capacity and knowledge bases in their states and districts, the call is that they “carry out these priorities primarily by organizing … networks of practitioners, policy makers and others in ‘research alliances.’”

The promotion of networks and alliances echoes our call for a networked improvement community framework where research and practice communities join to accomplish improvement at scale. Carnegie has made a concerted effort to provide leadership in the R&D field and the IES language is an indication that our ideas are gaining traction. We have been operating as a “thought partner” for the Knowledge Alliance and its president Jim Kohlmoos to encourage its members, who are mostly these federal education laboratories, to work together using a networked approach since 2008 when Kohlmoos interviewed Tony Bryk, just named as Carnegie’s president, for a video presentation to be shown at the organization’s summer retreat.

“It seems clear that the basic framework for the IES solicitation is informed by Carnegie’s vision,” Kohlmoos said, “just as Tony Bryk and Louis Gomez’s seminal work on reinventing R&D has been the catalyst for much of our collective visioning and thinking over the past several years.”