In The New York Times article, “Scholars Test Web Alternative to Peer Review,” Patricia Cohen writes that some humanities scholars are arguing “that in an era of digital media there is a better way to assess the quality of work. Instead of relying on a few experts selected by leading publications, they advocate using the Internet to expose scholarly thinking to the swift collective judgment of a much broader interested audience.”

Cohen quotes Katherine Rowe, a Renaissance specialist and media historian at Bryn Mawr College, as saying that “the way scholarly exchange is moving is radical…”

Here at Carnegie, we’ve thought about this radical transformation quite a bit. We’re reading the 2004 book, The Wisdom of Crowds, by James Surowiecki. Crowds and now the Times article reinforce our thinking that drawing information from invested people in what we’re calling a “networked improvement community” will yield better decisions, products, outcomes than individuals working alone.

Carnegie President Tony Bryk has said that educational research and development is not leading to solutions for major educational problems and has called for a reengineering of how we carry out educational R&D and the institutional environs in which this work occurs. Thus, in our new work, we’re enlisting designers, developers and researchers to work in close collaboration with educational practitioners and students from the beginning, with their efforts emerging from and connected to the day-to-day work of teaching and learning and the institutions in which these activities occur. As Surowiecki says (speaking of a car manufacturer) in Crowds, “the further you got from the front line, the more watered down the solution became.” We want to be on the front lines.

And of course, we’re interested in spreading what is learned. Surowiecki writes that “invention may still be an individual enterprise, but selecting among inventions is a collective one.” And he adds, “used well, imitation is a powerful tool for spreading good ideas fast.”

Carnegie is building a space both for the inventions and the imitations. We know that information must be shared widely and openly. In Carnegie’s current work addressing the extraordinarily high failure rate of community college students in developmental mathematics, we are creating a Web 2.0 environment as a platform for encouraging contributions from the field. In the spirit of harvesting learning from the wisdom of crowds, Carnegie is identifying targeted problems and inviting contributions that might clarify concerns and point out promising new developments. We are going to analyze how others in the Web 2.0 environment analyze and evaluate these ideas, synthesize this with our own judgments, vet this with key experts and summarize these conversations as feedback to the field. We’re hoping for active participation across a diverse community of members who judge the network as an important resource advancing their improvement efforts.

We believe that this approach will not only produce powerful solutions to educational challenges, but will help to create a new infrastructure for research and development more broadly. Too often, R&D is undertaken as a purely academic endeavor, driven by individual researchers’ own professional interests and agendas, and unconnected to realities and felt needs in the world of practice. In contrast, Carnegie is calling for a reconceived approach: situated on “the job floor” and organized around the high-leverage problems that real people care about and are invested in solving.

And we share the belief of one of the professors in the Times article who said, “There is an ethical imperative to share information.”