The What Works Clearinghouse reports on “studies that provide credible and reliable evidence of the effectiveness of a given practice, program, or policy.” Its highest rating of confidence is given only to well-implemented randomized controlled trials (RCTs), which randomly assign people to control or treatment groups. The What Works Clearinghouse obviously sees RCTs as the “gold standard” for minimizing bias and assessing the effect of interventions on outcomes. But is an RCT the best way to evaluate a very complex intervention like an education curriculum?
In “Do Randomized Controlled Trials Meet the ‘Gold Standard’?”, published by the American Enterprise Institute, Alan Ginsburg and Carnegie Foundation Senior Fellow Marshall (Mike) Smith analyze 27 RCT mathematics curriculum studies contained in the What Works Clearinghouse. They find that none of these RCTs provides enough useful information for consumers to make informed decisions about purchasing and using mathematics curricula. The authors suggest a more nuanced approach to evaluating curricula that is more timely and takes into consideration the complex contexts in which curricula are implemented.