This article serves as an exemplar for other effectiveness studies that may not replicate findings from previous efficacy trials. Beginning in 2002, researchers developed, implemented, and evaluated the efficacy of an English reading intervention for first-grade English learners using multiple randomized control trials (RCTs). As aresult of this efficacy work, researchers successfully competed for an Insitution of Education Sciences Goal 4 effectiveness study using the same intervention. Unlike the efficacy studies, the findings from the effectiveness study begun in 2011 did not yield significant differences in favor of the treatment.
The authors investigated the identification and preintervention performance levels of students in efficacy and effectiveness RCT studies and provided data and compared samples from these studies as one means of explaining variation in findings. Findings indicate that preintervention performance levels are higher for students in effectiveness RCT studies.
Barr, C. D., Reutebuch, C. K., Carlson, C. D., Vaughn, S., & Francis, D. J. (2019). Explaining variation in findings from efficacy and effectiveness studies for English reading interventions for English learners. Journal of Research on Educational Effectiveness, 12(1), 116–134. doi:10.1080/19345747.2018.1529211