School-level assignment to the Open Court Reading (Imagine It!) curriculum does not produce a statistically significant impact on elementary students' reading achievement relative to business-as-usual control curricula in either study year¶

Claim · All claims
Evidence · 1 study · causal r2 · q3 · i0 negligible
Currency · newest study 2016 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i0 In a multi-site cluster randomized trial, ITT analysis found no statistically significant impact of OCR on students' reading achievement compared to the business-as-usual curriculum across either year, with printed effect sizes of 0.038 (Year 1) and -0.06 (Year 2). → Vaden-Kiernan 2016

Evidence¶

Vaden-Kiernan 2016¶

Vaden-Kiernan, M., Borman, G., Caverly, S., Bell, N., Ruiz de Castilla, V., Sullivan, K., & Fleming, G. (2016). Findings from a Multi-Year Scale-Up Effectiveness Trial of Open-Court Reading (Imagine It!). SREE Spring 2016 Conference. https://eric.ed.gov/?id=ED567512

q3 · i0 · causal · r2

Intent-to-treat analysis of the multi-site cluster randomized trial (49 schools, 7 districts; Year 1 analytical sample 4,392 students with valid Spring GRADE scores), using a three-level model with school-mean GRADE pretest as covariate. Table 2 prints effect sizes of 0.038 (Year 1) and -0.06 (Year 2), both negligible; equivalence was not tested.

"ITT analysis indicates that the OCR program does not have a statistically significant impact on students' reading achievement compared to the business as usual curriculum in place in control schools across either year."

Discussion¶

Related Claims¶

• The OCR program shows a positive differential impact for Kindergarten students relative to other grades — related
• Assigning schools to use RM-CC5 as their core Grade 5 mathematics curriculum for a full year did not significantly change state assessment achievement compared with business-as-usual schools — a broader claim this one bears on
• The SPARK program produced a small but statistically significant positive impact on reading achievement of 0.12 standard deviations in a two-year RCT — related
• The multi-component intervention produced statistically significant advantages over business-as-usual on one decoding measure (ES = 0.58) and one fluency measure (ES = 0.46), but not on reading comprehension — related
• School-mean pretest scores predict posttest reading achievement in the OCR trial — related