Nunnery and Ross (2007) reported significant grade 5 but not grade 8 TAAS effects, which were non-significant in WWC recalculation¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q2–q3
Currency · newest study 2010 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The quasi-experiment's authors reported positive, statistically significant effects for grade 5 students but not for grade 8 students on the TAAS reading subtest. → Accelerated Reader™ 2010 q3 i? In WWC recalculations using untransformed scores accounting for clustering, none of the Nunnery and Ross effects were statistically significant and the average effect size fell below 0.25. → Accelerated Reader™ 2010 (2)

Evidence¶

Accelerated Reader™ 2010¶

Accelerated Reader™. (2010). WWC Intervention Report: Accelerated Reader™, Adolescent Literacy. What Works Clearinghouse, U.S. Department of Education. http://www.renlearn.com/ar/

q2 · i? · causal · r2

Quasi-experiment in Texas comparing Accelerated Reader schools with matched comparison schools on performance, ethnicity, English proficiency, poverty, and mobility; two grade 5 cohorts (912 and 891 students) plus grade 8 students (482 intervention, 510 comparison).

"Nunnery and Ross (2007) reported positive and statistically significant effects of the intervention for grade 5 students and did not find statistically significant effects of the intervention for grade 8 students on the reading subtest of the Texas Assessment of Academic Skills (TAAS) test."

Accelerated Reader™ 2010 (2)¶

Accelerated Reader™. (2010). WWC Intervention Report: Accelerated Reader™, Adolescent Literacy. What Works Clearinghouse, U.S. Department of Education. http://www.renlearn.com/ar/

q3 · i? · review · r3

WWC recalculation of the same quasi-experiment's findings. The authors had not accounted for clustering within classrooms and used transformed test scores; WWC corrections changed the significance conclusions.

"In WWC calculations, based on untransformed scores that account for clustering, none of these effects were statistically significant, and the calculated average effect size was not large enough to be considered substantively important according to WWC criteria (that is, an effect size of at least 0.25)."

Discussion¶

Related Claims¶

• Author-reported statistically significant program effects did not remain significant after WWC corrections for clustering and multiple comparisons — a broader claim this one bears on
• Greenwood et al. (1993) reported a statistically significant CWPT effect on CTBS-Reading, but the WWC-corrected analysis was not statistically significant — related
• Everyday Mathematics® showed positive effects on TAAS math subtests (Concepts, Operations, Problem Solving) with effect sizes of 0.25 to 0.31 that were not statistically significant after clustering correction — related
• Everyday Mathematics® has potentially positive effects on mathematics achievement for primary students, with a substantively important but statistically non-significant effect size of 0.27 — related
• Statistically significant effects reported by the CDP studies were not statistically significant as recalculated by the WWC — related
• Chambers et al. report a statistically significant expressive language effect for 3-year-olds but not 4-year-olds, which WWC recalculation does not confirm — related
• In a five-year quasi-experimental study, the intervention program showed a statistically significant positive effect on fifth-grade reading performance assessment but not on the CTBS Reading subtest — related