Supplemental analyses of I CAN Learn® in alternate second-year samples showed no effects that survived multiple-comparison adjustment¶

Claim · All claims
Evidence · 2 studies · 2 causal r2 · q2
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? In a supplemental alternate sample of students in 14 middle schools, the 2002 FCAT MJ-3 contrast had a positive effect size of 0.12 (improvement index +5) with p = .02, but a WWC critical p-value of .013 made it not statistically significant. → supplemental-14-schools q2 i? In the alternate sample of all 36 middle schools, the 2002 FCAT MJ-3 contrast favored the comparison group (effect size −0.03) and was not statistically significant. → supplemental-36-schools

Evidence¶

supplemental-14-schools¶

What Works Clearinghouse, Institute of Education Sciences, U.S. Department of Education. (2017, July). Primary Mathematics intervention report: I CAN Learn®. Retrieved from https://ies.ed.gov/ncee/wwc/InterventionReport/676

q2 · i? · causal · r2

Supplemental findings from the second-year (2001–02) analyses of Kerstyn (2002), restricted to students in the 14 pilot middle schools (113 classrooms, 1,676 students). Reported p = .02 did not survive the WWC's multiple-comparison correction; these supplemental findings do not factor into the intervention's effectiveness rating.

"A correction for multiple comparisons was needed and resulted in a WWC-computed critical p-value of .013 for the 2002 FCAT Math Scale Score for MJ-3 Students in 14 middle schools; therefore, the WWC does not find the result for this outcome to be statistically significant."

supplemental-36-schools¶

What Works Clearinghouse, Institute of Education Sciences, U.S. Department of Education. (2017, July). Primary Mathematics intervention report: I CAN Learn®. Retrieved from https://ies.ed.gov/ncee/wwc/InterventionReport/676

q2 · i? · causal · r2

Supplemental alternate-sample analysis from Kerstyn (2002) covering all 36 district middle schools (258 classrooms, 4,045 students). The mean difference of −0.90 (effect size −0.03, p = .47) slightly favored the comparison group and was not statistically significant; it is excluded from the effectiveness rating.

"2002 FCAT Math Scale Score Grade 8 (MJ-3 students in 36 middle schools) 258 classrooms/ 4,045 students 294.16 (33.46) 295.06 (33.91) –0.90 –0.03 –1 .47"

Discussion¶

Related Claims¶

• I CAN Learn® has no discernible effects on the mathematics test scores of eighth-grade students in primary mathematics courses — related
• Comparison classrooms in the qualifying study were district-matched traditional classrooms, but baseline equivalence required adjustment — related
• Sensitivity analyses restricting to students assessed in all 15 terms reproduced the full-sample between-group comparisons — related