I CAN Learn® has no discernible effects on the mathematics test scores of eighth-grade students in primary mathematics courses¶

Claim · All claims
Evidence · 2 studies · 1 causal r2, 1 quant-synthesis r1 · q3
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Across the one qualifying study, the average effect of I CAN Learn® on eighth-grade mathematics achievement was neither statistically significant nor substantively important (domain average effect size 0.05, improvement index +2). → kerstyn-domain-average q3 i? In the raw reported results, one contrast (2002 FCAT, MJ-3 classrooms) was positive with p = .04, but after correction for multiple comparisons (critical p = .013) the WWC found it not statistically significant. → kerstyn-2002-fcat-mj3

Evidence¶

kerstyn-domain-average¶

What Works Clearinghouse, Institute of Education Sciences, U.S. Department of Education. (2017, July). Primary Mathematics intervention report: I CAN Learn®. Retrieved from https://ies.ed.gov/ncee/wwc/InterventionReport/676

q3 · i? · quant-synthesis · r1

Systematic review finding based on one quasi-experimental study meeting WWC standards with reservations, covering 524 classrooms and 9,886 students across 36 schools. The domain average effect size was 0.05 and the improvement index +2; no comparison reached the WWC's substantively important threshold, "an effect size of at least 0.25".

"Kerstyn (2002) reported, and the WWC confirmed, no statistically significant difference between I CAN Learn® and the comparison group on the mathematics achievement domain for three analytic samples"

kerstyn-2002-fcat-mj3¶

What Works Clearinghouse, Institute of Education Sciences, U.S. Department of Education. (2017, July). Primary Mathematics intervention report: I CAN Learn®. Retrieved from https://ies.ed.gov/ncee/wwc/InterventionReport/676

q3 · i? · causal · r2

WWC re-analysis of the Kerstyn (2002) quasi-experiment. The 2002 FCAT Math MJ-3 contrast showed a mean difference of 3.91 and effect size 0.10 (improvement index +4) with p = .04, but a WWC-computed critical p-value of .013 left it not statistically significant; the study finding is characterized as an indeterminate effect.

"the author reported a positive and statistically significant difference between the I CAN Learn® group and the comparison group in the mathematics achievement domain. However, after adjusting this finding for multiple comparisons"

Discussion¶

Related Claims¶

• Brandt et al. (2013) finds a statistically significant positive effect of eMINTS on seventh- and eighth-grade mathematics achievement — related
• Martin et al. (2009) finds indeterminate effects of eMINTS on both mathematics and literacy achievement in grades 4–5 — related
• eMINTS shows no discernible effects on general literacy achievement — related
• Supplemental analyses of I CAN Learn® in alternate second-year samples showed no effects that survived multiple-comparison adjustment — related