SMART® effects on comprehension were not statistically significant on either subtest, though the domain-average effect size was substantively important¶

Claim · All claims
Evidence · 1 study · causal r? · q3
Currency · newest study 2007 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Neither the WRMT-R word comprehension subtest (ES 0.41) nor the passage comprehension subtest (ES 0.28) showed a statistically significant SMART® effect, but the domain-average effect size of 0.35 met the WWC's substantively important threshold of at least 0.25. → What Works Clearinghouse 2007

Evidence¶

What Works Clearinghouse 2007¶

What Works Clearinghouse, U.S. Department of Education. (2007). What Works Clearinghouse Intervention Report: Start Making a Reader Today® (SMART®). https://www.getsmartoregon.org

q3 · i? · causal · r?

In the randomized controlled trial's comprehension domain, the word comprehension subtest showed ES 0.41 and passage comprehension ES 0.28, both marked ns; the domain average of 0.35 was judged substantively important, with an average improvement index of +14 percentile points (Appendix A3.3).

"The WWC did not find that either of these effects was statistically significant. The average effect size across the two outcomes, however, was large enough to be considered substantively important according to WWC criteria (that is, an effect size of at least 0.25)."

Discussion¶

Related Claims¶

• Corrective Reading showed no statistically significant effects on reading comprehension outcomes — related
• Failure Free Reading shows potentially positive effects on reading comprehension — related
• In the Torgesen et al. (2006) RCT, SpellRead™ showed indeterminate effects for fifth graders, with no statistically significant or substantively important effects in any domain after six months — related
• Waterford Early Reading Program™ showed no discernible effects on comprehension, with a small non-significant effect size on vocabulary — related
• After one year of SMART® intervention, only the fluency outcome was statistically significant, while alphabetics and comprehension effects were not — related
• SMART® tutoring produced statistically significant positive effects on oral reading fluency on both first- and second-grade passages — related
• SMART® tutoring produced a statistically significant positive effect on first-graders' word identification after two years of intervention — related
• Read Naturally® shows potentially positive effects on writing for students with learning disabilities, with a substantively important but non-significant effect size on the WIAT Written Expression subtest — related
• The effect on SAT math scores was positive but not statistically significant, though large enough to be substantively important (effect size at least 0.25) — related
• Early Steps one-to-one tutoring substantially raises first graders' Word Attack and Passage Comprehension scores (average effect size +0.86) — related
• Interventions targeting different reading and skill content domains show mostly small differences, with average effect sizes of 0.14–0.22 all statistically significant — related
• The guide treats an effect size of +0.25 or higher as substantively important, equivalent to raising performance at least 10 percentile points — a broader claim this one bears on
• AI-ALOE technology deployments were associated with improved adult learner outcomes, including higher course grades and retention for Jill Watson and higher-quality written work for SMART, though VERA outcomes varied by course structure — related