The guide treats an effect size of +0.25 or higher as substantively important, equivalent to raising performance at least 10 percentile points¶

Claim · All claims
Evidence · 1 study · theoretical r1 · q1
Currency · newest study 2007 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? In accordance with What Works Clearinghouse criteria, the panel describes an effect size of +0.25 or higher as substantively important. → Gersten 2007

Evidence¶

Gersten 2007¶

Gersten, R., Baker, S.K., Shanahan, T., Linan-Thompson, S., Collins, P., & Scarcella, R. (2007). Effective Literacy and English Language Instruction for English Learners in the Elementary Grades: A Practice Guide (NCEE 2007-4011). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncee/wwc/PracticeGuide/6

q1 · i? · theoretical · r1

A methodological convention stated in the guide's introduction to govern how impact magnitudes are interpreted throughout the recommendations. The panel defines the threshold and its percentile interpretation; no study data underlie this statement, it is an adopted reporting standard.

"In accordance with the What Works Clearinghouse criteria we describe an effect size of +0.25 or higher as substantively important. This is equivalent to raising performance of a group of students at least 10 percentile points on a valid test."

Discussion¶

Related Claims¶

• The Lovaas Model has potentially positive effects on cognitive development for children with disabilities, with an improvement index of +27 percentile points — a narrower finding that bears on this claim
• SMART® effects on comprehension were not statistically significant on either subtest, though the domain-average effect size was substantively important — a narrower finding that bears on this claim
• EIR® had a positive, substantively important but statistically non-significant effect on first-grade reading comprehension — a narrower finding that bears on this claim
• One qualifying study found a substantively important but statistically non-significant positive effect of repeated reading on comprehension (effect size 0.28) — a narrower finding that bears on this claim
• The effect on SAT math scores was positive but not statistically significant, though large enough to be substantively important (effect size at least 0.25) — a narrower finding that bears on this claim
• Too Good for Violence shows a substantively important but not statistically significant effect on students' knowledge, attitudes, and values — a narrower finding that bears on this claim
• The Lovaas Model has no discernible effects on functional abilities for children with disabilities — related