Schools implementing ST Math with fidelity in at least one grade level showed larger math advantages (ES 0.31-0.35) than matched comparison clusters¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3 · i2 medium
Currency · newest study 2019 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 In the fidelity subsample (239 schools, 14 states), combined-grade proficiency effects were 0.35 SD and scale-score effects 0.31 SD, significant after BH correction. → Wendt 2019 q3 i? By grade in the fidelity subsample, proficiency effects ranged from a fifth to nearly half a standard deviation and scale-score effects from 0.27 to 0.43, all significant after correction. → Wendt 2019 (2)

Evidence¶

Wendt 2019¶

Wendt, S., Rice, J., & Nakamoto, J. (2019). A cross-state evaluation of MIND Research Institute's ST Math program and math performance. San Francisco, CA: WestEd. https://eric.ed.gov/?id=ED601907

q3 · i2 · causal · r2

HLM analysis of 497 treatment and 497 comparison grade-level clusters across 14 states, adjusting for baseline achievement, school demographics, and grade nesting. The fidelity subsample showed an effect size of "0.35 of a standard deviation" for proficiency rates and 0.31 for scale scores, both p < .01 with BH correction.

"The magnitude (or effect size) of the difference for math proficiency rates was 0.35 of a standard deviation and 0.31 for math scale scores."

Wendt 2019 (2)¶

Wendt, S., Rice, J., & Nakamoto, J. (2019). A cross-state evaluation of MIND Research Institute's ST Math program and math performance. San Francisco, CA: WestEd. https://eric.ed.gov/?id=ED601907

q3 · i? · causal · r2

Grade-specific regressions in the fidelity sample (N = 168, 156, 173 matched pairs). The report describes proficiency effects as "from a fifth to nearly a half of a standard deviation higher" and states scale-score effect sizes ranged from 0.27 to 0.43, all significant after the BH correction; no single standardized effect size is printed for this pooled statement.

"students in grade-level clusters 3 through 5 in schools that implemented the ST Math program with fidelity had z-score proficiency rates in math that were, on average, from a fifth to nearly a half of a standard deviation higher than those of students in grade-level clusters not provided with the ST Math program"

Discussion¶

Related Claims¶

• Implementation fidelity was defined as at least 85 percent student enrollment and at least 50 percent grade-level content coverage in ST Math — related
• Grade-level clusters provided with ST Math scored higher on state math proficiency rates and scale scores than matched comparison clusters across grades 3-5 — related
• Among grades that fully implemented ST Math, grades 2, 3, and 5 showed significant CST advantages but grade 4 did not — related
• Among grades provided with ST Math, only grade 2 showed statistically significant CST advantages; grades 3, 4, and 5 did not — related
• Matched treatment and comparison grade-level clusters were balanced on observable baseline characteristics, with differences below a quarter of a standard deviation — related
• Grades provided with ST Math have significantly larger proportions of students advanced, or proficient or advanced, on the mathematics CST than comparison grades — related
• The ST Math evaluation's quasi-experimental design cannot rule out selection bias because treatment grades volunteered to participate — related