Grade-level clusters provided with ST Math scored higher on state math proficiency rates and scale scores than matched comparison clusters across grades 3-5¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3 · i0 negligible
Currency · newest study 2019 · checked against its sources 2026-10-20 (automated, 2 of 2 entries) · not reviewed by a person

Subclaims¶

q3 i0 Across combined grade levels 3-5, clusters provided with ST Math had higher proficiency rates (ES = 0.17) and scale scores (ES = 0.13) than matched comparison clusters, significant after BH correction; both effects are below 0.2 SD. → Wendt 2019 q3 i0 Per-grade analyses showed significant proficiency advantages at grades 3, 4, and 5 (ES = 0.13, 0.23, 0.15), all surviving BH correction; the report's text says scale-score advantages were significant at grades 4 and 5 only, with grade 4 surviving the correction, though its table also marks the grade 3 scale score (ES = 0.12) as significant after correction. → Wendt 2019 (2)

Evidence¶

Wendt 2019¶

Wendt, S., Rice, J., & Nakamoto, J. (2019). A cross-state evaluation of MIND Research Institute's ST Math program and math performance. San Francisco, CA: WestEd. https://eric.ed.gov/?id=ED601907

q3 · i0 · causal · r2

Hierarchical linear modeling of 1,132 treatment and 1,132 matched comparison grade-level clusters across 16 states, adjusting for baseline achievement, demographics, and nesting of grades within schools. The analysis found "0.17 of a standard deviation for proficiency rates and 0.13 for scale scores," both reported as p = .01 (as rounded in the table) and significant after BH correction. Both effect sizes are below 0.2 SD.

"The magnitude (or effect size) of the difference was 0.17 of a standard deviation for proficiency rates and 0.13 for scale scores."

Wendt 2019 (2)¶

Wendt, S., Rice, J., & Nakamoto, J. (2019). A cross-state evaluation of MIND Research Institute's ST Math program and math performance. San Francisco, CA: WestEd. https://eric.ed.gov/?id=ED601907

q3 · i0 · causal · r2

Separate multiple regressions by grade level (392, 366 and 374 grade-level clusters in each group, matched one-to-many and weighted to equal group sizes) adjusting for baseline achievement and school demographics. The report states grade 3 proficiency rates were "0.13 of a standard deviation higher" than comparison clusters; Exhibit 3 prints effect sizes for every grade-level outcome: proficiency 0.13, 0.23 and 0.15 and scale scores 0.12, 0.17 and 0.10 for grades 3, 4 and 5. All but grade 4 proficiency are below 0.2 SD.

"students in clusters of grade level 3 provided with the ST Math program had z-score proficiency rates in math that were, on average, 0.13 of a standard deviation higher compared to students in clusters of grade level 3 not provided with the ST Math program"

Discussion¶

Related Claims¶

• Matched treatment and comparison grade-level clusters were balanced on observable baseline characteristics, with differences below a quarter of a standard deviation — related
• Schools implementing ST Math with fidelity in at least one grade level showed larger math advantages (ES 0.31-0.35) than matched comparison clusters — related
• Among grades that fully implemented ST Math, grades 2, 3, and 5 showed significant CST advantages but grade 4 did not — related
• Among grades provided with ST Math, only grade 2 showed statistically significant CST advantages; grades 3, 4, and 5 did not — related
• Grades provided with ST Math have significantly larger proportions of students advanced, or proficient or advanced, on the mathematics CST than comparison grades — related
• Implementation fidelity was defined as at least 85 percent student enrollment and at least 50 percent grade-level content coverage in ST Math — related