Evaluate ed-tech program effects at the grade-level cluster level with state-z-scored outcomes across states¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 1 study (1 causal), q3 · 1 of 1 report an effect size · 1 claim rests on one study

Description¶

The evaluation used the grade-level cluster (all classes teaching a given grade in a school) as the unit of analysis rather than classrooms or whole schools, and converted state-specific assessment outcomes to z-scores because the states used differing standardized tests. Matching was conducted within the same state, and combined-grade effects were estimated with hierarchical linear modeling to account for nesting of grades within schools, with the Benjamini-Hochberg correction applied to control false discovery across multiple outcome comparisons.

Design Implications¶

Context¶

Requirements¶

• Baseline-year and follow-up data must be available for both treatment and comparison clusters, including demographic variables from the baseline school year used in matching

Constraints¶

• Without randomization, the possibility that the groups differed on other characteristics besides exposure to ST Math impedes causal conclusion

Target Learners¶

• elementary students in grades 3-5

Target Learning Goals¶

• elementary mathematics achievement on state standardized assessments

Claims¶

• Grade-level clusters provided with ST Math scored higher on state math proficiency rates and scale scores than matched comparison clusters across grades 3-5 [+W]

Related Strategies¶

Examples¶

-

Key Sources¶

• Wendt, S., Rice, J., & Nakamoto, J. (2019). A cross-state evaluation of MIND Research Institute's ST Math program and math performance. San Francisco, CA: WestEd. https://eric.ed.gov/?id=ED601907