Baseline equivalence held between conditions, but both groups scored far below the grade 8 proficiency cut score¶

Claim · All claims
Evidence · 1 study · causal r3 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person · recheck by 2028: baselines and conditions change

Subclaims¶

q2 i? Intervention and comparison groups performed equivalently at baseline (Hedges' g = 0.08, not significant), and after the intervention both remained about 8 points below the Level 3 proficiency cut score. → Feng 2023

Evidence¶

Feng 2023¶

Feng, M., Huang, C.-W., & Collins, K. (2023). Technology-based support shows promising long-term impact on math learning: Initial results from a randomized controlled trial in middle schools. WestEd. https://osf.io/exqpn

q2 · i? · causal · r3

Baseline equivalency test of grade 6 EOG math scores between randomized groups. Authors also note the estimated intervention mean (535) is 8 points short of the Level 3 "Proficient" cut score of 543, highlighting urgency of pandemic learning-loss recovery.

"The difference is small and not statistically significant at the 0.05 level (545.70 for intervention versus 545.03 for comparison, Hedges' g = 0.08), indicating that both groups performed equally at the end of 6th grade prior to participating in the study."

Discussion¶

Related Claims¶

• Propensity score matching produced comparison groups equivalent at baseline (within +/- 0.25 standard deviations) on all pretest achievement measures across the three QEDs — related
• A virtual comparison group matched on pretest, locale, FRL, and test dates achieved baseline equivalence with the treatment group (effect size = -0.03) — related
• In the full-sample analysis, baseline demographic and prior-achievement differences between treatment and matched comparison groups were minimal, with effect sizes below the WWC threshold — related
• Propensity score matching produced baseline-equivalent treatment and comparison groups on all measured variables — related
• Baseline pre-assessment scores showed no statistical differences between treatment and control groups in either the matched or baseline sample. — related
• Matched treatment and comparison grade-level clusters were balanced on observable baseline characteristics, with differences below a quarter of a standard deviation — related
• Across the full sample, students receiving the ASSISTments intervention performed comparably to their virtual comparison peers on end-of-year MAP Growth — related
• ASSISTments use in seventh grade produces a statistically significant long-term math achievement effect of 0.10 SD one year after implementation ends — related
• ASSISTments use in grade 7 math homework produces a statistically significant long-term positive effect on grade 8 state test scores one year after the intervention ends — related
• Random assignment produced treatment and control groups balanced in mathematical preparedness before the intervention — related