One year of Utah Compose exposure raised students' SAGE ELA scale scores by 9.14 points (ES = 0.12) versus no exposure¶

Claim · All claims
Evidence · 1 study · causal r2 · q3 · i1 small
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i1 The estimated average treatment effect of one cumulative year of UC exposure on SAGE ELA scale score was 9.14 points, effect size 0.12, statistically significant. → Yue Huang 2024

Evidence¶

Yue Huang 2024¶

Yue Huang, Joshua Wilson, & Henry May. (2024). Exploring the Long-Term Effects of the Statewide Implementation of an Automated Writing Evaluation System on Students' State Test ELA Performance. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-024-00443-9

q3 · i1 · causal · r2

IPTW quasi-experimental analysis of statewide Utah Compose implementation among 134,425 Utah students in Grades 4-10, SY 2015-2018, estimating cumulative ATEs on SAGE ELA scale scores. The one-year treatment coefficient was "B = 9.14", with effect size "0.12".

"Coefficients for the one-year treatment (B = 9.14), two-year treatment (B = 4.38), and three-year treatment (B = 1.37) were all significant (p < 0.01). The effect sizes were 0.12, 0.06, and 0.02 for the one-, two-, and three-year treatment, respectively."

Discussion¶

Related Claims¶

• Cumulative Utah Compose effects on ELA scores diminished each year and peaked after three years of implementation — related
• The fourth year of Utah Compose exposure showed no statistically significant additional effect on ELA scores — related
• Students' ELA growth over the four years followed a logarithmic pattern, selected over a quadratic model by AIC — related
• Propensity score analysis estimated a Texas generalizability index of 0.91, and post-stratification yielded a population average treatment effect of 1.47 versus a conventional estimate of 1.44 — related