Middle school students with the lowest baseline argumentative-writing scores showed the largest growth across essay attempts on an AES platform, while mid- and high-baseline students showed limited or no significant change¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Across four growth outcomes, students with lower baseline scores experienced the highest average growth, whereas students starting mid-range or higher were predicted to have limited or no significant change. → Greene Nolan 2023 q2 i? Students starting at the lowest score of 4/16 still did not reach the 12-point expectations level by the final draft, ending around 8–9 points on average. → Greene Nolan 2023 (2)

Evidence¶

Greene Nolan 2023¶

Greene Nolan, H. L., & Vang, M. C. (2023). Automated essay scoring in middle school writing: Understanding key predictors of students' growth and comparing artificial intelligence- and teacher-generated scores and feedback. Digital Promise. https://doi.org/10.51388/20.500.12265/187

q2 · i? · associational · r2

Hierarchical linear models (students nested in teachers) on Topeka AES scores estimated four outcomes—growth within first and second prompts, transfer across baselines, and growth from first baseline to second final—as a function of baseline score, finding "the highest growth" for the lowest starters.

"students with lower baseline scores experienced, on average, the highest growth, whereas students who started with mid-range or higher scores were predicted to have limited or no significant change across essay attempts."

Greene Nolan 2023 (2)¶

Greene Nolan, H. L., & Vang, M. C. (2023). Automated essay scoring in middle school writing: Understanding key predictors of students' growth and comparing artificial intelligence- and teacher-generated scores and feedback. Digital Promise. https://doi.org/10.51388/20.500.12265/187

q2 · i? · associational · r2

Executive-summary analysis of the same score dataset reports that lowest-band students, despite the largest gains, ended "around 8–9 points" on the 16-point sum scale, below the 12-point expectations level.

"students who began at that lowest level of 4 points still did not meet expectations (12 points) by the final draft—ending, on average, around 8–9 points."

Discussion¶

Learner Variables¶

• Writing Skills — predictor: learners who differ on it differ in outcomes

Related Claims¶

• Teachers' years of experience and most self-reported instructional practices did not significantly predict students' AES-measured writing growth — related
• Teachers' belief that their students were ready for argumentative writing significantly predicted student growth within the first prompt — related
• The AES tool scored middle school essays more generously than teacher scorers overall and on three of four dimensions
• Neural network models predicted students' writing performance groups with 94–100% accuracy from self-regulation characteristics — related
• In Support & Evidence and Claim & Focus, teacher and AES scores agreed at the lowest and highest rubric points but diverged in the middle, with teachers more likely to score a 2 and the AES a 3 — related
• Prompt Coach's largest learning gains occurred on Inclusion of Constraints, Error Handling, and Context Awareness, the dimensions developers scored lowest on at baseline — a narrower finding that bears on this claim
• Scattering varies with intelligence-age level: smallest at the lowest mental ages and possibly largest in the middle range, with age-level differences exceeding diagnostic-category differences — related