Adding more learning activity data beyond Stage 1 did not significantly improve predictive performance of at-risk identification models¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Comparing the demographic-only stage with Stage 2, no significant differences in predictive performance were found on any metric, though effect sizes suggested potential for improvement. → Alison Cheng 2025

Evidence¶

Alison Cheng 2025¶

Alison Cheng, Bo Pei and Cheng Liu. (2025). Balancing Act: Early, Fair, and Accurate Identification of At-Risk Students. Journal of Learning Analytics, 12(3), 47–65. https://doi.org/10.18608/jla.2025.8761

q2 · i? · design · r2

Statistical comparison of the four ML models' average performance between the Demog Stage and Stage 2 across balanced accuracy, accuracy, recall, and precision found "no significant differences in predictive performance". The article notes effect sizes exceeding moderate levels suggest practical potential for improvement.

"Comparing the Demog Stage with Stage 2, which contains more learning activity data, there were no significant differences in predictive performance."

Discussion¶

Related Claims¶

• Balancing the dataset significantly improved model predictive performance but increased bias under demographic parity — related
• Incorporating more learning activity data reduced the potential bias caused by overreliance on demographic information — related
• Stage 1 (after the first unit review assignment) is the optimal point for identifying at-risk students, balancing timeliness, accuracy, and fairness — related
• Early-stage performance data is particularly important for predicting attrition, while demographic data has limited predictive value once performance data is available — related