Both prior performance flags and the machine learning algorithm are less accurate when predicting outcomes for students who are Black¶

Claim · All claims
Evidence · 1 study · associational r? · q2
Currency · newest study 2021 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2024: tool and model accuracy changes quickly

Subclaims¶

q2 i? Accuracy is lower for students who are Black under both the prior performance flags and the algorithm. → Lindsay Cattell 2021

Evidence¶

Lindsay Cattell 2021¶

Lindsay Cattell, Julie Bruch. (2021). Identifying Students At Risk Using Prior Performance Versus a Machine Learning Algorithm. Regional Educational Laboratory Mid-Atlantic, U.S. Department of Education, Institute of Education Sciences. https://ies.ed.gov/ncee/edlabs

q2 · i? · associational · r?

The report's findings summary states that "Both the prior performance flags and the algorithm are less accurate" for students who are Black, indicating an equity limitation shared by both identification approaches.

"Both the prior performance flags and the algorithm are less accurate when predicting outcomes for students who are Black."

Studies bearing on this claim¶

Are at-risk predictions equally accurate for every student group, and do fairness fixes close the gap? This table gathers the studies behind this claim and the 11 narrower claims that bear on it: 7 distinct studies (2 associational, 2 causal, 1 review, 1 design, 1 theoretical), q1–q2, 1 of 7 with an effect size coded. One row per study, codes as each entry records them; no effect is averaged and no verdict is drawn. Built by scripts/synthesize_claim.py from eval/synthesis/; a row that looks wrong is fixed on the claim it comes from.

Discussion¶

Related Claims¶

• A prior performance early warning system and a machine learning algorithm with same-percentage risk-score cutoffs are similarly accurate at identifying students who experience academic problems — related
• A machine learning algorithm with 10-percent risk-score cutoffs better targets students most likely to experience academic problems and has the advantage in predicting suspensions — related
• Prediction algorithms forecast student achievement from small windows of data, more accurately when data are enriched with instructional design and learning-process detail — related
• Predicting student outcomes was a common AIEd use in higher education, with ten studies predicting academic performance — related
• The review reports AI curriculum-design tools identify learning gaps and equity-analysis tools examine differential policy impacts on student populations — related
• Post-hoc fairness interventions on a vendor-controlled EWS redistributed disparities across demographic groups without consistently reducing them — related
• When demographic groups have different base rates, no method can satisfy calibration, equalized odds, and statistical parity simultaneously — related
• Removing the sensitive race feature caused little impact on predictive performance at Stage 1 and no significant fairness differences, with mixed fairness effects — related