Fairness of AI outputs remains under-evaluated in edtech research¶

Claim · All claims
Evidence · 1 study · theoretical r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Fairness — ensuring outputs do not systematically disadvantage certain populations — is under-evaluated in edtech research, partly due to limited access to demographic data and a lack of agreed-upon measurement standards. → Li 2026

Evidence¶

Li 2026¶

Li, W., Kurimchak, M., & Whitmer, J. (2026). ATS Field Note: AI Model Evaluation. ATS Hub Field Note Series. https://doi.org/10.51388/20.500.12265/300

q2 · i? · theoretical · r2

The field note's discussion of the fairness component states that in practice the property "remains under-evaluated in edtech research" and names two causes: limited demographic data access and absent measurement standards. This is the authors' assessment of the field, not a study result.

"In practice, this property remains under-evaluated in edtech research, due in part to limited access to demographic data and a lack of agreed-upon standards for how fairness should be measured in complex contexts."

Discussion¶

Related Claims¶

• Using AI to evaluate AI outputs was often challenging because AI act as yes people and humans struggled to agree on evaluation rubrics — related
• Evidence is mixed on whether including demographic variables as predictors improves model performance in educational prediction — related
• AI is currently not best practice for competency-based micro-credential assessment; human assessors remain indispensable — related
• Model accuracy of vision-based proctoring systems can vary across demographics, lighting conditions, and hardware setups, potentially disadvantaging some students — a narrower finding that bears on this claim
• AI-detection tools produce inequitable outcomes, disadvantaging non-native English speakers through false positives — a narrower finding that bears on this claim
• Post-hoc fairness interventions on a vendor-controlled EWS redistributed disparities across demographic groups without consistently reducing them — related