Evaluate AI systems in DLPs for fairness across learner groups, not just overall accuracy¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The paper recommends that researchers studying AI-enhanced DLPs go beyond aggregate accuracy: evaluate models for how they perform across different learner groups and contexts, ensure studies reflect diverse student populations (varied language backgrounds, learning needs, technology access), and clearly report limitations in sample coverage. The authors state that 'Researchers must critically examine the assumptions embedded in AI algorithms and prioritize fairness, accountability, and explainability in their use of DLPs,' since algorithmic decisions may reinforce disparities in data coverage or model performance.

Design Implications¶

Context¶

Requirements¶

• Studies must include diverse student populations and report subgroup performance of predictive models and AI systems

Constraints¶

• Fairness concerns are stated as especially salient in AI-enhanced platforms where training data coverage and model performance may be uneven across groups

Target Learners¶

• Diverse student populations, including English language learners and students with varied learning needs and technology access

Target Learning Goals¶

• Equitable performance of AI-driven personalization, prediction, and feedback in digital learning platforms

Affordances¶

• Dlp As Research Infrastructure

Related Strategies¶

• Mitigating Racial Bias in Edtech Products
• AI in Education Toolkit for Racial Equity
• Use this Toolkit: Mitigating Racial Bias in Edtech
• Regularly audit ML models for fairness and proactively apply bias mitigation during model training
• Evaluate AI systems before deploying them with students
• Audit educational LLM deployments for differential treatment before serving marginalized learners

Examples¶

-

Key Sources¶

• Wei, X., Roschelle, J., Pautz Stephenson, S. & Wortman, A. (2025). Researching digital learning platforms: Foundations, methods, and policy recommendations. Digital Promise. https://doi.org/10.51388/20.500.12265/257