Experts identified pervasive limitations of today's AI for learning, including biased data, inequity, non-graceful failure, and weak context grasp¶

Claim · All claims
Evidence · 1 study · qualitative r2 · q1
Currency · newest study 2020 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person · overdue for a recheck since 2023: tool and model accuracy changes quickly

Subclaims¶

q1 i? A panel of 22 experts enumerated specific limitations of current AI in education, including dataset weaknesses, bias, lack of attention to equity and learner differences, tendency to fail non-gracefully, and difficulty integrating multiple AI capabilities. → Roschelle 2020

Evidence¶

Roschelle 2020¶

Roschelle, J., Lester, J. & Fusco, J. (Eds.) (2020). AI and the future of learning: Expert panel report [Report]. Digital Promise. https://circls.org/reports/ai-report doi:10.51388/20.500.12265/106

q1 · i? · qualitative · r2

Expert-panel discussion at a facilitated online convening; the report records the limitations experts posted, including "Presence of bias in data" and failure to attend to equity and learner differences. This is qualitative expert opinion, not an empirical test.

"The experts were also highly aware of limitations and weaknesses of today’s AI. Some specific limitations that the experts called out include: • Limitation and weaknesses of available datasets, which limit the resulting AI progress • Presence of bias in data"

Discussion¶

Related Claims¶

• Weak grasp of context can make AI misdiagnoses systematically harmful when a flawed decision pattern is replicated across many students — a narrower finding that bears on this claim
• Fine-tuned GPT-4o-mini delivers equitable name-detection performance across cultural and gender groups, reducing cultural biases present in baseline models — related
• Current best learner performance models are severely biased outside the interval containing most of the data, hindering downstream adaptive policies and open learner models — related
• Across 28 K-12 AI pilot projects, AI demonstrated success at categorizing and coding datasets, translating language, and triangulating qualitative findings with quantitative data — related
• Generative AI systems carry strong potential for introduced and inherited bias — related
• Training constraints during AI development create potential biases that may be amplified by the population data used — related
• AI-based proctoring systems trained on western behavioural norms may exacerbate cultural and social challenges in non-English-speaking regions — a narrower finding that bears on this claim
• Algorithmic bias and data privacy are central ethical risks of AI in adult education — related
• The evidence base on AI in higher education shows geographical, publication, and time-horizon biases, with successful implementations overreported — related
• The most reported risks of AI tools in science and chemistry education are ethical issues, including gender and racial bias, hallucinations, copyright infringement, and plagiarism — related
• Data-based recording of educational processes risks narrowing education to quantifiable output indicators and raises data sovereignty and algorithmic ethics issues — related
• Three recurring AI failure modes arose in this project-based learning context: plausible-but-incorrect code, missing specialized knowledge, and limited long-term context — a narrower finding that bears on this claim