Recurring risks of GenAI in STEAM education include hallucination, bias, superficial completion strategies, and compromised assessment validity¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? The review reports that the same systems that draft lesson plans and generate examples can hallucinate, amplify bias, encourage superficial completion strategies, and complicate the validity of assessment. → Dahal N 2026

Evidence¶

Dahal N 2026¶

Dahal N, Hasan MK and Islam MT (2026). GenAI and AI tools for STEAM educators: evidence-informed strategies and resources for the future of teaching. Front. Educ. 11:1948064. https://doi.org/10.3389/feduc.2026.1948064

q2 · i? · review · r2

Narrative synthesis: the review attributes these failure modes to multiple cited reviews and conceptual analyses, describing GenAI outputs as probabilistic rather than inherently truthful, which requires active verification and contextual judgment. No effect sizes are printed.

"Yet the same systems can hallucinate, amplify bias, encourage superficial completion strategies, and complicate the validity of assessment (Dwivedi et al., 2023; Farrokhnia et al., 2024; Kasneci et al., 2023; Lo, 2023; Tlili et al., 2023)."

Discussion¶

Related Claims¶

• Recurring benefits of GenAI in STEAM education include engagement, teacher productivity, and creativity when supported by sound pedagogy — related
• A structured narrative synthesis of GenAI/AI in STEAM education screened 107 records and included 15 sources in the reviewed corpus — related
• The most reported risks of AI tools in science and chemistry education are ethical issues, including gender and racial bias, hallucinations, copyright infringement, and plagiarism — a narrower finding that bears on this claim
• LLM-assisted inductive qualitative coding carries risks of superficial themes, broad or redundant codes, and hallucinated interpretations, so LLMs should augment rather than replace human researchers — a narrower finding that bears on this claim
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — a narrower finding that bears on this claim
• The most frequently cited ethical concerns about GenAI in research were hallucinated facts, fabricated references, and misleading content — related
• Phase VI AI-enabled systems face six documented challenges: explainability, hallucination risk, prompt sensitivity, computational cost, validation complexity, and limited large-scale evidence — related
• Zero-shot LLM errors follow four recurring patterns: over-interpretation, failure to detect relevant information, hallucination, and failure to generate a response — related
• Across 28 project contexts, human-in-the-loop and independent peer review identified no conceptual or mathematical hallucinations, and Python visualization scripts executed without compilation errors on the zero-shot attempt. — reports the opposite
• The most significant reported benefit of AI in instructional design is assistance with administrative and repetitive tasks (18 of 28 studies, 64.2%) — related
• Qualitative error analysis yields five recurring error types, with over-inference most prevalent (33.5%), followed by evidence misidentification (23.4%) and hallucination (19.8%) — related
• LLM-based systems mediated family education through four recurring forms: producing materials, structuring dialogue, interpreting context, and connecting to external knowledge and values — related
• Current LLMs misalign instructional-strategy sensitivity with pedagogical relevance, including hallucinated relevance of irrelevant traits — a narrower finding that bears on this claim
• Disciplinary expertise alone is insufficient for teachers to detect GenAI representational bias — a narrower finding that bears on this claim
• GenAI improves administrative coordination of group work but its analyses of team members can be inaccurate and hallucinated — a narrower finding that bears on this claim
• LLMs in educational applications hallucinate in 15–25% of responses and can amplify bias against non-native language inputs — a narrower finding that bears on this claim
• Of 178 reference sources produced by AI dialogue systems, 69 lacked a DOI and 28 could not be found via Google search and had no existing DOI — a narrower finding that bears on this claim
• In a narrative educator assessment, coding assistants (Cursor and Claude Code) produced the most accurate, complete, and pedagogically sound slides from textbook chapters — related
• Vision-assisted extraction identified chart type in all sampled images but showed ±5–10% numeric estimation error and two failure modes correctable via mandatory review — related