The most reported risks of AI tools in science and chemistry education are ethical issues, including gender and racial bias, hallucinations, copyright infringement, and plagiarism¶

Claim · All claims
Evidence · 1 study · review r2 · q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Ethical issues are the most frequently reported risk/limitations theme (n = 3), including gender and racial biases in generated visuals, hallucinations, copyright infringement, and plagiarism and biased information production. → Ali Kürşat Erümit 2025

Evidence¶

Ali Kürşat Erümit 2025¶

Ali Kürşat Erümit, Rabia Özdemir Sarıalioğlu. (2025). Artificial intelligence in science and chemistry education: a systematic review. Discover Education. https://doi.org/10.1007/s44217-025-00622-3

q3 · i? · review · r2

Content analysis of risks/limitations across the 18 included studies, coded into themes (Table 7). The review reports that researchers "mostly report risks/limitations regarding ethical issues (n = 3)". Descriptive synthesis; no effect size.

"researchers mostly report risks/limitations regarding ethical issues (n = 3) in the use of artificial intelligence tools in education"

Discussion¶

Related Claims¶

• AI applications in science and chemistry education most frequently affect learning-process outcomes, reported in 9 of 18 studies — related
• ChatGPT and conversational robots are the most used AI tools in science and chemistry education research, with ChatGPT used in 50% of reviewed studies — related
• Generative AI systems carry strong potential for introduced and inherited bias — a broader claim this one bears on
• Experts identified pervasive limitations of today's AI for learning, including biased data, inequity, non-graceful failure, and weak context grasp — related
• Algorithmic bias and data privacy are central ethical risks of AI in adult education — related
• LLM-assisted inductive qualitative coding carries risks of superficial themes, broad or redundant codes, and hallucinated interpretations, so LLMs should augment rather than replace human researchers — related
• Zero-shot LLM errors follow four recurring patterns: over-interpretation, failure to detect relevant information, hallucination, and failure to generate a response — related
• Phase VI AI-enabled systems face six documented challenges: explainability, hallucination risk, prompt sensitivity, computational cost, validation complexity, and limited large-scale evidence — related
• Academic integrity, professional development, and equity concerns about AI span all surveyed units, though interpreted differently — related
• State AI guidance describes risks but risk-management work is at an early stage, with some documents possibly too upbeat given school readiness — related
• Qualitative findings identified plagiarism, student overreliance on AI, and a barrier of reconciling institutional policy gaps with personal ethical values — related
• Faculty members and students identify both user-related and algorithm-related ethical risks in AI use in education — a broader claim this one bears on
• Human-AI relationship concerns persist across all CAI generations while academic integrity and data privacy are emerging ethical concerns — related
• Ethical and governance dimensions of AI were less prominent in teachers' narratives than pedagogical opportunities — reports the opposite
• Educators' use of AI in PBL is accompanied by persistent ethical concerns about data privacy, algorithmic bias, and educator autonomy — related
• Three major research themes—Ethical AI, Responsible AI, and Energy-efficient AI—each reached 50 per cent trend presence by 2025 — related
• AI literacy requires discipline-specific calibration beyond prompt-writing skills — related
• Recurring risks of GenAI in STEAM education include hallucination, bias, superficial completion strategies, and compromised assessment validity — a broader claim this one bears on
• Privacy protections lag behind reputational concerns: institutions rarely consider implications of capturing audio-visual and behavioral data from private spaces, and AI algorithms are seldom transparent about error rates — related
• Reviewed studies identify data privacy, algorithmic bias, unequal access, and epistemic trust as central ethical concerns in educational AI — related
• Qualitative error analysis yields five recurring error types, with over-inference most prevalent (33.5%), followed by evidence misidentification (23.4%) and hallucination (19.8%) — related
• Layer 1 survey (N=54) found mostly daily or weekly but largely passive AI use, with persistent ethical anxiety described as cheating fears or hallucination worries — related
• Students perceive LLM chatbots as useful for learning but requiring critical use, reporting hallucinations, overconfidence, and dependency risks — related
• Over-reliance on AI in chemical experimentation carries ethical considerations and pitfalls as AI automates routine tasks and alters how experiments are conceptualized — a broader claim this one bears on
• Disciplinary expertise alone is insufficient for teachers to detect GenAI representational bias — related
• Review reports ethical concerns of educational AI including algorithmic bias and exploitation of personal data — related
• LLM-based simulated students surpass rule-based systems in natural language generation and situational flexibility, but concerns about bias and evaluation reliability persist — related
• GenAI raises privacy, transparency, bias and accuracy concerns in group work, though empirical evidence remains limited — related
• Common risks of AI integration in nursing education include technological inequities, faculty preparedness gaps, and ethical concerns around privacy and bias — related
• Educator open-ended responses yield four themes: ChatGPT dominance, need for methodological guidance, ethical concerns, and infrastructure barriers — related
• General-purpose LLMs show accuracy and reliability limits, including hallucinations and task-specific accuracy below perfect, making them risky as expert advice sources — a narrower finding that bears on this claim
• Faculty opt-out of generative AI is more substantial than student non-use, with value-, identity- and risk-based reasons — related
• LLMs in educational applications hallucinate in 15–25% of responses and can amplify bias against non-native language inputs — related
• Awareness of Creative Commons licensing lags behind copyright and public domain awareness among both faculty and academic leaders — related
• Only a small fraction of AIEd research addresses ethical implications, challenges and risks — related
• Scholars raise three main concerns about AI chatbots in education: reliability and accuracy, fair assessment, and ethical issues — a broader claim this one bears on
• Ethical considerations are a sidelined research theme, addressed by only seven of 183 publications, mostly on stereotype perpetuation by generative AI — related
• Most reviews reporting neurophysiological AI use stressed the need for further ethical considerations, including participant consent and transparent development — related
• Academic integrity concerns were the third most prevalent pedagogical limitation, reported in 20 of 54 studies (37%) — related
• AI hiring tools have demonstrated biases disproportionately affecting women, individuals with disabilities, and underrepresented ethnic and racial groups — related
• Topic modelling yielded 16 topics dominated by four: AI in Education & Pedagogy (39.43%), Applied ML & Data Science (21.33%), AI Ethics Risk & Governance (20.83%), and ChatGPT for Teaching, Writing & Coding (7.13%) — related
• AI governance vocabulary (bias, transparency, privacy, accountability, regulation, fairness) is a structural zero in students' AI associative networks — related
• Respondents identified algorithmic bias (31.9%), data privacy (27.5%), and technical reliability (27.5%) as the primary risks of AI in education — related
• Existing legal and sector guidance each address only part of ethical learning analytics governance, and none integrates all six LEAGUE dimensions — related
• Data-based recording of educational processes risks narrowing education to quantifiable output indicators and raises data sovereignty and algorithmic ethics issues — related
• Faculty advisors and mentors are considered pivotal in promoting doctoral students' development as responsible and ethical researchers — related