Scholars raise three main concerns about AI chatbots in education: reliability and accuracy, fair assessment, and ethical issues¶

Claim · All claims
Evidence · 1 study · review r3 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The main skeptical arguments concern biased or inaccurate chatbot responses, difficulty assessing AI-generated student work, and ethical issues around data privacy and responsible AI use. → Lasha Labadze 2023

Evidence¶

Lasha Labadze 2023¶

Lasha Labadze, Maya Grigolia and Lela Machaidze. (2023). Role of AI chatbots in education: systematic literature review. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-023-00426-1

q2 · i? · review · r3

Narrative synthesis in the concerns section of the review. The review reports chatbots "may provide biased responses or non-accu- rate information", that AI-generated text detection is not yet foolproof and can produce false negatives or positives, and that ethical implications concern data privacy, security, and responsible AI use.

"AI chatbots may provide biased responses or non-accu- rate information (Kasneci et al., 2023; Sedaghat, 2023). If the chatbot provides incorrect information or guidance, it could mislead students and hinder their learning progress."

Discussion¶

Related Claims¶

• Students report negative experiences with ChatGPT including inaccurate outputs, lack of emotional connection, and risk of overreliance, which constrain competence — a narrower finding that bears on this claim
• Students perceive LLM chatbots as useful for learning but requiring critical use, reporting hallucinations, overconfidence, and dependency risks — a narrower finding that bears on this claim
• The review identifies data privacy, algorithmic fairness, academic integrity, and unequal access as interconnected ethical concerns in AI-supported mathematics education — a narrower finding that bears on this claim
• The most frequently cited ethical concerns about GenAI in research were hallucinated facts, fabricated references, and misleading content — a narrower finding that bears on this claim
• Teachers report concerns about infrastructure, reliability, age appropriateness, teacher competence, and student over-reliance on AI — a narrower finding that bears on this claim
• The most reported risks of AI tools in science and chemistry education are ethical issues, including gender and racial bias, hallucinations, copyright infringement, and plagiarism — a narrower finding that bears on this claim
• Limited output accuracy and over-reliance are the most pressing challenges Chinese HSS students report in GenAI use, alongside strong ethical concern — a narrower finding that bears on this claim
• Community adults entering an AI education session held broad, generalized concerns about AI, including distrust of self-regulation and privacy worries — related
• AI systems can produce biased or inaccurate diagnostic outputs, requiring critical teacher evaluation before integration — a narrower finding that bears on this claim
• Institutional approval processes, privacy regulations, technology access, and limited classroom time constrained chatbot adoption beyond instructional value — related
• Data-based recording of educational processes risks narrowing education to quantifiable output indicators and raises data sovereignty and algorithmic ethics issues — related