Rigorous investigation of GenAI's actual educational and cognitive effects remains limited despite increasing integration into learning environments¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Articles focusing only on GenAI and LLM-based agents address educational impact and cognitive challenges less than other challenges (38.1% highest), indicating research prioritizes technical, ethical, and operational aspects over pedagogical outcomes. → Amrita Ganguly 2025

Evidence¶

Amrita Ganguly 2025¶

Amrita Ganguly, Nafisa Mehjabin, Aqdas Malik, Aditya Johri. (2025). Conversational AI agents in education: an umbrella review of current utilization, challenges, and future directions for ethical and responsible use. AI and Ethics. https://doi.org/10.1007/s43681-025-00916-0

q2 · i? · review · r2

The review's analysis of challenge-theme distribution by agent type (RQ2), observing that GenAI-focused reviews emphasize system capabilities and risk mitigation. The review states that "rigorous investigation of their actual educational and cognitive effects remains limited".

"rigorous investigation of their actual educational and cognitive effects remains limited, underscoring the need for more studies that systematically evaluate their pedagogical value."

Discussion¶

Related Claims¶

• Interaction and usability challenges affect 85.3% of reviewed CAI articles, with usability problems most observable in mature rule-based agents — related
• LLM-based tools may interfere with reading comprehension and retention despite learners finding them helpful — related
• Student-facing AI raises serious wellbeing, equity, and safety concerns, and some educators express more confidence in teacher-facing tools — related
• Systematic review finds operational teaming most common (62.5%) and no empirical evidence of synergistic teacher-AI teaming — related
• Capabilities and skills are explicitly addressed in only 3 of 36 reviewed GenAI studies, marking a gap in AI-literacy and educator-capability research — related
• Teacher educators show shallow understanding of GenAI tools and gaps in aligning AI use with pedagogical goals — related
• Research on supports teachers need to teach about complex systems has been sparse despite 15 years of tools-and-theories work — a narrower finding that bears on this claim
• AI tutors explicitly designed according to educational theory outperformed traditional active learning in rigorous academic settings — related
• CAI-in-education research shows a framework gap: no end-to-end design guidance, weak CAI-specific usability methods, unclear pedagogical guidance, and limited AI literacy support — related
• A mixed-methods study addresses groupthink in newly formed student teams by introducing a genAI agent into small-group processes of a VIPE problem-based learning session — a narrower finding that bears on this claim
• GenAI models span multiple agent types, mainly model-based and goal-based, but categorisation is difficult without model specifics — related
• Only 4 of 23 reviewed conversational agent studies explicitly apply learning theories, indicating a gap between CA development and pedagogical grounding — related