Positive LLM tutor perceptions do not necessarily translate to learning gains, creating an over-reliance risk¶

Claim · All claims
Evidence · 1 study · associational r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? High learner satisfaction with LLM tutors can coexist with limited learning gains when content-relevant AI responses foster superficial engagement. → Kaşarcı İsmail 2026

Evidence¶

Kaşarcı İsmail 2026¶

Kaşarcı İsmail, Yurt Eyüp. (2026). Effectiveness of AI-Supported Game-Based Learning: A Systematic Review of Outcomes, Challenges, and Future Directions. Behavioral Sciences. https://doi.org/10.3390/bs16071050

q2 · i? · associational · r1

The review reports a single-group observational study of beginner CFL learners using an LLM tutor that was well-received, in which "positive LLM tutor perceptions do not necessarily translate to learning gains" and content-relevant responses were associated with more superficial engagement.

"Fang et al. (2025) critically found that positive LLM tutor perceptions do not necessarily translate to learning gains, with content-relevant AI responses associated with more superficial engagement (the over-reliance risk)."

Discussion¶

Related Claims¶

• Higher reliance on AI tools is correlated with lower critical-thinking performance, mediated by cognitive offloading — the general claim this finding bears on, with the studies behind it
• Over-reliance on GAI risks weakening teaching and social presence and depriving students of foundational skill development — related
• Standalone LLM tutors replicate Phase II's profile: strong single-interaction intelligence without systematic control over the learning trajectory — related
• Unguarded answer-giving AI harmed unaided exam performance while a guarded version of the same model erased the harm (Bastani et al., 2025, as reported) — related
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — related
• Viewers of learning-oriented content describe ChatGPT as a tutor or thinking partner, while viewers of output-oriented content raise over-reliance and cognitive-offloading concerns — related
• Individual trajectories suggest heavy LLM use focused on support other than full answer generation can coexist with improving scores, whereas reliance on LLM-generated summaries was associated with declining scores — related
• Over-reliance on AI dialogue systems reduces students' critical and analytical thinking skills, especially when they become overly dependent on AI-generated content — related
• System-controlled adaptive instruction does not necessarily yield higher transfer than nonadaptive static approaches — related