Prior meta-analytic evidence indicates chatbot technology has a substantial positive effect on overall learning outcomes and knowledge retention, but not on critical thinking, engagement, or motivation¶

Claim · All claims
Evidence · 1 study · review r3 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The review reports, citing Deng and Yu (2023), that chatbot technology demonstrated a substantial impact on overall learning outcomes and significant enhancements in knowledge retention, with nonsignificant effects on critical thinking, learning engagement, and motivation. → Lasha Labadze 2023

Evidence¶

Lasha Labadze 2023¶

Lasha Labadze, Maya Grigolia and Lela Machaidze. (2023). Role of AI chatbots in education: systematic literature review. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-023-00426-1

q2 · i? · review · r3

The review reports, citing Deng and Yu (2023), that "Chatbot technology demonstrated a substantial impact on overall learning outcomes" and significant enhancements in knowledge retention, with nonsignificant effects on critical thinking, learning engagement, and motivation. No effect sizes are printed in the review.

"Chatbot technology demonstrated a substantial impact on overall learning outcomes, regardless of fac- tors such as intervention duration and learning content"

Discussion¶

Related Claims¶

• Initial meta-analyses indicate the effect of AI chatbots and generative AI on learning outcomes is lower in secondary education than in higher education — related
• GenAI shows a dual-mechanism profile: meta-analytic learning benefits coexist with dependence-related costs to critical thinking — related
• Gen-AI has a moderate positive overall effect on students' higher-order thinking (g = 0.609) across 29 experimental and quasi-experimental studies — reports the opposite
• The educational value of GAI is contingent on pedagogical integration, not on the technology itself — a broader claim this one bears on
• The review reports mixed findings on AI's motivational effects: some studies show enhancement, others nonsignificant or negative effects — a broader claim this one bears on
• Meta-reviews reach contradictory conclusions on whether AI reliance harms or helps critical thinking, with substantial design limitations limiting generalizability — related
• AI-supported instruction has a moderate positive effect on student achievement in STEM education (g = 0.670 under a random effects model) — related
• Meta-analytic evidence on GenAI and critical thinking shows a positive mean association with a prediction interval spanning harm to benefit — related